Evaluation of Springboard - a community-based alcohol and other drug (AOD) recovery day program: Communify
Status: Completed August 2018
Client: Communify
Background
The purpose of this project was to undertake an evaluation of the outcomes of the Springboard program, a community-based alcohol and other drug (AOD) recovery day program led by Communify for individuals with problematic AOD use and a co-occurring mental health issue.
Approach
The evaluation of the Springboard program aimed to achieve the following:
Increased understanding of the factors that underpin successful delivery of the Springboard program
Increased understanding of the outcomes of the Springboard program delivered by Communify
Increased visibility of opportunities to strengthen future program delivery
The overall evaluation approach consisted of a number of components, including:
Program logic model — A program logic model aims to visually represent how the program leads to results, by describing a program’s:
Inputs—financial, human and other resources available to deliver activities and produce outputs
Activities—specific actions and processes taken by the program that contribute to the identified outputs
Outputs—products or services delivered by the program that reach people who participate in the program
Outcomes—changes for individuals, groups, communities or systems over time, including short-term (e.g. knowledge, values, motivation, behaviours, skills) and long-term (e.g. conditions people experience, how systems operate)
Assumptions—statements and judgements made that concern how and why the program will work in its context.
External factors—the environment in which a program exists and the factors that can positively or negatively influence its delivery.
Evaluation questions — Drafting evaluation questions helps to focus the scope of inquiry within an evaluation process. Evaluation questions are drafted against key domains to help gauge the extent to which the program meets its objects from different perspectives. These five domains include:
Appropriateness—are the needs of clients being met?
Accessibility—is the program reaching its intended target population?
Efficiency—does the program provide value for money?
Effectiveness—is the program achieving measurable outcomes?
Sustainability—what are the prospects for growth of the program?
Quantitative data analysis — Existing datasets were available for analysis including individual client data collected at various time points for standardised assessment measures such as:
Behaviour and Symptom Identification Scale (BASIS-24)
Depression, Anxiety and Stress Scale (DASS-21)
Drug Taking Confidence Questionnaire (DTCQ-8)
Severity of Dependence Scale (SDS)
Connor-Davidson Resilience Scale (CD-RISC-10)
Qualitative data collection — Semi-structured interviews with project staff collected additional qualitative data regarding the design and implementation of the program. Topics of inquiry included:
Underpinning practice frameworks
Adherence to program theory
Referral pathways (inward and outward)
Implementation success factors
Barriers and challenges
Literature review—Best practice relating to models of AOD treatment were summarised based on a brief scan of academic research and published evaluation reports
Evaluation report—A comprehensive report on the findings and learnings of the evaluation, including recommendations for consideration by Communify was generated
Presentation—A short presentation of the high-level findings for Communify and other relevant stakeholders was provided