summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Alachua Academy
Sequel Youth and Family Services
(Contract Provider)
3430 NE 39th Avenue
Gainesville, Florida 32609
Primary Service: Impact of Crime
SPEP Review Period: 12/15/19 - 12/14/20
SPEP Review Conducted: 12/15/20
Florida Department of Juvenile Justice
Report Date: 1/28/21
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes the Impact of Crime curriculum as a primary service. A review of
documentation for the annual review period of December 2019 through December 2020 showed
there were four staff members trained in the primary service. The training is four days provided by
a qualified trainer and involves role plays, lecture, and activities to allow the staff the ability to gain
experience using the curriculum.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the curriculum Impact of Crime: Addressing the Harm v 2.0. The facilitator’s
manual is divided into seven chapters, an appendix with handouts, and the student guide section
(which includes the student exercises). The manual addresses the order in which to deliver the
content and gives the facilitators an outline to follow. The facility does not have separate student
workbooks to give out. The facilitators make copies of the exercises before each session to pass
out to the youth.
The Impact of Crime is delivered as a closed group with no more than twelve youth in each
cohort. The curriculum is to be delivered to the youth over a minimum of sixteen weeks for thirty-
two sessions, with at least one day between lessons. During this review period, there were eleven
instances of groups being held on back-to-back days, accounting for twenty-two total lessons.
This was addressed with both the Clinical Director and facilitator at the time of the review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Impact of Crime was observed on 12/15/2020. The group began at 3:59p. The
facilitator asked youth to state the class expectations. The facilitator then went into the icebreaker
activity. The co-facilitator wrote the objectives on the board while the youth were completing the
icebreaker activity. The co-facilitator then went over the chapter objectives with the youth. The
facilitator gave each youth a definition on a sticky note and asked the youth to place it next to the
word it defines. The facilitator then reviewed the homework from the prior session. The facilitator
ripped up the pictures the youth drew (part of the activity) and then discussed how others feel
when property crime happens. The facilitator then went over the myth vs reality statements. Each
youth were to answer if the statement represented a myth or a reality. Each of the youth were
then given a copy of student exercise 6. Youth were asked to read the scenario and answer
questions based on each scenario. The facilitator assigned the youth homework to complete prior
to next session. The group ended at 5:02p.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
According to the documentation reviewed, there has been no facilitator turnover at the program.
One facilitator was promoted to the assistant facility administrator after her cohort ended.
There was a gap of two weeks from 6/3/2020-6/18/2020. The facilitator went on bereavement
leave during that time. There was no other trained facilitator to take over the group during the
facilitator’s absence.
There was one gap of just over one month between the cohort which ended on 9/12/2020 and the
cohort which began on 10/27/2020. The clinical director stated the gap was due to a low census
because of COVID-19. The program began the primary service once there were enough youth to
begin a cohort. There was also a gap of two and a half weeks during the cohort which began on
10/27/2020. The clinical director stated this was due to COVID-19. A staff member tested positive
and the remaining staff all had to isolate until they received a negative test result. The results
came back about two weeks after the initial test. The program resumed the service once the staff
members were cleared to work. This gap does not qualify as an unnecessary gap in service.
Four cohorts completed during the review period and one cohort was running at the time of the
review. The dates of the cohorts are 10/20/2019-2/20/2020, 12/3/2020-4/4/2020, 4/16/2020-
8/6/2020, 5/22/2020-9/12/2020, and 10/27/2020-current.
The program can earn 2 points if there are no unnecessary gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of documentation for Impact of Crime shows there were sixteen fidelity monitorings
completed during the annual review period, accounting for each facilitator and each month. Each
fidelity monitoring was completed by either the Clinical Director, Regional Clinical Director, or a
trained facilitator. The Clinical Director or Regional Clinical Director discusses the fidelity check
with the facilitator upon completion.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process in place to address corrective actions resulting from internal fidelity
monitoring. The process begins with the monthly internal fidelity monitoring completed by the
either the Clinical Director or Regional Clinical Director, who then discuss any findings with the
facilitator. The facilitator is afforded two weeks to make corrections before being re-evaluated. A
review of fidelity monitoring for Impact of Crime shows there were no corrective actions noted nor
taken.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Evaluations were reviewed for two of the four Impact of Crime facilitators. Each evaluation has an
area titled "facilitator skill delivering the evidence-based programming". Each evaluation had
specific information based on the facilitator's delivery of Impact of Crime.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 16 weeks for this type of service. Of the 27 youth in the
sample, 63% (17 of 27) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 24 contact hours for this type of service. Of the 27 youth in the
sample, 67% (18 of 27) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 10 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 40%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 85%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 7 youth
Moderate-High = 12 youth
High = 4 youth
Total Youth in Sample = 27 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 15%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 4 youth
Total Youth in Sample = 27 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 85% (23 of 27) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 15% (4 of 27) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 10 25 40%
Totals 62 85 73%
This SPEP report evaluates Impact of Crime, an intervention delivered at Alachua Academy.
27 youth completed the program during the review period. Of those youth, 26 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 6 points for Amount of Service: Duration. Of the 26 total youth sampled, only
19 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 17
received at least the recommended weeks of service. Youth in the sample completed between 1
and 34 weeks of service, with an average of 17 weeks.
The program earned 6 points for Amount of Service: Contact Hours. Of the 26 total youth
sampled, only 19 included dosage in the EBS Module. Of those youth with correct dosage, 18
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
received at least the recommended hours of service. Youth in the sample completed between 12
and 70 hours of service, with an average of 35 hours.
The program was awarded 10 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 91%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Alachua Academy can optimize their SPEP Quality of Service Delivery score by ensuring there
are no unnecessary gaps in service.
Alachua Academy can optimize their SPEP Amount of Service score by ensuring that dosage for
all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9


state: FL

facility_name: Alachua Academy

summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Alachua Academy
Sequel Youth and Family Services
(Contract Provider)
3430 NE 39th Avenue
Gainesville, Florida 32609
Primary Service: Living in Balance
SPEP Review Period: 12/15/19 - 12/14/20
SPEP Review Conducted: 12/15/20
Florida Department of Juvenile Justice
Report Date: 1/28/21
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Living in Balance.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes the Living in Balance curriculum as a primary service. A review of
documentation for the annual review period of December 2019 through December 2020 shows
there are three staff who are trained in the primary service. The program has a written training
protocol, requiring two hours conducted as in-service training delivered by the Clinical Director or
Regional Clinical Director.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the Living in Balance: Moving from a Life of Addiction to a Life of Recovery
as the manual. The manual addresses the order in which to deliver the content and gives the
facilitators an outline to follow. The facility does not have separate student workbooks to give out.
The facilitators make copies of the exercises before each session to pass out to the youth. Living
in Balance is an open group run year-round. There were three staff who facilitated groups during
the review period.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Living in Balance was observed on 12/15/2020. The group began at 2:49p. The
facilitator began group with a check in. The lesson was on Sex, Drugs, and Alcohol. The youth
and facilitator discussed triggers for sex and drugs. They also discussed alcohol use and sex
problems. The facilitator went into discussing sex as a trigger to use drugs. The youth were asked
to participate in the role plays. After the youth read the first scenario, the facilitator decided to just
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
read each scenario and then ask the corresponding questions. The group then moved onto Part
2: Sexual Problems. The youth and facilitator took turns reading the information from the lesson.
The facilitator then read the visualization activity from the workbook. The facilitator asked the
youth to close their eyes to visualize what she was reading. The facilitator then asked the youth to
share their personal strengths and weaknesses after they had time to reflect. The youth were then
asked about their ultimate goal. The facilitator assigned homework for the youth to complete prior
to next session. The group ended at 3:59p.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation showed there was no facilitator turnover at the program. Living in
Balance was provided by a qualified facilitator during the review period. There are currently two
facilitators who can facilitate the primary service with the clinical director available as needed. The
two trained therapists each were running one cohort continuously during the review period.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of documentation for Living in Balance shows there were twenty-four fidelity monitorings
completed during the annual review period, accounting for both facilitators each month. Each
fidelity monitoring was completed by a trained facilitator.
The Regional Clinical Director or Clinical Director conduct monthly fidelity monitoring of each
facilitator. The Regional Clinical Director discusses the fidelity monitoring with the facilitator to
ensure the facilitator is completing groups with fidelity of the model.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process in place to address corrective actions resulting from internal fidelity
monitoring. The process begins with the monthly internal fidelity monitoring completed by the
either the Clinical Director or Regional Clinical Director, who then discuss any findings with the
facilitator. The facilitator is afforded two weeks to make corrections before being re-evaluated. A
review of fidelity monitoring for Living in Balance shows there were no corrective actions noted
nor taken.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The completed evaluations were reviewed for the four staff trained in Living in Balance. Each
evaluation has an area titled "facilitator skill delivering the evidence-based programming". Each
evaluation had specific information based on the facilitator's delivery of Living in Balance.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 24 weeks for this type of service. Of the 24 youth in the
sample, 92% (22 of 24) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 40 contact hours for this type of service. Of the 24 youth in the
sample, 96% (23 of 24) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 10 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 40%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 88%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 7 youth
Moderate-High = 11 youth
High = 3 youth
Total Youth in Sample = 24 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 13%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 3 youth
Total Youth in Sample = 24 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 88% (21 of 24) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 13% (3 of 24) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 10 25 40%
Totals 76 95 80%
This SPEP report evaluates Living in Balance, an intervention delivered at Alachua Academy.
27 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 23 total youth sampled, all
23 included dosage with end dates in the EBS Module. Of those youth, 22 received at least the
recommended weeks of service. Youth in the sample completed between 22 and 49 weeks of
service, with an average of 35 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 23 total youth
sampled, all 23 included dosage in the EBS Module. Of those youth, 23 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
recommended hours of service. Youth in the sample completed between 45 and 87 hours of
service, with an average of 62 hours.
The program was awarded 10 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 96%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Alachua Academy can maintain their SPEP Quality of Service Delivery score by continuing the
practices in place at the time of this review.
Alachua Academy can optimize their SPEP Amount of Service score by ensuring that dosage for
all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

source: DJJ

report_type: SPEP

program_name_raw: Alachua Academy - Sequel

program_status_note: Closed

service_type: Living in Balance


summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Alachua Academy
Sequel Youth and Family Services
(Contract Provider)
3430 NE 39th Avenue
Gainesville, Florida 32609
Primary Service: Female Healthy Relationships
SPEP Review Period: December 15, 2020-December 14, 2021
SPEP Review Conducted: December 28, 2021
Florida Department of Juvenile Justice
Report Date: 2/21/22
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Female Healthy Relationships.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 6 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes the Female Healthy Relationships bundle as a primary service. The Female
Healthy Relationships bundle is comprised of the VOICES curriculum and the Teen Relationship
Workbook curriculum. A review of documentation for the annual review period showed there were
six staff members trained in VOICES and four staff members trained in Teen Relationships. The
trained staff for the VOICES curriculum were the former clinical director, the current clinical
director, the regional clinical director, and three therapists. The trained staff for the Teen
Relationships curriculum were the clinical director and three therapists. An email from the
developer of VOICES states there is no formal training needed to conduct the curriculum;
however, most staff appreciate the training. The training for VOICES and Teen Relationships is
completed as an in-service training. The training for VOICES is sixteen hours in duration. The
training for Teen Relationships is four hours in duration. The program has a written protocol for
both services they follow for training facilitators. The clinical director or regional clinical director
conducts the training for Teen Relationships and VOICES. The documentation showed there were
two therapists who conducted Female Healthy Relationships groups during the annual review
period. There were four trained staff (former clinical director, current clinical director, regional
clinical director, and therapist) who conducted the monthly internal fidelity monitoring for the
VOICES service and three trained staff (therapist, the former clinical director, and current clinical
director) who conducted the monthly internal fidelity monitoring for the Teen Relationships
service. A trained staff member can conduct the monthly fidelity monitoring. The VOICES primary
service can be ran as a closed group or an open group. The regional clinical director stated
VOICES will now be ran as an open group. The Teen Relationships primary service is an open
group ran year-round.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
The program utilizes the VOICES: A program of Self-Discovery and Empowerment for Girls 2nd
edition as the manual. The manual is divided into two parts. The first part gives the facilitator
background information about girls and about the formation of the program. It includes the reality
of girl's lives: socialization and identity, culture and class, sexuality, violence, and aggressiveness.
It addresses expressions of risk. The second part includes four modules. The recommended
dosage is eighteen ninety-minute sessions. The sessions may be arranged in a variety of ways.
The curriculum is laid out in the suggested sequence. Groups are suggested to be between six
and ten girls and the groups can be closed or open. The youth have their own separate workbook.
VOICES is held on Wednesdays. The program keeps all documentation in a binder labeled
VOICES SPEP in the clinical director’s office. All youth who are admitted to the program receive
the primary service.
The program also utilizes Teen Relationships in conjunction with VOICES. The Teen
Relationships Workbook is broken down into six sections. The first section is evaluating your
relationship. Section two is understanding abuse. Section three is social influences. Section four
is building healthy relationships. Section five is making good decisions. Section six is the review.
There are sixty-eight reproducible worksheets that are given to the youth during the sessions.
Each lesson has information on how the facilitator should present the material. There are no
separate student workbooks. The facilitators make copies of the current lessons to hand out to the
youth. Teen Relationships is an open group ran year-round. The youth who are not in VOICES
will be placed in Teen Relationships until a new VOICES cohort begins. The program keeps all
documentation in a binder labeled Teen Relationships SPEP in the clinical director’s office. All
youth who are admitted to the program receive the primary service. The Teen Relationships
primary service is bundled with VOICES to ensure the youth receive the correct dosage for the
level of service.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of the VOICES component was observed on 12/28/2021 via Microsoft Teams. The
group began at 3:10p. There were eight youth and one staff. The therapist was facilitating the
group in one of the program’s classrooms. The facilitator followed the lesson, but did not read the
lesson verbatim. The facilitator used her own words and ensured the youth understood the lesson
concepts. The facilitator was knowledgeable of the material. The group ended at 4:05p.
A session of the Teen Relationship Workbook component was observed on 12/21/2021. The
group began at 1:48p. There were eight youth and two staff. The therapist was facilitating the
group in one of the program’s classrooms. The facilitator followed the lesson, but did not read the
lesson verbatim. The facilitator used her own words and ensured the youth understood the lesson
concepts. The facilitator was knowledgeable of the material. The group ended at 2:50p.
Basic Score: 0 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The documentation reviewed for the annual review period of December 15, 2020 through
December 14, 2021 shows the program experienced facilitator turnover. The former clinical
director and one therapist left the program. The therapist was facilitating the VOICES group at the
time she left. There was a gap in service from 1/29/2021 to July 2021. The executive director
stated the program had to evacuate the building on 1/29/2021 due to mold. All youth and staff
were transferred to another facility. The youth and staff were unable to go back to their home
location until June 2021. The last youth was transferred from the program’s care on 3/1/2021. The
program did not receive any additional youth until 6/4/2021. According to the regional clinical
director, the program officially resumed facilitating the primary service in July 2021 when the
program had six youth admitted.
The VOICES service can be ran as an open group or a closed group. The program was running
VOICES as a closed group. The regional clinical director stated the program will be running
VOICES as an open group from now on. There were two cohorts run during the annual review
period when the program was running the primary service as a closed group. The dates of the
cohorts were 12/4/2020-1/26/2021 and 7/29/2021-current. The cohort which began on 12/4/2020
began with three youth and ended with three youth. The cohort which began on 7/29/2021 began
with seven youth and currently has nine youth. The program is now running VOICES as an open
group.
The Teen Relationships service is ran as an open group ran year-round. The program ran the
service every month during the quarterly review period except the months the youth and staff
were relocated. There was a gap in service from February 2021 through the end of May 2021.
The executive director also stated the program continued to provide the primary service to the
youth while at the other program, but they could not locate any of the documentation. The
program began running the primary service again in June 2021. The program began admitting
youth on 6/4/2021. According to the regional clinical director, the SPEP service officially restarted
in July 2021. The program reached six youth admissions by July 2021.
According to the clinical director, the program’s practice guidelines, in the event of turnover, is one
of the other trained facilitators on staff will continue with the group cohort.
The program can earn 2 points if facilitator turnover does not impact groups and there are no
gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of the documentation for the annual review period of December 15, 2020 through
December 14, 2021 shows there were thirteen monthly internal fidelity monitorings completed
during the review period for Female Healthy Relationships. The dates of the fidelity monitoring for
VOICES were 1/15/2021, 12/14/2020, 9/23/2021, 8/26/2021, 10/28/2021, and 11/19/2021. The
dates of the fidelity monitoring for Teen Relationships were 12/21/2021, 11/16/2021, 10/28/2021,
9/23/2021, 8/31/2021, 1/25/2021, and 12/30/2020. There were no fidelities completed during the
months of February 2021, March 2021, April 2021, May 2021, June 2021, and July 2021 for both
services due to the program evacuating their home location on 1/29/2021. The program
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
transferred the last youth on 3/1/2021 and did not return to the facility until 6/4/2021. The fidelity
monitoring for VOICES were completed by the former clinical director, the current clinical director,
the regional clinical director, and one therapist. The fidelity monitoring for the Teen Relationships
service were completed by the former clinical director, current clinical director, and one therapist.
According to the clinical director, the clinical director monitors each group monthly and provides
SPEP fidelity checklist and status report for documentation. The clinical director or regional
clinical director discusses the fidelity check with the facilitator upon completion.
The regional clinical director stated the program could not find any additional fidelity monitorings
for the review period. All the documentation was boxed up when the program evacuated on
1/29/2021. The executive director stated all the program’s documentation was scattered during
the time the program was not in their home building. The program was unable to find any
additional documentation.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
According to the documentation reviewed for the annual review period of December 15, 2020
through December 14, 2021, the program did not identify any corrective actions on their monthly
internal fidelity monitorings of Female Healthy Relationships. The clinical director or regional
clinical director will discuss any identified corrective actions with the facilitator and will re-evaluate
during the next month’s fidelity check.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program provided three staff evaluations to review for the annual review period. There were
two ninety-day evaluations and one annual staff evaluation. The ninety-day evaluations were
completed on 12/21/2021 and 12/16/2021 for a therapist and a case manager. The annual
evaluation was completed on 12/15/2021 for a therapist. The three evaluations have an area to
document the facilitator’s skill delivering Female Healthy Relationships, but none contained any
information about the facilitator’s skill delivering the service. The ratings for the section were
checked, but lacked specific information for the facilitator’s skill delivering the primary service.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service - Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 24 weeks for this type of service. Of the 6 youth in the
sample, 0% (0 of 6) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 40 contact hours for this type of service. Of the 6 youth in the
sample, 67% (4 of 6) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 7 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 28%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 83%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 2 youth
Moderate-High = 3 youth
High = 0 youth
Total Youth in Sample = 6 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 0%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 0 youth
Total Youth in Sample = 6 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 83% (5 of 6) youth scored Moderate to High-Risk to reoffend, for a score of
7 points.
Of the SPEP sample, 0% (0 of 6) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 7 25 28%
Totals 53 95 56%
This SPEP report evaluates Female Healthy Relationships, an intervention delivered at Alachua
Academy.
Alachua Academy was temporarily relocated to another site during this review period, which
resulted in service interruption. These reports are not representative of a full review period. This
information should be taken into account when interpreting these reports.
6 youth completed the program during the review period. Of those youth, all 6 were included in
the sample.
The program scored Medium for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 6 total youth sampled, only
4 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 0
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
received at least the recommended weeks of service. Youth in the sample completed 19 weeks of
service.
The program earned 6 points for Amount of Service: Contact Hours. Of the 6 total youth sampled,
all 6 included dosage in the EBS Module. Of those youth, 4 received at least the recommended
hours of service. Youth in the sample completed between 12 and 51 hours of service, with an
average of 36 hours.
The program was awarded 7 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 75%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Alachua Academy can optimize their SPEP Quality of Service Delivery score by ensuring
facilitator turnover does not impact groups, there are no gaps in service, fidelity monitoring is
completed every month for each facilitator, and each facilitator’s performance evaluation includes
discussion of their skill in delivering this primary service.
Alachua Academy can optimize their SPEP Amount of Service score by ensuring that dosage for
all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

file_name: FY2021-22 Alachua - FHR.pdf

service_type: Female Healthy Relationships


summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Alachua Academy
Sequel Youth and Family Services
(Contract Provider)
3430 NE 39th Avenue
Gainesville, Florida 32609
Primary Service: Impact of Crime
SPEP Review Period: December 15, 2020-December 14, 2021
SPEP Review Conducted: December 28, 2021
Florida Department of Juvenile Justice
Report Date: 2/21/22
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 4 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes Impact of Crime (IOC) as a primary service. A review of documentation for
the annual review period showed nine staff were trained in IOC. The trained staff were the former
regional clinical director, former clinical director, the assistant facility administrator, one therapist,
and five case managers. The training is a three-day, 24-hour training currently being provided
online via Microsoft Teams. The documentation showed there were three case managers who
conducted groups during the annual review period. There were two trained staff who conducted
the monthly internal fidelity monitoring. A case manager and a therapist conducted the only two
fidelity monitorings for the annual review period. A trained staff member can conduct the monthly
fidelity monitoring.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the Impact of Crime: Addressing the Harm v 2.0 as the manual. The
facilitator’s manual is divided into seven chapters, an appendix with handouts, and the student
guide section (which includes the student exercises). The manual addresses the order in which to
deliver the content and gives the facilitators an outline to follow. The facility does not have
separate student workbooks to give out. The facilitators make copies of the exercises before each
session to pass out to the youth. Impact of Crime is a closed group with no more than twelve
youth in each cohort. The curriculum is to be delivered over a minimum of sixteen weeks for thirty-
two sessions. Impact of Crime is held on Mondays and Fridays. The program keeps all
documentation in a binder labeled IOC SPEP in the clinical director’s office. All youth who are
admitted to the program receive the primary service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Impact of Crime was observed on 12/21/2021. The group began at 10:05a. There
were nine youth and two staff. One staff was facilitating the group. The other staff member just
recently completed the facilitator training and was shadowing the facilitator. The group was held in
one of the program’s classrooms. The facilitator followed the lesson, but did not read the lesson
verbatim. The facilitator used her own words and ensured the youth understood the lesson
concepts. The facilitator was knowledgeable of the material. The group ended at 11:06a.
Basic Score: 0 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The documentation reviewed for the annual review period of December 15, 2020 through
December 14, 2021 shows the program experienced facilitator turnover. The program had four
case managers and the clinical director leave the program during the annual review period. The
program conducted three cohorts of Impact of Crime during the review period. The dates of the
cohorts were 10/4/2021-current, 7/29/2021-8/26/2021, and 10/27/2020-1/21/2021. There was a
gap of thirty-nine days between the cohort which abruptly ended on 8/26/2021 and the cohort
which began on 10/4/2021.
The cohort which began on 10/27/2020 began with six youth and ended with four youth. The last
attendance sheet provided for the cohort was dated for 1/21/2021. The facilitator documented the
cohort was on chapter five objective two. There were no other attendance sheets provided for the
cohort. The executive director stated the program had to evacuate the building on 1/29/2021 due
to mold. All youth and staff were transferred to another facility. The youth and staff were unable to
go back to their home location until June 2021. The last youth was transferred from the program’s
care on 3/1/2021. The program did not receive any additional youth until 6/4/2021.
The cohort which began on 7/29/2021 began with seven youth and ended with seven youth. The
facilitator left the program prior to the youth completing the cohort. The youth were transferred to
another IOC cohort. The regional clinical director stated the program only had one other trained
facilitator but the program did not have any other trained staff available to conduct the monthly
fidelity monitoring so the program waited until there were two additional staff trained in the service
before beginning a new cohort.
The cohort which began on 10/4/2021 began with ten youth and currently has ten youth. The
attendance sheets show the facilitator ran the group back to back on 10/7/2021 and 10/8/2021.
The cohort began with a facilitator and co-facilitator, but the co-facilitator left the program after the
first session.
According to the clinical director, the program’s practice guidelines in the event of turnover is one
of the other trained facilitators on staff will continue with the group cohort.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The program can earn 2 points if facilitator turnover does not impact groups and there are no
unnecessary gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 0 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of the documentation for the annual review period of December 15, 2020 through
December 14, 2021 shows there were two fidelity monitorings completed for facilitators of Impact
of Crime. The dates of the fidelity monitorings were 10/29/2021 and 11/4/2021. The program was
missing fidelity monitorings for the months of December 2020, and January through September
2021. The fidelity monitoring was completed by a case manager and a therapist.
According to the clinical director, the clinical director monitors each group monthly and provides
SPEP fidelity checklist and status report for documentation. The clinical director or regional
clinical director discusses the fidelity check with the facilitator upon completion.
The regional clinical director stated the program could not find any additional fidelity monitorings
for the review period. All of the documentation was boxed up when the program evacuated on
1/29/2021. The executive director stated all of the program’s documentation was scattered during
the time the program was not at the program. The program was unable to find any additional
documentation.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 0 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
According to the documentation reviewed for the annual review period of December 15, 2020
through December 14, 2021, the program did not identify any corrective actions on their two
monthly internal fidelity monitorings of Impact of Crime. The clinical director or regional clinical
director will discuss any identified corrective actions with the facilitator and will re-evaluate during
the next month’s fidelity check.
Internal fidelity monitoring was not conducted regularly throughout the review period; therefore no
points may be awarded in this section.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program provided three staff evaluations to review for the annual review period of December
15, 2020 through December 14, 2021. There were two ninety-day evaluations and one annual
staff evaluation. The ninety-day evaluations were completed on 12/21/2021 and 12/16/2021 for a
therapist and a case manager. The annual evaluation was completed on 12/15/2021 for a
therapist. The three evaluations have an area to document the facilitator’s skill delivering Impact
of Crime, but none contained any information about the facilitator’s skill delivering the service. The
ratings for the section were checked, but lacked specific information for the facilitator’s skill
delivering the primary service.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service - Duration
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates the target duration of 16 weeks for this type of service. Of the 6 youth in the
sample, 50% (3 of 6) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates a target of 24 contact hours for this type of service. Of the 6 youth in the
sample, 50% (3 of 6) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 7 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 28%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 83%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 2 youth
Moderate-High = 3 youth
High = 0 youth
Total Youth in Sample = 6 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 0%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 0 youth
Total Youth in Sample = 6 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 83% (5 of 6) youth scored Moderate to High-Risk to reoffend, for a score of
7 points.
Of the SPEP sample, 0% (0 of 6) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 4 10 40%
Amount of Service: Contact Hours 4 10 40%
Risk Level of Youth Served 7 25 28%
Totals 45 85 53%
This SPEP report evaluates Impact of Crime, an intervention delivered at Alachua Academy.
Alachua Academy was temporarily relocated to another site during this review period, which
resulted in service interruption. These reports are not representative of a full review period. This
information should be taken into account when interpreting these reports.
6 youth completed the program during the review period. Of those youth, 6 were included in the
sample.
The program scored Medium for Quality of Service Delivery.
The program earned 4 points for Amount of Service: Duration. Of the 6 total youth sampled, only
3 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 3
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
received at least the recommended weeks of service. Youth in the sample completed 17 weeks of
service.
The program earned 4 points for Amount of Service: Contact Hours. Of the 6 total youth sampled,
all 6 included dosage in the EBS Module. Of those youth, 3 received at least the recommended
hours of service. Youth in the sample completed between 22 and 38 hours of service, with an
average of 30 hours.
The program was awarded 7 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 74%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Alachua Academy can optimize their SPEP Quality of Service Delivery score by ensuring
facilitator turnover does not impact groups, there are no unnecessary gaps in service, fidelity
monitoring is completed every month for each facilitator, any corrective action needed is
identified, applied, and demonstrated, and each facilitator’s performance evaluation includes
discussion of their skill in delivering this primary service.
Alachua Academy can optimize their SPEP Amount of Service score by ensuring that dosage for
all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

file_name: FY2021-22 Alachua - IOC.pdf

service_type: Impact of Crime


summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Alachua Academy
Sequel Youth and Family Services
(Contract Provider)
3430 NE 39th Avenue
Gainesville, Florida 32609
Primary Service: Living In Balance
SPEP Review Period: December 15, 2020-December 14, 2021
SPEP Review Conducted: December 28, 2021
Florida Department of Juvenile Justice
Report Date: 2/21/22
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Living in Balance.
The program was awarded 25 points because the Primary Service is identified as a Group
<---Select from Drop-Down---> Service. The specific Sub-Component Service Type identified is
Group Counseling. The Primary Service was identified as this type of service as it focuses on
psychological or interpersonal problems or issues faced by an individual and involves a group of
youths interacting with each other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 6 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes Living in Balance (LIB). A review of documentation for the annual review
period showed four staff were trained in LIB. The trained staff were the former clinical director, the
current clinical director, and two therapists. The training is a two-hour in-service training delivered
by the clinical director or the regional clinical director. The program has a written protocol for the
in-service training which documents the schedule for the training. The documentation showed
there were two therapists who conducted groups during the annual review period. There were four
trained staff who conducted the monthly internal fidelity monitoring. Three therapists and the
current clinical director conducted the monthly internal fidelity monitorings for the annual review
period. Any trained staff member can conduct the monthly fidelity monitoring.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the Living in Balance: Moving from a Life of Addiction to a Life of Recovery
as the manual. The manual addresses the order in which to deliver the content and gives the
facilitators an outline to follow. The program does not have separate student workbooks to give
out. The facilitators make copies of the exercises before each session to pass out to the youth.
The primary service is an open group run year-round. Living in Balance is held on Tuesdays and
Thursdays. The program keeps all documentation in a binder labeled LIB SPEP in the clinical
director’s office. All youth who are admitted to the program receive LIB.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Living in Balance was observed on 12/21/2021. The group began at 11:15a. There
were thirteen youth and two staff. The therapist was facilitating the group and the clinical director
was conducting the monthly internal fidelity for the month of December 2022 for the facilitator. The
group was held in one of the program’s classrooms. The facilitator followed the lesson, but did not
read the lesson verbatim. The facilitator used her own words and ensured the youth understood
the lesson concepts. The facilitator was knowledgeable of the material. The group ended at 12p.
Basic Score: 0 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The documentation reviewed for the annual review period of December 15, 2020 through
December 14, 2021 shows the program experienced facilitator turnover. The program had one
therapist and one clinical director leave the program during the annual review period. Living in
Balance (LIB) was provided by a qualified facilitator during the review period. There is currently
one facilitator who can facilitate LIB, with the clinical director available if needed. LIB is an open
group run year-round. There are no more than fifteen youth in each cohort, as evidenced by the
sign in sheets.
There was a gap in service from February 2021 through the end of May 2021. The program
evacuated the building on 1/29/2021 and went to another program. The program transferred their
last two youth on 3/1/2021. The executive director stated the program had to evacuate due to a
mold problem in the building. The executive director also stated the program continued to provide
the primary service to the youth while at the other program, but they could not locate any of the
documentation.
The program began running the primary service again in June 2021, with fewer than the minimum
required youth to ensure all youth receive the necessary substance abuse treatment. The
program began admitting youth on 6/4/2021. According to the regional clinical director, the SPEP
service officially restarted in July 2021, when the program reached six youth admissions and
therefore qualified as a "group".
According to the clinical director, the program’s practice guidelines in the event of turnover is one
of the other trained facilitators on staff will continue with the group cohort.
The program can earn 2 points if facilitator turnover does not impact groups and there are no
unnecessary gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of documentation for the annual review period of December 15, 2020 through December
14, 2021 shows there were nine fidelity monitorings completed for facilitators of Living in Balance.
The dates of the fidelity monitorings were 12/21/2021, 11/9/2021, 10/27/2021, 9/22/2021,
8/19/2021, 1/12/2021, 1/12/2021, 12/17/2020, and 12/17/2020. The program was missing fidelity
monitorings for the months of June 2021 and July 2021 for one therapist. There were no fidelities
completed during the months of February 2021, March 2021, April 2021, and May 2021 due to the
program evacuating their home location on 1/29/2021. The program transferred the last youth on
3/1/2021 and did not return to the facility until 6/4/2021. The fidelity monitoring were completed by
three therapists. The clinical director or regional clinical director discusses the fidelity check with
the facilitator upon completion.
According to the clinical director, the clinical director monitors each group monthly and provides
SPEP fidelity checklist and status report for documentation. The clinical director or regional
clinical director discusses the fidelity check with the facilitator upon completion.
The regional clinical director stated the program could not find any additional fidelity monitorings
for the review period. All the documentation was boxed up when the program evacuated on
1/29/2021. The executive director stated all the program’s documentation was scattered during
the time the program was not in their home building. The program was unable to find any
additional documentation.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
According to the documentation reviewed for the annual review period of December 15, 2020
through December 14, 2021, the program did not identify any corrective actions on their monthly
internal fidelity monitorings of Living in Balance. The clinical director or regional clinical director
will discuss any identified corrective actions with the facilitator and will re-evaluate during the next
month’s fidelity check.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
The program provided three staff evaluations to review for the annual review period of December
15, 2020 through December 14, 2021. There were two ninety-day evaluations and one annual
staff evaluation. The ninety-day evaluations were completed on 12/21/2021 and 12/16/2021 for a
therapist and a case manager. The annual evaluation was completed on 12/15/2021 for a
therapist. The three evaluations have an area to document the facilitator’s skill delivering Living in
Balance, but none contained any information about the facilitator’s skill delivering the service. The
ratings for the section were checked, but lacked specific information for the facilitator’s skill
delivering the primary service.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service - Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 24 weeks for this type of service. Of the 6 youth in the
sample, 0% (0 of 6) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 40 contact hours for this type of service. Of the 6 youth in the
sample, 67% (4 of 6) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 7 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 28%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 83%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 2 youth
Moderate-High = 3 youth
High = 0 youth
Total Youth in Sample = 6 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 0%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 0 youth
Total Youth in Sample = 6 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 83% (5 of 6) youth scored Moderate to High-Risk to reoffend, for a score of
7 points.
Of the SPEP sample, 0% (0 of 6) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 7 25 28%
Totals 53 95 56%
This SPEP report evaluates Living in Balance, an intervention delivered at Alachua Academy.
Alachua Academy was temporarily relocated to another site during this review period, which
resulted in service interruption. These reports are not representative of a full review period. This
information should be taken into account when interpreting these reports.
6 youth completed the program during the review period. Of those youth, 6 were included in the
sample.
The program scored Medium for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. There were no end dates entered
into the Juvenile Justice Information system (JJIS) Evidence-Based Services (EBS) module for
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
any youths. With no JJIS data to review, the SPEP score was reduced. The program must
correctly enter dosage data into JJIS in order to receive any score in this area.
The program earned 6 points for Amount of Service: Contact Hours. Of the 6 total youth sampled,
all 6 included dosage in the EBS Module. Of those youth, 4 received at least the recommended
hours of service. Youth in the sample completed between 35 and 89 hours of service, with an
average of 62 hours.
The program was awarded 7 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 75%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Alachua Academy can optimize their SPEP Quality of Service Delivery score by ensuring
facilitator turnover does not impact groups, there are no unnecessary gaps in service, fidelity
monitoring is completed every month for each facilitator, and each facilitator’s performance
evaluation includes discussion of their skill in delivering this primary service.
Alachua Academy can optimize their SPEP Amount of Service score by ensuring that dosage for
all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

file_name: FY2021-22 Alachua - LIB.pdf