summary: SPEP - FY18-19

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Alachua Academy
Sequel Youth and Family Services
(Contract Provider)
3430 NE 39th Avenue
Gainesville, Florida 32609
Primary Service: Living in Balance
SPEP Review Date(s): 12/11-14/2018
Florida Department of Juvenile Justice
Report Date(s): 1/3/2019
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Living in Balance. The program was awarded 25 points
because the Primary Service is identified as a Group 4 Service. The specific Sub-Component
Service Type identified is Group Counseling. The Primary Service was identified as this type of
service as it focuses on psychological or interpersonal problems or issues faced by an individual
and involves a group of youths interacting with each other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes the Living in Balance curriculum as a primary service. A review of the
documentation from the prior 12 months showed there were two staff members trained in the
primary service. The two trained staff are therapists. One therapist was trained on 6/8/18 and the
other was trained on 10/17/17. The training is a two-hour in-service training delivered by the
Clinical Director or the Assistant Clinical Director. The program has a written protocol for the in-
service training.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the Living in Balance: Moving from a Life of Addiction to a Life of Recovery
as the manual. The program uses an electronic version of the manual. The Regional Clinical
Director stated it is easier to make copies of the lessons from the disk for the youth. The manual
addresses the order in which to deliver the content and gives the facilitators an outline to follow.
The facility does not have separate student workbooks to give out. The facilitators make copies of
the exercises before each session to pass out to the youth.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Living in Balance was observed on Thursday from 3p-4p. The facilitator provided a
copy of the lesson, Session 4 - Planning for Sobriety. The facilitator began with a check in and a
review of the previous lesson. The youth to role played exercises 4 and 5, and identified their
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
relapse warning signs. The group then read the full program of recovery. The facilitator wrapped
up the session and assigned homework to be completed before the next session.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
According to the documentation reviewed, there has been facilitator turnover at the program.
There are two staff members and the Clinical Director who can facilitate the Living in Balance
program. The program documentation from the prior 12 months shows there were four facilitators
running groups during the review period. Documentation showed the former Clinical Director and
a therapist ran a cohort in January 2018. Another former staff member facilitated a group in
February 2018. There were cohorts run during every month except March 2018. The facility was
on an admission freeze from December 2017 until May 2018. According to the Clinical Director,
the program tries to have at least 3 staff trained to facilitate the Living in Balance group in the
event of staff turnover.
The program can earn 2 points if there are no unnecessary gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Documentation was reviewed for the 12-month review period. Fidelity monitoring was conducted
for all but one of the applicable service months. No fidelity monitoring was conducted in January
2018; no groups were run March 2018. The Clinical Director and former Clinical Director
completed all the internal fidelity monitoring. The program has a process in place for conducting
fidelity monitoring. The Clinical Director conducts monthly fidelity monitoring of each facilitator
and discusses the results with the facilitator to ensure the facilitator is completing groups to the
fidelity of the model.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process in place to address corrective action resulting from fidelity monitoring.
The Clinical Director conducts monthly fidelity monitoring and discusses any findings with the
facilitator. The Clinical Director allows the facilitator two weeks to make corrections before being
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
re-evaluated. According to the reviewed documentation for the prior 12 months, there was no
corrective action noted on the fidelity monitoring reports.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Evaluations were reviewed for the three facilitators that were completed during the review period.
Each evaluation has an area on the evaluation called "Facilitator Skill Delivering the Evidence-
Based Programming". Each of the evaluations reviewed had specific information based on the
facilitator's delivery of the evidence-based programming.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 24 weeks for this type of service. Of the 26 youth in the
sample, 85% (22 of 26) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 40 contact hours for this type of service. Of the 26 youth in the
sample, 62% (16 of 26) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 96%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 7 youth
Moderate-High = 6 youth
High = 12 youth
Total Youth in Sample = 26 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 46%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 12 youth
Total Youth in Sample = 26 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 96% (25 of 26) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 46% (12 of 26) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 25 25 100%
Totals 89 95 94%
This SPEP report evaluates Living in Balance, an intervention delivered at Alachua Academy.
26 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 26 total youth sampled, only
24 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 22
received at least the recommended weeks of service. Youth in the sample completed between 21
and 61 weeks of service, with an average of 36 weeks.
The program earned 6 points for Amount of Service: Contact Hours. Of the 26 total youth
sampled, all 26 included dosage in the EBS Module. Of those youth with correct dosage, 16
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
received at least the recommended hours of service. Youth in the sample completed between 11
and 68 hours of service, with an average of 42 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Alachua Academy can optimize their SPEP Quality of Service Delivery score by ensuring there
are no unnecessary gaps in service, and by ensuring fidelity monitoring is completed every month
for each facilitator.
Alachua Academy can optimize their SPEP Amount of Service score by ensuring that dosage for
all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9


state: FL

facility_name: Alachua Academy

summary: SPEP - FY18-19

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Alachua Academy
Sequel Youth and Family Services
(Contract Provider)
3430 NE 39th Avenue
Gainesville, Florida 32609
Primary Service: VOICES
SPEP Review Date(s): 12/11-14/2018
Florida Department of Juvenile Justice
Report Date(s): 12/21/2018
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is VOICES.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes the VOICES curriculum as a primary service. A review of the documentation
from the prior 12 months showed seven staff members were trained in the primary service. There
are only two trained staff who are currently employed and can facilitate the service. Both of those
staff are therapists. One therapist was trained on 8/30/18-8/31/18 and the other was trained on
3/5/18-3/6/18. The other five staff are no longer employed at the facility. An email from the
developer of VOICES states there is no formal training required to conduct the curriculum;
however, the program has a written protocol they follow for training. The 16-hour training is
completed as an in-service training and is conducted by the Clinical Director.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the VOICES: A program of Self-Discovery and Empowerment for Girls 2nd
ed as the manual. A copy of the manual was provided for review. The manual is divided into two
parts. The 1st part gives the facilitator background information about girls and about the
formation of the program. It includes the reality of girl's lives: socialization and identity, culture
and class, sexuality, violence, and aggressiveness. It addresses expressions of risk. The 2nd
part includes four modules. The recommended dosage is 18 90-minute sessions. The sessions
may be arranged in a variety of ways. The curriculum is laid out in the suggested sequence.
Groups are suggested to be between six and 10 girls, and the groups should be closed.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
VOICES groups were held during a time the reviewer was not on-site, so this curriculum was
unable to be observed; therefore, this indicator rates as non-applicable.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
According to the documentation reviewed, there has been facilitator turnover at the program.
There were five staff members who were trained to facilitate VOICES and have left the program.
The program documentation from the prior 12 months shows there were two facilitators running
groups from April 2018-December 2018. Documentation showed there was a staff member who
began a cohort in January 2018 and stopped in February 2018. That staff member left the facility
in March 2018. There was no documentation of any groups running from February 2018 until
April 2018. The facility was on an admission freeze from December 2017 until May 2018. A
cohort began on 4/5/2018 with six youth, and ended on 8/9/2018. There was documentation the
Clinical Director began a cohort on 4/5/18 with the 6 youth due to the facility not currently running
the VOICES groups. The Clinical Director ended the cohort on 8/9/18 prematurely with none of
the six youth completing the curriculum. The Clinical Director stated she ended the cohort
prematurely, so the program could start a new cohort to include 20 youth. According to the
Clinical Director, the program tries to have at least three staff trained to facilitate the VOICES
group in the event of staff turnover.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
TAS reviewed the program's documentation for the 12-month review period. Fidelity monitoring
was conducted by the Clinical Director each month of service for each facilitator. There was no
fidelity monitoring completed in March 2018 because there were no groups being run during that
month. The program has a process in place for conducting fidelity monitoring. The Clinical
Director conducts monthly fidelity monitoring of each facilitator and discusses the results with the
facilitator to ensure the facilitator is completing groups to the fidelity of the model.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process in place to address corrective action resulting from fidelity monitoring.
The Clinical Director conducts monthly fidelity monitoring and discusses any findings with the
facilitator. The Clinical Director allows the facilitator two weeks to make corrections before being
re-evaluated. According to the reviewed documentation for the prior 12 months, there was no
corrective action noted on the fidelity monitoring reports.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Evaluations were reviewed for the three facilitators that were completed during the review period.
Each evaluation has an area on the evaluation called "Facilitator Skill Delivering the Evidence-
Based Programming". Each of the evaluations reviewed had specific information based on the
facilitator's delivery of the evidence-based programming.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 24 weeks for this type of service. Of the 24 youth in the
sample, 0% (0 of 24) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 40 contact hours for this type of service. Of the 24 youth in the
sample, 0% (0 of 24) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 96%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 7 youth
Moderate-High = 5 youth
High = 11 youth
Total Youth in Sample = 24 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 46%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 11 youth
Total Youth in Sample = 24 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 96% (23 of 24) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 46% (11 of 24) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 25 25 100%
Totals 75 95 79%
This SPEP report evaluates VOICES, an intervention delivered at Alachua Academy.
26 youth completed the program during the review period. Of those youth, 24 had not previously
received this intervention in another setting.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 24 total youth sampled, only
14 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 0
received at least the recommended weeks of service. Youth in the sample completed between 4
and 19 weeks of service, with an average of 14 weeks.
The program earned 0 points for Amount of Service: Contact Hours. Of the 24 total youth
sampled, only 20 included dosage in the EBS Module. Of those youth with correct dosage, 0
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
received at least the recommended hours of service. Youth in the sample completed between 4
and 27 hours of service, with an average of 16 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Alachua Academy can optimize their SPEP Quality of Service Delivery score by ensuring
facilitator turnover does not impact groups.
Alachua Academy can optimize their SPEP Amount of Service score by ensuring that dosage for
all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

source: DJJ

report_type: SPEP

program_name_raw: Alachua Academy - Sequel

program_status_note: Closed

service_type: VOICES: A Program of Self-Exploration and Empowerment


summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Alachua Academy
Sequel Youth and Family Services
(Contract Provider)
3430 NE 39th Avenue
Gainesville, Florida 32609
Primary Service: Impact of Crime
SPEP Review Date(s): 12/19‐20/2019
Florida Department of Juvenile Justice
Report Date(s): 12/20/2019
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
 Service Type: service type points are awarded based on the group category that the
service falls within
 Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
 Amount of Service: or “dosage” refers to the amount and duration of services that were
received
 Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime. The program was awarded 15 points
because the Primary Service is identified as a Group 3 Service. The specific Sub-Component
Service Type identified is Social Skills Training. The Primary Service was identified as this type of
service as it focuses on developing social skills required for an individual to interact in a positive
way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes the Impact of Crime curriculum as a primary service. A review of the
documentation from the prior twelve months showed there were seven staff members trained in
the primary service. Four of the trained staff are no longer employed with the facility. The three
remaining trained staff are the regional clinical director and two case managers. The training is
provided by a qualified trainer and lasts four days. The training involves role plays, lecture, and
activities to allow the staff the ability to gain experience using the curriculum during the training.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the Impact of Crime: Addressing the Harm v 2.0 as the manual. The
facilitator’s manual is divided into seven chapters, an appendix with handouts, and the student
guide section, which includes the student exercises. The manual addresses the order in which to
deliver the content and gives the facilitators an outline to follow. The facility does not have
separate student workbooks to give out. The facilitators make copies of the exercises before each
session to pass out to the youth. The group is a closed group with no more than twelve youth in
each cohort. The curriculum is to be delivered to the youth over a minimum of sixteen weeks for
thirty-two sessions.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The Technical Assistance Specialist observed a session of Impact of Crime. The group began late
due to an altercation between two youth before the group session started. The group began at
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
4:23p and ended at 5p. The group was working on chapter four, exercise 7: consequences of
making decisions under the influence of alcohol. The youth took turns reading the scenarios and
answering the questions. The facilitator read from the instructor’s manual. The facilitator did allow
conversations about side topics, but always brought the conversation back to the topic of poor
decision making. The group ended at 5p.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
According to the documentation reviewed, there has been facilitator turnover at the program. The
program had three different clinical directors during the review period. The first clinical director left
in June 2019. The next clinical director left in September 2019. The current clinical director began
in November 2019. The program also had two case managers and the facility administrator leave.
The facilitator turnover did not negatively affect the facilitation of Impact of Crime. The turnover of
the trained facilitators happened in-between the cohorts.
The documentation reviewed showed the program did have gaps in service. One gap was
between 4/25/19 and 6/4/19. The other gap was from 9/20/19 to 10/20/19. Three cohorts were
completed and two cohorts are currently running. The dates of the cohorts are 1/8/19-4/25/19,
6/4/19-9/20/19, 6/4/19-9/20/19, 10/20/19-current, and 12/3/19-current.
The program can earn 2 points if there are no unnecessary gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of documentation for the review period shows there were fourteen fidelity monitoring
completed during the annual review period, with monitoring take place at least monthly during the
months groups were held. There were no groups in May 2019. Each fidelity monitoring was
completed by either the clinical director, regional clinical director, or a trained facilitator. The
Technical Assistance Specialist spoke with the regional clinical director about the program's
internal fidelity monitoring process. The regional clinical director conducts monthly fidelity
monitoring of each facilitator until the new clinical director is trained. A trained facilitator may also
conduct the fidelity monitoring. The regional clinical director discusses the fidelity monitoring with
the facilitator to ensure the facilitator is completing groups to the fidelity of the model.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program's corrective action process begins with the monthly internal fidelity monitoring
completed by the either the clinical director or the regional clinical director. The regional clinical
director or the clinical director then discusses any findings with the facilitator. The facilitator will be
allowed two weeks to make corrections and then will be re-evaluated. A review of the fidelity
monitoring shows there were no corrective actions noted nor taken during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The Technical Assistance Specialist reviewed the evaluations for two of the three facilitators
completed during the review period. There is one case manager who just reached her ninety-days
in the program and the evaluation is waiting to be reviewed. Each evaluation has an area on the
evaluation called facilitator skill delivering the evidence-based programming. Each of the
evaluations reviewed had specific information based on the facilitator's delivery of the evidence-
based programming.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service ‐ Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 16 weeks for this type of service. Of the 37 youth in the
sample, 73% (27 of 37) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 24 contact hours for this type of service. Of the 37 youth in the
sample, 73% (27 of 37) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 13 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 52%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 59%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 4 youth
Moderate-High = 8 youth
High = 10 youth
Total Youth in Sample = 37 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 27%
High Score: 8 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 62%
Table 2
High = 10 youth
Total Youth in Sample = 37 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 59% (22 of 37) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 27% (10 of 37) youth scored High-Risk to reoffend, for a score of 8 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 13 25 52%
Totals 65 85 76%
This SPEP report evaluates Impact of Crime, an intervention delivered at Alachua Academy.
37 youth completed the program during the review period. Of those youth, all 37 were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 6 points for Amount of Service: Duration. Of the 37 total youth sampled, only
29 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 27
received at least the recommended weeks of service. Youth in the sample completed between 2
and 19 weeks of service, with an average of 16 weeks.
The program earned 6 points for Amount of Service: Contact Hours. Of the 37 total youth
sampled, only 29 included dosage in the EBS Module. Of those youth with correct dosage, 27
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
received at least the recommended hours of service. Youth in the sample completed between 8
and 36 hours of service, with an average of 33 hours.
The program was awarded 13 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 91%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Alachua Academy can optimize their SPEP Quality of Service Delivery score by ensuring there
are no unnecessary gaps in service.
Alachua Academy can optimize their SPEP Amount of Service score by ensuring that dosage for
all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

service_type: Impact of Crime


summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Alachua Academy
Sequel Youth and Family Services
(Contract Provider)
3430 NE 39th Avenue
Gainesville, Florida 32609
Primary Service: Living in Balance
SPEP Review Date(s): 12/19‐20/2019
Florida Department of Juvenile Justice
Report Date(s): 12/20/2019
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
 Service Type: service type points are awarded based on the group category that the
service falls within
 Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
 Amount of Service: or “dosage” refers to the amount and duration of services that were
received
 Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Living in Balance. The program was awarded 25 points
because the Primary Service is identified as a Group 4 Service. The specific Sub-Component
Service Type identified is Group Counseling. The Primary Service was identified as this type of
service as it focuses on psychological or interpersonal problems or issues faced by an individual
and involves a group of youths interacting with each other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes the Living in Balance curriculum as a primary service. A review of
documentation for the annual review period shows there are six staff who were trained in the
primary service. There are now only two of those staff who are still employed with the program.
The training is a 2-hour in-service training delivered by the clinical director or the regional clinical
director. The program has a written protocol for the in-service training.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the Living in Balance: Moving from a Life of Addiction to a Life of Recovery
as the manual. The manual addresses the order in which to deliver the content and gives the
facilitators an outline to follow. The facility does not have separate student workbooks to give out.
The facilitators make copies of the exercises before each session to pass out to the youth.
The primary service is an open group which is run year-round. There were four staff who
facilitated the groups during the review period.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The Technical Assistance Specialist observed a session on 12/19/19. The group began at 3:11p
and ended at 3:59p. The facilitator began with a group check in. The facilitator asked for
volunteers to conduct a role play. After the role play, the facilitator asked the group what
forgiveness means. The group passed out the workbooks. The youth read one question from the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
exercise then asked another youth to answer that question. The facilitator then asked the youth
what are the three reasons you feel guilty today. One youth answered the question and the two
other follow up questions for the exercise. Another youth wanted to answer the questions as well.
The facilitator connected the fact the two youth are feeling the same way for certain things. A
youth asked the facilitator what she feels guilty about. The facilitator shared with the youth what
she feels guilty about. The youth read childhood fear and answered questions for exercise
eighteen. The facilitator explained what the group will be doing for the next session. The group did
the check out and ended the session.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation showed there was facilitator turnover at the program. The program had
two different clinical directors and two therapists leave. The program began the year with two
therapists facilitating the service and ended the year with two different therapists facilitating the
service. There is a new clinical director at the program. Living in Balance was provided by a
qualified facilitator during the review period. There are currently two facilitators who can facilitate
the primary service.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of documentation for the review period shows there were eighteen fidelity monitorings
completed during the annual review period. Internal fidelity monitoring was conducted at least
monthly for each facilitator of Living in Balance, without exception. Each of the fidelity monitorings
was completed by either the clinical director or the regional clinical director. The regional clinical
director or the clinical director conducts monthly fidelity monitoring of each facilitator. The
regional clinical director discusses the fidelity monitoring with the facilitator to ensure the facilitator
is completing groups to the fidelity of the model.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program's corrective action process begins with the monthly internal fidelity monitoring
completed by the either the clinical director or the regional clinical director. The regional clinical
director or the clinical director then discusses any findings with the facilitator. The facilitator will be
allowed two weeks to make corrections and then will be re-evaluated. A review of the fidelity
monitoring during the review period shows there were no corrective actions noted nor taken.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The Technical Assistance Specialist reviewed the evaluations for the four facilitators completed
during the review period. Each evaluation has an area on the evaluation called facilitator skill
delivering the evidence-based programming. Each of the evaluations that were reviewed had
specific information based on the facilitator's delivery of the evidence-based programming.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service ‐ Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 24 weeks for this type of service. Of the 37 youth in the
sample, 95% (35 of 37) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 40 contact hours for this type of service. Of the 37 youth in the
sample, 92% (34 of 37) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 13 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 52%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 59%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 4 youth
Moderate-High = 8 youth
High = 10 youth
Total Youth in Sample = 37 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 27%
High Score: 8 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 62%
Table 2
High = 10 youth
Total Youth in Sample = 37 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 59% (22 of 37) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 27% (10 of 37) youth scored High-Risk to reoffend, for a score of 8 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 13 25 52%
Totals 79 95 83%
This SPEP report evaluates Living in Balance, an intervention delivered at Alachua Academy.
37 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 37 total youth sampled, all
37 included dosage with end dates in the EBS Module. Of those youth, 35 received at least the
recommended weeks of service. Youth in the sample completed between 13 and 61 weeks of
service, with an average of 34 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 37 total youth
sampled, all 37 included dosage in the EBS Module. Of those youth, 34 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
recommended hours of service. Youth in the sample completed between 35 and 91 hours of
service, with an average of 57 hours.
The program was awarded 13 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 96%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Alachua Academy can optimize their SPEP Quality of Service Delivery score by ensuring
facilitator turnover does not impact groups.
Alachua Academy can optimize their SPEP Amount of Service score by ensuring that dosage for
all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

service_type: Living in Balance


summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Alachua Academy
Sequel Youth and Family Services
(Contract Provider)
3430 NE 39th Avenue
Gainesville, Florida 32609
Primary Service: VOICES
SPEP Review Date(s): 12/19/19‐12/20/19
Florida Department of Juvenile Justice
Report Date(s): 12/20/2019
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
 Service Type: service type points are awarded based on the group category that the
service falls within
 Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
 Amount of Service: or “dosage” refers to the amount and duration of services that were
received
 Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is VOICES. The program was awarded 25 points because
the Primary Service is identified as a Group 4 Service. The specific Sub-Component Service
Type identified is Group Counseling. The Primary Service was identified as this type of service as
it focuses on psychological or interpersonal problems or issues faced by an individual and
involves a group of youths interacting with each other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes the VOICES curriculum as a primary service. A review of documentation
showed there were three staff members trained in the primary service. All three of those staff are
therapists. One therapist was trained on 8/30/18-8/31/18 another was trained on 3/5/18-3/6/18
and the third staff was trained on 5/9/19-5/10/19. Two of the therapists who were facilitating
groups are no longer employed with the program. An email from the developer of VOICES states
there is no formal training needed to conduct the curriculum; however, most staff appreciate the
training. The training is completed as an in-service training. The training is sixteen hours in
duration. The program has a written protocol they follow for training. The clinical director or
regional clinical director conducts the training.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the VOICES: A program of Self-Discovery and Empowerment for Girls, 2nd
edition, as the manual. The manual is divided into two parts. The first part gives the facilitator
background information about girls and about the formation of the program. It includes the reality
of girl's lives: socialization and identity, culture and class, sexuality, violence, and aggressiveness.
It addresses expressions of risk. The second part includes four modules. The recommended
dosage is eighteen ninety-minute sessions. The sessions may be arranged in a variety of ways.
The curriculum is laid out in the suggested sequence. Groups are suggested to be between six
and ten girls and the groups should be closed.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The Technical Assistance Specialist observed a session of VOICES. The session began at 9a
and ended at 9:50a. The facilitator had the youth read specific sections in the workbook. The
group discussed stressors the youth had to overcome in the past few months. They also
discussed what is in the youth’s power to help avoid the stressor. They differentiated between
positive and negative stressors. There was one youth who sat in a corner of the room and did not
want to participate. The youth would give short answers or wouldn’t answer the questions at all.
The facilitator asked the youth how they cope with stress. They discussed positive coping
strategies the youth use, as well as two skills the youth can work on. The facilitator also asked
how the youth’s substance use impacted their family. This group ended the substance abuse unit.
The facilitator assigned the youth homework to complete before Monday’s session. The clinical
director brought the therapy dog into the session. The youth loved having the therapy dog in the
session. They all took turns petting the dog.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
According to documentation reviewed, there has been facilitator turnover at the program. The
documentation for the review period shows there were three facilitators running the groups. One
facilitator left the program in May 2019 and another left in July 2019. Both facilitators left after the
end of their cohort. The clinical director also left in June 2019. There was another clinical director
who was at the program from 7/8/19-9/6/19. That clinical director is no longer at the program
either. The turnover did not affect delivery of VOICES. According to the regional clinical director,
the program tries to have at least three staff trained to facilitate the VOICES group in the event of
staff turnover. The regional clinical director stated those three staff would be two therapist and the
clinical director.
There were two cohorts completed during the review period. There is currently one cohort in
session. The dates of the cohorts are 9/14/18-4/8/19, 9/19/18-3/29/19, and 7/25/19-current. The
two completed cohorts were running at least two times per week. The cohort which began on
7/25/19 was only being run one time a week until November, when the program began providing
the service two times per week. There were no groups run from April 2019 through the end of July
2019. The regional clinical director stated the program did not provide the service due to waiting
on workbooks. There was also a gap of two weeks between 8/9/19-8/28/19.
The program can earn 2 points if there are no unnecessary gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of documentation for the review period shows there were twelve fidelity monitorings
completed during the review period. Internal fidelity monitoring was conducted at least monthly for
each facilitator during the months groups were held. Each of the fidelity monitorings was
completed by either the clinical director or the regional clinical director. There was no fidelity
monitoring completed during April or May, 2019, as there were no groups being ran during that
time.
The regional clinical director or the clinical director conducts monthly fidelity monitoring of each
facilitator. The regional clinical director discusses the fidelity monitoring with the facilitator to
ensure the facilitator is completing groups to the fidelity of the model.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program's corrective action process begins with the monthly internal fidelity monitoring
completed by the clinical director or the regional clinical director. The regional clinical director then
discusses any findings with the facilitator. The facilitator will be allowed two weeks to make
corrections and then will be re-evaluated the facilitator. According to the reviewed documentation
for the prior twelve months, there was no corrective action noted on the fidelity monitoring reports.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The Technical Assistance Specialist reviewed the evaluations for two of the three facilitators
completed during the review period. There were two staff members who left the program who had
facilitated VOICES. One of the facilitators left before her annual evaluation was due. The other
facilitator transferred to another program. The evaluations have an area on the evaluation called
facilitator skill delivering the evidence-based programming. The of the evaluations reviewed had
specific information based on the facilitator's delivery of the evidence-based programming.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service ‐ Duration
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates the target duration of 24 weeks for this type of service. Of the 36 youth in the
sample, 25% (9 of 36) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 40 contact hours for this type of service. Of the 36 youth in the
sample, 0% (0 of 36) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 13 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 52%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 58%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 4 youth
Moderate-High = 8 youth
High = 9 youth
Total Youth in Sample = 36 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 25%
High Score: 8 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 62%
Table 2
High = 9 youth
Total Youth in Sample = 36 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 58% (21 of 36) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 25% (9 of 36) youth scored High-Risk to reoffend, for a score of 8 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 2 10 20%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 13 25 52%
Totals 65 95 68%
This SPEP report evaluates VOICES, an intervention delivered at Alachua Academy.
37 youth completed the program during the review period. Of those youth, 36 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 2 points for Amount of Service: Duration. Of the 36 total youth sampled, only
18 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 9
received at least the recommended weeks of service. Youth in the sample completed between 15
and 31 weeks of service, with an average of 23 weeks.
The program earned 0 points for Amount of Service: Contact Hours. Of the 36 total youth
sampled, only 18 included dosage in the EBS Module. Of those youth with correct dosage, 0
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
received at least the recommended hours of service. Youth in the sample completed between 11
and 32 hours of service, with an average of 22 hours.
The program was awarded 13 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 81%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Alachua Academy can optimize their SPEP Quality of Service Delivery score by ensuring there
are no unnecessary gaps in service.
Alachua Academy can optimize their SPEP Amount of Service score by ensuring that dosage for
all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9


summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Alachua Academy
Sequel Youth and Family Services
(Contract Provider)
3430 NE 39th Avenue
Gainesville, Florida 32609
Primary Service: Female Healthy Relationships
SPEP Review Period: 12/15/19 - 12/14/20
SPEP Review Conducted: 12/15/20
Florida Department of Juvenile Justice
Report Date: 1/28/21
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Female Healthy Relationships.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes the Female Health Relationships bundle as a primary service. The Female
Healthy Relationships bundle is comprised of "Voices: A Program of Self-Discovery and
Empowerment for Girls" curriculum and the Teen Relationships Workbook. A review of
documentation for the annual review period of December 2019 through December 2020 showed
there were three staff members trained in both components of the Female Healthy Relationships
bundle. The training for Female Healthy Relationships is completed as an in-service training by
the Clinical Director or Regional Clinical Director, with sixteen (16) hours' training specific to
Voices and four hours specific to The Teen Relationships Workbook. The training for VOICES is
sixteen hours in duration. The training for Teen Relationships is four hours in duration. The
program has a written protocol for both services.
The program designated the Female Healthy Relationships bundle in March 2020.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the VOICES: A program of Self-Discovery and Empowerment for Girls 2nd
edition as the manual. The manual is divided into two parts. The first part gives the facilitator
background information about girls and about the formation of the program. It includes the reality
of girl's lives: socialization and identity, culture and class, sexuality, violence, and aggressiveness.
It addresses expressions of risk. The second part includes four modules. The recommended
dosage is eighteen (18) ninety-minute sessions. The sessions may be arranged in a variety of
ways. The curriculum is laid out in the suggested sequence. Groups are suggested to be between
six and ten girls and the groups should be closed.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
As the second component of the Female Healthy Relationships bundle, the program also utilizes
the Teen Relationships Workbook. The Teen Relationships Workbook is broken down into six
sections. The first section is evaluating your relationship. Section two is understanding abuse.
Section three is social influences. Section four is building healthy relationships. Section five is
making good decisions. Section six is the review. There are sixty-eight reproducible worksheets
that are given to the youth during the sessions. Each lesson has information on how the facilitator
should present the material. There are no separate student workbooks. The facilitators make
copies of the current lessons to hand out to the youth. Teen Relationships is an open group run
year-round.
All youth receive this service. The youth who are not in VOICES will be placed in Teen
Relationships until a new VOICES cohort begins.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of VOICES was observed on 12/14/2020. The group began at 3:03p. The youth were
spread out in the classroom and all youth and staff had their masks on. The facilitator began
group by sending the sign in sheet around and had the youth check in. The youth were asked to
restate the group rules. The facilitator then asked for a volunteer to write on the board. The
facilitator switched the lessons for the session because the technical assistance specialist was
observing group. The group was supposed to be going over their individual life story, but the
facilitator felt it would be best for the youth to complete another lesson because the youth might
not fully get everything out of the session with the technical assistance specialist observing. The
youth were asked to define what self-esteem means to them. The youth were then asked to read
the activity instructions from the workbook. The youth were then asked to put statements
(messages) they have heard others state about them on the roots of a tree. The facilitator then
asked the youth to create their own tree with roots and leaves. The youth then looked through
magazines to find words and phrases which represent the messages they have heard about them
and then cut out words and phrases or pictures and place them on the tree roots. The activity will
be continued next session. The facilitator ended group with a check out at 4:03p.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
According to documentation reviewed for the annual review period, there has not been facilitator
turnover at the program. The documentation shows there were two facilitators running both
components of the Female Healthy Relationships bundle. According to the Regional Clinical
Director, the program tries to have at least three staff trained to facilitate groups to avoid gaps in
service in the event of staff turnover.
There was a gap in service delivery from 12/16/2019 until 1/23/2020, as the program did not want
to begin the service during the holiday season. While the Female Healthy Relationships bundle
was designated in March 2020, the program did not begin using The Teen Relationships
Workbook until 9/16/2020.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
There were two cohorts of VOICES completed during the annual review period. There is currently
one cohort of Female Healthy Relationships in session. The dates of the cohorts are 1/23/2020-
6/1/2020, 6/17/2020-10/26/2020 (partially utilizing the Teen Relationships Workbook), and
11/17/2020-current (fully utilizing the Teen Relationships Workbook). The two completed cohorts
were running at least two times per week. The current cohort is being run at least two times per
week.
The program can earn 2 points if there are no unnecessary gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of documentation shows there were fourteen fidelity monitorings completed during the
review period for Female Healthy Relationships. Fidelity monitoring took place for each facilitator
each month. Each fidelity monitoring was completed by either the Clinical Director or the Regional
Clinical Director.
The Regional Clinical Director stated she or the Clinical Director conducts monthly fidelity
monitoring of each facilitator. The Regional Clinical Director discusses the fidelity monitoring with
the facilitator to ensure the facilitator is completing groups to the fidelity of the model.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The Regional Clinical Director stated the program's corrective action process begins with the
monthly internal fidelity monitoring completed by the Clinical Director or the Regional Clinical
Director. The monitor will discuss any findings with the facilitator and allow two weeks to make
corrections before being re-evaluated the facilitator. According to the reviewed documentation for
the prior twelve months, there were no corrective actions noted nor taken.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The annual evaluations of both facilitators have an area called "facilitator skill delivering the
evidence-based programming". The of the evaluations reviewed had specific information based
on the facilitator's delivery of Female Healthy Relationships.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 24 weeks for this type of service. Of the 26 youth in the
sample, 15% (4 of 26) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates a target of 24 contact hours for this type of service. Of the 26 youth in the
sample, 31% (8 of 26) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 10 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 40%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 85%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 7 youth
Moderate-High = 11 youth
High = 4 youth
Total Youth in Sample = 26 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 15%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 4 youth
Total Youth in Sample = 26 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 85% (22 of 26) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 15% (4 of 26) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 2 10 20%
Risk Level of Youth Served 10 25 40%
Totals 62 95 65%
This SPEP report evaluates Female Healthy Relationships, an intervention delivered at Alachua
Academy.
27 youth completed the program during the review period. Of those youth, 25 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 25 total youth sampled, only
15 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 4
received at least the recommended weeks of service. Youth in the sample completed between 10
and 24 weeks of service, with an average of 20 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 2 points for Amount of Service: Contact Hours. Of the 25 total youth
sampled, only 15 included dosage in the EBS Module. Of those youth with correct dosage, 8
received at least the recommended hours of service. Youth in the sample completed between 10
and 46 hours of service, with an average of 36 hours.
The program was awarded 10 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 81%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Alachua Academy can optimize their SPEP Quality of Service Delivery score by ensuring there
are no unnecessary gaps in service.
Alachua Academy can optimize their SPEP Amount of Service score by ensuring that dosage for
all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

service_type: Female Healthy Relationships