summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Lake Academy Secure
TrueCore Behavioral Solutions, LLC
(Contract Provider)
9504 East Columbus Drive
Tampa, Florida 33619
Primary Service: Impact of Crime
SPEP Review Period: 3/1/2022-2/28/2023
SPEP Review Conducted: 3/20/2023
Florida Department of Juvenile Justice
Report Date: 9/13/2023
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes Impact of Crime (IOC) as a primary service. There were five staff trained to
facilitate IOC groups during the annual review period- one Director of Case Management, one
Clinical Director, one Facility Administrator, one Assistant Facilitator, and one Delinquency
Intervention Coordinator. All training records were provided for review. There were two main
facilitators. The Clinical Director conducts internal fidelity monitoring.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the current Impact of Crime (IOC) manual and student workbooks. The manual
includes a script, facilitator notes, facilitation guidelines, and lesson plans. Impact of Crime groups
are held Tuesdays and Thursdays. All documentation for IOC groups are in a binder kept in the
Clinical Director's office. All youth are eligible to attend IOC.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Impact of Crime was observed on Tuesday, 12/6/22, from 3:37pm to 4:33pm. There
was one facilitator and three youth; Chapter Six: Managing Conflict was covered during this
session. The facilitator was knowledgeable of the material and used a combination of her own
word and the script in delivering the lesson while ensuring the youth understood the concepts
before moving on. The youth were very engaged and participated by reading, writing their
thoughts, and participating in the roleplay exercise.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation and an interview with the Clinical Director and the Assistant Facility
Administrator confirmed there was no facilitator turnover during the review period. In the event of
facilitator turnover, other staff trained in the Impact of Crime curriculum, or the co-facilitator would
take over facilitation of the group.
There were two cohorts completed during the review period; one additional cohort was ongoing at
the time of the review. The first cohort ran 5/11/22 through 8/3/22. This group started with three
youth and ended with two. The second cohort began 9/27/22 with five youth and ended 12/29/22
with three youth.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The Director of Case Management, Clinical Director and Assistant Clinical Director conduct
internal fidelity monitoring for Impact of Crime (IOC) groups. There were two main facilitators for
IOC during the annual review period. The internal fidelity forms were provided for a review. A
review of the documentation for the annual review period showed the program completed internal
fidelity monitorings for each facilitator during the months IOC groups were held, without exception.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The Director of Case Management addresses any issues or corrective actions for the facilitators
of Impact of Crime (IOC). The internal fidelity forms include a section to document any corrective
actions for the facilitators. All internal fidelity forms were provided for review and there were no
significant issues or corrective actions noted for the annual review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The annual performance evaluation includes a section to document which curricula the
employees are trained in and evaluated their skill. There were two main facilitators for Impact of
Crime groups during the annual review period. The annual performance evaluation were provided
for review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 16 weeks for this type of service. Of the 2 youth in the
sample, 0% (0 of 2) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 2 youth in the
sample, 100% (2 of 2) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 0 youth
Moderate-High = 1 youth
High = 1 youth
Total Youth in Sample = 2 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 50%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 1 youth
Total Youth in Sample = 2 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (2 of 2) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 50% (1 of 2) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 25 25 100%
Totals 75 85 88%
This SPEP report evaluates Impact of Crime, an intervention delivered at Lake Academy Secure.
2 youth completed the program during the review period. Of those youth, both were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 2 total youth sampled, both
included dosage with end dates in the EBS Module. Of those youth, 0 received at least the
recommended weeks of service. Youth in the sample completed 13 weeks of service.
The program earned 10 points for Amount of Service: Contact Hours. Of the 2 total youth
sampled, both included dosage in the EBS Module. Of those youth, 2 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
recommended hours of service. Youth in the sample completed between 46 and 57 hours of
service, with an average of 52 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
RECOMMENDATION(S):
Lake Academy Secure can maintain their SPEP Quality of Service Delivery score by continuing
the practices in place at the time of this review.
Lake Academy Secure can optimize their SPEP Amount of Service score by ensuring dosage for
all youth is recorded accurately in EBS and by ensuring youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8

file_name: FY2022-23 Lake Secure IOC.pdf


state: FL

facility_name: Lake Academy (Secure)

summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Lake Academy Secure
TrueCore Behavioral Solutions, LLC
(Contract Provider)
9504 East Columbus Drive
Tampa, Florida 33619
Primary Service: Aggression Replacement Training
SPEP Review Period: 3/1/2022-2/28/2023
SPEP Review Conducted: 3/20/2023
Florida Department of Juvenile Justice
Report Date: 9/13/2023
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Aggression Replacement Training.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes Aggression Replacement Training (ART) as a primary service. There were
six staff trained to facilitate ART groups during the annual review period- one Clinical Director,
one Licensed Mental Health Therapist, three Therapists and one Assistant Clinical Director. All
training records were provided for review. There was one main facilitator during the annual review
period. The Licensed Mental Health Therapist conducted internal fidelity monitoring.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the current Aggression Replacement Training (ART) manual and student
workbooks. The program runs ART groups on Tuesday, Thursday, and Saturday. The manual
includes a script, facilitator notes, facilitation guidelines, and lesson plans.
All documentation for ART groups are in a binder kept in the Clinical Director's office. All youth are
eligible to attend ART groups.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Aggression Replacement Training (ART) was observed on December 8, 2022.
Group started at 3:37 pm and ended at 4:33 pm. In attendance were one facilitator and three
youths. The facilitator made sure the youth fully grasped the concepts before moving on to the
next section. The youth participated in the group by reading and role play. The youth seemed to
enjoy the group and stated the therapist facilitating was their favorite in the program.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
Aggression Replacement Training (ART) was designated as a primary service as the result of a
newly negotiated contract; there was a delay in providing this service due to a slow intake of
youth. Lake Academy Secure did not have enough youth to form a proper group until late July
2022.
There was one cohort during the review period, running 8/4/22 through 10/11/22. The group
started with seven youth and ended with five youth; two youth were arrested and removed from
the group. A second cohort is slated to begin after the end of this review period, due to a
consistently low census and inability to form a full group..
A review of documentation and an interview with the Clinical Director confirmed there was
facilitator turnover during the review period. This turnover did not adversely affect delivery of
services. The Licensed Mental Health Therapist left the program in September this affected the
internal fidelity for the month of October. The program has since employed and trained four staff
in the ART curriculum. In the event of facilitator turnover, other staff trained in the Impact of Crime
curriculum, or the co-facilitator would take over facilitation of the group.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The Licensed Mental Health Therapist conducted internal fidelity monitoring for the Aggression
Replacement Training (ART) groups. There was one main facilitator for ART during the annual
review period. The internal fidelity forms were provided for a review. The LMHT left the program in
September; internal fidelity monitoring was not conducted the month of October.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The Clinical Director addresses any issues or corrective actions for the facilitators of Aggression
Replacement Training (ART). The internal fidelity forms include a section to document any
corrective actions for the facilitators. All internal fidelity forms were provided for review and there
were no significant issues or corrective actions noted for the annual review period.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The annual performance evaluation includes a section to document which curricula the
employees are trained in and evaluated their skill. There was one main facilitator for Aggression
Replacement Training groups during the annual review period. The annual performance
evaluation was provided for review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 10 weeks for this type of service. Of the 2 youth in the
sample, 100% (2 of 2) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 30 contact hours for this type of service. Of the 2 youth in the
sample, 100% (2 of 2) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 0 youth
Moderate-High = 1 youth
High = 1 youth
Total Youth in Sample = 2 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 50%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 1 youth
Total Youth in Sample = 2 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (2 of 2) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 50% (1 of 2) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 25 25 100%
Totals 100 100 100%
This SPEP report evaluates Aggression Replacement Training, an intervention delivered at Lake
Academy Secure.
2 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The targeted number of weeks for this specific intervention is 10 weeks; the targeted number of
hours for this specific intervention is 30 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 5 services (15 weeks; 45
hours).
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 10 points for Amount of Service: Duration. Of the 2 total youth sampled, both
included dosage with end dates in the EBS Module. Of those youth, 2 received at least the
recommended weeks of service. Youth in the sample completed 10 weeks of service.
The program earned 10 points for Amount of Service: Contact Hours. Of the 2 total youth
sampled, both included dosage in the EBS Module. Of those youth, 2 received at least the
recommended hours of service. Youth in the sample completed 30 hours of service.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
RECOMMENDATION(S):
Lake Academy Secure can optimize their SPEP Quality of Service Delivery score by ensuring
fidelity monitoring is completed every month for each facilitator.
Lake Academy Secure can optimize their SPEP Amount of Service score by ensuring dosage for
all youth is recorded accurately in EBS and by ensuring youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

source: DJJ

report_type: SPEP

file_name: FY2022-23 Lake Secure ART.pdf

program_name_raw: Lake Academy (Secure)

program_status_note: Closed

service_type: Aggression Replacement Training


summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Lake Academy (Non-Secure)
TrueCore Behavioral Solutions
(Contract Provider)
9504 East Columbus Drive
Tampa, Florida 33619
Primary Service: Individual Counseling
SPEP Review Period: 3/15/2023-3/14/2024
SPEP Review Conducted: 12/5/2023 and 12/6/2023
Florida Department of Juvenile Justice
Report Date: 9/22/2025
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 10 Points
1. Primary Service and Supplemental Service Types POS: 10 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Individual Counseling.
The program was awarded 5 points because the Primary Service is identified as a Group 1
Service. The specific Sub-Component Service Type identified is Individual Counseling. The
Primary Service was identified as this type of service as it includes any range of treatment
techniques that focus on psychological or interpersonal problems or issues faced by an individual
that involves a one-on-one relationship with a therapist or counselor.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The intervention being reviewed is Individual Counseling. There were twelve (12) trained
facilitators: four therapists, two assistant clinical directors, one licensed clinical psychologist, two
recreational therapists, one transition service manager, one delinquency coordinator, and one
clinical director. The therapists provide Individual Counseling to the youth in the program and the
clinical director provides monitoring of those clinicians. All clinicians provided counseling during
the review period. The program maintains all information regarding Individual Counseling in the
youth's mental health/substance abuse record. There was documentation supporting that each
Therapist had the education and experience to provide counseling sessions. This primary service
does not have a formalized curriculum. The facilitators receive additional training as needed,
which is documented during their clinical supervision.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
There is no written curriculum/manual for Individual Counseling. The program has a written
protocol for this intervention. The dosage is stated on the youth’s individualized treatment plan.
The plan is then reviewed and signed by a licensed clinician.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
There was no observation completed during the review due to the confidential nature of Individual
Counseling; therefore, this indicator was rated as non-applicable.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of the documentation and an interview with the clinical director confirmed there was no
facilitator turnover. In the event of turnover, the facility has other therapists employed by the
program that could assist in the delivery of Individual Counseling to their youth.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program documents fidelity monitoring through weekly supervisions and reviewing weekly
progress notes. The clinical director reviews and ensures compliance with dosage. There was
documentation of the clinical director conducting this review. The fidelity monitoring was
completed each month. The program documents fidelity monitoring through weekly supervisions
and reviewing weekly progress notes. The clinical director reviews and ensures compliance with
dosage. The fidelity monitoring was completed each month and the primary service was provided
for this review period. A review of the documentation showed that the program completed internal
fidelity monitorings.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process for corrective actions based on the fidelity monitoring conducted. A
review of documentation provided and an interview with the Clinical Director verified that there
was no corrective action taken during the review period.
According to the program's Standardized Program Evaluation Protocol, based on the results of
the monthly fidelity monitoring, corrective action may be needed. The first stage of addressing
areas identified as needing improvement through fidelity monitoring is by providing coaching and
supervision to the facilitator on that issue by a qualified facilitator in the primary service. The
additional coaching and/or supervision will be documented by the person conducting the fidelity
monitoring in a note and placed in the clinical supervision file. If during additional fidelity
monitoring the previously identified are addressed continue to be an issue, the next stage is
completing a written warning and placing the facilitator on a performance improvement plan. The
performance improvement plan will state the issue identified, the tasks the facilitator needs to
complete to correct the issues, and a target date for completion. The plan will then be reviewed
with the facilitator by the qualified facilitator conducting the monitoring and facilitator’s supervisor.
If an issue identified during the primary service is detrimental to the youth’s safety and /or
treatment, the matter will be taken up directly with the program’s Human Resource Manager and
be addressed in accordance with the employee handbook.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The annual performance evaluation the program utilizes has a section to document what curricula
the employees are trained in and evaluates the facilitation. There were twelve (12) facilitators for
Individual Counseling during the annual review period. Twelve (12) facilitator's annual
performance evaluations were provided for review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 25 weeks for this type of service. Of the 39 youth in the
sample, 67% (26 of 39) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 30 contact hours for this type of service. Of the 39 youth in the
sample, 74% (29 of 39) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 7 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 28%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 77%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 8 youth
Moderate-High = 18 youth
High = 4 youth
Total Youth in Sample = 39 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 10%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 4 youth
Total Youth in Sample = 39 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 77% (30 of 39) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 10% (4 of 39) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 10 10 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 7 25 28%
Totals 49 75 65%
This SPEP report evaluates Individual Counseling, an intervention delivered at Lake Academy .
40 youth completed the program during the review period. Of those youth, 39 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 6 points for Amount of Service: Duration. Of the 39 total youth sampled, only
30 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 26
received at least the recommended weeks of service. Youth in the sample completed between 18
and 59 weeks of service, with an average of 35 weeks.
The program earned 6 points for Amount of Service: Contact Hours. Of the 39 total youth
sampled, only 39 included dosage in the EBS Module. Of those youth with correct dosage, 29
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
received at least the recommended hours of service. Youth in the sample completed between 18
and 60 hours of service, with an average of 36 hours.
The program was awarded 7 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 89%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Lake Academy can maintain their SPEP Quality of Service Delivery score by continuing the
practices in place at the time of this review.
Lake Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2023-24 Lake Individual.pdf

service_type: Individual Counseling


summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Lake Academy (Secure)
TrueCore Behavioral Solutions
(Contract Provider)
9504 East Columbus Drive
Tampa, Florida 33619
Primary Service: Aggression Replacement Training
SPEP Review Period: 3/15/2023-3/14/2024
SPEP Review Conducted: 12/5/2023 and 12/6/2023
Florida Department of Juvenile Justice
Report Date: 9/22/2025
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Aggression Replacement Training.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The intervention being reviewed is Aggression Replacement Training (ART) for the annual review
period of March 15, 2023, through March 14, 2024. The program utilizes Aggression Replacement
Training (ART) as their primary service and is held on Mondays, Wednesdays and Fridays. There
are two staff who are trained to facilitate the primary service. The trained staff are the clinical
director and a therapist. Aggression Replacement Training is provided by a qualified trainer over a
three-day period. The primary service is a closed group. The assistant clinical director conducts
the monthly internal fidelity checks.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the current Aggression Replacement Therapy (ART) manual and student
workbooks. The manual and student workbooks were reviewed during the annual review period.
The program runs ART groups on Mondays, Wednesdays, and Fridays. The manual includes a
script, facilitator notes, facilitation guidelines, and lesson plans. All documentation for ART groups
are kept in a binder in the Clinical Director's office. All youth are eligible to attend ART groups.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Aggression Replacement Therapy group was observed on 12/6/23 by TAS Jerry
Dasher. The session started at 2:30pm and ended at 3:20pm. There were six youth, one staff and
one facilitator present. The session opened with the youth reviewing their homework. The youth
were very attentive and participated by reading and actively participating in the discussions being
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
held. The facilitator seemed very knowledgeable of the material and made sure the youth clearly
understood the topics being covered prior to moving to the next section. The facilitator used a
combination of her own words and reading from the lesson in delivery of the curriculum.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation and an interview with the Clinical Director confirmed that there was no
facilitator turnover during the review period. All youth receive the primary service. The facilitator is
assigned based on whether the youth participate in ART for clinical needs or criminogenic needs
as identified by the Community Assessment Tool and the treatment plan.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The Clinical Director conducts internal fidelity monitoring for the ART groups. There were six
facilitators for ART during the annual review period. The internal fidelity forms were provided for a
review. There were no fidelity monitoring conducted during the months of July, August, and
September due to a lack of census. The monthly internal fidelity monitoring was conducted on the
following dates: 3/30/23, 4/18/23, 5/16/23, 6/1/23, 10/30/23 and 11/13/23.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The Clinical Director addresses any issues or corrective actions for the facilitators of Aggression
Replacement Therapy (ART). The internal fidelity forms include a section to document any
corrective actions for the facilitators. Internal fidelity forms were provided for review and there
were no significant issues or corrective actions noted for the annual review period.
According to the facility's Standard Program Evaluation Protocol, based on the results of the
monthly fidelity monitoring corrective action may be needed. The first stage of addressing areas
identified as needing improvement through fidelity monitoring is by providing coaching and
supervision to the facilitator on that issue by a qualified facilitator in the primary service. The
additional coaching and/or supervision will be documented by the person conducting the fidelity
monitoring in a note and placed in the clinical supervision file. If during additional fidelity
monitoring the previously identified area addressed continues to be an issue, the next stage is
completing a written warning and placing the facilitator on a Performance Improvement Plan. The
Performance Improvement Plan will state the issue identified, the tasks the facilitator needs to
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
complete to correct the issue, and a target date for completion. The plan will then be reviewed
with the facilitator by the qualified facilitator conducting the monitoring, and the facilitator’s
supervisor. In the event that an issue identified during the primary service is detrimental to the
youth’s safety and/or treatment, the matter will be taken up directly with the program’s Human
Resource Manager and be addressed in accordance with the employee handbook.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The annual performance evaluation the program utilizes a section to document what curricula the
employees are trained in and evaluates the facilitation. There were six facilitators for Aggression
Replacement Therapy during the annual review period. The annual performance evaluations were
provided for review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 10 weeks for this type of service. Of the 5 youth in the
sample, 100% (5 of 5) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 30 contact hours for this type of service. Of the 5 youth in the
sample, 100% (5 of 5) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 10 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 40%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 60%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 0 youth
Moderate-High = 2 youth
High = 1 youth
Total Youth in Sample = 5 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 20%
High Score: 5 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 38%
Table 2
High = 1 youth
Total Youth in Sample = 5 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 60% (3 of 5) youth scored Moderate to High-Risk to reoffend, for a score of
5 points.
Of the SPEP sample, 20% (1 of 5) youth scored High-Risk to reoffend, for a score of 5 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 10 25 40%
Totals 85 100 85%
This SPEP report evaluates Aggression Replacement Training, an intervention delivered at Lake
Academy.
6 youth completed the program during the review period. Of those youth, 5 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 5 total youth sampled, only
5 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 5
received at least the recommended weeks of service. Youth in the sample completed between 10
and 10 weeks of service, with an average of 10 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 10 points for Amount of Service: Contact Hours. Of the 5 total youth
sampled, only 5 included dosage in the EBS Module. Of those youth with correct dosage, 5
received at least the recommended hours of service. Youth in the sample completed between 30
and 30 hours of service, with an average of 30 hours.
The targeted number of weeks for this specific intervention is 10 weeks; the targeted number of
hours for this specific intervention is 30 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 5 services (15 weeks; 45
hours).
The program was awarded 10 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 100%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Lake Academy can optimize their SPEP Quality of Service Delivery score by ensuring fidelity
monitoring is completed every month for each facilitator.
Lake Academy can maintain their SPEP Amount of Service score by continuing to ensure dosage
for all youth is recorded accurately in EBS and by continuing to ensure youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2023-24 Lake Academy Secure ART.pdf


summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Lake Academy (Secure)
TrueCore Behavioral Solutions
(Contract Provider)
9504 East Columbus Drive
Tampa, Florida 33619
Primary Service: Impact of Crime
SPEP Review Period: 3/15/2023-3/14/2024
SPEP Review Conducted: 12/5/2023 and 12/6/2023
Florida Department of Juvenile Justice
Report Date: 9/22/2025
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes Impact of Crime as a primary service for the secure side of the program.
During the annual review period of March 15, 2023, to March 14, 2024, there were eight staff
trained to facilitate IOC groups: one director of case management (DCM), four case managers
(CM), one clinical director (CD), two transition service managers (TSM) and one delinquency
coordinator (DC). There were no IOC groups conducted during this review period due to no
census on the secure side of the program.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the current Impact of Crime (IOC) manual and student and workbooks.
These were reviewed during the annual review period. The manual includes a script, facilitator
notes, facilitator guidelines, and lesson plans. Impact of crime groups are held Tuesdays and
Thursdays. All documentation for IOC groups is in a binder kept in the clinical director’s office. All
youth are eligible to attend IOC groups. Impact of Crime groups are closed and held on a constant
basis.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
During this review there was no IOC group observed due to a lack of census on the secure side of
the program.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
The program can earn 1 point if the program follows the manual/protocol when delivering the
primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
An interview with the clinical director confirmed there was no facilitator turnover. In the event of
turnover, the co-facilitator or other trained IOC facilitator employed by the program that could
assist in the delivery of Impact of Crime to their youth.
According to the program's Standardized Program Evaluation Protocol, Lake Academy utilizes a
probationary review status process designed for effective management of risk though
individualized coaching, training, and verification activities of all new employees during their first
ninety days. The process encourages personal growth and development, recognizes areas of
improvement, thus reducing staff turnover. In addition, the program provides positive incentives,
which in our experience has shown to improve employee motivation and dedication and thus
leads to better employee retention. Through this policy, employees receive credit for years of
service, employment referrals, and for contributing to the positive culture, teamwork environment
and for going above and beyond normal job expectations. Each employee that receives credit for
doing these things is entered in a monthly drawing at each facility. In the event of facilitator
turnover, the co-facilitator will assume the role of the facilitator for the group. If a co-facilitator is
not present, a new qualified facilitator for the primary service will take over conducting the service.
Basic Score: 0 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
According to the program's Standardized Program Evaluation Protocol, designated staff are
responsible for conducting fidelity monitoring of each facilitator of a primary service monthly. The
staff conducting the fidelity monitoring has received formal training by a qualified trainer in the
primary service. Fidelity monitoring is documented on fidelity monitoring tool developed
specifically for that primary service by the developer of the manual, or internal fidelity checklist
created for manualized or non-manualized curriculum delivered in the program. The staff
performing the fidelity monitoring of the primary services verifying the facilitator is conducting the
group in accordance with the manual/protocol. This includes verifying the facilitator adheres to the
session plan, group size, population, frequency, needed materials, duration, and interventions.
For the primary service of individual counseling, fidelity monitoring is conducted by licensed
clinicians in the review of therapy notes to verify provision of effective and appropriate intervention
in line with the treatment plan and review of trackers to ensure compliance with frequency and
duration.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process for corrective actions based on the fidelity monitoring conducted. The
program uses internal fidelity monitoring/corrective action checklists specific to Impact of Crime. A
review of documentation provided and an interview with the Clinical Director verified that there
was no corrective action taken during the review period.
According to the program's Standardized Program Evaluation Protocol, based on the results of
the monthly fidelity monitoring, corrective action may be needed. The first stage of addressing
areas identified as needing improvement through fidelity monitoring is by providing coaching and
supervision to the facilitator on that issue by a qualified facilitator in the primary service. The
additional coaching and/or supervision will be documented by the person conducting the fidelity
monitoring in a note and placed in the clinical supervision file. If during additional fidelity
monitoring the previously identified are addressed continue to be an issue, the next stage is
completing a written warning and placing the facilitator on a performance improvement plan. The
performance improvement plan will state the issue identified, the tasks the facilitator needs to
complete to correct the issues, and a target date for completion. The plan will then be reviewed
with the facilitator by the qualified facilitator conducting the monitoring and facilitator’s supervisor.
In the event that an issue identified during the primary service is detrimental to the youth’s safety
and /or treatment, the matter will be taken up directly with the program’s Human Resource
Manager and be addressed in accordance with the employee handbook.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The annual performance evaluation the program utilizes a section to document what curricula the
employees are trained in and evaluates the facilitation. There were eight facilitators for Impact of
Crime during the annual review period. The annual performance evaluations were provided for
review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 16 weeks for this type of service. Of the 6 youth in the
sample, 83% (5 of 6) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 24 contact hours for this type of service. Of the 6 youth in the
sample, 83% (5 of 6) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 8 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 32%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 67%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 1 youth
Moderate-High = 2 youth
High = 1 youth
Total Youth in Sample = 6 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 17%
High Score: 3 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 23%
Table 2
High = 1 youth
Total Youth in Sample = 6 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 67% (4 of 6) youth scored Moderate to High-Risk to reoffend, for a score of
5 points.
Of the SPEP sample, 17% (1 of 6) youth scored High-Risk to reoffend, for a score of 3 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 8 25 32%
Totals 54 85 64%
This SPEP report evaluates Impact of Crime, an intervention delivered at Lake Academy.
6 youth completed the program during the review period. Of those youth, 6 were included in the
sample.
The program scored Medium for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 6 total youth sampled, only
6 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 5
received at least the recommended weeks of service. Youth in the sample completed between 10
and 17 weeks of service, with an average of 15 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 6 total youth sampled,
only 6 included dosage in the EBS Module. Of those youth with correct dosage, 5 received at
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
least the recommended hours of service. Youth in the sample completed between 15 and 32
hours of service, with an average of 29 hours.
The program was awarded 8 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 84%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Lake Academy can optimize their SPEP Quality of Service Delivery score by ensuring fidelity
monitoring is completed every month for each facilitator.
Lake Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2023-24 Lake Academy Secure IOC.pdf

service_type: Impact of Crime


facility_name: Saint Johns Youth Academy

summary: SPEP - FY15-16

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
St John's Youth Academy
Sequel TSI of Florida, LLC
(Contract Provider)
4500 Avenue D
St Augustine, Florida 32085
Primary Service: Impact of Crime
SPEP Review Date(s): April 26‐29, 2016
Florida Department of Juvenile Justice
Report Date(s): 4/17/2017
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: St John's Youth Academy QI Program Code: 1266
Provider Name: Sequel TSI of Florida, LLC Contract Number: 10173
Location: St Johns County / Circuit: 7 Number of Beds: 70
Review Date(s): April 26-29, 2016 Lead Reviewer Code: 143
Persons Interviewed
Program Director Corporate QI/QA staff 1 # Program Supervisors
DJJ Monitor 3 # Case Managers # Youth
DHA or designee # Clinical Staff 1 # Other (listed by title): Corporate
DMHCA or designee # Healthcare Staff Clinical
Documents Reviewed
Written Protocol/Manual Logbooks 2 # Personnel Records
Fidelity Monitoring Documents Program Schedules 6 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts # Youth Records (Closed)
Staff Evaluations Table of Organization 9 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports # Health Records
Contract Scope of Services # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime. The program was awarded 15 points
because the Primary Service is identified as a Group 3 Service. The specific Sub-Component
Service Type identified is Social Skills Training. The Primary Service was identified as this type of
service as it focuses on developing social skills required for an individual to interact in a positive
way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program’s documentation reflected two case management staff had received formal training
in the delivery of Impact of Crime prior to facilitating any groups. Four additional case
management staff were trained in April 2016.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the Impact of Crime manual, version 2.0, to facilitate groups. The Impact of
Crime curriculum has lesson plans and handouts. The curriculum includes implementation
guidelines, identifies the order the lessons are to be delivered, and provides detailed instructions
for each lesson.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
This curriculum was not able to be observed; therefore, this indicator rates as non-applicable. The
program conducts this group twice weekly when groups are running.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The reviewed documentation and interview with the case management supervisor confirmed there
has been no turnover adversely affecting youth since the start of the contract. The trained
facilitators, who are conducting groups, have been in their positions since the start of the contract.
Documentation shows the program completed two cycles between September 2015 and March
2016. One cycle was September 2015 to January 2016 and one cycle was December 2015 to
March 2016. Each group had a different facilitator. Each facilitator started and ended with the
same group of youth. A new cycle is scheduled to begin May 2016.
BasicScore: 1Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has an internal process to conduct fidelity monitoring specific to Impact of Crime.
The program utilizes the Department’s fidelity monitoring checklist. The facilitators conduct
internal fidelity monitoring on each other. There was documentation fidelity monitoring was
conducted on one facilitator for two of the five required months. For the other facilitator, there
were two fidelity monitoring forms partially filled out. The date, time, group title, and number of
youth was blank.
The program can earn 2 points if fidelity monitoring is completed each month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
BasicScore: 1Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process for identifying deficiencies with the delivery of the primary service. A
review of the two complete fidelity monitoring checklists found the facilitator had adhered to the
written protocol. Fidelity monitoring reports did not identify a need for corrective action related to
the delivery of Impact of Crime.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program's policy states staff receive a probationary evaluation at ninety-days. Both facilitators
were eligible for a probationary performance evaluation. One facilitator had an evaluation dated
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
March 2016. The evaluation did not include an assessment of the facilitator skill in the delivery of
Impact of Crime. The other facilitator had not been given a performance evaluation at the time of
the review.
The program can earn 1 point if facilitators are evaluated on their skill in the delivery of this
primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 16 weeks for this type of service. Of the 4 youth in the
sample, 75% (3 of 4) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 4 youth in the
sample, 100% (4 of 4) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 1 youth
Moderate-High = 2 youth
High = 1 youth
Total Youth in Sample = 4 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 25%
High Score: 8 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 62%
Table 2
High = 1 youth
Total Youth in Sample = 4 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (4 of 4) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 25% (1 of 4) youth scored High-Risk to reoffend, for a score of 8 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 20 25 80%
Totals 66 85 78%
This SPEP report evaluates Impact of Crime, an intervention delivered at St John's Youth
Academy.
The program scored Medium for Quality of Service Delivery. This score can be optimized by
conducting fidleity monitoring and assessing facilitator skill in the employee performance
evaluations.
The program earned 6 points for Amount of Service: Duration. Of the 4 total youth sampled, 3
received at least the recommended weeks of service. Youth in the sample completed between 12
and 17 weeks of service, with an average of 15 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 4 total youth
sampled, all received at least the recommended hours of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
St John's Youth Academy can optimize their SPEP Quality of Service Delivery score by ensuring
fidelity monitoring is done monthly and evaluations address delivery of this primary service.
St John's Youth Academy can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

program_name_raw: Saint Johns Youth Academy