summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Les Peters Girls Academy
TrueCore Behavioral Solutions, LLC
(Contract Provider)
3930 West Martin Luther King Jr. Blvd.
Tampa, Florida 33614
Primary Service: Individual Counseling
SPEP Review Period: 08/01/20-07/31/21
SPEP Review Conducted: 08/03/21
Florida Department of Juvenile Justice
Report Date: 10/14/2021
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 10 Points
1. Primary Service and Supplemental Service Types POS: 10 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Individual Counseling.
The program was awarded 5 points because the Primary Service is identified as a Group 1
Service. The specific Sub-Component Service Type identified is Individual Counseling. The
Primary Service was identified as this type of service as it includes any range of treatment
techniques that focus on psychological or interpersonal problems or issues faced by an individual
that involves a one-on-one relationship with a therapist or counselor.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
During the review period, there were eight clinicians trained to provide Individual Counseling to
the youth- five Therapists, one as needed Therapist (PRN), one Assistant Clinical Director and
one Director of Clinical Services. All eight provided services and the Director of Clinical Services
provided fidelity monitoring. The program maintains all information regarding counseling in the
youth's mental health/substance abuse record. There was documentation to support each
therapist had the education and experience to provide counseling.
For this service, which does not have a formal curriculum, the facilitator receives training from a
licensed clinician. The facilitator receives additional training as needed, which is documented
during their clinical supervision. Three Therapists and the PRN Therapist are no longer employed
at Les Peters Academy. After those staff left, the two remaining Therapists and the Assistant
Clinical Director and Director of Clinical Services continued to facilitate Individual Counseling for
the youth.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
There is no written curriculum/manual for individual counseling. The program has written protocol
for this intervention. The dosage/frequency is stated on each youth's individualized treatment
plan. This plan is then reviewed and signed by a licensed clinician.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
This service was not observed during the review due to the nature of Individual Counseling;
therefore, this indicator rates as non-applicable.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
Upon review of documentation and an interview with the Director of Clinical Services, there has
been no facilitator turnover during the review period which adversely affected Individual
Counseling services to the youth. In the event of turnover, other trained Therapists would facilitate
individual counseling.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has documented fidelity monitoring for observations of Individual Counseling. he
fidelity monitoring was completed by the program's Director of Clinical Services and their
Assistant Clinical Director. The monitoring is completed through a review of therapy notes to verify
effective and appropriate interventions which are in line with the youth's treatment plan. The
Director of Clinical Services reviews trackers to ensure compliance with frequency and duration.
There was documentation of the Director of Clinical Services conducting this review. The
monitoring was completed monthly and the service was provided from 08/01/20-07/31/21.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The Director of Clinical Services provides fidelity monitoring. There was no documentation on any
reviewed monitoring sheets which suggested any corrective action had been taken.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Of the four remaining trained facilitators, only two (one therapist and the Director of Clinical
Services) were eligible for performance evaluations at the time of the annual review. The other
two staff had not been employed long enough to have received an annual performance
evaluation. The annual evaluations for the Therapists and PRN Therapist who were no longer
employed had been completed prior to their departures.
Both facilitators were evaluated on their skill in delivering the service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 25 weeks for this type of service. Of the 18 youth in the
sample, 94% (17 of 18) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates a target of 30 contact hours for this type of service. Of the 18 youth in the
sample, 22% (4 of 18) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 5 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 20%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 72%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 4 youth
Moderate-High = 7 youth
High = 2 youth
Total Youth in Sample = 18 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 11%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 2 youth
Total Youth in Sample = 18 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 72% (13 of 18) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 11% (2 of 18) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 10 10 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 2 10 20%
Risk Level of Youth Served 5 25 20%
Totals 45 75 60%
This SPEP report evaluates Individual Counseling, an intervention delivered at Les Peters Girls
Academy.
18 youth completed the program during the review period. Of those youth, all 18 were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 18 total youth sampled, all
18 included dosage with end dates in the EBS Module. Of those youth, 17 received at least the
recommended weeks of service. Youth in the sample completed between 22 and 47 weeks of
service, with an average of 31 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
The program earned 2 points for Amount of Service: Contact Hours. Of the 18 total youth
sampled, all 18 included dosage in the EBS Module. Of those youth, 4 received at least the
recommended hours of service. Youth in the sample completed between 19 and 48 hours of
service, with an average of 30 hours.
The program was awarded 5 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 87%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Les Peters Girls Academy can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Les Peters Girls Academy can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8

file_name: FY2021-22 LPGA Individual.pdf


state: FL

facility_name: Les Peters Academy

summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Les Peters Girls Academy
TrueCore Behavioral Solutions, LLC
(Contract Provider)
3930 West Martin Luther King Jr. Blvd.
Tampa, Florida 33614
Primary Service: Impact of Crime
SPEP Review Period: 08/01/20-07/31/21
SPEP Review Conducted: 08/03/21
Florida Department of Juvenile Justice
Report Date: 10/14/2021
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There were six staff members formally trained in the Impact of Crime (IOC) curriculum- one
Transition Services Manager, three Case Managers, one Assistant Facility Administrator, and one
Facility Administrator. Four facilitated group during the review period. All six received training
from a qualified trainer who was formally trained in the Impact of Crime curriculum.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the Impact of Crime-Addressing the Harm curriculum. The manual contains a
script for the facilitator for the groups, provides lessons and includes implementation guidelines,
identifies the order the lessons should be delivered in and includes detailed instructions for the
delivery of each. All youth admitted to Les Peters Girls Academy are assessed by staff as to need
for evidence-based services. Based on various criteria, the youth are placed into program
services.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
An Impact of Crime group was observed on Wednesday08/03/21, from 11:50 to 12:50. The
facilitator followed the lesson plan, did not read verbatim, explained the lesson in her own words
and was easy for the youth to understand. The facilitator adhered to the lesson plan in every
aspect.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of Impact of Crime documentation provided revealed gaps in service from 07/29/20-
08/17/20 and 11/20/20-12/04/20. The Director of Clinical Services stated the gaps were due to
enrollment numbers and waiting on enough new youth to restart the group. Of the eighteen (18)
successful completers, sixteen (16) received Impact of Crime
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation for fidelity monitoring of Impact of Crime was reviewed. Fidelity monitoring
was completed for each staff who facilitated the Impact of Crime curriculum on a monthly basis.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the Director of Clinical Services verified the program has a process for
corrective actions based on the fidelity monitoring conducted. The program uses internal fidelity
monitoring/corrective action checklists specific to the Impact of Crime curriculum. A review of the
documentation provided and an interview with the Director of Clinical Services verified there was
no corrective action taken.
g. Evaluation of Facilitator Skill Delivering the Basic Score: N/A Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The performance evaluations of all trained facilitators found three facilitators did not have
evaluations for the year, as they are not due until 10/19/21. TrueCore does not complete 90-day
evaluations. The evaluations will be reviewed during the next quarterly SPEP Review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates the target duration of 16 weeks for this type of service. Of the 18 youth in the
sample, 50% (9 of 18) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 24 contact hours for this type of service. Of the 18 youth in the
sample, 89% (16 of 18) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 5 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 20%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 72%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 4 youth
Moderate-High = 7 youth
High = 2 youth
Total Youth in Sample = 18 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 11%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 2 youth
Total Youth in Sample = 18 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 72% (13 of 18) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 11% (2 of 18) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 4 10 40%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 5 25 20%
Totals 57 85 67%
This SPEP report evaluates Impact of Crime, an intervention delivered at Les Peters Girls
Academy .
18 youth completed the program during the review period. Of those youth, all 18 were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 4 points for Amount of Service: Duration. Of the 18 total youth sampled, only
16 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 9
received at least the recommended weeks of service. Youth in the sample completed between 14
and 18 weeks of service, with an average of 16 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
The program earned 8 points for Amount of Service: Contact Hours. Of the 18 total youth
sampled, only 16 included dosage in the EBS Module. Of those youth with correct dosage, 16
received at least the recommended hours of service. Youth in the sample completed between 33
and 41 hours of service, with an average of 36 hours.
The program was awarded 5 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 91%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Les Peters Girls Academy can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Les Peters Girls Academy can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8

source: DJJ

report_type: SPEP

file_name: FY2021-22 LPGA IOC.pdf

program_name_raw: Les Peters Academy - Females

program_status_note: Closed

service_type: Impact of Crime


summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Les Peters Girls Academy
TrueCore Behavioral Solutions, LLC
(Contract Provider)
3930 West Dr. Martin Luther King Jr. Blvd.
Tampa , Florida 33614
Primary Service: Female Healthy Relationships
SPEP Review Period: 08/01/21-07/31/22
SPEP Review Conducted: 08/09/22
Florida Department of Juvenile Justice
Report Date: 1/6/23
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Female Healthy Relationships.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The Female Healthy Relationships intervention is being reviewed. This bundled service includes
Voices: A Program of Self-Discovery and Empowerment for Girls and The Teen Relationships
Workbook. The program has formal training protocols for both curricula. Facilitators for Teen
Relationships are trained by the Clinical Director or other licensed mental health representative.
The training is a comphrensive overview of the course coupled with opportunities to practice the
skills of engagement as outlined by the curriculum requirements. Facilitators for Voices attend a
three day training conducted by the Regional Compliance Manager who maintains a Train the
Trainer status for Voices. The training is a comphrehensive overview of the course, coupled with
activities and mock group presentations which allow the Therapists to learn and practice the skills
of engagement as outlined by the curriculum requirements.
There were three staff trained in the bundled service- one Assistant Clinical Director and two non-
licensed Therapists. There were also two previous trained non-licensed Therapists who have left
the program. The previous Clinical Director and the current Assistant Clinical Director provided
fidelity monitoring for this service.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program has brand name manuals for the delivery of Female Healthy Relationships. The
Teen Relationships Workbook manual is divided into fifty (50) different topics related to healthy
relationships and addressing abuse. The manual contains worksheets for students and
intervention suggestions for facilitators. The Voices manual is divided into eighteen (18) lessons
with specific interventions, activities and an associated workbook. The lessons are further divided
into the modules of Self, Others, Healthy Living, and Journey Ahead. The lessons are outlined
with detail with some flexibility for idiosyncratic group dynamics. There is a separate facilitator
manual and student workbook. Teen Relationships is an open ongoing group; Voices is a closed
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
group. Both groups are held on Tuesdays and Thursdays. The program maintains their records
for this servive in a SPEP binder tabbed by intervention and seperated into the different SPEP
indicator sections. All youth at Les Peters Academy receive Female Healthy Relationships.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Female Healthy Relationships, specifically the Teen Relationships Workbook
component, was observed on Thursday, 08/11/22 from 2:30-3:30p. The facilitator provided a copy
of the lesson plan from the manual prior to the observation. The lesson for the day was "Gender
Roles-Where Do I Stand". The facilitator followed the lesson according to the manual, did not read
the lesson verbatim, used her own words to convey the topic, and did so in an easy to follow
manner. The facilitator adhered to the lesson plan in every aspect.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The reviewed documentation and an interview with the Clinical Director confirmed two non-
licensed Therapists left during the review period. However, this turnover did not adversely affect
the delivery of Female Healthy Relationships. The program's policy regarding turnover to ensure
any potentially affected groups are co-facilitated, if possible, at the outset and that all staff are
trained in these curricula. Teen Relationships is an open ongoing group, which the youth receive
for the duration of their stay. As a closed group, the Voices component had four completed
cohorts and one ongoing cohort during the review period. There were no gaps in service during
the review period.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Twelve months of internal fidelity monitoring reports were reviewed for Female Healthy
Relationships. The program has an internal process to conduct fidelity monitoring specific to this
service. The Assistant Clinical Director or her trained designee perform all fidelity monitoring.
Fidelity monitoring was completed monthly for each facilitator of Female Healthy Relationships,
without exception.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the Assistant Clinical Director verified the program has a process for corrective
actions based on fidelity monitoring conducted. The program uses internal fidelity monitoring /
corrective action checklists specific to Female Healthy Relationships. Each fidelity monitoring
checklist includes a section for feedback specifying either the facilitator did not require any
corrective action or included recommendations for fuure improvement. The following month's
checklist would include comments the previous month's recommendations were applied and
demonstrated. There were no corrective actions noted nor taken during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of the three facilitators of Female Healthy Relationships found evaluations were
completed within the last twelve (12) months for each. A review of the performance evaluations
found an entry for the evaluation of the staff's performance in delivering the service. These
included comments on any strengths or weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 24 weeks for this type of service. Of the 23 youth in the
sample, 87% (20 of 23) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 40 contact hours for this type of service. Of the 23 youth in the
sample, 96% (22 of 23) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 8 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 32%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 65%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 5 youth
Moderate-High = 6 youth
High = 4 youth
Total Youth in Sample = 23 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 17%
High Score: 3 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 23%
Table 2
High = 4 youth
Total Youth in Sample = 23 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 65% (15 of 23) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 17% (4 of 23) youth scored High-Risk to reoffend, for a score of 3 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 8 25 32%
Totals 74 95 78%
This SPEP report evaluates Female Healthy Relationships, an intervention delivered at Les
Peters Girls Academy.
24 youth completed the program during the review period. Of those youth, 23 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 23 total youth sampled, all
23 included dosage with end dates in the EBS Module. Of those youth, 20 received at least the
recommended weeks of service. Youth in the sample completed between 14 and 61 weeks of
service, with an average of 34 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 8 points for Amount of Service: Contact Hours. Of the 23 total youth
sampled, all 23 included dosage in the EBS Module. Of those youth, 22 received at least the
recommended hours of service. Youth in the sample completed between 10 and 89 hours of
service, with an average of 65 hours.
The program was awarded 8 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 96%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Les Peters Girls Academy can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Les Peters Girls Academy can optimize their SPEP Amount of Service score by ensuring dosage
for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted dosage
of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2022-23 LPGA FHR.pdf

service_type: Female Healthy Relationships


summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Les Peters Girls Academy
TrueCore Behavioral Solutions, LLC
(Contract Provider)
3930 West Dr. Martin Luther King Jr. Blvd.
Tampa , Florida 33614
Primary Service: Individual Counseling
SPEP Review Period: 08/01/21-07/31/22
SPEP Review Conducted: 08/09/22
Florida Department of Juvenile Justice
Report Date: 1/6/23
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 10 Points
1. Primary Service and Supplemental Service Types POS: 10 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Individual Counseling.
The program was awarded 5 points because the Primary Service is identified as a Group 1
Service. The specific Sub-Component Service Type identified is Individual Counseling. The
Primary Service was identified as this type of service as it includes any range of treatment
techniques that focus on psychological or interpersonal problems or issues faced by an individual
that involves a one-on-one relationship with a therapist or counselor.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes Individual Counseling as a primary service. Documentation confirmed the
use of a training protocol provided by Master Level Therapists trained in a relevant field. Non-
licensed Therapists provide therapy under the supervision of a licensed Therapist. There are
three current facilitators for Individual Counseling- one Assistant Clinical Director and two non-
licensed Therapists. One Clinical Director and two non-licensed Therapists also faciliated
counseling during the review period; however, these staff are no longer employed by the program.
The previous Clinical Director and the current Assistant Clinical Director provided fidelity
monitoring for the non-licensed staff.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
There is no formal curriculum/manual for Individual Counseling. The program has a written
protocol for this intervention. The dosage is stated on the youth's individualized treatment plan.
This plan is then reviewed and signed by a licensed clinician. The main theoretical approach is
Cognitive Behavioral Therapy, an evidence-based framework within a trauma-informed model.
Therapists also utilize other interventions as appropriate to client needs, including Motivational
Interviewing, Dialectical-Behavior Therapy and Solution-Based Therapy. Individual Therapy notes
are submitted to the EBS system within 48-hours of the service. They are printed out weekly as
part of the Weekly Progress Note and put in the file. Individual Counseling is considered open, as
no cohorts are ever actually completed.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Individual Counseling is unable to be observed due to the confidential nature of the counseling.
Therefore this indicator is rated as non-applicable.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The reviewed documentation and an informal interview with the Assistant Clinical Director
confirmed there was some turnover during the review period. The program's policy regarding
turnover is for the other trained Therapists to take over and provide counseling under the
supervision of the Master Level Therapist. The program lost their Clinical Director and two
unlicensed Therapists. Individual Counseling is an open and ongoing service provided for each
youth, for one hour each week, starting from the week they arrive until the week they are
discharged from the program. There were no gaps in services during the review period.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Twelve (12) months of internal fidelity monitoring reports were provided for Individual Counseling.
The program has an internal process to conduct monitoring specific to Individual Counseling. The
Assistant Clinical Director reviews progress notes for each counselor weekly to ensure each youth
is receiving services for one hour each week. Individual therapy is provided by Master Level
Therapists trained in a relevant field. Non-licensed Therapists provided therapy under the
supervision of a licensed therapist.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the Assistant Clinical Director and review of the documentation provided verified
the program has a process for corrective actions based on fidelity monitoring conducted. The
program uses internal fidelity monitoring/corrective action checklists specific to Individual
Counseling. Each fidelity monitoring checklist includes a section for feedback and
recommendations for facilitator improvement. Les Peter's Academy follows Truecore's
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
progressive correction action process; coaching, oral, written, and critical. Each completed
checklist included feedback specifying the facilitator did not require any corrective action or
included recommendations for future improvement.
There were no corrective actions noted nor taken by the program during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of the three facilitator's employee performance evaluations found an entry for evaluation
of the staff's performance in providing Individual Counseling. These included comments on any
strengths or weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 25 weeks for this type of service. Of the 23 youth in the
sample, 78% (18 of 23) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates a target of 30 contact hours for this type of service. Of the 23 youth in the
sample, 48% (11 of 23) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 8 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 32%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 65%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 5 youth
Moderate-High = 6 youth
High = 4 youth
Total Youth in Sample = 23 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 17%
High Score: 3 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 23%
Table 2
High = 4 youth
Total Youth in Sample = 23 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 65% (15 of 23) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 17% (4 of 23) youth scored High-Risk to reoffend, for a score of 3 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 10 10 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 4 10 40%
Risk Level of Youth Served 8 25 32%
Totals 48 75 64%
This SPEP report evaluates Individual Counseling, an intervention delivered at Les Peters Girls
Academy.
24 youth completed the program during the review period. Of those youth, 23 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 6 points for Amount of Service: Duration. Of the 23 total youth sampled, all
23 included dosage with end dates in the EBS Module. Of those youth, 18 received at least the
recommended weeks of service. Youth in the sample completed between 14 and 53 weeks of
service, with an average of 29 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 4 points for Amount of Service: Contact Hours. Of the 23 total youth
sampled, all 23 included dosage in the EBS Module. Of those youth, 11 received at least the
recommended hours of service. Youth in the sample completed between 14 and 52 hours of
service, with an average of 30 hours.
The program was awarded 8 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 87%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Les Peters Girls Academy can optimize their SPEP Quality of Service Delivery score by ensuring
facilitator turnover does not impact groups.
Les Peters Girls Academy can optimize their SPEP Amount of Service score by ensuring dosage
for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted dosage
of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2022-23 LPGA Individual.pdf

service_type: Individual Counseling


summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Les Peters Girls Academy
TrueCore Behavioral Solutions, LLC
(Contract Provider)
3930 West Dr. Martin Luther King Jr. Blvd.
Tampa , Florida 33614
Primary Service: Impact of Crime
SPEP Review Period: 08/01/21-07/31/22
SPEP Review Conducted: 08/09/22
Florida Department of Juvenile Justice
Report Date: 1/6/23
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 6 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
An annual SPEP Review is being conducted at Les Peters Academy for the Impact of Crime
intervention for the dates of 08/01/21-07/31/22. The program utilizes a formal training program
provided by the Florida Department of Juvenile Justice. During the review period, there were three
current staff formally trained in the Impact of Crime curriculum- one Director of Case
Management, one Case Manager, and one Transporter (Youth Care Worker). Two other staff
have left the program who facilitated the IOC group during the review period- one Lead Case
Manager and one Case Manager. One Program Director and one Case Manager, who are no
longer at the program, provided fidelity monitoring, along with the current Director of Case
Management.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the Impact of Crime; Addressing the Harm curriculum. The manual provides
a script, lessons, instructions for the lessons, implementation guidelines, identifies the order the
lessons are to be delivered, and includes detailed instructions for the delivery of each lesson.
Impact of Crime is held twice a week on Mondays and Wednesdays from 11:30a to 12:30p. The
program maintains their records for the Impact of Crime intervention in a SPEP binder which is
tabbed and separated into the required categories. All youth at Les Peters Academy receive IOC
as a part of the program services requirement for restorative justice.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Impact of Crime was observed virtually via Microsoft Teams on Monday, 08/15/22,
from 11:45 to 12:45 pm. There was one staff and six youth present for the group session. A copy
of the lesson, Chapter 6: "Resolving Conflict" was provided to the reviewer prior to the session.
The facilitator was knowledgeable of the material. She ensured all youth grasped the concepts
before moving on and she did not read the script verbatim. She used her own words to convey the
discussion topic. The facilitator adhered to the lesson plan in every aspect.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program experienced some facilitator turnover during the review period. Two Case Managers
left the program, one during an Impact of Crime cohort. The program handled the turnover by
having additional trained facilitators resume the group sessions in order to prevent a lapse in
services to the youth. One group had a co-facilitator that took over the group. There were no gaps
in services over one month.
The program completed four cohorts of Impact of Crime during the review period and there was
one ongoing cohort at the time of the annual review. There were between two and eight youth in
each cohort.
The cohorts took place 04/07/21 to 08/12/21, 08/30/21 to 12/15/21, 12/03/21 to 04/12/22, and
03/31/22 to 07/22/22. There is one cohort currently ongoing which began on 05/25/22.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 0 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Twelve months of internal fidelity monitoring forms were reviewed for facilitators of Impact of
Crime. The program has a process to conduct internal fidelity monitoring using checklists specific
to Impact of Crime. The Director of Case Management or her trained designee performs all fidelity
monitoring. The Director of Case Management is formally trained in the curriculum.
Fidelity monitoring was not completed for the months of 02/22 and 03/22 for any of the facilitators.
A monitoring sheet was provided for 06/29/22; however, during a review of video footage by
Monitoring and Quality Improvement, it was determined this session did not occur. A subsequent
review of The Lead Case Manager and one Case Manager have since left the program.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 0 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the Director of Case Management and review of the documentation provided
verified the program has a process for corrective actions based on fidelity monitoring conducted
on facilitators of Impact of Crime (IOC). The program uses internal fidelity monitoring/corrective
action checklists specific to IOC.
The Director of Case Management and all Impact of Crime facilitators were formally trained in the
curriculum. Fidelity monitoring was not completed for the months of 02/22 and 03/22 for any of the
facilitators. A monitoring sheet was provided for 06/29/22 with Natasha Stewart as the observer
for the group and Alexis Merrick was the facilitator. This date was involved in the investigation by
MQI which revealed 06/29/22 was one of the dates the group did not occur. The Lead Case
Manager and one Case Manager have since left the program.. Each facilitator was monitored
monthly except for the months of 02/22 and 03/22.Subsequently this staff member who was
facilitating the group and responsible for the documentation and sign in sheets was terminated as
a result of these falsifications.
The program can earn 1 point if any corrective action needed is identified, applied, and
demonstrated.
At the time of the review, the program did not meet criteria to earn a score of 1.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of two of the three current facilitator's employee performance evaluations found an entry
for evaluation of the staff's performance in delivering Impact of Crime. These included comments
on any strengths or weaknesses of which the facilitator should be aware. The third facilitator has
not been with the program long enough for an annual review to be completed.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 16 weeks for this type of service. Of the 22 youth in the
sample, 64% (14 of 22) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 24 contact hours for this type of service. Of the 22 youth in the
sample, 95% (21 of 22) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 8 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 32%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 64%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 4 youth
Moderate-High = 6 youth
High = 4 youth
Total Youth in Sample = 22 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 18%
High Score: 3 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 23%
Table 2
High = 4 youth
Total Youth in Sample = 22 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 64% (14 of 22) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 18% (4 of 22) youth scored High-Risk to reoffend, for a score of 3 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 8 25 32%
Totals 52 85 61%
This SPEP report evaluates Impact of Crime, an intervention delivered at Les Peters Girls
Academy.
24 youth completed the program during the review period. Of those youth, 22 were included in the
sample.
The program scored Medium for Quality of Service Delivery.
The program earned 6 points for Amount of Service: Duration. Of the 22 total youth sampled, only
21 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 14
received at least the recommended weeks of service. Youth in the sample completed between 12
and 17 weeks of service, with an average of 16 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 8 points for Amount of Service: Contact Hours. Of the 22 total youth
sampled, only 21 included dosage in the EBS Module. Of those youth with correct dosage, 21
received at least the recommended hours of service. Youth in the sample completed between 32
and 36 hours of service, with an average of 33 hours.
The program was awarded 8 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 81%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Les Peters Girls Academy can optimize their SPEP Quality of Service Delivery score by ensuring
facilitator turnover does not impact groups, fidelity monitoring is completed every month for each
facilitator, and any corrective action needed is identified, applied, and demonstrated.
Les Peters Girls Academy can optimize their SPEP Amount of Service score by ensuring dosage
for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted dosage
of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2022-23 LPGA IOC.pdf