state: FL

facility_name: Les Peters Academy

summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Les Peters Academy
TrueCore Behavioral Solutions, LLC
(Contract Provider)
3930 W Dr. Martin Luther King Jr. Blvd
Tampa, Florida 33614
Primary Service: Female Healthy Relationships
SPEP Review Period: 08/01/2022 - 07/31/2023
SPEP Review Conducted: 08/01/2023
Florida Department of Juvenile Justice
Report Date: 01/24/2024
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Female Healthy Relationships.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 6 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilized Female Healthy Relationships as a primary service for the review period of
August 1, 2022, to July 31,2023. This bundled service includes Voices: A Program of Self-
Discovery and Empowerment for Girls and The Teen Relationships Workbook. The program has
formal training protocols for both curricula.
Facilitators for Teen Relationships are trained by the Clinical Director or other licensed mental
health professionals. The training is a comprehensive overview of the course coupled with
opportunities to practice the skills of engagement as outlined by the curriculum requirements.
Facilitators for Voices attend a three-day training conducted by the regional compliance manager
who maintains a Train the Trainer status for Voices. The training is a comprehensive overview of
the course, coupled with activities and mock group presentations which allows the facilitators to
learn and practice the skills of engagement as outlined by the curriculum requirements.
There were nine staff trained in the bundled service: three clinical directors, two assistant clinical
directors, and four therapists. Two clinical directors, two assistant clinical directors and one
therapist left the program during this review period. The previous clinical director, previous
assistant facility administrator, and the current assistant clinical director provided fidelity
monitoring for this service.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program has brand name manuals for the delivery of Female Healthy Relationships. The
Teen Relationships Workbook manual is divided into fifty (50) different topics related to healthy
relationships and addressing abuse. The manual contains worksheets for students and
intervention suggestions for facilitators. The Voices manual is divided into eighteen (18) lessons
with specific interventions, activities, and an associated workbook. The lessons are further divided
into the modules of Self, Others, Healthy Living, and Journey Ahead. The lessons are outlined
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
with some flexibility for specific group needs. There is a separate facilitator manual and student
workbook.
Teen Relationships is an open ongoing group; Voices is a closed group. Groups are scheduled on
Tuesdays and Thursdays. The program maintains their records for this service in a SPEP binder
tabbed by intervention and separated into the different SPEP indicator sections. All youth at Les
Peters Academy receive Female Healthy Relationships.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Female Healthy Relationships was observed on Thursday, 08/11/22 from 2:30pm to
3:30pm. The facilitator provided a copy of the lesson plan from the manual prior to the
observation. The lesson for the day was "The World Girls Live In". The facilitator followed the
lesson according to the manual, did not read the lesson verbatim, used her own words to convey
the topic, and did so in an easy-to-follow manner. In attendance were two staff members, ten (10)
youth, and the facilitator. The youth were receptive of the lesson and were respectful of each
other’s opinions and feelings. The facilitator adhered to the lesson plan in every aspect.
Basic Score: 0 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation and an interview with the facility administrator confirmed two clinical
directors, two assistant clinical directors, and a therapist left during the review period. This
turnover adversely affected the delivery of Female Healthy Relationships. This service was not
provided during the months of November 2022 and December 2022 as there were no trained staff
available to deliver the Female Healthy Relationship bundle. The program's policy regarding
turnover is to ensure any potentially affected groups are co-facilitated, if possible, and all staff are
trained in these curricula. Teen Relationships is an open ongoing group, which the youth receive
for the duration of their stay. Voices is a closed group.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Internal fidelity monitoring reports were reviewed for Female Healthy Relationships. The program
has an internal process to conduct fidelity monitoring specific to this service. The previous clinical
director, previous assistant facility administrator and the current assistant clinical director or her
trained designee perform all fidelity monitoring. An interview with the facility administrator
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
confirmed there were missing internal fidelity monitoring forms dating prior to her arrival in
February 2023. The available documentation shows fidelity monitoring was conducted on the
following dates: 3/7/23, 3/9/23, 4/27/23, and 5/30/23.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the facility administrator verified the program has a process for corrective
actions based on the internal fidelity monitoring. The program uses internal fidelity monitoring
and/or corrective action checklists specific to Female Healthy Relationships. Each fidelity
monitoring checklist includes a section for feedback specifying whether the facilitator required any
corrective action and/or suggested recommendations for future improvement. The following
month's checklist would include whether the previous month's recommendations were applied and
demonstrated. There were no corrective actions noted nor taken during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: N/A Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of documentation and an interview with the facility administrator confirmed there were no
annual evaluations completed within the last twelve (12) months due to all the clinical employees
only being with the program for under a year. This indicator is rated as non-applicable.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 24 weeks for this type of service. Of the 21 youth in the
sample, 86% (18 of 21) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 40 contact hours for this type of service. Of the 21 youth in the
sample, 67% (14 of 21) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 5 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 20%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 62%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 4 youth
Moderate-High = 8 youth
High = 1 youth
Total Youth in Sample = 21 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 5%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 1 youth
Total Youth in Sample = 21 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 62% (13 of 21) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 5% (1 of 21) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 5 25 20%
Totals 59 95 62%
This SPEP report evaluates Female Healthy Relationships, an intervention delivered at Les
Peters Academy.
26 youth completed the program during the review period. Of those youth, 21 were included in the
sample.
The program scored Medium for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 21 total youth sampled, all
21 included dosage with end dates in the EBS Module. Of those youth, 18 received at least the
recommended weeks of service. Youth in the sample completed between 11 and 49 weeks of
service, with an average of 36 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 6 points for Amount of Service: Contact Hours. Of the 21 total youth
sampled, all 21 included dosage in the EBS Module. Of those youth, 14 received at least the
recommended hours of service. Youth in the sample completed between 4 and 76 hours of
service, with an average of 49 hours.
The program was awarded 5 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 83%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Les Peters Academy can optimize their SPEP Quality of Service Delivery score by ensuring
facilitator turnover does not impact groups and fidelity monitoring is completed every month for
each facilitator.
Les Peters Academy can optimize their SPEP Amount of Service score by ensuring dosage for all
youth is recorded accurately in EBS and by ensuring youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

source: DJJ

report_type: SPEP

file_name: FY2023-24 LPGA FHR.pdf

program_name_raw: Les Peters Academy - Females

program_status_note: Closed

service_type: Female Healthy Relationships


summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Les Peters Academy
TrueCore Behavioral Solutions, LLC
(Contract Provider)
3930 W Dr. Martin Luther King Jr. Blvd
Tampa, Florida 33614
Primary Service: Individual Counseling
SPEP Review Period: 08/01/2022 - 07/31/2023
SPEP Review Conducted: 08/01/2023
Florida Department of Juvenile Justice
Report Date: 01/24/2024
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 10 Points
1. Primary Service and Supplemental Service Types POS: 10 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Individual Counseling.
The program was awarded 5 points because the Primary Service is identified as a Group 1
Service. The specific Sub-Component Service Type identified is Individual Counseling. The
Primary Service was identified as this type of service as it includes any range of treatment
techniques that focus on psychological or interpersonal problems or issues faced by an individual
that involves a one-on-one relationship with a therapist or counselor.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilized Individual Counseling as a primary service for the review period of August 1,
2022, to July 31, 2023. During the review period, there were nine clinicians trained to provide
Individual Counseling to the youth — four therapists, three assistant clinical directors and two
clinical directors. All nine provided services during this annual review period. The program
maintains all information regarding counseling in the youth's mental health/substance abuse
record. There was documentation to support each therapist had the education and experience to
provide counseling.
As this service does not have a formal curriculum, the facilitators receive training from a licensed
clinician and are provided ongoing supervision by the clinical director. The facilitator receives
additional training as needed, which is documented during their clinical supervision. One
therapist, two clinical directors and an assistant clinical director are no longer employed at Les
Peters Academy. After those staff left, the remaining therapists and the assistant clinical director
continued to facilitate individual counseling for the youth.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
There is no written curriculum/manual for Individual Counseling. The program has written protocol
for this intervention. The dosage/frequency is stated on each youth's individualized treatment
plan. This plan is then reviewed and signed by a licensed clinician. All youth at Les Peters Girls
Academy receive Individual Counseling.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
This service was not observed during the review due to the nature of Individual Counseling,
therefore, this indicator rates as non-applicable.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of the documentation and an interview with the facility administrator confirmed there was
facilitator turnover during the review period. In the event of turnover, other trained therapists
would facilitate individual counseling. There were no gaps in service.
The program can earn 2 points if facilitator turnover does not impact service delivery.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has documented fidelity monitoring for observations of individual counseling. The
fidelity monitoring was completed by the program's clinical director and the assistant clinical
director. The monitoring is completed through a review of therapy notes to verify effective and
appropriate interventions which are in line with the youth's treatment plan. The clinical director
reviews trackers to ensure compliance with frequency and duration. There was documentation of
the director of clinical services conducting this review. The monitoring was completed monthly
without exception.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the facility administrator verified the program has a process for corrective
actions based on fidelity monitoring conducted by the clinical director. There was no
documentation on any reviewed monitoring sheets which suggested any corrective action had
been taken.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
g. Evaluation of Facilitator Skill Delivering the Basic Score: N/A Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of documentation and an interview with the facility administrator confirmed there were no
annual evaluations completed within the last twelve (12) months due to all the clinical employees
only being with the program for under a year. This indicator is rated as non-applicable.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 25 weeks for this type of service. Of the 21 youth in the
sample, 81% (17 of 21) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 30 contact hours for this type of service. Of the 21 youth in the
sample, 14% (3 of 21) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 5 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 20%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 71%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 4 youth
Moderate-High = 9 youth
High = 2 youth
Total Youth in Sample = 21 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 10%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 2 youth
Total Youth in Sample = 21 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 71% (15 of 21) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 10% (2 of 21) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 10 10 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 5 25 20%
Totals 33 75 44%
This SPEP report evaluates Individual Counseling, an intervention delivered at Les Peters
Academy.
26 youth completed the program during the review period. Of those youth, 21 were included in the
sample.
The program scored Medium for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 21 total youth sampled, all
21 included dosage with end dates in the EBS Module. Of those youth, 17 received at least the
recommended weeks of service. Youth in the sample completed between 1 and 45 weeks of
service, with an average of 27 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 0 points for Amount of Service: Contact Hours. Of the 21 total youth
sampled, all 21 included dosage in the EBS Module. Of those youth, 3 received at least the
recommended hours of service. Youth in the sample completed between 7 and 38 hours of
service, with an average of 20 hours.
The program was awarded 5 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 71%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Les Peters Academy can optimize their SPEP Quality of Service Delivery score by ensuring
facilitator turnover does not impact service delivery.
Les Peters Academy can optimize their SPEP Amount of Service score by ensuring dosage for all
youth is recorded accurately in EBS and by ensuring youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2023-24 LPGA Individual.pdf

service_type: Individual Counseling


summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Les Peters Academy
TrueCore Behavioral Solutions, LLC
(Contract Provider)
3930 W Dr. Martin Luther King Jr. Blvd
Tampa, Florida 33614
Primary Service: Impact of Crime
SPEP Review Period: 08/01/2022 - 07/31/2023
SPEP Review Conducted: 08/01/2023
Florida Department of Juvenile Justice
Report Date: 01/24/2024
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilized Impact of Crime (IOC) as a primary service for the review period of August 1,
2022, to July 31, 2023. There were six staff members formally trained in the curriculum: one case
manager, one director of case management, the assistant facility administrator, and the facility
administrator. Each received training from a qualified trainer. Four facilitators conducted group
during the review period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the Impact of Crime: Addressing the Harm curriculum. The manual contains a
script for the facilitator, identifies the order the lessons should be delivered, and includes detailed
instructions for the delivery of each lesson. All youth admitted to Les Peters Girls Academy are to
complete Impact of Crime during their stay.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Impact of Crime on Tuesday, 08/2/23 from 11:35am to 12:32pm. The facilitator
provided a copy of the lesson plan from the manual prior to the observation. The lesson for the
day was "Chapter 3: Introduction to Harm". The facilitators followed the lesson according to the
manual, did not read the lesson verbatim, used their own words to convey the topic, and did so in
an easy-to-follow manner. In attendance were the facilitator, co-facilitator, and eight youth. The
group participated in an activity involving a crossword puzzle intended to reinforce the information
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
provided during the previous lesson. The facilitator adhered to the lesson plan and ensured the
youth understood the lesson prior to moving forward.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation and an interview with the Facility Administrator confirmed there was
facilitator turnover during the review period. The information provided indicated that this turnover
did not adversely affect the delivery of Impact of Crime (IOC). The program's policy regarding
turnover is to ensure any potentially affected groups are co-facilitated, if possible, at the outset
and that all staff are trained in the IOC curriculum.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Twelve (12) months of internal fidelity monitoring reports were reviewed for Impact of Crime
(IOC). The program utilizes an internal process to conduct fidelity monitoring specific to the
curriculum. The director of case management or other designated trained staff perform the fidelity
monitoring. The documents reviewed show there were IOC groups held throughout the entire
review period; however, an interview with the facility administrator revealed that there were
missing or misplaced internal fidelity monitoring forms from prior to her arrival at the end of
February 2023. The available documentation showed internal fidelity monitoring was conducted
March through July 2023.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the facility administrator verified the program has a process for corrective
actions based on fidelity monitoring conducted. The program uses internal fidelity monitoring in
determining corrective action based on a checklist specific to the Impact of Crime curriculum.
Each fidelity monitoring checklist includes a section for feedback specifying if the facilitator
required any corrective action and/or included recommendations for future improvement.
Subsequent fidelity monitoring checklists would then include comments to indicate whether the
previous month's recommendations were applied and demonstrated. There were no corrective
actions noted nor taken during the review period.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of the trained staff facilitating Impact of Crime groups confirmed ninety (90)-day
evaluations were completed within the last twelve (12) months. These evaluations included
comments on any strengths or weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 16 weeks for this type of service. Of the 21 youth in the
sample, 81% (17 of 21) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 24 contact hours for this type of service. Of the 21 youth in the
sample, 90% (19 of 21) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 5 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 20%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 71%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 4 youth
Moderate-High = 9 youth
High = 2 youth
Total Youth in Sample = 21 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 10%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 2 youth
Total Youth in Sample = 21 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 71% (15 of 21) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 10% (2 of 21) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 5 25 20%
Totals 61 85 72%
This SPEP report evaluates Impact of Crime, an intervention delivered at Les Peters Academy.
26 youth completed the program during the review period. Of those youth, 21 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 21 total youth sampled, all
21 included dosage with end dates in the EBS Module. Of those youth, 17 received at least the
recommended weeks of service. Youth in the sample completed between 9 and 23 weeks of
service, with an average of 17 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 21 total youth
sampled, all 21 included dosage in the EBS Module. Of those youth, 19 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
recommended hours of service. Youth in the sample completed between 17 and 63 hours of
service, with an average of 34 hours.
The program was awarded 5 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 95%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Les Peters Academy can optimize their SPEP Quality of Service Delivery score by ensuring
fidelity monitoring is completed every month for each facilitator.
Les Peters Academy can optimize their SPEP Amount of Service score by ensuring dosage for all
youth is recorded accurately in EBS and by ensuring youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2023-24 LPGA IOC.pdf

service_type: Impact of Crime