summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Cypress Creek Juvenile Offender Correctional Center / Cypress
Creek Treatment Center
TrueCore Behavioral Solutions, LLC
(Contract Provider)
2855 West Woodland Ridge Drive
Lecanto, Florida 34461
Primary Service: Impact of Crime
SPEP Review Period: 4/15/2022-4/14/2023
SPEP Review Conducted: 5/9/2023
Florida Department of Juvenile Justice
Report Date: 9/27/2023
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The Standardized Program Evaluation Protocol (SPEP) review encompasses twelve (12) months
of records. Impact of Crime is based on the belief that crime is more than just a legal definition,
but rather crime affects the victim, the offender, their families, and the community. The harm
caused brings with it the moral responsibility to all involved. The focus of Impact of Crime is to
help offenders understand the harm they have created, and then assisting them in taking personal
accountability for their actions. Together, the victim, offender, and the community arrive at a viable
solution to the repair the harm caused and the offender. Once the harm is repaired, the offender
subsequently reintegrates back into the community. Facilitators are encouraged to bring in victim
impact speakers throughout the course of the curriculum.
The program showed evidence of eight staff trained in the Impact of Crime curriculum- five Case
Managers, one Intake Specialist, one Transitional Service Manager and one Director of Case
Management. The training is a twenty-eight (28)-hour in-person training, conducted over three
days by a qualified trainer. The director of case management or transition service manager
conduct the monthly internal fidelity checks.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the curriculum Impact of Crime: Addressing the Harm to Victims and the
Community, originally published in 2009. It is highly detailed, providing a facilitator's manual and
student workbook. The facilitator's manual provides lesson plans, classroom management ideas,
and tips for engaging youth discussions. The student manual provides reproducible worksheets to
help facilitate group discussions and review concepts learned during the group sessions.
Homework materials are also utilized in the student manual to re-enforce learned concepts. The
primary service is facilitated as a closed group, meaning once a cohort commences, no new youth
may be added.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Impact of Crime was observed on January 17, 2023 from 2:20pm until 3:16pm. In
attendance were one facilitator, one Director of Case Management (DCM), and six youth. The
DCM was conducting fidelity monitoring. A copy of the lesson was provided, Chapter Two:
Owning One’s Action. The facilitator was knowledgeable of the material and ensured all youth
grasped the concepts prior to moving on to the next topic. The youth were engaged and eager to
participate in the activities.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation for the annual review period of April 15, 2022 through April 14, 2023
shows there was facilitator turnover; however, the turnover did not negatively affect the delivery of
Impact of Crime. The program has a process in place to address facilitator turnover and prevent
gaps in service.
There were two cohorts held from 4/5/22-7/28/22. One cohort was conducted in the Alpha dorm
which started with nine youth and ended seven youth. The other cohort was conducted in the
Omega dorm which started with ten (10) youth ended with ten (10) youth.
There were three cohorts run 8/16/22-12/13/22. Alpha group started with two facilitators and eight
youth and ended with nine youth. One youth transferred from another program and was added to
this cohort because he was on the exact chapter as this group. Echo group started with one
facilitator and ten (10) youth and ended with ten (10) youth. Omega group started with one
facilitator and seven youth and ended with six youth. One youth was arrested on 11/17/22.
There were three cohorts run 1/3/23-4/27/23. Bravo group started with seven youth and ended
with six youth. One youth was dropped on 1/17/2023 for lack of participation. Echo group started
with six youth and ended with five youth. One youth successfully completed the program on
4/20/23. Omega group started with seven youth and ended with five youth. Two youth where
moved to a echo group on 3/30/2023 and later dropped for lack of participation.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has a process in place to conduct monthly internal fidelity monitoring of Impact of
Crime facilitators. A review of documentation for the annual period of April 15, 2022 through April
14, 2023 showed the program completed all required fidelity monitoring, without exception. Thirty-
five (35) of thirty-five (35) required internal fidelity monitoring forms were provided. Fidelity
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
monitoring was conducted by the Transitional Service Manager, Director of Case Management,
and Case Manager.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process in place to address corrective actions identified during monthly
internal fidelity monitoring. A review of documentation for the review period showed there were no
corrective actions noted nor taken on the fidelity monitoring of Impact of Crime facilitators.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The annual performance evaluation the program utilizes includes a section to document what
curricula the employees are trained in and evaluated the facilitation. There were three main
facilitators for Impact of Crime groups during the annual review period.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 16 weeks for this type of service. Of the 49 youth in the
sample, 100% (49 of 49) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 49 youth in the
sample, 100% (49 of 49) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 18 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 72%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 86%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 10 youth
Moderate-High = 19 youth
High = 13 youth
Total Youth in Sample = 49 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 27%
High Score: 8 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 62%
Table 2
High = 13 youth
Total Youth in Sample = 49 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 86% (42 of 49) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 27% (13 of 49) youth scored High-Risk to reoffend, for a score of 8 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 18 25 72%
Totals 78 85 92%
This SPEP report evaluates Impact of Crime, an intervention delivered at Cypress Creek Juvenile
Offender Correctional Center / Cypress Creek Treatment Center.
65 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 49 total youth sampled, all
49 included dosage with end dates in the EBS Module. Of those youth, 49 received at least the
recommended weeks of service. Youth in the sample completed between 17 and 19 weeks of
service, with an average of 18 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 49 total youth
sampled, all 49 included dosage in the EBS Module. Of those youth, 49 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
recommended hours of service. Youth in the sample completed between 24 and 28 hours of
service, with an average of 24 hours.
The program was awarded 18 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 100%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Impact of Crime is an intervention suitable for all youth. The Department recommends providing
this service to every youth in the program.
Cypress Creek Juvenile Offender Correctional Center / Cypress Creek Treatment Center can
maintain their SPEP Quality of Service Delivery score by continuing the practices in place at the
time of this review.
Cypress Creek Juvenile Offender Correctional Center / Cypress Creek Treatment Center can
optimize their SPEP Amount of Service score by ensuring dosage for all youth is recorded
accurately in EBS and by ensuring youth receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2022-23 Cypress Creek IOC.pdf