summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Cypress Creek Juvenile Offender Correctional Center / Cypress
Creek Treatment Center
TrueCore Behavioral Solutions, LLC
(Contract Provider)
2855 West Woodland Ridge Drive
Lecanto, Florida 34461
Primary Service: Impact of Crime
SPEP Review Period: 4/15/2020-4/14/2021
SPEP Review Conducted: 5/19/2021-5/20/2021
Florida Department of Juvenile Justice
Report Date: 7/26/21
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A review of the documentation for the annual review period of April 15, 2020 through April 14,
2021 shows the program utilizes Impact of Crime (IOC) as their primary service. There are ten
staff who are trained to facilitate the primary service. The trained staff are the director of case
management, six case managers, two transition services managers, and one intake specialist.
There were seven staff who facilitated a cohort of the primary service during the annual review
period. The training is a twenty-eight hour in-person training, conducted over three days by a
qualified trainer. The director of case management assigns a trained staff member to conduct the
monthly internal fidelity checks. All trained staff members can conduct the internal fidelity checks.
According to the facility's Standardized Program Evaluation Protocol procedure, the facilitator for
each primary service will have completed a formalized training for that primary service from a
qualified trainer. To enhance the continuity of the service due to unexpected facilitator absence
and provide more individualized instruction as needed during the service, when possible, Cypress
Creek will make all efforts to have a co-facilitator for the primary service. The co-facilitator will
have completed a formalized training for the primary service from a qualified trainer.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The documentation was reviewed for the annual review period of April 15, 2020 through April 14,
2021. The program utilizes the Impact of Crime: Addressing the Harm manual. The manual opens
with an introduction to the Impact of Crime Curriculum. Then the manual goes over all seven
chapters of the group and provides a script for the facilitator to follow. The manual also has an
appendix and a student guide with all the student exercises. The program does not have separate
student workbooks to give out. The facilitators make copies of the exercises before each session
to pass out to the youth. The group is a closed group with no more than twelve youth in each
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
cohort. The curriculum is to be delivered to the youth over a minimum of sixteen weeks for thirty-
two sessions. The primary service is a closed group.
The program keeps all SPEP documents in a binder labeled Impact of Crime. The program
provides the service on Mondays and Wednesdays for one hour each. All youth receive Impact of
Crime. The director of case management stated the youth are prioritized by length of time in the
program. The level eight youth are put in a cohort immediately. The level ten youth are put in on
how many youth need it per dorm. They have more time available to receive the primary service.
According to the facility's Standard Program Evaluation Protocol procedure, each primary service
will be facilitated in accordance to its manual. The primary service will be conducted as indicated
in the manual/protocol; this includes adherence to the session plan, group size, population,
frequency, needed materials, duration, and interventions. The facilitator of the primary service will
specify the topic covered, interventions utilized, duration of the service, youth present, and
progress for each session on the sign-in sheet.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Impact of Crime was observed on 5/19/2021. There were seven youth present for the
group along with one staff, the facilitator, and co-facilitator. The group began at 2:35p and was
held in the dorm’s classroom. The facilitator and co-facilitator were knowledgeable of the material
and followed the lesson plan. The youth played around and made jokes at the beginning of the
group, but did settle down and participated in group. The youth inside the dorm next to the
classroom were loud and that caused the some of the youth in group to lose focus. The facilitator
and co-facilitator were able to refocus the youth after each disruption. The group ended at 3:30p.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The documentation reviewed for the annual review period of April 15, 2020 through April 14, 2021
showed the program did not experience facilitator turnover for Impact of Crime. There were
eleven cohorts completed during the annual review period. The dates of the cohorts are:
5/4/2020-8/31/2020, 5/4/2020-8/31/2020, 5/4/2020-8/31/2020, 5/4/2020-8/31/2020, 9/21/2020-
1/25/2021, 9/21/2020-1/25/2021, 9/21/2020-1/25/2021, 9/21/2020-1/25/2021, 2/15/2021-current,
2/15/2021-current, and 2/15/2021-current. The director of case management stated the program
runs multiple cohorts at a time, which are based on the dorm assignments and each group covers
the same lesson. There were three gaps of twenty-one days between cohorts. The technical
assistance specialist spoke with the director of case management about the gap in service. The
director of case management stated she believed the program could have a gap of no more than
three weeks between cohorts. Technical assistance specialist informed the director of case
management there could be no more than two weeks between cohorts.
The cohorts which began on 5/4/2020 were run on four dorms: Alpha, Charlie, Delta, and Omega
dorms. The cohort on Alpha dorm began with six youth and ended with seven youth. A youth was
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
added to the Alpha dorm cohort on 6/1/2020. The cohort on Charlie dorm began with five youth
and ended with five youth. Two youth transferred out on 6/1/2020 and 6/8/2020 and two youth
were transferred in on 8/24/2020 and 8/26/2020. The cohort on Delta dorm began with seven
youth and ended with seven youth. One youth was transferred in and one youth was transferred
out. The cohort on Omega dorm began with ten youth and ended with eight.
The cohorts which began on 9/21/2020 were run on four dorms: Alpha, Charlie, Delta, and Echo
dorms. The cohort on Alpha dorm began with eight youth and ended with six. Two youth moved
dorms on 10/21/2020. The cohort on Charlie dorm began with ten youth and ended with nine
youth. One youth was removed for refusing two groups. There was a youth added on 9/28/2020
and a youth was arrested on 12/23/2020. The cohort on Delta dorm began with nine youth and
ended with eight. One youth was moved to another dorm. The cohort on Echo dorm began with
seven youth and ended with five. Two youth refused groups and were removed. Two other youth
were transferred in, but refused group and were removed.
The cohorts which began on 2/15/2021 are run on three dorms: Bravo, Delta, and Echo dorms.
The cohort on Bravo dorm began with nine youth and currently has eight. The cohort on Delta
dorm began with ten youth and currently has eight. One youth moved dorms and one youth was
arrested. The cohort on Echo dorm began with six and currently has seven. There was one youth
added.
According to the facility's Standard Program Evaluation Protocol procedure, TrueCore utilizes a
Probationary Review Status Process designed for effective management of risk through
individualized coaching, training and verification activities of all new employees during their first
90 days. The process encourages personal development and recognizes areas of improvement in
an effort to reduce staff turnover. In addition, the program provides positive incentives which, in
their experience has shown to improve employee motivation and dedication and leads to better
employee retention. Through this policy, employees receive credit for years of service,
employment referrals, and for contributing to the positive culture, teamwork environment and for
going above and beyond normal job expectations. Each employee who receives credit for doing
these things is entered in a monthly drawing at each facility. In the event of facilitator turnover, the
co-facilitator will assume the role of the facilitator for the group. If a co-facilitator is not present, a
new qualified facilitator for the primary service will take over conducting the service.
The program can earn 2 points if there are no unnecessary gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of the documentation for the annual review period of April 15, 2020 through April 14,
2021 showed the program completed forty-five fidelity monitorings of Impact of Crime, accounting
for each facilitator, each month services were provided. Five of the ten trained staff conducted the
program’s internal fidelity monitoring for the annual review period. The staff conducting the
internal fidelity checks were the two transition service managers, two case managers, and the
intake specialist. The director of case management assigns fidelity checks to be conducted on
staff facilitating groups.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
According to the facility's Standard Program Evaluation Protocol procedure, designated staff are
responsible for conducting fidelity monitoring of each facilitator of a primary service monthly. The
staff conducting the fidelity monitoring has received formal training by a qualified trainer in the
primary service. Fidelity monitoring is documented on a fidelity monitoring tool developed
specifically for that primary service by the developer of the manual, or an internal fidelity checklist
created for manualized or non-manualized curriculum delivered in the program. The staff
performing the fidelity monitoring of the primary service is verifying the facilitator is conducting the
group in accordance with the manual/protocol. This includes verifying the facilitator adheres to the
session plan, group size, population, frequency, needed materials, duration, and interventions.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
A review of the documentation for the review period showed there were no corrective actions
noted nor taken for Impact of Crime during the review period.
According to the facility's Standard Program Evaluation Protocol procedure, based on the results
of the monthly fidelity monitoring corrective action may be needed. The first stage of addressing
areas identified as needing improvement through fidelity monitoring is by providing coaching and
supervision to the facilitator on that issue by a qualified facilitator in the primary service. The
additional coaching and/or supervision will be documented by the person conducting the fidelity
monitoring in a note and placed in the clinical supervision file. If during additional fidelity
monitoring the previously identified area addressed continues to be an issue, the next stage is
completing a written warning and placing the facilitator on a Performance Improvement Plan. The
Performance Improvement Plan will state the issue identified, the tasks the facilitator needs to
complete to correct the issue, and a target date for completion. The plan will then be reviewed
with the facilitator by the qualified facilitator conducting the monitoring, and the facilitator’s
supervisor. In the event that an issue identified during the primary service is detrimental to the
youth’s safety and/or treatment, the matter will be taken up directly with the program’s Human
Resource Manager and be addressed in accordance with the employee handbook.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The documentation reviewed for the annual review period of April 15, 2020 through April 14, 2021
showed the evaluations have an area where they list the primary services and provide feedback
for the facilitator. Ten annual performance evaluations were provided for review. Nine of the ten
annual performance evaluations only listed the primary services and/or the numerical rating.
There was no specific information documenting the facilitator’s skill delivering Impact of Crime.
The director of case management had an annual evaluation, but the section was listed as N/A
because the director of case management was not trained in the primary service prior to the
annual evaluation.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
The facility's Standardized Program Evaluation Protocol procedure states an annual evaluation of
performance will be conducted on all the facilitators of each primary service. The performance
evaluation is comprised of feedback from the fidelity monitoring tools as well as an assessment of
the facilitator’s skill in delivering the primary service.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 16 weeks for this type of service. Of the 46 youth in the
sample, 100% (46 of 46) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 46 youth in the
sample, 100% (46 of 46) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 78%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 4 youth
Moderate-High = 14 youth
High = 18 youth
Total Youth in Sample = 46 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
Percentage of Youth with High-Risk Level to Reoffend: 39%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 39 youth
Total Youth in Sample = 46 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 78% (36 of 46) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 39% (18 of 46) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 20 25 80%
Totals 80 85 94%
This SPEP report evaluates Impact of Crime, an intervention delivered at Cypress Creek Juvenile
Offender Correctional Center / Cypress Creek Treatment Center.
47 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 46 total youth sampled, all
46 included dosage with end dates in the EBS Module. Of those youth, 46 received at least the
recommended weeks of service. Youth in the sample completed between 18 and 20 weeks of
service, with an average of 19 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10
The program earned 10 points for Amount of Service: Contact Hours. Of the 46 total youth
sampled, all 46 included dosage in the EBS Module. Of those youth, 46 received at least the
recommended hours of service. Youth in the sample completed 24 hours of service.
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 100%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Impact of Crime is an intervention suitable for all youth. The Department recommends providing
this service to every youth in the program.
Cypress Creek Juvenile Offender Correctional Center / Cypress Creek Treatment Center can
optimize their SPEP Quality of Service Delivery score by ensuring there are no unnecessary gaps
in service and each facilitator’s performance evaluation includes discussion of their skill in
delivering this primary service.
Cypress Creek Juvenile Offender Correctional Center / Cypress Creek Treatment Center can
optimize their SPEP Amount of Service score by ensuring that dosage for all youth is recorded
accurately in EBS and by ensuring that youth receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 11


state: FL

facility_name: Cypress Creek

summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Cypress Creek Juvenile Offender Correctional Center / Cypress
Creek Treatment Center
TrueCore Behavioral Solutions, LLC
(Contract Provider)
2855 West Woodland Ridge Drive
Lecanto, Florida 34461
Primary Service: Aggression Replacement Training
SPEP Review Period: April 15, 2021-April 14, 2022
SPEP Review Conducted: May 17, 2022
Florida Department of Juvenile Justice
Report Date: 7/29/22
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Aggression Replacement Training.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A review of documentation for the annual review period of April 15, 2021, through April 14, 2022,
shows the program utilizes Aggression Replacement Training (ART) as their primary service.
There are five staff who are trained to facilitate the primary service- the clinical director, assistant
clinical director and three therapists. Aggression Replacement Training is provided by a qualified
trainer over a three-day period. The clinical director or senior therapist conducts the monthly
internal fidelity checks.
According to the facility's Standardized Program Evaluation Protocol procedure, the facilitator for
each primary service will have completed a formalized training for that primary service from a
qualified trainer. To enhance the continuity of the service due to unexpected facilitator absence
and provide more individualized instruction as needed during the service, when possible, Cypress
Creek will make all efforts to have a co-facilitator for the primary service. The co-facilitator will
have completed a formalized training for the primary service from a qualified trainer.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The documentation reviewed for the annual review period of April 15, 2021 through April 14, 2022
confirms the program utilizes the Aggression Replacement Training (ART) manual. The manual is
broken down into six sections. The first section covers Skillstreaming. The second section deals
with anger control. The third section covers moral reasoning. Each of the first three sections has a
script for the facilitator to use for each lesson. Sections four through six are for the facilitator's
use. It includes training step guides, fidelity forms and hassle logs, and the program materials.
The program does not have separate student workbooks to give out. The facilitators make copies
of the exercises before each session to pass out to the youth.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
ART is facilitated as a closed group. The curriculum is to be delivered to the youth over a
minimum of ten (10) weeks for thirty (30) hours. The program keeps all SPEP documents in a
binder labeled "Aggression Replacement Training". The program provides the service on Monday,
Wednesday, and Friday for one hour from 4:15 to 5:15. All youth receive Aggression
Replacement Training. The clinical director stated a tracker is maintained based on the youth’s
arrival and anticipated release date. The clinical director schedules the youth to complete ART
prior to their discharge date.
Pursuant to the facility's Standardized Program Evaluation Protocol procedure, each primary
service will be facilitated in accordance to its manual. The primary service will be conducted as
indicated in the manual/protocol; this includes adherence to the session plan, group size,
population, frequency, needed materials, duration, and interventions. The facilitator of the primary
service will specify the topic covered, interventions utilized, duration of the service, youth present,
and progress for each session on the sign-in sheet.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Aggression Replacement Training was observed on 5/16/2021. There were eight
youth present for the group along with the facilitator and the assistant clinical director who
conducted internal fidelity monitoring. The group began at 4:15 p.m. The group was held in the
dorm’s classroom. The lesson covered was Skill 38, Responding to Failure. The staff was
knowledgeable of the material and followed the lesson plan. All eight youth participated in group
and provided good group discussion. The youth were not disruptive and were excited about their
upcoming group graduation date and party. The group ended at 5:11 p.m.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The documentation reviewed for the annual review period of April 15, 2021 through April 14, 2022
showed the program did experience facilitator turnover. One therapist transtioned to the education
department and one therapist left the program. The facilitator turnover did not negatively affect the
delivery of Aggression Replacement Training (ART).
There were four cohorts completed during the annual review period and a current cohort is set to
complete on 5/20/22. The dates of the cohorts are: 4/27/2021-7/1/2021, 7/13/21-9/16/21, 9/28/21-
12/3/21, 12/21/21-2/25/22, and 3/14/22-current. The program did not have any gaps in service.
There were two cohorts which running 4/27/2021-7/1/21. The cohort on Omega dorm began with
seven youth and ended with six youth. The youth was removed from the group on 5/19/2021 for
behavioral issues. The cohort on Delta dorm began with six youth and ended with five youth. A
youth was removed from the group immediately after the first session. The clinical director stated
the youth had completed all ten weeks of the primary service on 1/27/2020.
There were three cohorts run from 7/13/2021 to 9/16/21. The cohort on Omega dorm began with
six youth and ended with six youth. The cohort on Echo began with seven youth and ended with
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
six youth. One youth was moved to Brovo dorm. The cohort on Bravo dorm began with eight
youth and ended with nine youth.
There were two cohorts which began on 9/28/21 and ran through 12/3/21. The cohort on Omega
dorm began with seven youth and ended with seven youth. The cohort on Delta dorm began with
eight youth and ended with eight youth.
One cohort ran 12/21/21 to 2/25/22 on Charlie dorm. The cohort started with nine youth and
ended with seven youth. One youth was discharged from the program and one youth was
dropped ffrom the group for lack of perticipation.
There is one cohort on Alpha dorm which began on 3/14/22 and is ongoing at the time of this
annual review. The cohort started with nine youth and is set to complete on 5/20/22.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of documentation for the annual review period of April 15, 2021, through April 14, 2022
showed the program completed forty-eight (48) fidelity monitorings for facilitators of Aggression
Replacement Training (ART). The program is missing a Moral Reasoning fidelity in April 2021 for
one therapist; an Anger Control and Moral Reasoning fidelity for one therapist and a
Skillstreaming fidelity for the other therapist in May 2021; a Skillstreaming fidelity for both
therapists in June 2021; a Moral Reasoning fidelity for one therapist for the months of July 2021,
August 2021, and October 2021; an Anger Control fidelity for one therapist for September 2021
and November 2021; a Moral Reasoning fidelity for a therapist for July 2021, October 2021, and
November 2021; an Anger Control fidelity for one therapist for July 2021 and September 2021; a
Skillstreaming fidelity for one therapist in July 2021; a Moral Reasoning fidelity for a therapist for
July 2021, August 2021, and September 2021; and an Anger Control and Skillstreaming fidelity
for July 2021 and September 2021. The Clinical Director or Senior Therapist conducted the
monthly internal fidelity checks during the annual review period.
The technical Assistance Specialist informed the clinical director on 5/19/2021 during the previous
annual review that all monthly internal fidelity checks needed to be completed by a different staff
member trained in the primary service. The Clinical Director stated the new facilitators conduct
“self-fidelity” checks monthly. The fidelity checks which were completed for the months of April
2021 and May 2021 were conducted as “self-fidelity” checks which did not count for this annual
review. The Clinical Director was informed each facilitator needed to have one fidelity check per
section of ART per month. This would cover the developer’s requirement of twenty percent fidelity
checks for Skillstreaming, twenty percent fidelity checks for anger control, and twenty percent
fidelity checks for moral reasoning for the ten-week cohort.
The fidelity check completed on 6/24/2021 was completed by the same observer on two different
facilitators who were running groups at the same time. The Clinical Director stated she asked the
observer to split the time between both groups. The observer spent thirty minutes with one
facilitator and the last thirty minutes with the other facilitator. The Technical Assistance Specialist
informed the Clinical Director the observer needed to remain with one group for the full hour to
ensure the group is running to the fidelity of the model.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
According to the facility's Standardized Program Evaluation Protocol procedure, designated staff
are responsible for conducting fidelity monitoring of each facilitator of a primary service monthly.
The staff conducting the fidelity monitoring has received formal training by a qualified trainer in the
primary service. Fidelity monitoring is documented on a fidelity monitoring tool developed
specifically for that primary service by the developer of the manual, or an internal fidelity checklist
created for manualized or non-manualized curriculum delivered in the program. The staff
performing the fidelity monitoring of the primary service is verifying the facilitator is conducting the
group in accordance with the manual/protocol. This includes verifying the facilitator adheres to the
session plan, group size, population, frequency, needed materials, duration, and interventions.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
A review of the documentation for the review period showed there were no corrective actions
noted on the fidelity monitoring for facilitators of Aggression Replacement Training.
Pursuant to the facility's Standardized Program Evaluation Protocol procedure, based on the
results of the monthly fidelity monitoring corrective action may be needed. The first stage of
addressing areas identified as needing improvement through fidelity monitoring is by providing
coaching and supervision to the facilitator on that issue by a qualified facilitator in the primary
service. The additional coaching and/or supervision will be documented by the person conducting
the fidelity monitoring in a note and placed in the clinical supervision file. If during additional
fidelity monitoring the previously identified area addressed continues to be an issue, the next
stage is completing a written warning and placing the facilitator on a Performance Improvement
Plan. The Performance Improvement Plan will state the issue identified, the tasks the facilitator
needs to complete to correct the issue, and a target date for completion. The plan will then be
reviewed with the facilitator by the qualified facilitator conducting the monitoring, and the
facilitator’s supervisor. In the event an issue identified during the primary service is detrimental to
the youth’s safety and/or treatment, the matter will be taken up directly with the program’s Human
Resource Manager and be addressed in accordance with the employee handbook.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The documentation reviewed for the annual review period of April 15, 2021 through April 14, 2022
showed the evaluations have an area where they list the primary services and provide feedback
for the facilitator. Four annual performance evaluations were provided for review. All four annual
performance evaluations listed the primary services and/or the numerical rating. There was also
specific information highlighting the facilitator’s skill delivering Aggression Replacement Training.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
The facility's Standardized Program Evaluation Protocol procedure states an annual evaluation of
performance will be conducted on all the facilitators of each primary service. The performance
evaluation is comprised of feedback from the fidelity monitoring tools as well as an assessment of
the facilitator’s skill in delivering the primary service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 10 weeks for this type of service. Of the 65 youth in the
sample, 95% (62 of 65) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 30 contact hours for this type of service. Of the 65 youth in the
sample, 98% (64 of 65) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 10 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 40%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 72%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 12 youth
Moderate-High = 20 youth
High = 15 youth
Total Youth in Sample = 65 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
Percentage of Youth with High-Risk Level to Reoffend: 23%
High Score: 5 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 38%
Table 2
High = 15 youth
Total Youth in Sample = 65 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 72% (47 of 65) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 23% (15 of 65) youth scored High-Risk to reoffend, for a score of 5 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 10 25 40%
Totals 81 100 81%
This SPEP report evaluates Aggression Replacement Training, an intervention delivered at
Cypress Creek Juvenile Offender Correctional Center / Cypress Creek Treatment Center.
65 youth completed the program during the review period. Of those youth, all 65 were included in
the sample.
The program scored High for Quality of Service Delivery.
The targeted number of weeks for this specific intervention is 10 weeks; the targeted number of
hours for this specific intervention is 30 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 5 services (15 weeks; 45
hours).
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10
The program earned 8 points for Amount of Service: Duration. Of the 65 total youth sampled, only
62 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 62
received at least the recommended weeks of service. Youth in the sample completed between 10
and 19 weeks of service, with an average of 10 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 65 total youth
sampled, only 64 included dosage in the EBS Module. Of those youth with correct dosage, 64
received at least the recommended hours of service. Youth in the sample completed between 30
and 60 hours of service, with an average of 31 hours.
The program was awarded 10 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 96%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Cypress Creek Juvenile Offender Correctional Center / Cypress Creek Treatment Center can
optimize their SPEP Quality of Service Delivery score by ensuring fidelity monitoring is completed
every month for each facilitator.
Cypress Creek Juvenile Offender Correctional Center / Cypress Creek Treatment Center can
optimize their SPEP Amount of Service score by ensuring dosage for all youth is recorded
accurately in EBS and by ensuring youth receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 11

source: DJJ

report_type: SPEP

file_name: FY2021-22 Cypress Creek ART.pdf

program_name_raw: Cypress Creek Treatment Center - TrueCore

program_status_note: Closed

service_type: Aggression Replacement Training


summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Cypress Creek Juvenile Offender Correctional Center / Cypress
Creek Treatment Center
TrueCore Behavioral Solutions, LLC
(Contract Provider)
2855 West Woodland Ridge Drive
Lecanto, Florida 34461
Primary Service: Impact of Crime
SPEP Review Period: April 15,2021-April 14, 2022
SPEP Review Conducted: May 17, 2022
Florida Department of Juvenile Justice
Report Date: 7/29/22
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others
Job-Related Training: includes vocational counseling, job training arranged to provide work
experience. The overall emphasis is on preparing the juveniles to enter the work force
Individual Counseling: includes any range of treatment techniques that focus on psychological or
interpersonal problems or issues faced by an individual that involves a one-on-one relationship
with a therapist or counselor
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A review of documentation for the quarterly review period of April 15, 2021 through April 14, 2022
shows the program utilizes Impact of Crime (IOC) as their primary service. There are ten (10) staff
who are trained to facilitate the primary service: the Director of Case Management, six Case
Managers, two Transition Services Managers, and one Intake Specialist. Eight staff facilitated a
cohort of IOC during the annual review period. Training for IOC is conducted in-person for twenty-
eight (28) hours over three days by a qualified trainer. The Director of Case Management assigns
a trained staff member to conduct the monthly internal fidelity checks. All trained staff members
can conduct the internal fidelity checks.
According to the facility's Standardized Program Evaluation Protocol procedure, the facilitator for
each primary service will have completed a formalized training for that primary service from a
qualified trainer. To enhance the continuity of the service due to unexpected facilitator absence
and provide more individualized instruction as needed during the service, when possible, Cypress
Creek will make all efforts to have a co-facilitator for the primary service. The co-facilitator will
have completed a formalized training for the primary service from a qualified trainer.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The documentation reviewed for the annual review period of April 15, 2021 through April 14, 2022
confirmed the program utilizes the Impact of Crime: Addressing the Harm manual. The manual
opens with an introduction to the Impact of Crime Curriculum. Then the manual goes over all
seven chapters of the group and provides a script for the facilitator to follow. The manual also has
an appendix and a student guide with all the student exercises. The program does not have
separate student workbooks to give out. The facilitators make copies of the exercises before each
session to pass out to the youth. The group is a closed group with no more than twelve youth in
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
each cohort. The curriculum is to be delivered to the youth over a minimum of sixteen (16) weeks
for thirty-two (32) sessions.
The program keeps all SPEP documents in a binder labeled "Impact of Crime". The program
previously provided the service on Mondays and Wednesdays; however, the program now
provides the service for one hour on Tuesdays and Thursdays from 2:30 to 3:30. All youth receive
Impact of Crime. The Director of Case Management stated the youth are prioritized by length of
time in the program. The level eight youth are put in a cohort immediately. The level ten youth are
put in on how many youth need it per dorm. They have more time available to receive the primary
service.
Pursuant to the facility's Standardized Program Evaluation Protocol procedure, each primary
service will be facilitated in accordance to its manual. The primary service will be conducted as
indicated in the manual/protocol; this includes adherence to the session plan, group size,
population, frequency, needed materials, duration, and interventions. The facilitator of the primary
service will specify the topic covered, interventions utilized, duration of the service, youth present,
and progress for each session on the sign-in sheet.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Impact of Crime was observed on 5/17/2022. There were nine youth, two staff and
the facilitator and co-facilitator present for the group. The group began at 2:30 p.m. and was held
in the dorm’s classroom. The facilitator and co-facilitator were knowledgeable of the material and
followed the lesson plan. One youth refused to perticipate in the group and left the classroom. The
other youth would periodically get off topic to express their personal issues with the program. The
facilitator and co-facilitator were able to refocus the youth. The group ended at 3:37 p.m.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The documentation reviewed for the annual review period of April 15, 2021 through April 14, 2022
showed the program did experience facilitator turnover. One trained facilitator leave the case
management unit and another trained facilitator is no longer employed at the program. The
facilitator who left the program was running groups at the time she left the program. The program
had another trained facilitator take over the group with no gap in service.
There were ten (10) cohorts of IOC completed during the annual review period. Three cohorts
began on 2/15/2021. One cohort began with six youth and ended with eight youth. Another cohort
began with ten youth and ended with eight youth. The third cohort began with nine youth and
ended with seven youth. The Director of Case Management stated the program runs multiple
cohorts at a time which are based on the dorm assignments and each group covers the same
lesson. Some of the youth are transferred to another dorm which is assigned to a different
facilitator, but are completing the same lesson.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Four cohorts began on 6/28/2021. The cohort on Alpha dorm began with eight youth and ended
with five youth. Three youth were moved to another dorm. There were several different staff
facilitating the cohort. According to the Director of Case Management, there was no set Case
Manager to run the cohort on Alpha dorm. The Case Manager assigned to the dorm left the
program and the position was vacant throughout the entire cohort. The cohort in Charlie dorm
began with seven youth and ended with seven youth. The cohort on Echo dorm began with five
youth and ended with seven youth. Some of the youth were transferred to another dorm which is
assigned to a different facilitator, but are completing the same lesson.
Two cohorts began on 11/15/2021. The cohort on Bravo dorm began with seven youth and ended
with eight youth. The cohort on Echo dorm began with five youth and ended with five youth.
There is currently one cohort being held in the Alpha dorm which began on 4/5/22 with 10 (10)
youth.
According to the facility's Standardized Program Evaluation Protocol procedure, TrueCore utilizes
a Probationary Review Status Process designed for effective management of risk through
individualized coaching, training and verification activities of all new employees during their first
90 days. The process encourages personal development and recognizes areas of improvement in
an effort to reduce staff turnover. In addition, the program provides positive incentives to improve
employee motivation and dedication. Through this policy, employees receive credit for years of
service, employment referrals, and for contributing to the positive culture, teamwork environment
and for going above and beyond normal job expectations. Each employee who receives credit for
doing these things is entered in a monthly drawing at each facility. In the event of facilitator
turnover, the co-facilitator will assume the role of the facilitator for the group. If a co-facilitator is
not present, a new qualified facilitator for the primary service will take over conducting the service.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of documentation for the quarterly review period of April 15, 2021 through July 14, 2021
showed the program completed thirty-eight (38) fidelity monitoring sessions for facilitators of
Impact of Crime. The Intake Specialist, two Transition Service Managers, and the Director of
Case Management conducted the monthly internal fidelity checks. There were no missing internal
fidelity monitorings for the annual review.
According to the facility's Standardized Program Evaluation Protocol procedure, designated staff
are responsible for conducting fidelity monitoring of each facilitator of a primary service monthly.
The staff conducting the fidelity monitoring has received formal training by a qualified trainer in the
primary service. Fidelity monitoring is documented on a fidelity monitoring tool developed
specifically for that primary service by the developer of the manual, or an internal fidelity checklist
created for manualized or non-manualized curriculum delivered in the program. The staff
performing the fidelity monitoring of the primary service is verifying the facilitator is conducting the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
group in accordance with the manual/protocol. This includes verifying the facilitator adheres to the
session plan, group size, population, frequency, needed materials, duration, and interventions.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
A review of documentation for the review period and an interview with the Director of Case
Management showed there were no corrective actions noted on the fidelity monitoring for
facilitators of Impact of Crime.
Pursuant to the facility's Standardized Program Evaluation Protocol procedure, based on the
results of the monthly fidelity monitoring corrective action may be needed. The first stage of
addressing areas identified as needing improvement through fidelity monitoring is by providing
coaching and supervision to the facilitator on that issue by a qualified facilitator in the primary
service. The additional coaching and/or supervision will be documented by the person conducting
the fidelity monitoring in a note and placed in the clinical supervision file. If during additional
fidelity monitoring the previously.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The documentation reviewed for the annual review period of April 15, 2020 through April 14, 2021
showed the evaluations have an area where they list the primary services and provide feedback
for the facilitator. Ten annual performance evaluations were provided for review. The performance
evaluation is comprised of feedback from the fidelity monitoring tools as well as an assessment of
the facilitator’s skill in delivering Impact of Crime.
The facility's Standardized Program Evaluation Protocol procedure states an annual evaluation of
performance will be conducted on all the facilitators of each primary service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 16 weeks for this type of service. Of the 64 youth in the
sample, 100% (64 of 64) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 64 youth in the
sample, 100% (64 of 64) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 10 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 40%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 72%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 11 youth
Moderate-High = 20 youth
High = 15 youth
Total Youth in Sample = 64 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 23%
High Score: 5 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 38%
Table 2
High = 15 youth
Total Youth in Sample = 64 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 72% (46 of 64) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 23% (15 of 64) youth scored High-Risk to reoffend, for a score of 5 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 10 25 40%
Totals 70 85 82%
This SPEP report evaluates Impact of Crime, an intervention delivered at Cypress Creek Juvenile
Offender Correctional Center / Cypress Creek Treatment Center.
65 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 64 total youth sampled, all
64 included dosage with end dates in the EBS Module. Of those youth, 64 received at least the
recommended weeks of service. Youth in the sample completed between 18 and 28 weeks of
service, with an average of 19 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 64 total youth
sampled, all 64 included dosage in the EBS Module. Of those youth, 64 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
recommended hours of service. Youth in the sample completed between 24 and 34 hours of
service, with an average of 24 hours.
The program was awarded 10 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 100%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Cypress Creek Juvenile Offender Correctional Center / Cypress Creek Treatment Center can
optimize their SPEP Quality of Service Delivery score by ensuring facilitator turnover does not
impact groups.
Cypress Creek Juvenile Offender Correctional Center / Cypress Creek Treatment Center can
optimize their SPEP Amount of Service score by ensuring dosage for all youth is recorded
accurately in EBS and by ensuring youth receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

file_name: FY2021-22 Cypress Creek IOC.pdf

service_type: Impact of Crime


summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Cypress Creek Juvenile Offender Correctional Center / Cypress
Creek Treatment Center
TrueCore Behavioral Solutions, LLC
(Contract Provider)
2855 West Woodland Ridge Drive
Lecanto, Florida 34461
Primary Service: Aggression Replacement Training
SPEP Review Period: 4/15/2022-4/14/2023
SPEP Review Conducted: 5/9/2023
Florida Department of Juvenile Justice
Report Date: 9/27/2023
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Aggression Replacement Training.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 4 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A review of documentation for the review period of April 15, 2022 through April 14, 2023 shows
there were six staff members trained to facilitate Aggression Replacement Training (ART)- one
clinical director, two assistant clinical directors, and three therapists. Fidelity was conducted by
the clinical director, assistant clinical director, and therapist. During the review period, all trained
staff left employment at the program. There were no staff available to provide ART as of
November 2022. Facilitator training is provided by a qualified trainer over a three-day period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the current version of Aggression Replacement Training (ART). ART is held
three days a week on Monday, Wednesday, and Friday. There is a facilitator manual which
outlines the three components of ART, Skillstreaming, anger control, and moral reasoning.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
An observation of Aggression Replacement Training was unable to be conducted during the
review period, therefore this indicator is rated as non-applicable.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 0 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program has a process in place to address facilitator turnover for Aggression Replacement
Training (ART). In the event of facilitator turnover, the co-facilitator will assume the role of the
facilitator for the group. If a co-facilitator is not present, a new qualified facilitator for the primary
service will take over conducting the service.
The documentation reviewed showed the program experienced significant facilitator turnover
during the review period. As of November 2022, there were no trained staff to conduct ART
groups.
There were five cohorts conducted during the review period. The first cohort started 3/14/22 with
nine youth and ended 5/20/22 with nine youth.
The second, third, and fourth cohorts all ran from 6/6/22 through 8/12/22. The second cohort
started and ended with six youth. The third cohort started and ended with five youth. The fourth
cohort started with eight and ended with seven, with one youth being removed for refusing to
participate.
The fifth cohort started 8/9/22 with seven youth and ended 11/4/22 with six youth. One youth was
released prior to completing.
There have been no ART cohorts since 11/4/22.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has a process in place to conduct monthly internal fidelity monitoring of Aggression
Replacement Training facilitators. A review of documentation showed the program completed
fifteen (15) fidelity monitorings. There was no internal fidelity monitoring for August through
November 2022. Groups were not held after November.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 0 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The program has a process in place to address corrective actions identified during monthly
internal fidelity monitoring of Aggression Replacement Training (ART) facilitators. A review of
documentation showed there were no corrective actions noted nor taken during the months ART
groups were held.
There was no fidelity monitoring conducted for eight of the twelve (12) months of the review
period, with the service not being run for the most recent five months. Therefore, this indictor is
ineligible to receive any points.
The program can earn 1 point if any corrective action needed is identified, applied, and
demonstrated.
At the time of the review, the program did not meet criteria to earn a score of 1.
g. Evaluation of Facilitator Skill Delivering the Basic Score: N/A Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
At the time of the annual review, there were no facilitators eligible for an annual performance
evaluation to include evaluation of their skill in delivering Aggression Replacement Training.
Therefore, this indicator is rated as non-applicable.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 10 weeks for this type of service. Of the 50 youth in the
sample, 78% (39 of 50) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 30 contact hours for this type of service. Of the 50 youth in the
sample, 78% (39 of 50) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 18 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 72%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 86%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 10 youth
Moderate-High = 19 youth
High = 14 youth
Total Youth in Sample = 50 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 28%
High Score: 8 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 62%
Table 2
High = 14 youth
Total Youth in Sample = 50 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 86% (43 of 50) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 28% (14 of 50) youth scored High-Risk to reoffend, for a score of 8 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 18 25 72%
Totals 75 100 75%
This SPEP report evaluates Aggression Replacement Training, an intervention delivered at
Cypress Creek Juvenile Offender Correctional Center / Cypress Creek Treatment Center.
65 youth completed the program during the review period. Of those youth, 50 were included in the
sample.
The program scored High for Quality of Service Delivery.
The targeted number of weeks for this specific intervention is 10 weeks; the targeted number of
hours for this specific intervention is 30 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 5 services (15 weeks; 45
hours).
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 6 points for Amount of Service: Duration. Of the 50 total youth sampled, only
39 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 39
received at least the recommended weeks of service. Youth in the sample completed between 10
and 15 weeks of service, with an average of 10 weeks.
The program earned 6 points for Amount of Service: Contact Hours. Of the 50 total youth
sampled, only 39 included dosage in the EBS Module. Of those youth with correct dosage, 39
received at least the recommended hours of service. Youth in the sample completed between 30
and 42 hours of service, with an average of 30 hours.
The program was awarded 18 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 82%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Cypress Creek Juvenile Offender Correctional Center / Cypress Creek Treatment Center can
optimize their SPEP Quality of Service Delivery score by ensuring facilitator turnover does not
impact groups, fidelity monitoring is completed every month for each facilitator, and any corrective
action needed is identified, applied, and demonstrated.
Cypress Creek Juvenile Offender Correctional Center / Cypress Creek Treatment Center can
optimize their SPEP Amount of Service score by ensuring dosage for all youth is recorded
accurately in EBS and by ensuring youth receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2022-23 Cypress Creek ART.pdf