state: FL

facility_name: Jacksonville Youth Academy

summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Jacksonville Youth Academy
TrueCore Behavioral Solutions, LLC.
(Contract Provider)
4501 Lannie Road
Jacksonville, Florida 32218
Primary Service: Cannabis Youth Treatment (CYT)
SPEP Review Period: March 1, 2023-February 29, 2024
SPEP Review Conducted: 2/29/2024 and 3/25/2024
Florida Department of Juvenile Justice
Report Date: 9/17/2025
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Cannabis Youth Treatment.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A review of the documentation for Cannabis Youth Treatment (CYT) for the annual review period
of March 1, 2023 through February 29, 2024 shows the program utilizes Cannabis Youth
Treatment (CYT) as a primary service. Cannabis Youth Treatment (CYT) training is an in-service
training provided by the Clinical Director or other approved person. The documentation shows
there were six staff trained to facilitate Cannabis Youth Treatment during the annual review
period. The trained staff are two part-time licensed therapists, a full-time licensed therapist, a non-
licensed therapist, the part-time Master’s Level Certified Addiction Professional (MCAP), and the
current Clinical Director. The current Clinical Director and the non-licensed therapist facilitated the
primary service during the annual review period. Two part-time licensed therapists, the Clinical
Director, a non-licensed therapist, and the part-time Master’s Level Certified Addiction
Professional (MCAP) conducted the internal fidelity checks for the annual review period.
The program has a policy and procedure in place to address facilitator training. The facilitator for
the primary service will have training from a licensed clinician or other qualified trainer. This
training will be documented in the facilitators on the job training plan. The facilitator will receive
additional training on the curriculum as needed from the licensed clinician documented within the
clinical supervision. All facilitators will receive training prior to conducting the service.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The documentation reviewed for Cannabis Youth Treatment for the annual review period of March
1, 2023 through February 29, 2024 shows the program utilizes the Cannabis Youth Treatment
Series Volumes one and two. Technical Assistance Specialist reviewed the manual. The manuals
provide step by step procedures for implementing the program. There are no student manuals.
The facilitators make copies of the worksheets and pass them out to the youth during group. The
first two lessons are conducted as individual sessions. These sessions are an hour in length.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Lessons three through twelve are conducted as a group. All youth in the program are required to
receive the primary service. The program maintains all documentation in a binder labeled
Cannabis Youth Treatment (CYT) located in the Clinical Director’s office. The binder contains all
attendance sheets, internal fidelity checklists, and training certificates. The program provides the
primary service one time per week on Wednesdays for one hour. The primary service is a closed
group.
The program has a policy and procedure in place to address the treatment manual/protocol. The
primary service will be conducted as indicated in the manual/protocol; this includes adherence to
the session plan, group size, population, frequency, needed materials, duration, and interventions.
The facilitator of the primary service will specify the topic covered, interventions utilized, duration
of the service, youth present, and progress for each session on the sign-in sheet.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Technical Assistance Specialist observed a session of Cannabis Youth Treatment (CYT) on
2/29/2024 via Microsoft Teams. The group began at 2p. The group was held in the program’s
conference room. There were ten youth, one facilitator, and one youth care worker present during
the group. The facilitator was knowledgeable of the material and followed the lesson plan. The
group was on the final lesson of the curriculum. The facilitator asked the youth “Why would you
want to go back to smoking marijuana?” There were about four or five youth who began rolling
“joints” with Playdoh. The facilitator attempted to get the youth back on track, but the youth
seemed to put their focus on themselves and not the facilitator. There were some youths who
tried to actively participate in the group, but they were overshadowed by the other youth. The
group ended at 2:50p.
The Clinical Director stated the youth in the current cohort were all in pre-contemplation about
changing their marijuana usage. The Clinical Director stated the youth would continually talk
about using marijuana when they get out no matter how many times she tried to redirect the
youth.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The documentation reviewed for Cannabis Youth Treatment for the annual review period of March
1, 2023 through February 29, 2024 shows the program did have facilitator turnover. The facilitator
turnover did not affect the delivery of the primary service. The full-time licensed therapist left the
program, but was not facilitating any cohorts at the time he left. The program completed four
cohorts during the annual review period. The dates of the cohorts are 2/21/2023-5/10/2023,
6/14/2023-8/30/2023, 11/15/2023-1/17/2024, and 12/28/2023-2/29/2024. There was a gap in
service of thirty-five (35) days between the cohort which ended on 5/10/2023 and the cohort which
began on 6/14/2023. The Regional Compliance Director stated the program waited until a youth
on the waitlist was admitted to the program prior to beginning the service since the service is a
closed group and all youth other than the one new youth would have completed the service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
There was another gap in service of seventy-seven (77) days between the cohort which ended on
8/30/2023 and the cohort which began on 11/15/2023. The Clinical Director stated the program
did not have any youth who needed the service during that time. The program had a limited
capacity due to construction in the dorms so the program could only use one side of the dorm at a
time. This limited the program to twelve (12) youth. The dorm construction was completed on
9/29/2023 and the program began receiving new intakes. The program had enough youth
admitted to the program to begin a cohort on 11/15/2023.
The cohort which began on 2/21/2023 started with nine youth and ended with nine youth.
The cohort which began on 6/14/2023 started with seven youth and ended with six youth. One
youth was transferred on 7/26/2023.
The cohort which began on 11/15/2023 began with nine youth and ended with eight youth. The
attendance sheet dated 12/6/2023 and 12/13/2023 did not have the facilitator’s signature or name
listed. One youth was arrested on 1/14/2024.
The cohort which began on 12/28/2023 began with ten (10) youth and ended with ten (10) youth.
The program has a policy in place to limit facilitator turnover and increase employee retention.
The program utilizes positive incentives and a Probationary Review Status Process designed to
improve employee motivation and dedication.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation reviewed for Cannabis Youth Treatment for the annual review period of March
1, 2023 through February 29, 2024 shows the program conducted eleven (11) monthly internal
fidelity checks. The dates of the fidelity monitoring were: 3/27/2023, 6/28/2023, 7/5/2023,
8/9/2023, 11/22/2023, 11/15/2023, 12/28/2023, 12/21/2023, 1/17/2024, 1/25/2024, and 2/7/2024.
Two part-time licensed therapists, the Clinical Director, a non-licensed therapist, and the part-time
Master’s Level Certified Addiction Professional (MCAP) conducted the internal fidelity checks for
the annual review period. There were two missing internal fidelity checks during the annual review
period. The missing fidelity checks were for the Clinical Director for the months of April 2023 and
May 2023. The program conducts the fidelity monitoring during lessons three through twelve since
those lessons are conducted as a group. Lessons one and two are conducted as individual
sessions.
The internal fidelity check dated 8/9/2023 shows the part time licensed therapist conducted the
fidelity check on herself. The Clinical Director stated she was the facilitator and that was a
documentation error. The attendance sheets confirm the Clinical Director was facilitating the
group on the day the fidelity check was completed.
The program has a policy in place to address internal fidelity monitoring. There are designated
staff responsible for conducting monthly fidelity monitoring of each facilitator. The staff performing
the fidelity monitoring of the primary service is verifying the facilitator is conducting the group in
accordance with the manual/protocol. This includes verifying the facilitator adheres to the session
plan, group size, population, frequency, needed materials, duration, and interventions.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The documentation reviewed for Cannabis Youth Treatment for the annual review period of March
1, 2023 through February 29, 2024 shows there were no corrective actions noted.
The program has a policy in place to address corrective actions identified during the monthly
fidelity monitoring. The program utilizes several steps to address corrective actions beginning with
additional coaching and increasing to written warnings or placing facilitator on a Performance
Improvement Plan if the corrective action has not been ameliorated.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The documentation reviewed for Cannabis Youth Treatment for the annual review period of March
1, 2023 through February 29, 2024 shows the program did not provide any staff performance
evaluations for the trained facilitators. The interim Facility Administrator stated there were no staff
who were at the program long enough for an annual performance evaluation or the staff’s annual
performance evaluations did not state specific information about the facilitator’s skill delivering the
primary service. The program did not provide a ninety-day (90) evaluation for any trained staff.
The program has a policy and procedure in place for annual staff performance evaluations. The
program’s policy states an annual evaluation of performance will be conducted on all the
facilitators of each primary service. The performance evaluation is comprised of feedback from
the fidelity monitoring tools as well as an assessment of the facilitator’s skill in delivering the
primary service.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 12 weeks for this type of service. Of the 26 youth in the
sample, 100% (26 of 26) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 12 contact hours for this type of service. Of the 26 youth in the
sample, 100% (26 of 26) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 92%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 3 youth
Moderate-High = 11 youth
High = 10 youth
Total Youth in Sample = 26 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 38%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 10 youth
Total Youth in Sample = 26 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 92% (24 of 26) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 38% (10 of 26) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 23 25 92%
Totals 98 100 98%
This SPEP report evaluates Cannabis Youth Treatment, an intervention delivered at Jacksonville
Youth Academy.
26 youth completed the program during the review period. Of those youth, 26 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 26 total youth sampled,
only 26 included dosage with end dates in the EBS Module. Of those youth with correct dosage,
26 received at least the recommended weeks of service. Youth in the sample completed between
12 and 13 weeks of service, with an average of 12.5 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program earned 10 points for Amount of Service: Contact Hours. Of the 26 total youth
sampled, only 26 included dosage in the EBS Module. Of those youth with correct dosage, 26
received at least the recommended hours of service. Youth in the sample completed between 12
and 13 hours of service, with an average of 12.15 hours.
The targeted number of weeks for this specific intervention is 12 weeks; the targeted number of
hours for this specific intervention is 12 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 5 services (15 weeks; 45
hours).
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
RECOMMENDATION(S):
Jacksonville Youth Academy can optimize their SPEP Quality of Service Delivery score by
ensuring fidelity monitoring is completed every month for each facilitator and each facilitator’s
performance evaluation includes discussion of their skill in delivering this primary service.
Jacksonville Youth Academy can maintain their SPEP Amount of Service score by continuing to
ensure dosage for all youth is recorded accurately in EBS and by continuing to ensure youth
receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

source: DJJ

report_type: SPEP

file_name: FY2023-24 Jacksonville Youth Academy CYT.pdf

program_name_raw: Jacksonville Youth Academy - TrueCore

program_status_note: Closed

service_type: Cannabis Youth Treatment


summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Jacksonville Youth Academy
TrueCore Behavioral Solutions, LLC.
(Contract Provider)
4501 Lannie Road
Jacksonville, Florida 32218
Primary Service: Impact of Crime (IOC)
SPEP Review Period: March 1, 2023-February 29, 2024
SPEP Review Conducted: 2/29/2024 and 3/25/2024
Florida Department of Juvenile Justice
Report Date: 9/17/2025
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The documentation reviewed for Impact of Crime for the annual review period of March 1, 2023
through February 29, 2024 shows the program utilizes Impact of Crime (IOC) as a primary
service. Impact of Crime (IOC) training is a three-day training currently conducted via Microsoft
Teams or in person by a qualified trainer. The documentation shows there were eight program
staff and two corporate staff (interim Assistant Facility Administrator and interim Facility
Administrator) trained to facilitate Impact of Crime during the annual review period. The trained
staff are the former Training Coordinator, the former Facility Administrator, the former Transition
Service Manager, the current Transition Service Manager, three former Case Managers, one
current case manager, the interim Facility Administrator, and the interim Assistant Facility
Administrator. Three former case managers, the former Transition Service Manager, and the
former Facility Administrator facilitated the primary service during the annual review period. The
former Facility Administrator, the interim Assistant Facility Administrator, and the former Transition
Service Manager conducted the internal fidelity checks for the annual review period.
The program has a policy and procedure in place to address facilitator training. The facilitator for
the primary service will have training from a licensed clinician or other qualified trainer. This
training will be documented in the facilitators on the job training plan. The facilitator will receive
additional training on the curriculum as needed from the licensed clinician documented within the
clinical supervision. All facilitators will receive training prior to conducting the service.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The documentation reviewed for Impact of Crime for the annual review period of March 1, 2023
through February 29, 2024 shows the program utilizes the Impact of Crime: Addressing the Harm
manual. Technical Assistance Specialist reviewed the manual. The manual opens with an
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
introduction to the Impact of Crime Curriculum. Then the manual goes over all seven chapters of
the service and provides a script for the facilitator to follow. The manual also has an appendix and
a student guide with all the student exercises. The program makes copies of the student
exercises and passes them out to the youth during each lesson. The program conducts the
primary service on Mondays and Wednesdays or Tuesdays and Thursdays. The program keeps
all SPEP documentation (attendance sheets, fidelity checks, and training certificates) in a binder
labeled IOC SPEP documentation in the Case Management office. All youth admitted to the
program are required to participate in the primary service. The primary service is a closed group
with no more than twelve (12) youth assigned to one cohort.
The program has a policy and procedure in place to address the treatment manual/protocol. The
primary service will be conducted as indicated in the manual/protocol; this includes adherence to
the session plan, group size, population, frequency, needed materials, duration, and interventions.
The facilitator of the primary service will specify the topic covered, interventions utilized, duration
of the service, youth present, and progress for each session on the sign-in sheet.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Technical assistance specialist observed a session of Impact of Crime on 3/25/2024. The group
began at 3:14p. The group was held in one of the program’s classrooms. There were ten (10)
youth, one facilitator, and two staff present during the group. The facilitator was knowledgeable of
the material and followed the lesson plan. The facilitator attempted to make sure the youth knew
the concepts from the primary service. The group was working on chapter seven exercise two.
The facilitator made the youth explain what the concepts were during their discussion. There were
about two or three youth who were actively trying to participate and learn from the primary
service. The other youth were making jokes about the content. Several youths did not take the
group seriously. The facilitator tried to get the youth to see how the consequences of their actions
caused them to be committed to the program. The youth were still not taking ownership of their
actions. Technical Assistance Specialist informed the facilitator that he could have the youth go
back to previous chapters to review the content before completing the final session. The group
ended at 4:07p.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The documentation reviewed for Impact of Crime for the annual review period of March 1, 2023
through February 29, 2024 shows the program did have facilitator turnover. The facilitator
turnover affected the delivery of the primary service. A former Case Manager left the program in
March 2023. The former Case Manager was facilitating a cohort when she left the program. The
program had another trained facilitator take over the cohort to ensure there were no gaps in
service. There was another former Case Manager who left the program in October 2023. The
former Case Manager who left in October 2023 was co-facilitating a cohort. The other co-
facilitator continued facilitating the primary service to ensure there were no gaps in service. The
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
program completed three cohorts and currently have two cohorts ongoing during the annual
review period. The dates of the cohorts are 12/23/2022-4/20/2023, 5/22/2023-9/8/2023,
10/23/2023-2/8/2024, 12/11/2023-current (as of 2/29/2024), and 2/20/2024-current (as of
2/29/2024). There was a gap in service of thirty-two (32) days between the cohort which ended on
4/20/2023 and the cohort which began on 5/22/2023. The program stated the gap in service was
due to the lack of trained staff available. The former Transition Service Manager transferred in
from another program. The Facility Administrator stated he waited until the Transition Service
Manager arrived at the program before beginning a new cohort. The Facility Administrator stated
he did not want to have to switch facilitators during the middle of the cohort. There was another
gap in service of forty-five (45) days between the cohort which ended on 9/8/2023 and the cohort
which began on 10/23/2023. The former Transition Service Manager stated the program did not
have any new youth who needed the service during that time. The program had a limited capacity
due to construction in the dorms so the program could only use one side of the dorm at a time.
This limited the program to twelve (12) youth. The dorm construction was completed on 9/29/2023
and the program began receiving new intakes.
The cohort which began on 12/23/2022 began with ten (10) youth and ended with ten (10) youth.
The attendance sheet dated 3/3/2023 shows the facilitator was working on chapter four objective
four. The attendance sheet dated 3/6/2023 shows the facilitator went to chapter six objective one.
The attendance sheet dated 3/8/2023 shows the facilitator went back to chapter five objective one
to cover the chapter five. The attendance sheet dated 3/24/2023 shows the facilitator then went
back to chapter six objective two. The former Case Manager left the program. The attendance
sheet dated 4/5/2023 shows the Transition Service Manager facilitated on group with the Facility
Administrator. The Facility Administrator took over the cohort on 4/7/2023.
The cohort which began on 5/22/2023 started with nine youth and ended with eight youth. One
youth missed group on 6/26/2023 (there was no signature for the youth on the attendance sheet)
and there was no make-up session documented. The session the youth missed was chapter three
objective three. One youth was transferred on 7/26/2023. The attendance sheet dated on
9/6/2023 did not list the group times.
The cohort which began on 10/23/2023 began with eight youth and ended with seven youth. An
attendance sheet dated 12/11/2023 did not have the group start time or group end time. The
session was a make-up session for two youth who refused group on 10/25/2023. Technical
Assistance Specialist informed the Transition Service Manager the make-up sessions should be
completed as soon as possible since the curriculum builds upon the lesson before. The
attendance sheet dated 12/13/2023 did not list the group time. One youth was arrested on
1/14/2024.
The cohort which began on 12/11/2023 began with ten (10) youth and currently has ten (10) youth
(as of 2/29/2024). The attendance sheet dated 12/13/2023 did not list the end time. The
attendance sheets dated 12/20/2023 and 12/29/2023 did not list the group start or end time.
The cohort which began on 2/20/2024 began with four youth and currently has four youth (as of
2/29/2024).
The program has a policy in place to limit facilitator turnover and increase employee retention.
The program utilizes positive incentives and a Probationary Review Status Process designed to
improve employee motivation and dedication.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
The program can earn 2 points if facilitator turnover does not impact groups and there are no
unnecessary gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation reviewed for Impact of Crime for the annual review period of March 1, 2023
through February 29, 2024 shows the program conducted fifteen (15) monthly internal fidelity
checks. The dates of the fidelity monitoring were: 4/5/2023, 6/14/2023, 6/28/2023, 7/31/2023,
8/14/2023, 9/4/2023, 10/25/2023, 11/28/2023, 12/18/2023, 12/28/2023, 1/29/2024, 1/30/2024,
2/5/2024, 2/8/2024, and 2/29/2024. The former Facility Administrator, the interim Assistant Facility
Administrator, and the former Transition Service Manager conducted the internal fidelity checks
for the annual review period. The program was missing an internal fidelity check for the former
Case manager for the month of March 2023 and for the Transition Service Manager for the month
of May 2023.
The fidelity monitoring checklist dated for 2/5/2024 did not have the observer’s name listed on the
form.
The program has a policy in place to address internal fidelity monitoring. There are designated
staff responsible for conducting monthly fidelity monitoring of each facilitator. The staff performing
the fidelity monitoring of the primary service is verifying the facilitator is conducting the group in
accordance with the manual/protocol. This includes verifying the facilitator adheres to the session
plan, group size, population, frequency, needed materials, duration, and interventions.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The documentation reviewed for Impact of Crime for the annual review period of March 1, 2023
through February 29, 2024 shows there were no corrective actions noted.
The program has a policy in place to address corrective actions identified during the monthly
fidelity monitoring. The program utilizes several steps to address corrective actions beginning with
additional coaching and increasing to written warnings or placing facilitator on a Performance
Improvement Plan if the corrective action has not been ameliorated.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The documentation reviewed for Impact of Crime for the annual review period of March 1, 2023
through February 29, 2024 shows the program did not provide any staff performance evaluations
for the trained facilitators. The interim Facility Administrator stated there were no staff who were at
the program long enough for an annual performance evaluation or the staff’s annual performance
evaluations did not state specific information about the facilitator’s skill delivering the primary
service. The program did not provide a ninety-day (90) evaluation for any trained staff.
The program has a policy and procedure in place for annual staff performance evaluations. The
program’s policy states an annual evaluation of performance will be conducted on all the
facilitators of each primary service. The performance evaluation is comprised of feedback from
the fidelity monitoring tools as well as an assessment of the facilitator’s skill in delivering the
primary service.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 16 weeks for this type of service. Of the 26 youth in the
sample, 100% (26 of 26) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 26 youth in the
sample, 100% (26 of 26) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 92%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 3 youth
Moderate-High = 11 youth
High = 10 youth
Total Youth in Sample = 26 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
Percentage of Youth with High-Risk Level to Reoffend: 38%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 10 youth
Total Youth in Sample = 26 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 92% (24 of 26) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 38% (10 of 26) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 23 25 92%
Totals 73 85 86%
This SPEP report evaluates Impact of Crime, an intervention delivered at Jacksonville Youth
Academy.
26 youth completed the program during the review period. Of those youth, 26 were included in the
sample.
The program scored Medium for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 26 total youth sampled,
only 26 included dosage with end dates in the EBS Module. Of those youth with correct dosage,
26 received at least the recommended weeks of service. Youth in the sample completed between
16 and 17 weeks of service, with an average of 16 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10
The program earned 10 points for Amount of Service: Contact Hours. Of the 26 total youth
sampled, only 26 included dosage in the EBS Module. Of those youth with correct dosage, 26
received at least the recommended hours of service. Youth in the sample completed between 28
and 34 hours of service, with an average of 32 hours.
The program was awarded XXXX available points for Risk Level of Youth Served. This is
calculated using data from the Community Assessment Tool (CAT) assessment. This score
reflects youths' most recent CAT score prior to placement at the program. The program itself has
no control over youths' CAT risk level because the scored assessment was administered prior to
the youths' admission.
RECOMMENDATION(S):
Jacksonville Youth Academy can optimize their SPEP Quality of Service Delivery score by
ensuring facilitator turnover does not impact groups, fidelity monitoring is completed every month
for each facilitator, and each facilitator’s performance evaluation includes discussion of their skill
in delivering this primary service.
Jacksonville Youth Academy can maintain their SPEP Amount of Service score by continuing to
ensure dosage for all youth is recorded accurately in EBS and by continuing to ensure youth
receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 11

file_name: FY2023-24 Jacksonville Youth Academy IOC.pdf

service_type: Impact of Crime