summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Jacksonville Youth Academy
TrueCore Behavioral Solutions, LLC
(Contract Provider)
4501 Lannie Road
Jacksonville, Florida 32218
Primary Service: Cannabis Youth Treatment
SPEP Review Period: March 15, 2022-March 14, 2023
SPEP Review Conducted: April 20, 2023
Florida Department of Juvenile Justice
Report Date: 9/14/2023
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Cannabis Youth Treatment.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 6 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A review of documentation for the annual review period of March 15, 2022 through March 14,
2023 showed the program utilizes Cannabis Youth Treatment (CYT) as a primary service. CYT
training is an in-service training provided by the clinical director or other approved person. The
documentation shows there were five staff trained to facilitate Cannabis Youth Treatment during
the annual review period- the former clinical director, the current clinical director, and three
licensed therapists. Two licensed therapists, the former clinical director, and the current clinical
director facilitated the primary service during the annual review period. The licensed therapist and
the former clinical director conducted internal fidelity checks for the annual review period.
According to the Facility's operating procedure, the facilitator for the primary service will have
training from a licensed clinician. This training will be documented in the facilitators on the job
training plan. The facilitator will receive additional training on the curriculum as needed from the
licensed clinician documented within the clinical supervision. All facilitators will receive training
prior to conducting the service.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the Cannabis Youth Treatment (CYT) Series Volumes one and two. The
manuals provide step by step procedures for implementing the program. There are no student
manuals; the facilitators make copies of the worksheets and pass them out to the youth during
group. The first two lessons of CYT are conducted as individual sessions. These sessions are an
hour in length. Lessons three through twelve (12) are conducted as a group. All youth are
required to receive CYT.
The program maintains all documentation in a binder labeled "Cannabis Youth Treatment (CYT)",
located in the clinical director’s office. The binder contains all attendance sheets, internal fidelity
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
checklists, and training certificates. The program provides the primary service one time per week
on Wednesdays for one hour. CYT is run as a closed group.
Pursuant to the facility's Standardized Program Evaluation Protocol procedure, each primary
service will be facilitated in accordance to its manual. The primary service will be conducted as
indicated in the manual/protocol; this includes adherence to the session plan, group size,
population, frequency, needed materials, duration, and interventions. The facilitator of the primary
service will specify the topic covered, interventions utilized, duration of the service, youth present,
and progress for each session on the sign-in sheet.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Cannabis Youth Treatment was observed on 4/12/2023 via Teams. The group began
at 2:06pm and was held in the program’s cafeteria. There were nine youth, one facilitator, and a
youth care worker present during the group. The facilitator was knowledgeable of the material and
followed the lesson plan. The facilitator discussed the topic of Anger Management with the youth.
The facilitator asked the youth “What makes you feel angry?” The youth were assigned homework
to complete before the next group. The group ended at 3:05pm.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
Documentation reviewed for the annual review period of March 15, 2022 through March 14, 2023
shows the program experienced facilitator turnover which adversely affected the delivery of
Cannabis Youth Treatment. The former clinical director went to PRN status and then left the
program shortly after. Two licensed therapists also left the program during the annual review
period.
The program completed five cohorts and had one ongoing at the time of the annual review period.
The first cohort began on 2/7/2022 with nine youth and ended 5/10/2022 with ten (10) youth. One
youth was added to session three after completing the first two individual sessions.
The second cohort began on 5/28/2022 with eight youth and ended 8/13/2022 with seven youth.
One youth was removed from the group on 6/21/2022. The licensed therapist and former clinical
director alternated facilitation of the group.
The third cohort began on 7/10/2022 with two youth and ended 9/11/2022 with two youth. The two
youth were transferred from another facility and the program did not have the youth’s records to
determine if they successfully received the service. The program decided to ensure the youth
received all thirteen (13) sessions prior to their discharge. The youth received the first two
individual sessions and then jumped to lesson nine. The program did not provide the service in
order. The former clinical director stated the program decided to conduct the service this way for
these youth so they would be able to receive all lessons.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The fourth cohort began on 8/3/2022 with nine youth and ended 10/29/2022 with nine youth. One
youth missed group on 8/18/2022. A second youth missed group on 8/25/2022. A third youth
missed group on 9/24/2022. The former clinical director stated she will conduct the make-up
sessions for the three youth. There were no make-up sessions conducted for the youth.
The fifth cohort began on 11/9/2022 began with eleven (11) youth and ended 2/5/2023 with ten
(10) youth. The licensed therapist who began the group left the program. Another licensed
therapist took over the group with no gap in service. One youth was listed on first attendance
sheet, but there was no youth signature. The youth was not listed on any subsequent attendance
sheets.
The ongoing cohort began on 2/28/2023 and was in the process of completing the two individual
sessions.
Pursuant to the facility's Standardized Program Evaluation Protocol procedure, TrueCore utilizes
a Probationary Review Status Process designed for effective management of risk through
individualized coaching, training and verification activities of all new employees during their first
ninety (90) days. The process encourages personal development and recognizes areas of
improvement in an effort to reduce staff turnover. In addition, the program provides positive
incentives to improve employee motivation and dedication and lead to better employee retention.
Through this policy, employees receive credit for years of service, employment referrals, and for
contributing to the positive culture, teamwork environment and for going above and beyond
normal job expectations. In the event of facilitator turnover, the co-facilitator will assume the role
of the facilitator for the group. If a co-facilitator is not present, a new qualified facilitator for the
primary service will take over conducting the service.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation reviewed for the annual review period of March 15, 2022 through March 14,
2023 shows the program conducted eight monthly internal fidelity checks for facilitators of
Cannabis Youth Treatment. The former clinical director and a licensed therapist conducted the
fidelity checks. There were three missing internal fidelity checks during the review period. The
missing fidelity checks were for the licensed therapist for May 2022 and the former clinical director
for June 2022 and July 2022. The program did not provide any fidelity checks for the months of
October 2022, November 2022, December 2022, January 2023, February 2023, and March 2023.
The Regional Compliance Director stated the program did not conduct those fidelity checks. The
program conducts the fidelity monitoring during lessons three through twelve (12) since those
lessons are conducted as a group. Lessons one and two are conducted as individual sessions.
According to the facility's Standardized Program Evaluation Protocol procedure, designated staff
are responsible for conducting fidelity monitoring of each facilitator of a primary service on a
monthly basis. The staff conducting the fidelity monitoring has received formal training by a
qualified trainer in the primary service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 0 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
According to the documentation, there were no corrective actions noted during fidelity monitoring
of Cannabis Youth Treatment facilitators. The program did not provide any fidelity monitorings for
the most recent six months.
Pursuant to the facility's Standardized Program Evaluation Protocol procedure, based on the
results of the monthly fidelity monitoring, corrective action may be needed. There is a process in
place to address corrective actions.
The program can earn 1 point if any corrective action needed is identified, applied, and
demonstrated.
At the time of the review, the program did not meet criteria to earn a score of 1.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program provided an annual evaluation for a licensed therapist and Recreation Therapist.
The Recreation Therapist’s evaluation did not contain specific information related to their delivery
of Cannabis Youth Treatment (CYT). A licensed therapist’s annual evaluation did not contain
specific information related to their delivery of CYT, only the generic statement “Demonstrates
appropriate level of skill when delivering curriculum as supported by the fidelity checks.” The
program also provided a ninety (90)-day evaluation for the same licensed therapist which did
contain specific information about the staff’s skill delivering the primary service. Two additional
ninety (90)-day evaluations were provided for another licensed therapist and the current clinical
director. Neither evaluation contained specific information related to their delivery of CYT.
Pursuant the facility's Standardized Program Evaluation Protocol procedure, an annual evaluation
of performance will be conducted on all facilitators of each primary service. The performance
evaluation is comprised of feedback from the fidelity monitoring tools as well as an assessment of
the facilitator’s skill in delivering the primary service.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 12 weeks for this type of service. Of the 31 youth in the
sample, 100% (31 of 31) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 12 contact hours for this type of service. Of the 31 youth in the
sample, 100% (31 of 31) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 3 youth
Moderate-High = 14 youth
High = 14 youth
Total Youth in Sample = 31 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 45%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 14 youth
Total Youth in Sample = 31 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (31 of 31) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 45% (14 of 31) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 25 25 100%
Totals 90 100 90%
This SPEP report evaluates Cannabis Youth Treatment, an intervention delivered at Jacksonville
Youth Academy.
33 youth completed the program during the review period.
The program scored Medium for Quality of Service Delivery.
The targeted number of weeks for this specific intervention is 12 weeks; the targeted number of
hours for this specific intervention is 12 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 5 services (15 weeks; 45
hours).
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program earned 10 points for Amount of Service: Duration. Of the 31 total youth sampled, all
31 included dosage with end dates in the EBS Module. Of those youth, 31 received at least the
recommended weeks of service. Youth in the sample completed between 13 and 20 weeks of
service, with an average of 14 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 31 total youth
sampled, all 31 included dosage in the EBS Module. Of those youth, 31 received at least the
recommended hours of service. Youth in the sample completed between 12 and 22 hours of
service, with an average of 13 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
RECOMMENDATION(S):
Jacksonville Youth Academy can optimize their SPEP Quality of Service Delivery score by
ensuring facilitator turnover does not impact groups, fidelity monitoring is completed every month
for each facilitator, any corrective action needed is identified, applied, and demonstrated, and
each facilitator’s performance evaluation includes discussion of their skill in delivering this primary
service.
Jacksonville Youth Academy can optimize their SPEP Amount of Service score by ensuring
dosage for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

file_name: FY2022-23 JYA CYT.pdf


state: FL

facility_name: Jacksonville Youth Academy

summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Jacksonville Youth Academy
TrueCore Behavioral Solutions, LLC.
(Contract Provider)
4501 Lannie Road
Jacksonville, Florida 32218
Primary Service: Impact of Crime
SPEP Review Period: March 15, 2021-March 14, 2022
SPEP Review Conducted: April 20, 2023
Florida Department of Juvenile Justice
Report Date: 9/14/2023
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A review of documentation for the annual review period of March 15, 2022 through March 14,
2023 showed the program utilizes Impact of Crime as a primary service. Impact of Crime (IOC)
training is a three-day training currently conducted via Microsoft Teams, provided by a qualified
trainer. The documentation shows there were seven staff trained to facilitate Impact of Crime
during the annual review period- two former directors of case management/transition managers,
the former clinical director, a transition service manager, a former licensed therapist, and two case
managers. Four of the seven facilitated the primary service during the annual review period. A
former director of case management/transition manager, a licensed therapist, a case manager,
and the former clinical director conducted the internal fidelity checks for the annual review period.
Pursuant to the facility's operating procedure, the facilitator for the primary service will have
training from a licensed clinician. This training will be documented in the facilitator's on the job
training plan. The facilitator will receive additional training on the curriculum as needed from the
licensed clinician documented within the clinical supervision. All facilitators will receive training
prior to conducting the service.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the Impact of Crime: Addressing the Harm manual. The manual opens with
an introduction to the Impact of Crime (IOC) curriculum. Then the manual goes over all seven
chapters of the service and provides a script for the facilitator to follow. The manual also has an
appendix and a student guide with all the student exercises. The program makes copies of the
student exercises and passes them out to the youth during each lesson.
The program conducts the primary service on Tuesdays and Thursdays. The program keeps all
SPEP documentation (attendance sheets, fidelity checks, and training certificates) in a binder
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
labeled "IOC SPEP documentation" in the case management office. All youth admitted to the
program are required to participate in IOC. The primary service is a closed group with no more
than twelve (12) youth assigned to one cohort.
According to the facility's Standardized Program Evaluation Protocol procedure, each primary
service will be facilitated in accordance to its manual. The primary service will be conducted as
indicated in the manual/protocol; this includes adherence to the session plan, group size,
population, frequency, needed materials, duration, and interventions. The facilitator of the primary
service will specify the topic covered, interventions utilized, duration of the service, youth present,
and progress for each session on the sign-in sheet.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Impact of Crime was observed on 4/17/2023 via Microsoft Teams. The group began
at 4:06pm and was held in the program’s conference room. There were ten (10) youth, one
facilitator, and one staff present during the group. The facilitator was knowledgeable of the
material and followed the lesson plan. The facilitator made sure the youth knew the concepts from
the primary service. The facilitator had the youth explain the concepts covered during their
discussion. The group ended at 5:08p.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The documentation reviewed for the annual review period of March 15, 2022 through March 14,
2023 shows the program experience facilitator turnover which adversely affected the delivery of
Impact of Crime (IOC). Two directors of case management/transition manager, one transition
service manager, the former clinical director, and a case manager left the program during the
annual review period.
The program ran five cohorts during the annual review period. The first cohort began 2/15/2022
started with ten (10) youth and ended 6/2/2022 with eight youth. Two youth were discharged from
the program. The two youth received individual sessions to ensure they were able to complete the
service.
The second cohort began 5/16/2022 with six youth and ended 9/22/2022 with five youth. One
youth left the program prior to completing the primary service. There were several staff facilitating
the cohort. The cohort began with a transition service manager and former clinical director co-
facilitating the service. The transition service manager left the program and the former clinical
director continued to facilitate the group. The licensed therapist also facilitated and then another
case manager took over for the final two sessions.
The third cohort began 7/5/2022 with ten (10) youth and ended 9/8/2022 with eight youth. Two
youth were discharged from the program, but did receive all sessions due to beginning their own
individual cohort. The two youth’s cohort which began on 7/5/2022 started with two youth and
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
ended with two youth. Two youth started on the last session and worked their way backwards
through the lessons. At the same time working through the lessons with the larger group. The two
youth were transferred from another program. The program wanted to make sure the youth
received all lessons prior to being discharged. The youth were receiving all sessions during their
individual cohort and the full group cohort.
The fourth cohort began 10/3/2022 with ten (10) youth and ended 1/27/2023 with ten (10) youth.
One youth missed group on 10/10/2022 and 10/13/2022. There were no attendance sheets
showing the youth made up the missed sessions. The attendance sheet dated 12/8/2022 had a
note stating a youth missed the group, but the youth signed the attendance sheet. There was no
make up session noted.
The fifth cohort began 12/23/2022 with ten (10) youth and was ongoing at the time of this review
with the same ten (10) youth. One youth was sick on 12/29/2022. The note on the attendance
sheet stated the facilitator provided the youth with the information and would go over it with the
youth once the youth was better. There was no additional documentation showing that happened.
There were two attendance sheets dated 1/25/2023. One listed seven youth and the other listed
three youth. Both attendance sheets documented the group was held at the same time.
According to the facility's Standardized Program Evaluation Protocol procedure, in the event of
facilitator turnover, the co-facilitator will assume the role of the facilitator for the group. If a co-
facilitator is not present, a new qualified facilitator for the service will take over facilitation.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation reviewed for the annual review period of March 15, 2022 through March 14,
2023 shows the program conducted twelve (12) monthly internal fidelity checks for facilitators of
Impact of Crime. A former director of case management/transition manager, a licensed therapist,
a case manager, and the former clinical director conducted the internal fidelity checks for the
annual review period. The program was missing internal fidelity checks for the transition service
manager for the months of March-June 2022. The program was missing a fidelity check for the
former director of case management/transition manager for the month of September 2022. The
former clinical director stated she forgot to conduct the fidelity checks during that time. The
program did not provide any fidelity checks for the months of January 2023 and February 2023.
The fidelity checklists dated 12/28/2022 and 12/29/2022 were both signed by the observer and
showed the date as 1/6/2022.
According to the facility's Standardized Program Evaluation Protocol procedure, designated staff
are responsible for conducting fidelity monitoring of each facilitator of a primary service on a
monthly basis. The staff conducting the fidelity monitoring has received formal training by a
qualified trainer in the primary service. Fidelity monitoring is documented on a fidelity monitoring
tool developed specifically for that primary service by the developer of the manual, or an internal
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
fidelity checklist created for manualized or non-manualized curriculum delivered in the program.
The staff performing the fidelity monitoring of the primary service is verifying the facilitator is
conducting the group in accordance with the manual/protocol.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
According to the documentation provided, there were no corrective actions noted nor taken during
the fidelity monitoring of Impact of Crime.
According to the facility's Standardized Program Evaluation Protocol procedure, based on the
results of the monthly fidelity monitoring, corrective action may be needed. The first stage of
addressing areas identified as needing improvement through fidelity monitoring is by providing
coaching and supervision to the facilitator on that issue by a qualified facilitator in the primary
service. The additional coaching and/or supervision will be documented by the person conducting
the fidelity monitoring in a note and placed in the clinical supervision file. In the event an issue
identified during the primary service is detrimental to the youth’s safety and/or treatment, the
matter will be taken up directly with the program’s Human Resource Manager and be addressed
in accordance with the employee handbook.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program provided an annual evaluations for the former director of case
management/transition manager and one case manager. The evaluations did not contain specific
information related to their delivery of Impact of Crime, only the generic statement “Demonstrates
appropriate level of skill when delivering curriculum as supported by the fidelity checks.” A case
manager’s ninety (90)-day evaluation did not contain specific information related to their delivery
of the primary service.
According to the facility's Standardized Program Evaluation Protocol procedure, an annual
evaluation of performance will be conducted on all the facilitators of each primary service. The
performance evaluation is comprised of feedback from the fidelity monitoring tools as well as an
assessment of the facilitator’s skill in delivering the primary service.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 16 weeks for this type of service. Of the 31 youth in the
sample, 94% (29 of 31) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 31 youth in the
sample, 100% (31 of 31) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 3 youth
Moderate-High = 14 youth
High = 14 youth
Total Youth in Sample = 31 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 45%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 14 youth
Total Youth in Sample = 31 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (31 of 31) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 45% (14 of 31) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 25 25 100%
Totals 73 85 86%
This SPEP report evaluates Impact of Crime, an intervention delivered at Jacksonville Youth
Academy.
33 youth completed the program during the review period.
The program scored Medium for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 31 total youth sampled, all
31 included dosage with end dates in the EBS Module. Of those youth, 29 received at least the
recommended weeks of service. Youth in the sample completed between 15 and 20 weeks of
service, with an average of 17 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 31 total youth
sampled, all 31 included dosage in the EBS Module. Of those youth, 31 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
recommended hours of service. Youth in the sample completed between 29 and 40 hours of
service, with an average of 32 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
RECOMMENDATION(S):
Jacksonville Youth Academy can optimize their SPEP Quality of Service Delivery score by
ensuring facilitator turnover does not impact groups, fidelity monitoring is completed every month
for each facilitator, and each facilitator’s performance evaluation includes discussion of their skill
in delivering this primary service.
Jacksonville Youth Academy can optimize their SPEP Amount of Service score by ensuring
dosage for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

source: DJJ

report_type: SPEP

file_name: FY2022-23 JYA IOC.pdf

program_name_raw: Jacksonville Youth Academy - TrueCore

program_status_note: Closed

service_type: Impact of Crime


summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Jacksonville Youth Academy
TrueCore Behavioral Solutions, LLC
(Contract Provider)
4501 Lannie Road
Jacksonville, Florida 32218
Primary Service: Male Healthy Relationships
SPEP Review Period: March 15, 2022-March 14, 2023
SPEP Review Conducted: April 20, 2023
Florida Department of Juvenile Justice
Report Date: 9/14/2023
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Male Healthy Relationships.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 6 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 0 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A review of documentation for the annual review period of March 15, 2022 through March 14,
2023 showed the program utilizes the Male Healthy Relationship bundle as a primary service. The
Male Healthy Relationships bundle consists of two services: The Teen Relationships Workbook
and Young Men’s Work. The program has a written training protocol for both Young Men’s Work
and Teen Relationships. The Male Healthy Relationships bundle training is an in-service training
conducted by the director of clinical services, clinical director, or assistant clinical director. The
documentation shows there were four staff trained in the primary service- two licensed therapists,
the recreation therapist, and the former clinical director. The program did not provide proof of
training for two other licensed therapists and the current clinical director. Three of the four trained
staff conducted groups during the annual review period. The three staff who do not have proof of
training also conducted groups during the annual review period. The current clinical director,
former clinical director, and the recreation therapist conducted the monthly internal fidelity checks
during the annual review period.
According to the facility's operating procedure, the facilitator for the primary service will have
training from a licensed clinician. This training will be documented in the facilitator's on the job
training plan. The facilitator will receive additional training on the curriculum as needed from the
licensed clinician documented within the clinical supervision. All facilitators will receive training
prior to conducting the service.
The program can earn 1 point if all facilitators receive formal training prior to delivering services to
youth.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilized the The Teen Relationship Workbook and the Young Men's Work manual.
The program maintains the attendance sheets, fidelity checklists, and training certificates in the
SPEP binder labeled "Male Healthy Relationships bundle" in the clinical director’s office. The
Teen Relationships Workbook is broken down into six sections. The first section is evaluating your
relationship. Section two is understanding abuse. Section three is on social influences. Section
four is building healthy relationships. Section five is making good decisions. Section six is the
review. There are sixty-eight (68) reproducible worksheets given to the youth during the sessions.
Each lesson has information on how the facilitator should present the material. There are no
separate student workbooks.
The Young Men's Work manual is broken down into twenty-six (26) sessions. Each session has at
least one student exercise in them. There is a total of thirty-eight (38) exercises during the group.
The manual provides information for the facilitator to conduct each session.
Young Men's Work and Teen Relationships are delivered one time per week and are run as open
groups throughout the youth's length of stay. The youth will participate in the components until a
minimum of forty (40) combined contact hours are reached. The program runs the service on
Thursdays and Mondays.
According to the facility's Standardized Program Evaluation Protocol procedure, each primary
service will be facilitated in accordance to its manual. The primary service will be conducted as
indicated in the manual/protocol; this includes adherence to the session plan, group size,
population, frequency, needed materials, duration, and interventions. The facilitator of the primary
service will specify the topic covered, interventions utilized, duration of the service, youth present,
and progress for each session on the sign-in sheet.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Male Healthy Relationships-Young Men’s Work was observed on 4/11/2023 via
Microsoft Teams. The group began at 2:06pm and was held in one of the program’s classrooms.
There were eight youth, one facilitator, and one staff present during the group. The facilitator was
knowledgeable of the material and followed the lesson plan. The lesson completed was Creating
Relationships 1. The youth arrived to the group late due to a meeting that ran over. The youth
were engaging in good discussions about the session topic. The group ended at 3:10p.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The documentation reviewed for the annual review period of March 15, 2022 through March 14,
2023 shows the program experienced facilitator turnover which adversely affected the delivery of
Male Healthy Relationships. The former clinical director and a licensed therapist left the program.
Both were facilitating the group when they ended employment. The program did not have any
gaps in service. The youth were transitioned to another therapist to ensure continued delivery of
the service. Each service is an open group run year-round.
According to the facility's Standardized Program Evaluation Protocol procedure, in the event of
facilitator turnover, the co-facilitator will assume the role of the facilitator for the group. If a co-
facilitator is not present, a new qualified facilitator for the primary service will take over conducting
the service.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation reviewed for the annual review period of March 15, 2022 through March 14,
2023 shows the program conducted twenty-one (21) monthly internal fidelity checks for The Teen
Relationships Workbook and thirteen (13) monthly internal fidelity checks for Young Men’s Work.
The current clinical director, former clinical director, and the recreation therapist conducted the
monthly internal fidelity checks. Teen Relationships: The program was missing an internal fidelity
monitoring for the months of May 2022, October 2022, and December 2022 for a licensed
therapist; and February 2023 for the current clinical director. Young Men’s Work: The program
was missing an internal fidelity monitoring for the months of August - December 2022 for a
licensed therapist.
The two fidelity checklists for 4/7/2022 for Teen Relationships show the same monitor (former
clinical director) ending one fidelity check at 3:43p, but also beginning another fidelity check at
3:24p. The former clinical director stated she left the group early so she could conduct another
fidelity check on a different facilitator. The technical assistance specialist stated the fidelity
monitor must be in the group for the full session. The monitor cannot leave one group early to
begin another fidelity check on a different facilitator.
According to the facility's Standardized Program Evaluation Protocol procedure, designated staff
are responsible for conducting fidelity monitoring of each facilitator of a primary service on a
monthly basis. The staff conducting the fidelity monitoring has received formal training by a
qualified trainer in the primary service. Fidelity monitoring is documented on a fidelity monitoring
tool developed specifically for that primary service by the developer of the manual, or an internal
fidelity checklist created for manualized or non-manualized curriculum delivered in the program.
The staff performing the fidelity monitoring of the primary service is verifying the facilitator is
conducting the group in accordance with the manual/protocol. This includes verifying the facilitator
adheres to the session plan, group size, population, frequency, needed materials, duration, and
interventions.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
According to the documentation provided, there were no corrective actions noted nor taken during
the fidelity monitoring of Male Healthy Relationships.
According to the facility's Standardized Program Evaluation Protocol procedure, based on the
results of the monthly fidelity monitoring, corrective action may be needed. The first stage of
addressing areas identified as needing improvement through fidelity monitoring is by providing
coaching and supervision to the facilitator on that issue by a qualified facilitator in the primary
service. The additional coaching and/or supervision will be documented by the person conducting
the fidelity monitoring in a note and placed in the clinical supervision file. In the event an issue
identified during the primary service is detrimental to the youth’s safety and/or treatment, the
matter will be taken up directly with the program’s Human Resource Manager and be addressed
in accordance with the employee handbook.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program provided an annual evaluation for a licensed therapist and Recreation Therapist.
Neither evaluation contained specific information related to their delivery of Male Healthy
Relationships, only the generic statement “Demonstrates appropriate level of skill when delivering
curriculum as supported by the fidelity checks.” The program also provided a ninety (90)-day
evaluation for the same licensed therapist which did contain specific information about the staff’s
skill delivering the primary service. Two additional ninety (90)-day evaluations were provided for
another licensed therapist and the current clinical director, neither evaluation contained specific
information related to their delivery of the primary service.
According to the facility's Standardized Program Evaluation Protocol procedure, an annual
evaluation of performance will be conducted on all the facilitators of each primary service. The
performance evaluation is comprised of feedback from the fidelity monitoring tools as well as an
assessment of the facilitator’s skill in delivering the primary service.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 24 weeks for this type of service. Of the 31 youth in the
sample, 100% (31 of 31) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 40 contact hours for this type of service. Of the 31 youth in the
sample, 97% (30 of 31) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 3 youth
Moderate-High = 14 youth
High = 14 youth
Total Youth in Sample = 31 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 45%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 14 youth
Total Youth in Sample = 31 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (31 of 31) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 45% (14 of 31) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 25 25 100%
Totals 83 95 87%
This SPEP report evaluates Male Healthy Relationships, an intervention delivered at Jacksonville
Youth Academy.
33 youth completed the program during the review period. Of those youth, 31 were included in the
sample.
The program scored Medium for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 31 total youth sampled, all
31 included dosage with end dates in the EBS Module. Of those youth, 31 received at least the
recommended weeks of service. Youth in the sample completed between 25 and 44 weeks of
service, with an average of 33 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program earned 8 points for Amount of Service: Contact Hours. Of the 31 total youth
sampled, all 31 included dosage in the EBS Module. Of those youth, 30 received at least the
recommended hours of service. Youth in the sample completed between 35 and 84 hours of
service, with an average of 61 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
RECOMMENDATION(S):
Jacksonville Youth Academy can optimize their SPEP Quality of Service Delivery score by
ensuring all facilitators receive formal training prior to delivering services to youth, facilitator
turnover does not impact groups, fidelity monitoring is completed every month for each facilitator,
and each facilitator’s performance evaluation includes discussion of their skill in delivering this
primary service.
Jacksonville Youth Academy can optimize their SPEP Amount of Service score by ensuring
dosage for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

file_name: FY2022-23 JYA MHR.pdf

service_type: Male Healthy Relationships