state: FL

facility_name: Miami Youth Academy

summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Miami Youth Academy
TrueCore Behavioral Solutions, LLC
(Contract Provider)
10855 Southwest 84th Street
Miami , Florida 33173
Primary Service: Cannabis Youth Treatment
SPEP Review Period: November 1, 2022-October 31, 2023
SPEP Review Conducted: November 15, 2023
Florida Department of Juvenile Justice
Report Date: 6/26/2024
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Cannabis Youth Treatment.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The Standardized Program Evaluation Protocol (SPEP) review encompasses twelve (12) months
of records. The Cannabis Youth Treatment program consists of two individual sessions of
Motivational Enhancement Therapy (MET) and three weekly group sessions of Cognitive-
Behavioral Therapy (CBT; MET/CBT5). The MET sessions focus on factors which motivate
participants who abuse substances to change. While in the CBT sessions, participants learn skills
to cope with problems and meet needs in ways which do not involve turning to marijuana or
alcohol, increase their social support network and non-drug activities, and how to avoid and cope
with relapses. The MET/CBT12 model comprises the complete five-session model combined with
seven supplemental cognitive behavioral sessions covering additional coping skills.
The program provided documentation of one therapist and one director of clinical services being
trained in the primary service. The therapist facilitates the group and director provides internal
fidelity monitoring. A former director of clinical services, also trained in the service, works in an "as
needed" basis to provide internal fidelity monitoring.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the commercially-produced Cannabis Youth Treatment (CYT) curriculum to
facilitate the service. The curriculum provides for large group and individual discussions topics to
further explore the youth's perception of substance abuse issues. The curriculum provides
facilitators guide to provide enrichment activities if appropriate. CYT is delivered as a closed
group after the initial individual sessions, meaning once a cohort has began, no new youth may
be added.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Cannabis Youth Treatment was observed during the review period. The group
consisted of nine youth, one facilitator, and one direct care staff. The group was held in the day
room of the program. The facilitator led discussion focused on the way substances can influence
decision making skills when in the community. The youth were engaged in the discussion, and no
behavioral issues were observed.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
According to documentation provided by the program and an informal conversation with the
clinical director, no turnover in facilitators occurred during the review period which affected service
delivery of Cannabis Youth Treatment (CYT). The former clinical director retired and the therapist
was promoted to new clinical director. A new therapist was brought on board between the cohorts,
so no break in service occurred.
The program provided documentation of four cohorts of the CYT being facilitated during the
review period. The first cohort began November 23, 2022, with nine youth. The cohort was
completed on January 25, 2023, with the same nine youth. A second cohort began on April 5,
2023, with nine youth and was completed on May 24, 2023, with nine youth. A third cohort began
on August 23, 2023, and was completed on October 25, 2023, with eight youth. One youth
successfully completed the program prior to finishing the service. A fourth cohort began on
October 17, 2023, with nine youth and was ongoing at the time of the review.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
According to documentation provided by the program, monthly internal fidelity monitoring took
place monthly for facilitators of Cannabis Youth Treatment. The groups were observed by a
person trained in the primary service. There was no missing internal fidelity monitoring.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process in place to address corrective actions identified during monthly
internal fidelity monitoring. A review of documentation confirmed no corrective action was initiated
during the review period. Facilitators of Cannabis Youth Treatment were found to be delivering the
service in an acceptable manner.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Copies of the annual employee evaluations were provided as part of the review. Each employee
who facilitated Cannabis Youth Treatment during the review period, and was eligible for an annual
review, was rated in their effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 12 weeks for this type of service. Of the 26 youth in the
sample, 4% (1 of 26) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 12 contact hours for this type of service. Of the 26 youth in the
sample, 100% (26 of 26) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 88%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 5 youth
Moderate-High = 8 youth
High = 10 youth
Total Youth in Sample = 26 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 38%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 10 youth
Total Youth in Sample = 26 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 88% (23 of 26) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 38% (10 of 26) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 23 25 92%
Totals 88 100 88%
This SPEP report evaluates Cannabis Youth Treatment, an intervention delivered at Miami Youth
Academy.
27 youth completed the program during the review period. Of those youth, 26 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 26 total youth sampled, all
26 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 1
received at least the recommended weeks of service. Youth in the sample completed between 9
and 11 weeks of service, with an average of 11 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 10 points for Amount of Service: Contact Hours. Of the 26 total youth
sampled, all 26 included dosage in the EBS Module. Of those youth with correct dosage, 26
received at least the recommended hours of service. Youth in the sample completed between 12
and 12 hours of service, with an average of 12 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
The targeted number of weeks for this specific intervention is 12 weeks; the targeted number of
hours for this specific intervention is 12 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 5 services (15 weeks; 45
hours).
RECOMMENDATION(S):
Miami Youth Academy can maintain their SPEP Quality of Service Delivery score by continuing
the practices in place at the time of this review.
Miami Youth Academy can optimize their SPEP Amount of Service score by ensuring dosage for
all youth is recorded accurately in EBS and by ensuring youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

source: DJJ

report_type: SPEP

file_name: FY2023-24 MYA CYT.pdf

program_name_raw: Miami Youth Academy - TrueCore

program_status_note: Closed

service_type: Cannabis Youth Treatment


summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Miami Youth Academy
Truecore Behavioral Solutions, LLC
(Contract Provider)
10855 Southwest 84th Street
Miami , Florida 33173
Primary Service: Impact of Crime
SPEP Review Period: November 1, 2022-October 31, 2023
SPEP Review Conducted: November 15, 2023
Florida Department of Juvenile Justice
Report Date: 6/26/2024
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The Standardized Program Evaluation Protocol (SPEP) review encompasses twelve (12) months
of records. Impact of Crime is based on the belief that crime is more than just a legal definition,
but rather crime affects the victim, the offender, their families, and the community. The harm
caused brings with it the moral responsibility to all involved. The focus of Impact of Crime is to
help offenders understand the harm they have created, and then assisting them in taking personal
accountability for their actions. Together, the victim, offender, and the community arrive at a viable
solution to the repair the harm caused and the offender. Once the harm is repaired, the offender
subsequently reintegrates back into the community. Facilitators are encouraged to bring in victim
impact speakers throughout the course of the curriculum.
At the time of the review two case managers and the director of case management were trained in
the primary service. The case managers facilitate group and the director of case management
provides internal fidelity monitoring.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the curriculum Impact of Crime: Addressing the Harm to Victims and the
Community. It is highly detailed providing a facilitator's manual, and student workbook. The
facilitator's manual provides lessons plans, classroom management ideas, and tips for engaging
youth discussions. The student manual provides reproducible worksheets to help facilitate group
discussions, and review concepts learned during the group sessions. Homework materials are
also utilized in the student manual to reinforce learned concepts.
Impact of Crime is facilitated as a closed group, meaning once a cohort begins, no new youth can
be added to the group. All youth are to receive this service during their stay at the program.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Impact of Crime was observed during the review period. The group took place in a
classroom with one facilitator, two direct care staff and nine youth present. The facilitator did not
read a script word for word, but did generalize the information, and added their own words to
provide meaning to covered concepts. The group was discussing the ripple effect in which the
actions of one person affect other people's lives indirectly. The youth were generally engaged and
did participate in the discussion. No behavioral issues were observed during the group.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There were no gaps in service delivery or facilitator turnover for Impact of Crime (IOC) during the
past year. A review of documentation and an informal interview with both the director of case
management no turnover of staff lead to break in service delivery.
Three cohorts of IOC were conducted during the review period. The first cohort began before the
review period with nine youth. It was completed on May 2, 2023, with eight youth. One youth was
detained for new charges and removed from the cohort.
A second cohort began on May 9, 2023, with eleven (11) youth and was completed on August 29,
2023, with ten (10) youth. One youth was taken into custody on new charges and did not return to
the cohort.
A third cohort began on September 5, 2023, with eight youth and was on going at the time of the
review with seven youth. One youth had refused to attend group three times and was removed
from the cohort.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of twelve (12) months of documentation confirmed the program has conducted monthly
internal fidelity monitoring each month for facilitators of Impact of Crime. Additionally, the internal
fidelity monitoring was completed by staff member trained in the service. The facilitator also
signed the form saying feedback was given to the facilitator and understood by the facilitator.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process in place to address corrective actions identified during monthly
internal fidelity monitoring. A review of documentation confirmed no corrective action was initated
during the review period for facilitators of Impact of Crime. The facilitators were found to be
delivering the service in an acceptable manner.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Copies of the annual employee evaluations were provided as part of the review. Each employee
who facilitated Impact of Crime was rated in their effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 16 weeks for this type of service. Of the 26 youth in the
sample, 100% (26 of 26) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 26 youth in the
sample, 100% (26 of 26) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 88%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 5 youth
Moderate-High = 8 youth
High = 10 youth
Total Youth in Sample = 26 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 38%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 10 youth
Total Youth in Sample = 26 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 88% (23 of 26) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 38% (10 of 26) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 23 25 92%
Totals 83 85 98%
This SPEP report evaluates Impact of Crime, an intervention delivered at Miami Youth Academy.
27 youth completed the program during the review period. Of those youth, 26 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 26 total youth sampled, all
26 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 26
received at least the recommended weeks of service. Youth in the sample completed between 16
and 17 weeks of service, with an average of 17 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 26 total youth
sampled, all 26 included dosage in the EBS Module. Of those youth with correct dosage, 26
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
received at least the recommended hours of service. Youth in the sample completed between 32
and 34 hours of service, with an average of 32 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
RECOMMENDATION(S):
Miami Youth Academy can maintain their SPEP Quality of Service Delivery score by continuing
the practices in place at the time of this review.
Miami Youth Academy can maintain their SPEP Amount of Service score by continuing to ensure
dosage for all youth is recorded accurately in EBS and by continuing to ensure youth receive the
full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2023-24 MYA IOC.pdf

service_type: Impact of Crime


summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Miami Youth Academy
Truecore Behavioral Solutions, LLC
(Contract Provider)
10855 Southwest 84th Street
Miami , Florida 33173
Primary Service: Thinking for a Change
SPEP Review Period: November 1, 2022-October 31, 2023
SPEP Review Conducted: November 15, 2023
Florida Department of Juvenile Justice
Report Date: 6/26/2024
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
1. Primary Basic Score: 35
Service and Points
Supplemental POS: 35 Points
Service Types POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The Standardized Program Evaluation Protocol (SPEP) review encompasses twelve months of
SPEP records. T4C utilizes a problem-solving program with both cognitive restructuring, and
social skills interventions bringing the identification of thinking, beliefs, attitudes, and values to
behavior. Effort is focused on helping youth become aware of their thoughts, feelings, and beliefs
in particular circumstances.
The program provided documentation of three staff members being trained in the primary service.
The director of case management, recreational therapist and youth transitional services manager
are trained in the primary service.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
A review of documents and records revealed the evidence-based curriculum Thinking for a
Change. The program utilizes a highly detailed, commercially produced manual that provides
lessons plans, facilitator notes, ideas to facilitate the curriculum, and long-term planning
schedules. The program is currently conducting the groups on Monday, Wednesday, and Friday.
Groups are conducted as closed groups, meaning once a group commenced, no new youth can
be added to the group.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
A session of the primary service was observed during an earlier quarterly review. The group
session took place in a classroom with ten youth in attendance. The facilitator modelled the role
play, with student volunteer. Youth were broken into pairs, and was able to practice the new skill.
There were no behavior issues observed during the group. The facilitator did not read the script
verbatim but did have the facilitators guide open on the desk to ensure all points of the lesson
were covered.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
Review of documentation, and informal conversation with director of case management, no
facilitator turnover lead to a break in service delivery.
The program provided documentation of four cohorts of the primary service being facilitated
during the review period. The first cohort began before the review period with ten youth. It was
completed on January 4, 2023 with ten (10) youth. A second cohort began January 11, 2023, with
ten (10) youth and was completed on May 3, 2023, with nine youth. One youth was
unsuccessfully removed from the program. A third cohort began on May 8, 2023, with ten (10)
youth, and was completed on August 28, 2023, with ten youth. A fourth cohort began on
September 6, 2023, and was ongoing at the time of the review.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
According to documentation provided by the program, internal fidelity monitoring took place on a
monthly basis. The fidelity monitoring was completed by another trained facilitator in the primary
service.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
A review of documentation revealed that no corrective action was initated during the review
period. The facilitators were found to be delivering the primary service in an acceptable manner.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A copy of the annual employee evaluations were provided as part of the review. Each employee
who facilitated the primary service were rated in their effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 15 weeks for this type of service. Of the 26 youth in the
sample, 100% (26 of 26) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 45 contact hours for this type of service. Of the 26 youth in the
sample, 100% (26 of 26) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 88%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 5 youth
Moderate-High = 8 youth
High = 10 youth
Total Youth in Sample = 26 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 38%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 10 youth
Total Youth in Sample = 26 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 88% (23 of 26) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 38% (10 of 26) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 23 25 92%
Totals 98 100 98%
This SPEP report evaluates Thinking for a Change, an intervention delivered at Miami Youth
Academy.
27 youth completed the program during the review period. Of those youth, 26 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 26 total youth sampled, all
26 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 26
received at least the recommended weeks of service. Youth in the sample completed between 16
and 28 weeks of service, with an average of 17 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 10 points for Amount of Service: Contact Hours. Of the 26 total youth
sampled, all 26 included dosage in the EBS Module. Of those youth with correct dosage, 26
received at least the recommended hours of service. Youth in the sample completed between 45
and 78 hours of service, with an average of 46 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
RECOMMENDATION(S):
Miami Youth Academy can maintain their SPEP Quality of Service Delivery score by continuing
the practices in place at the time of this review.
Miami Youth Academy can maintain their SPEP Amount of Service score by continuing to ensure
dosage for all youth is recorded accurately in EBS and by continuing to ensure youth receive the
full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2023-24 MYA T4C.pdf

service_type: Thinking for a Change