summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
South Florida Youth Academy
True Core Behavioral Solutions, LLC
(Contract Provider)
3300 NW 27th Avenue
Miami, Florida 33142
Primary Service: Skillstreaming the Adolescent
SPEP Review Period: July 1, 2023-June 30, 2024
SPEP Review Conducted: July 30-31, 2024
Florida Department of Juvenile Justice
Report Date: 9/24/2025
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Skillstreaming the Adolescent.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A review of the documentation for the annual review period of July 1, 2023 through June 30, 2024
shows the program utilizes Skillstreaming the Adolescent as a primary service. Skillstreaming
training is provided as an in-service training conducted by the clinical director or another trained
clinician. Documentation provided by the program shows there were seven staff members trained
in the primary service during this annual review period including two clinical directors, one
assistant clinical director, and four therapists.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the Skillstreaming the Adolescent curriculum for all youth assigned to the
program. There are no student manuals for this service. The curriculum includes a facilitator’s
manual and skill cards for the youth. This service is provided once a week and groups are run
year-round with no more than ten (10) youth per group. Skillstreaming is an open group.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
An observation of the primary service took place on site on April 24, 2024. The group began at
3:00pm. A copy of the lesson plan was not provided to the technical assistance specialist for
review. There were six youth, one facilitator, and two direct care staff present for this group. The
facilitator had a copy of the facilitator’s manual available during this session. The group began
with a check-in/ice breaker, a discussion of the group rules, a review of the previous lesson, and a
homework review. The group engaged in a discussion about problems that occur in life and how
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
to identify the actions or decisions that led to the problem. No behavior management issues were
observed. The group ended at 4:00pm.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
Documentation reviewed showed the program had facilitator turnover during this annual review
period. One clinical director and one therapist left the program during this annual review period.
This turnover did not adversely affect the delivery of the service, as another therapist took over
the affected Skillstreaming group at the time of the therapist’s departure. The former clinical
director was not facilitating a Skillstreaming group at the time of her departure. No gap in service
occurred during this annual review period due to a lack of trained facilitators.
An informal interview with the clinical director confirmed the program will replace one facilitator
with another qualified facilitator should there be any staff turnover in the middle of a cohort.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of the documentation for the annual review period from July 1, 2023 through June 30,
2024 showed there were eighteen (18) out of the required twenty-two (22) internal fidelity
monitoring checks completed for the facilitators of Skillstreaming. Any facilitator that is trained in
Skillstreaming can conduct the internal fidelity monitoring. During this review period, internal
fidelity monitoring checks were conducted by two clinical directors who were trained in the
service.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
A review of the documentation shows there was a need for corrective actions identified for one
facilitator of Impact of Crime during the annual review period of July 1, 2023 through June 30,
2024. During an informal interview with the director of case management, it was explained that if a
need for corrective action is found, the facilitator will receive progressive discipline to include
coaching, retraining, and formal write-up(s). Informal coaching was provided for one therapist in
January 2024 to address the need to be prepared prior to group with all necessary materials,
redirection of sidebar conversations, and a reminder to emphasize the individual skill steps during
the group lesson. All subsequent fidelity monitoring forms reflect improvement and praise for this
facilitator’s skills.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
At the time of the review, the program did meet criteria to earn a score of 1.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A copy of one annual employee evaluation for a Skillstreaming facilitator was provided to the
technical assistance specialist for the annual review period of July 1, 2023 through June 30, 2024.
This employee evaluation did not provide specific details about the facilitator’s skill in the delivery
of this service.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates the target duration of 16 weeks for this type of service. Of the 15 youth in the
sample, 27% (4 of 15) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 24 contact hours for this type of service. Of the 15 youth in the
sample, 67% (10 of 15) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 15 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 60%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 73%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 2 youth
Moderate-High = 4 youth
High = 5 youth
Total Youth in Sample = 15 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 33%
High Score: 10 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 77%
Table 2
High = 5 youth
Total Youth in Sample = 15 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 73% (11 of 15) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 33% (5 of 15) youth scored High-Risk to reoffend, for a score of 10 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 2 10 20%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 15 25 60%
Totals 63 85 74%
This SPEP report evaluates Skillstreaming the Adolescent, an intervention delivered at South
Florida Youth Academy.
17 youth completed the program during the review period. Of those youth, 15 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 2 points for Amount of Service: Duration. Of the 15 total youth sampled, only
4 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 4
received at least the recommended weeks of service. Youth in the sample completed between 18
and 38 weeks of service, with an average of 28 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 6 points for Amount of Service: Contact Hours. Of the 15 total youth
sampled, only 15 included dosage in the EBS Module. Of those youth with correct dosage, 10
received at least the recommended hours of service. Youth in the sample completed between 9
and 34 hours of service, with an average of 24 hours.
The program was awarded 15 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 86%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program
RECOMMENDATION(S):
South Florida Youth Academy can optimize their SPEP Quality of Service Delivery score by
ensuring fidelity monitoring is completed every month for each facilitator and each facilitator’s
performance evaluation includes discussion of their skill in delivering this primary service.
South Florida Youth Academy can optimize their SPEP Amount of Service score by ensuring
dosage for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

report_type: SPEP

file_name: FY2023-24 South Florida Youth Academy Skillstreaming.pdf

program_name_raw: South Florida Youth Academy - TrueCore

program_status_note: Closed

service_type: Skillstreaming the Adolescent


state: FL

facility_name: South Florida Youth Academy

summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
South Florida Youth Academy
TrueCore Behavioral Solutions, LLC
(Contract Provider)
3300 NW 27th Avenue
Miami, Florida 33142
Primary Service: Cannabis Youth Treatment
SPEP Review Period: July 1, 2023-June 30, 2024
SPEP Review Conducted: July 30-31, 2024
Florida Department of Juvenile Justice
Report Date: 9/26/2025
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Cannabis Youth Treatment.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A review of the documentation for the annual review period of July 1, 2023 through June 30, 2024
shows the program uses Cannabis Youth Treatment as a primary service. Cannabis Youth
Treatment training is provided as an in-service training conducted by the clinical director or
another trained clinician. Documentation provided by the program shows there were six staff
trained in the primary service during this annual review period including two clinical directors, one
assistant clinical director, and three therapists.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the Cannabis Youth Treatment Series Volumes One and Two. The first two
lessons of Cannabis Youth Treatment are conducted as individual sessions, each one hour in
length. Lessons three through twelve (12) are conducted in a group setting. All SAOS youth
admitted into the program are required to receive Cannabis Youth Treatment. There are no
student manuals for this service. The facilitator will make copies of the necessary worksheets to
distribute to the youth during group. This service is provided once a week. Cannabis Youth
Treatment is a closed group.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
An observation of the primary service took place on site on April 25, 2024. The group began at
4:00pm. A copy of the lesson plan was not provided to the technical assistance specialist for
review. There were five youth, one facilitator, one clinical director, and two direct care staff
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
present for group. The facilitator had a copy of the facilitator’s manual available during this
session. The group began with a check-in/ice breaker, a discussion of the group rules, and a brief
overview of the topics that would be covered during this cohort. The topic for this group was
Lesson Three—Marijuana Refusal Skills. The group engaged in a discussion about ways to say
no when offered marijuana in the future. The youth were encouraged to engage in roleplays to
allow the youth to practice saying no to a variety of individuals who might encourage the youth to
use marijuana upon their release. No behavior management issues were observed. The group
ended at 5:00pm.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
Facilitator turnover occurred during this annual review period. This turnover did not adversely
affect the delivery of the service, as the staff member was not facilitating Cannabis Youth
Treatment at the time of their departure. No gap in service occurred during this annual review
period due to a lack of trained facilitators. The gaps in this service were due to a lack of SAOS
youth admissions into the program during the annual review period.
An informal interview with the clinical director confirmed the program will replace one facilitator
with another qualified facilitator should there be any staff turnover in the middle of a cohort.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of the documentation for the annual review period from July 1, 2023 through June 30,
2024 showed there were five out of the required six internal fidelity monitoring checks completed
for the facilitators of Cannabis Youth Treatment. One fidelity monitoring form was missing for June
2024. Any facilitator that is trained in Cannabis Youth Treatment can conduct the internal fidelity
monitoring. During this review period, internal fidelity monitoring checks were conducted by the
clinical director.There were two cohorts completed during this annual review period.
One cohort began on September 20, 2023 with six youth and ended on November 15, 2023 with
six youth.
One cohort began on April 25, 2024 with five youth and ended on June 14, 2024 with four youth.
One youth was dropped from the cohort due to being released from the program.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
A review of the documentation shows there was a need for corrective action identified for the
facilitator of Cannabis Youth Treatment. Documentation shows one cohort, initiated on April 25,
2024, ended on Lesson Ten with no explanation why Lesson Eleven and Lesson Twelve were
omitted. The assistant clinical director served as the facilitator for this cohort. During an informal
interview with the assistant clinical director, he stated he was trained to omit the final two lessons
for this service. The regional director of clinical services was interviewed to determine if the
training information provided by the assistant clinical director was accurate. The regional director
of clinical services stated the information provided was incorrect and the cohort should have been
ended after Lesson Twelve instead of Lesson Ten. He advised this situation would be addressed
with the assistant clinical director and additional training will be provided to prevent this from
happening during future cohorts.
During an informal interview with the clinical director, it was explained that if a need for corrective
action is found, the facilitator will receive progressive discipline to include coaching, retraining,
and formal write-up(s).
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A copy of one annual employee evaluation for a Cannabis Youth Treatment facilitator was
provided to the technical assistance specialist for the annual review period of July 1, 2023 through
June 30, 2024. This employee evaluation did not provide specific details about the facilitator’s
skill in the delivery of this service.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 12 weeks for this type of service. Of the 6 youth in the
sample, 0% (0 of 6) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 12 contact hours for this type of service. Of the 6 youth in the
sample, 100% (6 of 6) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 8 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 32%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 67%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 0 youth
Moderate-High = 3 youth
High = 1 youth
Total Youth in Sample = 6 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 17%
High Score: 3 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 23%
Table 2
High = 1 youth
Total Youth in Sample = 6 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 67% (4 of 6) youth scored Moderate to High-Risk to reoffend, for a score of
5 points.
Of the SPEP sample, 17% (1 of 6) youth scored High-Risk to reoffend, for a score of 3 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 8 25 32%
Totals 73 100 73%
This SPEP report evaluates Cannabis Youth Treatment, an intervention delivered at South Florida
Youth Academy.
17 youth completed the program during the review period. Of those youth, 6 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 6 total youth sampled, only
6 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 0
received at least the recommended weeks of service. Youth in the sample completed between 11
and 11 weeks of service, with an average of 11 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 10 points for Amount of Service: Contact Hours. Of the 6 total youth
sampled, only 6 included dosage in the EBS Module. Of those youth with correct dosage, 6
received at least the recommended hours of service. Youth in the sample completed between 12
and 12 hours of service, with an average of 12 hours.
The targeted number of weeks for this specific intervention is 12 weeks; the targeted number of
hours for this specific intervention is 12 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 5 services (15 weeks; 45
hours).
The program was awarded 8 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 90%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
South Florida Youth Academy can optimize their SPEP Quality of Service Delivery score by
ensuring fidelity monitoring is completed every month for each facilitator and each facilitator’s
performance evaluation includes discussion of their skill in delivering this primary service.
South Florida Youth Academy can optimize their SPEP Amount of Service score by ensuring
dosage for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

source: DJJ

file_name: FY2023-24 South Florida Youth Academy CYT.pdf

service_type: Cannabis Youth Treatment


summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
South Florida Youth Academy
True Core Behavioral Solutions, LLC
(Contract Provider)
3300 NW 27th Avenue
Miami, Florida 33142
Primary Service: Impact of Crime
SPEP Review Period: July 1, 2023-June 30, 2024
SPEP Review Conducted: July 30-31, 2024
Florida Department of Juvenile Justice
Report Date: 9/25/2025
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A review of the documentation for the annual review period of July 1, 2023 through June 30, 2024
shows the program utilizes Impact of Crime as a primary service. Facilitator training consists of a
three-day, twenty-four (24) hour training provided by the Department of Juvenile Justice. This
training is available to new facilitators both in-person and via Microsoft Teams. The
documentation shows there were four staff members trained in the primary service during this
review period: one director of case management and three case managers. The director of case
management and two case managers facilitated Impact of Crime during the annual review period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the manual titled Impact of Crime: Addressing the Harm. The manual includes
seven chapters and provides a script for the facilitator to follow. The curriculum includes a student
guide with exercises for the participants to complete to reinforce the material presented during
group sessions. The program does not assign a student guide to each individual participant,
rather the facilitator makes copies of the exercises and provides one to each youth during the
lesson. The program maintains all SPEP documentation (attendance sheets and internal fidelity
checks) in a separate binder labeled Impact of Crime. All youth admitted to the program are
required to participate in this primary service. Impact of Crime is a closed group with no more than
twelve (12) youth assigned to a cohort.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
A session of Impact of Crime was observed on-site on April 24, 2024. The facilitator prepared the
classroom prior to the arrival of the youth by arranging the desks and setting out the materials the
youth would be utilizing during this group session. The group began at 6:00pm. A copy of the
lesson plan was not provided to the technical assistance specialist for review. There were six
youth, one facilitator, and two direct care staff in attendance. The group began with an ice breaker
activity and a brief review of the previous lesson before proceeding to the lesson of the day. The
facilitator did have a copy of the facilitator’s manual available during this session. No behavior
management issues were observed. The group ended at 7:02pm.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program has not experienced facilitator turnover for staff trained in the Impact of Crime
curriculum. There were no gaps in service during the annual review period caused by a lack of
trained facilitators. There were three cohorts were completed and one cohort ongoing at the time
of this annual review.
The first cohort began on August 21, 2023 with twelve (12) youth and ended on December 13,
2023 with eight youth. Four youth were dropped from the cohort due to being arrested.
The second cohort began on October 25, 2023 with eight youth and ended on February 28, 2024
with seven youth. One youth was dropped from the cohort due to his release from the program.
The third cohort began on December 27, 2023 with seven youth and ended on May 1, 2024 with
seven youth.
The fourth cohort began on March 11, 2024 with seven youth and was on going at the end of the
annual review period with five youth. One youth was dropped from the cohort for repeated
refusals to attend and one youth was dropped from the cohort upon his release from the program.
An informal interview with the director of case management confirmed the program will replace
one facilitator with another qualified facilitator should there be any staff turnover in the middle of a
cohort.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
There were nineteen (19) out of the required nineteen (19) internal fidelity monitoring checks
completed for the facilitators of Impact of Crime. Any facilitator that is trained in Impact of Crime
can conduct the internal fidelity monitoring. During this review period, internal fidelity monitoring
checks were conducted by the one director of case management and one case manager who
were trained in the service.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The documentation shows there was no need for corrective actions identified for the facilitators of
Impact of Crime during the annual review period of July 1, 2023 through June 30, 2024. During an
informal interview with the director of case management, it was explained that if a need for
corrective action is found, the facilitator will receive progressive discipline to include coaching,
retraining, and formal write-up(s).
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Copies of the annual employee evaluations for the Impact of Crime facilitators were not provided
to the technical assistance specialist for the annual review period of July 1, 2023 through June 30,
2024.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 16 weeks for this type of service. Of the 15 youth in the
sample, 100% (15 of 15) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 15 youth in the
sample, 100% (15 of 15) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 15 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 60%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 73%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 2 youth
Moderate-High = 4 youth
High = 5 youth
Total Youth in Sample = 15 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 33%
High Score: 10 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 77%
Table 2
High = 5 youth
Total Youth in Sample = 15 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 73% (11 of 15) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 33% (5 of 15) youth scored High-Risk to reoffend, for a score of 10 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 15 25 60%
Totals 75 85 88%
This SPEP report evaluates Impact of Crime, an intervention delivered at South Florida Youth
Academy.
17 youth completed the program during the review period. Of those youth, 15 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 15 total youth sampled,
only 15 included dosage with end dates in the EBS Module. Of those youth with correct dosage,
15 received at least the recommended weeks of service. Youth in the sample completed between
16 and 19 weeks of service, with an average of 18 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 10 points for Amount of Service: Contact Hours. Of the 15 total youth
sampled, only 15 included dosage in the EBS Module. Of those youth with correct dosage, 15
received at least the recommended hours of service. Youth in the sample completed between 26
and 32 hours of service, with an average of 31 hours.
The program was awarded 15 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 100%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
South Florida Youth Academy can optimize their SPEP Quality of Service Delivery score by
ensuring each facilitator’s performance evaluation includes discussion of their skill in delivering
this primary service.
South Florida Youth Academy can maintain their SPEP Amount of Service score by continuing to
ensure dosage for all youth is recorded accurately in EBS and by continuing to ensure youth
receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2023-24 South Florida Youth Academy IOC.pdf

service_type: Impact of Crime