summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Youth Environmental Services
AMIkids, Inc
(Contract Provider)
4337 Saffold Road
Wimauma, Florida 33598
Primary Service: Cannabis Youth Treatment
SPEP Review Date(s): 10/23/19
Florida Department of Juvenile Justice
Report Date(s): 10/28/19
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
 Service Type: service type points are awarded based on the group category that the
service falls within
 Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
 Amount of Service: or “dosage” refers to the amount and duration of services that were
received
 Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Cannabis Youth Treatment. The program was awarded
30 points because the Primary Service is identified as a Group 5 Service. The specific Sub-
Component Service Type identified is Cognitive Behavioral Therapy. The Primary Service was
identified as this type of service as it is intended to correct faulty cognitions and perceptions and
provides skills individuals can use to monitor thought patterns and correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
At the time of the annual Standardized Program Evaluation Protocol (SPEP) review, the program
had five staff members trained to facilitate the Cannabis Youth Treatment (CYT) curriculum-one
Director of Treament Services, one Compliance Manager and three Therapists. One Therapist
facilitated the group during the review period. The Director of Treatment Services conducted the
fidelity monitoring.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program is using the Cannabis Youth Treatment (CYT) curriculum from the U.S. Department
of Health and Human Services, volumes one and two. The first two sessions of the curriculum are
individual treatment sessions with the facilitator, and then the youth begin group sessions. The
CYT manual is detailed and explains how each specific session is to be delivered, which includes
an outline for each group session. The faciliators maintain a copy of their group's sign in sheets
and have their own CYT manual.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The Cannabis Youth Treatment curriculum was unable to be observed during the review dates as
the group was not running. The facility had just completed a cohort on 10/18/19. They are waiting
on enough new youth to be assigned to the program in order to begin a new group at this time.
Therefore this indicator rates as non-applicable.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of the program's documentation provided and an interview with the Director of Treatment
Services confirmed there was staff turnover during the review period; however, this did not
adversely affect the services of Cannabis Youth Treatment to the youth. The program, in the
event of turnover, utilizes other trained staff to facilitate the curriculum. There was a gap in
services of 34 days from 11/21/18-12/25/18. This was due to the facility waiting on enough new
youth to begin a new group after the prior group had ended on 11/20/18. The new group started
on 12/26/18. A review of the Juvenile Justice Information System confirmed an average daily
population of 29 youth.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The Director of Treatment Services completed the fidelity monitoring. The fidelity monitor has
received certification by a qualified trainer in the Cannabis Youth Treatment curriculum and fidelity
monitoring. The program provided internal fidelity monitoring records for the review period for the
facilitators. The program uses an internal fidelity checklist specific to Cannabis Youth Treatment
(CYT) to document the monitoring. The program had completed checklists for all faciliators for
each month they completed their CYT groups. All completed fidelity checklists are maintained in a
binder entitled "Fidelity Monitoring for CYT", which is organized by monthly tabs. Each tab had a
copy of the fidelity checklist completed on the facilitator or facilitators who performed groups
during the specific month.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process by which corrective action is acknowledged and demonstrated based
on the fidelity monitoring of the primary service. An interview with the Director of Treatment
Services and a review of documentation verified the program has a process for corrective actions
based on the fidelity monitoring conducted. The program uses internal fidelity monitoring /
corrective action checklists specific to Cannabis Youth Treatment. The program did not have any
corrective actions noted nor taken during the review period.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of employee records for the facilitators and the Director Treament Services who
conducted fidelity monitoring found all staff received an evaluation within the last year.The
performance evaluations of the trained facilitators found entries on the evaluations of the staff's
performance in delivering Cannabis Youth Treatment. These included comments on any strengths
or weaknesses of which the facilitator delivering the curriculum should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service ‐ Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 12 weeks for this type of service. Of the 19 youth in the
sample, 95% (18 of 19) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 12 contact hours for this type of service. Of the 19 youth in the
sample, 100% (19 of 19) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 18 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 72%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 74%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 2 youth
Moderate-High = 5 youth
High = 7 youth
Total Youth in Sample = 19 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 37%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 7 youth
Total Youth in Sample = 19 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 74% (14 of 19) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 37% (7 of 19) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 18 25 73%
Totals 91 100 91%
This SPEP report evaluates Cannabis Youth Treatment, an intervention delivered at Youth
Environmental Services.
35 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The targeted number of weeks for this specific intervention is 12 weeks; the targeted number of
hours for this specific intervention is 12 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 5 services (15 weeks; 45
hours).
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 8 points for Amount of Service: Duration. Of the 19 total youth sampled, all
19 included dosage with end dates in the EBS Module. Of those youth, 18 received at least the
recommended weeks of service. Youth in the sample completed between 9 and 16 weeks of
service, with an average of 12 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 19 total youth
sampled, all 19 included dosage in the EBS Module. Of those youth, 19 received at least the
recommended hours of service. Youth in the sample completed 14.5 hours of service.
The program was awarded 18 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 98%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Youth Environmental Services can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Youth Environmental Services can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9


state: FL

facility_name: youth environmental services

summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Youth Environmental Services
AMIkids, Inc.
(Contract Provider)
4337 Saffold Road
Wimauma, Florida 33598
Primary Service: Aggression Replacement Training
SPEP Review Period: 09/15/2019 - 09/14/2020
SPEP Review Conducted: 09/22/2020
Florida Department of Juvenile Justice
Report Date: 11/30/2020
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Aggression Replacement Training.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
During the review period, the program had five staff members trained to facilitate the Aggression
Replacement Training curriculum- one Director of Treatment Services, two Therapists, one Case
Manager, and one Business Manager. Two staff facilitated groups (Director of Treatment Services
and one Therapist); two staff provided fidelity monitoring (the other therapist and Business
Manager).
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program is uses Aggression Replacement Training (ART)- A Comprehensive Intervention for
Aggressive Youth, 3rd Edition curriculum. The ART manual is detailed and explains how each
specific session is to be delivered and includes an outline for each group session. The facilitators
maintain a copy of their group's sign in sheets and have their own manual.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Aggression Replacement Training was observed via Microsoft Teams on Monday,
09/21/20, from 2:30-3:30. The observation was completed off-site due to the COVID-19
pandemic. The facilitators followed the lesson according to the manual, did not read the lesson
verbatim, but rather used their own words. They were easy for the youth to follow and adhered to
the lesson plan in every aspect.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of the program's documentation provided and an interview with the Director of Treatment
Services confirmed there was some staff turnover during the review period; however, the turnover
did not adversely affect the services of Aggression Replacement Training to the youth.The
program, in the case of turnover, utilizes administrative or other trained staff to facilitate the
curriculum. There were no gaps in service.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Staff who are formally trained in the Aggression Replacement Training (ART) currciulum
conducted fidelity monitoring and have received certification by a qualified trainer in the ART
curriculum and fidelity monitoring. The program provided internal fidelity monitoring records for the
review period. The program uses an internal fidelity checklist specific to ART to document the
monitoring. The program had completed checklists for all faciliators for each month they
completed their ART groups. All completed fidelity checklists are maintained in a binder entitled
"Fidelity Monitoring for ART", which is organized by monthly tabs. Each tab had a copy of the
fidelity checklist completed on the facilitator or facilitators who performed groups during the
specific month.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process by which corrective action is acknowledged and demonstrated based
on the fidelity monitoring of Aggression Replacement Training. The program uses internal fidelity
monitoring/corrective action checklists specific to Aggression Replacement Training. The program
did not have any corrective action during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of employee records for the two facilitators and the one staff who conducted fidelity
monitoring found all three received an evaluation within the last year. The performance
evaluations of the trained facilitators found entries of the staff's performance in delivering
Aggression Replacement Training. These included comments on any strengths or weaknesses of
which the facilitator delivering the curriculum should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 10 weeks for this type of service. Of the 24 youth in the
sample, 96% (23 of 24) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 30 contact hours for this type of service. Of the 24 youth in the
sample, 96% (23 of 24) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 92%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 4 youth
Moderate-High = 8 youth
High = 10 youth
Total Youth in Sample = 24 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 42%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 10 youth
Total Youth in Sample = 24 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 92% (22 of 24) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 42% (10 of 24) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 23 25 92%
Totals 94 100 94%
This SPEP report evaluates Aggression Replacement Training, an intervention delivered at Youth
Environmental Services.
35 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The targeted number of weeks for this specific intervention is 10 weeks; the targeted number of
hours for this specific intervention is 30 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 5 services (15 weeks; 45
hours).
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
The program earned 8 points for Amount of Service: Duration. Of the 24 total youth sampled, all
24 included dosage with end dates in the EBS Module. Of those youth, 23 received at least the
recommended weeks of service. Youth in the sample completed between 8 and 13 weeks of
service, with an average of 11 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 24 total youth
sampled, all 24 included dosage in the EBS Module. Of those youth, 23 received at least the
recommended hours of service. Youth in the sample completed between 18 and 33 hours of
service, with an average of 30 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
RECOMMENDATION(S):
Youth Environmental Services can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Youth Environmental Services can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8

source: DJJ

report_type: SPEP

program_name_raw: Youth Environmental Services

program_status_note: Closed

service_type: Aggression Replacement Training


summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Youth Environmental Services
AMIkids, Inc.
(Contract Provider)
4337 Saffold Road
Wimauma, Florida 33598
Primary Service: Cannabis Youth Treatment
SPEP Review Period: 09/15/2019 - 09/14/2020
SPEP Review Conducted: 09/22/2020
Florida Department of Juvenile Justice
Report Date: 11/30/2020
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Cannabis Youth Treatment.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
At the time of the annual SPEP review, the program had two staff members trained to facilitate the
Cannabis Youth Treatment curriculum- one Director of Treatment Services and one Mental Health
Professional (Therapist). One Therapist facilitated the group during the review period. The
Director of Treatment Services conducted the fidelity monitoring.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program is using the Cannabis Youth Treatment (CYT) curriculum from the U.S. Department
of Health and Human Services, volumes one and two. The first two sessions of the curriculum are
individual treatment sessions with the facilitator and then the youth begin group sessions. The
CYT manual is detailed and explains how each specific session is to be delivered, which includes
an outline for each group session. The facilitators maintain a copy of their group's sign in sheets
and have their own CYT manual.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The Cannabis Youth Treatment curriculum was observed via Microsoft Teams on Monday,
09/21/20 from 3:30-4:30PM. The observation was completed off-site due to the COVID-19
pandemic. The facilitators followed the lesson according to the manual, did not read it verbatim
and put the lesson in their own easy to understand terms for the youth. They were easy to
understand and followed the manual/protocol in every aspect.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of the program's documentation provided and an interview with the Director of Treatment
Services confirmed there was no turnover during the review period which adversely affected the
delivery of Cannabis Youth Treatment to youth. The program, in the case of turnover, utilizes
other trained staff to facilitate the curriculum. There were no gaps in services.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The Director of Treatment Services completed the fidelity monitoring. The fidelity monitor has
received certification by a qualified trainer in the Cannabis Youth Treatment curriculum and fidelity
monitoring. The program uses an internal fidelity checklist specific to Cannabis Youth Treatment
to document the monitoring. The program had completed checklists for all facilitators for each
month they completed their CYT groups except for the months of 11/19 and 03/20. All completed
fidelity checklists are maintained in a binder entitled "Fidelity Monitoring for CYT", which is
organized by monthly tabs. Each tab had a copy of the fidelity checklist completed on the
facilitator or facilitators who performed groups during the specific month.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process by which corrective action is acknowledged and demonstrated based
on the fidelity monitoring of the primary service. The program uses internal fidelity monitoring /
corrective action checklists specific to Cannabis Youth Treatment. The program did not have any
corrective action during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of employee records for facilitators and fidelity monitor found all staff received an
evaluation within the last year. The performance evaluations of facilitators found entries of the
staff's performance in delivering Cannabis Youth Treatment. These included comments on any
strengths or weaknesses of which the facilitator delivering the curriculum should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 12 weeks for this type of service. Of the 15 youth in the
sample, 60% (9 of 15) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 12 contact hours for this type of service. Of the 15 youth in the
sample, 93% (14 of 15) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 18 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 72%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 87%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 3 youth
Moderate-High = 6 youth
High = 4 youth
Total Youth in Sample = 15 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 27%
High Score: 8 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 62%
Table 2
High = 4 youth
Total Youth in Sample = 15 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 87% (13 of 15) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 27% (4 of 15) youth scored High-Risk to reoffend, for a score of 8 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 18 25 72%
Totals 87 100 87%
This SPEP report evaluates Cannabis Youth Treatment, an intervention delivered at Youth
Environmental Services.
35 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The targeted number of weeks for this specific intervention is 12 weeks; the targeted number of
hours for this specific intervention is 12 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 5 services (15 weeks; 45
hours).
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
The program earned 6 points for Amount of Service: Duration. Of the 15 total youth sampled, all
15 included dosage with end dates in the EBS Module. Of those youth, 9 received at least the
recommended weeks of service. Youth in the sample completed between 10 and 16 weeks of
service, with an average of 12 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 15 total youth
sampled, all 15 included dosage in the EBS Module. Of those youth, 14 received at least the
recommended hours of service. Youth in the sample completed between 11 and 15 hours of
service, with an average of 14 hours.
The program was awarded 18 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 94%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Youth Environmental Services can optimize their SPEP Quality of Service Delivery score by
ensuring fidelity monitoring is completed every month for each facilitator.
Youth Environmental Services can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8

service_type: Cannabis Youth Treatment


summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Youth Environmental Services
AMIkids, Inc
(Contract Provider)
4337 Saffold Road
Wimauma, Florida 33598
Primary Service: Aggression Replacement Training
SPEP Review Period: 09/15/20-09/14/21
SPEP Review Conducted: 09/14/21
Florida Department of Juvenile Justice
Report Date: 11/5/21
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Aggression Replacement Training.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The intervention being evaluated is Aggression Replacement Training. The program has a formal
training protocol for the curriculum. Staff attend a two day in-person training conducted by official
trainers through AMIKids. The training consists of learning the curriculum and conducting mock
groups. Due to the Covid19 Pandemic, the most recent facilitators for ART were trained through
TEAMS and had to complete a post-test demonstrating knowledge of the materials in order to
receive a certificate. The program had four staff members trained to facilitate the Aggression
Replacement Training (ART) curriculum- two mental health therapists, one Transition Coordinator
and one direct care staff. The Transition Coordinator was trained to conduct fidelity monitoring
only; the direct care staff meets the qualifications to provide clinical services and facilitates groups
under the supervision of the Clinical Director.
Three staff facilitated the group during the review period, two therapists and one direct care staff
member. The same three staff trained in ART also conducted fidelity monitoring for each other.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses a brand name manual for the group- Aggression Replacement Training-A
Comprehensive Intervention for Aggressive Youth, 3rd Edition curriculum. There is a facilitator
manual only and no student manual. The ART manual provides a script, is detailed and explains
how each specific session is to be delivered and includes an outline for each group session. The
facilitators maintain a copy of their group's sign in sheets and have their own manual.
This group is held on Mondays, Tuesdays, and Wednesdays from 2:30 p.m-3:30 p.m. The
program maintains their records for this intervention in a SPEP Binder that is individually tabbed
and includes staff training, evaluations, sign in sheets, fidelity monitoring and all other relevant
SPEP documentation. Youth are reviewed once they arrive at the program to see if they qualify to
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
be placed in ART. The criteria are: lack of pro-social skills, aggression and poor decision making.
After completing the Comprehensive Assessment and Master Treatment Plan, the Therapist
decides whether or not to place them in the group. ART is a closed group.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Aggression Replacement Training was observed on Tuesday, 09/14/21 from 2:30
p.m-3:30 p.m. There were five youth and three staff members present. The lesson was Week 4:
Anger Control and Reminders and a copy of the lesson was provided to the reviewer prior to the
observation. The youth learn how to control anger in steps. The facilitators followed the lesson
according to the manual, did not read the lesson verbatim, but rather used their own words. They
were easy for the youth to follow and adhered to the lesson plan in every aspect. They were both
very knowledgeable of the materials and ensure the youth grasped each section prior to moving
on in the lesson.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of the documentation provided and an interview with the therapist confirmed there was
no staff turnover which adversely affected the services of Aggression Replacement Training to the
youth. In the event of facilitator turnover, the program would utilize administrative or other trained
staff to facilitate the curriculum. There were gaps in services from 11/19/20-01/10/21, 04/04/21-
05/02/21 and from 07/25/21-08/22/21. The therapist stated these gaps were not caused by staff
turnover, but rather were caused by the freeze on receiving new youth back in September 2020.
The program completed three cohorts of ART and have one cohort currently ongoing. The dates
of the cohorts were 09/14/20-11/18/20 with four youth, 01/11/21-04/03/21 with six youth,
05/03/21-07/24/21 with seven youth and the latest cohort started on 08/23/21 with five youth.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Two therapists and one direct care staff facilitated the Aggression Replacement Training (ART)
group during the review period. The trained staff conducted fidelity monitoring for each other and
have received certification by a qualified trainer in the ART curriculum and fidelity monitoring. The
program provided internal fidelity monitoring records for the review period for each facilitator. The
program uses an internal fidelity checklist specific to ART to document the monitoring. The
program had completed checklists for all facilitators for each month they completed their ART
groups except for the months of April 2021 and June 2021. All completed fidelity checklists are
maintained in a binder entitled "Fidelity Monitoring for ART", which is organized by monthly tabs.
Each tab had a copy of the fidelity checklist completed on the facilitator or facilitators who
performed groups during the specific month.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process by which corrective action is acknowledged and demonstrated based
on the fidelity monitoring of the primary service. The program uses internal fidelity
monitoring/corrective action checklists specific to the Aggression Replacement Training
curriculum. Each completed checklist includes feedback specifying either the facilitator did not
require any corrective action or if the facilitator did require corrective action, it would include
recommendations for future improvement. The following month's checklist would include
comments that the previous month's recommendations were applied and demonstrated. However,
an interview with the Therapist and a review of the documentation verified that the program did
not take any corrective action during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of employee records for facilitators and staff who conducted fidelity monitoring found all
received an evaluation within the last year. The performance evaluations of the trained facilitators
were reviewed and entries were found of the staff's performance in delivering Aggression
Replacement Training. These included comments on any strengths or weaknesses of which the
facilitator delivering the curriculum should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 10 weeks for this type of service. Of the 18 youth in the
sample, 100% (18 of 18) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 30 contact hours for this type of service. Of the 18 youth in the
sample, 100% (18 of 18) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 8 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 32%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 72%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 1 youth
Moderate-High = 9 youth
High = 3 youth
Total Youth in Sample = 18 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 17%
High Score: 3 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 23%
Table 2
High = 3 youth
Total Youth in Sample = 18 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 72% (13 of 18) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 17% (3 of 18) youth scored High-Risk to reoffend, for a score of 3 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 8 25 32%
Totals 83 100 83%
This SPEP report evaluates Aggression Replacement Training, an intervention delivered at Youth
Environmental Services.
24 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The targeted number of weeks for this specific intervention is 10 weeks; the targeted number of
hours for this specific intervention is 30 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 5 services (15 weeks; 45
hours).
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 10 points for Amount of Service: Duration. Of the 18 total youth sampled, all
18 included dosage with end dates in the EBS Module. Of those youth, 18 received at least the
recommended weeks of service. Youth in the sample completed between 10 and 12 weeks of
service, with an average of 11 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 18 total youth
sampled, all 18 included dosage in the EBS Module. Of those youth, 18 received at least the
recommended hours of service. Youth in the sample completed 30 hours of service.
The program was awarded 8 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 100%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Youth Environmental Services can optimize their SPEP Quality of Service Delivery score by
ensuring fidelity monitoring is completed every month for each facilitator.
Youth Environmental Services can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2021-22 YES ART.pdf


summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Youth Environmental Services
AMIkids, Inc.
(Contract Provider)
4337 Saffold Road
Wimauma, Florida 33598
Primary Service: Cannabis Youth Treatment
SPEP Review Period: 09/15/20-09/14/21
SPEP Review Conducted: 09/14/21
Florida Department of Juvenile Justice
Report Date: 11/5/21
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Cannabis Youth Treatment.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There is a formal two-day training protocol for Cannabis Youth Treatment (CYT), conducted by
official CYT trainers through AMIKids. The training consists of learning the curriculum and
conducting mock groups. At the time of the annual compliance review, the program had two staff
members trained to facilitate CYT- two therapists. Both therapists facilitated groups and
conducted fidelity monitoring for each other during the review period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the brand name Cannabis Youth Treatment (CYT) curriculum from the U.S.
Department of Health and Human Services, volumes one and two. The reviewer looked over the
manual during the review. There is a facilitator manual only and no student manual. The first two
sessions of the curriculum are individual treatment sessions with the facilitator, and then the youth
begin group sessions. This is a closed group. The CYT manual has a script and is detailed and
explains how each specific session is to be delivered, which includes an outline for each group
session. The facilitators maintain a copy of their group's sign in sheets and have their own CYT
manual. The intervention is currently not running as they just completed a cohort on 09/06/21 and
are currently not receiving new youth.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
A session of Cannabis Youth Treatment (CYT) could not be observed during the review period.
The program had just completed a cohort of CYT on 09/06/21 and a new one had not been
started yet. Therefore, this indicator rates as non-applicable.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of the program's documentation provided and an interview with the therapist confirmed
there was no turnover during the review period which adversely affected the services of CYT. In
the event of staff turnover, the program would utilize administrative or other trained staff to
facilitate the curriculum. The gaps in services were not due to staff turnover, but rather due to a
freeze on receiving youth back in September of 2020, which just ended on 09/15/21. The gaps
were from 10/02/20-01/28/21, 04/21/21-07/10/21, and 09/07/21 to present. The program
completed three cohorts during the review period. The dates of the cohorts were: 07/14/20-
10/01/20 with five youth in group, 01/29/21-04/20/21 with seven youth in group and 07/11/21-
09/06/21 with nine youth in group.
The program can earn 2 points if there are no unnecessary gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has an in-house process to perform internal fidelity monitoring. The two therapists
completed fidelity monitoring for each other. The fidelity monitors have received certification by a
qualified trainer in the Cannabis Youth Treatment (CYT) curriculum and fidelity monitoring. The
program provided internal fidelity monitoring records for the review period for the facilitators. The
program uses an internal fidelity checklist specific to CYT to document the monitoring. The
program had completed checklists for all facilitators for each month they completed their CYT
groups except for the months of September 2020, October 2020, and January 2021. All
completed fidelity checklists are maintained in a binder entitled "Fidelity Monitoring for CYT",
which is organized by monthly tabs. Each tab had a copy of the fidelity checklist completed on the
facilitator or facilitators who performed groups during the specific month.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The program has a process by which corrective action is acknowledged and demonstrated based
on the fidelity monitoring of the primary service. The program uses internal fidelity
monitoring/corrective action checklists specific to the Cannabis Youth Treatment curriculum. Each
completed checklist includes feedback specifying either the facilitator did not require any
corrective action or if the facilitator did require corrective action, it would include recommendations
for future improvement. The following month's checklist would include comments that the previous
month's recommendations were applied and demonstrated.
An interview with the Therapist and a review of the documentation verified the program did not
take any corrective action during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of employee records for the facilitators found all staff received an evaluation within the
last year. The performance evaluations of the trained facilitators found entries of the staff's
performance in delivering Cannabis Youth Treatment. These included comments on any strengths
or weaknesses of which the facilitator delivering the curriculum should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 12 weeks for this type of service. Of the 7 youth in the
sample, 86% (6 of 7) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 12 contact hours for this type of service. Of the 7 youth in the
sample, 100% (7 of 7) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 5 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 20%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 71%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 2 youth
Moderate-High = 3 youth
High = 0 youth
Total Youth in Sample = 7 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 0%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 0 youth
Total Youth in Sample = 7 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 71% (5 of 7) youth scored Moderate to High-Risk to reoffend, for a score of
5 points.
Of the SPEP sample, 0% (0 of 7) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 5 25 20%
Totals 68 100 68%
This SPEP report evaluates Cannabis Youth Treatment, an intervention delivered at Youth
Environmental Services.
24 youth completed the program during the review period.
The program scored Medium for Quality of Service Delivery.
The targeted number of weeks for this specific intervention is 12 weeks; the targeted number of
hours for this specific intervention is 12 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 5 services (15 weeks; 45
hours).
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 8 points for Amount of Service: Duration. Of the 7 total youth sampled, all 7
included dosage with end dates in the EBS Module. Of those youth, 6 received at least the
recommended weeks of service. Youth in the sample completed between 9 and 13 weeks of
service, with an average of 12 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 7 total youth
sampled, all 7 included dosage in the EBS Module. Of those youth, 7 received at least the
recommended hours of service. Youth in the sample completed between 15 and 16 hours of
service, with an average of 15 hours.
The program was awarded 5 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 88%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Youth Environmental Services can optimize their SPEP Quality of Service Delivery score by
ensuring there are no unnecessary gaps in service and fidelity monitoring is completed every
month for each facilitator.
Youth Environmental Services can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9


summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Youth Environmental Services
AMIkids, Inc.
(Contract Provider)
4337 Saffold Road
Wimauma, Florida 33598
Primary Service: Cannabis Youth Treatment
SPEP Review Period: 11/01/21-10/31/22
SPEP Review Conducted: 11/01/22
Florida Department of Juvenile Justice
Report Date: 2/2/23
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Cannabis Youth Treatment .
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The Cannabis Youth Treatment (CYT) intervention is being reviewed for the period of 11/01/21-
10/31/22. There is a formal two-day training protocol for the intervention conducted by official CYT
trainers through AMIKids. The training consists of learning the CYT curriculum and conducting
mock groups. At the time of the annual review, there were two staff trained to facilitate CYT- one
therapist and the Director of Treatment. Both facilitated groups during the review period and
conducted fidelity monitoring for each other.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the brand-name Cannabis Youth Treatment (CYT) curriculum from the U.S.
Department of Health and Human Services, volumes one and two. There is a facilitator manual
only and no student manual. The first two sessions of the curriculum are individual treatment
sessions with the facilitator, then the youth begin group sessions and end with individual
counseling sessions. The CYT manual has a script, is detailed and explains how each specific
session is to be delivered, which includes an outline for each group session.
CYT is currently being held on Tuesdays from 2:30-3:45. This is a closed group. The facilitators
maintain a copy of their group's sign in sheets and have their own CYT manual. Once a
Comprehensive Assessment and Master Treatment Plan is completed for the youth and if it is
determined the youth has a substance abuse diagnosis, typically they are placed in CYT.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The Cannabis Youth Treatment curriculum could not be observed during the review period. The
group which began on 10/29/22 is currently in the individual counseling portion of the curriculum.
Due to the confidential nature of this counseling sessions, an observation could not be completed.
Therefore this indicator rates as non-applicable.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation provided and an interview with the Director of Treatment confirmed
there was no turnover during the review period which adversely affected the services of Cannabis
Youth Treatment (CYT) to the youth. In the event of staff turnover, the program would utilize
administrative or other trained staff to facilitate the curriculum. There were no gaps in service
during cohorts of CYT.
The program completed two cohorts during the review period from 12/22/21-04/08/22 (one youth)
and from 05/25/22-08/30/22 (four youth). There is currently a new cohort which began on
10/29/22. At the time of this review, the group is still in the individual counseling portion of the
curriculum. There are seven youth in this cohort.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has an in-house process to perform internal fidelity monitoring. One Therapist and
one Director of Treatment completed fidelity monitoring for each other. The fidelity monitors have
received certification by a qualified trainer in the Cannabis Youth Treatment curriculum. The
program provided internal fidelity monitoring records for the review period. The program uses an
internal fidelity checklist specific to Cannabis Youth Treatment to document the monitoring.
The program completed checklists for both facilitators for each month they facilitated a CYT
group. All completed fidelity checklists are maintained in a binder entitled "Fidelity Monitoring for
CYT", which is organized by monthly tabs. Each tab had a copy of the fidelity checklist completed
on the facilitator or facilitators who performed groups during the specific month.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process by which corrective action is acknowledged and demonstrated based
on the fidelity monitoring of Cannabis Youth Treatment (CYT). The program uses internal fidelity
monitoring/corrective action checklists specific to the CYT curriculum. Each completed checklist
includes feedback specifying either the facilitator did not require any corrective action or, if the
facilitator did require corrective action, it would include recommendations for future improvement.
The following month's checklist would include comments that the previous month's
recommendations were applied and demonstrated.
An interview with the Director of Treatment and a review of the documentation verified the
program did not take any corrective action during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of employee records for the facilitators found all staff received an evaluation within the
last year. The performance evaluation of the trained facilitators found entries of the staff's
performance in delivering Cannabis Youth Treatment. These included comments on any strengths
or weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates the target duration of 12 weeks for this type of service. Of the 4 youth in the
sample, 50% (2 of 4) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 12 contact hours for this type of service. Of the 4 youth in the
sample, 75% (3 of 4) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 13 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 48%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 50%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 1 youth
Moderate-High = 0 youth
High = 1 youth
Total Youth in Sample = 4 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 25%
High Score: 8 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 62%
Table 2
High = 1 youth
Total Youth in Sample = 4 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 50% (2 of 4) youth scored Moderate to High-Risk to reoffend, for a score of
5 points.
Of the SPEP sample, 25% (1 of 4) youth scored High-Risk to reoffend, for a score of 8 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 4 10 40%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 13 25 48%
Totals 78 100 78%
This SPEP report evaluates Cannabis Youth Treatment , an intervention delivered at Youth
Environmental Services.
28 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The targeted number of weeks for this specific intervention is 12 weeks; the targeted number of
hours for this specific intervention is 12 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 5 services (15 weeks; 45
hours).
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 4 points for Amount of Service: Duration. Of the 4 total youth sampled, all 4
included dosage with end dates in the EBS Module. Of those youth, 2 received at least the
recommended weeks of service. Youth in the sample completed between 11 and 16 weeks of
service, with an average of 13 weeks.
The program earned 6 points for Amount of Service: Contact Hours. Of the 4 total youth sampled,
all 4 included dosage in the EBS Module. Of those youth, 3 received at least the recommended
hours of service. Youth in the sample completed between 11 and 16 hours of service, with an
average of 14 hours.
The program was awarded 13 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 90%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Youth Environmental Services can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Youth Environmental Services can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2022-23 YES CYT.pdf