state: FL

facility_name: Everglades Youth Academy

summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Everglades Youth Academy
Youth Opportunity Investments, LLC
(Contract Provider)
5050 NE 168th Street
Okeechobee, Florida 34972
Primary Service: Impact of Crime
SPEP Review Period: December 1, 2020-November 30, 2021
SPEP Review Conducted: December 7, 2021
Florida Department of Juvenile Justice
Report Date: 2/18/22
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The annual SPEP review consists of an examination of twelve months of primary service
documents. All case managers at the program are trained to be facilitators of Impact of Crime.
During the past twelve months, the director of case management, transition services manager,
and three case managers have facilitated the primary service. Currently, only the director of case
management is facilitating groups; however, four newly hired case managers are being trained in
the next train-the-trainer class. The regional compliance manager is also trained, and has been
providing internal fidelity monitoring.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the curriculum Impact of Crime: Addressing the Harm to Victims and the
Community, published in 2009. It is highly detailed, providing a facilitator's manual and student
workbook. The facilitator's manual provides lessons plans, classroom management ideas, and
tips for engaging youth discussions. The student manual provides reproducible worksheets to
help facilitate group discussions, and review concepts learned during the group sessions.
Homework materials are also utilized in the student manual to reinforce learned concepts. Impact
of Crime is facilitated as a closed group, meaning once a cohort begins, no new youth can be
added to the group.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Impact of Crime was observed during the review period. The facilitator did not read
the script word for word, but generalized the information and added her own words to provide
meaning to covered concepts. The youth were generally engaged and participated in the
discussion. No behavioral issues were observed during the group.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation and an informal interview with the regional compliance manager, two
cohorts of Impact of Crime (IOC) had facilitators leave the groups. One group saw both co-
facilitators leave the group and another trained facilitator completed the cohort.
The program showed documentation of four cohorts being facilitated during the review period.
The first cohort of IOC began on February 22, 2021 and concluded on July 1, 2021. The cohort
started and ended with ten youth. The cohort was co-facilitated and one facilitator left during the
cohort but groups continued to be held with the remaining facilitator.
A second cohort began on February 22, 2021 and ended on June 24, 2021. The cohort began
and ended with six youth.
A third cohort began on July 20, 2021 and ended on November 18, 2021. The cohort began and
ended with seven youth. During this cohort both co-facilitators left, and another trained staff
member completed the cohort.
A fourth cohort began on August 3, 2021 and ended on December 2, 2021. The cohort started
with seven youth and ended with six youth. One youth was transferred to adult jail during the
cohort and has not returned.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
According to documentation provided by the program, internal fidelity monitoring took place
monthly for facilitators of Impact of Crime. The fidelity monitoring was completed by the regional
compliance manager and former director of case management, both were trained in the primary
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a policy for corrective action plans for facilitators of Impact of Crime. In the
event a facilitator needs help with service delivery, the regional compliance manager or director of
case management, will sit with the facilitator and develop a plan to improve the facilitator's skills.
No corrective actions were noted nor taken during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of annual evaluations documented the facilitators were given credit for their group
facilitation. The facilitators' skills in the delivery of Impact of Crime was evaluated by supervisors.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 16 weeks for this type of service. Of the 3 youth in the
sample, 67% (2 of 3) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 24 contact hours for this type of service. Of the 3 youth in the
sample, 67% (2 of 3) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 2 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 8%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 33%
Moderate to High Score: 2 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 17%
Table 1
Moderate = 0 youth
Moderate-High = 1 youth
High = 0 youth
Total Youth in Sample = 3 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 0%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 0 youth
Total Youth in Sample = 3 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 33% (1 of 3) youth scored Moderate to High-Risk to reoffend, for a score of
2 points.
Of the SPEP sample, 0% (0 of 3) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 2 25 8%
Totals 54 85 64%
This SPEP report evaluates Impact of Crime, an intervention delivered at Everglades Youth
Academy.
16 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 6 points for Amount of Service: Duration. Of the 3 total youth sampled, only
2 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 2
received at least the recommended weeks of service. Youth in the sample completed between 16
and 19 weeks of service, with an average of 18 weeks.
The program earned 6 points for Amount of Service: Contact Hours. Of the 3 total youth sampled,
all 3 included dosage in the EBS Module. Of those youth, 2 received at least the recommended
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
hours of service. Youth in the sample completed between 23 and 24 hours of service, with an
average of 28 hours.
The program was awarded 2 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 91%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Impact of Crime is an intervention suitable for all youth. The Department recommends providing
this service to every youth in the program.
Everglades Youth Academy can optimize their SPEP Quality of Service Delivery score by
ensuring facilitator turnover does not impact groups.
Everglades Youth Academy can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

source: DJJ

report_type: SPEP

file_name: FY2021-22 Everglades YA - IOC.pdf

program_name_raw: Everglades Youth Academy

program_status_note: Open

service_type: Impact of Crime


summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Everglades Youth Academy
Youth Opportunity Investments, LLC
(Contract Provider)
5050 NE 168th Street
Okeechobee, Florida 34972
Primary Service: Moral Reconation Therapy
SPEP Review Period: December 1, 2020-November 30, 2021
SPEP Review Conducted: December 7, 2021
Florida Department of Juvenile Justice
Report Date: 2/18/22
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Moral Reconation Therapy.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The annual SPEP review consists of an examination twelve months of primary service
documents.Moral Reconation Therapy (MRT) groups are facilitated by master level therapists at
the program. The clinical director provided documentation of each therapist being trained in the
primary service.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
Moral Reconation Therapy (MRT) was initially designed as a treatment program for adults
residing in prison-based substance abuse therapeutic communities. MRT is designed to help
participants reexamine their choices and develop cognitive structures that will improve decision-
making skills. The curriculum for juveniles, entitled “Juvenile MRT- How to Escape Your Prison” is
appropriate for both juvenile offenders and juvenile substance abusers, and is appropriate for
general treatment and substance abuse treatment. Based on cognitive-behavioral therapy
principles, characteristics that guide MRT practice include: 1) self-assessment of attitudes, beliefs,
defense mechanisms, and behavior, 2) current relationship assessment, 3) positive
reinforcement, 4) alteration of self-concept in the positive direction, 5) changes in gratification
stimuli, and 6) enhance moral reasoning.
The clinical director provided a copy of the manual. It is a commercially produced volume which
has both student and facilitator editions. Every youth who enters the program is enrolled in the
primary service. MRT runs as an open group, meaning as new youth enter the program, they are
immediately added to a group. The groups are organized by living modules with each living
module having its own. MRT groups are conducted twice per week on Mondays and
Wednesdays.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Moral Reconation Therapy was observed during the review period. The group
session took place in a classroom with seven youth and one direct care staff present. The
facilitator modeled the role play, then each member was able to practice the new skill. The
facilitator then led a large group discussion over feelings and emotions and how they influence a
person's behavior. There were no behavior issues observed during the group. The facilitator did
not read the script verbatim, but did have the curriculum open on the desk to ensure all points of
the lesson were covered.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
Moral Reconation Therapy (MRT) is delivered as open group run year-round and several
facilitators left employment during the review period. As such, facilitator turnover did occur;
however, groups were transitioned to new facilitators and no gaps in service occurred. The
program provided documentation of six cohorts of MRT being facilitated.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
According to documentation provided by the program, internal fidelity monitoring was completed
by the clinical director for facilitators of Moral Reconation Therapy. A total of five fidelity
monitorings took place during the review year. In informal conversation with the clinical director,
by being so short staffed in therapists, and himself facilitating groups, it is nearly impossible to
provide internal fidelity monitoring monthly.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a policy for corrective action plans for facilitators of Moral Reconation Therapy.
In the event a facilitator needs help with service delivery, the director of case management will sit
with the facilitator and develop a plan to improve the facilitator's skills.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of annual evaluations documented the facilitators were given credit for their group
facilitation. The facilitators' skills in the delivery of Moral Reconation Therapy was evaluated by
supervisors.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates the target duration of 15 weeks for this type of service. Of the 13 youth in the
sample, 54% (7 of 13) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 45 contact hours for this type of service. Of the 13 youth in the
sample, 69% (9 of 13) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 8 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 32%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 54%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 2 youth
Moderate-High = 3 youth
High = 2 youth
Total Youth in Sample = 13 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 15%
High Score: 3 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 23%
Table 2
High = 2 youth
Total Youth in Sample = 13 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 54% (7 of 13) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 15% (2 of 13) youth scored High-Risk to reoffend, for a score of 3 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 4 10 40%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 8 25 32%
Totals 73 100 73%
This SPEP report evaluates Moral Reconation Therapy, an intervention delivered at Everglades
Youth Academy.
16 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 4 points for Amount of Service: Duration. Of the 13 total youth sampled, all
13 included dosage with end dates in the EBS Module. Of those youth, 7 received at least the
recommended weeks of service. Youth in the sample completed between 6 and 35 weeks of
service, with an average of 19 weeks.
The program earned 6 points for Amount of Service: Contact Hours. Of the 13 total youth
sampled, all 13 included dosage in the EBS Module. Of those youth, 9 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
recommended hours of service. Youth in the sample completed between 14 and 84 hours of
service, with an average of 60 hours.
The program was awarded 8 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 90%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Everglades Youth Academy can optimize their SPEP Quality of Service Delivery score by
ensuring facilitator turnover does not impact groups and fidelity monitoring is completed every
month for each facilitator.
Everglades Youth Academy can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2021-22 Everglades YA - MRT.pdf

service_type: Moral Reconation Therapy


summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Everglades Youth Academy
Youth Opportunity Investments, LLC
(Contract Provider)
5050 NE 168th Street
Okeechobee, Florida 34972
Primary Service: Skillstreaming the Adolescent
SPEP Review Period: December 1, 2020-November 30, 2021
SPEP Review Conducted: December 7, 2021
Florida Department of Juvenile Justice
Report Date:
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Skillstreaming the Adolescent.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
Skillstreaming the Adolescent employs a four-part training approach—modeling, role-playing,
performance feedback, and generalization—to teach essential prosocial skills to adolescents. The
curriculum provides a complete description of the Skillstreaming intervention, with instructions for
teaching 50 prosocial skills. According to documentation provided by the program, All facilitators
are trained in Skillstreaming the Adolescent prior to facilitating groups.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The Skillstreaming the Adolescent curiculum is a commercially developed, highly organized
textbook with fifty skills which provide both discussion and practice in each skill. The text provides
for role playing situations, skill cards, and discussions for each skill.
Skillstreaming the Adolescent groups operate as open groups and are facilitated on the living
modules.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Skillstreaming the Adolescent was observed during the review period. The group
took place in a classroom and was attended by seven youth, one facilitator, and one direct care
staff. The facilitator led the group in a discussion about short and long term goals. The goals
centered around career paths, and options if primary careers don't happen (ie NFL football, Rap
star). The youth were engaged as they explored possible interest in other fields that may lead to
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
long term employment. The group ended with a worksheet where the youth listed goals in one,
five and ten year increments. No behavioral issues were observed during the group.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program has had turnover in facilitators/therapists of Skillstreaming the Adolescent
during the review period; however, since it is an open group, no gap in service delivery occurred.
When a facilitator left a group, other facilitators covered the groups.
As the census count grew during the past year, more cohorts were initated. The program began
with two cohorts of youth in December 2020 and grew to six cohorts in April 2021. The program
continues to facilitate six cohorts of the primary service.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
According to documentation provided by the program, internal fidelity monitoring took place on a
limited basis for facilitators of Skillstreaming the Adolescent. Due to a low number of therapists on
site and the clinical director facilitating groups, four total internal fidelity monitoring sessions were
documented for the review period.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a policy for corrective action plans for facilitators of Skillstreaming the
Adolescent. In the event a facilitator is in need of help with service delivery, the director of case
management will sit with the facilitator and develop a plan to improve the facilitator's skills.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of annual evaluations documented the facilitators were given credit for their group
facilitation. The facilitators' skills in the delivery of Skillstreaming the Adolescent was evaluated by
supervisors.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 16 weeks for this type of service. Of the 13 youth in the
sample, 100% (13 of 13) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 13 youth in the
sample, 100% (13 of 13) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 8 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 32%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 54%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 2 youth
Moderate-High = 3 youth
High = 2 youth
Total Youth in Sample = 13 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 15%
High Score: 3 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 23%
Table 2
High = 2 youth
Total Youth in Sample = 13 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 54% (7 of 13) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 15% (2 of 13) youth scored High-Risk to reoffend, for a score of 3 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 8 25 32%
Totals 68 85 80%
This SPEP report evaluates Skillstreaming the Adolescent, an intervention delivered at
Everglades Youth Academy.
16 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 13 total youth sampled, all
13 included dosage with end dates in the EBS Module. Of those youth, 13 received at least the
recommended weeks of service. Youth in the sample completed between 23 and 51 weeks of
service, with an average of 35 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 13 total youth
sampled, all 13 included dosage in the EBS Module. Of those youth, 13 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
recommended hours of service. Youth in the sample completed between 31 and 56 hours of
service, with an average of 52 hours.
The program was awarded 8 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 100%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Everglades Youth Academy can optimize their SPEP Quality of Service Delivery score by
ensuring facilitator turnover does not impact groups and fidelity monitoring is completed every
month for each facilitator.
Everglades Youth Academy can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2021-22 Everglades YA - Skillstreaming.pdf

service_type: Skillstreaming the Adolescent


summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Everglades Youth Academy Sex Offender Program
Youth Opportunity Investments, LLC
(Contract Provider)
5050 NE 168th Street
Okeechobee, Florida 34972
Primary Service: Impact of Crime
SPEP Review Period: December 1, 2020-November 30, 2021
SPEP Review Conducted: December 7, 2021
Florida Department of Juvenile Justice
Report Date: 2/18/22
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The annual SPEP review consists of an examination of twelve months of primary service
documents. All case managers at the program are trained to be facilitators of Impact of Crime.
During the past twelve months, the director of case management, transition services manager,
and three case managers have facilitated the primary service. Currently, only the director of case
management is facilitating groups; however, four newly hired case managers are being trained in
the next train-the-trainer class. The regional compliance manager is also trained, and has been
providing internal fidelity monitoring.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the curriculum Impact of Crime: Addressing the Harm to Victims and the
Community, published in 2009. It is highly detailed, providing a facilitator's manual and student
workbook. The facilitator's manual provides lessons plans, classroom management ideas, and
tips for engaging youth discussions. The student manual provides reproducible worksheets to
help facilitate group discussions, and review concepts learned during the group sessions.
Homework materials are also utilized in the student manual to reinforce learned concepts. Impact
of Crime is facilitated as a closed group, meaning once a cohort begins, no new youth can be
added to the group.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Impact of Crime was observed during the review period. The facilitator did not read
the script word for word, but generalized the information and added her own words to provide
meaning to covered concepts. The youth were generally engaged and participated in the
discussion. No behavioral issues were observed during the group.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation and an informal interview with the regional compliance manager, two
cohorts of Impact of Crime (IOC) had facilitators leave the groups. One group saw both co-
facilitators leave the group and another trained facilitator completed the cohort.
The program showed documentation of four cohorts of IOC being completed during the review
period. The first cohort began in February 2021 and was completed in May 2021. Eleven youth
started and completed the cohort. A second cohort began in July 2021 and was completed in
October 2021. Ten youth were enrolled in the program and all completed the cohort. A third
cohort began in August 2021 and was completed in November 2021. Ten youth were enrolled in
the cohort and all ten completed the cohort. A fourth cohort had just begun with eight youth in
December 2021 and was on going at the time of the review.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
According to documentation provided by the program, internal fidelity monitoring took place
monthly for facilitators of Impact of Crime. The fidelity monitoring was completed by the regional
compliance manager and former director of case management, both were trained in the primary
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a policy for corrective action plans for facilitators of Impact of Crime. In the
event a facilitator needs help with service delivery, the regional compliance manager or director of
case management, will sit with the facilitator and develop a plan to improve the facilitator's skills.
No corrective actions were noted nor taken during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of annual evaluations documented the facilitators were given credit for their group
facilitation. The facilitators' skills in the delivery of Impact of Crime was evaluated by supervisors.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 16 weeks for this type of service. Of the 4 youth in the
sample, 0% (0 of 4) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 24 contact hours for this type of service. Of the 4 youth in the
sample, 0% (0 of 4) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 12 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 48%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 75%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 1 youth
Moderate-High = 1 youth
High = 1 youth
Total Youth in Sample = 4 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 25%
High Score: 5 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 38%
Table 2
High = 1 youth
Total Youth in Sample = 4 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 75% (3 of 4) youth scored Moderate to High-Risk to reoffend, for a score of
7 points.
Of the SPEP sample, 25% (1 of 4) youth scored High-Risk to reoffend, for a score of 5 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 12 25 48%
Totals 52 85 61%
This SPEP report evaluates Impact of Crime, an intervention delivered at Everglades Youth
Academy Sex Offender Program.
8 youth completed the program during the review period. Of those youth, 4 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration and Amount of Service: Contact
Hours. There was no data entered into the Juvenile Justice Information system (JJIS) Evidence-
Based Services (EBS) module for any youths. With no JJIS data to review, the SPEP score was
reduced. The program must enter dosage data into JJIS in order to receive any score in this
area.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program was awarded 12 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Based on the risk level of the youth placed at the program (48% Program Optimization
Percentage), the Department should work to ensure a larger proportion of higher risk youth are
recommended. As the program serves youth with a history of sexual behavior problems, this
includes a multitier strategy of working with stakeholders to achieve that practice, as well as
ensuring appropriate treatment services are available in the community to reduce the reliance on
residential commitment for lower risk youth presenting with sexually-related offenses.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 76%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Everglades Youth Academy Sex Offender Program can optimize their SPEP Quality of Service
Delivery score by ensuring facilitator turnover does not impact groups.
Everglades Youth Academy Sex Offender Program can optimize their SPEP Amount of Service
score by ensuring that dosage for all youth is recorded accurately in EBS and by ensuring that
youth receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2021-22 Everglades YA SOP - IOC.pdf


summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Everglades Youth Academy Sex Offender Program
Youth Opportunity Investments, LLC
(Contract Provider)
5050 NE 168th Street
Okeechobee, Florida 34792
Primary Service: Pathways- A Guided Workbook for Youth Beginning
Treatment
SPEP Review Period: December 1, 2020-November 30, 2021
SPEP Review Conducted: December 7, 2021
Florida Department of Juvenile Justice
Report Date: 2/18/22
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Pathways- A Guided Workbook for Youth Beginning
Treatment.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The clinical director is the only person on site trained to facilitate Pathways- A Guided Workbook
for Youth Beginning Treatment. An assistant clinical director was in the process of being hired and
trained at the time of the annual review.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
Pathways- A Guided Workbook for Youth Beginning Treatment, Fourth Edition is a structured,
guided workbook for use with adolescent males and females with sexual behavior problems,
including but not limited to sexual offending. The treatment focuses extensively on helping clients
develop healthy and responsible patterns of thinking and behavior, and also provides concrete
guidance about how to control sexual feelings in healthy and responsible ways, including
guidelines for successful friendships and dating relationships. Relapse prevention techniques are
reinforced, as is a new focus on helping clients develop healthy and responsible lifestyles by
creating a Healthy Living Project to follow.
The Pathways- A Guided Workbook for Youth Beginning Treatment curriculum is a 4th edition
textbook which contains fourteen chapters. It consists of group discussion ideas, as well as
reproducible worksheets and group readings. Each chapter also has a chapter review and test at
the end to review the concepts covered it the chapter.
All youth in the program attend Pathways. The group operates as a open group, meaning youth
are added as they arrive in the program. The group meets twice per week for one-hour sessions.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Pathways- A Guided Workbook for Youth Beginning Treatment was observed during
the review period. The group session took place in a classroom with nine youth present. The
facilitator modeled the role play, then each member was able to practice the new skill. There were
no behavior issues observed during the group. The facilitator did not read the script verbatim, but
did have the manual open on the desk to ensure all points of the lesson were covered.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The clinical director is the only staff member trained in Pathways- A Guided Workbook for Youth
Beginning Treatment. No turnover occurred during the review period.
The program showed documentation of two cohorts of Pathways being completed during the
review period. One cohort began in November 2020 with eight youth. That cohort grew to fourteen
youth by April 2021 and was divided into two cohorts. Those two cohorts were completed on July
2021 with eight youth in each cohort. Also in July 2020, two cohorts began with eight youth in
each cohort and was on going at the time of the review.
Basic Score: 0 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
According to documentation provided by the program, internal fidelity monitoring took place once
during the twelve months Pathways- A Guided Workbook for Youth Beginning Treatment was
facilitated.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 0 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a policy for corrective action plans for facilitators of Pathways- A Guided
Workbook for Youth Beginning Treatment. In the event a facilitator is in need of help with service
delivery, the director of case management will sit with the facilitator and develop a plan to improve
the facilitator's skills.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
As no points were awarded for indicator e., this indicator is automatically 0 points. Internal fidelity
monitoring must take place regularly for points to be awarded in this section.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of annual evaluations documented the facilitators were given credit for their group
facilitation. The facilitators' skills in the delivery of Pathways- A Guided Workbook for Youth
Beginning Treatment was evaluated by supervisors.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 24 weeks for this type of service. Of the 5 youth in the
sample, 100% (5 of 5) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 40 contact hours for this type of service. Of the 5 youth in the
sample, 100% (5 of 5) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 10 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 40%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 60%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 1 youth
Moderate-High = 1 youth
High = 1 youth
Total Youth in Sample = 5 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 20%
High Score: 5 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 38%
Table 2
High = 1 youth
Total Youth in Sample = 5 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 60% (3 of 5) youth scored Moderate to High-Risk to reoffend, for a score of
5 points.
Of the SPEP sample, 20% (1 of 5) youth scored High-Risk to reoffend, for a score of 5 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 10 25 40%
Totals 70 95 74%
This SPEP report evaluates Pathways- A Guided Workbook for Youth Beginning Treatment, an
intervention delivered at Everglades Youth Academy Sex Offender Program.
8 youth completed the program during the review period. Of those youth, 5 were included in the
sample.
The program scored Medium for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 5 total youth sampled, all 5
included dosage with end dates in the EBS Module. Of those youth, 5 received at least the
recommended weeks of service. Youth in the sample completed between 26 and 40 weeks of
service, with an average of 31 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 10 points for Amount of Service: Contact Hours. Of the 5 total youth
sampled, all 5 included dosage in the EBS Module. Of those youth, 5 received at least the
recommended hours of service. Youth in the sample completed between 59 and 90 hours of
service, with an average of 66 hours.
The program was awarded 10 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Based on the risk level of the youth placed at the program (40% Program Optimization
Percentage), the Department should work to ensure a larger proportion of higher risk youth are
recommended. As the program serves youth with a history of sexual behavior problems, this
includes a multitier strategy of working with stakeholders to achieve that practice, as well as
ensuring appropriate treatment services are available in the community to reduce the reliance on
residential commitment for lower risk youth presenting with sexually-related offenses.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 89%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Everglades Youth Academy Sex Offender Program can optimize their SPEP Quality of Service
Delivery score by ensuring fidelity monitoring is completed every month for each facilitator.
Everglades Youth Academy Sex Offender Program can optimize their SPEP Amount of Service
score by ensuring that dosage for all youth is recorded accurately in EBS and by ensuring that
youth receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2021-22 Everglades YA SOP - Pathways SO.pdf

service_type: Pathways: A Guided Workbook for Youth Beginning Treatment


summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Everglades Youth Academy Sex Offender Program
Youth Opportunity Investment, LLC
(Contract Provider)
5050 NE 168th Street
Okeechobee, Florida 34972
Primary Service: Skillstreaming the Adolescent
SPEP Review Period: December 1, 2020-November 30, 2021
SPEP Review Conducted: December 7, 2021
Florida Department of Juvenile Justice
Report Date: 2/18/22
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Skillstreaming the Adolescent.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
Skillstreaming the Adolescent employs a four-part training approach—modeling, role-playing,
performance feedback, and generalization—to teach essential prosocial skills to adolescents. The
curriculum provides a complete description of the Skillstreaming intervention, with instructions for
teaching 50 prosocial skills. According to documentation provided by the program, All facilitators
are trained in Skillstreaming the Adolescent prior to facilitating groups.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The Skillstreaming the Adolescent curiculum is a commercially developed, highly organized
textbook with fifty skills which provide both discussion and practice in each skill. The text provides
for role playing situations, skill cards, and discussions for each skill.
Skillstreaming the Adolescent groups operate as open groups and are facilitated on the living
modules.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Skillstreaming the Adolescent was observed during the review period. The group
took place in a classroom and was attended by seven youth, one facilitator, and one direct care
staff. The facilitator led the group in a discussion about short and long term goals. The goals
centered around career paths, and options if primary careers don't happen (ie NFL football, Rap
star). The youth were engaged as they explored possible interest in other fields that may lead to
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
long term employment. The group ended with a worksheet where the youth listed goals in one,
five and ten year increments. No behavioral issues were observed during the group.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program has had turnover in facilitators/therapists of Skillstreaming the Adolescent
during the review period; however, since it is an open group, no gap in service delivery occurred.
When a facilitator left a group, other facilitators covered the groups.
As the census count grew during the past year, more cohorts were initated. The program began
with two cohorts of youth in December 2020 and grew to six cohorts in April 2021. The program
continues to facilitate six cohorts of the primary service.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
According to documentation provided by the program, internal fidelity monitoring took place on a
limited basis for facilitators of Skillstreaming the Adolescent. Due to a low number of therapists on
site and the clinical director facilitating groups, four total internal fidelity monitoring sessions were
documented for the review period.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a policy for corrective action plans for facilitators of Skillstreaming the
Adolescent. In the event a facilitator is in need of help with service delivery, the director of case
management will sit with the facilitator and develop a plan to improve the facilitator's skills.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of annual evaluations documented the facilitators were given credit for their group
facilitation. The facilitators' skills in the delivery of Skillstreaming the Adolescent was evaluated by
supervisors.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 16 weeks for this type of service. Of the 5 youth in the
sample, 100% (5 of 5) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 5 youth in the
sample, 100% (5 of 5) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 10 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 40%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 60%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 1 youth
Moderate-High = 1 youth
High = 1 youth
Total Youth in Sample = 5 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 20%
High Score: 5 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 38%
Table 2
High = 1 youth
Total Youth in Sample = 5 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 60% (3 of 5) youth scored Moderate to High-Risk to reoffend, for a score of
5 points.
Of the SPEP sample, 20% (1 of 5) youth scored High-Risk to reoffend, for a score of 5 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 10 25 40%
Totals 70 85 82%
This SPEP report evaluates Skillstreaming the Adolescent, an intervention delivered at
Everglades Youth Academy Sex Offender Program.
8 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 5 total youth sampled, all 5
included dosage with end dates in the EBS Module. Of those youth, 5 received at least the
recommended weeks of service. Youth in the sample completed between 25 and 39 weeks of
service, with an average of 30 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 10 points for Amount of Service: Contact Hours. Of the 5 total youth
sampled, all 5 included dosage in the EBS Module. Of those youth, 5 received at least the
recommended hours of service. Youth in the sample completed 50 hours of service.
The program was awarded 10 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Based on the risk level of the youth placed at the program (40% Program Optimization
Percentage), the Department should work to ensure a larger proportion of higher risk youth are
recommended. As the program serves youth with a history of sexual behavior problems, this
includes a multitier strategy of working with stakeholders to achieve that practice, as well as
ensuring appropriate treatment services are available in the community to reduce the reliance on
residential commitment for lower risk youth presenting with sexually-related offenses.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 100%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Everglades Youth Academy Sex Offender Program can optimize their SPEP Quality of Service
Delivery score by ensuring facilitator turnover does not impact groups and fidelity monitoring is
completed every month for each facilitator.
Everglades Youth Academy Sex Offender Program can optimize their SPEP Amount of Service
score by ensuring that dosage for all youth is recorded accurately in EBS and by ensuring that
youth receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2021-22 Everglades YA SOP - Skillstreaming.pdf