state: FL

facility_name: Everglades Youth Academy

summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Everglades Youth Academy Sex Offender Program
Youth Opportunity Investments, LLC
(Contract Provider)
5050 NE 168th Street
Okeechobee, Florida 34972
Primary Service: Pathways: A Guided Workbook for Youth Beginning
Treatment
SPEP Review Period: December 1, 2022-November 30, 2023
SPEP Review Conducted: December 5-6, 2023
Florida Department of Juvenile Justice
Report Date: 6/25/2024
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Pathways: A Guided Workbook for Youth Beginning
Treatment.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The documentation reviewed for the annual review period of December 1, 2022 through
November 30, 2023 shows the program uses Pathways: A Guided Workbook for Youth Beginning
Treatment as a primary service. The curriculum is designed to help juvenile offenders develop
responsible patterns of thinking and behavior, as well as to provide guidance for how to control
sexual feelings and cultivate healthy friendships and dating relationships. Pathways groups are
conducted by therapists who have been formally trained in the primary service. The program
documentation indicates one clinical director, one assistant clinical director and one state director
of clinical services have received the formal training to provide this service. There were two staff
members who facilitated Pathways-one clinical director and one assistant clinical director. The
clinical director or state director of clinical services conducted the monthly internal fidelity
monitoring for this service. The program provided documentation of all three facilitators having
completed training to provide this service.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The documentation reviewed for the annual review period of December 1, 2022 through
November 30, 2023 shows the program uses the Pathways: A Guided Workbook for Youth
Beginning Treatment 4th edition textbook. The curriculum is comprised of fourteen (14) chapters.
Each lesson includes topics for group discussion, group readings, and worksheets developed to
reinforce each topic for the participants. There is a formal facilitator’s manual but no separate
student manual available for this service. Pathways is designed to be an open group and group
sessions are held year-round. Every youth who enters the program is immediately enrolled in the
service and are required to participate in the service as a part of their individual treatment plan.
Pathways groups were held twice weekly on Mondays and Wednesdays until October 2023, when
the group schedule transitioned to Tuesdays and Thursdays from 1:20 p.m.-2:20 p.m. and 2:30
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
p.m.-3:30 p.m. The attendance sheets are maintained by the program in a separate binder
labeled “Pathways.”
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Pathways: A Guided Workbook for Youth Beginning Treatment was observed on-site
on October 17, 2023. The session was conducted in a classroom setting with one facilitator, 9
youth, and one direct care staff. The session began with a check-in and an ice breaker, followed
by the youth sharing something they learned in the previous lesson. The facilitator led a group
discussion which included youth reading passages from the student handout titled “Controlling
and Expressing My Sexual Feelings in Positive Ways.” The facilitator utilized the manual
throughout the session, conveyed the information in his own words, and was knowledgeable
about the material being presented to the group. A copy of the lesson and the student handout
were provided to this technical assistance specialist to review and follow along. There were no
behavior issues observed during the session and all youth actively participated in the group
discussion.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The documentation reviewed for the annual review period of December 1, 2022 through
November 30, 2023 shows there was one instance of facilitator turnover for Pathways: A Guided
Workbook for Youth Beginning Treatment. The assistant clinical director left the program in
January 2023. This turnover did adversely affect the delivery of the service. Once the assistant
clinical director left the program, the clinical director began facilitating the service. The clinical
director explained if a facilitator should leave the program during a cohort, another trained
therapist will then take over the group. In the absence of another trained facilitator on staff at the
program, the state director of clinical services would then fill in as a facilitator. There were no gaps
in service during the annual review period.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation reviewed for the annual review period of December 1, 2022 through
November 30, 2023 shows fidelity checks were conducted monthly, except for one fidelity check
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
for the clinical director for the month of January 2023. The internal fidelity monitoring was
completed by the clinical director and the state director of clinical services. Both were formally
trained to facilitate Pathways. There was a total of twelve (12) of the thirteen (13) required fidelity
checks documented for the annual review period. The documentation for the monthly fidelity
checks is maintained by the program in a separate binder labeled “Fidelity Monitoring."
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The documentation reviewed for the annual review period of December 1, 2022 through
November 30, 2023 shows the program has a system in place to address the need for any
corrective actions discovered during the monthly internal fidelity monitoring. According to the
clinical director, facilitators in need of corrective action related to their delivery of the service will
receive progressive discipline in the form of coaching, additional training, and formal write-up(s).
There were no corrective actions taken during the annual review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Copies of the annual employee evaluations were not provided to the technical assistance
specialist for this annual review period of December 1, 2022 through November 30, 2023.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 24 weeks for this type of service. Of the 15 youth in the
sample, 93% (14 of 15) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 40 contact hours for this type of service. Of the 15 youth in the
sample, 93% (14 of 15) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 2 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 8%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 40%
Moderate to High Score: 2 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 17%
Table 1
Moderate = 2 youth
Moderate-High = 2 youth
High = 2 youth
Total Youth in Sample = 15 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 13%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 2 youth
Total Youth in Sample = 15 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 40% (6 of 15) youth scored Moderate to High-Risk to reoffend, for a score
of 2 points.
Of the SPEP sample, 13% (2 of 15) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 2 25 8%
Totals 58 95 61%
This SPEP report evaluates Pathways: A Guided Workbook for Youth Beginning Treatment, an
intervention delivered at Everglades Youth Academy Sex Offender Program.
15 youth completed the program during the review period. Of those youth, 15 were included in the
sample.
The program scored Medium for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 15 total youth sampled, only
14 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 14
received at least the recommended weeks of service. Youth in the sample completed between 52
and 93 weeks of service, with an average of 69 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 8 points for Amount of Service: Contact Hours. Of the 15 total youth
sampled, only 14 included dosage in the EBS Module. Of those youth with correct dosage, 14
received at least the recommended hours of service. Youth in the sample completed between 52
and 158 hours of service, with an average of 99 hours.
The program was awarded 2 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Based on the risk level of the youth placed at the program (8% Program Optimization
Percentage), the Department should work to ensure a larger proportion of higher risk youth are
recommended. As the program serves youth with a history of sexual behavior problems, this
includes a multitier strategy of working with stakeholders to achieve that practice, as well as
ensuring appropriate treatment services are available in the community to reduce the reliance on
residential commitment for lower risk youth presenting with sexually-related offenses.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 85%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Everglades Youth Academy Sex Offender Program can optimize their SPEP Quality of Service
Delivery score by ensuring facilitator turnover does not impact groups, fidelity monitoring is
completed every month for each facilitator, and each facilitator’s performance evaluation includes
discussion of their skill in delivering this primary service.
Everglades Youth Academy Sex Offender Program can optimize their SPEP Amount of Service
score by ensuring dosage for all youth is recorded accurately in EBS and by ensuring youth
receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

source: DJJ

report_type: SPEP

file_name: FY2023-24 EYA Pathways.pdf

program_status_note: Open

service_type: Pathways to Self‐Discovery and Change


summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Everglades Youth Academy (Maximum Risk)
Youth Opportunity Investments, LLC.
(Contract Provider)
5050 NE 168th Street
Okeechobee, Florida 34972
Primary Service: Impact of Crime
SPEP Review Period: December 1, 2022-November 30, 2023
SPEP Review Conducted: December 5-6, 2023
Florida Department of Juvenile Justice
Report Date: 6/25/2024
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 6 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The documentation for the annual review period of December 1, 2022 through November 30,
2023 shows the program utilizes Impact of Crime as a primary service. Facilitator training consists
of a three-day, twenty-four (24) hour training. This training is available to new facilitators via
Microsoft Teams. There were three trained in the primary service during this review period: one
director of case management and two case managers. The following trained staff have facilitated
Impact of Crime: one director of case management and one case manager. Internal fidelity
monitoring checks were conducted by the director of case management.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the Impact of Crime: Addressing the Harm manual. The manual includes seven
chapters and provides a script for the facilitator to follow. The curriculum includes a student guide
with exercises for the participants to complete which are aimed at reinforcing the material
presented during group sessions. The program does not assign a student guide to each individual
participant, rather the facilitator makes copies of the exercises.The program conducts Impact of
Crime on Mondays and Wednesdays. The program maintains all SPEP documentation
(attendance sheets and internal fidelity checks) in a separate binder labeled Impact of Crime. All
youth admitted to the program are required to participate in this primary service. Impact of Crime
is a closed group with no more than twelve (12) youth assigned to a cohort.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Technical assistance specialist observed a session of Impact of Crime on 12/6/2023. The group
began at 2:27 p.m and was held in a classroom. The facilitator was prepared with a copy of the
facilitator’s guide, handouts for the group participants, and a copy of the lesson plan/handout for
the technical assistance specialist. The group began with an ice breaker, followed by a review of
the previous lesson. The current lesson was presented by the facilitator and the youth took turns
reading from their handouts. The group participated in a discussion about the economic impacts
of crime. There were no behavioral issues. The group ended at 3:00 p.m.
Basic Score: 0 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The documentation for the annual review period of December 1, 2022 through November 30,
2023 shows the program has experienced facilitator turnover. This turnover did adversely affect
the delivery of Impact of Crime. One case manager left the program in May 2022. This case
manager had served as a co-facilitator for this cohort, so the group continued with the main
facilitator. Another case manager left the program in November 2023. At this time, the director of
case management took over the group.
There was a gap in service during the annual review period. An informal interview with the director
of case management confirmed there were no cohorts running during the first quarter of the
annual review period. The director of case management confirmed this gap was caused by a
combination of a lack of trained facilitators and a lack of youth in the population that needed this
primary service.
The first cohort began on May 2, 2023 with ten (10) youth and was completed on August 22, 2023
with ten (10) youth.
The second cohort began on October 30, 2023 with eight youth and was ongoing at the time of
the annual review with six youth. One youth was dropped from the cohort due to physical
aggression and one youth was dropped due to the youth’s refusal to participate in the group.
The director of case management explained the program will replace one facilitator with another
qualified facilitator should there be any staff turnover in the middle of a cohort.
The program can earn 2 points if facilitator turnover does not impact groups and there are no
unnecessary gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation for the annual review period of December 1, 2022 through November 30,
2023 showed there were six out of the required seven internal fidelity monitoring checks
completed. There was one internal fidelity monitoring check missing from November 2023.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
According to the director of case management, either the director of case management or another
trained facilitator conducts the monthly internal fidelity monitoring checks for this primary service.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The documentation reviewed for the annual review period of December 1, 2022 through
November 30, 2023 showed there were no corrective actions identified for the facilitators of
Impact of Crime. The director of case management explained if there is a need for corrective
action, the facilitator will receive progressive discipline to include coaching, retraining, and formal
write-up(s).
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Copies of the annual employee evaluations were not provided to the technical assistance
specialist for the annual review period of December 1, 2022 through November 30, 2023.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 16 weeks for this type of service. Of the 20 youth in the
sample, 100% (20 of 20) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 20 youth in the
sample, 100% (20 of 20) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 18 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 72%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 65%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 2 youth
Moderate-High = 4 youth
High = 7 youth
Total Youth in Sample = 20 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 35%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 7 youth
Total Youth in Sample = 20 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 65% (13 of 20) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 35% (7 of 20) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 18 25 72%
Totals 68 85 80%
This SPEP report evaluates Impact of Crime, an intervention delivered at Everglades Youth
Academy (Maximum Risk).
26 youth completed the program during the review period. Of those youth, 20 were included in the
sample.
The program scored Medium for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 20 total youth sampled, all
20 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 20
received at least the recommended weeks of service. Youth in the sample completed between 17
and 20 weeks of service, with an average of 19 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 10 points for Amount of Service: Contact Hours. Of the 20 total youth
sampled, all 20 included dosage in the EBS Module. Of those youth with correct dosage, 20
received at least the recommended hours of service. Youth in the sample completed between 32
and 36 hours of service, with an average of 35 hours.
The program was awarded 18 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 88%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Everglades Youth Academy (Maximum Risk) can optimize their SPEP Quality of Service Delivery
score by ensuring facilitator turnover does not impact groups, there are no gaps in service, fidelity
monitoring is completed every month for each facilitator, and each facilitator’s performance
evaluation includes discussion of their skill in delivering this primary service.
Everglades Youth Academy (Maximum Risk) can maintain their SPEP Amount of Service score
by continuing to ensure dosage for all youth is recorded accurately in EBS and by ensuring youth
receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2023-24 EYA-MAX IOC.pdf

program_name_raw: Everglades Youth Academy

service_type: Impact of Crime


summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Everglades Youth Academy (Maximum Risk)
Youth Opportunity Investments, LLC
(Contract Provider)
5050 NE 168th Street
Okeechobee, Florida 34972
Primary Service: Moral Reconation Therapy
SPEP Review Period: December 1, 2022-November 30, 2023
SPEP Review Conducted: December 5-6, 2023
Florida Department of Juvenile Justice
Report Date: 6/26/2024
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Moral Reconation Therapy.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The documentation for the annual review period of December 1, 2022 through November 30,
2023 shows that the program uses Moral Reconation Therapy (MRT) as a primary service. The
curriculum is designed to help juvenile offenders reexamine their choices and develop appropriate
decision-making skills. MRT groups are conducted by therapists who have been formally trained
in the service. MRT training is conducted by a qualified trainer either in-person or virtually. The
documentation shows there were nine staff members who facilitated MRT during the review
period: one clinical director, one assistant clinical director, and seven therapists. The clinical
director and one therapist conducted the monthly internal fidelity monitoring for this service.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The documentation for the annual review period of December 1, 2022 through November 30,
2023 shows the program uses the Moral Reconation Therapy curriculum, Juvenile MRT-How to
Escape Your Prison. There is a formal facilitator’s manual and separate student manuals. Moral
Reconation Therapy is designed to be an open group and group sessions are held year-round.
Every youth who enters the program is immediately enrolled in the service based on their
assigned living modules. The program maintains the attendance sheets in a SPEP binder labeled
“Moral Reconation Therapy.” Documentation of internal fidelity monitoring is maintained in a
separate binder labeled “Fidelity Monitoring.” MRT groups are held twice weekly on Mondays and
Wednesdays at 1:20 p.m and 2:20 p.m.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
An observation of a Moral Reconation Therapy group took place on-site on July 12, 2023. The
group consisted of one facilitator, eight youth, and two direct care staff. The group was held in a
classroom. The facilitator used the facilitator’s manual throughout this group session and was
knowledgeable about the material being presented to the group. The facilitator used her own
words to convey the information. This session began with an ice breaker, followed by a review of
the previous lesson and a review of the homework assigned during the last group. The group
actively participated in a discussion about “The Pyramid of the Present.” The facilitator utilized
effective motivational interviewing skills to encourage the youth to share more about their
personal experiences and to assess for understanding. There were no behavioral issues
observed during this group.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The documentation reviewed for the annual review period of December 1, 2022 through
November 30, 2023 shows the program had facilitator turnover (one assistant clinical director and
four therapists). This turnover did adversely affect the delivery of this primary service. The
affected groups were reassigned to a new facilitator immediately. There were no gaps in service
during this annual review period.
An informal interview with the clinical director confirmed that if a facilitator should leave the
program, another trained therapist will then take over the group. The program provided
documentation of four on-going cohorts for this service during the review period.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of the documentation for the annual review period of December 1, 2022 through
November 30, 2023 shows a total of fifty (50) fidelity checks conducted by the clinical director and
one therapist for the staff members facilitating Moral Reconation Therapy groups. The clinical
director and the therapist were formally trained in this primary service.
Documentation further shows that internal fidelity monitoring was conducted monthly for all
facilitators.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The documentation reviewed for the annual review period of December 1, 2022 through
November 30, 2023 shows the program has a system in place to address the need for corrective
actions identified during monthly internal fidelity monitoring of Moral Reconation Therapy.
According to the clinical director, facilitators in need of corrective action related to their delivery of
the service will receive progressive discipline in the form of coaching, additional training, and
formal write-ups.
A review of the documentation shows there were no corrective actions identified during the annual
review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Copies of the annual employee evaluations and/or ninety (90) day reviews were not provided to
the technical assistance specialist for this annual review period of December 1, 2022 through
November 30, 2023.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 15 weeks for this type of service. Of the 26 youth in the
sample, 92% (24 of 26) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 45 contact hours for this type of service. Of the 26 youth in the
sample, 27% (7 of 26) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 18 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 72%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 58%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 2 youth
Moderate-High = 4 youth
High = 9 youth
Total Youth in Sample = 26 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 35%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 9 youth
Total Youth in Sample = 26 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 58% (15 of 26) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 35% (9 of 26) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 18 25 72%
Totals 81 100 81%
This SPEP report evaluates Moral Reconation Therapy, an intervention delivered at Everglades
Youth Academy (Maximum Risk).
26 youth completed the program during the review period. Of those youth, 26 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 26 total youth sampled, only
25 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 24
received at least the recommended weeks of service. Youth in the sample completed between 3
and 100 weeks of service, with an average of 50 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 0 points for Amount of Service: Contact Hours. Of the 26 total youth
sampled, all 26 included dosage in the EBS Module. Of those youth with correct dosage, 7
received at least the recommended hours of service. Youth in the sample completed between 16
and 122 hours of service, with an average of 43 hours.
The program was awarded 18 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 88%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Everglades Youth Academy (Maximum Risk) can optimize their SPEP Quality of Service Delivery
score by ensuring facilitator turnover does not impact groups and each facilitator’s performance
evaluation includes discussion of their skill in delivering this primary service.
Everglades Youth Academy (Maximum Risk) can optimize their SPEP Amount of Service score
by ensuring dosage for all youth is recorded accurately in EBS and by ensuring youth receive the
full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2023-24 EYA-MAX MRT.pdf

service_type: Moral Reconation Therapy


summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Everglades Youth Academy Sex Offender Program
Youth Opportunity Investments, LLC
(Contract Provider)
5050 NE 168th Street
Okeechobee, Florida 34972
Primary Service: Impact of Crime
SPEP Review Period: December 1, 2022-November 30, 2023
SPEP Review Conducted: December 5-6, 2023
Florida Department of Juvenile Justice
Report Date: 6/25/2024
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The documentation reviewed for the annual review period of December 1, 2022 through
November 30, 2023 shows the program uses Impact of Crime as a primary service. Facilitator
training consists of a three-day, twenty-four (24) hour training. This training is available to new
facilitators via Microsoft Teams. Three staff members are trained in the primary service: one
director of case management, one transitional service manager, and one case manager. The
transitional service manager and case manager facilitated Impact of Crime. Any facilitator who is
trained in Impact of Crime can conduct the internal fidelity monitoring. Internal fidelity monitoring
checks were conducted by the director of case management, transitional service manager, and
one therapist.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The documentation reviewed for the annual review period of December 1, 2022 through
November 30, 2023 shows the program uses the Impact of Crime: Addressing the Harm manual.
The manual includes seven chapters and provides a script for the facilitator to follow. The
curriculum includes a student guide with exercises for the participants to complete which are
aimed at reinforcing the material presented during group sessions. The program does not assign
a student guide to each individual participant, rather the facilitator makes copies of the exercises
and provides one to each youth during the lesson. The director of case management indicated the
program conducts Impact of Crime on Tuesdays and Thursdays. The program maintains all SPEP
documentation (attendance sheets and internal fidelity checks) in a separate binder labeled
Impact of Crime. All youth admitted to the program are required to participate in this primary
service. Impact of Crime is a closed group with no more than twelve (12) youth assigned to a
cohort.
c. Observed Adherence to the Manual/Protocol Basic Score: 1 Point(s)
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Technical assistance specialist observed a session of Impact of Crime on-site on July 11, 2023.
The group began at 1:30 p.m. and was held in a classroom. A copy of the lesson plan was not
provided to the technical assistance specialist for review. There were four youth, one facilitator,
and one direct care staff present. The group began with an ice breaker activity and a review of the
previous lesson before proceeding to the lesson of the day. The facilitator did not have a copy of
the facilitator’s manual available during this session. The group engaged in a discussion about the
ripple effect of crime. Technical assistance specialist observed no behavior management issues.
The group ended at 2:30 p.m.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The documentation reviewed for the annual review period of December 1, 2022 through
November 30, 2023 shows the program has experienced facilitator turnover. This turnover did
adversely affect the delivery of Impact of Crime. One case manager left the program in June 2023
in the middle of a cohort. The transition service manager took over as the facilitator for this cohort.
There were no gaps in service during the annual review period caused by a lack of trained
facilitators.
The first cohort began on November 10, 2022 with eleven (11) youth and ended on March 9, 2023
with seven youth. Three youth were removed from the group due to being arrested, and one youth
was discharged from the program prior to the completion of the cohort.
The second cohort began on May 9, 2023 with seven youth and ended on August 24, 2023 with
four youth. Three youth were dropped from the cohort due to one youth being transferred out of
the program and two youth refusing to participate in the service.
The third cohort began on October 31, 2023 with four youth and was ongoing at the time of the
annual review with four youth.
An informal interview with the director of case management confirmed that the program will
replace one facilitator with another qualified facilitator should there be any staff turnover in the
middle of a cohort.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The documentation reviewed for the annual review period of December 1, 2022 through
November 30, 2023 shows there were eight out of the required ten (10) internal fidelity monitoring
checks completed for the facilitators of Impact of Crime. Documentation of fidelity checks were not
provided for November 2022 and March 2023.
According to the director of case management, either the director of case management or another
trained facilitator conducts the monthly internal fidelity monitoring checks for this primary service.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
A review of the documentation for the annual review period of December 1, 2022 through
November 30, 2023 showed there was no need for corrective actions identified for the facilitators
of Impact of Crime given during the review period of 12/1/22 through 11/30/23. During an informal
interview with the director of case management, it was explained that if a need for corrective
action is found, the facilitator will receive progressive discipline to include coaching, retraining,
and formal write-up(s).
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Copies of the annual employee evaluations were not provided to the technical assistance
specialist for the annual review period of December 1, 2022 through November 30, 2023.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 16 weeks for this type of service. Of the 15 youth in the
sample, 93% (14 of 15) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 24 contact hours for this type of service. Of the 15 youth in the
sample, 93% (14 of 15) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 2 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 8%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 40%
Moderate to High Score: 2 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 17%
Table 1
Moderate = 2 youth
Moderate-High = 2 youth
High = 2 youth
Total Youth in Sample = 15 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 13%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 2 youth
Total Youth in Sample = 15 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 40% (6 of 15) youth scored Moderate to High-Risk to reoffend, for a score
of 2 points.
Of the SPEP sample, 13% (2 of 15) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 2 25 8%
Totals 48 85 56%
This SPEP report evaluates Impact of Crime, an intervention delivered at Everglades Youth
Academy Sex Offender Program.
15 youth completed the program during the review period. Of those youth, 15 were included in the
sample.
The program scored Medium for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 15 total youth sampled, only
14 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 14
received at least the recommended weeks of service. Youth in the sample completed between 17
and 18 weeks of service, with an average of 18 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 8 points for Amount of Service: Contact Hours. Of the 15 total youth
sampled, only 14 included dosage in the EBS Module. Of those youth with correct dosage, 14
received at least the recommended hours of service. Youth in the sample completed between 33
and 35 hours of service, with an average of 34 hours.
The program was awarded 2 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Based on the risk level of the youth placed at the program (8% Program Optimization
Percentage), the Department should work to ensure a larger proportion of higher risk youth are
recommended. As the program serves youth with a history of sexual behavior problems, this
includes a multitier strategy of working with stakeholders to achieve that practice, as well as
ensuring appropriate treatment services are available in the community to reduce the reliance on
residential commitment for lower risk youth presenting with sexually-related offenses.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 84%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Everglades Youth Academy Sex Offender Program can optimize their SPEP Quality of Service
Delivery score by ensuring facilitator turnover does not impact groups, fidelity monitoring is
completed every month for each facilitator, and each facilitator’s performance evaluation includes
discussion of their skill in delivering this primary service.
Everglades Youth Academy Sex Offender Program can optimize their SPEP Amount of Service
score by ensuring dosage for all youth is recorded accurately in EBS and by ensuring youth
receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2023-24 EYA-SOP IOC.pdf