facility_name: Daytona Juvenile Residential Facility

summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Daytona Juvenile Residential Facility
Rite of Passage, Inc.
(Contract Provider)
1386 Indian Lake Road
Daytona Beach, Florida 32124
Primary Service: Impact of Crime
SPEP Review Period: 10/1/2020-9/30/2021
SPEP Review Conducted: 11/3/2021
Florida Department of Juvenile Justice
Report Date: 11/18/2021
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes the Impact of Crime Curriculum (IOC) for their primary service. The
facilitators are trained by a qualified trainer to provide the primary service. The training is a three-
day, 24-hour training currently being provided online via Microsoft Teams. The documentation
reviewed for the annual review period of October 1, 2020 through September 30, 2021 shows
there were three staff trained in the primary service- one case manager, one shift supervisor, and
the Florida regional compliance director. The documentation provided shows one case manager
and one shift supervisor facilitated the cohorts during the annual review period. A trained staff
member conducts the monthly internal fidelity checks. One shift supervisor, another trained staff
member from another program, and the Florida regional compliance director conducted all of the
monthly internal fidelity checks during the annual review period.
The program's Standardized Program Evaluation Protocol procedure states the facilitator for each
primary service will have completed a formalized training for that primary service from a qualified
trainer. In order to enhance the continuity of the service due to unexpected facilitator absence and
provide more individualized instruction as needed during the service, when possible, Daytona
Juvenile Residential Facility will make all efforts to have a co-facilitator for the primary service.
The co-facilitator will have completed a formalized training for the primary service from a qualified
trainer.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the Impact of Crime: Addressing the Harm manual. The manual opens with
an introduction to the Impact of Crime Curriculum. Then the manual goes over all seven chapters
of the service and provides a script for the facilitator to follow. The manual also has an appendix
and a student guide with all the student exercises. The program makes copies of the student
exercises and passes them out to the youth during each lesson.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
The program conducts the primary service on Tuesdays and Thursdays. The program keeps all
SPEP documentation (attendance sheets, fidelity checks, and training certificates) in a binder
labeled "IOC SPEP documentation". All youth admitted to the program are required to participate
in the primary service. The primary service is a closed group with no more than twelve youth
assigned to one cohort.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Impact of Crime was observed on 6/24/2021. There were six youth present for the
group along with the facilitator and the clinical director. The group began at 2:35p. The group was
held in the clinical director’s office. The facilitator was knowledgeable of the material and followed
the lesson plan. The lesson for the day was on property crimes and their impacts. The facilitator
discussed the four victim impacts and explained misconceptions about insurance. The youth got
sidetracked during the group, but the facilitator was able to redirect them. The group ended at
3:28p.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of the documentation for the annual review period of October 1, 2020 through
September 30, 2021 shows there was facilitator turnover. The facilitator turnover did not adversely
affect the delivery of Impact of Crime. A case manager transferred to another program, but that
case manager was not facilitating any groups at the time of the transition. The program ran three
cohorts during the annual review period. The dates of the cohorts were 8/24/2020-11/24/2020,
11/19/2020-5/7/2021, and 5/18/2021-9/14/2021.
The cohort which began on 8/24/2020 ran groups on back-to-back days three times during the
cohort. The attendance sheets showed the days were 9/15/2020 and 9/16/2020; 9/28/2020,
9/29/2020, and 9/30/2020; and 11/16/2020 and 11/17/2020. Two of the three times were during
the previous annual review period. The Technical Assistance Specialist informed the facilitator
and the Florida regional compliance director there must be a day in between each session of the
primary service.
The cohort which began on 11/19/2020 began and ended with seven youth. The cohort was
stopped on 11/25/2020 due to a COVID-19 outbreak in the program. The youth were quarantined
and the staff were not allowed to conduct the groups until given the all clear. The director of
culture and operations stated the stoppage was to limit the spread of the virus in the program. The
youth in the cohort were from two different pods. The program did not want the youth from the two
different pods to be in the same group. The primary service resumed on 1/25/2021. The facilitator
skipped chapter two when the program restarted the cohort. The former clinical director stated
she caught it on 2/11/2021 and had the facilitator go back to chapter two on 2/15/2021. The
facilitator did not conduct the pre-test survey or post-test survey for this cohort.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The cohort which began on 5/18/2021 began and ended with six youth. The facilitator did not
conduct the pre-test survey for this cohort. Technical assistance specialist informed the facilitator
and clinical director that the pre-test and post-test surveys needed to be completed for all cohorts
moving forward.
According to the regional clinical director, the program’s protocol in the event of facilitator turnover
is to inform the department the reason and explanation for the turnover and also inform the
department of the gap if it was longer than a month. The program has a tracker and additional
trained staff in place to prevent any gaps in service.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation reviewed for the annual review period showed there were ten internal fidelity
monitoring checks completed. The date of the internal fidelity monitorings were 10/27/2020,
11/24/2020, 1/28/2021, 3/8/2021, 4/12/2021, 5/27/2021, 6/15/2021, 7/25/2021, 8/3/2021, and
9/12/2021. There was no internal fidelity monitoring completed for December 2020 due to the
program’s COVID-19 protocol in place. The program stopped the cohort at the end of November
2020 to attempt to limit the youth’s exposure to COVID-19. The program is missing a fidelity
check for the month of February 2021. The Florida regional compliance director, a shift
supervisor, and another trained facilitator from another program conducted the internal fidelity
checks during the annual review period.
The regional clinical director stated the program’s trained facilitator conducts the monthly internal
fidelity monitoring.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a policy in place to address corrective actions identified during monthly internal
fidelity monitoring. According to the regional clinical director, if there is an issue with services
being provided or fidelity monitoring being completed, the program will issue a corrective action
plan for correction in an appropriate time frame.
A review of the internal fidelity monitoring during the annual review period showed there were no
corrective actions needed.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of the annual staff evaluations during the review period of October 1, 2020 through
September 30, 2021 shows there is an area where the facilitators are evaluated on their
performance in facilitating the primary services. The program provided one case manager’s
annual evaluation. The evaluation provided information specific to the staff’s delivery of Impact of
Crime.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service - Duration
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates the target duration of 16 weeks for this type of service. Of the 15 youth in the
sample, 27% (4 of 15) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 24 contact hours for this type of service. Of the 15 youth in the
sample, 73% (11 of 15) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 2 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 8%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 40%
Moderate to High Score: 2 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 17%
Table 1
Moderate = 1 youth
Moderate-High = 3 youth
High = 2 youth
Total Youth in Sample = 15 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 13%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 2 youth
Total Youth in Sample = 15 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 40% (6 of 15) youth scored Moderate to High-Risk to reoffend, for a score
of 2 points.
Of the SPEP sample, 13% (2 of 15) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 2 10 20%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 2 25 8%
Totals 50 85 59%
This SPEP report evaluates Impact of Crime, an intervention delivered at Daytona Juvenile
Residential Facility.
30 youth completed the program during the review period. Of those youth, 15 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 2 points for Amount of Service: Duration. Of the 15 total youth sampled, only
10 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 4
received at least the recommended weeks of service. Youth in the sample completed between 14
and 25 weeks of service, with an average of 18 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program earned 6 points for Amount of Service: Contact Hours. Of the 15 total youth
sampled, only 11 included dosage in the EBS Module. Of those youth with correct dosage, 11
received at least the recommended hours of service. Youth in the sample completed between 24
and 32 hours of service, with an average of 30 hours.
The program was awarded 2 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Based on the risk level of the youth placed at the program (8% Program Optimization
Percentage), the Department should work to ensure a larger proportion of higher risk youth are
recommended. As the program serves youth with a history of sexual behavior problems, this
includes a multitier strategy of working with stakeholders to achieve that practice, as well as
ensuring appropriate treatment services are available in the community to reduce the reliance on
residential commitment for lower risk youth presenting with sexually-related offenses.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 86%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Daytona Juvenile Residential Facility can optimize their SPEP Quality of Service Delivery score
by ensuring fidelity monitoring is completed every month for each facilitator.
Daytona Juvenile Residential Facility can optimize their SPEP Amount of Service score by
ensuring that dosage for all youth is recorded accurately in EBS and by ensuring that youth
receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

report_type: SPEP

file_name: FY2021-22 Daytona IOC.pdf

program_name_raw: Daytona Juvenile Residential Facility

program_status_note: Open

service_type: Impact of Crime


state: FL

summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Daytona Juvenile Residential Facility
Rite of Passage, Inc.
(Contract Provider)
1386 Indian Lake Road
Daytona Beach, Florida 32124
Primary Service: Pathways: A Guided Workbook for Youth Beginning
Treatment
SPEP Review Period: 10/1/2020-9/30/2021
SPEP Review Conducted: 11/7/2021
Florida Department of Juvenile Justice
Report Date: 11/18/2021
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Pathways: A Guided Workbook for Youth Beginning
Treatment.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes Pathways: A Guided Workbook for Youth Beginning Treatment curriculum
for their primary service. The training is an in-service training provided by the clinical director or
other qualified trainer. The documentation reviewed for the annual review period of October 1,
2020 through September 30, 2021 shows there were four staff trained in the primary service. The
program had three trained staff from another facility assist the program with facilitating the primary
service. The four trained staff at the program were three therapists and the former clinical director.
The three trained staff who assisted the program were a clinical director and two therapists. The
documentation provided shows all seven trained staff facilitated the cohorts during the annual
review period. A trained staff member conducts the monthly internal fidelity checks. Three
therapists, the former clinical director, and the clinical director from another program conducted all
of the monthly internal fidelity checks during the annual review period.
The program's Standardized Program Evaluation Protocol procedure states the facilitator for each
primary service will have completed a formalized training for that primary service from a qualified
trainer. In order to enhance the continuity of the service due to unexpected facilitator absence and
provide more individualized instruction as needed during the service, when possible, Daytona
Juvenile Residential Facility will make all efforts to have a co-facilitator for the primary service.
The co-facilitator will have completed a formalized training for the primary service from a qualified
trainer.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the Pathways: A Guided Workbook for Youth Beginning Treatment 4th ed.
manual. The program also has Footprints and Roadmaps to Recovery if the youth is not on a high
school level. There is not a separate manual for facilitators and youth. The manual is broken down
into fourteen chapters. Each chapter covers a specific topic and there are activities for each topic
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
in each chapter. Pathways is run as a group and individually. Groups are moved through each
chapter in order from one through fourteen and then back to chapter one. During individual
sessions, Pathways protocol is given in order by the youth's level at the program. The program
conducts the primary service on Mondays, Wednesdays, and Fridays. The program keeps all
SPEP documentation (attendance sheets, fidelity checks, and training certificates) in a binder
labeled Pathways SPEP documentation. All youth admitted to the program are required to
participate in the primary service. The primary service is ran as an open group with no more than
ten youth assigned to one cohort.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Pathways: A Guided Workbook for Youth Beginning Treatment was observed on
6/24/2021. There were nine youth present for the group along with the facilitator and three staff.
The group began at 1:21p and was held in the day room. The Technical Assistance Specialist
was provided the lesson for the session observed. The facilitator was knowledgeable of the
material and followed the lesson plan. The lesson was continued from the last session. The youth
were asked to read different sections from the workbook. The youth provided good group
discussions and the facilitator asked probing questions to ensure the youth understood the
lesson. The group ended at 2:13p.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of the documentation for the annual review period of October 1, 2020 through
September 30, 2021 shows there was turnover of facilitators of Pathways: A Guided Workbook for
Youth Beginning Treatment. The program originally had the youth split into four groups. There
were several groups that utilized different facilitators during the annual review period. The
documentation for Group A shows the group had three facilitators running the group during the
review period. Group B had one consistent facilitator running the group during the review period.
Group C had three different facilitators running the group during the review period. Group D had
two different facilitators running the group during the review period. The former clinical director
merged the four groups into two groups (Group A merged with Group B to become Group B;
Group C merged with Group D to become Group C) in June 2021. One of the trained facilitators
left in November 2020. Two other therapists and the clinical director left during the annual review
period. The youth were transitioned to another trained facilitator with no gap in service. Pathways
is an open group and is run continually.
The program’s contract was updated as of July 1, 2021 to state the program needed licensed
therapists to run the primary service. The program utilized licensed therapists from another
program to ensure compliance with the contract.
According to the regional clinical director, the program’s protocol in the event of facilitator turnover
is to inform the department the reason and explanation for the turnover and also inform the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
department of the gap if it was longer than a month. The program has a tracker and additional
trained staff in place to prevent any gaps in service.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation reviewed for the annual review period showed there were thirty-one internal
fidelity monitoring checks completed for Pathways: A Guided Workbook for Youth Beginning
Treatment. The program is missing a fidelity check for the month of February 2021 for one
therapist, the month of June 2021 for two therapists and the former clinical director, and
September 2021 for the clinical director from another program. Three therapists, the former
clinical director, and the clinical director from another program conducted all of the monthly
internal fidelity checks during the annual review period.
The regional clinical director stated the program’s trained facilitator conducts the monthly internal
fidelity monitoring.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a policy to address corrective actions identified during monthly internal fidelity
monitoring. According to the regional clinical director, if there is an issue with services being
provided or fidelity monitoring being completed, the program will issue a corrective action plan for
correction in an appropriate time frame.
A review of the internal fidelity monitoring during the annual review period showed there were no
corrective actions needed for facilitators of Pathways: A Guided Workbook for Youth Beginning
Treatment.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of the annual staff evaluations during the annual review period of October 1, 2020
through September 30, 2021 shows there is an area where the facilitators are evaluated on their
performance in facilitating the primary services. The program provided one therapist’s annual
evaluation. The program also provided two staff’s evaluations who assisted with the delivery of
Pathways: A Guided Workbook for Youth Beginning Treatment, but who are not assigned to
Daytona Juvenile Residential Facility. The evaluation for the therapist employed by the program
provided information specific to the staff’s delivery of the primary service.
There was a note from the regional clinical director dated on 10/18/2021 which stated one of the
therapists and the former clinical director left the program in July 2021 prior to their yearly
evaluation, which was to be completed in August 2021.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 24 weeks for this type of service. Of the 19 youth in the
sample, 89% (17 of 19) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 40 contact hours for this type of service. Of the 19 youth in the
sample, 89% (17 of 19) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 7 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 28%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 42%
Moderate to High Score: 2 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 17%
Table 1
Moderate = 1 youth
Moderate-High = 3 youth
High = 4 youth
Total Youth in Sample = 19 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 21%
High Score: 5 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 38%
Table 2
High = 4 youth
Total Youth in Sample = 19 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 42% (8 of 19) youth scored Moderate to High-Risk to reoffend, for a score
of 2 points.
Of the SPEP sample, 21% (4 of 19) youth scored High-Risk to reoffend, for a score of 5 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 7 25 28%
Totals 73 95 77%
This SPEP report evaluates Pathways: A Guided Workbook for Youth Beginning Treatment, an
intervention delivered at Daytona Juvenile Residential Facility.
30 youth completed the program during the review period. Of those youth, 19 were included in the
sample. All youth received this service; however, 11 of 30 received the majority of this service
under another contracted provider and were excluded from this sample.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 19 total youth sampled, all
19 included dosage with end dates in the EBS Module. Of those youth, 17 received at least the
recommended weeks of service. Youth in the sample completed between 14 and 54 weeks of
service, with an average of 35 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program earned 8 points for Amount of Service: Contact Hours. Of the 19 total youth
sampled, all 19 included dosage in the EBS Module. Of those youth, 17 received at least the
recommended hours of service. Youth in the sample completed between 26 and 117 hours of
service, with an average of 73 hours.
The program was awarded 7 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Based on the risk level of the youth placed at the program (28% Program Optimization
Percentage), the Department should work to ensure a larger proportion of higher risk youth are
recommended. As the program serves youth with a history of sexual behavior problems, this
includes a multitier strategy of working with stakeholders to achieve that practice, as well as
ensuring appropriate treatment services are available in the community to reduce the reliance on
residential commitment for lower risk youth presenting with sexually-related offenses.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 80%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Daytona Juvenile Residential Facility can optimize their SPEP Quality of Service Delivery score
by ensuring facilitator turnover does not impact groups and fidelity monitoring is completed every
month for each facilitator.
Daytona Juvenile Residential Facility can optimize their SPEP Amount of Service score by
ensuring that dosage for all youth is recorded accurately in EBS and by ensuring that youth
receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

source: DJJ

file_name: FY2021-22 Daytona Pathways SO.pdf


summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Daytona Juvenile Residential Facility
Rite of Passage, Inc.
(Contract Provider)
1386 Indian Lake Road
Daytona Beach, Florida 32124
Primary Service: Pathways: A Guided Workbook for Youth Beginning
Treatment
SPEP Review Period: October 1, 2021-September 30, 2022
SPEP Review Conducted: November 4, 2022
Florida Department of Juvenile Justice
Report Date: 1/17/23
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Pathways: A Guided Workbook for Youth Beginning
Treatment.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
Documentation reviewed for the annual review period of October 1, 2021 through September 30,
2022 shows the program utilizes Pathways: A Guided Workbook for Youth Beginning Treatment
for their primary service. The training is an in-service training provided by the clinical director or
other qualified trainer. There were three program staff trained- two therapists and the clinical
director. Two trained staff from another facility assisted the program with facilitating the primary
service- one therapist and the other program’s clinical director. The documentation provided
shows all five trained staff facilitated the primary service during the annual review period.
A trained staff member conducts the monthly internal fidelity checks. The clinical director and the
clinical director from another program conducted the monthly internal fidelity checks during the
annual review period. The regional clinical director states it is best practice that the clinical
director conduct the monthly fidelity monitoring.
The program's Standardized Program Evaluation Protocol procedure states the facilitator for each
primary service will have completed a formalized training for that primary service from a qualified
trainer. In order to enhance the continuity of the service due to unexpected facilitator absence and
provide more individualized instruction as needed during the service, when possible, Daytona
Juvenile Residential Facility will make all efforts to have a co-facilitator for the primary service.
The co-facilitator will have completed a formalized training for the primary service from a qualified
trainer.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the Pathways: A Guided Workbook for Youth Beginning Treatment 4th ed.
manual. The program also has Footprints and Roadmaps to Recovery if the youth is not on a high
school level. There is not a separate manual for facilitators and youth as the manual is the same
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
for both the facilitator and the youth. The manual is broken down into fourteen (14) chapters. Each
chapter covers a specific topic and there are activities for each topic in each chapter. Pathways is
run as a group and individually. Groups are moved through each chapter in order from one
through fourteen (14) and then back to chapter one. During individual sessions, Pathways
protocol is given in order by the youth's level at the program.
The program conducts the primary service on Mondays, Wednesdays, and Fridays with no more
than ten (10) youth assigned to one cohort. The program keeps all SPEP documentation
(attendance sheets, fidelity checks, and training certificates) in a binder labeled "Pathways SPEP
documentation". All youth admitted to the program are required to participate in the primary
service.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Pathways: A Guided Workbook for Youth Beginning Treatment was observed via
Microsoft Teams on 10/12/2022. There were eight youth present for the group along with the
facilitator and the clinical director. The group began at 1:45p and was held in the main group
room. The facilitator was knowledgeable of the material and followed the lesson plan. The lesson
for the day was chapter eleven (11). The facilitator had the youth go back to chapter three and
read the sections because they lined up with the current session. The facilitator asked the youth
to provide their reasoning for going back to cover chapter three again. The youth went through the
Dear Becky letter to determine which statements were thinking errors or inappropriate statements.
All youth were engaged in the session. The group ended at 2:42p.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation for the annual review period of October 1, 2021 through September
30, 2022 shows there was no turnover of facilitators for Pathways: A Guided Workbook for Youth
Beginning Treatment. The program ran the primary service as an open group during the annual
review period, without any gaps in service.
There were twenty-one (21) attendance sheets which listed a youth receiving a supportive
session each of those days. Sixteen (16) of the twenty-one (21) attendance sheets show there
were ten (10) youth in those groups. The supportive sessions were run for fifteen (15) to twenty
(20) minutes each. The clinical director stated “the youth who received supportive sessions on all
of the dates identified with the exception of four dates are now released. The released youth did
have hours entered for each session and exceeded the dosage due to their length of stay. There
were three additional youth who had supportive sessions. One youth had a supportive as a new
intake due to our COVID-19 protocol at the time and he did not attend the group with the rest of
the youth. The other two youth were not in the group due to behavioral issues and causing
disruptions, so they received supportive sessions. The hours in the Evidence-Based Services
Module (EBS) reflect those youth having supportive sessions.”
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
According to the regional clinical director, the program’s protocol in the event of facilitator turnover
is to inform the department the reason and explanation for the turnover and also inform the
department of the gap if it was longer than a month. The program has a tracker and additional
trained staff in place to prevent any gaps in service.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation reviewed for the annual review period of October 1, 2021 through September
30, 2022 showed there were eighteen (18) internal fidelity monitoring checks completed for
facilitators of Pathways: A Guided Workbook for Youth Beginning Treatment. The clinical director
and another program’s clinical director conducted the internal fidelity checks. The program did not
have any missing fidelity checks during the annual review period.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
A review of the internal fidelity monitoring during the annual review period showed there were no
corrective actions noted nor taken for facilitators of Pathways: A Guided Workbook for Youth
Beginning Treatment.
According to the regional clinical director, if there is an issue with services being provided or
fidelity monitoring being completed, the program will issue a corrective action plan for correction in
an appropriate timeframe.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of the annual staff evaluations during the annual review period of October 1, 2021
through September 30, 2021 shows there is an area where the facilitators are evaluated on their
performance in facilitating Pathways: A Guided Workbook for Youth Beginning Treatment. The
program provided one therapist’s annual evaluation. The annual evaluation contained specific
information relating to the therapist’s delivery of the primary service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 24 weeks for this type of service. Of the 9 youth in the
sample, 100% (9 of 9) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 40 contact hours for this type of service. Of the 9 youth in the
sample, 100% (9 of 9) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 7 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 28%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 44%
Moderate to High Score: 2 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 17%
Table 1
Moderate = 1 youth
Moderate-High = 1 youth
High = 2 youth
Total Youth in Sample = 9 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 22%
High Score: 5 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 38%
Table 2
High = 2 youth
Total Youth in Sample = 9 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 44% (4 of 9) youth scored Moderate to High-Risk to reoffend, for a score of
2 points.
Of the SPEP sample, 22% (2 of 9) youth scored High-Risk to reoffend, for a score of 5 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 7 25 28%
Totals 77 95 81%
This SPEP report evaluates Pathways: A Guided Workbook for Youth Beginning Treatment, an
intervention delivered at Daytona Juvenile Residential Facility.
9 youth completed the program during the review period. Of those youth, all 9 were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 9 total youth sampled, all 9
included dosage with end dates in the EBS Module. Of those youth, 9 received at least the
recommended weeks of service. Youth in the sample completed between 46 and 79 weeks of
service, with an average of 57 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 10 points for Amount of Service: Contact Hours. Of the 9 total youth
sampled, all 9 included dosage in the EBS Module. Of those youth, 9 received at least the
recommended hours of service. Youth in the sample completed between 40 and 118 hours of
service, with an average of 81 hours.
The program was awarded 7 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Based on the risk level of the youth placed at the program (28% Program Optimization
Percentage), the Department should work to ensure a larger proportion of higher risk youth are
recommended. As the program serves youth with a history of sexual behavior problems, this
includes a multitier strategy of working with stakeholders to achieve that practice, as well as
ensuring appropriate treatment services are available in the community to reduce the reliance on
residential commitment for lower risk youth presenting with sexually-related offenses.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 100%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Daytona Juvenile Residential Facility can maintain their SPEP Quality of Service Delivery score
by continuing the practices in place at the time of this review.
Daytona Juvenile Residential Facility can optimize their SPEP Amount of Service score by
ensuring dosage for all youth is recorded accurately in EBS and by ensuring youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY22-23 Daytona JRF Pathways SO.pdf


summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Daytona Juvenile Residential Facility
Rite of Passage, Inc.
(Contract Provider)
1386 Indian Lake Road
Daytona Beach, Florida 32124
Primary Service: Aggression Replacement Training
SPEP Review Period: October 1, 2021-September 30, 2022
SPEP Review Conducted: November 4, 2022
Florida Department of Juvenile Justice
Report Date: 1/17/23
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Aggression Replacement Training.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The documentation reviewed for the annual review period of October 1, 2021 through September
30, 2022 shows the program utilizes Aggression Replacement Training (ART) for their primary
service. ART facilitator training is provided by a qualified trainer over a three-day period. There
were four staff trained in the primary service- two therapists, the clinical director, and the regional
clinical director. The documentation provided shows one therapist facilitated all three cohorts
during the annual review period. The regional clinical director, a therapist, and the clinical director
conducted the monthly internal fidelity checks during the annual review period. The regional
clinical director states it is best practice that the clinical director conduct the monthly fidelity
monitoring.
The program's Standardized Program Evaluation Protocol procedure states the facilitator for each
primary service will have completed a formalized training for that primary service from a qualified
trainer. In order to enhance the continuity of the service due to unexpected facilitator absence and
provide more individualized instruction as needed during the service, when possible, Daytona
Juvenile Residential Facility will make all efforts to have a co-facilitator for the primary service.
The co-facilitator will have completed a formalized training for the primary service from a qualified
trainer.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the Aggression Replacement Training (ART) manual. The manual is broken
down into six sections. The first section covers Skillstreaming, the second section deals with
Anger Control, the third section covers Moral Reasoning. Each of the first three sections has a
script for the facilitator to use for each lesson. Sections four through six are for the facilitator's
use. It includes training step guides, fidelity forms and hassle logs, and the program materials.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
The program does not have separate student workbooks to give out. The facilitators make copies
of the exercises before each session to pass out to the youth.
ART is delivered as a closed group over a minimum of ten (10) weeks for thirty (30) hours. The
program conducts ART on Sundays, Tuesdays, and Thursdays for one hour each session. The
program keeps all SPEP documentation (attendance sheets, fidelity checks, and training
certificates) in a binder labeled "Aggression Replacement Training (ART) SPEP documentation".
All youth admitted to the program are required to participate in the primary service.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Aggression Replacement Training was observed via Microsoft Teams on
10/19/2022. There were four youth present for the group along with the facilitator and the clinical
director. The group began at 1:50p and was held in the Therapist’s office. The facilitator was
knowledgeable of the material and followed the lesson plan. The lesson for the day was session
sixteen: Helping Others. The facilitator demonstrated the skill first before having the youth perform
their role plays. One youth took over the group for about fifteen (15) minutes talking about his
issues in the program. The facilitator did not immediately redirect the youth. The facilitator stated
during the debrief that she should have redirected the youth sooner. The facilitator did not assign
the youth a skill step to observe during the role plays. The group ended at 2:52p.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation for the annual review period of October 1, 2021 through September
30, 2022 shows there was no facilitator turnover. There were no gaps in service during a cohort of
Aggression Replacement Training.
The program completed two cohorts and there was one cohort ongoing at the time of the annual
review. The dates of the cohorts are: 11/30/2021-2/11/2022, 5/3/2022-7/8/2022, and 9/6/2022-
current (as of 9/30/2022).
The 11/30/2021 cohort began and ended with three youth. The 5/3/2022 cohort began and ended
with four youth. The 9/6/2022 cohort began with five youth and has four youth (as of 9/30/2022).
One youth was removed from the cohort due to persistent disruption of the group.
According to the regional clinical director, the program’s protocol in the event of facilitator turnover
is to inform the department the reason and explanation for the turnover and also inform the
department of the gap if it was longer than a month. The program has a tracker and additional
trained staff in place to prevent any gaps in service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Documentation reviewed for the annual review period of October 1, 2021 through September 30,
2022 showed there were twenty-one (21) internal fidelity monitoring checks completed for
facilitators of Aggression Replacement Training. The regional clinical director, clinical director, and
another program’s therapist conducted the internal fidelity checks. The program did not have any
missing fidelity checks during the annual review period. The regional clinical director stated the
program’s trained facilitators conduct the monthly internal fidelity monitoring.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
A review of internal fidelity monitoring for facilitators of Aggression Replacement Training showed
there were no corrective actions noted nor taken during the review period.
According to the regional clinical director, if there is an issue with services being provided or
fidelity monitoring being completed, the program will issue a corrective action plan for correction in
an appropriate time frame.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of the annual staff evaluations during the annual review period of October 1, 2021
through September 30, 2021 shows there is an area where the facilitators are evaluated on their
performance in facilitating Aggression Replacement Training (ART). The program provided one
therapist’s annual evaluation. The annual evaluation contained specific information relating to the
therapist’s delivery of ART. This therapist was the only facilitator for ART during the review period.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 10 weeks for this type of service. Of the 9 youth in the
sample, 89% (8 of 9) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 30 contact hours for this type of service. Of the 9 youth in the
sample, 89% (8 of 9) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 7 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 28%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 44%
Moderate to High Score: 2 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 17%
Table 1
Moderate = 1 youth
Moderate-High = 1 youth
High = 2 youth
Total Youth in Sample = 9 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 22%
High Score: 5 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 38%
Table 2
High = 2 youth
Total Youth in Sample = 9 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 44% (4 of 9) youth scored Moderate to High-Risk to reoffend, for a score of
2 points.
Of the SPEP sample, 22% (2 of 9) youth scored High-Risk to reoffend, for a score of 5 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 7 25 28%
Totals 78 100 78%
This SPEP report evaluates Aggression Replacement Training, an intervention delivered at
Daytona Juvenile Residential Facility.
9 youth completed the program during the review period. Of those youth, all 9 were included in
the sample.
The program scored High for Quality of Service Delivery.
The targeted number of weeks for this specific intervention is 10 weeks; the targeted number of
hours for this specific intervention is 30 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 5 services (15 weeks; 45
hours).
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 8 points for Amount of Service: Duration. Of the 9 total youth sampled, only
8 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 8
received at least the recommended weeks of service. Youth in the sample completed 11 weeks of
service.
The program earned 8 points for Amount of Service: Contact Hours. Of the 9 total youth sampled,
only 8 included dosage in the EBS Module. Of those youth with correct dosage, 8 received at
least the recommended hours of service. Youth in the sample completed 30 hours of service.
The program was awarded 7 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Based on the risk level of the youth placed at the program (28% Program Optimization
Percentage), the Department should work to ensure a larger proportion of higher risk youth are
recommended. As the program serves youth with a history of sexual behavior problems, this
includes a multitier strategy of working with stakeholders to achieve that practice, as well as
ensuring appropriate treatment services are available in the community to reduce the reliance on
residential commitment for lower risk youth presenting with sexually-related offenses.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 96%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Daytona Juvenile Residential Facility can maintain their SPEP Quality of Service Delivery score
by continuing the practices in place at the time of this review.
Daytona Juvenile Residential Facility can optimize their SPEP Amount of Service score by
ensuring dosage for all youth is recorded accurately in EBS and by ensuring youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY22-23 Daytona JRF ART.pdf

service_type: Aggression Replacement Training


summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Daytona Juvenile Residential Facility
Rite of Passage, Inc.
(Contract Provider)
1386 Indian Lake Road
Daytona Beach, Florida 32124
Primary Service: Impact of Crime
SPEP Review Period: October 1, 2021-September 30, 2022
SPEP Review Conducted: November 4, 2022
Florida Department of Juvenile Justice
Report Date: 1/17/23
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The documentation reviewed for the annual review period of October 1, 2021 through September
30, 2022 shows the program utilizes the Impact of Crime (IOC) Curriculum for their primary
service. The facilitators are trained by a qualified trainer to provide the primary service. The
training is a three-day, twenty-four (24)-hour training currently being provided online via Microsoft
Teams. There were six staff trained in IOC- two shift supervisors, a transition services
coordinator, the regional compliance director, and two case managers. The documentation
provided shows a shift supervisor and a case manager facilitated the cohorts during the annual
review period. A trained staff member conducts the monthly internal fidelity checks. The regional
compliance director, transition services coordinator, shift supervisor, and a case manager
conducted the monthly internal fidelity checks during the annual review period.
The program's Standardized Program Evaluation Protocol procedure states the facilitator for
each primary service will have completed a formalized training for that primary service from a
qualified trainer. In order to enhance the continuity of the service due to unexpected facilitator
absence and provide more individualized instruction as needed during the service, when possible,
Daytona Juvenile Residential Facility will make all efforts to have a co-facilitator for the primary
service. The co-facilitator will have completed a formalized training for the primary service from a
qualified trainer.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the Impact of Crime: Addressing the Harm manual. The manual opens with
an introduction to the Impact of Crime curriculum. Then the manual goes over all seven chapters
of the service and provides a script for the facilitator to follow. The manual also has an appendix
and a student guide with all the student exercises. The program makes copies of the student
exercises and passes them out to the youth during each lesson.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
The program conducts the primary service on Tuesdays and Sundays. The program keeps all
SPEP documentation (attendance sheets, fidelity checks, and training certificates) in a binder
labeled "IOC SPEP documentation". All youth admitted to the program are required to participate
in the primary service. The primary service is a closed group with no more than twelve (12) youth
assigned to one cohort.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Impact of Crime was observed via Microsoft Teams on 11/1/2022. There were seven
youth present for the group along with the facilitator and the clinical director. The group began at
4:20p. The group began thirty minutes later than the scheduled start time. The group was held in
the group room. The facilitator was knowledgeable of the material and followed the lesson plan.
The lesson for the day was the second half of chapter one. The facilitator discussed restorative
justice and went over the words to know. The facilitator had each youth write a definition of one of
the words to know in their own words. The group ended at 5:12p.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation for the annual review period of October 1, 2021 through September
30, 2022 shows there was no facilitator turnover. The program completed two cohorts of Impact of
Crime (IOC) during the annual review period. There were no unnecessary gaps in service during
a cohort of IOC.
The dates of the cohorts are 12/2/2021-4/1/2022 and 5/24/2022-9/22/2022. There was a gap in
service of seven weeks between the cohort which ended on 4/1/2022 and the cohort which began
on 5/24/2022. The clinical director stated the gap from April to May was due to not having enough
youth to start a new cohort. As of the last day of the cohort ending in April 2022, there was only 1
youth in need of IOC. The cohort beginning in May 2022 started with four youth (one of whom was
participating for a second time) rather than waiting any longer to have a larger group.
The 12/2/2021 cohort started and ended with three youth. The attendance sheet dated for
1/5/2022 shows the group was on chapter three objective three. The next attendance sheet was
dated on 1/13/2022 shows the group went to chapter four objective one. The facilitator went back
to chapter three objective four on 1/19/2022. The facilitator skipped ahead to chapter four
objective three on 1/27/2022 and then went back and covered chapter four objective two on
1/29/2022.
The 5/24/2022 cohort began and ended with five youth. The transition coordinator facilitated
groups on 6/7/2022, 6/9/2022, 7/26/2022, 8/9/2022, 8/11/2022, and 8/16/2022. The shift
supervisor facilitated groups on 7/28/2022, 8/30/2022, 9/6/2022, 9/12/2022, and 9/20/2022. The
documentation showed the facilitator skipped chapter seven objective two, but went back and
covered the lesson on the last day of the cohort.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
According to the regional clinical director, the program’s protocol in the event of facilitator turnover
is to inform the department the reason and explanation for the turnover and also inform the
department of the gap if it was longer than a month. The program has a tracker and additional
trained staff in place to prevent any gaps in service.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Documentation reviewed for the annual review period of October 1, 2021 through September 30,
2022 showed there were twelve (12) internal fidelity monitoring checks completed for facilitators of
Impact of Crime. The transition services coordinator, a shift supervisor, a case manager, and the
regional compliance director conducted the monthly internal fidelity checks during the annual
review period. The program is missing fidelity checks for the month of January 2022 for a shift
supervisor. The regional clinical director stated the program’s trained facilitators conduct the
monthly internal fidelity monitoring.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
A review of internal fidelity monitoring of Impact of Crime facilitators showed there were no
corrective actions noted nor taken during the annual review period.
According to the regional clinical director, if there is an issue with services being provided or
fidelity monitoring being completed, the program will issue a corrective action plan for correction in
an appropriate time frame.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of the annual staff evaluations during the annual review period of October 1, 2021
through September 30, 2021 shows there is an area where the facilitators are evaluated on their
performance in facilitating Impact of Crime (IOC). The program provided the annual evaluations
for two shift supervisors and a ninety (90)-day evaluation for the transition services coordinator.
The evaluations for the two shift supervisors briefly mentions IOC and provides a non-specific
statement about the staff’s delivery of the service. The evaluation for the transition services
coordinator does not mention IOC.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 16 weeks for this type of service. Of the 8 youth in the
sample, 100% (8 of 8) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 8 youth in the
sample, 100% (8 of 8) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 2 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 8%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 38%
Moderate to High Score: 2 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 17%
Table 1
Moderate = 1 youth
Moderate-High = 1 youth
High = 1 youth
Total Youth in Sample = 8 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 13%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 1 youth
Total Youth in Sample = 8 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 38% (3 of 8) youth scored Moderate to High-Risk to reoffend, for a score of
2 points.
Of the SPEP sample, 13% (1 of 8) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 2 25 8%
Totals 62 85 73%
This SPEP report evaluates Impact of Crime, an intervention delivered at Daytona Juvenile
Residential Facility.
9 youth completed the program during the review period. Of those youth, 8 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 8 total youth sampled, all 8
included dosage with end dates in the EBS Module. Of those youth, all 8 received at least the
recommended weeks of service. Youth in the sample completed between 17 and 25 weeks of
service, with an average of 20 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program earned 10 points for Amount of Service: Contact Hours. Of the 8 total youth
sampled, all 8 included dosage in the EBS Module. Of those youth, all 8 received at least the
recommended hours of service. Youth in the sample completed between 24 and 32 hours of
service, with an average of 27 hours.
The program was awarded 2 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Based on the risk level of the youth placed at the program (8% Program Optimization
Percentage), the Department should work to ensure a larger proportion of higher risk youth are
recommended. As the program serves youth with a history of sexual behavior problems, this
includes a multitier strategy of working with stakeholders to achieve that practice, as well as
ensuring appropriate treatment services are available in the community to reduce the reliance on
residential commitment for lower risk youth presenting with sexually-related offenses.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 100%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Daytona Juvenile Residential Facility can optimize their SPEP Quality of Service Delivery score
by ensuring fidelity monitoring is completed every month for each facilitator and each facilitator’s
performance evaluation includes discussion of their skill in delivering this primary service.
Daytona Juvenile Residential Facility can optimize their SPEP Amount of Service score by
ensuring dosage for all youth is recorded accurately in EBS and by ensuring youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

file_name: FY22-23 Daytona JRF IOC.pdf