state: FL

facility_name: Daytona Juvenile Residential Facility

summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Daytona Juvenile Residential Facility
Rite of Passage, Inc.
(Contract Provider)
1386 Indian Lake Road
Daytona Beach, Florida 32124
Primary Service: Agression Replacement Training
SPEP Review Period: October 1, 2022 to September 30, 2023
SPEP Review Conducted: October 4, 2023
Florida Department of Juvenile Justice
Report Date: May 1, 2024
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Aggression Replacement Training.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The documentation reviewed for the annual review period of October 1, 2023 - September 30,
2023 shows the program uses Aggression Replacement Training (ART) for their primary service.
ART facilitator training is provided by a qualified trainer over a three-day period. There are four
staff trained in the primary service - two therapists, the clinical director, and the director of clinical
services. The two therapists facilitated the cohorts. A trained staff member conducts the monthly
internal fidelity checks. The director of clinical services, a therapist, and the clinical director
conducted the monthly internal fidelity checks during the annual review period. The director of
clinical services confirmed it is best practice that the clinical director conducts the monthly fidelity
monitoring.
The program's Standardized Program Evaluation Protocol procedure states that the facilitator for
each primary service will have completed a formalized training for that primary service from a
qualified trainer. Daytona Juvenile Residential Facility will make all efforts to have a co-facilitator
for the primary service. The co-facilitator will have completed a formalized training for the primary
service from a qualified trainer.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the Aggression Replacement Training (ART) manual. The manual is broken
down into six sections. The first section covers Skillstreaming, the second section deals with
Anger Control, the third section covers Moral Reasoning. Each of the first three sections has a
script for the facilitator to use for each lesson. Sections four through six are for the facilitator's
use. It includes training step guides, fidelity forms and hassle logs, and the program materials.
The program does not have separate student workbooks to give out. The facilitators make copies
of the exercises before each session to pass out to the youth.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
ART is delivered as a closed group over a minimum of ten (10) weeks for thirty (30) hours. The
program conducts ART on Sundays, Tuesdays, and Thursdays for one hour each session. The
program keeps all SPEP documentation (attendance sheets, fidelity checks, and training
certificates) in a binder labeled "Aggression Replacement Training (ART) SPEP documentation".
All youth admitted to the program are required to participate in the primary service.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Aggression Replacement Training (ART) was observed on 6/21/2023. There were
four youth present for the group along with the facilitator. A coach counselor showed up after
group began. The group began at 1:45p. The facilitator began group with check in. The youth then
went over their homework from last group. The facilitator was engaging and was making the youth
laugh. Facilitator made sure all youth participated and the youth showed eagerness to do so.
Youth were encouraging others to participate and were actively engaged during role-play. The
facilitator made the youth state the steps for the expressing affection skill from memory. The youth
then went over the full Anger Control Chain. The facilitator was knowledgeable of the material and
provided it in a way the youth understood the concepts. Facilitator effectively redirected and de-
escalated when needed. The facilitator reminded the youth to complete their hassle logs and to
get ready for the upcoming “test.” The youth concluded group with the close out. Group ended at
2:50p.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation for Aggression Replacement Training (ART) for the annual review
period of October 1, 2023 - September 30, 2023 shows there was no facilitator turnover. The
program completed three cohorts and has one cohort currently ongoing at the end of the review
period.
The first cohort began on 9/6/2022 with five youth and ended 11/18/2022 with four youth. One
youth was dropped from the group on 9/15/2022 due to persistent disruption of sessions. One
session dated 11/17/2023 did not have an end time.
The second cohort was held 12/8/2022 through 2/17/2023. This cohort began and ended with
seven youth.
The third cohort began 4/26/2023 and ended 7/5/2023 with four youth.
The fourth cohort began 8/20/2023 with six youth and was ongoing at the time of the anndual
review with five youth. One youth was dropped from the group on 9/25/2023 due to aggressive
behavior.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
There was another note stating there was a gap in service of longer than two weeks due to the
census being low. The gap occurred between the cohort which ended on 2/17/2023 and the new
cohort which began on 4/26/2023. This does not qualify as an unnecessary gap in services.
According to the director of clinical services, the program’s protocol in the event of facilitator
turnover is to inform the department the reason and explanation for the turnover and also inform
the department of the gap if it was longer than a month. The program has a tracker and additional
trained staff in place to prevent any gaps in service.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation reviewed for Aggression Replacement Training (ART) for the annual review
period of October 1, 2023 - September 30, 2023 showed there were thirty-one (31) internal fidelity
monitoring checks completed. The program did not have any missing fidelity checks during the
review period.
The director of clinical services, a therapist, and the clinical director conducted the monthly
internal fidelity checks during the annual review period.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
A review of the documentation for Aggression Replacement Training showed there were no
corrective actions noted during the annual review period of October 1, 2023 - September 30,
2023.
According to the director of clinical services, if there is an issue with services being provided or
fidelity monitoring being completed, the program will issue a corrective action plan for correction in
an appropriate time frame.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of the annual staff evaluations during the annual review period of October 1, 2022
through September 30, 2023 shows there is an area where the facilitators are evaluated on their
performance in facilitating Aggression Replacement Training (ART). The program provided two
therapist annual evaluations. The annual evaluation contained specific information relating to the
therapist’s delivery of ART.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 10 weeks for this type of service. Of the 12 youth in the
sample, 100% (12 of 12) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 30 contact hours for this type of service. Of the 12 youth in the
sample, 100% (12 of 12) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 13 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 52%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 50%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 41%
Table 1
Moderate = 2 youth
Moderate-High = 1 youth
High = 3 youth
Total Youth in Sample = 12 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 25%
High Score: 8 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 61%
Table 2
High = 3 youth
Total Youth in Sample = 12 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 50% (6 of 12) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 25% (3 of 12) youth scored High-Risk to reoffend, for a score of 8 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 13 25 52%
Totals 88 100 88%
This SPEP report evaluates Aggression Replacement Training, an intervention delivered at
Daytona Juvenile Residential Facility.
12 youth completed the program during the review period. Of those youth, 12 were included in the
sample.
The program scored High for Quality of Service Delivery.
The targeted number of weeks for this specific intervention is 10 weeks; the targeted number of
hours for this specific intervention is 30 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 5 services (15 weeks; 45
hours).
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 10 points for Amount of Service: Duration. Of the 12 total youth sampled, all
12 included dosage with end dates in the EBS Module. Of those youth, 12 received at least the
recommended weeks of service. Youth in the sample completed between 10 and 15 weeks of
service, with an average of 12 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 12 total youth
sampled, all 12 included dosage in the EBS Module. Of those youth, 12 received at least the
recommended hours of service. Youth in the sample completed between 30 and 38 hours of
service, with an average of 32 hours.
The program was awarded 12 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Based on the risk level of the youth placed at the program (52% Program Optimization
Percentage), the Department should work to ensure a larger proportion of higher risk youth are
recommended. As the program serves youth with a history of sexual behavior problems, this
includes a multitier strategy of working with stakeholders to achieve that practice, as well as
ensuring appropriate treatment services are available in the community to reduce the reliance on
residential commitment for lower risk youth presenting with sexually-related offenses.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 100%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Daytona Juvenile Residential Facility can maintain their SPEP Quality of Service Delivery score
by continuing the practices in place at the time of this review.
Daytona Juvenile Residential Facility can optimize their SPEP Amount of Service score by
ensuring dosage for all youth is recorded accurately in EBS and by ensuring youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

source: DJJ

report_type: SPEP

file_name: FY2023-24 DJRF ART.pdf

program_name_raw: Daytona Juvenile Residential Facility

program_status_note: Open

service_type: Aggression Replacement Training


summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Daytona Juvenile Residential Facility
Rite of Passage, Inc.
(Contract Provider)
1386 Indian Lake Road
Daytona Beach, Florida 32124
Primary Service: Impact of Crime
SPEP Review Period: October 1, 2022 to September 30, 2023
SPEP Review Conducted: October 4, 2023
Florida Department of Juvenile Justice
Report Date: May 1, 2024
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The documentation reviewed for the annual review period of October 1, 2022 through September
30, 2023 shows the program uses the Impact of Crime Curriculum (IOC) for their primary service.
The facilitators are trained by a qualified trainer to provide the service. The training is a three-day,
twenty-four (24)-hour training currently being provided online via Microsoft Teams. There were six
staff trained in IOC - one shift supervisor, a former shift supervisor, the former transition services
manager, the current transition services manager, and two coach counselors. The documentation
provided shows one coach counselor facilitated both cohorts during the annual review period. A
trained staff member conducts the monthly internal fidelity checks. A shift supervisor, the former
transition services manager, and the current transition services manager conducted the monthly
internal fidelity checks.
The program's Standardized Program Evaluation Protocol procedure states the facilitator for each
primary service will have completed a formalized training for that primary service from a qualified
trainer. Daytona Juvenile Residential Facility will make all efforts to have a co-facilitator for the
primary service to ensure continuity of the service due to facilitator absence and provide more
individualized instruction as needed during the service, when possible. The co-facilitator will have
completed a formalized training for the primary service from a qualified trainer.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the Impact of Crime: Addressing the Harm manual. The manual opens with
an introduction to the Impact of Crime (IOC) curriculum. Then the manual goes over all seven
chapters of the service and provides a script for the facilitator to follow. The manual also has an
appendix and a student guide with all the student exercises. The program makes copies of the
student exercises and passes them out to the youth during each lesson.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
IOC groups are held on Sundays and Tuesday the majority of the time, but may vary based on
staff schedules. The program keeps all SPEP documentation (attendance sheets, fidelity checks,
and training certificates) in a binder labeled "IOC SPEP documentation". All youth admitted to the
program are required to participate in the service. IOC is a closed group with no more than twelve
(12) youth assigned to one cohort.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Impact of Crime (IOC) wasw observed on 7/11/2023. There were six (6) youth
present for group along with the facilitator and an additional coach counselor. The group began at
4:13p. The lesson was on chapter six, objective four. The facilitator was knowledgeable of the
material and followed the lesson plan. The class started with definitions from last session. The
facilitator kept the youth engaged.The facilitator also helped the youth through questions they had
and made sure all youth were participating, calling on those who weren't as involved. The
facilitator and the group had good rapport shown by celebrations of correct answers, the youth
lifting each other up when telling personal stories, and both staff present encouraging the youth
participation. The lesson topics were related to current and historic real-life events. The facilitator
effectively redirected youth from tangent conversations and the support staff helped with keeping
side conversations from starting.
Three youth left for five minutes for a medication call, but were quickly caught up when they
returned. There were no handouts or youth participation guides; however, the facilitator was
giving the youth talking points and passed around the facilitator manual for the youth to read
scenarios from. The group reviewed empathy and how they would feel in the victim's place, how
to properly resolve conflicts, and times when they had intense emotion.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation for Impact of Crime (IOC) for the annual review period of October 1,
2022 through September 30, 2023 shows there was facilitator turnover; however, the turnover did
not negatively affect the delivery of the service. A transition services manager and a shift
supervisor left the program. Neither were facilitating groups at the time.
The program completed two cohorts and no cohorts are currently ongoing for this annual review
period. The dates of the cohorts are 10/25/2022 to 2/28/2023 and 4/16/2023 to 8/1/2023. The
clinical director stated they were waiting to have enough youth who needed the IOC to run the
service between 2/28/2023 and 4/16/2023. There has not been a new cohort started since
8/1/2023 due to a facilitator injury, however, the next cohort is scheduled to start 10/4/2023 (as of
10/1/2023).
The 10/25/2022 cohort began and ended with seven youth. The documents reviewed were from
2/1/2023-2/28/2023 for this cohort. The beginning of the cohort through 1/31/2023 were reviewed
during the previous review period. Most of the attendance sheets reviewed have the IOC cohort
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
end date listed. The IOC cohort end date should not be on any attendance sheet other than the
final class’s attendance sheet. The attendance sheet on 11/23/2022 shows the group was on
chapter four objective two. The next attendance sheet on 11/27/2022 shows the facilitator went
back to chapter three objective three. On 11/29/2022, the facilitator covered chapter three
objective four. On 12/4/2022, the attendance sheet shows the facilitator covered chapter three
objective five (which does not exist). The facilitator stated the group needed to go back over
chapter three for a review. The technical assistance specialist informed the facilitator that it was
ok to go back and review, but to document the reason(s) on each attendance sheet. The
attendance sheet dated 12/25/2022 does not have the group end time listed and a checkmark
outside of the objectives box. The facilitator thought he should add an objective box outside of the
ones provided because there were only four objectives for chapter four, and he was working on
the fifth hour covering chapter four. The technical assistance specialist told the facilitator the
objectives boxes only aligned with the objectives for each chapter, not the recommended
minimum hours. The attendance sheets after 1/8/2023 did not list the facilitator.
The 4/16/2023 cohort began and ended with six youth. The attendance sheet dated 6/27/2023
shows the group was on chapter six objective. The next attendance sheets showed on 7/4/2023 -
chapter six objective four, 7/9/2023 - chapter six objective five, 7/11/2023 - chapter six objective
four, then continuing on to chapter seven. Chapter six objective three, was covered on the last
day of the cohort, 8/1/2023.
According to the director of clinical services, the program’s protocol in the event of facilitator
turnover is to inform the department the reason and explanation for the turnover and also inform
the department of the gap if it was longer than a month. The program has a tracker and additional
trained staff in place to prevent any gaps in service.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Documentation reviewed for Impact of Crime (IOC) for the annual review period of October 1,
2022 through September 30, 2023 showed there were ten (10) internal fidelity monitoring checks
completed. The the former transition service manager and a shift supervisor conducted the
monthly internal fidelity checks. The program did not have any missing internal fidelity monitoring.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
A review of internal fidelity monitoring of Impact of Crime (IOC) showed there were no corrective
actions noted nor taken during the annual review period of October 1, 2022 through September
30, 2023.
According to the director of clinical services, if there is an issue with services being provided or
fidelity monitoring being completed, the program will issue a corrective action plan for correction in
an appropriate time frame.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of documentation for the annual review period of October 1, 2022 through September
30, 2023 shows there is an area where the facilitators are evaluated on their performance in
facilitating Impact of Crime (IOC). The program provided annual evaluations for one case
manager. The evaluation provided was from 8/8/2022 and does not mention IOC as the coach
counselor was not facilitating at that time. The clinical director stated that the couch counselor is
due for another annual evaluation and IOC will be covered in that one. All of the evaluations had a
numerical value assigned for the performance of the facilitator.
According to the Director of Clinical Services, all facilitators are evaluated yearly on group
services provided. Evaluations include implementation and youth engagement.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 16 weeks for this type of service. Of the 12 youth in the
sample, 100% (12 of 12) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 12 youth in the
sample, 100% (12 of 12) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 13 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 52%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 50%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 2 youth
Moderate-High = 1 youth
High = 3 youth
Total Youth in Sample = 12 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 25%
High Score: 8 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 62%
Table 2
High = 3 youth
Total Youth in Sample = 12 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 50% (6 of 12) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 25% (3 of 12) youth scored High-Risk to reoffend, for a score of 8 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 13 25 52%
Totals 73 85 86%
This SPEP report evaluates Impact of Crime, an intervention delivered at Daytona Juvenile
Residential Facility.
12 youth completed the program during the review period. Of those youth, 12 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 12 total youth sampled, all
12 included dosage with end dates in the EBS Module. Of those youth, 12 received at least the
recommended weeks of service. Youth in the sample completed between 18 and 36 weeks of
service, with an average of 20 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program earned 10 points for Amount of Service: Contact Hours. Of the 12 total youth
sampled, all 12 included dosage in the EBS Module. Of those youth, 12 received at least the
recommended hours of service. Youth in the sample completed between 33 and 68 hours of
service, with an average of 38 hours.
The program was awarded 13 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Based on the risk level of the youth placed at the program (52% Program Optimization
Percentage), the Department should work to ensure a larger proportion of higher risk youth are
recommended. As the program serves youth with a history of sexual behavior problems, this
includes a multitier strategy of working with stakeholders to achieve that practice, as well as
ensuring appropriate treatment services are available in the community to reduce the reliance on
residential commitment for lower risk youth presenting with sexually-related offenses.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 100%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Daytona Juvenile Residential Facility can optimize their SPEP Quality of Service Delivery score
by ensuring each facilitator’s performance evaluation includes discussion of their skill in delivering
this primary service.
Daytona Juvenile Residential Facility can optimize their SPEP Amount of Service score by
ensuring dosage for all youth is recorded accurately in EBS and by ensuring youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

file_name: FY2023-24 DJRF IOC.pdf

service_type: Impact of Crime


summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Daytona Juvenile Residential Facility
Rite of Passage, Inc.
(Contract Provider)
1386 Indian Lake Road
Daytona Beach, Florida 32124
Primary Service: Pathways: A Guided Workbook for Youth Beginning
Treatment
SPEP Review Period: October 1, 2022 to September 30, 2023
SPEP Review Conducted: October 4, 2023
Florida Department of Juvenile Justice
Report Date: May 1, 2024
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Pathways: A Guided Workbook for Youth Beginning
Treatment .
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
Documentation reviewed for the annual review period of October 1, 2022 through September 30,
2023 shows the program uses Pathways: A Guided Workbook for Youth Beginning Treatment for
a primary service. The training is an in-service training provided by the clinical director or other
qualified trainer. There were four program staff trained - two therapists, a former therapist, and the
clinical director. Two trained staff from another facility assisted the program with facilitating the
primary service - one therapist and the other program’s clinical director. Three therapists, a former
therapist, and the clinical director facilitated the primary service during the annual review period.
A trained staff member conducts the monthly internal fidelity checks. The clinical director and the
clinical director from another program conducted the monthly internal fidelity checks during the
annual review period. The regional clinical director states it is best practice that the clinical
director conduct the monthly fidelity monitoring.
The program's Standardized Program Evaluation Protocol procedure states the facilitator for each
primary service will have completed a formalized training for that primary service from a qualified
trainer. Daytona Juvenile Residential Facility will make all efforts to have a co-facilitator for the
primary service, in order to enhance the continuity of the service due to facilitator absence and
provide more individualized instructionwhen possible. The co-facilitator will have completed a
formalized training for the primary service from a qualified trainer.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the Pathways: A Guided Workbook for Youth Beginning Treatment 4th ed.
manual. There is not a separate manual for facilitators and youth as the manual is the same for
both the facilitator and the youth. The manual is broken down into fourteen (14) chapters. Each
chapter covers a specific topic and there are activities for each topic in each chapter. Pathways is
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
run as a group and individually. Groups are moved through each chapter in order from one
through fourteen (14) and then back to chapter one. During individual sessions, Pathways
protocol is given in order by the youth's level at the program.
The program conducts the primary service on Tuesdays, Thursdays, and Fridays the majority of
the time, with no more than ten (10) youth assigned to one cohort. The program keeps all SPEP
documentation (attendance sheets, fidelity checks, and training certificates) in a binder labeled
"Pathways SPEP documentation". All youth admitted to the program are required to participate in
the primary service.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Pathways: A Guided Workbook for Youth Beginning Treatment was observed on
10/3/2023 at the facility. There were nine youth present for the group along with the facilitator and
two floor staff. The group began at 1:52p and was held in the main group room. The facilitator was
knowledgeable of the material and followed the lesson plan. The lesson for the day was chapter
twelve (12). There were two newer youth in group and the establishd youth helped them with
rules, procedures, and encouragement. Topics discussed included thinking errors, getting
something out of treatment, and treatment starts at accountablility. The facilitator did a great job of
guiding the youth and helping when the group got stuck. All youth were engaged in the session.
The group ended at 2:52p.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation for the annual review period of October 1, 2022 through September
30, 2023 shows there was turnover of facilitators for Pathways: A Guided Workbook for Youth
Beginning Treatment. A therapist left in May 2023. The turnover did not affect the delivery of the
service as it is an open group and a new facilitator took over. The program ran the primary service
as an open group during the annual review period, without any gaps in service.
According to the regional clinical director, the program’s protocol in the event of facilitator turnover
is to inform the department the reason and explanation for the turnover and also inform the
department of the gap if it was longer than a month. The program has a tracker and additional
trained staff in place to prevent any gaps in service.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation reviewed for the annual review period of October 1, 2022 through September
30, 2023 showed there were eighteen (18) internal fidelity monitoring checks completed for
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
facilitators of Pathways: A Guided Workbook for Youth Beginning Treatment. The clinical director
and another program’s clinical director conducted the internal fidelity checks. The program did not
have any missing fidelity checks.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
A review of the internal fidelity monitoring during the annual review period showed there were no
corrective actions noted nor taken for facilitators of Pathways: A Guided Workbook for Youth
Beginning Treatment.
According to the regional clinical director, if there is an issue with services being provided or
fidelity monitoring being completed, the program will issue a corrective action plan for correction in
an appropriate timeframe.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of the annual staff evaluations during the annual review period of October 1, 2022
through September 30, 2023 shows there is an area where the facilitators are evaluated on their
performance in facilitating Pathways: A Guided Workbook for Youth Beginning Treatment. The
program provided annual evaluations for two therapists, a former therapist, and the clinical
director. The annual evaluation contained specific information relating to the therapist’s delivery of
the primary service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 24 weeks for this type of service. Of the 12 youth in the
sample, 100% (12 of 12) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 40 contact hours for this type of service. Of the 12 youth in the
sample, 100% (12 of 12) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 13 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 52%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 50%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 2 youth
Moderate-High = 1 youth
High = 3 youth
Total Youth in Sample = 12 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 25%
High Score: 8 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 62%
Table 2
High = 3 youth
Total Youth in Sample = 12 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 50% (6 of 12) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 25% (3 of 12) youth scored High-Risk to reoffend, for a score of 8 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 13 25 52%
Totals 83 95 87%
This SPEP report evaluates Pathways: A Guided Workbook for Youth Beginning Treatment, an
intervention delivered at Daytona Juvenile Residential Facility.
12 youth completed the program during the review period. Of those youth, all 12 were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 12 total youth sampled, all
12 included dosage with end dates in the EBS Module. Of those youth, 12 received at least the
recommended weeks of service. Youth in the sample completed between 51 and 76 weeks of
service, with an average of 58 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 10 points for Amount of Service: Contact Hours. Of the 12 total youth
sampled, all 12 included dosage in the EBS Module. Of those youth, 12 received at least the
recommended hours of service. Youth in the sample completed between 139 and 216 hours of
service, with an average of 162 hours.
The program was awarded 13 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Based on the risk level of the youth placed at the program (52% Program Optimization
Percentage), the Department should work to ensure a larger proportion of higher risk youth are
recommended. As the program serves youth with a history of sexual behavior problems, this
includes a multitier strategy of working with stakeholders to achieve that practice, as well as
ensuring appropriate treatment services are available in the community to reduce the reliance on
residential commitment for lower risk youth presenting with sexually-related offenses.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 100%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Daytona Juvenile Residential Facility can maintain their SPEP Quality of Service Delivery score
by continuing the practices in place at the time of this review.
Daytona Juvenile Residential Facility can optimize their SPEP Amount of Service score by
ensuring dosage for all youth is recorded accurately in EBS and by ensuring youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2023-24 DJRF Pathways.pdf

service_type: Pathways to Self‐Discovery and Change