state: FL

facility_name: Duval Academy

summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Duval Academy
Wayne Halfway House
(Contract Provider)
4501 Lannie Road
Jacksonville, Florida 32218
Primary Service: Aggression Replacement Training (ART)
SPEP Review Period: July 1, 2023-June 30, 2024
SPEP Review Conducted: 7/2/2024 and 7/9/2024
Florida Department of Juvenile Justice
Report Date: 9/30/2025
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Aggression Replacement Training.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 5 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 25%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 1 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A review of the documentation for Aggression Replacement Training (ART) for the annual review
period of July 1, 2023 through June 30, 2024 shows the program uses Aggression Replacement
Training (ART) as a primary service. Aggression Replacement Training is provided by a qualified
trainer over a three-day period.
The documentation shows there were four staff trained to facilitate Aggression Replacement
Training during the annual review period. The trained staff were three therapists and the clinical
director. All four trained staff facilitated the primary service. The clinical director stopped running
ART after September 2024. The clinical director stated there were no youth at the program with a
Mental Health Overlay Service (MHOS) designation. All trained staff can conduct the monthly
internal fidelity checks. One trained staff member (clinical director) conducted a monthly internal
fidelity check during the review period. There were no other internal fidelity checks completed.
The primary service was conducted on Mondays, Tuesdays, and Thursdays.
Basic Score: 0 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program did not provide documentation showing they utilize the Aggression Replacement
Training (ART) manual. The program ran ART until September 2023. After, the clinical director
stated there were no youth at the program with a Mental Health Overlay Service (MHOS)
designation.
The program can earn 2 points if the program utilizes a manual developed for the curriculum.
At the time of the review, the program did not meet criteria to earn a score of 2.
c. Observed Adherence to the Manual/Protocol Basic Score: 0 Point(s)
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Technical assistance specialist did not observe a session of Aggression Replacement Training
(ART). The clinical director stated there were no youth at the program with a Mental Health
Overlay Service (MHOS) designation, therefore, the program was not providing the service.
The program can earn 1 point if the program provides the service as developed and outlined in
the curriculum.
At the time of the review, the program did not meet criteria to earn a score of 1.
Basic Score: 0 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The documentation reviewed for Aggression Replacement Training (ART) shows the program did
have facilitator turnover. There were two therapists who left the program. One therapist left the
program in July 2023 and the other left in August 2023. The documentation reviewed shows the
primary service was not being provided as intended. The program was running the primary
service as an open group instead of a closed group. All four trained staff members (clinical
director and three therapists) facilitated the primary service during the annual review period.
There were no identified cohorts due to the program running the primary service as an open
group. The primary service curriculum is supposed to be delivered three times a week with the
Skillstreaming component on the first day, the Anger Control component on the second day, and
the Moral Reasoning component on the third day. The program did not follow this outline.
The program had weeks where they provided one component two times and another component
once during a week, the same component three times during a week, or only providing two
components for a week. The clinical director stated it was difficult to ensure the primary service
was being run as developed due to the staff shortage in the clinical department. The clinical
director stated the program is now fully staffed in the clinical department and will ensure the
primary service is delivered as prescribed from now on. The program was also providing the
primary service with more than ten (10) youth in each group. A mental health service is only
allowed to have a maximum of ten (10) youth in a group. The program has twenty-seven (27)
youth in the program and is running two groups each day the primary service is delivered.
The clinical director stopped running ART after September 2024. The clinical director stated there
were no youth at the program with a Mental Health Overlay Service (MHOS) designation.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 0 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of the documentation for Aggression Replacement Training (ART) for the annual review
period of July 1, 2023 through June 30, 2024 showed there was one internal fidelity check
completed. The date of the internal fidelity check was: 7/11/2023. The clinical director conducted
the one internal fidelity monitoring for the review period. The clinical director stated she was
unable to conduct the missing fidelity checks due to being short staffed. The clinical director was
the only trained staff conducting primary service during most of the review period.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 0 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The documentation for Aggression Replacement Training (ART) for the annual review period of
July 1, 2023 through June 30, 2024 shows there were no corrective actions noted on the fidelity
check. It is unknown if there were any corrective actions needed as there was only one fidelity
check completed during the review period.
The program can earn 1 point if any corrective action needed is identified, applied, and
demonstrated.
At the time of the review, the program did not meet criteria to earn a score of 1.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program did not provide any facilitator performance evaluations for Aggression Replacement
Training for the annual review period of July 1, 2023 through June 30, 2024.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
There was no data entered into the Juvenile Justice Information System (JJIS) Evidence Based
Module (EBS) for the primary service for the annual review period. There was no data available to
enter into the report.
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 10 weeks for this type of service. Of the 30 youth in the
sample, 0% (0 of 30) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
There was no data entered into the Juvenile Justice Information System (JJIS) Evidence Based
Module (EBS) for the primary service for the annual review period. There was no data available to
enter into the report.
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 30 contact hours for this type of service. Of the 30 youth in the
sample, 0% (0 of 30) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 90%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 7 youth
Moderate-High = 9 youth
High = 11 youth
Total Youth in Sample = 30 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 37%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 11 youth
Total Youth in Sample = 30 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 90% (27 of 30) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 37% (11 of 30) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 5 20 25%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 23 25 92%
Totals 63 100 63%
This SPEP report evaluates Aggression Replacement Training, an intervention delivered at Duval
Academy.
There was no data entered into the Juvenile Justice Information System (JJIS) Evidence Based
Module (EBS) for the primary service for the annual review period. There was no data available to
enter into the report.
31 youth completed the program during the review period. Of those youth, 30 youth were selected
for the sample to determine risk level of youth served for the population.
The program scored Low for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 30 total youth sampled, only
0 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 0
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
received at least the recommended weeks of service. Youth in the sample completed between 0
and 0 weeks of service, with an average of 0 weeks.
The program earned 0 points for Amount of Service: Contact Hours. Of the 30 total youth
sampled, only 30 included dosage in the EBS Module. Of those youth with correct dosage, 0
received at least the recommended hours of service. Youth in the sample completed between 5
and 17 hours of service, with an average of 11.77 hours.
The program earned 0 points for Amount of Service: Duration and Amount of Service: Contact
Hours. There was no data entered into the Juvenile Justice Information system (JJIS) Evidence-
Based Services (EBS) module for any youths. With no JJIS data to review, the SPEP score was
reduced. The program must enter dosage data into JJIS in order to receive any score in this
area.
The targeted number of weeks for this specific intervention is 10 weeks; the targeted number of
hours for this specific intervention is 30 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 5 services (15 weeks; 45
hours).
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
RECOMMENDATION(S):
Duval Academy can optimize their SPEP Quality of Service Delivery score by ensuring the
program utilizes a specific written manual/protocol detailing the delivery of the primary service,
provides the primary service regularly throughout the year so it can be observed for adherence to
the manual/protocol, facilitator turnover does not impact groups, fidelity monitoring is completed
every month for each facilitator, any corrective action needed is identified, applied, and
demonstrated, and each facilitator’s performance evaluation includes discussion of their skill in
delivering this primary service.
Duval Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

source: DJJ

report_type: SPEP

file_name: FY2023-24 Duval Academy ART .pdf

program_status_note: Open

service_type: Aggression Replacement Training


summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Duval Academy
Wayne Halfway House
(Contract Provider)
4501 Lannie Road
Jacksonville, Florida 32218
Primary Service: Cannabis Youth Treatment (CYT)
SPEP Review Period: July 1, 2023-June 30, 2024
SPEP Review Conducted: 7/2/2024 and 7/9/2024
Florida Department of Juvenile Justice
Report Date: 9/30/2025
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Cannabis Youth Treatment .
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 5 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 0 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A review of the documentation for Cannabis Youth Treatment (CYT) for the annual review period
of July 1, 2023 through June 30, 2024 shows the program uses Cannabis Youth Treatment (CYT)
as a primary service. Cannabis Youth Treatment (CYT) training is an in-service training provided
by the clinical director or other approved person. The documentation shows there were three staff
trained to facilitate Cannabis Youth Treatment during the annual review period. The trained staff
are two therapists and the clinical director.
There was a therapist who conducted a few groups before he left the program. The program was
not able to provide his proof of training. All trained staff can conduct the monthly internal fidelity
checks. The clinical director and the two therapists facilitated the primary service. The clinical
director was running the cohorts as an open group during the first half of the review period. The
clinical director switched to a closed group during the second part of the review period after
speaking with the technical assistance specialist. There were two monthly internal fidelity checks
completed by the clinical director during the annual review period.
The program can earn 1 point if all facilitators are trained in the primary service prior to facilitating
groups.
At the time of the review, the program did not meet criteria to earn a score of 1.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The documentation reviewed for Cannabis Youth Treatment shows the program began the review
period using the Cannabis Youth Treatment Series Volume Four. Technical assistance specialist
informed the clinical director the appropriate manuals were volumes one and two. Volume four
focuses on youth who are in a community setting, not a residential setting. The clinical director
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
switched over to volumes one and two in February 2024. The manuals provide step by step
procedures for implementing the program. There are no student manuals. The facilitators make
copies of the worksheets and pass them out to the youth during group. The first two lessons are
conducted as individual sessions. These sessions are an hour in length. Lessons three through
twelve (12) are conducted as a group. All youth in the program are required to receive the primary
service. The program maintains all documentation in a binder labeled Cannabis Youth Treatment
(CYT) located in the Clinical Director’s office. The binder contains all attendance sheets, internal
fidelity checklists, and training certificates. The program provides the primary service one time per
week on Wednesdays for one hour. The primary service is a closed group.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Technical assistance specialist observed a session of Cannabis Youth Treatment (CYT) on
7/2/2024. The group began at 1:45p. The group was held in the program’s classroom. There were
a total of seven youth and one facilitator present during the group. The group began with five
youth in the classroom. There were two other youth who were not present at the start of the
group. The program was providing all youth with a reward for positive behaviors. The two youth
arrived to the group at 2:06p. The facilitator was knowledgeable of the material and followed the
lesson plan. The group was on session 9 effective communication. The group ended at 2:50p.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The documentation reviewed for Cannabis Youth Treatment for the annual review periodshows
the program did have facilitator turnover. There were three therapists who left the program. One
therapist left the program in July 2023, one left in August 2023, and the third left in June 2024.
The therapist who left the program in August 2023 was facilitating the groups when he left. This
therapist did not have proof of training in the primary service. The therapist who left in June 2024
was facilitating groups at the time she left the program.
The documentation reviewed shows the primary service was not being provided as intended. The
program was running the primary service as an open group instead of a closed group and the
program was using volume four instead of volumes one and two. There were no identified cohorts
during the first half of the review period due to the program running the primary service as an
open group. The primary service curriculum is supposed to be delivered once a week for twelve
(12) weeks. The first two weeks are supposed to be individual sessions. Weeks three through ten
(10) are group sessions. The clinical director stated she facilitated the first two weeks in a group
setting instead of the individual setting. The clinical director stated it was difficult to conduct all the
individual sessions as required because she was the only person in the clinical department at that
time. The clinical director stated the program is now fully staffed in the clinical department and will
ensure the primary service is delivered as prescribed from now on. The clinical director began
running the primary service as developed in February 2024. There were eight attendance sheets
which did not list group end times. The dates of those attendance sheets were 11/14/2023,
12/5/2023, 12/12/2023, and 12/19/2023.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The dates of the cohorts are 2/1/2024-4/18/2024 and 5/23/2024-current (as of 6/30/2024).
The cohort which began on 2/1/2024 began with fifteen (15) youth and ended with fourteen (14)
youth. The trained staff (clinical director and two therapists) rotated facilitating the cohort.
The cohort which began on 5/23/2024 began with seven youth and currently has seven youth (as
of 6/30/2024). One youth did not sign the attendance sheets on 5/23/2024 and 5/28/2024. The
program was not running groups on the same day each week. The clinical director stated the
program was trying to ensure the groups could be run at least once per week, even if the day of
the week changed.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation reviewed for Cannabis Youth Treatment for the annual review period of July
1, 2023 through June 30, 2024 shows the program conducted two monthly internal fidelity checks.
The dates of the fidelity monitoring were: 2/8/2024 and 3/14/2024. The clinical director conducted
the internal fidelity checks for the annual review period. There were no internal fidelity checks
completed for the months of July 2023 through January 2024 or for May 2024 and June 2024.
The program conducts the fidelity monitoring during lessons three through twelve since those
lessons are conducted as a group. Lessons one and two are conducted as individual sessions.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 0 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The documentation reviewed for Cannabis Youth Treatment for the annual review period of July
1, 2023 through June 30, 2024 shows there were no corrective actions noted on the two fidelity
checks. It is unknown if there were any corrective actions needed during the first half of the review
period as there were no fidelity checks completed during that time.
The program can earn 1 point if any corrective action needed is identified, applied, and
demonstrated.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program did not provide any facilitator performance evaluations for Cannabis Youth
Treatment for the annual review period of July 1, 2023 through June 30, 2024.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service - Duration
Due to the Juvenile Justice Information System (JJIS) outage, this report does not contain data for
the period of March 2024 through June 2024. Data contained in this report is representative from
the period of July 2023 through February 2024.
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 12 weeks for this type of service. Of the 27 youth in the
sample, 0% (0 of 27) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Due to the Juvenile Justice Information System (JJIS) outage, this report does not contain data for
the period of March 2024 through June 2024. Data contained in this report is representative from
the period of July 2023 through February 2024.
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 12 contact hours for this type of service. Of the 27 youth in the
sample, 67% (18 of 27) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 89%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 7 youth
Moderate-High = 8 youth
High = 9 youth
Total Youth in Sample = 27 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 33%
High Score: 10 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 77%
Table 2
High = 9 youth
Total Youth in Sample = 27 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 89% (24 of 27) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 33% (9 of 27) youth scored High-Risk to reoffend, for a score of 10 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 20 25 80%
Totals 71 100 71%
This SPEP report evaluates Cannabis Youth Treatment, an intervention delivered at Duval
Academy.
Due to the Juvenile Justice Information System (JJIS) outage, this report does not contain data for
the period of March 2024 through June 2024. Data contained in this report is representative from
the period of July 2023 through February 2024.
31 youth completed the program during the review period. Of those youth, 27 were included in the
sample.
The program scored Medium for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 27 total youth sampled, only
9 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 0
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
received at least the recommended weeks of service. Youth in the sample completed between 10
and 10 weeks of service, with an average of 10 weeks.
The program earned 0 points for Amount of Service: Duration. There was no data entered into the
Juvenile Justice Information system (JJIS) Evidence-Based Services (EBS) module for any
youths. With no JJIS data to review, the SPEP score was reduced. The program must enter
dosage data into JJIS in order to receive any score in this area.
The program earned 6 points for Amount of Service: Contact Hours. Of the 27 total youth
sampled, only 27 included dosage in the EBS Module. Of those youth with correct dosage, 18
received at least the recommended hours of service. Youth in the sample completed between 6
and 25 hours of service, with an average of 14.67 hours.
The targeted number of weeks for this specific intervention is 12 weeks; the targeted number of
hours for this specific intervention is 12 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 5 services (15 weeks; 45
hours).
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
RECOMMENDATION(S):
Duval Academy can optimize their SPEP Quality of Service Delivery score by ensuring all
facilitators are trained prior to facilitating groups, facilitator turnover does not impact groups,
fidelity monitoring is completed every month for each facilitator, any corrective action needed is
identified, applied, and demonstrated, and each facilitator’s performance evaluation includes
discussion of their skill in delivering this primary service.
Duval Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

file_name: FY2023-24 Duval Academy CYT .pdf

program_name_raw: Duval Academy

service_type: Cannabis Youth Treatment


summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Duval Academy
Wayne Halfway House
(Contract Provider)
7500 Ricker Rd
Jacksonville, Florida 32244
Primary Service: Impact of Crime
SPEP Review Period: July 1, 2023-June 30, 2024
SPEP Review Conducted: 7/2/2024 and 7/9/2024
Florida Department of Juvenile Justice
Report Date: 9/30/2025
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program uses Impact of Crime (IOC) as a primary service. There were five staff trained in the
primary service. The staff trained are two case managers, the assistant facility administrator, the
transition case manager, and the clinical director. The training is an online training and is twenty-
eight (28) hours long conducted over three days by a qualified trainer. The primary service is a
closed group. All trained staff can conduct the monthly internal fidelity checks. There were three
staff (two case managers and the assistant facility administrator) who facilitated the primary
service during the annual review period. Three trained staff members (a case manager, the
assistant facility administrator, and the clinical director) conducted the monthly internal fidelity
checks.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the Impact of Crime: Addressing the Harm manual. The manual opens with an
introduction to the Impact of Crime Curriculum. Then the manual goes over all seven chapters of
the service and provides a script for the facilitator to follow. The manual also has an appendix and
a student guide section with all the student exercises. The program does not have separate
student workbooks to give out. The facilitators make copies of the exercises before each session
to pass out to the youth. The group is a closed group with no more than twelve youth in each
cohort. The curriculum is to be delivered to the youth over a minimum of sixteen (16) weeks for
thirty-two (32) sessions. The program keeps all SPEP documents in a binder labeled Impact of
Crime in the clinical director’s office. The program provides the service on Tuesdays and
Thursdays for one hour each. All youth receive Impact of Crime.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Technical assistance specialist observed a session of Impact of Crime on 7/9/2024. There were
nine youth present for the group along with the facilitator. The group began at 4:13pm. The group
was held in the program's multi-purpose room. The facilitator attempted to get the youth to focus
on the lesson. The youth were holding side conversations and challenging the facilitator. One
youth wanted to leave the group at 4:39 pm. The youth asked to go back to the mod. The
facilitator called the clinical director and asked if she could take the youth. The clinical director
spoke with the youth and the youth returned to the group at 4:52 pm. The facilitator stated he was
disappointed the youth were being disrespectful while there was a visitor in the room. The
facilitator was able to go over the words to know and the objectives for the chapter. The facilitator
also had the youth complete student exercise one from chapter one. The group ended at 5:17pm.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program did not have facilitator turnover. The documentation reviewed shows there were four
cohorts completed. There were three staff (two case managers and the assistant facility
administrator) who facilitated the primary service during the annual review period. The dates of
the cohorts are 3/14/2023-7/19/2023, 6/8/2023-10/9/2023, 9/20/2023-1/25/2024, and 3/7/2024-
5/28/2024.
The cohort which began on 3/14/2023 began with ten (10) youth and ended with seven youth
successfully completing the service. There were two youth who were transferred from the
program in April 2023. One youth was discharged from the program on 7/7/2023. This cohort
began before the current review period and ended during the current review period.
The cohort which began on 6/8/2023 began with eight youth and ended with seven youth
successfully completing the service. One youth was discharged from the program on 9/8/2023.
There were six attendance sheets which did not have the group times noted. There was a note
stating IOC was not conducted during the week of 7/16/2023-7/22/2023 due to COVID leave.
There was another note stating IOC was not conducted the week of 8/13/2023-8/19/2023 due to
multiple critical incidents. The facilitator ran the group on back-to-back days (10/5/2023 and
10/6/2023).
The cohort which began on 9/20/2023 began with nine youth and ended with one youth
successfully completing the service. There was a gap in service from 10/5/2023-10/25/2023. The
attendance sheet dated 10/31/2023 shows the group was on chapter three objective one. The
next attendance sheet dated 11/2/2023 shows the group jumped to chapter four objectives one
and two. The next three attendance sheets (11/7/2023, 11/9/2023, and 11/14/2023) show the
facilitator went back to cover the remaining objectives for chapter three. There were youth
discharged on 11/9/2023 (one youth), 12/11/2023 (one youth), 12/13/2023 (one youth), and
12/20/2023 (four youth). The clinical director stated the program chose not to run two cohorts at
the same time due to the strain on the staff. Youth who were discharged early were not projected
to be released from the program as early as they were released. The clinical director stated the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
youth released had completed their treatment goals early so they were released from the
program.
The cohort which began on 3/7/2024 began with eight youth and ended with seven youth
successfully completing the service. One youth was arrested on 4/19/2024. The attendance
sheets show the case manager and the clinical director co-facilitating the group. The clinical
director stated she was not co-facilitating the group. The attendance sheets should not have her
name on them.
The program can earn 2 points if there are no unnecessary gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
There were twelve (12) internal fidelity checks completed during the review period. Four of the
twelve (12) fidelity checks (7/7/2023, 7/28/2023, 9/26/2023, and 9/27/2023) were completed by
the Clinical Director who was not trained in the primary service. The clinical director stated she
conducted the fidelity checks due to being short staffed. The clinical director did not realize she
could not conduct fidelity observations if she was not trained. The technical assistance specialist
stated the only staff who can complete a fidelity observation are the staff who were trained in the
primary service. The clinical director was trained in the primary service in February 2024. The
dates of the internal fidelity checks were: 7/7/2023, 7/28/2023, 8/25/2023, 9/26/2023, 9/27/2023,
9/28/2023, 10/5/2023, 11/22/2023, 12/5/2023, 3/19/2024, 4/4/2024, and 5/9/2024. Three trained
staff members (a case manager, the assistant facility administrator, and the clinical director)
conducted the monthly internal fidelity checks. The program was missing one fidelity monitoring
for the month of June 2023 for one case manager and January 2024 for another case manager.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
There were no corrective actions noted during the monthly internal fidelity checks. The clinical
director stated when corrective action is needed the process will be for the facilitator to meet with
the clinical director to review findings from the completed fidelity monitoring to address concerns
and provide support. The facilitator will be re-assessed at the next month’s fidelity monitoring.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program did not provide any facilitator performance evaluations for Impact of Crime for the
annual review period of July 1, 2023 through June 30, 2024.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service - Duration
Due to the Juvenile Justice Information System (JJIS) outage, this report does not contain data for
the period of March 2024 through June 2024. Data contained in this report is representative from
the period of July 2023 through February 2024.
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 16 weeks for this type of service. Of the 31 youth in the
sample, 3% (1 of 31) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Due to the Juvenile Justice Information System (JJIS) outage, this report does not contain data for
the period of March 2024 through June 2024. Data contained in this report is representative from
the period of July 2023 through February 2024.
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates a target of 24 contact hours for this type of service. Of the 31 youth in the
sample, 42% (13 of 31) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 90%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 7 youth
Moderate-High = 10 youth
High = 11 youth
Total Youth in Sample = 31 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 35%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 11 youth
Total Youth in Sample = 31 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 90% (28 of 31) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 35% (11 of 31) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 4 10 40%
Risk Level of Youth Served 23 25 92%
Totals 57 85 67%
This SPEP report evaluates Impact of Crime, an intervention delivered at Duval Academy.
Due to the Juvenile Justice Information System (JJIS) outage, this report does not contain data for
the period of March 2024 through June 2024. Data contained in this report is representative from
the period of July 2023 through February 2024.
31 youth completed the program during the review period. Of those youth, 31 were included in the
sample.
The program scored Medium for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 31 total youth sampled, only
7 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 1
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
received at least the recommended weeks of service. Youth in the sample completed between 10
and 19 weeks of service, with an average of 14 weeks.
The program earned 0 points for Amount of Service: Duration. There was no data entered into the
Juvenile Justice Information system (JJIS) Evidence-Based Services (EBS) module for any
youths. With no JJIS data to review, the SPEP score was reduced. The program must enter
dosage data into JJIS in order to receive any score in this area.
The program earned 4 points for Amount of Service: Contact Hours. Of the 31 total youth
sampled, only 14 included dosage in the EBS Module. Of those youth with correct dosage, 13
received at least the recommended hours of service. Youth in the sample completed between 19
and 47.25 hours of service, with an average of 37.50 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
RECOMMENDATION(S):
Duval Academy can optimize their SPEP Quality of Service Delivery score by ensuring there are
no unnecessary gaps in service, fidelity monitoring is completed every month for each facilitator,
and each facilitator’s performance evaluation includes discussion of their skill in delivering this
primary service.
Duval Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

file_name: FY2023-24 Duval Academy IOC .pdf

service_type: Impact of Crime