summary: SPEP - FY18-19

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Duval Academy
Sequel Youth and Family Services
(Contract Provider)
7500 Ricker Road
Jacksonville, Florida 32244
Primary Service: Impact of Crime
SPEP Review Date(s): 4/23/19‐4/26/19
Florida Department of Juvenile Justice
Report Date(s): 4/26/19
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
 Service Type: service type points are awarded based on the group category that the
service falls within
 Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
 Amount of Service: or “dosage” refers to the amount and duration of services that were
received
 Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime. The program was awarded 15 points
because the Primary Service is identified as a Group 3 Service. The specific Sub-Component
Service Type identified is Social Skills Training. The Primary Service was identified as this type of
service as it focuses on developing social skills required for an individual to interact in a positive
way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
During the review period, four staff were eligible to facilitate Impact of Crime groups, based on a
review of documentation provided; two of those four staff facilitated groups. Documentation
indicated each staff was trained by a qualified trainer prior to facilitating groups.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the Impact of Crime; Addressing the Harm curriculum, version 2.0, updated
September 2011. The program has a manual detailing how Impact of Crime should be delivered,
and a workbook for the students. The curriculum provides a script and instructions or seven
chapters which requires a minimum of thirty-two contact hours to facilitate. The curriculum
includes implementation guidelines, identifies the order the lessons are to be delivered, and
provides detailed instructions for the delivery of each lesson.
Documentation for this service in maintained in a binder specific to this intervention. The binder is
organized and easy to locate, and includes facilitator training documentation, sign in sheets,
fidelity monitoring, copies of pre- and post-tests and end of curriculum surveys. All youth in the
program are assigned to Impact of Crime during their stay. Groups are held Tuesdays and
Thursdays, from 1:00PM to 2:00PM.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
An observation could not be conducted as the facility is between cohorts at this time. This
curriculum was not able to be observed; therefore, this indicator rates as non-applicable.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation provided indicated there was facilitator turnover that did not adversely
affect services to the youth, as the turnover occurred between cohorts. The program has a
process in place to avoid gaps in service. This process includes having multiple people trained in
the service.
There is a current gap in service, for a little over one month. There is currently one facilitator for
Impact of Crime on site, who is conducting a cohort of Thinking for a Change. The program has
new staff registered for an upcoming Impact of Crime training. A recommendation was given to
regional clinical director to remove pre-populations of group end dates on the sign in sheets.
The program can earn 2 points if there are no unnecessary gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has a process in place to monitor the delivery of the Impact of Crime intervention,
and they use the Impact of Crime fidelity monitoring sheet provided by the Department of Juvenile
Justice. Fidelity monitoring is conducted by the Clinical Director, who is trained in the curriculum.
During the review period, fidelity monitoring was conducted each month, except for January and
April 2019.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process in place to address corrective action based on fidelity monitoring. An
informal discussion with the regional clinical director indicated fidelity monitoring is conducted
once a month, with the Clinical Director then following up on the monitorings during group
supervision meetings to discuss a plan of action for any corrective actions identified. The clinical
director will then re-address any issues during the next month’s fidelity monitoring. A review of
documentation provided indicated there were no corrective actions noted nor taken during the
review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Of the two facilitators who conducted groups during the review period, one received an annual
evaluation. The other staff left employment at the program before receiving an annual evaluation.
The facilitator’s performance evaluation included discussion of their skill in delivering the primary
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service ‐ Duration
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates the target duration of 16 weeks for this type of service. Of the 46 youth in the
sample, 37% (17 of 46) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates a target of 24 contact hours for this type of service. Of the 46 youth in the
sample, 54% (25 of 46) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 98%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 4 youth
Moderate-High = 13 youth
High = 28 youth
Total Youth in Sample = 46 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 61%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 28 youth
Total Youth in Sample = 46 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 98% (45 of 46) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 61% (28 of 46) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 2 10 20%
Amount of Service: Contact Hours 4 10 40%
Risk Level of Youth Served 25 25 100%
Totals 71 85 84%
This SPEP report evaluates Impact of Crime, an intervention delivered at Duval Academy.
46 youth completed the program during the review period. Of those youth, 46 were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 2 points for Amount of Service: Duration. Of the 46 total youth sampled, only
23 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 17
received at least the recommended weeks of service. Youth in the sample completed between 5
and 18 weeks of service, with an average of 15 weeks.
The program earned 4 points for Amount of Service: Contact Hours. Of the 46 total youth
sampled, only 31 included dosage in the EBS Module. Of those youth with correct dosage, 25
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
received at least the recommended hours of service. Youth in the sample completed between 8
and 33 hours of service, with an average of 30 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Duval Academy can optimize their SPEP Quality of Service Delivery score by ensuring there are
no unnecessary gaps in service, and by ensuring fidelity monitoring is completed every month for
each facilitator.
Duval Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9


state: FL

facility_name: Duval Academy

summary: SPEP - FY18-19

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Duval Academy
Sequel Youth and Family Services
(Contract Provider)
7500 Ricker Road
Jacksonville, Florida 32244
Primary Service: Living in Balance
SPEP Review Date(s): 4/23/19 ‐ 4/26/19
Florida Department of Juvenile Justice
Report Date(s): 4/26/19
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
 Service Type: service type points are awarded based on the group category that the
service falls within
 Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
 Amount of Service: or “dosage” refers to the amount and duration of services that were
received
 Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Living in Balance. The program was awarded 25 points
because the Primary Service is identified as a Group 4 Service. The specific Sub-Component
Service Type identified is Group Counseling. The Primary Service was identified as this type of
service as it focuses on psychological or interpersonal problems or issues faced by an individual
and involves a group of youths interacting with each other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
During the review period, two therapists and the clinical director were eligible to facilitate Living in
Balance groups. The program has an internal training protocol to ensure training is consistent
among each facilitator.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the commercially produced curriculum “Living in Balance – Moving from a Life
of Addiction to a Life of Recovery”, published by Jeffery Hoffman, Mim J. Landry and Barry
Caudill. Living in Balance is a researched-based substance abuse treatment intended to help
clients address issues in lifestyle areas that may have been neglected during addiction. It is
specifically oriented for group settings. The manual uses didactic education and instruction, group
process interaction through role plays and discussion, daily relaxation and visualization exerciese,
informational handouts, videotapes, and group-oriented recreational therapy exercises.
Documentation for this service is maintained in an organized binder, which includes sign in sheets
and fidelity monitoring checklists. All youth in the facility receive Living in Balance, and groups
are held Tuesdays, Wednesdays, and Thursdays from 2:00PM until 3:00PM.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The facilitator began the group with a sign in sheet and check in. Youth used a one to ten scale
on how they were that day and give a word to describe how they were feeling. The group was
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
working on session two: Alcohol and other drug education. The facilitator discussed standard
drugs and asked the youth which drugs are stimulants. The facilitator had to redirect the youth a
few times, as the youth were very excited to answer the questions. The facilitator discussed
withdraw symptoms from stimulants. Then the group discussed psychedelic and hallucinogenic
drugs. A staff member added to the discussion by detailing his experiences with an associate’s
drug use. The last drug the group discussed was marijuana. The youth gave examples of when
they used the drugs and their experiences with those drugs. The facilitator ended the group with a
review of the lesson.
The facilitator followed the manual and script for the lesson. There were no visual aids used for
the group. The facilitator stated she was not able to prepare visual aids for the group. There was
one youth who appeared to be throwing gang signs during group. The Technical Assistance
Specialist informed the facilitator of this.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
Documentation provided for the review period confirmed there was no facilitator turnover. One
staff was on medical leave for a period and the Clinical Director stepped in to ensure groups were
conducted without delay. Groups were run consistently throughout the review period, with no
gaps in service.
The program has a process in place to avoid gaps in service due to facilitator turnover. The
clinical director conducts weekly meetings with therapist to prepare/plan prior to assigning
facilitators to groups, in an attempt to avoid any potential gaps in service.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Fidelity monitoring was conducted each month for each facilitator, without exception. The clinical
director conducts monthly fidelity monitoring using an Internal checklist, which includes feedback
from the observer and facilitator, with space for corrective actions and signatures. Feedback
provided was thoughtful and specific.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process in place to address corrective action based on fidelity monitoring. An
informal discussion with the regional clinical director indicated fidelity monitoring is conducted
once a month, with the Clinical Director then following up on the monitorings during group
supervision meetings to discuss a plan of action for any corrective actions identified. The clinical
director will then re-address any issues during the next month’s fidelity monitoring. A review of
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
documentation provided indicated there were no corrective actions noted nor taken during the
review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Performance evaluations for the therapists facilitating Living in Balance were reviewed. All
included an evaluation of the therapists’ skill in delivering the service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 24 weeks for this type of service. Of the 46 youth in the
sample, 17% (8 of 46) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates a target of 40 contact hours for this type of service. Of the 46 youth in the
sample, 37% (17 of 46) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 98%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 4 youth
Moderate-High = 13 youth
High = 28 youth
Total Youth in Sample = 46 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 61%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 28 youth
Total Youth in Sample = 46 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 98% (45 of 46) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 61% (28 of 46) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 2 10 20%
Risk Level of Youth Served 25 25 100%
Totals 77 95 81%
This SPEP report evaluates Living in Balance, an intervention delivered at Duval Academy.
46 youth completed the program during the review period. Of those youth, 46 were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 46 total youth sampled, only
10 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 8
received at least the recommended weeks of service. Youth in the sample completed between 23
and 47 weeks of service, with an average of 30 weeks.
The program earned 2 points for Amount of Service: Contact Hours. Of the 46 total youth
sampled, only 18 included dosage in the EBS Module. Of those youth with correct dosage, 17
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
received at least the recommended hours of service. Youth in the sample completed between 18
and 99 hours of service, with an average of 91 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Duval Academy can maintain their SPEP Quality of Service Delivery score by continuing the
practices in place at the time of this review.
Duval Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

source: DJJ

report_type: SPEP

program_name_raw: Duval Academy - Sequel

program_status_note: Closed

service_type: Living in Balance


summary: SPEP - FY18-19

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Duval Academy
Sequel Youth and Family Services
(Contract Provider)
7500 Ricker Road
Jacksonville, Florida 32244
Primary Service: Thinking for a Change
SPEP Review Date(s): 4/23/19 ‐ 4/26/19
Florida Department of Juvenile Justice
Report Date(s): 4/26/19
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
 Service Type: service type points are awarded based on the group category that the
service falls within
 Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
 Amount of Service: or “dosage” refers to the amount and duration of services that were
received
 Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change. The program was awarded 30
points because the Primary Service is identified as a Group 5 Service. The specific Sub-
Component Service Type identified is Cognitive Behavioral Therapy. The Primary Service was
identified as this type of service as it is intended to correct faulty cognitions and perceptions and
provides skills individuals can use to monitor thought patterns and correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
During the review period there were five staff trained to facilitate Thinking for a Change (T4C).
Two of those staff are no longer employed at the program. Each facilitator was trained in the T4C
by a qualified trainer prior to delivering the service to the youth.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes Thinking for a Change, version 3.1. The curriculum is detailed with clear
explanation of how each specific session is to be delivered. The manual provides a detailed script
for each of the 25 lessons. The curriculum provides a guide for implementation, the order of the
lessons, and detailed instructions for each specific lesson.
Documentation for this service in maintained in a binder specific to this intervention. The binder is
organized and easy to locate, and includes facilitator training documentation, sign in sheets,
fidelity monitoring, copies of pre- and post-tests and end of curriculum surveys. All youth in the
program are assigned to Impact of Crime during their stay. Groups are held Mondays,
Wednesdays, and Fridays from 4:00PM until 5:00PM.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Thinking for a Change was observed by the Technical Assistance Specialist (TAS).
The group was working on lesson eleven-Social Skill: understanding the feelings of others. The
facilitator reviewed the previous session, lesson five: knowing your feelings, and provided
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
information on role play expectations for the day’s group. One youth began getting upset during
the group because he wanted to watch a movie instead of being in the group. The facilitator
demonstrated the skill for the youth and had each of the youth role play the skill back for the
group.
The facilitator did not have any visual aids. The TAS informed the facilitator there were skill cards
for each skill that is taught and recommended to the facilitator he should print those skill cards
each group and give them to the youth, so they can see the steps of the skill.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation revealed two gaps in service, May 1 – July 9, 2018, and December 11,
2018 – February 11, 2019. Additionally, one cohort transitioned to a new facilitator mid-stream
without a gap in service. Two cohorts were completed during the review period, and one was
ongoing at the time of the review. The program’s process to avoid gaps in service is to have the
clinical director conduct weekly meetings with therapists to prepare/plan prior to assigning
facilitators to groups.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has a process in place to monitor the delivery of the Thinking for a Change
intervention. Available documentation confirmed fidelity monitoring was conducted each month
for each facilitator during the months groups were held.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process in place to address corrective action based on fidelity monitoring. An
informal discussion with the regional clinical director indicated fidelity monitoring is conducted
once a month, with the Clinical Director then following up on the monitorings during group
supervision meetings to discuss a plan of action for any corrective actions identified. The clinical
director will then re-address any issues during the next month’s fidelity monitoring. A review of
documentation provided indicated there were no corrective actions noted nor taken during the
review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Performance evaluations for the staff facilitating Thinking for a Change were reviewed. Each
included an evaluation of the therapists’ skill in delivering the service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service ‐ Duration
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates the target duration of 15 weeks for this type of service. Of the 46 youth in the
sample, 33% (15 of 46) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 45 contact hours for this type of service. Of the 46 youth in the
sample, 11% (5 of 46) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 98%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 4 youth
Moderate-High = 13 youth
High = 28 youth
Total Youth in Sample = 46 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 61%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 28 youth
Total Youth in Sample = 46 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 98% (45 of 46) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 61% (28 of 46) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 2 10 20%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 25 25 100%
Totals 82 100 82%
This SPEP report evaluates Thinking for a Change, an intervention delivered at Duval Academy.
46 youth completed the program during the review period. Of those youth, 46 were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 2 points for Amount of Service: Duration. Of the 46 total youth sampled, only
17 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 15
received at least the recommended weeks of service. Youth in the sample completed between 7
and 22 weeks of service, with an average of 19 weeks.
The program earned 0 points for Amount of Service: Contact Hours. Of the 46 total youth
sampled, only 17 included dosage in the EBS Module. Of those youth with correct dosage, 5
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
received at least the recommended hours of service. Youth in the sample completed between 16
and 46 hours of service, with an average of 40 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Duval Academy can optimize their SPEP Quality of Service Delivery score by ensuring facilitator
turnover does not impact groups.
Duval Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

service_type: Thinking for a Change


summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Duval Academy
Sequel Youth and Family Services
(Contract Provider)
7500 Ricker Rd
Jacksonville, Florida 32244
Primary Service: Impact of Crime
SPEP Review Date(s): 6/26/2020
Florida Department of Juvenile Justice
Report Date(s): 6/26/2020
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A review of documentation for the annual review period of April 2019 through April 2020 shows
the program utilizes Impact of Crime as a primary service. The documentation showed there were
five staff trained in the primary service prior to delivering services to youth. Impact of Crime
training is delivered over four days by a qualified trainer.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the Impact of Crime: Addressing the Harm manual. The manual opens with
an introduction to the Impact of Crime Curriculum. Then the manual goes over all seven chapters
of the group and provides a script for the facilitator to follow. The manual also has an appendix
and a student guide with all the student exercises.The program does not have separate student
workbooks to give out. The facilitators make copies of the exercises before each session to pass
out to the youth. The group is a closed group with no more than twelve youth in each cohort. The
curriculum is to be delivered to the youth over a minimum of sixteen weeks for thirty-two sessions.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Impact of Crime was unable to be observed due to the COVID-19 pandemic;
therefore, this indicator is rated as non-applicable.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation for the annual review period confirmed the program had facilitator
turnover. There was one facilitator who began a cohort in July 2019; another facilitator took over
the cohort in August 2019. The documentation showed there was no gap in service due to the
facilitator change. Three cohorts were completed during the review period; the two other cohorts
began and ended with the same facilitator. There were no identifiable gaps in service. The
program was evacuated due to Hurricane Dorian on 9/2/2019.
One cohort began on 7/2/19 began with nine youth. Two of the youth were released from the
program in August 2019. One youth was removed from the group for refusing to attend multiple
groups in August 2019. There were four other youth who were also released from the program in
September 2019. This left two youth in the cohort. The attendance sheets for October 2019 show
there was two youth at the beginning of the month, but only one in the cohort at the end of the
month. There was no documentation why the youth was no longer in the cohort. This cohort
ended on 11/4/2019 with only one youth successfully completing group.
Another cohort began on 11/4/2019 with eleven (11) youth. Six of the youth were discharged from
the program during November 2019 shortly after the cohort began. Two more youth were
discharged in January 2020. One youth was discharged in February 2020. Out of the eleven (11)
youth starting the cohort, only two successfully completed the primary service.
A third cohort was documented during the annual review period. The cohort began on 2/13/2020
with eleven (11) youth. This cohort was ongoing at the end of the review period with all youth
remaining in the cohort.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of documentation for the annual review period of April 2019 through April 2020 showed
there were nine internal fidelity checks completed. There was a letter dated October 2019 from
the clinical director which stated a fidelity check was not completed in October 2019 due to not
having a second trained staff member in the primary service.
The program has a process in place to conduct internal fidelity monitoring. The regional clinical
director conducted monthly fidelity monitoring of each facilitator until the new clinical director was
trained. A trained facilitator may also conduct the fidelity monitoring if the regional clinical director
is unavailable. The regional clinical director discusses the fidelity monitoring with the facilitator to
ensure the facilitator is completing groups to the fidelity of the model.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process in place to address corrective actions resulting from internal fidelity
monitoring. The process begins with the monthly internal fidelity monitoring completed by the
either the clinical director or the regional clinical director. The regional clinical director or the
clinical director conducts the fidelity monitoring and then discusses any findings with the facilitator.
The facilitator will be allowed two weeks to make corrections before being re-evaluated.
The documentation reviewed for the annual review period showed there were no identified
corrective actions noted during internal fidelity monitoring.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The documentation reviewed shows each evaluation has an area called "facilitator skill delivering
the evidence-based programming". Each evaluation reviewed had specific information based on
the facilitator's delivery of the evidence-based programming.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 16 weeks for this type of service. Of the 28 youth in the
sample, 18% (5 of 28) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates a target of 24 contact hours for this type of service. Of the 28 youth in the
sample, 21% (6 of 28) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 93%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 4 youth
Moderate-High = 5 youth
High = 17 youth
Total Youth in Sample = 28 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 61%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 17 youth
Total Youth in Sample = 28 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 93% (26 of 28) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 61% (17 of 28) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 2 10 20%
Risk Level of Youth Served 23 25 92%
Totals 55 85 65%
This SPEP report evaluates Impact of Crime, an intervention delivered at Duval Academy.
32 youth completed the program during the review period. Of those youth, 28 were included in the
sample.
The program scored Medium for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 28 total youth sampled, only
21 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 5
received at least the recommended weeks of service. Youth in the sample completed between 1
and 18 weeks of service, with an average of 11 weeks.
The program earned 2 points for Amount of Service: Contact Hours. Of the 28 total youth
sampled, only 21 included dosage in the EBS Module. Of those youth with correct dosage, 6
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
received at least the recommended hours of service. Youth in the sample completed between 2
and 57 hours of service, with an average of 18 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Duval Academy can optimize their SPEP Quality of Service Delivery score by ensuring facilitator
turnover does not impact groups, and by ensuring fidelity monitoring is completed every month for
each facilitator.
Duval Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

service_type: Impact of Crime


summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Duval Academy
Sequel Youth and Family Services
(Contract Provider)
7500 Ricker Rd
Jacksonville, Florida 32244
Primary Service: Living in Balance
SPEP Review Date(s): 6/26/2020
Florida Department of Juvenile Justice
Report Date(s): 6/29/2020
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Living in Balance.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 0 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A review of documentation for the annual review period of April 2019 through April 2020 showed
there were three staff trained in Living in Balance. The training is an in-service training provided
by the clinical director or assistant clinical director. There were eight staff who facilitated the
primary service during the annual review period. There was no documentation of five facilitators'
training.
The program can earn 1 point if all facilitators receive formal training prior to delivering services to
youth.
At the time of the review, the program did not meet criteria to earn a score of 1.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program's Standardized Program Evaluation Protocol (SPEP) annual review period is April
2019 through April 2020. The program conducts Living in Balance (LIB) as an open group and
has consistently held group during the review period. The curriculum is offered to youth who are
diagnosed with a substance abuse diagnosis as one of the top three criminogenic needs on the
youth's Residential Assessment for Youth. Each manual also includes CDs for facilitator
resources and session handouts. The manuals include facilitation guidelines. There are a total of
forty-seven lessons for LIB. The sessions are to be sixty to seventy-five minutes. The manuals
include specific instructions for each lesson and utilizes role plays, video, and discussion.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Living in Balance was unable to be observed due to the COVID-19 pandemic;
therefore, this indicator is rated as non-applicable.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation for the annual review period showed the program had facilitator
turnover; however, the facilitator turnover did not adversely affect facilitation of Living in Balance..
The former clinical director left the program in August 2019. There were no gaps in service.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of documentation for the annual review period of April 2019 through April 2020 showed
there were twenty-three internal fidelity checks completed. Internal fidelity monitoring was
conducted each month for each facilitator, without exception.
The program has a process in place to conduct internal fidelity monitoring. The regional clinical
director conducted monthly fidelity monitoring of each facilitator until the new clinical director was
trained. A trained facilitator may also conduct the fidelity monitoring if the regional clinical director
is unavailable. The regional clinical director discusses the fidelity monitoring with the facilitator to
ensure the facilitator is completing groups to the fidelity of the model.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process in place to address corrective actions resulting from internal fidelity
monitoring. The process begins with the monthly internal fidelity monitoring completed by the
either the clinical director or the regional clinical director. The regional clinical director or the
clinical director conducts the fidelity monitoring and then discusses any findings with the facilitator.
The facilitator will be allowed two weeks to make corrections before being re-evaluated.
The documentation reviewed for the annual review period showed there were no identified
corrective actions noted during the internal fidelity checks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The documentation reviewed shows each evaluation has an area called "facilitator skill delivering
the evidence-based programming". Each evaluation reviewed had specific information based on
the facilitator's delivery of the evidence-based programming.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 24 weeks for this type of service. Of the 31 youth in the
sample, 58% (18 of 31) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates a target of 40 contact hours for this type of service. Of the 31 youth in the
sample, 29% (9 of 31) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 94%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 4 youth
Moderate-High = 8 youth
High = 17 youth
Total Youth in Sample = 31 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 55%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 17 youth
Total Youth in Sample = 31 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 94% (29 of 31) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 55% (17 of 31) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 2 10 20%
Risk Level of Youth Served 23 25 92%
Totals 81 95 85%
This SPEP report evaluates Living in Balance, an intervention delivered at Duval Academy.
32 youth completed the program during the review period. Of those youth, 31 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 6 points for Amount of Service: Duration. Of the 31 total youth sampled, only
19 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 18
received at least the recommended weeks of service. Youth in the sample completed between 2
and 43 weeks of service, with an average of 31 weeks.
The program earned 2 points for Amount of Service: Contact Hours. Of the 31 total youth
sampled, only 20 included dosage in the EBS Module. Of those youth with correct dosage, 9
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
received at least the recommended hours of service. Youth in the sample completed between 10
and 82 hours of service, with an average of 36 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Duval Academy can optimize their SPEP Quality of Service Delivery score by ensuring all
facilitators receive formal training prior to delivering services to youth.
Duval Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9


summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Duval Academy
Sequel Youth and Family Services
(Contract Provider)
7500 Ricker Rd
Jacksonville, Florida 32244
Primary Service: Thinking for a Change
SPEP Review Date(s): 6/26/2020
Florida Department of Juvenile Justice
Report Date(s): 6/29/2020
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A review of documentation for the annual review period of April 2019 through April 2020 shows
the program utilizes Thinking for a Change as a primary service. The documentation reviewed
showed there were four staff trained in the primary service prior to delivering services to youth.
Thinking for a Change training is four-days, conducted by a qualified trainer.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes Thinking for a Change (T4C) as a primary service. The manual provides a
script and instructions for all twenty-five lessons. The curriculum also includes the facilitation
guidelines, skill cards, and homework pages for the youth.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Thinking for a Change was unable to be observed due to the COVID-19 pandemic;
therefore, this indicator is rated as non-applicable.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
A review of the documentation for the annual review period showed the program had facilitator
turnover. The documentation reviewed showed there were four cohorts during the annual review
period. The first cohort began before April 2019 and ended on 6/23/2019. There was a note in the
file stating the cohort ended due to the facilitator going on paternity leave. The clinical director
then left the program in August 2019.
The second cohort began on 7/17/2019 and ended on 10/23/2019. The cohort began and ended
with nine youth. The third cohort began on 10/25/2019 and ended on 2/5/2020. The cohort began
with five youth. One youth completed the program in November 2019 as documented by the
attendance sheets. A youth refused group on 1/31/2020, but was given an individual session after
group as documented on the attendance sheet. The fourth cohort began on 2/19/2020 and was
ongoing at the end of the review period. The cohort began with ten (10) youth; all ten (10) youth
remained in the group as of the end of the review period.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of documentation for the annual review period of April 2019 through April 2020 showed
there were ten internal fidelity checks completed. There was no fidelity monitoring provided for
April 2019 through June 2019.
The program has a process in place to conduct internal fidelity monitoring. The regional clinical
director conducted monthly fidelity monitoring of each facilitator until the new clinical director was
trained. A trained facilitator may also conduct the fidelity monitoring if the regional clinical director
is unavailable. The regional clinical director discusses the fidelity monitoring with the facilitator to
ensure the facilitator is completing groups to the fidelity of the model.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process in place to address corrective actions resulting from internal fidelity
monitoring. The process begins with the monthly internal fidelity monitoring completed by the
either the clinical director or the regional clinical director. The regional clinical director or the
clinical director conducts the fidelity monitoring and then discusses any findings with the facilitator.
The facilitator will be allowed two weeks to make corrections before being re-evaluated.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The documentation reviewed for the annual review period showed there were no identified
corrective actions noted during the internal fidelity checks.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The documentation reviewed shows each evaluation has an area called "facilitator skill delivering
the evidence-based programming". Each evaluation reviewed had specific information based on
the facilitator's delivery of the evidence-based programming.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates the target duration of 15 weeks for this type of service. Of the 31 youth in the
sample, 48% (15 of 31) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 45 contact hours for this type of service. Of the 31 youth in the
sample, 10% (3 of 31) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 94%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 4 youth
Moderate-High = 8 youth
High = 17 youth
Total Youth in Sample = 31 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 55%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 17 youth
Total Youth in Sample = 31 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 94% (29 of 31) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 55% (17 of 31) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 4 10 40%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 23 25 92%
Totals 72 100 72%
This SPEP report evaluates Thinking for a Change, an intervention delivered at Duval Academy.
32 youth completed the program during the review period. Of those youth, 31 were included in the
sample.
The program scored Medium for Quality of Service Delivery.
The program earned 4 points for Amount of Service: Duration. Of the 31 total youth sampled, only
19 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 15
received at least the recommended weeks of service. Youth in the sample completed between 3
and 44 weeks of service, with an average of 20 weeks.
The program earned 0 points for Amount of Service: Contact Hours. Of the 31 total youth
sampled, only 19 included dosage in the EBS Module. Of those youth with correct dosage, 3
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
received at least the recommended hours of service. Youth in the sample completed between 11
and 48 hours of service, with an average of 27 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Duval Academy can optimize their SPEP Quality of Service Delivery score by ensuring facilitator
turnover does not impact groups, and by ensuring fidelity monitoring is completed every month for
each facilitator.
Duval Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9