summary: SPEP - FY17-18

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Okaloosa Youth Academy
Gulf Coast Youth Services
(Contract Provider)
4445 Straightline Road
Crestview , Florida 32455
Primary Service: ARISE
SPEP Review Date(s): October 3 ‐ 6, 2017
Florida Department of Juvenile Justice
Report Date(s): 12/20/2017
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Okaloosa Youth Academy/Crestview Sex Offender Program
MQI Program Code: 830/1161
Provider Name: Gulf Coast Youth Services Contract Number: 10288
Location: Okaloosa County / Circuit: 1 Number of Beds: 60/15
Review Date(s): October 3 - 6, 2017 Lead Reviewer Code: 145/144
Persons Interviewed
Program Director Corporate QI/QA staff # Program Supervisors
DJJ Monitor # Case Managers # Youth
DHA or designee # Clinical Staff # Other (listed by title):
DMHCA or designee # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 7 # Personnel Records
Fidelity Monitoring Documents Program Schedules 7 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts # Youth Records (Closed)
Staff Evaluations Table of Organization # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports # Health Records
Contract Scope of Services # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is ARISE. The program was awarded 25 points because
the Primary Service is identified as a Group 4 Service. The specific Sub-Component Service
Type identified is Group Counseling. The Primary Service was identified as this type of service as
it focuses on psychological or interpersonal problems or issues faced by an individual and
involves a group of youths interacting with each other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 6 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
Training certificates were provided to all therapy staff who completed the 14-hour ARISE training
curriculum which is conducted by the ARISE founders. Five of the seven therapists trained in
ARISE conducted this curriculum at least once over the review period of 12 months.
Basic Score: 1 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The ARISE curriculum is split into various workbooks that are completed by the youth during
group. Each workbook covers various topics and the youth are able to work at their own pace
during group. The group has a range of topics they discuss during their sessions.
There is an instructor’s manual with each set of workbooks. The instructor's manual does not
have a script for the facilitator to use. The facilitator is supposed to work with the youth where they
are in their stages of change based on the workbook topic. Within the ARISE curriculum, the
youth may work in smaller groups or as one large group, if the facilitator desires. The facilitator
should measure how well the youth interact and participate during the group to determine their
performance in this curriculum. The instructor’s manual is separated by lessons and worksheets
based on the youth’s workbook. There is also a resource manual with extra worksheets and
activities for the youth to complete.
The program can earn 2 points if the manual specifically details the delivery of this primary
service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The ARISE groups are conducted on Saturdays and Sundays and were not monitored during this
review.
This curriculum was not able to be observed; therefore, this indicator rates as non-applicable.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The attendance sheets for this curriculum were unorganized. A random sampling was taken of the
attendance sheets. The attendance sheets included the title of session, youth names and
signatures, date of group, and a place for the therapist to sign. Many of the attendance sheets
from the sample did not have a therapists name and/or signature. It was difficult for this reviewer
to determine which therapist facilitated group and on what date.
The facility consistently employed six therapists over the past 12 months. Per discussion with the
Designated Mental Health Clinical Authority, as a new therapist is hired, they are trained in the
ARISE curriculum. Since this group is held on the weekends, the facilitators of this group change
every weekend. If a therapist leaves, there are others that can take that weekend group so there
is no disruption to the curriculum.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The internal fidelity monitoring sheets were provided by the Designated Mental Health Clinical
Authority for the review period. One of the six facilitators for this group had all 12 months of
internal fidelity monitoring. The internal fidelity monitoring was sporadic for the other five
facilitators trained in the ARISE curriculum.
It was not possible to determine if fidelity monitoring was completed every month for every
facilitator due to the missing information on sign-in sheets.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
There were no corrective actions indicated on the internal fidelity monitoring forms that were
reviewed.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Five out of the six therapists trained had a completed yearly performance evaluation. The sixth
therapist left employment prior to the first evaluation being completed.
On the yearly performance evaluations for therapists, there is a designated section for group
facilitation. Under this section, multiple service types are listed and a check mark is placed next to
anything the therapist facilitates.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 6%
Research indicates the target duration of 24 weeks for this type of service. Of the 30 youth in the
sample, 63% (19 of 30) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 40 contact hours for this type of service. Of the 30 youth in the
sample, 100% (30 of 30) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 90%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 5 youth
Moderate-High = 10 youth
High = 12 youth
Total Youth in Sample = 30 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 40%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 12 youth
Total Youth in Sample = 30 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 90% (27 of 30) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 40% (12 of 30) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 23 25 92%
Totals 79 95 83%
This SPEP report evaluates ARISE, an intervention delivered at Okaloosa Youth
Academy/Crestview Sex Offender Program.
The program scored Medium for Quality of Service Delivery.
The program earned 6 points for Amount of Service: Duration. 63% received at least the
recommend weeks of service, with sample youth completing an average of 31 weeks of service.
The program earned 10 points for Amount of Service: Contact Hours. 100% received at least the
recommended hours of service, with sample youth completing an average of 54 hours of service.
Dosage was calculated for both Okaloosa Youth Academy and for Crestview Sex Offender
Program together as the programs were reviewed together.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
itself has no control over youths' C-PACT risk level because the scored assessment was
administered just prior to the youths' admission.
RECOMMENDATION(S):
Okaloosa Youth Academy/Crestview Sex Offender Program can optimize their SPEP Quality of
Service Delivery score by minimizing facilitator turnover, ensuring internal fidelity monitoring is
conducted at least monthly for all facilitators, and by utililizing detailed manuals and plans.
Okaloosa Youth Academy/Crestview Sex Offender Program can optimize their SPEP Amount of
Service score by ensuring that youth receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

service_type: ARISE


state: FL

facility_name: Okaloosa Youth Academy

summary: SPEP - FY17-18

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Okaloosa Youth Academy
Gulf Coast Youth Services
(Contract Provider)
4445 Straightline Road
Crestview, Florida 32455
Primary Service: Impact of Crime (IOC)
SPEP Review Date(s): October 3 ‐ 6, 2017
Florida Department of Juvenile Justice
Report Date(s): 12/20/2017
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Okaloosa Youth Academy/Crestview Sex Offender Program
MQI Program Code: 830/1161
Provider Name: Gulf Coast Youth Academy Contract Number: 10288
Location: Okaloosa County / Circuit: 1 Number of Beds: 60/15
Review Date(s): October 3 - 6, 2017 Lead Reviewer Code: 145/144
Persons Interviewed
Program Director Corporate QI/QA staff 1 # Program Supervisors
DJJ Monitor 3 # Case Managers 3 # Youth
DHA or designee # Clinical Staff # Other (listed by title):
DMHCA or designee # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 5 # Personnel Records
Fidelity Monitoring Documents Program Schedules # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts # Youth Records (Closed)
Staff Evaluations Table of Organization # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports # Health Records
Contract Scope of Services # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime (IOC). The program was awarded 15
points because the Primary Service is identified as a Group 3 Service. The specific Sub-
Component Service Type identified is Social Skills Training. The Primary Service was identified
as this type of service as it focuses on developing social skills required for an individual to interact
in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program employs five case managers formally trained in the Impact of Crime (IOC)
curriculum. Two of the case managers received formal training in August 2012. Three case
managers received formal training in March 2017. Two of the facilitators have been consistently
facilitating this group. The three case managers who were trained in March of 2017 recently
completed their external fidelity monitoring by the Technical Assistance Specialist.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The manual provided was written in 2003. This manual did not have the new exercise (exercise
nine) in Chapter five for the youth to complete, however, the updated IOC exercises were
provided to the case managemers so that all case managers have up-to-date training material.
The manual contains a script for the facilitator to cover the three sections included in the manual.
The first section is the facilitator manual, the second is the appendix which includes helpful and
detailed information about IOC and extra sets of forms for the training. The third section is the
student handbook, which is divided into seven chapters and contained all of the student
worksheets.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
An IOC group was observed on October 5, 2017. This session happened to be the first meeting
for this particular group. Prior to session beginning, this reviewer had the opportunity to walk with
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
the youth to the dining hall and sit with them while they ate. This group was facilitated by one case
manager, however, a second case manager was sitting in to assist with the group as needed.
The group had eight youth registered for this cohort but on this particular day, only six youth
attended the group. The facilitator welcomed the youth and began with explaining what Impact of
Crime was and explained Restorative Justice. The facilitator then completed an icebreaker.
During this group there was one behavioral issue. The facilitator and case manager spoke to the
youth and maintained a calm demeanor throughout the incident. They both followed the program's
policy and procedure and the situation was handled without further incident.
Overall, the facilitator handled the class well and explained the first objective well.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There has been no facilitator turnover in the past 12 months. The program increased the amount
of Impact of Crime facilitators from two to five within the past 12 months.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Two of the five facilitators for Impact of Crime conducted internal fidelity monitoring for all of the
groups. Once the other three have completed their external fidelity monitoring they will be able to
conduct internal fidelity monitoring.
There were three cohorts of Impact of Crime completed during the review period of 12 months.
The first cohort was held April 19, 2017 through July 31, 2017, the second cohort was held July 6,
2017 through September 9, 2017, and the third cohort started April 17, 2017 and is still being
held. All internal fidelity monitoring was completed.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The case management supervisor provided all internal fidelity documents for Impact of Crime
groups. There were no corrective actions indicated on the internal fidelity monitoring forms. There
were various suggestions made to some of the facilitators.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Five yearly performance evaluations were provided for review. There was no specific section for
group facilitation or discussion of IOC.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 16 weeks for this type of service. Of the 6 youth in the
sample, 0% (0 of 6) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 24 contact hours for this type of service. Of the 6 youth in the
sample, 0% (0 of 6) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 1 youth
Moderate-High = 1 youth
High = 4 youth
Total Youth in Sample = 6 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 67%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 4 youth
Total Youth in Sample = 6 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (6 of 6) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 67% (4 of 6) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 25 25 100%
Totals 65 85 76%
This SPEP report evaluates Impact of Crime (IOC), an intervention delivered at Okaloosa Youth
Academy/Crestview Sex Offender Program.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. 0% received at least the
recommend weeks of service, with sample youth completing an average of 13 weeks of service.
four of the six youth sampled did not include a service end date, so it was impossible to calculate
the accurate weeks of service. The program earned 0 points for Amount of Service: Contact
Hours. 0% received at least the recommended hours of service, with sample youth completing an
average of 8 hours of service. Dosage was calculated for both Okaloosa Youth Academy and for
Crestview Sex Offender Program together as the programs were reviewed together; however,
there was no dosage recorded for this intervention for any youth from Crestview Sex Offender
Program.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered just prior to the youths' admission.
RECOMMENDATION(S):
Okaloosa Youth Academy/Crestview Sex Offender Program can optimize their SPEP Quality of
Service Delivery score by ensuring facilitators' performance evaluations include an assessment of
their delivery of IOC.
Okaloosa Youth Academy/Crestview Sex Offender Program can optimize their SPEP Amount of
Service score by ensuring that data is entered correctly in the EBS Module of JJIS and by
ensuring youth receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

source: DJJ

report_type: SPEP

program_name_raw: Okaloosa Youth Academy

program_status_note: Open

service_type: Impact of Crime


summary: SPEP - FY18-19

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Okaloosa Youth Academy
Gulf Coast Youth Services
(Contract Provider)
4445 Straightline Road
Crestview, Florida 32539
Primary Service: Seven Challenges
SPEP Review Date(s): 4/29/19‐5/3/19
Florida Department of Juvenile Justice
Report Date(s): 5/7/19
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
 Service Type: service type points are awarded based on the group category that the
service falls within
 Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
 Amount of Service: or “dosage” refers to the amount and duration of services that were
received
 Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. 1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Seven Challenges.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program provided all facilitator training records for therapy staff at the program. Eight
therapists were trained to facilitate Seven Challenges during the review period. The designated
mental health clinical authority conducts informal training for therapy staff once hired. Each
therapist trained received a certificate of completion. Two therapists have completed the follow up
courses for seven challenges.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
Seven Challenges is a substance abuse curriculum provided to the youth who have current
substance abuse as a top criminogenic need on the Residential Positive Achievement Change
Tool. The group is held Monday through Friday from 2:15pm to 3:15pm. A second group is held
on Mondays, Wednesdays, and Fridays from 3:15pm to 4:15pm. Seven Challenges is held as an
open group, meaning as youth come into the program and qualify for the seven challenges
curriculum, they can join the group immediately.
The program provided the seven challenges manual for review. The youth have a separate
workbook for this curriculum they work throughout their time in the group. Facilitators use the
guide to read through scenarios and utilize the discussion questions with the group applying the
scenarios to real-life events and experiences. There is no specific script for the curriculum.
The manual contains an introductory chapter that describes the curriculum and the goals of the
program. Chapter seven of the manual descrives how the curriculum should be implemented.
This group includes books and journals so youth can record their thoughts and complete activities
in the sessions. The workbooks and journals provide the worksheets and journaling pages for the
youth for those sessions. The working sessions are meant for the youth to work through their
stages of change when it comes to substance use and abuse.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A Seven Challenges group was observed on Tuesday, April 30, 2019 from 3:15pm to 4:15pm.
This contradicts the schedule provided; however, some of the group schedules were changed
during the annual review week and based upon changes in the therapists schedules.
There were eight youth in the group. The facilitator conducted the check-in with all the youth. All
the youth participated in the group and provided good examples and discussion. The youth
volunteered to read and assisted in group. The facilitator had good control of the group and there
was a healthy relationship between the therapist and the youth attending group. The facilitator did
use the manual to follow along with the reading of the scenario and discussion questions, but was
comfortable with the content in the lesson and the guided discussion.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
According to a review of program records there has been some turnover with therapy staff;
however, it has not adversely affected group facilitation. A review of the JJIS Evidence-Based
Services module, and a random sample of the attendance sheets for Seven Challenges, indicated
there have not been gaps in service. When a therapist is out or leaves employment, another
therapist is able to pick up that group. There are currently seven trained facilitators.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of program documentation indicated eight therapists have facilitated seven challenges
groups during the review period. A review of the internal fidelity monitoring forms revealed
therapists observe one another to conduct internal fidelity monitoring. All eight therapists received
internal fidelity monitoring for the months they facilitated seven challenges.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The designated mental health clinical authority conducts weekly supervisions with therapy staff.
During those meetings any issues or corrective actions are addressed. A review of the internal
fidelity monitoring forms did not indicate corrective actions were needed during the review period.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Six of the eight therapists who facilitated Seven Challenges groups received a yearly performance
evaluation for review. One therapist does not facilitate seven challenges and the other therapist
began employment April 5, 2019. This therapist had not received an evaluation at the time of the
review. The yearly performance evaluations include a section to address group faciltiation for
therapists. There is a list of all curriculum and each is checked based on what curricula the
therapist facilitates.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service ‐ Duration
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates the target duration of 24 weeks for this type of service. Of the 93 youth in the
sample, 43% (40 of 93) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 40 contact hours for this type of service. Of the 93 youth in the
sample, 87% (81 of 93) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 91%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 8 youth
Moderate-High = 25 youth
High = 52 youth
Total Youth in Sample = 93 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 56%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 52 youth
Total Youth in Sample = 93 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 91% (85 of 93) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 56% (52 of 93) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 4 10 40%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 23 25 92%
Totals 85 95 89%
This SPEP report evaluates Seven Challenges, an intervention delivered at Okaloosa Youth
Academy.
115 youth completed the program during the review period. Of those youth, 93 were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 4 points for Amount of Service: Duration. Of the 93 total youth sampled, only
88 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 40
received at least the recommended weeks of service. Youth in the sample completed between 1
and 60 weeks of service, with an average of 25 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 8 points for Amount of Service: Contact Hours. Of the 93 total youth
sampled, all 93 included dosage in the EBS Module. Of those youth, 81 received at least the
recommended hours of service. Youth in the sample completed between 29 and 147 hours of
service, with an average of 81 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Okaloosa Youth Academy can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Okaloosa Youth Academy can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

service_type: Seven Challenges


summary: SPEP - FY18-19

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Okaloosa Youth Academy
Gulf Coast Youth Services
(Contract Provider)
4445 Straightline Road
Crestview, Florida 32539
Primary Service: Anger Management for Substance Abuse and Mental
Health Clients
SPEP Review Date(s): 4/30/19‐5/3/19
Florida Department of Juvenile Justice
Report Date(s): 5/6/19
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
 Service Type: service type points are awarded based on the group category that the
service falls within
 Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
 Amount of Service: or “dosage” refers to the amount and duration of services that were
received
 Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Anger Management for Substance Abuse and Mental
Health Clients.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The review period for Anger Management for Substance Abuse and Mental Health Clients runs
from October 16, 2018 through April 2019. The program designated this curriculum as a primary
on October 16, 2018. The groups did not begin as a primary service until November 1 for the
program.
The program's documentation reflected the program has six therapists trained in the primary
service. Five therapists were trained by the Designated Mental Health Clinical Authority (DMHCA)
on October 31, 2018 and one therapist was trained on April 1, 2019. The DMHCA indicated
during an interview all therapists receive informal training when hired for the curriculum and can
immediately begin facilitating group.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
Anger Management for Substance Abuse and Mental Health Clients is held on Tuesdays and
Thursdays from 2:15pm to 3:15pm and 3:15pm to 4:15pm. The attendance sheets for the review
period were provided for review. The attendance sheets are kept in the Designated Mental Health
Clinical Authority's (DMHCA) office, rubber-banded together by month with all other mental health
groups.
Youth will receive this curriculum based upon the results of the Residential Positive Achievement
Change Tool (R-PACT). The curriculum can be provided to youth for just mental health; a
substance abuse diagnosis is not required, pursuant to the facilitator manual. The requirements to
facilitate this curriculum are facilitators should be trained in cognitive behavioral therapy, group
therapy, and substance abuse treatment.
The DMHCA provided the manual for review. It is maintained in a binder, with twelve chapters to
complete. The instructor's manual provides the dosage infromation of twelve weeks and ninety
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
minute sessions, but can be held as sixty minutes. The sessions provide a script and outline of
how the session should be facilitated. The student guide provides all worksheets for the lessons.
The youth do not receive individual workbooks. The facilitators make copies of the worksheets for
the youth to complete during group.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A group session of Anger Management for Substance Abuse and Mental Health Clients was
observed on Tuesday, April 30, 2019 from 2:15pm to 3:15pm. The group had eight youth in
attendance. Group is held in a classroom with chairs formed into a half circle. The group started
and ended on time. The lesson of anger control plans was continued from the previous group
session. The facilitator had good control of the group and the youth had a good level of respect for
the facilitator. All youth participated in group. The facilitator conducted the lesson as provided in
the manual.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of the JJIS Evidence-Based Services module and a random sampling of attendance
sheets indicated there were no gaps in service. There has not been turnover with therapist staff.
When a therapist does leave or is out, there are numerous therapists who are able to step in and
continue the groups. All therapists are trained by the Designated Mental Health Clinical Authority.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Upon review of internal fidelity monitoring forms, the therapists typically observe one another for
internal fidelity monitoring. Of the eight therapists who conducted Anger Management for
Substance Abuse and Mental Health Clients, all received internal fidelity monitoring for the
months they facilitated groups.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The Designated Mental Health Clinical Authority (DMHCA) conducts weekly supervision with the
therapy staff members. These meeting notes were reviewed by the Monitoring and Quality
Improvement review team. The DMHCA will discuss any issues or corrective actions during those
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
meetings or during one-to-one meetings. Upon review of internal fidelity monitoring records for
the review period, no corrective actions were noted for therapy staff.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program provided six of seven annual performance evaluations for the therapy staff. The
other therapist was employed at the program on March 8, 2019 and has not yet received an
annual evaluation. The annual performance evaluations include a section for group facilitation. It
also lists all therapeutic groups provided at the program. Each therapist has a check mark next to
each curriculum they are trained in to facilitate.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 24 weeks for this type of service. Of the 30 youth in the
sample, 0% (0 of 30) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 40 contact hours for this type of service. Of the 30 youth in the
sample, 0% (0 of 30) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 93%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 3 youth
Moderate-High = 6 youth
High = 19 youth
Total Youth in Sample = 30 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 63%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 19 youth
Total Youth in Sample = 30 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 93% (28 of 30) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 63% (19 of 30) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 23 25 92%
Totals 73 95 77%
This SPEP report evaluates Anger Management for Substance Abuse and Mental Health Clients,
an intervention delivered at Okaloosa Youth Academy.
115 youth completed the program during the review period. Of those youth, 30 were included in
the sample. This interention was added as a primary service November 1, 2018.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 30 total youth sampled, all
30 included dosage with end dates in the EBS Module. Of those youth, 0 received at least the
recommended weeks of service. Youth in the sample completed between 2 and 22 weeks of
service, with an average of 12 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 0 points for Amount of Service: Contact Hours. Of the 30 total youth
sampled, all 30 included dosage in the EBS Module. Of those youth, 0 received at least the
recommended hours of service. Youth in the sample completed between 1 and 12 hours of
service, with an average of 11 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Okaloosa Youth Academy can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Okaloosa Youth Academy can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9


summary: SPEP - FY18-19

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Okaloosa Youth Academy
Gulf Coast Youth Services
(Contract Provider)
4445 Straightline Road
Crestview, Florida 32455
Primary Service: Impact of Crime
SPEP Review Date(s): 4/30/19‐5/3/19
Florida Department of Juvenile Justice
Report Date(s): 5/6/19
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
 Service Type: service type points are awarded based on the group category that the
service falls within
 Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
 Amount of Service: or “dosage” refers to the amount and duration of services that were
received
 Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime. The program was awarded 15 points
because the Primary Service is identified as a Group 3 Service. The specific Sub-Component
Service Type identified is Social Skills Training. The Primary Service was identified as this type of
service as it focuses on developing social skills required for an individual to interact in a positive
way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 6 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program documentation reflected there are two case managers, one training coordinator, and
the case management supervisor currently trained Impact of Crime (IOC) curriculum. All four staff
members had formal IOC facilitator training prior to facilitating groups. One of three staff members
conducted an IOC group during the review period.
An interview with the case management supervisor confirmed all case management staff is to be
trained to facilitate IOC groups in the program. There are currently two case management staff
who are enrolled in the next available IOC facilitator training. One of the two case managers is
responsible for conducting IOC groups at this program.
Basic Score: 1 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The review encompasses twelve months of records examinations for the Impact of Crime (IOC)
curriculum. Upon review of the attendance records, there have been two cohorts held during the
review period from July 3, 2018 to September 13, 2018 and January 15, 2019 to April 25, 2019.
The group prior to July 2018 was held February 15, 2018 to May 24, 2018. During this group five
out of eight youth were released from the program prior to completing the group. The group
conducted July 3, 2018 to September 13, 2018 had attendance records through chapter four.
There is no record of the eight youth in this group completing IOC. The group held January 15,
2019 to April 25, 2019 ended after chapter six and did not complete chapter seven. The training
protocol for IOC indicates the curriculum must be provided in chapter order and the group should
complete the entire curriculum for it to be effective.
The youth enrolled in the IOC groups during the review period were unable to complete the
groups due to leaving the program. This reviewer discussed with the case management
supervisor the youth placed in the group must have the opportunity to complete the IOC group
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
prior to release from the program. It was discussed to place youth in a group that come in
together to ensure they are able to successfully complete IOC prior to release.
The program does maintain the correct IOC manual for faciltiation. The manual includes all
facilitator script and lessons. The manual also includes the facilitation information and youth
workbook.
The program can earn 2 points if the manual specifically details the delivery of this primary
service, and the program adheres to the protocol.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
This reviewer was unable to observe an Impact of Crime (IOC) group during the review week. The
program is not currently running a group. This curriculum was not able to be observed; therefore,
this indicator rates as non-applicable.
Basic Score: 0 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The reviewed documentation reflected there had been turnover in case management during the
review period; however, this turnover did not adversely affect the Impact of Crime (IOC) group for
the program. When the other case managers left, there were still four trained facilitators at the
program. The program also has an internal trainer for IOC facilitators, but no trainings were
scheduled during the review period at the program.
There have been two gaps in service during the review period that were not due to trained
facilitators leaving the program. One gap occurred between May 2018 and July 2018 and the
second occurred between September 2018 and January 2019. The case management supervisor
indicated Thinking for a Change (T4C) groups were running during the gaps in service.
The program can earn 2 points if there are no unnecessary gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program currently has the case managers monitoring each other's Impact of Crime (IOC)
groups. This reviewer read twelve months of internal fidelity monitoring reports for IOC. For the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
two cohorts held during the review period, all internal fidelity monitoring forms were accounted for
both faciltiators.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a protocol in place to address any corrective actions based on fidelity
monitoring. The case management supervisor will address the issues with the case manager
one-on-one. A review of documentation provided indicated no corrective actions were noted nor
taken.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The case manager who conducts Impact of Crime for this program has received a yearly
performance evaluation. The yearly performance evaluations have a specific section for group
facilitation. All curricula the case managers are trained in and facilitate are included in this section.
The training coordinator and case management supervisor, although trained, do not facilitate
Impact of Crime groups with the youth.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 16 weeks for this type of service. Of the 110 youth in
the sample, 5% (5 of 110) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 24 contact hours for this type of service. Of the 110 youth in the
sample, 7% (8 of 110) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 90%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 12 youth
Moderate-High = 29 youth
High = 58 youth
Total Youth in Sample = 110 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 53%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 58 youth
Total Youth in Sample = 110 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 90% (99 of 110) youth scored Moderate to High-Risk to reoffend, for a
score of 10 points.
Of the SPEP sample, 53% (58 of 110) youth scored High-Risk to reoffend, for a score of 13
points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 23 25 92%
Totals 53 85 62%
This SPEP report evaluates Impact of Crime, an intervention delivered at Okaloosa Youth
Academy.
115 youth completed the program during the review period. Of those youth, 110 were included in
the sample.
The program scored Medium for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 110 total youth sampled,
only 19 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 5
received at least the recommended weeks of service. Youth in the sample completed between 5
and 20 weeks of service, with an average of 14 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 0 points for Amount of Service: Contact Hours. Of the 110 total youth
sampled, only 27 included dosage in the EBS Module. Of those youth with correct dosage, 8
received at least the recommended hours of service. Youth in the sample completed between 3
and 24 hours of service, with an average of 15 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Okaloosa Youth Academy can optimize their SPEP Quality of Service Delivery score by ensuring
the manual specifically details the delivery of this primary service, and the program adheres to the
protocol, and by ensuring there are no unnecessary gaps in service
Okaloosa Youth Academy can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9


summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Okaloosa Youth Academy
Gulf Coast Youth Services
(Contract Provider)
4445 Straightline Road
Crestview, Florida 32539
Primary Service: Seven Challenges
SPEP Review Date(s): 5/22/2020
Florida Department of Juvenile Justice
Report Date(s): 5/22/2020
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Seven Challenges.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
Therapy staff are trained to conduct the Seven Challenges curriculum. Seven therapists have
been trained and all seven have conducted the Seven Challenges curriculum throughout both
sides of the program.
All training records were provided for review. The therapy staff are trained by the designated
mental health clinician authority (DMHCA). The training records are maintained in the DMHCA's
office.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
Seven Challenges is a substance abuse curriculum provided to youth who have current
substance abuse as a top criminogenic need on the Residential Assessment for Youth or have a
substance abuse diagnosis. The group is held Monday through Friday from 2:15pm to 3:15pm,
according to the program schedule. The designated mental health clinician authority (DMHCA)
stated the program has incorporated Anger Control and Arise material into the Seven Challenges
curriculum three days per week, while delivering Seven Challenges 5 days per week. Seven
Challenges is held as an open group, meaning as youth come into the program and qualify for the
Seven Challenges curriculum, they can join the group immediately.
The program provided the seven challenges manual for review. The youth have a separate
workbook for this curriculum they work throughout their time in the group. Facilitators use the
guide to read through scenarios and utilize the discussion questions with the group applying the
scenarios to real-life events and experiences. There is no specific script for the curriculum.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Seven Challenges was not observed during this review period. Group observations
will resume during quarterly reviews throughout this next annual review period. This indicator is
rated as non-applicable.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
All therapy staff are trained in the Seven Challenges curriculum and rotate leading groups. Should
there be turnover, another therapist will continue facilitating the group to avoid gaps in service.
Upon review of the attendance sheets for the review period there were no gaps in service. Any
turnover in therapy staff did not affect the groups.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Trained therapists conduct internal fidelity monitoring for Seven Challenges groups. All internal
fidelity monitoring forms were provided for review. There were no missing internal fidelity
monitoring forms for this review period
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
Any issues or corrective actions noted are addressed by the therapist observing the group or the
designated mental health clinician authority (DMHCA). Upon review of the internal fidelity
monitoring forms there were no significant issues or corrective actions noted.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Four therapists have received an annual performance evaluation. Three therapistis have not been
at the program for a full year to receive a performance evaluation. The performance evaluations
completed were provided for review. There is a section designated on the evaluation to address
what curricula the therapist is trained in and an evaluation of facilitation.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates the target duration of 24 weeks for this type of service. Of the 72 youth in the
sample, 35% (25 of 72) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 40 contact hours for this type of service. Of the 72 youth in the
sample, 75% (54 of 72) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 92%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 8 youth
Moderate-High = 24 youth
High = 34 youth
Total Youth in Sample = 72 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 47%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 34 youth
Total Youth in Sample = 72 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 92% (66 of 72) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 47% (34 of 72) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 2 10 20%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 23 25 92%
Totals 81 95 85%
This SPEP report evaluates Seven Challenges, an intervention delivered at Okaloosa Youth
Academy.
91 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 2 points for Amount of Service: Duration. Of the 72 total youth sampled, all
72 included dosage with end dates in the EBS Module. Of those youth, 25 received at least the
recommended weeks of service. Youth in the sample completed between 4 and 51 weeks of
service, with an average of 24 weeks.
The program earned 6 points for Amount of Service: Contact Hours. Of the 72 total youth
sampled, all 72 included dosage in the EBS Module. Of those youth, 54 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
recommended hours of service. Youth in the sample completed between 7 and 100 hours of
service, with an average of 75 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Okaloosa Youth Academy can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Okaloosa Youth Academy can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8