summary: SPEP - FY17-18

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Crestview Youth Academy Secure
Youth Opportunities Investment
(Contract Provider)
4445 Straightline Road
Crestview, Florida 32539
Primary Service: Impact of Crime
SPEP Review Date(s): May 15-18, 2018
Florida Department of Juvenile Justice
Report Date(s): 5/16/18
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Crestview Youth Academy MQI Program Code: XXXX
Provider Name: Youth Opportunities Investment Contract Number: XXXX
Location: Okaloosa County / Circuit: 1 Number of Beds: 60
Review Date(s): 5/15/18-5/18/18 Lead Reviewer Code: 144
Persons Interviewed
Program Director Corporate QI/QA staff # Program Supervisors
DJJ Monitor 1 # Case Managers # Youth
DHA or designee 1 # Clinical Staff # Other (listed by title):
DMHCA or designee # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks # Personnel Records
Fidelity Monitoring Documents Program Schedules 5 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts # Youth Records (Closed)
Staff Evaluations Table of Organization 5 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports # Health Records
Contract Scope of Services # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime. The program was awarded 15 points
because the Primary Service is identified as a Group 3 Service. The specific Sub-Component
Service Type identified is Social Skills Training. The Primary Service was identified as this type of
service as it focuses on developing social skills required for an individual to interact in a positive
way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program's documentation reflected that five staff have been formally trained to facilitate the
Impact of Crime (IOC) curriculum. The staff included four case managers and the transition case
manager. One of the case managers was promoted to assistant facility administrator and is no
longer facilitating IOC. Two of the case managers have facilitated IOC groups. A third case
manager begins cofacilitating a group the month of this review.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
All of the trained staff maintain the most recent Impact of Crime manual. There are four sections
to the IOC manual. The manual includes all needed worksheets and facilitator forms to conduct
IOC. The facilitators also had the victim videos to go along with the curriculum.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
At the time of the review, the facilitators recently completed external fidelity monitoring with the
Technical Assistance Team. The program started IOC groups in December 2017 and the group is
held on Mondays and Wednesdays, from 2:15pm to 3:15pm. The facilitator creates a lesson plan
for each group and maintains these records. The facilitator provided a copy of the attendance
sheet with the new group of youth beginning group today, a lesson plan, and a crossword puzzle
fo the youth to complete with the words to know in chapter one, along with a coloring sheet. The
facilitator explained that the first session is for the youth to get to know one another.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Groups began in December 2017. The group began with nine youth and three of these youth
completed IOC successfully.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The facilitators for Impact of Crime have experienced turnover with one case manager being
promoted to assistance facility administrator. This did not impact any group. There have been no
gaps in service during the review period.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has a process for internal fideilty monitoring. At this time the two facilitators for
Impact of Crime (IOC) monitor each other once a month. Internal fidelity forms were completed
from January 2018 through April 2018. The only month missing was December 2017.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program's process for any corrective actions or issues that arise during group facilitation is
addressed during weekly clinical supervisions. Review of the internal fidelity forms did not show
corrective action noted for the faciltiators of Impact of Crime.
g. Evaluation of Facilitator Skill Delivering the Basic Score: N/A Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program has not conducted yearly performance evaluations. The program will be doing their
first yearly performance evaluations in or about July 2018.
The program did provide a blank example of a yearly performance evaluation and it did not have a
section to address group facilitation. It was suggested that when performance evaluations are
conducted that the case managers and therapists have a section on their evaluations that
addresses group facilitation.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 16 weeks for this type of service. Of the 4 youth in the
sample, 0% (0 of 4) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 4 youth in the
sample, 100% (4 of 4) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 0 youth
Moderate-High = 2 youth
High = 2 youth
Total Youth in Sample = 4 youth
Percentage of Youth with High-Risk Level to Reoffend: 50%
High Score: 13 Points
Program Optimization Score: 13 Points
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Program Optimization Percentage: 100%
Table 2
High = 2 youth
Total Youth in Sample = 4 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (4 of 4) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 50% (2 of 4) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 25 25 100%
Totals 75 85 88%
This SPEP report evaluates Impact of Crime, an intervention delivered at Crestview Youth
Academy Secure.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. There were no end dates entered
into the Juvenile Justice Information System (JJIS) Evidence-Based Services Module (EBS)
Module for any youth in the sample. The program must enter end dates into JJIS in order to
receive any score in this area.
The program earned 10 points for Amount of Service: Contact Hours. Of the 4 total youth
sampled, all 4 included dosage in the EBS Module. Of those youth, 4 received at least the
recommended hours of service. Youth in the sample completed 32 hours of service.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
itself has no control over youths' C-PACT risk level because the scored assessment was
administered just prior to the youths' admission.
RECOMMENDATION(S):
Crestview Youth Academy Secure can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Crestview Youth Academy Secure can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

program_name_raw: Crestview Youth Academy Secure


state: FL

facility_name: Crestview Youth Academy

summary: SPEP - FY17-18

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Crestview Youth Academy Secure
Youth Opportunities Investments
(Contract Provider)
4445 Straightline Road
Crestview, Florida 32539
Primary Service: Young Men's Work
SPEP Review Date(s): May 15-18, 2018
Florida Department of Juvenile Justice
Report Date(s): 5/16/18
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Crestview Youth Academy MQI Program Code: XXXX
Provider Name: Youth Opportunities Investment Contract Number: XXXX
Location: Okaloosa County / Circuit: 1 Number of Beds: 60
Review Date(s): 5/15/18-5/18/18 Lead Reviewer Code: 144
Persons Interviewed
Program Director Corporate QI/QA staff # Program Supervisors
DJJ Monitor # Case Managers # Youth
DHA or designee # Clinical Staff # Other (listed by title):
DMHCA or designee # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks # Personnel Records
Fidelity Monitoring Documents Program Schedules 5 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts # Youth Records (Closed)
Staff Evaluations Table of Organization # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports # Health Records
Contract Scope of Services # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Young Men's Work. The program was awarded 25 points
because the Primary Service is identified as a Group 4 Service. The specific Sub-Component
Service Type identified is Group Counseling. The Primary Service was identified as this type of
service as it focuses on psychological or interpersonal problems or issues faced by an individual
and involves a group of youths interacting with each other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program requires the therapy staff to be trained and facilitate Young Men's Work (YMW).
There are currently five therapists and the clinical director who are trained to facilitate YMW. The
clinical director can also train facilitators. Upon further review, the curriculum does not require
formal training in order to facilitate or train facilitators. The program maintains training records and
provided copies to this reviewer.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the manual for Young Men's Work: Building Skills to Stop Violence: A Ten
Session Group Program. The authors for this curriculum are Allan Creighton and Paul Kivel. The
manual includes the following for each lesson: objectives, agenda, session outline, and script.
Each lesson also has a worksheet included in the manual. Young Men's Work group is held on
Fridays at 3:00pm.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Young Men's Work (YMW) is held as an open group on Fridays at 3:00pm. There are two groups
held at the same time at the program. The program maintains a binder with all of the attendance
sheets for the groups and provided the binder for review. The average number of youth in a group
was 10. The attendance sheets were completed on a consistent basis. The facilitators need to
ensure they are signing the attendance sheets on a consistent basis.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
Clinical staff are trained to facilitate Young Men's Work (YMW). During the review period one
therapist has left, but it was not during a cohort and did not cause a gap in service. The program
maintains five therapist positions and a clinical director, all of whom are trained in the YMW
curriculum. If the program experiences turnover there are other therapists who are able to step in
and facilitate groups. The program also has the therapists rotate facilitation on a regular basis so
it is not the same facilitators each week. There were no gaps in service and the group has met on
a consistent basis.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program provided documentation of all internal fidelity monitoring forms. The forms are kept
in a binder in the clinical director's office. The clinical director conducts all internal fidelity
monitoring. All internal fidelity forms were accounted for and located in the fidelity monitoring
folder.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program's process for corrective actions and facilitation issues is for the clinical director to
address these in clinical supervisions, which are conducted once a week.
g. Evaluation of Facilitator Skill Delivering the Basic Score: N/A Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program has not conducted yearly performance evaluations. The program will be doing their
first yearly performance evaluations in or about July 2018.
The program did provide a blank example of a yearly performance evaluation and it did not have a
section to address group facilitation. It was suggested that when performance evaluations are
conducted that the case managers and therapists have a section on their evaluations that
addresses group facilitation.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 24 weeks for this type of service. Of the 4 youth in the
sample, 0% (0 of 4) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 40 contact hours for this type of service. Of the 4 youth in the
sample, 0% (0 of 4) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 0 youth
Moderate-High = 2 youth
High = 2 youth
Total Youth in Sample = 4 youth
Percentage of Youth with High-Risk Level to Reoffend: 50%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Table 2
High = 2 youth
Total Youth in Sample = 4 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (4 of 4) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 50% (2 of 4) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 25 25 100%
Totals 75 95 79%
This SPEP report evaluates Young Men's Work, an intervention delivered at Crestview Youth
Academy Secure.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. There were no end dates entered
into the Juvenile Justice Information System (JJIS) Evidence-Based Services Module (EBS)
Module for any youth in the sample. The program must enter end dates into JJIS in order to
receive any score in this area.
The program earned 0 points for Amount of Service: Contact Hours. Of the 4 total youth sampled,
4 included dosage in the EBS Module. Of those youth, 0 received at least the recommended
hours of service. Youth in the sample completed between 4 and 7 hours of service, with an
average of 5 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered just prior to the youths' admission.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
RECOMMENDATION(S):
Crestview Youth Academy Secure can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Crestview Youth Academy Secure can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

source: DJJ

report_type: SPEP

program_status_note: Open

service_type: Young Men's Work


summary: SPEP - FY17-18

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Crestview Sex Offender Program
Gulf Coast Youth Services
(Contract Provider)
4445 Straightline Road
Crestview , Florida 32455
Primary Service: ARISE
SPEP Review Date(s): October 3 ‐ 6, 2017
Florida Department of Juvenile Justice
Report Date(s): 12/20/2017
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Okaloosa Youth Academy/Crestview Sex Offender Program
MQI Program Code: 830/1161
Provider Name: Gulf Coast Youth Services Contract Number: 10288
Location: Okaloosa County / Circuit: 1 Number of Beds: 60/15
Review Date(s): October 3 - 6, 2017 Lead Reviewer Code: 145/144
Persons Interviewed
Program Director Corporate QI/QA staff # Program Supervisors
DJJ Monitor # Case Managers # Youth
DHA or designee # Clinical Staff # Other (listed by title):
DMHCA or designee # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 7 # Personnel Records
Fidelity Monitoring Documents Program Schedules 7 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts # Youth Records (Closed)
Staff Evaluations Table of Organization # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports # Health Records
Contract Scope of Services # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is ARISE. The program was awarded 25 points because
the Primary Service is identified as a Group 4 Service. The specific Sub-Component Service
Type identified is Group Counseling. The Primary Service was identified as this type of service as
it focuses on psychological or interpersonal problems or issues faced by an individual and
involves a group of youths interacting with each other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 6 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
Training certificates were provided to all therapy staff who completed the 14-hour ARISE training
curriculum which is conducted by the ARISE founders. Five of the seven therapists trained in
ARISE conducted this curriculum at least once over the review period of 12 months.
Basic Score: 1 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The ARISE curriculum is split into various workbooks that are completed by the youth during
group. Each workbook covers various topics and the youth are able to work at their own pace
during group. The group has a range of topics they discuss during their sessions.
There is an instructor’s manual with each set of workbooks. The instructor's manual does not
have a script for the facilitator to use. The facilitator is supposed to work with the youth where they
are in their stages of change based on the workbook topic. Within the ARISE curriculum, the
youth may work in smaller groups or as one large group, if the facilitator desires. The facilitator
should measure how well the youth interact and participate during the group to determine their
performance in this curriculum. The instructor’s manual is separated by lessons and worksheets
based on the youth’s workbook. There is also a resource manual with extra worksheets and
activities for the youth to complete.
The program can earn 2 points if the manual specifically details the delivery of this primary
service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The ARISE groups are conducted on Saturdays and Sundays and were not monitored during this
review.
This curriculum was not able to be observed; therefore, this indicator rates as non-applicable.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The attendance sheets for this curriculum were unorganized. A random sampling was taken of the
attendance sheets. The attendance sheets included the title of session, youth names and
signatures, date of group, and a place for the therapist to sign. Many of the attendance sheets
from the sample did not have a therapists name and/or signature. It was difficult for this reviewer
to determine which therapist facilitated group and on what date.
The facility consistently employed six therapists over the past 12 months. Per discussion with the
Designated Mental Health Clinical Authority, as a new therapist is hired, they are trained in the
ARISE curriculum. Since this group is held on the weekends, the facilitators of this group change
every weekend. If a therapist leaves, there are others that can take that weekend group so there
is no disruption to the curriculum.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The internal fidelity monitoring sheets were provided by the Designated Mental Health Clinical
Authority for the review period. One of the six facilitators for this group had all 12 months of
internal fidelity monitoring. The internal fidelity monitoring was sporadic for the other five
facilitators trained in the ARISE curriculum.
It was not possible to determine if fidelity monitoring was completed every month for every
facilitator due to the missing information on sign-in sheets.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
There were no corrective actions indicated on the internal fidelity monitoring forms that were
reviewed.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Five out of the six therapists trained had a completed yearly performance evaluation. The sixth
therapist left employment prior to the first evaluation being completed.
On the yearly performance evaluations for therapists, there is a designated section for group
facilitation. Under this section, multiple service types are listed and a check mark is placed next to
anything the therapist facilitates.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 6%
Research indicates the target duration of 24 weeks for this type of service. Of the 30 youth in the
sample, 63% (19 of 30) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 40 contact hours for this type of service. Of the 30 youth in the
sample, 100% (30 of 30) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 90%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 5 youth
Moderate-High = 10 youth
High = 12 youth
Total Youth in Sample = 30 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 40%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 12 youth
Total Youth in Sample = 30 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 90% (27 of 30) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 40% (12 of 30) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 23 25 92%
Totals 79 95 83%
This SPEP report evaluates ARISE, an intervention delivered at Okaloosa Youth
Academy/Crestview Sex Offender Program.
The program scored Medium for Quality of Service Delivery.
The program earned 6 points for Amount of Service: Duration. 63% received at least the
recommend weeks of service, with sample youth completing an average of 31 weeks of service.
The program earned 10 points for Amount of Service: Contact Hours. 100% received at least the
recommended hours of service, with sample youth completing an average of 54 hours of service.
Dosage was calculated for both Okaloosa Youth Academy and for Crestview Sex Offender
Program together as the programs were reviewed together.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
itself has no control over youths' C-PACT risk level because the scored assessment was
administered just prior to the youths' admission.
RECOMMENDATION(S):
Okaloosa Youth Academy/Crestview Sex Offender Program can optimize their SPEP Quality of
Service Delivery score by minimizing facilitator turnover, ensuring internal fidelity monitoring is
conducted at least monthly for all facilitators, and by utililizing detailed manuals and plans.
Okaloosa Youth Academy/Crestview Sex Offender Program can optimize their SPEP Amount of
Service score by ensuring that youth receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

program_name_raw: Crestview Sex Offender Program

program_status_note: Closed

service_type: ARISE


summary: SPEP - FY17-18

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Crestview Sex Offender Program
Gulf Coast Youth Services
(Contract Provider)
4445 Straightline Road
Crestview, Florida 32455
Primary Service: Impact of Crime (IOC)
SPEP Review Date(s): October 3 ‐ 6, 2017
Florida Department of Juvenile Justice
Report Date(s): 12/20/2017
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Okaloosa Youth Academy/Crestview Sex Offender Program
MQI Program Code: 830/1161
Provider Name: Gulf Coast Youth Academy Contract Number: 10288
Location: Okaloosa County / Circuit: 1 Number of Beds: 60/15
Review Date(s): October 3 - 6, 2017 Lead Reviewer Code: 145/144
Persons Interviewed
Program Director Corporate QI/QA staff 1 # Program Supervisors
DJJ Monitor 3 # Case Managers 3 # Youth
DHA or designee # Clinical Staff # Other (listed by title):
DMHCA or designee # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 5 # Personnel Records
Fidelity Monitoring Documents Program Schedules # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts # Youth Records (Closed)
Staff Evaluations Table of Organization # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports # Health Records
Contract Scope of Services # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime (IOC). The program was awarded 15
points because the Primary Service is identified as a Group 3 Service. The specific Sub-
Component Service Type identified is Social Skills Training. The Primary Service was identified
as this type of service as it focuses on developing social skills required for an individual to interact
in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program employs five case managers formally trained in the Impact of Crime (IOC)
curriculum. Two of the case managers received formal training in August 2012. Three case
managers received formal training in March 2017. Two of the facilitators have been consistently
facilitating this group. The three case managers who were trained in March of 2017 recently
completed their external fidelity monitoring by the Technical Assistance Specialist.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The manual provided was written in 2003. This manual did not have the new exercise (exercise
nine) in Chapter five for the youth to complete, however, the updated IOC exercises were
provided to the case managemers so that all case managers have up-to-date training material.
The manual contains a script for the facilitator to cover the three sections included in the manual.
The first section is the facilitator manual, the second is the appendix which includes helpful and
detailed information about IOC and extra sets of forms for the training. The third section is the
student handbook, which is divided into seven chapters and contained all of the student
worksheets.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
An IOC group was observed on October 5, 2017. This session happened to be the first meeting
for this particular group. Prior to session beginning, this reviewer had the opportunity to walk with
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
the youth to the dining hall and sit with them while they ate. This group was facilitated by one case
manager, however, a second case manager was sitting in to assist with the group as needed.
The group had eight youth registered for this cohort but on this particular day, only six youth
attended the group. The facilitator welcomed the youth and began with explaining what Impact of
Crime was and explained Restorative Justice. The facilitator then completed an icebreaker.
During this group there was one behavioral issue. The facilitator and case manager spoke to the
youth and maintained a calm demeanor throughout the incident. They both followed the program's
policy and procedure and the situation was handled without further incident.
Overall, the facilitator handled the class well and explained the first objective well.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There has been no facilitator turnover in the past 12 months. The program increased the amount
of Impact of Crime facilitators from two to five within the past 12 months.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Two of the five facilitators for Impact of Crime conducted internal fidelity monitoring for all of the
groups. Once the other three have completed their external fidelity monitoring they will be able to
conduct internal fidelity monitoring.
There were three cohorts of Impact of Crime completed during the review period of 12 months.
The first cohort was held April 19, 2017 through July 31, 2017, the second cohort was held July 6,
2017 through September 9, 2017, and the third cohort started April 17, 2017 and is still being
held. All internal fidelity monitoring was completed.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The case management supervisor provided all internal fidelity documents for Impact of Crime
groups. There were no corrective actions indicated on the internal fidelity monitoring forms. There
were various suggestions made to some of the facilitators.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Five yearly performance evaluations were provided for review. There was no specific section for
group facilitation or discussion of IOC.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 16 weeks for this type of service. Of the 6 youth in the
sample, 0% (0 of 6) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 24 contact hours for this type of service. Of the 6 youth in the
sample, 0% (0 of 6) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 1 youth
Moderate-High = 1 youth
High = 4 youth
Total Youth in Sample = 6 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 67%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 4 youth
Total Youth in Sample = 6 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (6 of 6) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 67% (4 of 6) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 25 25 100%
Totals 65 85 76%
This SPEP report evaluates Impact of Crime (IOC), an intervention delivered at Okaloosa Youth
Academy/Crestview Sex Offender Program.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. 0% received at least the
recommend weeks of service, with sample youth completing an average of 13 weeks of service.
four of the six youth sampled did not include a service end date, so it was impossible to calculate
the accurate weeks of service. The program earned 0 points for Amount of Service: Contact
Hours. 0% received at least the recommended hours of service, with sample youth completing an
average of 8 hours of service. Dosage was calculated for both Okaloosa Youth Academy and for
Crestview Sex Offender Program together as the programs were reviewed together; however,
there was no dosage recorded for this intervention for any youth from Crestview Sex Offender
Program.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered just prior to the youths' admission.
RECOMMENDATION(S):
Okaloosa Youth Academy/Crestview Sex Offender Program can optimize their SPEP Quality of
Service Delivery score by ensuring facilitators' performance evaluations include an assessment of
their delivery of IOC.
Okaloosa Youth Academy/Crestview Sex Offender Program can optimize their SPEP Amount of
Service score by ensuring that data is entered correctly in the EBS Module of JJIS and by
ensuring youth receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

service_type: Impact of Crime


summary: SPEP - FY18-19

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Crestview Sex Offender Program
Gulf Coast Youth Services
(Contract Provider)
4445 Straightline Road
Crestview, Florida 32539
Primary Service: ARISE Lifeskills
SPEP Review Date(s): 4/30/19‐5/3/19
Florida Department of Juvenile Justice
Report Date(s): 5/7/19
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
 Service Type: service type points are awarded based on the group category that the
service falls within
 Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
 Amount of Service: or “dosage” refers to the amount and duration of services that were
received
 Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is ARISE Lifeskills.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program provided training documentation for seven therapists who facilitate the ARISE
curriculum. The designated mental health clinical authority conducts informal training with
therapists as they are hired. Certificates and training records were provided for review. Seven
therapists have facilitated ARISE during the review period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
ARISE is a life-skills curriculum conducted as a mental health group, pursuant to the program's
contract. ARISE is held on Fridays from 2:15pm to 3:15pm and 3:15pm to 4:15pm. All youth in the
program are eligible to attend ARISE group. All attendance sheets for the ARISE group are
bundled together with all other mental health and substance abuse groups and maintained in the
designated mental health clinical authority's (DMHCA) office. ARISE is held as an open group,
meaning youth can join in the group at any time.
The ARISE facilitator training manual was provided for review. There are multiple workbooks for
the youth to complete working through various types of life skills. The instructor's manual provides
a guide for the lessons, lesson plan forms, and all necessary worksheets. There is not a specific
script for the facilitators to follow.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Due to a change in the schedule, the reviewer was able to observe an ARISE group during the
review week. The group was observed on Wednesday, May 1, 2019 from 2:15 to 3:15pm. There
were nine youth in attendance for the group. There has been a low census in the program, so the
3:15pm group has been combined with the earlier group. The facilitator provided the worksheets
and handouts for the youth to follow along and participate in the discussion. The facilitator allowed
for open discussion and kept the group on task. All youth participated and appeared to feel
comfortable with the facilitator and staff member in the room. The facilitator followed along with
the worksheets and appeared to feel comfortable with the material.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program provided documentation showing there has not been significant turnover in therapy
staff. All therapy staff are trained to conduct ARISE curriculum. Currently, seven therapists
facilitate all ARISE groups. A review of attendance sheets showed there have not been gaps in
service. If a therapist leaves, another therapist steps in to complete facilitation of group.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of internal fidelity logs showed the therapists observe one another to conduct internal
fidelity monitoring. There were no missing internal fidelity monitoring forms for the seven
therapists who conducted ARISE group during the review period.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The designated mental health clinical authority conducts weekly supervisions with therapy staff.
Any issues or corrective actions documented on internal fidelity monitoring forms are discussed
during those meetings or during one-to-one meetings. A review of all internal fidelity monitoring
forms indicated no corrective actions were observed during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program provided six of the seven therapists' yearly performance evaluations for review. One
therapist was hired in April 2019 and had not received an evaluation at the time of the review. The
yearly performance evaluations include a section for group facilitation with all curricula listed.
Each therapist has a checkmark next to all curricula they are trained in to facilitate.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 24 weeks for this type of service. Of the 19 youth in the
sample, 11% (2 of 19) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates a target of 40 contact hours for this type of service. Of the 19 youth in the
sample, 47% (9 of 19) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 89%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 7 youth
Moderate-High = 4 youth
High = 6 youth
Total Youth in Sample = 19 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 32%
High Score: 10 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 77%
Table 2
High = 6 youth
Total Youth in Sample = 19 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 89% (17 of 19) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 32% (6 of 19) youth scored High-Risk to reoffend, for a score of 10 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 4 10 40%
Risk Level of Youth Served 20 25 80%
Totals 74 95 78%
This SPEP report evaluates ARISE Lifeskills, an intervention delivered at Crestview Sex Offender
Program.
19 youth completed the program during the review period. Of those youth, 19 were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 19 total youth sampled, only
9 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 2
received at least the recommended weeks of service. Youth in the sample completed between 6
and 24 weeks of service, with an average of 19 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
The program earned 4 points for Amount of Service: Contact Hours. Of the 19 total youth
sampled, only 17 included dosage in the EBS Module. Of those youth with correct dosage, 9
received at least the recommended hours of service. Youth in the sample completed between 3
and 52 hours of service, with an average of 34 hours.
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
Based on the risk level of the youth placed at the program (80% Program Optimization
Percentage), the Department should work to ensure a larger proportion of higher risk youth are
recommended. As the program serves youth with a history of sexual behavior problems, this
includes a multitier strategy of working with stakeholders to achieve that practice, as well as
ensuring appropriate treatment services are available in the community to reduce the reliance on
residential commitment for lower risk youth presenting with sexually-related offenses.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 83%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Crestview Sex Offender Program can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Crestview Sex Offender Programcan optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8


summary: SPEP - FY18-19

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Crestview Sex Offender Program
Gulf Coast Youth Services
(Contract Provider)
4445 Straightline Road
Crestview, Florida 32539
Primary Service: Impact of Crime
SPEP Review Date(s): 4/30/19‐5/3/19
Florida Department of Juvenile Justice
Report Date(s): 5/8/19
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
 Service Type: service type points are awarded based on the group category that the
service falls within
 Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
 Amount of Service: or “dosage” refers to the amount and duration of services that were
received
 Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime. The program was awarded 15 points
because the Primary Service is identified as a Group 3 Service. The specific Sub-Component
Service Type identified is Social Skills Training. The Primary Service was identified as this type of
service as it focuses on developing social skills required for an individual to interact in a positive
way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 6 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
This review encompasses twelve months of records examinations for the Impact of Crime (IOC)
curriculum. The program provided twelve months of documentation to review. The documentation
reflected there are four staff formally trained to faciltiate Impact of Crime (IOC) groups: two case
managers, the training coordinator, and the case management supervisor. All training
documentation was reviewed for all facilitators. The training coordinator is also trained to conduct
facilitator training. One case manager conducts IOC training for the program.
Basic Score: 1 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program maintains the current Impact of Crime (IOC) manual and youth workbook for the
program. The youth are provided the workbook chapters one at a time. The facilitator maintains
the workbooks in her office with the training material. The manual includes a script and lesson
plan for the facilitator to follow. Attendance sheets are maintained in the case management
supervisor's office.
IOC groups were being held on Wednesdays and Thursdays from 11:00am to 12:00pm. This
reviewer discussed with the case management supervisor there should be a day between groups
for homework and the group is not designed to be held two days in a row. The group is held as a
closed group, meaning once the group has started no youth are able to join.
A review of the program schedule indicated all deliquency interventions and gender specific
groups are being held during the lunch period from 11:30am to 12:30pm. This also includes med
pass. The youth retrieve their lunch from the cafeteria and go into a classroom to eat lunch and
attend group. This cuts into group time, along with med pass interrupting group for the last ten
minutes. The scheduling of groups was discussed with the case management supervisor and it
was reported the only time available for delinquency interventions are during the youth's lunch
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
time. All mental health groups are held in the afternoons after school between 2:15pm and
4:15pm. Additionally, this reviewer discussed a follow up meeting to talk about tracking youth to
properly receive IOC group, as all youth are eligible to attend IOC.
The program can earn 2 points if the manual specifically details the delivery of this primary
service, and the program adheres to the protocol.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
An Impact of Crime (IOC) group was observed on Wednesday, May 1, 2019 from 11:00am to
12:00pm. The facilitator gathered the youth at 10:45am and took them to get their lunches. Group
was held in the Crestview Sex Offender Program (CSOP) dorm. There were five youth in
attendance. The facilitator reported two other youth were released from the program prior to
completing IOC. The group started chapter five. The facilitator had chosen activities the youth
enjoyed participating in throughout the group time. The facilitator did a good job of going through
the start of the lesson without relying solely on the manual and appeared comfortable with the
lesson. The facilitator provided affirmations and feedback throughout the lesson. The youth were
open to receiving the feedback. The facilitator and youth had a healthy rapport built as well.
Basic Score: 0 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There has been some turnover in case management during the review period. This turnover did
not affect the Impact of Crime (IOC) group.The case management supervisor reported the case
management staff are responsible for conducting deliquency interventions for the youth. The goal
is to have all case management trained in the IOC curriculum. During the turnover there were still
four staff members trained to facilitate IOC.
A review of documentation showed one full cohort of IOC during the review period. A group was
started on January 2, 2019 with seven youth. Documentation received from the case
management supervisor showed a prior group beginning in early 2018 and ending in May of 2018.
There were no groups held between May 2018 and January 2019 (current review period is May
2018 to May 2019). It was reported there is an unknown amount of time between groups. During
this gap there were other staff available to facilitate IOC groups. The case management
supervisor did indicate Thinking for a Change groups were being conducted.
The program can earn 2 points if there are no unnecessary gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of the internal fidelity monitoring forms showed the case management staff observe one
another for fidelity monitoring. The current group of Impact of Crime starting January 2019 was
not missing internal fidelity monitoring for the facilitator.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process in place to address corrective actions based on fridelity monitoring.
The case management supervisor addresses any issues or corrective actions with facilitators
individually as needed.
A review of the four months of internal fidelity monitoring forms showed no corrective actions or
issues noted.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The facilitator for Impact of Crime did receive a yearly performance evaluation. There is a section
on the yearly performance evaluations for group facilitation with a check mark for all curricula the
case manager facilitates. The case management supervisor conducts the annual performance
evaluations.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 16 weeks for this type of service. Of the 19 youth in the
sample, 5% (1 of 19) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 24 contact hours for this type of service. Of the 19 youth in the
sample, 0% (0 of 19) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 89%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 7 youth
Moderate-High = 4 youth
High = 6 youth
Total Youth in Sample = 19 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 32%
High Score: 10 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 77%
Table 2
High = 6 youth
Total Youth in Sample = 19 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 89% (17 of 19) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 32% (6 of 19) youth scored High-Risk to reoffend, for a score of 10 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 20 25 80%
Totals 50 85 59%
This SPEP report evaluates Impact of Crime, an intervention delivered at Crestview Sex Offender
Program.
19 youth completed the program during the review period. Of those youth, 19 were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 19 total youth sampled, only
3 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 1
received at least the recommended weeks of service. Youth in the sample completed between 6
and 16 weeks of service, with an average of 12 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 0 points for Amount of Service: Contact Hours. Of the 19 total youth
sampled, only 9 included dosage in the EBS Module. Of those youth with correct dosage, 0
received at least the recommended hours of service. Youth in the sample completed between 13
and 23 hours of service, with an average of 19 hours.
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
Based on the risk level of the youth placed at the program (80% Program Optimization
Percentage), the Department should work to ensure a larger proportion of higher risk youth are
recommended. As the program serves youth with a history of sexual behavior problems, this
includes a multitier strategy of working with stakeholders to achieve that practice, as well as
ensuring appropriate treatment services are available in the community to reduce the reliance on
residential commitment for lower risk youth presenting with sexually-related offenses.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 65%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Crestview Sex Offender Program can optimize their SPEP Quality of Service Delivery score by
ensuring the manual specifically details the delivery of this primary service, and the program
adheres to the protocol; by ensuring there are no unnecessary gaps in service; and by ensuring
fidelity monitoring is completed every month for each facilitator.
Crestview Sex Offender Program can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9