summary: SPEP - FY18-19

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Crestview Youth Academy Secure
Youth Opportunities Investment, Inc.
(Contract Provider)
4449 Straighline Road
Crestview, Florida 32539
Primary Service: Male Healthy Relationships
SPEP Review Date(s): 2/12/19-2/15/19
Florida Department of Juvenile Justice
Report Date(s): 2/18/19
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime. The program was awarded 15 points
because the Primary Service is identified as a Group 3 Service. The specific Sub-Component
Service Type identified is Social Skills Training. The Primary Service was identified as this type of
service as it focuses on developing social skills required for an individual to interact in a positive
way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program's documentation reflected the program had a training protocol for all therapy staff to
be informally trained to conduct Male Healthy Relationships curriculum.The clinical director
maintains a training binder with all training sign-in sheets for training. The program had four
therapeutic staff facilitate Male Healthy Relationships during the review period as indicated on the
group attendance sheets. The clinical director provided informal training documentation for a two-
hour training. Pursuant to an interview with the clinical director, the training entails the clinical
director choosing a lesson and demonstrating how the lesson is facilitated.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The Standardized Program Evaluation Protocol (SPEP) review encompasses twelve months of
records examinations, specifically for Male Healthy Relationships. Prior to August 2018 Male
Healthy Relationships was not a primary service for the program.
Male Healthy Relationships is a bundle curriculum of the Teen Relationships Workbook and
Young Men's Work. The group is held for one hour, two days a week, pursuant to the program
group schedule. Currently this group is held on Fridays from 2:30pm to 3:30pm and Sundays
between 9:30am and 12:30pm for one hour. All youth are eligible to attend this group.
Attendance sheets are maintained in a binder kept in the clinical director's office. The program
maintains two manuals which therapists work from; Young Men's Work and the Teen
Relationships Workbook. The Young Men's Work manual is maintained in a binder in the clinical
director's office. It is approximately ten sessions and includes videos and handouts. The manual
is broken into individual sessions that include the objective, agenda, session outline, and all
handouts. The lessons also include a script to follow.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
The manual for the Teen Relationships Workbook is maintained in a binder in the clinical
director's office. It is five main topics broken down into individual lessons. All lessons and
handouts are included. Facilitator guidelines are also included, but no script is provided.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
This reviewer did not observe Male Healthy Relationships group due to it being held outside of the
time the reviewed was on site. This curriculum was not able to be observed; therefore, this
indicator rates as non-applicable.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The reviewed documentation confirmed there has been turnover that did not adversely affect the
youth during the review period. One therapist left during the review period. There were three other
therapists who continued to facilitate group. There were no gaps in service longer than one month
throughout the review period.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The clinical director is delegated to conduct internal fidelity monitoring for Male Healthy
Relationships. All internal fidelity monitoring forms were provided and reviewed. There were no
missing internal fidelity monitoring for the six months of the review period in which this primary
service had been designated. The clinical director did facilitate groups during the review period,
but did not receive internal fidelity monitoring. If the clinical director facilitates the group that
means there is a therapist out and all other therapists are running a group during the same time
frame. Therefore no internal fidelity monitoring forms are completed on the clinical director. This
information was confirmed during an interview with the clinical director.The suggestion was made
to have other staff trained to conduct internal fidelity monitoring when needed.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The clinical director conducts weekly supervisions for therapeutic staff and case management
staff. If any issues or corrective actions are noted it is addressed during weekly supervisions. The
clinical director maintains a binder with documentation of all weekly supervisions. Each meeting
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
contains attendees, signatures, dates, and an agenda. There were no corrective actions noted on
the internal fidelity monitoring forms reviewed.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program provided two yearly performance evaluation for the therapeutic staff who conducted
Male Healthy Relationships group during the review period. The yearly performance evaluations
contain a section for facilitators skill delivering evidence based practices. There is also a section
to list each curricula a staff member facilitates.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates the target duration of 24 weeks for this type of service. Of the 17 youth in the
sample, 47% (8 of 17) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates a target of 40 contact hours for this type of service. Of the 17 youth in the
sample, 47% (8 of 17) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 0 youth
Moderate-High = 6 youth
High = 11 youth
Total Youth in Sample = 17 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 71%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 11 youth
Total Youth in Sample = 17 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (17 of 17) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 71% (11 of 17) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 4 10 40%
Amount of Service: Contact Hours 4 10 40%
Risk Level of Youth Served 25 25 100%
Totals 83 95 87%
This SPEP report evaluates Male Healthy Relationships, an intervention delivered at Crestview
Youth Academy Secure.
24 youth completed the program during the review period. Of those youth, 17 had not received
this intervention in another setting.
The program scored High for Quality of Service Delivery.
The program earned 4 points for Amount of Service: Duration. Of the 17 total youth sampled, only
13 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 8
received at least the recommended weeks of service. Youth in the sample completed between 14
and 39 weeks of service, with an average of 29 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 4 points for Amount of Service: Contact Hours. Of the 17 total youth
sampled, all 17 included dosage in the EBS Module. Of those youth, 8 received at least the
recommended hours of service. Youth in the sample completed between 4 and 52 hours of
service, with an average of 30 hours.
This bundled primary service was designated August 2, 2018, six months into the review period.
Prior to that date, dosage was entered for only the Young Men's Work component. As such, each
youth in the sample documented dosage for only one part of the whole intervention, thus affecting
the overall score. As of the date of the review, dosage was being entered for the bundled service.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Crestview Youth Academy Secure can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Crestview Youth Academy Secure can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9


state: FL

facility_name: Crestview Youth Academy

summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Crestview Sex Offender Program
Gulf Coast Youth Services
(Contract Provider)
4445 Straightline Road
Crestview, Florida 32539
Primary Service: ARISE Lifeskills
SPEP Review Date(s): 05/21/2020
Florida Department of Juvenile Justice
Report Date(s): 05/21/2020
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
 Service Type: service type points are awarded based on the group category that the
service falls within
 Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
 Amount of Service: or “dosage” refers to the amount and duration of services that were
received
 Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is ARISE Lifeskills.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program provied training documentation. All seven therapists have been trained by a certified
ARISE Lifeskills trainer. The training is documented on sign in sheets maintained by the
Designated Mental Health Clinician Authority (DMHCA) and the therapists receive a certificate of
completion.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
ARISE Lifeskills is held as a life skills group for the program and all youth are eligible to attend.
The group is held as an open group to allow the youth to work through the program throughout
their stay at the program. As this is held as an open group, the youth enter the group at any time
and complete it at their own pace.
The ARISE Lifeskills facilitator training manual was provided for review. There are multiple
workbooks for the youth to complete working through various types of life skills. The instructor's
manual provides a guide for the lessons, lesson plan forms, and all necessary worksheets. There
is not a specific script for the facilitators to follow.
ARISE Lifeskills groups are held on Fridays from 2:15pm - 3:15pm and 3:15pm to 4:15pm. They
groups are held on Saturdays and Sundays from 8:30am - 9:30am and 9:30am to 10:30am. The
therapists rotate leading the groups. There are multiple groups held each day to ensure all youth
are able to attend the full curriculum.
The attendance sheets were provided for review. All attendance sheets were available for review
and there were no issues noted. They are kept in the Designated Mental Health Clinician
Authority's (DMCHA) office, rubber-banded together with all other groups.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of ARISE Lifeskills was not observed during the review period. Group observations will
be observed during quarterly reviews throughout the next review period. This indicator is rated as
non-applicable.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program has seven therapy staff trained to conduct ARISE Lifeskills groups. Should there be
any turnover, another trained therapist can continue the group. The program has not experienced
any turnover during the review period and there have been no gaps in service.
BasicScore: 2Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The Designated Mental Health Clinician Authority (DMHCA) or other trained therapists conduct
internal fidelity monitoring for ARISE Lifekills groups. All internal fidelity monitoring forms were
completed for the groups.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
Any suggestions made or corrective actions are reviewed with the observer of the group and the
Designated Mental Health Clinician Authority (DMHCA). Any suggestions made were documented
on the internal fidelity monitoring forms.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Four therapists had recievied their annual performance evauationsat the time of the annual
review. These evaluations included a section for what curriculum the therapist is trained in and an
evaulation of how they are facilitating the groups. Three therapists have not been employed at the
program long enough to have had an annual performance evaluation.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service ‐ Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 24 weeks for this type of service. Of the 17 youth in the
sample, 100% (17 of 17) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 40 contact hours for this type of service. Of the 17 youth in the
sample, 100% (17 of 17) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 76%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 5 youth
Moderate-High = 1 youth
High = 7 youth
Total Youth in Sample = 17 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 41%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 7 youth
Total Youth in Sample = 17 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 76% (13 of 17) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 41% (7 of 17) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 20 25 80%
Totals 90 95 95%
This SPEP report evaluates ARISE Lifeskills, an intervention delivered at Crestview Sex Offender
Program.
17 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 17 total youth sampled, all
17 included dosage with end dates in the EBS Module. Of those youth, all 17 received at least the
recommended weeks of service. Youth in the sample completed between 25 and 66 weeks of
service, with an average of 38 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
The program earned 10 points for Amount of Service: Contact Hours. Of the 17 total youth
sampled, all 17 included dosage in the EBS Module. Of those youth, all 17 received at least the
recommended hours of service. Youth in the sample completed 52 hours of service.
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
Based on the risk level of the youth placed at the program (80% Program Optimization
Percentage), the Department should work to ensure a larger proportion of higher risk youth are
recommended. As the program serves youth with a history of sexual behavior problems, this
includes a multitier strategy of working with stakeholders to achieve that practice, as well as
ensuring appropriate treatment services are available in the community to reduce the reliance on
residential commitment for lower risk youth presenting with sexually-related offenses.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 100%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Crestview Sex Offender Program can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Crestview Sex Offender Program can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8

source: DJJ

report_type: SPEP

program_name_raw: Crestview Sex Offender Program

program_status_note: Closed

service_type: ARISE


summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Crestview Sex Offender Program
Gulf Coast Youth Services
(Contract Provider)
4445 Straightline Road
Crestview, Florida 32539
Primary Service: Impact of Crime
SPEP Review Date(s): 05/21/2020
Florida Department of Juvenile Justice
Report Date(s): 05/21/2020
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
Case management staff are formally trained in the Impact of Crime (IOC) curriculum. There are
four case managers and the case management supervisor trained to facilitate IOC. There are no
specific case managers assigned to facilitate IOC for the youth in the sex offender program;
according to the case management supervisor, the case managers rotate based upon where the
need is for the group in the program. All training records were available for review.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
All records were provided for review for Impact of Crime (IOC) groups. IOC is held as closed
groups, meaning once a group has started youth cannot enter a group and must wait for the next
group to begin. The groups are held Tuesdays and Thursdays from 4:00pm to 5:00pm. The
program maintains the current IOC manual. Over the past year, the case management supervisor
has done a good job working to ensure all youth in the program receive IOC groups.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Impact of Crime was not observed at this time. Group observations will be scheduled
during quarterly reviews within the next review period.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There has not been any turnover in the case management staff during this review period.
Five cohorts of Impact of Crime were completed during the review period; two additional cohorts
were meeting at the time of the annual review. The first cohort started June 24, 2019 and was
held on Mondays and Wednesdays from 2:15pm to 3:15pm. The group started and completed
with nine youth.
The second cohort started May 7, 2019 and ended August 20, 2019. The group was held on
Tuesdays and Thursdays from 2:15pm-3:15pm.The group started and ended with eight youth.
According to the case management supervisor, three youth received an extra session outside of
the group due to needing to complete the group prior to being released from the program. The
reviewer discussed with the case management supervisor this is an extenuating circumstance
with trying to get the youth through IOC prior to release and will not be a regular practice.
Individual attendance sheets were completed for the youth completing individual sessions.
The third cohort began July 30, 2019 with two new facilitators. The group was held Tuesdays and
Thursdays from 4:00pm to 5:00pm with fifteen youth in the group. This is a large number of youth
for this curriculum. It was suggested a seasoned faciltiator for IOC sit in the group with the two
new facilitators to assist when needed. The group ended November 12, 2019 with nine youth.
There was a short gap during this group between September 12th to October 1. During that time
the group lost six youth. This was documented on the attendance forms the youth finished the
program, however they did not complete IOC.
The fourth cohort began November 19, 2019 and ended March 2, 2020. The group started with
eight youth and ended with seven youth. Once youth was released early and did not complete the
curriculum. The fifth cohort began November 19, 2010 and ended March 12, 2020. The group
started and ended with eight youth.
The sixth cohort was started on March 24, 2020 and continues to meet at the time of this review.
The group began with eight youth and currently has seven youth. There was no explanation as to
why the youth did not complete the group. A seventh cohort was started on March 24, 2020 and
continues to meet at the time of this review. Six youth began the group and there are currently still
sixth youth in the group.
The groups are being held with fidelity according to a review of the attendance sheets. The
groups are being held consistently for an hour. Some of the groups were held for an hour and
thirty minutes. The youth are signing in on a consistent basis. The groups are meeting dosage
requirements.
The program can earn 2 points if there are no unnecessary gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
All internal fidelity monitorng forms for Impact of Crime were provided for review. All internal
fidelity monitoring forms were accounted for all facilitators for Impact of Crime in the program. The
case management supervisor or other trained case managers conduct internal fidelity monitoring
for Impact of Crime groups. Feedback and suggestions were made on a consistent basis on the
internal fidelity monitoring forms.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
Upon review of the internal fidelity monitoring forms, there were no corrective actions noted. The
case management supervisor did make suggestions when needed.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The facilitators for Impact of Crime received an annual performance evaluation, which were
provided for review. The annual performance evaluations did not have a section to evaluation
group facilitation.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates the target duration of 16 weeks for this type of service. Of the 17 youth in the
sample, 59% (10 of 17) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 24 contact hours for this type of service. Of the 17 youth in the
sample, 88% (15 of 17) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 76%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 5 youth
Moderate-High = 1 youth
High = 7 youth
Total Youth in Sample = 17 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 41%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 7 youth
Total Youth in Sample = 17 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 76% (13 of 17) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 41% (7 of 17) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 4 10 40%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 20 25 80%
Totals 62 85 73%
This SPEP report evaluates Impact of Crime, an intervention delivered at Crestview Sex Offender
Program.
17 youth completed the program during the review period.
The program scored Medium for Quality of Service Delivery.
The program earned 4 points for Amount of Service: Duration. Of the 17 total youth sampled, only
12 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 10
received at least the recommended weeks of service. Youth in the sample completed between 12
and 24 weeks of service, with an average of 18 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 17 total youth
sampled, only 16 included dosage in the EBS Module. Of those youth with correct dosage, 15
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
received at least the recommended hours of service. Youth in the sample completed between 23
and 52 hours of service, with an average of 30 hours.
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
Based on the risk level of the youth placed at the program (80% Program Optimization
Percentage), the Department should work to ensure a larger proportion of higher risk youth are
recommended. As the program serves youth with a history of sexual behavior problems, this
includes a multitier strategy of working with stakeholders to achieve that practice, as well as
ensuring appropriate treatment services are available in the community to reduce the reliance on
residential commitment for lower risk youth presenting with sexually-related offenses.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 79%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Crestview Sex Offender Program can optimize their SPEP Quality of Service Delivery score by
ensuring there are no unnecessary gaps in service, and by ensuring each facilitator’s
performance evaluation includes discussion of their skill in delivering this primary service.
Crestview Sex Offender Program can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

service_type: Impact of Crime


summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Crestview Sex Offender Program
Gulf Coast Youth Services
(Contract Provider)
4445 Straightline Road
Crestview, Florida 32539
Primary Service: Pathways / Roadmaps to Recovery
SPEP Review Date(s): 05/21/2020
Florida Department of Juvenile Justice
Report Date(s): 5/21/2020
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Pathways / Roadmaps to Recovery.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There are seven therapists trained in the Pathways / Roadmaps to Recovery curriculum during
the review period. The therapists are trained by the designated mental health clinician authority
(DMHCA) in an informal training. The training is documented on sign in sheets maintained by the
DMHCA.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program provided documentation for the twelve month review period. Pathways / Roadmaps
to Recovery is a curriculum designed for sexual offenders. All youth admitted into the program
attend the Pathways group prior to being released. Pathways is provided four days a week,
Monday through Thursday from 2:15pm to 3:15pm and 3:15pm to 4:15pm, based upon current
census.
Attendance sheets are bound by month with all other substance abuse and mental health groups
and maintained in the designated mental health clinician authority's (DMHCA) office. The sign in
sheets, weekly clinical summaries, and fidelity monitoring forms are maintained in the DMHCA's
office. Pathways is held as an open group meaning youth can enter the group at any time.
The program provided the Pathways manual for review. The manual is a guided workbook the
youth work through on their own with a therapist. There are thirteen chapters with twelve-steps to
complete in chapter twelve. There is not a script for the facilitator to follow. The manual is broken
into individual assignments and reading passages.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Pathways / Roadmaps to Recovery was not observed during the annual review. This
group will be scheduled to be observed during a quarterly review over the next review period. This
indicator is rated as non-applicable.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There were no gaps in service during this review period. The facility has not had any turnover with
therapy staff. Should there be any turnover, another trained therapist would continue faciltiation of
the group to avoid gaps in service. A review of the attendance sheets showed the group is being
held on a consistent basis. The attendance sheets were much easier to read when updated in
December 2019.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program provided twelve months of internal fidelity monitoring forms for review. One therapist
mainly facilitates the Pathways / Roadmaps to Recovery curriculum. All twelve months of internal
fidelity monitoring were provided for review. The designated mental health clinician authority
conducted the internal fidelity monitoring for this curriculum.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The designated mental health clinician authority (DMHCA) conducts weekly supervisons with
therapy staff. Any issue or corrective actions are addressed during those meetings or one-to-one,
when necessary. There were no corrective actions noted on the internal fidelity monitorng forms.
The DMHCA addresses any issues and documents any suggestions on the forms.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The main facilitator for the Pathways / Roadmaps to Recovery curriculum had an annual
performance evaluation conducted. There is a section in the performance evaluation which
includes what the therapist is trained in and an evaluation of their facilitation skills.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 24 weeks for this type of service. Of the 17 youth in the
sample, 100% (17 of 17) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 40 contact hours for this type of service. Of the 17 youth in the
sample, 100% (17 of 17) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 76%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 5 youth
Moderate-High = 1 youth
High = 7 youth
Total Youth in Sample = 17 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 41%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 7 youth
Total Youth in Sample = 17 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 76% (13 of 17) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 41% (7 of 17) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 20 25 80%
Totals 90 95 95%
This SPEP report evaluates Pathways / Roadmaps to Recovery, an intervention delivered at
Crestview Sex Offender Program.
17 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 17 total youth sampled, all
17 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 17
received at least the recommended weeks of service. Youth in the sample completed between 25
and 78 weeks of service, with an average of 44 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 17 total youth
sampled, all 17 included dosage in the EBS Module. Of those youth with correct dosage, 17
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
received at least the recommended hours of service. Youth in the sample completed 52 hours of
service.
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
Based on the risk level of the youth placed at the program (80% Program Optimization
Percentage), the Department should work to ensure a larger proportion of higher risk youth are
recommended. As the program serves youth with a history of sexual behavior problems, this
includes a multitier strategy of working with stakeholders to achieve that practice, as well as
ensuring appropriate treatment services are available in the community to reduce the reliance on
residential commitment for lower risk youth presenting with sexually-related offenses.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 100%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Crestview Sex Offender Program can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Crestview Sex Offender Program can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8

service_type: Pathways/Roadmaps to Recovery


summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Crestview Youth Academy Nonsecure - Borderline
Developmental Disability / Developmental Disability
Youth Opportunity Investments, LLC
(Contract Provider)
4449 Straightline Road
Crestview, Florida 32459
Primary Service: Impact of Crime
SPEP Review Date(s): 2/12/20200
Florida Department of Juvenile Justice
Report Date(s): 2/12/2020
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
All training documentation was provided for review. There have been five case management staff
formally trained in the Impact of Crime IOC) curriculum during the review period. The program
protocol is for the case management staff to conduct IOC curriculum; there are two case
managers assigned to conduct IOC.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
Impact Of Crime (IOC) is considered a closed group, meaning no youth can begin the group once
it has completed the first lesson. All youth are eligible to attend and complete the IOC curriculum.
IOC group is held on Mondays and Thursdays from 5:00pm to 6:00pm. All sign-in sheets are
maintained in a SPEP binder in the case management supervisor's office.
The program has a manual for the delivery of services and provided a copy for this reviewer for
reference during the annual review. The program uses the current IOC manual. The youth are
provided copies of IOC workbooks with all of the needed worksheets. The manual includes
implementation guidelines, identifies the order the lessons should be delivered, includes detailed
instructions for lesson delivery, and all needed worksheets for the youth.
The facilitators of IOC group do an excellent job of documenting what information was disucssed
during the class and how the youth responded.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Impact of Crime was not observed during this review. An observation will be scheduled during a
quarterly review.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program's policy is for all case management staff to be trained in the Impact of Crime (IOC)
curriculum in order to take over a group should a case manager be out for a short or extended
period of time.
Two staff left during the twelve month review period, the transition manager and one case
manager. A case manager was able to facilitate one of the groups and ended the group timely.
The group was familiar with the case manager and the change did not cause a disruption with the
group. The other group was being co-facilitated and the co-facilitator was able to complete the
group with little disruption.
There was a gap in service for the cohort that started October 24, 2019. The gap took place
between December 18, 2019 and January 8, 2020. The program's policy is for the case manager,
who is going to be gone, to assign duties to another case manager to complete. This information
is signed off by the case management supervisor and designated mental health clinical authority
(DMHCA). The case manager assigned to conduct the group while the facilitator was out failed to
complete the task. When the facilitator came back, a review was done of the previous information
and the group was able to continue with fidelity.
A second gap was between December 11, 2019 when one cohort ended and January 31, 2020
when a cohort began. Per an internview with the DMHCA there were not enough youth to begin a
full group by January 11th. However, eleven youth came in between December and the end of
January so a new cohort was able to begin at that time.
There were four cohorts of IOC completed during the review period; two cohorts were ongoing at
the time of this review. The first cohort was conducted February 8, 2019 through June 7, 2019
and was faciliated by the previous transition case manager. Ten youth started the group and ten
youth successfully completed the group. One youth was set to be released prior the group
completion date, so the facilitator completed individual sessions with the youth in order to allow
the youth successful completion.
The second cohort was conducted from April 8, 2019 through August 8, 2019. Ten youth started
the group and ten youth successfully completed the group. A third cohort was conducted from
April 8, 2019 through August 12, 2019. One youth did not complete this group due to an early
release ordered by the commiting court.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
A fourth cohort was started on August 9, 2019 and ended on December 11, 2019. The group
began with ten youth and ended with seven youth. The three youth who graduated the program
early were able to completed IOC through individual sessions.
A fifth cohort was started on October 24, 2019 and is continuing at this time. The group started
with ten youth and continues to maintain the ten youth. All ten youth will be able to successfully
complete this IOC group prior to graduation. A sixth cohort was started on January 31, 2020, with
nine youth.
The program can earn 2 points if there are no unnecessary gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The transition manager conducted internal fidelity monitoring up to the time of separation. During
this time, a case management supervisor position was created. The staff who was hired for this
position was previously trained in the Impact of Crime (IOC) curriculum and had been facilitating
groups for over a year. The case management supervisor is now responsible for all internal fidelity
monitoring.
Two out of thirty-six internal fidelity monitoring forms were missing for the review period. All
internal fidelity monitoring forms are kept in a binder in the designated mental health clinical
authority's office. The case management supervisor reviewes the form with the facilitator after
each group observed.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program's previous policy was for the designated mental health clinical authority (DMHCA) to
conduct internal fidelity monitoring for the youth. The current DMHCA is not trained in the Impact
of Crime (IOC) curriculum. The transition case manager was conducting fidelity monitoring until
her separation from employment. At that time a case management supervisor position was
created and this position now monitors IOC groups.
The process for any corrective action is for the case management supervisor to meet with the
facilitator and address any needs and issues. The facilitator is monitored again to determine if the
issues has been corrected. If it has not been corrected, further action is taken.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
There was one corrective action noted due to the case manager not faciliitating groups as
assigned when the facilitator was out on leave. This was documented in the staff member's file.
There were no other corrective actions noted on the internal fidelity monitoring forms.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The case management staff receive an annual performance evaluation which addresses
faciliation of evidenced based curriculum. All of the case management staff have received an
annual performance evaluation at the time of this review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service - Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 90%
Research indicates the target duration of 16 weeks for this type of service. Of the 36 youth in the
sample, 64% (23 of 36) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 24 contact hours for this type of service. Of the 36 youth in the
sample, 69% (25 of 36) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 97%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 1 youth
Moderate-High = 14 youth
High = 20 youth
Total Youth in Sample = 35 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 56%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 20 youth
Total Youth in Sample = 36 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 97% (35 of 36) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 56% (20 of 36) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 25 25 100%
Totals 67 85 79%
This SPEP report evaluates Impact of Crime, an intervention delivered at Crestview Youth
Academy Nonsecure - Borderline Developmental Disability / Developmental Disability.
36 youth completed the program during the review period. Of those youth, 36 were included in the
sample.
The program scored Medium for Quality of Service Delivery.
The program earned 6 points for Amount of Service: Duration. Of the 36 total youth sampled, only
25 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 23
received at least the recommended weeks of service. Youth in the sample completed between 14
and 19 weeks of service, with an average of 17 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program earned 6 points for Amount of Service: Contact Hours. Of the 36 total youth
sampled, only 25 included dosage in the EBS Module. Of those youth with correct dosage, 25
received at least the recommended hours of service. Youth in the sample completed between 28
and 35 hours of service, with an average of 33 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Crestview Youth Academy Nonsecure - Borderline Developmental Disability / Developmental
Disability can optimize their SPEP Quality of Service Delivery score by ensuring there are no
unnecessary gaps in service and by ensuring fidelity monitoring is completed every month for
each facilitator
Crestview Youth Academy Nonsecure - Borderline Developmental Disability / Developmental
Disability can optimize their SPEP Amount of Service score by ensuring that dosage for all youth
is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

program_name_raw: Crestview Youth Academy Nonsecure

program_status_note: Open


summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Crestview Youth Academy Nonsecure - Borderline
Developmental Disability / Developmental Disability
Youth Opportunity Investments, LLC
(Contract Provider)
4449 Straightline Road
Crestview, Florida 32539
Primary Service: Male Healthy Relationships
SPEP Review Date(s): 2/11/2020
Florida Department of Juvenile Justice
Report Date(s): 2/11/2020
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Male Healthy Relationships.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
Therapy staff are informally trained to conduct Male Health Relationships pursuant to the
program's policy and contract. Six therapists and the designated mental health clinical authority
(DMHCA) were trained and facilitated Male Healthy Relationship groups during the twelve-month
review period. The curriculum is an informal training conducted by the designated mental health
clinical authority during scheduled supervisions. Training is documented on staff attendance
sheets maintained in the Specialized Program Evaluation Protocol (SPEP) binder.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
Male Healthy Relationships is a bundled group of The Teen Relationship Workbook and Young
Men's Work. Male Healthy Relationships is held as an open group and all youth are eligible for
this service. The program maintains two manuals the therapists work from; Young Men's Work
and The Teen Relationship Workbook.
The Young Men's Work manual is maintained in a binder in the clinical director's office. It is
approximately ten sessions and includes videos and handouts. The manual is broken into
individual sessions that include the objective, agenda, session outline, and all handouts. The
lessons also include a script to follow. The manual for The Teen Relationship Workbook is also
maintained in a binder in the clinical director's office. It is five main topics broken down into
individual lessons. All lessons and handouts are included. Facilitator guidelines are also included,
but no script is provided.
The group is held on Fridays from 3:30pm-4:30pm and Sundays from 1:30pm - 2:30pm. Three
groups are held during the scheduled times to include all the youth in the program. The amount of
groups change based upon the census of the program at the time. The sign-in sheets are
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
maintained in the designated mental health clinical authority's (DMHCA) office in a binder labeled
"Warriors sign-ins".
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The group was not observed during this review. An observation of this group will be scheduled
during a quarterly review.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program's protocol is for other therapists or the designated mental health clinical authority
(DMHCA) to cover a group until the therapist comes back or a new therapist is hired.
Five therapists have left during the twelve month review period. Two left over the summer of
2019, one left in December 2019, and two left in February 2020. The therapy staff is short at this
time, but are able to cover all groups. According to a review of the sign-in sheets for Male Healthy
Relationships there were not gaps in service during this review period.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program's process if for the designated mental health clinical authority (DMHCA) to observe
groups for internal fidelity monitoring. During the review period, seven therapists facilitated Male
Healthy Relationships.
The internal fidelity monitoring forms were provided for review. Four internal fidleity monitoring
forms were missing out of eighty for the twelve month review period. The internal fidelity
monitoring forms are kept in the designated mental health clinical authority's (DMHCA) office in
binder. The forms are separated by therapist.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program's process for corrective action is for the designated mental health clinical authority
(DMHCA) to meet with the facilitator one on one to discuss the issues. The DMHCA will observe
the group again to ensure the issue has been resolved. If the issue is not resolved further action is
taken.
Upon review of the internal fidelity monitoring forms, no corrective actions were noted nor taken.
There were suggestions made and feedback provided for the facilitator(s) on a regular basis.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The annual performance evaluations include a section to address facilitation of evidence-based
services. One therapist has received a ninety-day evaluation. The annual review for this therapist
is due in February.
Three therapists and the designated mental health clinical authority (DMHCA) had an annual
performance evaluation. The four therapists who left during the review period had not received a
performance evaluation prior to leaving.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 24 weeks for this type of service. Of the 36 youth in the
sample, 94% (34 of 36) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 40 contact hours for this type of service. Of the 36 youth in the
sample, 100% (36 of 36) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 97%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 1 youth
Moderate-High = 14 youth
High = 20 youth
Total Youth in Sample = 35 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 56%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 20 youth
Total Youth in Sample = 36 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 97% (35 of 36) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 56% (20 of 36) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 25 25 100%
Totals 93 95 98%
This SPEP report evaluates Male Healthy Relationships, an intervention delivered at Crestview
Youth Academy Nonsecure - Borderline Developmental Disability / Developmental Disability.
36 youth completed the program during the review period. Of those youth, all 36 were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 36 total youth sampled, all
36 included dosage with end dates in the EBS Module. Of those youth, 34 received at least the
recommended weeks of service. Youth in the sample completed between 22 and 43 weeks of
service, with an average of 29 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 10 points for Amount of Service: Contact Hours. Of the 36 total youth
sampled, all 36 included dosage in the EBS Module. Of those youth, 36 received at least the
recommended hours of service. Youth in the sample completed between 42 and 78 hours of
service, with an average of 58 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Crestview Youth Academy Nonsecure - Borderline Developmental Disability / Developmental
Disability can optimize their SPEP Quality of Service Delivery score by ensuring fidelity monitoring
is completed every month for each facilitator.
Crestview Youth Academy Nonsecure - Borderline Developmental Disability / Developmental
Disability can optimize their SPEP Amount of Service score by ensuring that dosage for all youth
is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

service_type: Male Healthy Relationships