summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Crestview Youth Academy Nonsecure
Youth Opportunity Investments, LLC
(Contract Provider)
4449 Straightline Road
Crestview, Florida 32539
Primary Service: Skillstreaming the Adolescent
SPEP Review Period: February 15, 2020 - February 14, 2021
SPEP Review Conducted: February 18, 2021
Florida Department of Juvenile Justice
Report Date: 5/4/21
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Skillstreaming the Adolescent.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A Standardized Program Evaluation Protocol (SPEP) review consists of twelve months of
documentation of the program's efforts in providing the primary service, Skillstreaming the
Adolescent.
Skillstreaming the Adolescent is facilitated by therapy staff at the program. There were six total
staff trained in the curriculum during the review period. There were four therapists, the assistant
clinical director, and the clinical director. There are four main facilitators for the Skillstreaming the
Adolescent curriculum. There were times during the review period other trained therapists
conducted the group due to a main facilitator being out.
Training consists of one-to-one training with the clinical director. The clinical director also
shadows the facilitation in training until the therapist feels comfortable facilitating the curriculum
and there are no concerns or issues with the facilitation.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program currently uses The Prepare Curriculum manual by Arnold Goldstein. The program
also utilizes the Skillstreaming the Adolescent skill cards. The manual does not have a specific
script for each lesson. It is divided into exercises for each lesson. Each lesson also contains the
objective, materials, procedure: guided questions and discussion. Once the lesson is completed,
the following exercise is practicing the skill using the identified skill card. There is no identified
dosage or modality information for Skillstreaming the Adolescent.
Skillstreaming the Adolescent is held on Mondays and Wednesdays with two groups being held
on both days. The program maintains records for this group through sign-in sheets, fidelity
monitoring, and training documentation which are kept in binders in the assistant clinical director
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
and the clinical director's offices. All youth are eligible to receive this curriculum with no more than
ten youth in a group at a time.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Skillstreaming the Adolescent was observed remotely on Monday, August 17, 2020 from 3:35pm
to 4:35pm. There were six youth in the group. The skill being taught was listening. One therapist
facilitated this group and there was a staff member in the group as well. The youth reviewed the
skill steps and then began role plays. One youth was not participating much at the beginning, but
the other youth in group and the facilitator did a good job encouraging this youth to participate.
The youth did end up participating in a role play and provided feedback. The facilitator did a good
job pausing the role-plays when needed to correct any behaviors and go through the steps again.
The facilitator also played the telelphone game with the group to further the skill steps for
listening.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
An informal interview with the clinical director revealed there was turnover during this review
period for Skillstreaming the Adolescent facilitators. One trained facilitator left on December 23,
2020. The group the therapist was facilitating was able to remain together and another therapist,
whom the youth were familiar with, continued with the group.
All sign-in sheets were provided for review. There was a gap in service from December 23, 2020
to January 6, 2021. The program reports all youth and most therapy staff were in quarantine due
to COVID-19 and groups could not be held as scheduled. This does not qualify as an
unnecessary gap in service. All groups were consistently held for one hour and the group size
averaged between six and seven youth during this review period. All sign-in sheets were provided
for review. Documentation showed all groups were held for one hour and the group size averaged
between six and seven youth during this review period. Two groups are held two days a week to
ensure all youth are able to receive the curriculum.
According to the clinical director and documentation with in the Evidence-Based Services module
(EBS), there were two cycles of Skillstreaming the Adolescent during the review period.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has a process in place to monitor the delivery of Skillstreaming the Adolescent. A
licensed therapist conducts internal fidelity monitoring on a monthly basis. Five therapists have
conducted Skillstreaming the Adolescent over the twelve month review period.
Skillstreaming the Adolescent is to be monitored once a month for each side of the program. All
internal fidelity monitoring forms were provided for review. Based upon the months the therapists
facilitated Skillstreaming the Adolescent there were missing internal fidelity monitoring forms
during December 2020. Per a discussion with the clinical director and documentation provided,
one therapist resigned abruptly without notice on December 23, 2020 and those remaining
fidelities were not completed. The clinical director and assistant clinical director were out due to
COVID from December 19, 2020 to December 28, 2020. Another therapist was out due to COVID
as of December 28, 2020 and returned January 11, 2021. Those internal fidelity monitorings were
not completed in December as well. The clinical director and assistant clinical director are the only
two licensed therapists in the program and conduct internal fidelity monitoring.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
Per documentation provided by the clinical director, the program does have an internal corrective
action process for fidelity monitoring of Skillstreamingthe Adolescent. If there are concerns or
issues during group facilitation the facilitator receives one-to-one coaching with the clinical
director and is retrained, if necessary.
Upon review of the internal fidelity monitoring forms there were no corrective actions noted during
the twelve month review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The annual performance evaluations were reviewed for all facilitators of Skillstreaming the
Adolescent. Annual performance evaluations were completed in October of 2020. There is a
specific section for "Facilitator Skill Delivering Evidenced Based Practices" and Skillstreaming the
Adolescent was listed in all of the performance evaluations.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 16 weeks for this type of service. Of the 26 youth in the
sample, 100% (26 of 26) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 10%
Research indicates a target of 24 contact hours for this type of service. Of the 26 youth in the
sample, 100% (26 of 26) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 92%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 2 youth
Moderate-High = 10 youth
High = 12 youth
Total Youth in Sample = 26 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 46%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 12 youth
Total Youth in Sample = 26 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 92% (24 of 26) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 46% (12 of 26) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 23 25 92%
Totals 83 85 98%
This SPEP report evaluates Skillstreaming the Adolescent, an intervention delivered at Crestview
Youth Academy Nonsecure.
27 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 26 total youth sampled, all
26 included dosage with end dates in the EBS Module. Of those youth, 26 received at least the
recommended weeks of service. Youth in the sample completed between 24 and 29 weeks of
service, with an average of 26 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 10 points for Amount of Service: Contact Hours. Of the 26 total youth
sampled, all 26 included dosage in the EBS Module. Of those youth, 26 received at least the
recommended hours of service. Youth in the sample completed 50 hours of service.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
RECOMMENDATION(S):
Crestview Youth Academy Nonsecure can optimize their SPEP Quality of Service Delivery score
by ensuring facilitator turnover does not impact groups and fidelity monitoring is completed every
month for each facilitator.
Crestview Youth Academy Nonsecure can optimize their SPEP Amount of Service score by
ensuring that dosage for all youth is recorded accurately in EBS and by ensuring that youth
receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

service_type: Seeking Safety


state: FL

facility_name: Crestview Youth Academy

summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Crestview Youth Academy Secure
Youth Opportunity Investments, LLC
(Contract Provider)
4449 Straightline Road
Crestview, Florida 32539
Primary Service: Seven Challenges
SPEP Review Period: February 15, 2020 - February 14, 2021
SPEP Review Conducted: February 18, 2021
Florida Department of Juvenile Justice
Report Date: 5/3/21
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Seven Challenges.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 20%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A Standardized Program Evaluation Protocol (SPEP) review consists of twelve months
documentation of the program's efforts in providing the primary service, Seven Challenges.
Seven Challenges training consistes of watching the three-hour curriculum video followed by an
instructor-led ten hour training, followed by shadowing of, at least, three Seven Challenges
sessions. Only a staff who is leader trained by Seven Challenges can conduct the training for the
therapy staff. Currently, training staff include the clinical director and the director of clinical
services.
There were six trained staff at the program during this review period. There were four therapists,
the assistant clinical director, and the clinical director. There are three main facilitators for the
Seven Challenges curriculum; however, the therapists rotate conducting Seven Challenges
curriculum throughout the review period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program has a manual for the delivery of services. The program currently uses the Seven
Challenges manual by Rober Schwebel, PhD. The youth also have workbooks to work through at
their own pace. These workbooks include the lesson readings and journaling pages. The
facilitator has a Master manual that has all of the lessons.
Seven Challenges is held as an open group with no more than fifteen youth in a group. The
program conducts Seven Challenges three times per week on Thursdays, Saturdays, and
Sundays. Two groups are held each of the three days. The program maintains the records for
Seven Challenges through sign-in sheets, fidelity monitoring, and training documentation in
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
binders kept in the assistant clinical director and clinical director's offices. Youth who have a
substance abuse diagnosis are able to attend Seven Challenges group.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Seven Challenges was observed virtually on September 10, 2020. There were
thirteen youth in the group. The session was on Trust. This reviewer had a copy of the lesson to
follow. The youth were polite and well behaved during the group. The session was held for a full
hour. All objectives were met for the group on this date. The facilitator was comfortable facilitating
the group and did not fully rely on the manual to conduct the group.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
An informal interview with the clinical director confirmed there was turnover during this review
period for Seven Challenges facilitators. One trained facilitator left on December 23, 2020. The
group the therapist was facilitating was able to remain together and another therapist, whom the
youth were familiar with, continued with the group.
All sign-in sheets were provided for review. There was a gap in service from December 23, 2020
to January 6, 2021. The program reports all youth and most therapy staff were in quarantine due
to COVID-19 and groups could not be held as scheduled. This does not qualify as an
unnecessary gap in service. All groups were consistently held for one hour and the group size
averaged between ten to fifteen youth during this review period. All groups were consistently held
for one hour three times a week. According to the clinical director and documentation within the
Evidence-Based Services module (EBS) there were ten cycles of Seven Challenges during the
review year.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has a process in place to monitor the delivery of Seven Challenges to ensure fidelity
to the model. A licensed therapist conducts internal fidelity monitoring monthly for the therapists
conducting Seven Challenges. Three therapists have consistently conducted Seven Challenges
with the other three trained facilitators conducting group as fill-ins during the twelve month review
period. The clinical director and assistant clinincal director conduct internal fidelity monitoring for
Seven Challenges. All internal fidelity monitoring forms were provided for review. All internal
fidelity monitoring forms had feedback for the facilitators.
Based upon the months the therapists facilitated Seven Challenges there were missing internal
fidelity monitoring forms during December 2020. Per a discussion with the clinical director and
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
documentation provided, one therapist resigned abruptly without notice on December 23, 2020
and those remaining fidelities were not completed. The clinical director and assistant clinical
director were out due to COVID from December 19, 2020 to December 28, 2020. Another
therapist was out due to COVID as of December 28, 2020 and returned January 11, 2021. Those
internal fidelity monitorings were not completed in December as well. The clinical director and
assistant clinical director are the only two licensed therapists in the program and conduct internal
fidelity monitoring.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
According to documentation provided by the clinical director, the program has an internal
corrective action process for fidelity monitoring of Seven Challenges. If there are concerns or
issues during group facilitation the facilitator receives one-to-one coaching with the clinical
director and is retrained, if necessary.
Upon review of the internal fidelity monitoring forms, there were no corrective actions noted nor
taken during the twelve month review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The annual performance evaluations were reviewed for all facilitators of Seven Challenges.
Annual performance evaluations were completed in October 2020. There is a specific section for
"Facilitator Skill Delivering Evidenced Based Practices" and Seven Challenges was listed in all of
the performance evaluations.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 24 weeks for this type of service. Of the 35 youth in the
sample, 91% (32 of 35) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 40 contact hours for this type of service. Of the 35 youth in the
sample, 100% (35 of 35) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 89%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 4 youth
Moderate-High = 10 youth
High = 17 youth
Total Youth in Sample = 35 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 49%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 17 youth
Total Youth in Sample = 35 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 89% (31 of 35) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 49% (17 of 35) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 23 25 92%
Totals 91 95 96%
This SPEP report evaluates Seven Challenges, an intervention delivered at Crestview Youth
Academy Secure.
36 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 35 total youth sampled, all
35 included dosage with end dates in the EBS Module. Of those youth, 32 received at least the
recommended weeks of service. Youth in the sample completed between 18 and 38 weeks of
service, with an average of 34 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 35 total youth
sampled, all 35 included dosage in the EBS Module. Of those youth, 35 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
recommended hours of service. Youth in the sample completed between 51 and 100 hours of
service, with an average of 94 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
RECOMMENDATION(S):
Crestview Youth Academy Secure can optimize their SPEP Quality of Service Delivery score by
ensuring fidelity monitoring is completed every month for each facilitator.
Crestview Youth Academy Secure can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

source: DJJ

report_type: SPEP

program_name_raw: Crestview Youth Academy Secure

program_status_note: Open

service_type: Seven Challenges


summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Crestview Youth Academy Secure
Youth Opportunity Investments, LLC
(Contract Provider)
4449 Straightline Road
Crestview, Florida 32539
Primary Service: Impact of Crime
SPEP Review Period: February 15, 2020 - February 14, 2021
SPEP Review Conducted: February 18, 2021
Florida Department of Juvenile Justice
Report Date: 5/3/21
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The Standardized Program Evaluation Protocol (SPEP) review consists of twelve months of
documentation of the program's efforts in providing the primary service, Impact of Crime (IOC).
IOC consists of four days of in-person training conducted by a qualified trainer. Documentation
provided and the Department's learning management system (SkillPro) shows there are four staff
formally trained in IOC: three case managers and the transition specialist. There are two main
facilitators for the secured program.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the current Impact of Crime (IOC) manual. The youth are provided copies of
IOC workbooks with all of the needed worksheets. The manual includes implementation
guidelines, identifies the order the lessons should be delivered, includes detailed instructions for
lesson delivery, and all needed worksheets for the youth.
IOC is held on Tuesdays and Thursdays for the secure program. The program maintains the
group records by utilizing sign-in sheets, fidelity monitoring, and training. This documentation is
kept in a binder in the clinical director or assistant clinical director's office. All youth are eligible to
attend IOC groups. IOC is held as a closed group with no more than twelve youth in the group at
a time.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Impact of Crime was observed on October 2, 2020 and October 23, 2020. The case manager is a
newly trained facilitator and will need to be observed three times.
On October 2, 2020, there were eleven youth in the group. The facilitator was running group alone
with one safety security specialist. The group was somewhat rowdy, but the faciliator handled the
youth appropriately. The group completed three exercises from chapter five and one exercise was
assigned as homework. The suggestion was made for the facilitator to have a co-facilitator to
assist with the youth due to it being such a large group. The group was held for a full hour.
On October 23, 2020, there were eleven youth in the group. There was a co-facilitator for this
group. The two faciltiator have different types of facilitation and did not mesh well. The suggestion
was made for the new facilitator to be assigned to one of the more seasoned facilitators to assist.
The group completed Chapter 6, exercise 7. The group was held for one hour. There were no
behavioral concerns during this group.
During the annual review there was documentation the facilitator was paired with a different co-
facilitator for the new cohort started. According to the clinical director, there did not seem to be
any issues with the facilitators running the group together.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The secure program experienced turnover during this review period. The case management
supervisor left employment in June 2020 and the transition specialist left employment on February
17, 2021. There was a co-facilitator when the transition specialist left employment and was able to
continue the group with minimal impact to the youth. There was not a co-facilitator for the group
the case management supervisor was facilitating upon leaving the program. Another trained
facilitator the youth were familiar with, completed this group.
There were three completed cohorts during the twelve month review period.
The first cohort was held from January 6, 2020 to April 23, 2020. There were twelve youth who
started the group and eleven youth successfully completed the group. One youth transferred to
another program during this group. The second cohort was held from March 31, 2020 to August 6,
2020. There were ten youth who started and successfully completed this group. The third cohort
was held from July 17, 2020 to November 13, 2020. The were eleven youth who started the
group and sucessfully completed the group.
A fourth cohort was started on December 1, 2020 and is continuing to be held at this time. The
group has ten youth at this time.
The program can earn 2 points if facilitator turnover does not impact groups.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has a process in place to monitor the delivery of Impact of Crime (IOC) to ensure
fidelity to the model. Another case management staff conducts internal fidelity monitoring. There
were two main facilitators for IOC groups during this review period. A third facilitator was added
into the group which started on December 1, 2020.
No internal fidelity monitoring was missing for the twelve month review period. All internal fidelity
monitoring forms contained feedback for the facilitator from the observer.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the clinical director and documentation provided verified the program has a
process for corrective actions based on the fidelity monitoring conducted. The program uses
internal fidelity monitoring/corrective action checklists specific to Impact of Crime.
There were no corrective actions noted nor taken during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The performance evaluations for the trained facilitators were reviewed. The evaluations were
completed and addressed "Facilitator Skill Delivering Evidence Based Practices" where the
specific groups the individuals were trained in were listed. This section contained feedback
regarding the delivery of Impact of Crime.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 16 weeks for this type of service. Of the 33 youth in the
sample, 100% (33 of 33) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 33 youth in the
sample, 100% (33 of 33) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 91%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 4 youth
Moderate-High = 10 youth
High = 16 youth
Total Youth in Sample = 33 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 48%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 16 youth
Total Youth in Sample = 33 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 91% (30 of 33) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 48% (16 of 33) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 23 25 92%
Totals 83 85 98%
This SPEP report evaluates Impact of Crime, an intervention delivered at Crestview Youth
Academy Secure.
36 youth completed the program during the review period. Of those youth, 33 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 33 total youth sampled, all
33 included dosage with end dates in the EBS Module. Of those youth, 33 received at least the
recommended weeks of service. Youth in the sample completed between 16 and 37 weeks of
service, with an average of 19 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 10 points for Amount of Service: Contact Hours. Of the 33 total youth
sampled, all 33 included dosage in the EBS Module. Of those youth, 33 received at least the
recommended hours of service. Youth in the sample completed between 31 and 64 hours of
service, with an average of 34 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
RECOMMENDATION(S):
Crestview Youth Academy Secure can optimize their SPEP Quality of Service Delivery score by
ensuring facilitator turnover does not impact groups.
Crestview Youth Academy Secure can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

service_type: Impact of Crime


summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Crestview Youth Academy Secure
Youth Opportunity Investments, LLC
(Contract Provider)
4449 Straightline Road
Crestview, Florida 32539
Primary Service: Male Healthy Relationships
SPEP Review Period: February 15, 2020 - February 14, 2021
SPEP Review Conducted: February 18, 2021
Florida Department of Juvenile Justice
Report Date: 3/1/21
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Male Healthy Relationships.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A Strandardized Program Evaluation Protocol (SPEP) review consists of twelve months of records
and documentation of the program's efforts in providing the primary service, Male Healthy
Relationships.
Male Healthy Relationships (MHR) is a combination of two curricula: Teen Relationships
Workbook and Young Men's Work. Therapists are trained to conduct MHR at this program.
Training consists of one-on-one training with the clinical director. The clinical director also
shadows the facilitation of groups until the therapist feels comfortable facilitating the curriculum
and there are no concerns or issues with the facilitation.
There were six staff trained to conduct MHR during the review period. There are four therapists,
the assistant clinical director, and the clinical director. There are three main facilitators for Teen
Relationships Workbook and two main facilitators for Young Men's Work. The facilitators rotate
throughout the twelve month review period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
Male Healthy Relationships (MHR) is a bundled group of Teen Relationships Workbook and
Young Men's Work and is utilized as a primary service for the program. The program maintains
two manuals the therapists work from; Young Men's Work and Teen Relationships Workbook.
The Young Men's Work manual is maintained in a binder in the clinical director's office. It is
approximately ten sessions and includes videos and handouts. The manual is broken into
individual sessions which include the objective, agenda, session outline, and all handouts. The
lessons also include a script to follow.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
The manual for Teen Relationships Workbook is maintained in a binder in the clinical director's
office. It is five main topics broken down into individual lessons. All lessons and handouts are
included. Facilitator guidelines are also included, but no script is provided.
The service is provided as an open group with no more than ten youth in a group. All youth are
eligible to attend this group. Male Healthy Relationships is held on Tuesday and Fridays with
three groups being held on each day to ensure all youth are able to attend the curriculum. The
program maintains the records for these groups through sign-in sheets, fidelity monitoring, and
training. All documentation is maintained in binders kept in the clinical director or assistant to the
clinical director's office.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Male Healthy Relationships was observed virtually on October 23, 2020. The session
observed was Young Men's Work. There were seven youth in the group. The group was on
session seventeen: drugs. This reviewer was provided a copy of the lesson to follow.
The group was held for one full hour. The youth were polite and all actively participated in the
session. The facilitator was comfortable facilitating the lesson and did not read from the manual.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A conversation with the clinical director confirmed there was turnover during this review period for
Male Healthy Relationships (MHR) facilitators. One trained facilitator left on December 23, 2020.
The group the therapist was facilitating was able to remain together and another therapist, whom
the youth were familiar with, continued with the group. It should be noted therapists rotate
facilitating each group.
All sign-in sheets were provided for review. There was a gap in service from December 23, 2020
to January 6, 2021. The program reports all youth and most therapy staff were in quarantine due
to COVID-19 and groups could not be held as scheduled. This does not qualify as an
unnecessary gap in service. All groups were consistently held for one hour and the group size
averaged between six and seven youth during this review period. Two groups are held at one time
on Tuesdays and Fridays to ensure all youth are able to attend Male Healthy Relationships.
According to the clinical director and documentation within the Evidence-Based Services System
(EBS) there were nine cycles of MHR during the review year.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The program has a process in place to monitor the delivery of Male Healthy Relationships (MHR)
to ensure fidelity to the model. A licensed therapist conducts internal fidelity monitoring monthly
for the therapists conducting MHR. Five therapists have conducted MHR on a rotating schedule
during the twelve month review period.
Male Healthy Relationships is required to be monitored once a month for each curriculum in the
bundle. Youth Opportunity Investments require the facilitators be observed a total of four times a
month. Two times on the non-secured side and two times on the secured side. This would total
288 internal fidelity monitorings during a twelve month review period.
All internal fidelity monitoring forms were provided for review. Based upon the months the
therapists facilitated MHR there were missing internal fidelity monitoring forms during December
2020. Per a discussion with the clinical director and documentation provided, one therapist
resigned abruptly without notice on December 23, 2020 and those remaining fidelities were not
completed. The clinical director and assistant clinical director were out due to COVID from
December 19, 2020 to December 28, 2020. Another therapist was out due to COVID as of
December 28, 2020 and returned January 11, 2021. Those internal fidelity monitorings were not
completed in December as well. The clinical director and assistant clinical director are the only
two licensed therapists in the program and conduct internal fidelity monitoring.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
According to documentation provided by the clinical director, the program has an internal
corrective action process for fidelity monitoring of Male Healthy Relationships. If there are
concerns or issues during group facilitation the facilitator receives one-to-one coaching with the
clinical director and is retrained, if necessary.
Upon review of the internal fidelity monitoring forms there were no corrective actions noted nor
taken during the twelve month review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The annual performance evaluations were reviewed for all facilitators of Male Healthy
Relationships. Annual performance evaluations were completed in October 2020. There is a
specific section for "Facilitator Skill Delivering Evidenced Based Practices" and Male Healthy
Relationships was listed in all of the performance evaluations.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 24 weeks for this type of service. Of the 35 youth in the
sample, 91% (32 of 35) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 40 contact hours for this type of service. Of the 34 youth in the
sample, 94% (33 of 35) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 89%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 4 youth
Moderate-High = 10 youth
High = 17 youth
Total Youth in Sample = 35 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 49%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 17 youth
Total Youth in Sample = 35 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 89% (31 of 35) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 49% (17 of 35) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 23 25 92%
Totals 89 95 94%
This SPEP report evaluates Male Healthy Relationships, an intervention delivered at Crestview
Youth Academy Secure.
36 youth completed the program during the review period. Of those youth, 35 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 35 total youth sampled, all
35 included dosage with end dates in the EBS Module. Of those youth, 32 received at least the
recommended weeks of service. Youth in the sample completed between 18 and 43 weeks of
service, with an average of 37 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 8 points for Amount of Service: Contact Hours. Of the 35 total youth
sampled, all 35 included dosage in the EBS Module. Of those youth, 33 received at least the
recommended hours of service. Youth in the sample completed between 35 and 78 hours of
service, with an average of 72 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
RECOMMENDATION(S):
Crestview Youth Academy Secure can optimize their SPEP Quality of Service Delivery score by
ensuring fidelity monitoring is completed every month for each facilitator.
Crestview Youth Academy Secure can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

service_type: Male Healthy Relationships


summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Crestview Sex Offender Program
Gulf Coast Youth Services
(Contract Provider)
4445 Straightline Road
Crestview, Florida 32539
Primary Service: ARISE Lifeskills
SPEP Review Period: May 1, 2020 - April 30, 2021
SPEP Review Conducted: June 14, 2021
Florida Department of Juvenile Justice
Report Date: 8/9/21
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is ARISE Lifeskills.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes the ARISE Lifeskills curriculum as a primary service. The program's
documentation reflected all seven therapy staff are trained in the curriculum. Therapists are
designated to conduct the ARISE Lifeskills groups for the program. The trained staff includes six
therapists and the clinical director. The therapy staff rotate conducting the ARISE Lifeskills
curriculum. All six therapists have conducted ARISE Lifeskills groups during the twelve month
review period.
The program provided all training documentation. All seven therapists have been trained by a
certified ARISE Lifeskills trainer. The therapists rotate leading the groups. The training is
documented on sign in sheets maintained by the clinical director and the therapists receive a
certificate of completion.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The ARISE Lifeskills facilitator training manual was provided for review. There are multiple
workbooks for the youth to complete working through various types of life skills. The instructor's
manual provides a guide for the lessons, lesson plan forms, and all necessary worksheets. There
is not a specific script for the facilitators to follow.
ARISE Lifeskills groups are held on Fridays from 2:15pm to 3:15pm and 3:15pm to 4:15pm. The
groups are held on Saturdays and Sundays from 8:30am to 9:30am and 9:30am to 10:30am. All
youth in the program are eligible to attend ARISE Lifeskills groups. ARISE Lifeskills is held as an
open group to allow the youth to work through the program throughout their stay. The attendance
sheets were provided for review and there were no issues noted. All documentation for the ARISE
Lifeskills curriculum is kept in the clinical director's office.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of ARISE Lifeskills was not able to be oberved; therefore this indicator is rated as non-
applicable. The reviewer attempted on three occassions to schedule a virtual observation, but was
unable to complete this due to internet issues at the program.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
An informal interview with a licensed therapist, who was sitting in for the clinical director on the
review date, confirmed the program has not experienced any turnover during the twelve month
review period. The therapists rotate conducting ARISE Lifeskills groups so the youth are familiar
with all facilitators. This is to decrease any significant disruption to the groups.
All group sign-in sheets were provided for review. There were no gaps in service during this
review period.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program's policy is for another trained facilitator to conduct monthly internal fidelity monitoring
for each ARISE Lifeskills facilitator. All internal fidelitly monitoring forms were provided for review.
Seventy-four out of seventy-four internal fidelity monitoring forms were reviewed. The internal
fidelity monitoring forms include specific information for feedback and corrective actions /
suggestions. There were no missing internal fidelity monitoring forms for this review period.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The clinical director addresses any issues or corrective actions for faciltatiors of the ARISE
Lifeskills curriculum. The clinical director conducts weekly supervisons with therapy staff. Any
issue or corrective actions are addressed during those meetings or one-to-one, when necessary.
There were no corrective actions noted on the internal fidelity monitoring forms for the twelve
month review period.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
All annual performance evaluations were provided for the twelve month review period. The annual
performance evaluations include a section for group facilitation and it is specific to what groups
the individual is trained to facilitate. All ARISE Lifeskills facilitators have received their annual
performance evaluations for this twelve month review period. There was no specific evaluation of
each facilitator's skill in delivering the intervention.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 24 weeks for this type of service. Of the 7 youth in the
sample, 100% (7 of 7) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 40 contact hours for this type of service. Of the 7 youth in the
sample, 100% (7 of 7) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 13 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 52%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 71%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 1 youth
Moderate-High = 2 youth
High = 2 youth
Total Youth in Sample = 7 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 29%
High Score: 8 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 62%
Table 2
High = 2 youth
Total Youth in Sample = 7 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 71% (5 of 7) youth scored Moderate to High-Risk to reoffend, for a score of
5 points.
Of the SPEP sample, 29% (2 of 7) youth scored High-Risk to reoffend, for a score of 8 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 13 25 52%
Totals 83 95 87%
This SPEP report evaluates ARISE Lifeskills, an intervention delivered at Crestview Sex Offender
Program.
7 youth completed the program during the review period. Of those youth, all 7 were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 7 total youth sampled, all 7
included dosage with end dates in the EBS Module. Of those youth, 7 received at least the
recommended weeks of service. Youth in the sample completed between 30 and 49 weeks of
service, with an average of 39 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 10 points for Amount of Service: Contact Hours. Of the 7 total youth
sampled, all 7 included dosage in the EBS Module. Of those youth, 7 received at least the
recommended hours of service. Youth in the sample completed 52 hours of service.
The program was awarded 13 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Based on the risk level of the youth placed at the program (52% Program Optimization
Percentage), the Department should work to ensure a larger proportion of higher risk youth are
recommended. As the program serves youth with a history of sexual behavior problems, this
includes a multitier strategy of working with stakeholders to achieve that practice, as well as
ensuring appropriate treatment services are available in the community to reduce the reliance on
residential commitment for lower risk youth presenting with sexually-related offenses.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 100%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Crestview Sex Offender Program can optimize their SPEP Quality of Service Delivery score by
ensuring each facilitator’s performance evaluation includes discussion of their skill in delivering
this primary service.
Crestview Sex Offender Program can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

program_name_raw: Crestview Sex Offender Program

program_status_note: Closed

service_type: ARISE