summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Crestview Youth Academy Nonsecure - Borderline
Developmental Disability / Developmental Disability
Youth Opportunity Investments, LLC
(Contract Provider)
4449 Straightline Road
Crestview, Florida 32539
Primary Service: Skillstreaming the Adolescent
SPEP Review Date(s): 2/11/2020
Florida Department of Juvenile Justice
Report Date(s): 2/11/2020
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Skillstreaming the Adolescent.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
Seven therapy staff were trained during the twelve-month review period to conduct Skillstreaming
the Adolescent groups. The program's policy and contract shows therapeutic staff conduct
Skillstreaming the Adolescent groups to the youth in the program. Six therapists and the
designated mental health clinical authority (DMHCA) were trained to conduct Skillstreaming
during the review period.
Skillstreaming is an informal training. The DMHCA is trained to conduct training to staff members.
This is documented on training sign-in sheets in the Standardized Program Evaluation Protocol
binder kept in the DMHCA's office.
The DMHCA typically will facilitate when needed based upon staffing at the time group.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program has a manual for the delivery of services and provided a copy for this reviewer to
reference during the review. The program currently uses The Prepare Curriculum manual by
Arnold Goldstein. The program also utilizes the Skillstreaming the Adolescent skill cards. The
manual is divided into three sections to address skills, anger control and moral reasoning training.
The manual does not contact specific script for each lesson. It is divided into exercises for each
lesson. Each lesson also contains the objective, materials, procedure: guided questions and
discussion. Once the lesson is completed the following exercise is practicing the skill using the
identified skill card.
The group is held as an open group. The group is designed as a cognitive behavioral group and is
designed to teach adolescents essential social skills. Skillstreaming is conducted on Mondays and
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Tuesdays from 3:30pm-4:30pm. All youth are eligible to attend Skillstreaming the Adolescent. The
youth are able to work through the curriculum throughout their stay at the program.
The program maintains the sign-in sheets in a binder labeled "Warriors Sign-In". The binder is
maintained in the designated mental health clinical authority's (DMHCA) office. Twelve months of
sign-in sheets were reviewed throughout the review period. The sign-in sheets are organized by
month and are put together by curriculum. Per the sign in sheets the groups have ten youth
attending on a consistent basis.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Group was not observed during this review. Skillstreaming the Adolescent will be observed during
a scheduled quarterly review.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
Three therapists have left employment during the twelve month review period. The program's
policy is for another therapist to cover the group the previous therapist was facilitating. Two
therapists turned in a two-week notice and one left employment the same day notice was given.
The therapists are taken off of the groups schedule throughout the two weeks prior to leaving. If
they leave without two weeks, another therapist or the designated mental health clinical authority
facilitates the group. Per review of the group sign-in sheets there were no gaps in service during
the review period.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program's process is for the designated mental health clinical authority (DMHCA) or the
assistant to the DMHCA to conduct internal fidelity monitoring. Four therapists facilitated
Skillstreaming the Adolescent consistently during the review period. Three out of forty-eight
internal fidelity monitoring forms were missing for the review period. All other internal fidelity
montoring forms were accounted for and provided for review. The forms are kept in a binder in the
DMHCA's office. The forms allow for feedback to be provided to the therapist and a section for
debriefing after a group to discuss what went well and any upgrades that might be needed.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The designated mental health clinical authority (DMHCA) conducts weekly supervision with
therapy staff. Any corrective actions or issues are discussed during weekly supervisions. The
DMHCA will then observe the following group to determine if the issue was corrected. No
corrective actions or issues were identified on the internal fidelity monitoring forms for the review
period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Three of the four therapists who consistently facilitated Skillstreaming the Adolescent had
received a yearly evaluation. The fourth therapist had received a ninety-day evaluation. The
ninety-day evaluation and annual evaluation included a section for "Facilitator Skill Delivering
Evidenced Based Practices."
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 16 weeks for this type of service. Of the 35 youth in the
sample, 100% (35 of 35) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 35 youth in the
sample, 100% (35 of 35) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 97%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 1 youth
Moderate-High = 13 youth
High = 20 youth
Total Youth in Sample = 35 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 57%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 20 youth
Total Youth in Sample = 35 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 97% (34 of 35) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 57% (20 of 35) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 25 25 100%
Totals 75 85 88%
This SPEP report evaluates Skillstreaming the Adolescent, an intervention delivered at Crestview
Youth Academy Nonsecure - Borderline Developmental Disability / Developmental Disability.
36 youth completed the program during the review period.
The program scored Medium for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 35 total youth sampled, all
35 included dosage with end dates in the EBS Module. Of those youth, 35 received at least the
recommended weeks of service. Youth in the sample completed between 22 and 33 weeks of
service, with an average of 27 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 35 total youth
sampled, all 35 included dosage in the EBS Module. Of those youth, 35 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
recommended hours of service. Youth in the sample completed between 43 and 50 hours of
service, with an average of 50 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Crestview Youth Academy Nonsecure - Borderline Developmental Disability / Developmental
Disability can optimize their SPEP Quality of Service Delivery score by ensuring facilitator
turnover does not impact groups and by ensuring fidelity monitoring is completed every month for
each facilitator.
Crestview Youth Academy Nonsecure - Borderline Developmental Disability / Developmental
Disability can optimize their SPEP Amount of Service score by ensuring that dosage for all youth
is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

service_type: Skillstreaming the Adolescent


state: FL

facility_name: Crestview Youth Academy

summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Crestview Youth Academy - Secure
Youth Opportunity Investments
(Contract Provider)
4449 Straightline Road
Crestview, Florida 32539
Primary Service: Seven Challenges
SPEP Review Date(s): 2/12/2020
Florida Department of Juvenile Justice
Report Date(s): 2/12/2020
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Seven Challenges.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program's protocol is for all therapy staff to be formally trained in the Seven Challenges
curriculum. All therapy staff take a one day course and then must attend a two day course in order
to facilitate groups.
There are six therapists who have been trained during the review period to facilitate the Seven
Challenges. The training documentation is located in the Standardized Program Evaluation
Protocol (SPEP) binder maintained in the designated mental health clinician authority's (DMHCA)
office. The training documentation was provided for review. The DMHCA was recently trained as
a trainer and can now train staff in the program. There is one therapist who will need the rest of
the training in order to faciltiate groups. This therapist has been sitting in groups with the DMHCA
to observe.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The Standardized Program Evaluation Protocol (SPEP) review encompasses twelve months of
SPEP records. Seven Challenges is held as an open group and can be held for up to ninety-
minutes one time per week. The Seven Challenges groups are held three days a week:
Wednesdays at 2:30pm-3:30pm, Thursdays at 2:30pm - 3:30pm, and Saturdays at 8:00am-
9:00am, 9:00am-10:00am, and 10:00am-11:00am. Sign in sheets are located in a binder labled
"Jaguars sign-in" in the designated mental health clinician authority's (DMHCA) office. The sign in
sheets are organized by month and group.
The program has a manual for the delivery of services and provided a copy for the reviewer. The
program currently uses the Seven Challenges manual by Rober Schwebel, PhD. The youth also
have workbooks to work through at their own pace. These workbooks include the lesson readings
and journaling pages. The facilitator has a Master manual that has all of the lessons. Therapists
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
rotate facilitating the group so all therapists have the opportunity to run the lessons. The DMHCA
maintains the facilitator schedule.
Youth with a substance abuse diagnosis and/or with substance abuse as a top criminogenic need
on their Residential Assessment for Youth (RAY) are eligible to attend the Seven Challenges
group.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
This group was not observed during the annual review. The observation for this group will be
scheduled during a quarterly review. This curriculum was not able to be observed; therefore, this
indicator rates as non-applicable.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program's process for facilitator turnover is for another therapist to facilitate a group when a
therapist leaves or is out for an extended period of time.
Five therapists have left employment during the twelve month review period. There are currently
four therapists fully trained to faciltate Seven Challenges groups. The fifth therapist is able to sit in
and observe groups until she has completed the second part of the required training.
According to a review of the sign-in sheets for the last twelve months, there have been no gaps in
service.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The designated mental health clinician authority (DMHCA) conducts internal fidelity monitoring for
Seven Challenges groups. The internal fidleity monitoring forms are kept in a binder in the
DMHCA's office and organized by month and facilitator.
Six out of eighty-four internal fidelity monitoring forms were missing for the review period.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program's process for corrective actions based on internal fideilty monitoring is for the
designated mental health clinician authority (DMHCA) to conduct a one-on-one supervision with
the facilitator and address the issue. The facilitator will be observed again to ensure the issue has
been corrected. If it has not been corrected further action is taken.
Upon review of the twelve months of internal fidelity monitoring forms, there were no corrective
actions noted.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The annual performance evaluations include a section for facilitation of evidenced based services.
Three therapists and the designated mental health clinician authority (DMHCA) have received
their annual performance evaluations.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 24 weeks for this type of service. Of the 28 youth in the
sample, 100% (28 of 28) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 40 contact hours for this type of service. Of the 28 youth in the
sample, 100% (28 of 28) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 96%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 3 youth
Moderate-High = 8 youth
High = 16 youth
Total Youth in Sample = 28 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 57%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 16 youth
Total Youth in Sample = 28 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 96% (27 of 28) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 57% (16 of 28) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 25 25 100%
Totals 95 95 100%
This SPEP report evaluates Seven Challenges, an intervention delivered at Crestview Youth
Academy - Secure.
28 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 28 total youth sampled, all
28 included dosage with end dates in the EBS Module. Of those youth, 28 received at least the
recommended weeks of service. Youth in the sample completed between 35 and 47 weeks of
service, with an average of 38 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 10 points for Amount of Service: Contact Hours. Of the 28 total youth
sampled, all 28 included dosage in the EBS Module. Of those youth, 28 received at least the
recommended hours of service. Youth in the sample completed 100 hours of service.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Crestview Youth Academy - Secure can optimize their SPEP Quality of Service Delivery score by
ensuring fidelity monitoring is completed every month for each facilitator.
Crestview Youth Academy - Secure can optimize their SPEP Amount of Service score by
ensuring that dosage for all youth is recorded accurately in EBS and by ensuring that youth
receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

source: DJJ

report_type: SPEP

program_name_raw: Crestview Youth Academy Secure

program_status_note: Open

service_type: Seven Challenges


summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Crestview Youth Academy - Secure
Youth Opportunity Investments, LLC
(Contract Provider)
4449 Straightline Road
Crestview, Florida 32539
Primary Service: Impact of Crime
SPEP Review Date(s): 2/12/2020
Florida Department of Juvenile Justice
Report Date(s): 2/12/2020
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
Impact Of Crime requires staff to be formally trained in the curriculum. There was documentation
of training for three case managers, located in the Standardized Program Evaluation Protocol
(SPEP) binder.
There have been two trained facilitators for IOC on the secured side of the program. One of the
faciltiators left employment at the beginning of October 2019. According to the designated mental
health clinical authority (DMHCA), the assistant facilitator administrator (AFA) stepped down from
this position and will be taking the open case management position for the secured program. The
AFA is a formally trained IOC facilitator and will be able to conduct groups.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The Standardized Program Evaluation Protocol (SPEP) review encompasses twelve months of
records. All youth at the program are eligible to attend Impact of Crime (IOC) group due to
receiving charges. The program has a manual for the delivery of services and provided a copy for
reference during the annual review. The program uses the current IOC manual. The youth are
provided copies of IOC workbooks with all of the needed worksheets. The manual includes
implementation guidelines, identifies the order the lessons should be delivered, includes detailed
instructions for lesson delivery, and all needed worksheets for the youth.
Groups are held Mondays and Fridays from 5:00pm - 6:00pm. The attendance forms are kept in a
binder in the designated mental health clinical authority's (DMHCA) office.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
This group was not observed during this review. An observation will be scheduled during a
quarterly review. This curriculum was not able to be observed; therefore, this indicator rates as
non-applicable.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
Three case management staff have left during the review period. The program's policy is for other
case management staff to facilitate the group if they are out for an extended period of time. The
groups being held during the time they left were not disrupted. One case manager was promoted
to the case management supervisor position and is available to facilitate a group when needed.
There was one gap in service between December 11, 2019 and January 31, 2020, but it was not
due to facilitator turnover. Per a conversation with the designated mental health clinical authority
(DMHCA) and the case management supervisor there were not enough youth at that time to start
a new group. There were eleven youth that came in between December and January so a group
was able to start at the end of January.
Three cohorts of Impact of Crime were completed during the review period; a fourth cohort was
ongoing at the time of the review. The first cohort was held February 22, 2019 through June 18,
2019. The group began with eleven youth and ended with eight youth. The youth who were
dropped from group did not have an explanation on the attendance sheets. It was recommended
if youth drop from the group there is an explanation included on the attendance forms in the
designated section.
The second group started on May 17, 2019 through September 28, 2019. There were ten youth
who completed the group. A third cohort began on August 12, 2019 and ended on December 3,
2019. The group began and ended with twelve youth in attendance.
A fourth cohort was started on January 6, 2020 with eight youth and is continuing to meet at this
time. All groups met for one hour. The finished cohorts were conducted with fidelity and met the
required dosage timeframes.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The previous transition manager conducted internal fidelity monitoring for the case management
staff. Now the case management supervisor conducts internal fidelity monitoring for all case
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
management staff. The case management supervisor has been formally trained in the Impact of
Crime curriculum.
The internal fidelity monitoring forms were maintained in a binder in the designated mental health
clinical authority's (DMHCA) office. The fidelity monitoring forms are kept by month in the binder.
Three out of thirty-six monitoring forms were missing for the twelve months of review.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program's process for corrective action is for the case management supervior to observe the
Impact of Crime groups. If there is an issue during the group the case management supervisor will
address it with the facilitator. The facilitator will be observed again to ensure the issue was
resolved. If it was not further action is taken.
Upon review of the internal fidelity forms there were no corrective actions during the review
period. The forms included feedback from the monitor and allowed space for facilitator feedback.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The annual performance evaluations have a sepcific section to address facilitation of evidence-
based services. The case management supervisor has received a yearly performance evaluation
at this time. The current case manager has not been in the position for a year yet.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 16 weeks for this type of service. Of the 26 youth in the
sample, 92% (24 of 26) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 24 contact hours for this type of service. Of the 26 youth in the
sample, 92% (24 of 26) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 96%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 3 youth
Moderate-High = 7 youth
High = 15 youth
Total Youth in Sample = 26 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 58%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 15 youth
Total Youth in Sample = 26 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 96% (25 of 26) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 58% (15 of 26) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 25 25 100%
Totals 81 85 95%
This SPEP report evaluates Impact of Crime, an intervention delivered at Crestview Youth
Academy - Secure.
28 youth completed the program during the review period. Of those youth, 26 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 26 total youth sampled, only
24 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 24
received at least the recommended weeks of service. Youth in the sample completed between 16
and 20 weeks of service, with an average of 18 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 8 points for Amount of Service: Contact Hours. Of the 26 total youth
sampled, only 24 included dosage in the EBS Module. Of those youth with correct dosage, 24
received at least the recommended hours of service. Youth in the sample completed 32 hours of
service.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Crestview Youth Academy - Secure can optimize their SPEP Quality of Service Delivery score by
ensuring fidelity monitoring is completed every month for each facilitator.
Crestview Youth Academy - Secure can optimize their SPEP Amount of Service score by
ensuring that dosage for all youth is recorded accurately in EBS and by ensuring that youth
receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

service_type: Impact of Crime


summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Crestview Youth Academy - Secure
Youth Opportunity Investments
(Contract Provider)
4449 Straightline Road
Crestview, Florida 32539
Primary Service: Male Healthy Relationships
SPEP Review Date(s): 2/12/2020
Florida Department of Juvenile Justice
Report Date(s): 2/12/2020
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Male Healthy Relationships.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
Therapy staff are informally trained to conduct Male Health Relationships pursuant to the
program's policy and contract. Six therapists and the designated mental health clinician authority
(DMHCA) were trained and facilitated Male Healthy Relationship groups during the twelve month
review period.
The curriculum is an informal training conducted by the designated mental health clinical authority
during scheduled supervisions. Training is documented on staff attendance sheets maintained in
the Specialized Program Evaluation Protocol (SPEP) binder.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
Male Healthy Relationships is a bundled group of The Teen Relationships Workbook and Young
Men's Work. Male Healthy Relationships is held as an open group and all youth are eligible for
this service. The program maintains two manuals the therapists work from; Young Men's Work
and Teen Relationships. The Young Men's Work manual is maintained in a binder in the clinical
director's office. It is approximately ten sessions and includes videos and handouts. The manual
is broken into individual sessions that include the objective, agenda, session outline, and all
handouts. The lessons also include a script to follow.
The manual for Teen Relationships is maintained in a binder in the clinical director's office. It is
five main topics broken down into individual lessons. All lessons and handouts are included.
Facilitator guidelines are also included, but no script is provided.
The group is held on Fridays from 2:30pm-3:30pm and Sundays 8:00am-9:00am, 9:00am -
10:00am, and 10:00am-11:00am. Three groups are held during the scheduled times to include all
the youth in the program. The amount of groups change based upon the census of the program at
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
the time. Sign in sheets are maintained in a binder labeled "Jaguars sign-in". They are separated
by month and group.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
This group was not monitored during the annual review. This will be monitored during a quarterly
review. This curriculum was not able to be observed; therefore, this indicator rates as non-
applicable.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program's protocol is for other therapists or the designated mental health clinician authority
(DMHCA) to cover a group until the therapist comes back or a new therapist is hired.
Five therapists have left during the twelve month review period. Two left over the summer of
2019, one left in December 2019, and two left in February 2020. The therapy staff is down at this
time, but are able to cover all groups. Pursuant to a review of the sign-in sheets for Male Healthy
Relationships, there were no gaps in service during this review period.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program's process is for the designated mental health clinician authority (DMHCA) to observe
groups for internal fidelity monitoring. During the review period seven therapists facilitated Male
Healthy Relationships.
The internal fidelity monitoring forms were provided for review. The internal fidelity monitoring
forms are kept in the designated mental health clinical authority's (DMHCA) office in binder.
The forms are separated by therapist. Fidelity monitoring was conducted each month for each
facilitator, without exception.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program's process for corrective action is for the designated mental health clinician authority
(DMHCA) to meet with the facilitator one on one to discuss the issues. The DMHCA will observe
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
the group again to ensure the issue has been resolved. If the issue is not resolved further action is
taken.
Upon review of the internal fidelity monitoring forms, no corrective actions were noted. There were
suggestions made and feedback provided for the facilitator(s) on a regular basis.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The annual performance evaluations include a section to address facilitation of evidenced-based
services. Three therapists and the designated mental health clinician authority (DMHCA) had an
annual performance evaluation. One therapist has received a ninety-day evaluation. The annual
review for this therapist is due in February. The therapists who left during the review period had
not received a performance evaluation prior to leaving.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 24 weeks for this type of service. Of the 28 youth in the
sample, 100% (28 of 28) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 40 contact hours for this type of service. Of the 28 youth in the
sample, 93% (26 of 28) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 96%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 3 youth
Moderate-High = 8 youth
High = 16 youth
Total Youth in Sample = 28 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 57%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 16 youth
Total Youth in Sample = 28 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 96% (27 of 28) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 57% (16 of 28) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 25 25 100%
Totals 93 95 98%
This SPEP report evaluates Male Healthy Relationships, an intervention delivered at Crestview
Youth Academy - Secure.
28 youth completed the program during the review period. Of those youth, all 28 were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 28 total youth sampled, all
28 included dosage with end dates in the EBS Module. Of those youth, 28 received at least the
recommended weeks of service. Youth in the sample completed between 30 and 47 weeks of
service, with an average of 38 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 8 points for Amount of Service: Contact Hours. Of the 28 total youth
sampled, all 28 included dosage in the EBS Module. Of those youth, 26 received at least the
recommended hours of service. Youth in the sample completed between 32 and 78 hours of
service, with an average of 66 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Crestview Youth Academy - Secure can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Crestview Youth Academy - Secure can optimize their SPEP Amount of Service score by
ensuring that dosage for all youth is recorded accurately in EBS and by ensuring that youth
receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

service_type: Male Healthy Relationships


summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Crestview Youth Academy Nonsecure
Youth Opportunity Investments, LLC
(Contract Provider)
4449 Straightline Road
Crestview, Florida 32539
Primary Service: Impact of Crime
SPEP Review Period: February 15, 2020 to February 14, 2021
SPEP Review Conducted: February 18, 2021
Florida Department of Juvenile Justice
Report Date: 5/3/21
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The Standardized Program Evaluation Protocol (SPEP) review consists of twelve months of
records and documentation of the program's efforts in providing the primary service, Impact of
Crime (IOC).
IOC consists of four days of in-person training conducted by a qualified trainer. Documentation
provided and the Department's learning management system (SkillPro) shows there are four staff
formally trained in IOC: three case managers and the transition specialist. There are two main
facilitators for the nonsecure program.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program has a manual for the delivery of services and provided a copy for this reviewer for
reference during the annual review. The program uses the current Impact of Crime (IOC) manual.
The youth are provided copies of IOC workbooks with all of the needed worksheets. The manual
includes implementation guidelines, identifies the order the lessons should be delivered, includes
detailed instructions for lesson delivery, and all needed worksheets for the youth.
Impact of Crime is held on Tuesdays and Fridays for the nonsecure program. The program
maintains the group records by utilizing sign-in sheets, fidelity monitoring, and training. This
documentation is kept in a binder in the clinical director or assistant clinical director's office. All
youth are eligible to attend IOC groups. IOC is held as a closed group with no more than twelve
youth in the group at a time.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
This group was observed remotely on Monday, August 17, 2020 from 2:30pm - 3:30pm. There
were five youth in the group. The faciltiator was comfortable facilitating the group and is consistent
in his facilitation methods. Each youth had their workbooks for this chapter. The group was
beginning chapter 5. The youth began with writing out each word to know and definition. The
terms and definitions were reviewed as a gorup. The group completed exercises one and three.
The youth were well behaved throughout group and all youth participated in the discussions.
There were no concerns with the facilitation of this group.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The non-secured program did not experience turnover during the review period. The program's
policy regarding turnover is for a co-faciltiator to continue running the group or have another case
manager, who the youth are familiar with, continue the group to avoid any gaps in service.
There have been three cohorts completed during this review period.
The first cohort was held from October 24, 2020 to February 26, 2020. There were ten youth who
started and successfully completed the group. The second cohort was held from February 3, 2020
to May 5, 2020. There were ten youth who started the group and seven youth who successfully
completed the group. Three youth were released from the program prior to successful completion.
The third cohort was held from June 3, 2020 to October 2, 2020. There were five youth who
started the group and five you successfully completed the cohort.
There is a fourth cohort that began October 8, 2020 and is still being held at this time. The group
was ending the week of this review. The group began with eight youth and will end with six youth
successfully completing the group. One youth was transferred and one youth left the program
prior to successful completion of IOC.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has a process in place to monitor the delivery of Impact of Crime (IOC) to ensure
fidelity to the model. Another case management staff conducts internal fidelity monitoring. There
were two main facilitators for IOC groups during this review period.
During the review period there were twenty-one fidelity monitorings completed, accounting for
each facilitator for each month they facilitated an IOC group.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the clinical director and documentation provided verified the program has a
process for corrective actions based on the fidelity monitoring conducted. The program uses
internal fidelity monitoring/corrective action checklists specific to Impact of Crime.
There were no corrective actions noted nor taken during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The performance evaluations for the trained facilitators were reviewed. The evaluations were
completed and addressed "Facilitator Skill Delivering Evidence Based Practices" where the
specific groups the individuals were trained in were listed. This section contained feedback
regarding the delivery of Impact of Crime.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 16 weeks for this type of service. Of the 26 youth in the
sample, 88% (23 of 26) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 26 youth in the
sample, 100% (26 of 26) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 92%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 2 youth
Moderate-High = 10 youth
High = 12 youth
Total Youth in Sample = 26 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 46%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 12 youth
Total Youth in Sample = 26 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 92% (24 of 26) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 46% (12 of 26) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 23 25 92%
Totals 81 85 95%
This SPEP report evaluates Impact of Crime, an intervention delivered at Crestview Youth
Academy Nonsecure.
27 youth completed the program during the review period. Of those youth, 26 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 26 total youth sampled, all
26 included dosage with end dates in the EBS Module. Of those youth, 23 received at least the
recommended weeks of service. Youth in the sample completed between 12 and 21 weeks of
service, with an average of 19 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 10 points for Amount of Service: Contact Hours. Of the 26 total youth
sampled, all 26 included dosage in the EBS Module. Of those youth, 26 received at least the
recommended hours of service. Youth in the sample completed between 32 and 35 hours of
service, with an average of 32 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
RECOMMENDATION(S):
Crestview Youth Academy Nonsecure can optimize their SPEP Quality of Service Delivery score
by ensuring facilitator turnover does not impact groups.
Crestview Youth Academy Nonsecure can optimize their SPEP Amount of Service score by
ensuring that dosage for all youth is recorded accurately in EBS and by ensuring that youth
receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

program_name_raw: Crestview Youth Academy Nonsecure


summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Crestview Youth Academy Nonsecure
Youth Opportunity Investments, LLC
(Contract Provider)
4449 Straightline Road
Crestview, Florida 32539
Primary Service: Male Healthy Relationships
SPEP Review Period: February 15, 2020 - February 14, 2021
SPEP Review Conducted: February 18, 2021
Florida Department of Juvenile Justice
Report Date: 5/4/21
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Male Healthy Relationships.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A Standardized Program Evaluation Protocol (SPEP) review consists of twelve months of
documentation of the program's efforts in providing the primary service, Male Healthy
Relationships.
Male Healthy Relationships (MHR) is a combination of two curricula: Teen Relationships
Workbook and Young Men's Work. Therapists are trained to conduct MHR at this program.
Training consists of one-to-one training with the clinical director. The clinical director also
shadows the facilitation in training until the therapist feels comfortable facilitating the curriculum
and there are no concerns or issues with the facilitation.
There were six staff trained to conduct MHR during the review period. There are four therapists,
the assistant clinical director, and the clinical director. The program rotates the therapists
conducting the group on a regular basis. Therefore the youth are familiar with all facilitators of the
curriculum. The clinical director and assistant clinical director are utilized as back-up facilitators as
needed and mainly conduct internal fidelity monitoring for the curriculum.
All training records have been provided for review.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
Male Healthy Relationships (MHR) is a bundled group of Teen Relationships Workbook and
Young Men's Work and is utilized as a primary service for the program. The program maintains
two manuals the therapists work from; Young Men's Work and Teen Relationships. The Young
Men's Work manual is maintained in a binder in the clinical director's office. It is approximately ten
sessions and includes videos and handouts. The manual is broken into individual sessions that
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
include the objective, agenda, session outline, and all handouts. The lessons also include a script
to follow.
The manual for Teen Relationships Workbook is maintained in a binder in the clinical director's
office. It is five main topics broken down into individual lessons. All lessons and handouts are
included. Facilitator guidelines are also included, but no script is provided.
The group is held as an open group with no more than ten youth in a group. All youth are eligible
to attend this group. Male Healthy Relationships is held on Tuesday and Fridays. The program
maintains the records for these groups through sign-in sheets, fidelity monitoring, and training. All
documentation is maintained in binders kept in the clinical director or assistant to the clinical
director's office.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Male Healthy Relationships was observed virtually on October 23, 2020. There were
seven participants in this group. The group was held for a full hour. The facilitator appeared
comfortable teaching the lesson and all youth were engaged and participated in the discussion.
There were no behavioral issues throughout the group time. The facilitator was engaged in the
group and did not read from the manual.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
An informal interview with the clinical director confirmed there was turnover during this review
period for Male Healthy Relationships facilitators. One trained facilitator left on December 24,
2020. The group the therapist was facilitating was able to remain together and another therapist,
whom the youth were familiar with, continued with the group.
All sign-in sheets were provided for review. There was a gap in service from December 23, 2020
to January 6, 2021. The program reports all youth and most therapy staff were in quarantine due
to COVID-19 and groups could not be held as scheduled. This does not qualify as an
unnecessary gap in service. All groups were consistently held for one hour and the group size
averaged between six and seven youth during this review period. Two groups are held at one time
on Tuesdays and Fridays to ensure all youth are able to attend Male Healthy Relationships.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has a process in place to monitor the delivery of Male Healthy Relationships (MHR)
to ensure fidelity to the model. A licensed therapist conducts internal fidelity monitoring monthly
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
for the therapists conducting MHR. Five therapists have conducted MHR on a rotating schedule
during the twelve month review period.
Male Healthy Relationships is required to be monitored once a month for each curriculum in the
bundle. Youth Opportunity Investments require the facilitators be observed a total of four times a
month. Two times on the non-secured side and two times on the secured side. This would total
288 internal fidelity monitorings during a twelve month review period. All internal fidelity monitoring
forms were provided for review. Based upon the months the therapists facilitated MHR there were
no missing internal fidelity monitoring forms during the twelve month review period. All internal
fidelity monitoring forms did provide feedback to the facilitators.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
Documentation provided by the clinical director confirmed the program has an internal corrective
action process for fidelity monitoring. If there are concerns or issues during group facilitation, the
facilitator receives one-to-one coaching with the clinical director and is retrained, if necessary.
Upon review of the internal fidelity monitoring forms there were no corrective actions noted nor
taken during the twelve month review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The annual performance evaluations were reviewed for facilitators of Male Healthy Relationships.
Annual performance evaluations were completed in October 2020. There is a specific section for
"Facilitator Skill Delivering Evidence Based Practices" and Male Healthy Relationships was listed
in all of the performance evaluations.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 24 weeks for this type of service. Of the 26 youth in the
sample, 100% (26 of 26) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 40 contact hours for this type of service. Of the 26 youth in the
sample, 100% (26 of 26) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 92%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 2 youth
Moderate-High = 10 youth
High = 12 youth
Total Youth in Sample = 26 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 46%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 12 youth
Total Youth in Sample = 26 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 92% (24 of 26) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 46% (12 of 26) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 23 25 92%
Totals 93 95 98%
This SPEP report evaluates Male Healthy Relationships, an intervention delivered at Crestview
Youth Academy Nonsecure.
27 youth completed the program during the review period. Of those youth, 26 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 26 total youth sampled, all
26 included dosage with end dates in the EBS Module. Of those youth, 26 received at least the
recommended weeks of service. Youth in the sample completed between 26 and 42 weeks of
service, with an average of 33 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 10 points for Amount of Service: Contact Hours. Of the 26 total youth
sampled, all 26 included dosage in the EBS Module. Of those youth, 26 received at least the
recommended hours of service. Youth in the sample completed between 50 and 78 hours of
service, with an average of 64 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
RECOMMENDATION(S):
Crestview Youth Academy Nonsecure can maintain their SPEP Quality of Service Delivery score
by continuing the practices in place at the time of this review.
Crestview Youth Academy Nonsecure can optimize their SPEP Amount of Service score by
ensuring that dosage for all youth is recorded accurately in EBS and by ensuring that youth
receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9