state: FL

summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Deep Creek Youth Academy-Comprehensive Mental Health /
Borderline Developmental Disability / Developmental
Disability
Youth Opportunity Investments, Inc
(Contract Provider)
765 E St. Johns Ave
Hastings, Florida 32145
Primary Service: Impact of Crime
SPEP Review Period: August 1, 2021-July 31, 2022
SPEP Review Conducted: September 1, 2022
Florida Department of Juvenile Justice
Report Date: 1/6/23
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A review of documentation for the annual review period of August 1, 2021 through July 31, 2022
showed the program utilizes Impact of Crime (IOC) as a primary service. The documentation
shows there were eleven staff trained in IOC- seven case managers, two transitional service
managers, the former facility administrator, and the director of case management. The director of
clinical services stated any staff trained in the primary service can conduct IOC on either side of
the program (Comprehensive-BDD or Intensive). The staff facilitating the primary service during
the annual review period for the Comprehensive-BDD side of the program were a transition
service manager and three case managers. The training is in-person or virtual via Teams, which
is twenty-eight (28) hours long conducted over three days by a qualified trainer. Each trained staff
member may conduct monthly internal fidelity checks. The internal fidelity checks were completed
by the director of case management or facility administrator during the annual review period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the Impact of Crime: Addressing the Harm manual. The manual opens with
an introduction to the Impact of Crime curriculum. The manual then goes over all seven chapters
of the service and provides a script for the facilitator to follow. The manual also has an appendix
and a student guide with all the student exercises. The program makes copies of the student
exercises and passes them out to the youth during each lesson.
Impact of Crime is held on Tuesdays and Thursdays. The program keeps all SPEP
documentation (attendance sheets, fidelity checks, and training certificates) in a binder labeled
"IOC SPEP documentation". All youth admitted to the program are required to participate in the
primary service. The primary service is a closed group with no more than twelve (12) youth
assigned to one cohort.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Impact of Crime was observed on 8/23/2022 via Microsoft Teams. The group began
at 11:45a. There were eight youth, the facilitator, and one additional staff member present for the
group. The group was held in classroom number three. The facilitator was knowledgeable of the
material and followed the lesson plan. The facilitator used a game to conduct a review of the
concepts learned from the previous three chapters. The facilitator provided the youth with words
to review for the next lesson. The group ended at 12:46p.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The documentation reviewed for the annual review period of August 1, 2021 through July 31,
2022 showed the program had facilitator turnover for Impact of Crime. The turnover negatively
affected the facilitation of the primary service. Two case managers left the program in the middle
of a cohort they were facilitating.
Documentation provided shows there were three cohorts completed and two cohorts ongoing as
of 7/31/2022. The dates of the cohorts are 8/17/2021-12/16/2021, 12/21/2021-4/14/2022,
2/1/2022-5/24/2022, 6/14/2022-current (as of 7/31/2022), and 7/7/2022-current (as of 7/31/2022).
The transition service manager and three case managers facilitated the cohorts.
The 8/17/2021 cohort began with five youth and ended with two youth. One youth was discharged
from the program on 11/11/2021. One youth was released from the program on 11/18/2021. One
youth was released from the program on 12/2/2021.
The 12/21/2021 cohort began with eight youth and ended with six youth. One youth was
transferred from the program on 3/9/2022. One youth was released from the program on
4/4/2022.
The 2/1/2022 cohort began with ten (10) youth and ended with eight youth. One youth was
released on 5/11/2022. One youth was discharged on 5/18/2022.
The 6/14/2022 cohort which began with five youth and currently has five youth (as of 7/31/2022).
The case manager left the program in July 2022. Another trained facilitator took over the cohort.
The 7/7/2022 cohort began with eight youth and currently has eight youth (as of 7/31/2022). The
case manager left the program in July 2022. The co-facilitator took over the cohort.
According to the director of clinical services, Youth Opportunity Investments provides staff
incentives to decrease the likelihood of turnover in implementation of the Standardized Program
Evaluation Protocol services.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of documentation for the annual review period of August 1, 2021 through July 31, 2022
showed there were sixteen (16) internal fidelity checks completed for facilitators of Impact of
Crime. The director of case management and facility administrator conducted all monthly internal
fidelity checks for the annual review period. The program was missing one fidelity checklist for the
month of April 2022 for a case manager.
According to the director of clinical services, the director of case management or another trained
facilitator conducts the monthly internal fidelity checks for the primary service.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The documentation reviewed for the annual review period of August 1, 2021 through July 31,
2022 showed there were no identified corrective actions noted during the monthly internal fidelity
checks for facilitators of Impact of Crime.
According to the director of clinical services, all facilitators who do not provide adequate
facilitation of Standardized Program Evaluation Protocol services will receive progressive
discipline to include retraining, coaching, and formal written write-up.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of documentation for the annual review period showed the evaluations have an area to
document if the facilitator has been trained in all facets of the position. The program provided
annual evaluations for three case managers, two transition service managers, and the director of
case management. The program also provided a ninety (90)-day evaluation for one case
manager. One case manager’s evaluation had a numerical value assigned for the performance of
the facilitator as well as detailed comments relating to the performance of that facilitator
conducting Impact of Crime. The other five annual evaluations only had a numerical value
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
assigned for the performance of the facilitator. The ninety (90)-day evaluation was for a case
manager who was not facilitating any groups at the time of the evaluation.
According to the director of clinical services, all facilitators are evaluated yearly on group services
provided. Evaluations include implementation and youth engagement.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service - Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 16 weeks for this type of service. Of the 24 youth in the
sample, 71% (17 of 24) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 24 contact hours for this type of service. Of the 24 youth in the
sample, 88% (21 of 24) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 3 youth
Moderate-High = 11 youth
High = 10 youth
Total Youth in Sample = 24 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 42%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 10 youth
Total Youth in Sample = 24 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (24 of 24) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 42% (10 of 24) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 25 25 100%
Totals 69 85 81%
This SPEP report evaluates Impact of Crime, an intervention delivered at Deep Creek Youth
Academy-Comprehensive Mental Health / Borderline Developmental Disability / Developmental
Disability.
32 youth completed the program during the review period. Of those youth, 24 were included in the
sample.
The program scored Medium for Quality of Service Delivery.
The program earned 6 points for Amount of Service: Duration. Of the 24 total youth sampled, only
21 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 17
received at least the recommended weeks of service. Youth in the sample completed between 13
and 18 weeks of service, with an average of 16 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program earned 8 points for Amount of Service: Contact Hours. Of the 24 total youth
sampled, only 21 included dosage in the EBS Module. Of those youth with correct dosage, 21
received at least the recommended hours of service. Youth in the sample completed between 26
and 34 hours of service, with an average of 32 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
RECOMMENDATION(S):
Deep Creek Youth Academy-Comprehensive Mental Health / Borderline Developmental Disability
/ Developmental Disability can optimize their SPEP Quality of Service Delivery score by ensuring
facilitator turnover does not impact groups, fidelity monitoring is completed every month for each
facilitator, and each facilitator’s performance evaluation includes discussion of their skill in
delivering this primary service.
Deep Creek Youth Academy-Comprehensive Mental Health / Borderline Developmental Disability
/ Developmental Disability can optimize their SPEP Amount of Service score by ensuring dosage
for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted dosage
of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

source: DJJ

report_type: SPEP

file_name: FY2022-23 Deep Creek CMH-BDD-DD IOC.pdf

program_name_raw: Deep Creek Youth Academy Comprehensive Mental Health / Deep Creek Youth Academy BIF

program_status_note: Open

service_type: Impact of Crime


facility_name: Deep Creek Youth Academy

summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Deep Creek Youth Academy-Comprehensive Mental Health /
Borderline Developmental Disability / Developmental
Disability
Youth Opportunity Investments, Inc
(Contract Provider)
765 E St. Johns Ave
Hastings, Florida 32145
Primary Service: Skillstreaming the Adolescent
SPEP Review Period: August 1, 2021-July 31, 2022
SPEP Review Conducted: September 1, 2022
Florida Department of Juvenile Justice
Report Date: 1/6/23
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Skillstreaming the Adolescent.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A review of documentation for the annual review period of August 1, 2021 through July 31, 2022
showed the program utilizes Skillstreaming the Adolescent as a primary service. The
documentation shows there were fourteen (14) staff trained in the primary service- ten (10)
therapists, a former assistant clinical director, the current assistant clinical director, the former
clinical director, and the current clinical director. Skillstreaming the Adolescent training is an
internal training conducted by the director of clinical services, clinical director, or assistant clinical
director. There were ten (10) staff (seven therapists, a former assistant clinical director, the former
clinical director, and the current clinical director) who facilitated the primary service during the
annual review period. Each trained staff member may conduct monthly internal fidelity checks.
The internal fidelity checks were completed by the former clinical director, current clinical director,
or assistant clinical director during the annual review period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilize the Skillstreaming the Adolescent, 3rd edition. The manual introduces
Skillstreaming, contains scripts for the facilitators to follow as well as specific instructions for each
skill taught, example role plays, and student exercises. There are no separate student manuals.
The facilitator makes copies of the skill cards for each skill and hands out those copies to the
youth in the group. There are fifty (50) skills taught to the youth. All youth assigned to the
Comprehensive-BDD-DD side of the program are required to complete the primary service. The
program maintains the attendance sheets and monthly internal fidelity checks in the SPEP binder
titled Skillstreaming. Skillstreaming is an open group with groups being run year-round on
Tuesdays and Thursdays.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Skillstreaming the Adolescent was observed on 8/23/2022 via Microsoft Teams. The
group began at 1:43p. There were ten (10) youth, the facilitator, and one additional staff member
present for the group. The group was held in one of the program’s classrooms. The facilitator was
knowledgeable of the material and followed the lesson plan. The facilitator properly modeled the
skill, asked the youth to explain where they seen the skill step used during the role play,
differentiated between an action step and a thinking step, and then asked the youth to explain
what they learned at the end of the session. The group ended at 2:44p.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The documentation reviewed for the annual review period of August 1, 2021 through July 31,
2022 showed the program had facilitator turnover which negatively affected the facilitation of
Skillstreaming the Adolescent. Seven therapists, an assistant clinical director, and the former
clinical director left the program. The program continued to run the service by transitioning the
youth to another trained facilitator to ensure no gaps in service. The nine staff were facilitating
groups at the time they left the program.
There were fifty-eight (58) dates between December 2021 and July 2022 where the program had
more than ten (10) youth listed on the attendance sheet for the mental health group. The
facilitators documented ten (10) youth received the traditional group for one hour and the
additional youth would receive fifteen (15) minutes of supportive counseling. The technical
assistance specialist informed the regional clinical director and assistant clinical director there
should only be ten (10) youth in a group. The additional youth should receive the full hour in a
separate group instead of only fifteen (15) minutes of supportive counseling. The youth receiving
the supportive counseling session should be listed on a separate attendance sheet with the date
and times the youth received the service. The regional clinical director stated supportive
counseling sessions are not documented in the Evidence-Based Services Module (EBS).
According to the director of clinical services, Youth Opportunity Investments provides staff
incentives to decrease the likelihood of turnover in implementation of the Standardized Program
Evaluation Protocol services.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of documentation for the annual review period of August 1, 2021 through July 31, 2022
showed there were fifty-one (51) internal fidelity checks completed for facilitators of Skillstreaming
the Adolescent. The former clinical director, current clinical director, and the current assistant
clinical director conducted all monthly internal fidelity checks for the annual review period. The
program was missing one fidelity checklist for the month of July 2022 for the current clinical
director.
According to the director of clinical services, the director of case management or another trained
facilitator conducts the monthly internal fidelity checks for the primary service.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The documentation reviewed for the annual review period of August 1, 2021 through July 31,
2022 showed there were no identified corrective actions noted during the monthly internal fidelity
checks for facilitators of Skillstreaming the Adolescent.
According to the director of clinical services, all facilitators who do not provide adequate
facilitation of Standardized Program Evaluation Protocol services will receive progressive
discipline to include retraining, coaching, and formal written write-up.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of documentation for the annual review period showed the evaluations have an area to
document if the facilitator has been trained in all facets of the position. The program provided
annual evaluations for three therapists. The program also provided a ninety (90)-day evaluation
for two therapists. Two of the three therapists' annual evaluations had a numerical value assigned
for the performance of the facilitator as well as non-detailed comments relating to the performance
of that facilitator conducting Skillstreaming the Adolescent. The comments stated the therapist
demonstrates an understanding of the curricula. One therapist’s annual evaluation had a
numerical value assigned for the performance of the facilitator with detailed comments. The two
ninety (90)-day evaluations mentioned the services the therapist facilitates, but did not contain
detailed, specific information about the therapists’ delivery of the primary service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
According to the director of clinical services, all facilitators are evaluated yearly on group services
provided. Evaluations include implementation and youth engagement.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 16 weeks for this type of service. Of the 28 youth in the
sample, 86% (24 of 28) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 24 contact hours for this type of service. Of the 28 youth in the
sample, 93% (26 of 28) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 96%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 3 youth
Moderate-High = 12 youth
High = 12 youth
Total Youth in Sample = 28 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 43%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 12 youth
Total Youth in Sample = 28 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 96% (27 of 28) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 43% (12 of 28) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 25 25 100%
Totals 71 85 84%
This SPEP report evaluates Skillstreaming the Adolescent, an intervention delivered at Deep
Creek Youth Academy-Comprehensive Mental Health / Borderline Developmental Disability /
Developmental Disability.
32 youth completed the program during the review period. Of those youth, 28 were included in the
sample.
The program scored Medium for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 28 total youth sampled, all
28 included dosage with end dates in the EBS Module. Of those youth, 24 received at least the
recommended weeks of service. Youth in the sample completed between 5 and 38 weeks of
service, with an average of 25 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program earned 8 points for Amount of Service: Contact Hours. Of the 28 total youth
sampled, all 28 included dosage in the EBS Module. Of those youth, 26 received at least the
recommended hours of service. Youth in the sample completed between 16 and 73 hours of
service, with an average of 46 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
RECOMMENDATION(S):
Deep Creek Youth Academy-Comprehensive Mental Health / Borderline Developmental Disability
/ Developmental Disability can optimize their SPEP Quality of Service Delivery score by ensuring
facilitator turnover does not impact groups, fidelity monitoring is completed every month for each
facilitator, and each facilitator’s performance evaluation includes discussion of their skill in
delivering this primary service.
Deep Creek Youth Academy-Comprehensive Mental Health / Borderline Developmental Disability
/ Developmental Disability can optimize their SPEP Amount of Service score by ensuring dosage
for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted dosage
of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

file_name: FY2022-23 Deep Creek CMH-BDD-DD Skillstreaming.pdf

service_type: Skillstreaming the Adolescent


summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Deep Creek Youth Academy
Youth Opportunity Investments, Inc
(Contract Provider)
765 E St. Johns Ave
Hastings, Florida 32145
Primary Service: Impact of Crime
SPEP Review Period: August 1, 2021-July 31, 2022
SPEP Review Conducted: September 1, 2022
Florida Department of Juvenile Justice
Report Date: 1/6/23
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A review of documentation for the annual review period of August 1, 2021 through July 31, 2022
showed the program utilizes Impact of Crime (IOC) as a primary service. The documentation
shows there were eleven (11) staff trained in IOC- seven case managers, two transitional service
managers, the former facility administrator, and the director of case management. The director of
clinical services stated any staff trained in the primary service may conduct IOC on either side of
the program. The staff facilitating IOC during the annual review period for the Intensive side of the
program were a transition service manager, three case managers, and the director of case
management. IOC training is in-person or a virtual training via Teams, which is twenty-eight (28)
hours long conducted over three days by a qualified trainer. Each trained staff member may
conduct monthly internal fidelity checks. The internal fidelity checks were completed by the
director of case management or facility administrator during the annual review period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the Impact of Crime: Addressing the Harm manual. The manual opens with
an introduction to the Impact of Crime (IOC) curriculum. The manual then goes over all seven
chapters of the service and provides a script for the facilitator to follow. The manual also has an
appendix and a student guide with all the student exercises. The program makes copies of the
student exercises and passes them out to the youth during each lesson.
IOC groups are held on Tuesdays and Thursdays. The program keeps all SPEP documentation
(attendance sheets, fidelity checks, and training certificates) in a binder labeled "IOC SPEP
documentation". All youth admitted to the program are required to participate in the primary
service. The primary service is a closed group with no more than twelve (12) youth assigned to
one cohort.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Impact of Crime was observed on 8/9/2022 via Microsoft Teams. The group began at
11:28a. There were ten (10) youth, the facilitator, and one additional staff member present for the
group. The group was held in classroom number five. The facilitator was knowledgeable of the
material and followed the lesson plan. The facilitator ensured the youth understood the concepts.
The facilitator had the youth write the words to know down and then what they think of when they
hear the word. The group ended at 12:37p.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The documentation reviewed for the annual review period of August 1, 2021 through July 31,
2022 showed the program had facilitator turnover for Impact of Crime (IOC). The turnover did not
negatively affect facilitation of IOC. One case manager left the program in the middle of a cohort
they were co-facilitating. The other case manager left the program after the cohort ended.
The documentation reviewed shows there were three cohorts completed and one cohort ongoing
as of 7/31/2022. The dates of the cohorts are 8/17/2021-12/16/2021, 12/21/2021-4/14/2022,
3/22/2022-7/14/2022, and 5/24/2022-current (as of 7/31/2022). The transition service manager,
three case managers, and the director of case management facilitated the cohorts during the
annual review period. There were no gaps in service
The 8/17/2021 cohort began with six youth and ended with four youth. Two youth were released
from the program on 12/2/2021. The 12/21/2021 cohort began with eight youth and ended with
seven youth. One youth was discharged from the program on 3/9/2022.
The 3/22/2022 cohort began with nine youth and ended with seven youth. One youth was
released on 6/14/2022. One youth was released on 6/16/2022. One youth was released on
7/5/2022, but was provided the final three lessons individually prior to leaving the program. A case
manager was co-facilitating the cohort with the director of case management. The case manager
left the program in May 2022. The director of case management continued facilitating the cohort.
The 5/24/2022 cohort began with ten (10) youth and currently has ten (10) youth, as of 7/31/2022.
According to the director of clinical services, Youth Opportunity Investments provides staff
incentives to decrease the likelihood of turnover in implementation of the Standardized Program
Evaluation Protocol services.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
A review of documentation for the annual review period of August 1, 2021 through July 31, 2022
showed there were seventeen (17) internal fidelity checks completed for facilitators of Impact of
Crime. The director of case management and facility administrator conducted all monthly internal
fidelity checks for the annual review period. The program was missing one fidelity checklist for the
month of April 2022 for a case manager.
According to the director of clinical services, the director of case management or another trained
facilitator conducts the monthly internal fidelity checks for the primary service.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The documentation reviewed for the annual review period of August 1, 2021 through July 31,
2022 showed there were no identified corrective actions noted during the monthly internal fidelity
checks for facilitators of Impact of Crime.
According to the director of clinical services, all facilitators who do not provide adequate
facilitation of Standardized Program Evaluation Protocol services will receive progressive
discipline to include retraining, coaching, and formal written write-up.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of documentation for the annual review period showed the evaluations have an area to
document if the facilitator has been trained in all facets of the position. The program provided
annual evaluations for three case managers, two transition service managers, and the director of
case management. One case manager’s evaluation had a numerical value assigned for the
performance of the facilitator as well as detailed comments relating to the performance of that
facilitator conducting Impact of Crime. The other five annual evaluations only had a numerical
value assigned for the performance of the facilitator.
According to the director of clinical services, all facilitators are evaluated yearly on group services
provided. Evaluations include implementation and youth engagement.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 16 weeks for this type of service. Of the 18 youth in the
sample, 72% (13 of 18) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 24 contact hours for this type of service. Of the 18 youth in the
sample, 83% (15 of 18) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 94%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 1 youth
Moderate-High = 5 youth
High = 11 youth
Total Youth in Sample = 18 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 61%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 11 youth
Total Youth in Sample = 18 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 94% (17 of 18) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 61% (11 of 18) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 23 25 92%
Totals 77 85 91%
This SPEP report evaluates Impact of Crime, an intervention delivered at Deep Creek Youth
Academy.
25 youth completed the program during the review period. Of those youth, 18 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 6 points for Amount of Service: Duration. Of the 18 total youth sampled, only
16 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 13
received at least the recommended weeks of service. Youth in the sample completed between 12
and 18 weeks of service, with an average of 16 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 8 points for Amount of Service: Contact Hours. Of the 18 total youth
sampled, only 16 included dosage in the EBS Module. Of those youth with correct dosage, 15
received at least the recommended hours of service. Youth in the sample completed between 23
and 34 hours of service, with an average of 31 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
RECOMMENDATION(S):
Deep Creek Youth Academy can optimize their SPEP Quality of Service Delivery score by
ensuring fidelity monitoring is completed every month for each facilitator, and each facilitator’s
performance evaluation includes discussion of their skill in delivering this primary service.
Deep Creek Youth Academy can optimize their SPEP Amount of Service score by ensuring
dosage for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2022-23 Deep Creek IOC.pdf

program_name_raw: Deep Creek Youth Academy


summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Deep Creek Youth Academy
Youth Opportunity Investments, Inc
(Contract Provider)
765 E St. Johns Ave
Hastings, Florida 32145
Primary Service: Moral Reconation Therapy
SPEP Review Period: August 1, 2021-July 31, 2022
SPEP Review Conducted: September 1, 2022
Florida Department of Juvenile Justice
Report Date: 1/6/23
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Moral Reconation Therapy.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A review of documentation for the annual review period of August 1, 2021 through July 31, 2022
showed the program utilizes Moral Reconation Therapy (MRT) as a primary service. The
documentation shows there were ten (10) staff trained in MRT- seven therapists, a former
assistant clinical director, the current assistant clinical director, and the former clinical director.
MRT training is conducted by a qualified trainer. There were seven staff (five therapists, a former
assistant clinical director, and the former clinical director) who facilitated the primary service
during the annual review period. Each trained staff member may conduct the monthly internal
fidelity checks. The internal fidelity checks were completed by the former clinical director or
assistant clinical director during the annual review period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the Moral Reconation Therapy Facilitator’s Handbook. The Moral Reconation
Therapy Facilitator’s Handbook contains information on the group process, specific objective
criteria and guidelines for all exercises and tasks in Moral Reconation Therapy, and specific “how
to” instructions. There are separate student manuals given to the youth. Moral Reconation
Therapy is designed to be an open group with groups being run year-round. All youth are required
to complete this primary service. The program provides the service on Mondays, Wednesdays,
and Fridays. The program maintains the attendance sheets, staff trainings, and monthly internal
fidelity checks in the SPEP binder titled "MRT".
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
A session of Moral Reconation Therapy was observed on 8/24/2022 via Microsoft Teams. The
group began at 1:43p. There were nine youth, the facilitator, and one additional staff member
present for the group. The group was held in classroom number seven. The facilitator was
knowledgeable of the material and followed the lesson plan. The facilitator went over to each
youth to ensure the youth understood the concepts of the step they were on. There were a group
of three to four youth who were having side conversations instead of working on their step in the
workbook. The direct care staff only attempted to redirect the youth once. The facilitator attempted
to redirect the youth but was unable to consistently redirect the youth since she was working with
one youth at a time on their workbook. The group ended at 2:43p.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The documentation reviewed for the annual review period of August 1, 2021 through July 31,
2022 showed the program had facilitator turnover for Moral Reconation Therapy (MRT) which
negatively affected the facilitation of the primary service. Three therapists, an assistant clinical
director, and the former clinical director left the program. The program continued to run the
service by transitioning the youth to another trained facilitator to ensure no gaps in service. The
five staff were facilitating groups at the time they left the program. Five therapists, a former
assistant clinical director, and the former clinical director facilitated the primary service during the
annual review period.
There were thirty-six (36) dates between April 2022 and July 2022 where the program had more
than ten (10) youth listed on the attendance sheet for the mental health group. The facilitators
documented ten (10) youth received the traditional group for one hour and the additional youth
would receive fifteen (15) minutes of supportive counseling. The technical assistance specialist
informed the regional clinical director and assistant clinical director there should only be ten (10)
youth in a group. The additional youth should receive the full hour in a separate group instead of
only fifteen (15) minutes of supportive counseling. The youth receiving the supportive counseling
session should be listed on a separate attendance sheet with the date and times the youth
received the service. The director of clinical services stated supportive counseling sessions are
not documented in the Evidence-Based Services Module (EBS).
According to the director of clinical services, Youth Opportunity Investments provides staff
incentives to decrease the likelihood of turnover in implementation of the Standardized Program
Evaluation Protocol services.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
A review of documentation for the annual review period of August 1, 2021 through July 31, 2022
showed there were thirty-nine (39) internal fidelity checks completed for facilitators of Moral
Reconation Therapy. The former clinical director and current assistant clinical director conducted
all monthly internal fidelity checks for the annual review period. The program was missing one
fidelity checklist for the month of March 2022 and June 2022 for one therapist and the month of
June 2022 for another therapist. According to the director of clinical services, the director of case
management or another trained facilitator conducts the monthly internal fidelity checks for the
primary service.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The documentation reviewed for the annual review period of August 1, 2021 through July 31,
2022 showed there were no identified corrective actions noted during the monthly internal fidelity
checks for facilitators of Moral Reconation Therapy.
According to the director of clinical services, all facilitators who do not provide adequate
facilitation of Standardized Program Evaluation Protocol services will receive progressive
discipline to include retraining, coaching, and formal written write-up.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of documentation showed the evaluations have an area to document if the facilitator has
been trained in all facets of the position. The program provided annual evaluations for three
therapists. The program also provided a ninety (90)-day evaluation for two therapists. Two of the
three therapists' annual evaluations had a numerical value assigned for the performance of the
facilitator as well as non-detailed comments relating to the performance of that facilitator
conducting Moral Reconation Therapy. The comments stated the therapist demonstrates an
understanding of the curricula. One therapist’s annual evaluation had a numerical value assigned
for the performance of the facilitator with detailed comments. The two ninety (90)-day evaluations
mentioned the services the therapist facilitates, but did not contain detailed, specific information
about the therapists’ delivery of the primary service.
According to the director of clinical services, all facilitators are evaluated yearly on group services
provided. Evaluations include implementation and youth engagement.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 15 weeks for this type of service. Of the 21 youth in the
sample, 90% (19 of 21) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 45 contact hours for this type of service. Of the 21 youth in the
sample, 90% (19 of 21) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 90%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 1 youth
Moderate-High = 5 youth
High = 13 youth
Total Youth in Sample = 21 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 62%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 13 youth
Total Youth in Sample = 21 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 90% (19 of 21) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 62% (13 of 21) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 23 25 92%
Totals 85 100 85%
This SPEP report evaluates Moral Reconation Therapy, an intervention delivered at Deep Creek
Youth Academy.
25 youth completed the program during the review period.
The program scored Medium for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 21 total youth sampled, all
21 included dosage with end dates in the EBS Module. Of those youth, 19 received at least the
recommended weeks of service. Youth in the sample completed between 12 and 43 weeks of
service, with an average of 27 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 21 total youth
sampled, all 21 included dosage in the EBS Module. Of those youth, 19 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
recommended hours of service. Youth in the sample completed between 28 and 107 hours of
service, with an average of 74 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
RECOMMENDATION(S):
Deep Creek Youth Academy can optimize their SPEP Quality of Service Delivery score by
ensuring facilitator turnover does not impact groups, fidelity monitoring is completed every month
for each facilitator, and each facilitator’s performance evaluation includes discussion of their skill
in delivering this primary service.
Deep Creek Youth Academy can optimize their SPEP Amount of Service score by ensuring
dosage for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2022-23 Deep Creek MRT.pdf

service_type: Moral Reconation Therapy


summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Deep Creek Youth Academy-Comprehensive Mental Health /
Borderline Developmental Disability/Developmental Disability
Youth Opportunity Investments, Inc
(Contract Provider)
765 E St. Johns Ave
Hastings, Florida 32145
Primary Service: Impact of Crime
SPEP Review Period: August 1, 2022-July 31, 2023
SPEP Review Conducted: August 15, 2023
Florida Department of Juvenile Justice
Report Date: 6/27/2024
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A review of the documentation for the annual review period of August 1, 2022 through July 31,
2023 showed the program utilizes Impact of Crime (IOC) as a primary service. The documentation
shows there were five staff trained in the primary service. The staff trained are two case
managers, two transitional service managers, and the director of case management. All five
trained staff ran cohorts during the annual review period. The director of clinical services stated
any staff trained in the primary service can conduct IOC on either side of the program
(Comprehensive-BDD or Intensive). As of October 2022, the cohorts have been combined with
both Comprehensive-BDD and Intensive youth to have sufficient numbers to run a cohort. The
training is offered virtually via Teams which is twenty-eight (28) hours long conducted over three
days by a qualified trainer. The primary service is a closed group. Each trained staff member can
conduct the monthly internal fidelity checks. The internal fidelity checks were completed by the
former and current director of case management or the former facility administrator during the
annual review period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the Impact of Crime: Addressing the Harm manual. The manual opens with
an introduction to the Impact of Crime Curriculum. Then the manual goes over all seven chapters
of the service and provides a script for the facilitator to follow. The manual also has an appendix
and a student guide with all the student exercises. The program makes copies of the student
exercises and passes them out to the youth during each lesson.
The program conducts the primary service on Tuesdays and Thursdays. The program keeps all
SPEP documentation (attendance sheets, fidelity checks, and training certificates) in a binder
labeled IOC SPEP documentation. All youth admitted to the program are required to participate in
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
the primary service. The primary service is a closed group with no more than twelve (12) youth
assigned to one cohort.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The technical assistance specialist observed a session of Impact of Crime (IOC) on 7/13/2023
while at the program. The group began at 3:16p. There were nine youth, the facilitator, and two
additional staff members present for the group, a Transition Service Manager and a Unit
Manager. The group was held in a classroom. The facilitator was knowledgeable of the material
and followed the lesson plan. The facilitator related the material to real-life situations and asked
though-provoking questions. The facilitator did an excellent job re-directing the youth when
needed. One youth left early at 3:33p with his Case Manager and did not return. Two additional
youth left group early at 4:11p, one for behavior disruptions and the other for a bathroom break.
The group ended at 4:19p.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The documentation reviewed for Impact of Crime (IOC) for the annual review period of August 1,
2022 through July 31, 2023 showed the program had facilitator turnover. The turnover did
negatively affect the facilitation of the primary service, however, the program transitioned to
another facilitator if staff left in the middle of a cohort. Two Case Managers left the program in the
middle of the cohorts they were facilitating (one of those facilitators left 7/28/2022, which was part
of the last review period, and a new facilitator took over the group as of 8/2/2022, part of this
review period). Two Case Managers, a Transitional Service Manager, and a Director of Case
Management left in between cohorts and did not negatively affect facilitation.
The documentation reviewed shows there were nine cohorts completed and two cohorts ongoing
as of 7/31/2023. The dates of the cohorts are 5/24/2022-9/15/2022, 6/14/2022-10/13/2022,
7/7/2022-11/1/2022, 8/2/2022-11/29/2022, 10/20/2022-2/9/2023, 11/15/2022-3/9/2023,
1/10/2023-5/2/2023, 1/10/2023-5/2/2023, 3/7/2023-6/29/2023, 4/18/2023-current (as of
7/31/2023) and 5/30/2023-current (as of 7/31/2023). One Transition Service Manager, a former
Transition Service Manager, a former Director of Case Management, a former Transition Service
Manager (now the Director of Case Management), and five case managers facilitated the cohorts
during the annual review period. There were no gaps in service.
The cohort which began on 5/24/2022 started with ten (10) youth and ended with ten (10) youth.
The cohort which began on 6/14/2022 began with five youth. Due to COVID issues with low
census, the Department allowed cohorts to start with fewer than six youth given they ask for
permission; however, the Department was not informed prior to starting and an explanation was
not given as to why. The cohort started with one facilitator but switched to a different facilitator on
8/2/2022.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The cohort which began on 7/7/2022 began with eight youth and ended with eight youth. The
cohort began with a facilitator and a co-facilitator. The co-facilitator took over as facilitator on
8/2/2022 when the main facilitator left.
The cohort which began on 8/2/2022 began with ten (10) youth and ended with nine youth. One
youth was transferred from the program on 9/14/2022. A co-facilitator was added to the group on
11/29/2022.
The cohort which began on 10/20/2022 started with seven youth and ended with five youth. One
youth was transferred on 12/6/2022 and the other youth was discharged on 1/26/2023, prior to
completing the service.
The cohort which began on 11/15/2022 started with eight youth and ended with eight youth.
The cohort which began on 1/10/2023 started with eight youth and ended with eight youth.
The cohort which began on 1/10/2023 started with ten (10) youth and ended with ten (10) youth.
The cohort started with one facilitator but switched to a different facilitator on 3/28/2023.
The cohort which began on 3/7/2023 started with ten (10) youth and ended with nine youth. One
youth was released on 6/27/2023 but was provided the final lesson individually prior to leaving the
program.
The cohort which began on 4/18/2023 started with nine youth and as of 7/31/2023, has eight
youth. One youth was sent to jail on 5/30/2023.
The cohort which began on 5/30/2023 started with ten youth and as of 7/31/2023, has nine youth.
One youth was sent to jail on 6/6/2023.
According to the director of clinical services, Youth Opportunity Investments provides staff
incentives to decrease the likelihood of turnover in implementation of the Standardized Program
Evaluation Protocol services.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of the documentation for the annual review period of August 1, 2022 through July 31,
2023 showed there were thirty-seven (37) internal fidelity checks completed for facilitators of
Impact of Crime. The director of case management, the former director of case management, and
the former facility administrator conducted all monthly internal fidelity checks for the annual review
period. The program was missing two fidelity checks for the months of November 2022 and May
2023 for a case manager.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
An additional five fidelity checks were conducted on 10/27/2022, 11/8/2022, 12/8/2022,
1/19/2023, and 2/9/2023. There were documentation issues with these five fidelity checks as they
were done in the cohort that started on 10/20/2022 and the facilitator listed was the former
director of case management and the co-facilitator was a former transition service manager
(currently the director of case management). The fidelity checks were done on the co-facilitator
and not the main facilitator. The last three fidelity checks were also done by the facilitator for the
co-facilitator.
When these issues were brought up to director of clinical services, it was stated that this was a
documentation issue, and the transition service manager was the main facilitator of the group and
they had put the director of case management on the form as a co-facilitator as they believed it
was required. The director of clinical services also stated that the issue was identified and
corrected and informed staff that a co-facilitator is only listed on the sign in sheets if they are co-
facilitating for that particular session.
According to the director of case management, the director of case management and the facility
administrator are the only ones that conduct the monthly internal fidelity checks for the primary
service.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The documentation reviewed for the annual review period of August 1, 2022 through July 31,
2023 showed there were no identified corrective actions noted during the monthly internal fidelity
checks.
According to the director of clinical services, all facilitators who do not provide adequate
facilitation of Standardized Program Evaluation Protocol services will receive progressive
discipline to include retraining, coaching, and formal written write-up.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of the documentation for the annual review period showed the evaluations have an area
to document if the facilitator has been trained in all facets of the position. The program provided
annual evaluations for one case manager and two transition service managers. All of the
evaluations had a numerical value assigned for the performance of the facilitator as well as
detailed comments relating to the performance of that facilitator conducting the primary service.
According to the director of clinical services, all facilitators are evaluated yearly on group services
provided. Evaluations include implementation and youth engagement.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 16 weeks for this type of service. Of the 34 youth in the
sample, 94% (32 of 34) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 24 contact hours for this type of service. Of the 34 youth in the
sample, 97% (33 of 34) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 91%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 4 youth
Moderate-High = 12 youth
High = 14 youth
Total Youth in Sample = 34 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
Percentage of Youth with High-Risk Level to Reoffend: 44%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 15 youth
Total Youth in Sample = 34 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 91% (31 of 34) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 44% (15 of 34) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 23 25 92%
Totals 69 85 81%
This SPEP report evaluates Impact of Crime, an intervention delivered at Deep Creek Youth
Academy-Comprehensive Mental Health / Borderline Developmental Disability/Developmental
Disability.
35 youth completed the program during the review period. Of those youth, 34 were included in the
sample.
The program scored Medium for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 34 total youth sampled, only
33 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 32
received at least the recommended weeks of service. Youth in the sample completed between 14
and 18 weeks of service, with an average of 17 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10
The program earned 8 points for Amount of Service: Contact Hours. Of the 34 total youth
sampled, only 33 included dosage in the EBS Module. Of those youth with correct dosage, 33
received at least the recommended hours of service. Youth in the sample completed between 27
and 68 hours of service, with an average of 34 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
RECOMMENDATION(S):
Deep Creek Youth Academy-Comprehensive Mental Health / Borderline Developmental
Disability/Developmental Disability can optimize their SPEP Quality of Service Delivery score by
ensuring facilitator turnover does not impact groups, fidelity monitoring is completed every month
for each facilitator, and each facilitator’s performance evaluation includes discussion of their skill
in delivering this primary service.
Deep Creek Youth Academy-Comprehensive Mental Health / Borderline Developmental
Disability/Developmental Disability can optimize their SPEP Amount of Service score by ensuring
dosage for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 11

file_name: FY2023-24 Deep Creek YA CMH-BDD-DD IOC.pdf