summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Deep Creek Youth Academy-Comprehensive Mental Health /
Borderline Developmental Disability/Developmental Disability
Youth Opportunity Investments, Inc
(Contract Provider)
765 E St. Johns Ave
Hastings, Florida 32145
Primary Service: Skillstreaming the Adolescent
SPEP Review Period: August 1, 2022-July 31, 2023
SPEP Review Conducted: August 15, 2023
Florida Department of Juvenile Justice
Report Date: 6/21/2024
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Skillstreaming the Adolescent.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 6 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 0 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A review of the documentation for the annual review period of August 1, 2022 through July 31,
2023 showed the program utilizes Skillstreaming the Adolescent as a primary service. The
documentation shows there were thirteen (13) staff trained in the primary service. The trained
staff are eight therapists, the former clinical director, the current assistant clinical director, the
former assistant clinical director, the designated mental health clinician authority (DMHCA), and
the lead therapist. All thirteen (13) trained staff facilitated the primary service during the annual
review period. There were two staff (therapists) who facilitated the primary service, but the
program did not provide the documentation for two of their therapists. Skillstreaming the
Adolescent training is an internal training conducted by the director of clinical services, the clinical
director, or assistant clinical director. Each trained staff member can conduct the monthly internal
fidelity checks. The internal fidelity checks were completed by the former clinical director, the
assistant clinical director, the DMHCA, the former assistant clinical director, and one therapist
during the annual review period.
The program can earn 1 points if all facilitators are trained in the primary service prior to
facilitating groups.
At the time of the review, the program did not meet criteria to earn a score of 1.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The documentation reviewed for Skillstreaming for the annual review period of August 1, 2022-
July 31, 2023 shows the program used the Skillstreaming the Adolescent 3rd edition as the
manual for the primary service. Technical Assistance Specialist reviewed the manual. The manual
introduces Skillstreaming, contains scripts for the facilitators to follow as well as specific
instructions for each skill taught, example role plays, and student exercises. There are no
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
separate student manuals. The facilitator makes copies of the skill cards for each skill and hands
out those copies to the youth in the group. There are fifty (50) skills taught to the youth.
Skillstreaming is an open group with groups being run year-round. All youth assigned to the
Comprehensive-Borderline Developmental Disability (BDD) side of the program are required to
complete the primary service. The program maintains the attendance sheets and monthly internal
fidelity checks in the SPEP binder titled Skillstreaming. Skillstreaming is held on Tuesdays and
Thursdays.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The technical Assistance Specialist observed a session of Skillstreaming the Adolescent on
8/15/2023 via Zoom. The group began at 2:00p. There were six youth, the facilitator, and the co-
facilitator. The facilitator was knowledgeable of the material and went over each of the concepts
and components of the skill step. However, the facilitator did not model the skill before having the
youth do the role-plays and did not provide an opportunity for the youth to give each other
sufficient feedback on each step of the skill. Each of the youth role-played the skill in groups of
two. The facilitator also did a good job on re-directing the youth from inappropriate examples. The
facilitator and co-facilitator could have more directly related activities to lesson objectives. The
group ended at 3:04p.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The documentation reviewed for Skillstreaming the Adolescent for the annual review period of
August 1, 2022 through July 31, 2023 showed the program had facilitator turnover. The turnover
did negatively affect the facilitation of the primary service. Seven therapists, the former assistant
clinical director, and the former clinical director left the program. The program continued to run the
service by transitioning the youth to another trained facilitator to ensure no gaps in service. The
nine staff were facilitating groups at the time they left the program. All thirteen (13) trained staff
facilitated the primary service during the annual review period.
There were twenty-three (23) dates during the annual review period where the program
documented a supportive counseling session. The supportive counseling sessions were
documented on the group attendance sheets. The facilitators documented the youth would
receive fifteen (15) to twenty-five (25) minutes of supportive counseling. Technical assistance
specialist informed the director of clinical ervices in March 2023 there should not be any
supportive counseling sessions completed unless there was a specific need. All youth should
receive the full hour in a separate group instead of only fifteen minutes of supportive counseling.
The youth receiving the supportive counseling session should be listed on a separate attendance
sheet with the date and times the youth received the service. The director of clinical services
stated supportive counseling sessions are not documented in the Evidence-Based Services
Module (EBS). The documentation after March 2023 only contained one supportive counseling
session completed in June 2023 due to a youth’s medical issue. No other supportive counseling
sessions were noted.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
An attendance sheet dated 7/26/2023 did not have the group times listed. Two attendance sheets
dated 8/30/2022 and 8/9/2022 were filled out by the former clinical director, but signed by the
designated mental health clinician authority (DMHCA).
According to the director of clinical services, Youth Opportunity Investments provides staff
incentives to decrease the likelihood of turnover in implementation of the Standardized Program
Evaluation Protocol (SPEP) services.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of the documentation for Skillstreaming the Adolescent for the annual review period of
August 1, 2022 through July 31, 2023 showed there were forty-six (46) internal fidelity checks
completed. The internal fidelity checks were completed by the former clinical director, the
assistant clinical director, the DMHCA, and two therapists. The program was missing a fidelity
checklist for the lead therapist and former clinical director for the month of August 2022; the
former clinical director for September 2022; the former assistant clinical director for October 2022;
the current assistant clinical Director for January 2023; a therapist and the current assistant
clinical director for February 2023; a therapist for March 2023; the designated mental health
clinician authority (DMHCA) for June 2023; and the current assistant clinical director for July 2023.
There were two fidelity checks dated 12/6/2022 and 3/1/2023 which did not have corresponding
attendance sheets.
According to the director of clinical services, the clinical director, assistant clinical director, or
another trained facilitator conducts the monthly internal fidelity checks for the primary service.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The documentation reviewed for Skillstreaming the Adolescent for the annual review period of
August 1, 2022 through July 31, 2023 showed there were no identified corrective actions noted
during the monthly internal fidelity checks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
According to the Director of Clinical Services, all facilitators who do not provide adequate
facilitation of Standardized Program Evaluation Protocol services will receive progressive
discipline to include retraining, coaching, and formal written write-up.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of the documentation for Skillstreaming the Adolescent for the annual review period of
August 1, 2022 through July 31, 2023 showed the evaluations have an area to document if the
facilitator has been trained in all facets of the position. The program provided annual evaluations
for five therapists, one lead therapist, one designated mental health clinician authority (DMHCA),
and one former clinical director. The program also provided a ninety-day (90) evaluation for one
therapist and the current assistant clinical director. The other three facilitators were not employed
with the program long enough to receive an evaluation. The facilitator’s annual performance
evaluations used very similar language when documenting the facilitator’s skill delivering the
primary service. Six of the eight annual performance evaluations were written by the former
clinical director. One of the two ninety-day (90) evaluations mentioned the services the therapist
facilitates and contained detailed, specific information about the therapists’ delivery of the primary
service. The other ninety-day (90) evaluation did not mention the evidence-based services the
staff facilitates.
According to the director of clinical services, all facilitators are evaluated yearly on group services
provided. Evaluations include implementation and youth engagement.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 16 weeks for this type of service. Of the 34 youth in the
sample, 97% (33 of 34) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 34 youth in the
sample, 100% (34 of 34) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 91%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 4 youth
Moderate-High = 12 youth
High = 15 youth
Total Youth in Sample = 34 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 44%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 15 youth
Total Youth in Sample = 34 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 91% (31 of 34) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 44% (15 of 34) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 23 25 92%
Totals 71 85 84%
This SPEP report evaluates Skillstreaming the Adolescent, an intervention delivered at Deep
Creek Youth Academy-Comprehensive Mental Health / Borderline Developmental Disability /
Developmental Disability .
35 youth completed the program during the review period. Of those youth, 34 were included in the
sample.
The program scored Medium for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 34 total youth sampled, only
33 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 33
received at least the recommended weeks of service. Youth in the sample completed between 22
and 54 weeks of service, with an average of 35 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program earned 10 points for Amount of Service: Contact Hours. Of the 34 total youth
sampled, all 34 included dosage in the EBS Module. Of those youth with correct dosage, 34
received at least the recommended hours of service. Youth in the sample completed between 45
and 105 hours of service, with an average of 63 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
RECOMMENDATION(S):
Deep Creek Youth Academy-Comprehensive Mental Health / Borderline Developmental Disability
/ Developmental Disability can optimize their SPEP Quality of Service Delivery score by ensuring
all facilitators are trained prior to facilitating groups, facilitator turnover does not impact groups,
fidelity monitoring is completed every month for each facilitator, and each facilitator’s performance
evaluation includes discussion of their skill in delivering this primary service.
Deep Creek Youth Academy-Comprehensive Mental Health / Borderline Developmental Disability
/ Developmental Disability can optimize their SPEP Amount of Service score by ensuring dosage
for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted dosage
of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

file_name: FY2023-24 Deep Creek YA CMH-BDD-DD Skillstreaming.pdf


state: FL

facility_name: Deep Creek Youth Academy

summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Deep Creek Youth Academy
Youth Opportunity Investments, Inc
(Contract Provider)
765 E St. Johns Ave
Hastings, Florida 32145
Primary Service: Impact of Crime
SPEP Review Period: August 1, 2022-July 31, 2023
SPEP Review Conducted: August 15, 2023
Florida Department of Juvenile Justice
Report Date: 6/21/2024
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A review of documentation for the annual review period of August 1, 2022 through July 31, 2023
showed the program uses Impact of Crime (IOC) as a primary service. The documentation shows
there were five staff trained in the primary service- two case managers, two transitional service
managers, and the director of case management. All nine trained staff ran cohorts during the
annual review period. The director of clinical services stated any staff trained in the primary
service can conduct IOC on either side of the program (Comprehensive-BDD or Intensive). As of
October 2022, the cohorts have been combined with both Comprehensive-BDD and Intensive
youth to have sufficient numbers to run a cohort. The training is offered virtually via Teams which
is twenty-eight (28) hours long conducted over three days by a qualified trainer. The primary
service is a closed group. Each trained staff member can conduct the monthly internal fidelity
checks. The internal fidelity checks were completed by the former and current director of case
management or the former facility administrator during the annual review period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the Impact of Crime: Addressing the Harm manual. The manual opens with an
introduction to the Impact of Crime Curriculum. Then the manual goes over all seven chapters of
the service and provides a script for the facilitator to follow. The manual also has an appendix and
a student guide with all the student exercises. The program makes copies of the student
exercises and passes them out to the youth during each lesson.
The program conducts the primary service on Tuesdays and Thursdays. The program keeps all
SPEP documentation (attendance sheets, fidelity checks, and training certificates) in a binder
labeled IOC SPEP documentation. All youth admitted to the program are required to participate in
the primary service. The primary service is a closed group with no more than twelve (12) youth
assigned to one cohort.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The technical assistance specialist observed a session of Impact of Crime (IOC) on 7/13/2023
while at the program. The group began at 3:16p. There were nine youth, the facilitator, and two
additional staff members present for the group, a Transition Service Manager and a Unit
Manager. The group was held in a classroom. The facilitator was knowledgeable of the material
and followed the lesson plan. The facilitator related the material to real-life situations and asked
though-provoking questions. The facilitator did an excellent job re-directing the youth when
needed. One youth left early at 3:33p with his case manager and did not return. Two additional
youth left group early at 4:11p, one for behavior disruptions and the other for a bathroom break.
The group ended at 4:19p.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The documentation reviewed for Impact of Crime (IOC) for the annual review period of August 1,
2022 through July 31, 2023 showed the program had facilitator turnover. The turnover did
negatively affect the facilitation of the primary service, however, the program transitioned to
another facilitator if staff left in the middle of a cohort. Two case managers left the program in the
middle of the cohorts they were facilitating (one of those facilitators left 7/28/2022, which was part
of the last review period, and a new facilitator took over the group as of 8/2/2022, part of this
review period). Two case managers, a transitional service manager, and a director of case
management left in between cohorts and did not negatively affect facilitation.
The documentation reviewed shows there were nine cohorts completed and two cohorts ongoing
as of 7/31/2023. The dates of the cohorts are 5/24/2022-9/15/2022, 6/14/2022-10/13/2022,
7/7/2022-11/1/2022, 8/2/2022-11/29/2022, 10/20/2022-2/9/2023, 11/15/2022-3/9/2023,
1/10/2023-5/2/2023, 1/10/2023-5/2/2023, 3/7/2023-6/29/2023, 4/18/2023-current (as of
7/31/2023) and 5/30/2023-current (as of 7/31/2023). One transition service manager, a former
transition service manager, a former director of case management, a former transition service
manager (now the director of case management), and five case managers facilitated the cohorts
during the annual review period. There were no gaps in service.
The cohort which began on 5/24/2022 started with ten (10) youth and ended with ten (10) youth.
The cohort which began on 6/14/2022 began with five youth. Due to COVID issues with low
census, the department allowed cohorts to start with fewer than six youth given they ask for
permission; however, the department was not informed prior to starting and an explanation was
not given as to why. The cohort started with one facilitator but switched to a different facilitator on
8/2/2022.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The cohort which began on 7/7/2022 began with eight youth and ended with eight youth. The
cohort began with a facilitator and a co-facilitator. The co-facilitator took over as facilitator on
8/2/2022 when the main facilitator left.
The cohort which began on 8/2/2022 began with ten (10) youth and ended with nine youth. One
youth was transferred from the program on 9/14/2022. A co-facilitator was added to the group on
11/29/2022.
The cohort which began on 10/20/2022 started with seven youth and ended with five youth. One
youth was transferred on 12/6/2022 and the other youth was discharged on 1/26/2023, prior to
completing the service.
The cohort which began on 11/15/2022 started with eight youth and ended with eight youth.
The cohort which began on 1/10/2023 started with eight youth and ended with eight youth.
The cohort which began on 1/10/2023 started with ten (10) youth and ended with ten (10) youth.
The cohort started with one facilitator but switched to a different facilitator on 3/28/2023.
The cohort which began on 3/7/2023 started with ten (10) youth and ended with nine youth. One
youth was released on 6/27/2023 but was provided the final lesson individually prior to leaving the
program.
The cohort which began on 4/18/2023 started with nine youth and as of 7/31/2023, has eight
youth. One youth was sent to jail on 5/30/2023.
The cohort which began on 5/30/2023 started with ten (10) youth and as of 7/31/2023, has nine
youth. One youth was sent to jail on 6/6/2023.
According to the director of clinical services, Youth Opportunity Investments provides staff
incentives to decrease the likelihood of turnover in implementation of the Standardized Program
Evaluation Protocol services.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of the documentation for the annual review period of August 1, 2022 through July 31,
2023 showed there were thirty-seven (37) internal fidelity checks completed for facilitators of
Impact of Crime. The director of case management, the former director of case management, and
the former facility administrator conducted all monthly internal fidelity checks for the annual review
period. The program was missing two fidelity checks for the months of November 2022 and May
2023 for a case manager.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
An additional five fidelity checks were conducted on 10/27/2022, 11/8/2022, 12/8/2022,
1/19/2023, and 2/9/2023. There were documentation issues with these five fidelity checks as they
were done in the cohort that started on 10/20/2022 and the facilitator listed was the former
director of case management and the co-facilitator was a former transition service manager
(currently the director of case management). The fidelity checks were done on the co-facilitator
and not the main facilitator. The last three fidelity checks were also done by the facilitator for the
co-facilitator.
When these issues were brought up to director of clinical services, it was stated this was a
documentation issue, and the transition service manager was the main facilitator of the group and
they had put the director of case management on the form as a co-facilitator as they believed it
was required. The director of clinical services also stated the issue was identified and corrected
and informed staff that a co-facilitator is only listed on the sign in sheets if they are co-facilitating
for that particular session.
According to the director of case management, the director of case management and the facility
administrator are the only ones that conduct the monthly internal fidelity checks for the primary
service.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The documentation reviewed for the annual review period of August 1, 2022 through July 31,
2023 showed there were no identified corrective actions noted during the monthly internal fidelity
checks for facilitators of Impact of Crime.
According to the director of clinical services, all facilitators who do not provide adequate
facilitation of Standardized Program Evaluation Protocol services will receive progressive
discipline to include retraining, coaching, and formal written write-up.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of the documentation for the annual review period showed the evaluations have an area
to document if the facilitator has been trained in all facets of the position. The program provided
annual evaluations for one case manager and two transition service managers. Each evaluation
had a numerical value assigned for the performance of the facilitator as well as detailed
comments relating to the performance of that facilitator conducting Impact of Crime.
According to the director of clinical services, all facilitators are evaluated yearly on group services
provided. Evaluations include implementation and youth engagement.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 16 weeks for this type of service. Of the 36 youth in the
sample, 100% (36 of 36) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 36 youth in the
sample, 100% (36 of 36) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 94%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 3 youth
Moderate-High = 14 youth
High = 17 youth
Total Youth in Sample = 36 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
Percentage of Youth with High-Risk Level to Reoffend: 47%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 17 youth
Total Youth in Sample = 36 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 94% (34 of 36) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 47% (17 of 36) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 23 25 92%
Totals 83 85 98%
This SPEP report evaluates Impact of Crime, an intervention delivered at Deep Creek Youth
Academy.
38 youth completed the program during the review period. Of those youth, 36 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 36 total youth sampled, all
36 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 36
received at least the recommended weeks of service. Youth in the sample completed between 16
and 18 weeks of service, with an average of 17 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10
The program earned 10 points for Amount of Service: Contact Hours. Of the 36 total youth
sampled, all 36 included dosage in the EBS Module. Of those youth with correct dosage, 36
received at least the recommended hours of service. Youth in the sample completed between 32
and 34 hours of service, with an average of 34 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
RECOMMENDATION(S):
Deep Creek Youth Academy can optimize their SPEP Quality of Service Delivery score by
ensuring facilitator turnover does not impact groups, fidelity monitoring is completed every month
for each facilitator, and each facilitator’s performance evaluation includes discussion of their skill
in delivering this primary service.
Deep Creek Youth Academy can maintain their SPEP Amount of Service score by continuing to
ensure dosage for all youth is recorded accurately in EBS and by ensuring youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 11

source: DJJ

report_type: SPEP

file_name: FY2023-24 Deep Creek YA IOC.pdf

program_name_raw: Deep Creek Youth Academy

program_status_note: Open

service_type: Impact of Crime


summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Deep Creek Youth Academy
Youth Opportunity Investments, Inc
(Contract Provider)
765 E St. Johns Ave
Hastings, Florida 32145
Primary Service: Moral Reconation Therapy
SPEP Review Period: August 1, 2022-July 31, 2023
SPEP Review Conducted: August 15, 2023
Florida Department of Juvenile Justice
Report Date: 6/21/2024
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Moral Reconation Therapy.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A review of the documentation for the annual review period of August 1, 2022 through July 31,
2023 showed the program utilizes Moral Reconation Therapy (MRT) as a primary service. The
documentation shows there were eleven (11) staff trained in the primary service. The staff trained
are seven therapists, the lead therapist, a former assistant clinical director, the current assistant
clinical director, and the former clinical director. MRT training is conducted by a qualified trainer.
There were ten (10) staff (six therapists, the lead therapist, a former assistant clinical director, the
current assistant clinical director, and the former clinical director) who facilitated the primary
service during the annual review period. Each trained staff member can conduct the monthly
internal fidelity checks. The internal fidelity checks were completed by the former clinical director,
the former assistant clinical director, current assistant clinical director, the lead therapist, and a
therapist during the annual review period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program used the Moral Reconation Therapy Facilitator’s Handbook. Technical assistance
specialist reviewed the manual. The Moral Reconation Therapy Facilitator’s Handbook contains
information on the group process, specific objective criteria and guidelines for all exercises and
tasks in Moral Reconation Therapy, and specific “how to” instructions. There are separate student
manuals that are given to the youth. Moral Reconation Therapy was designed to be an open
group with groups being facilitated year-round. All youth assigned to the Intensive Mental Health
side of the program are required to complete the primary service. The program provides the
service on Mondays, Wednesdays, and Fridays. The program maintains the attendance sheets,
staff trainings, and monthly internal fidelity checks in the SPEP binder titled MRT.
c. Observed Adherence to the Manual/Protocol Basic Score: 1 Point(s)
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Technical assistance specialist observed a session of Moral Reconation Training (MRT) on
8/9/2023. The group began at 2:06p. There were ten (10) youth, the facilitator, and three
additional staff members present for the group. The three staff members rotated in and out of the
group during the primary service. The facilitator was knowledgeable of the material and followed
the lesson plan. The facilitator went over to each youth to ensure the youth understood the
concepts of the step they were working on. Some of the youth were assisting the other youth with
their step while the facilitator was assisting other youth. The youth were beginning to get rowdy
and began laughing at the end of group. The facilitator redirected the youth. The group ended at
3:00p.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The documentation reviewed for Moral Reconation Training (MRT) for the annual review period of
August 1, 2022 through July 31, 2023 showed the program had facilitator turnover. The turnover
did negatively affect the facilitation of the primary service. Five therapists, the former assistant
clinical director, and the former clinical director left the program. The program continued to run the
service by transitioning the youth to another trained facilitator to ensure no gaps in service. Six of
the seven staff were facilitating groups at the time they left the program. Six therapists, the lead
therapist, a former assistant clinical director, the current assistant clinical director, and the former
clinical director facilitated the primary service.
There were ten (10) dates between August 2022 and July 2023 where the program documented
youth receiving supportive counseling. The facilitators documented the supportive counseling on
the same attendance sheet as the traditional group. Technical Assistance Specialist informed the
director of clinical services all youth should receive the full hour of group, not a fifteen (15) minute
supportive counseling session. The youth receiving the supportive counseling session should be
listed on a separate attendance sheet with the date and times the youth received the service. The
director of clinical services stated supportive counseling sessions are not documented in the
Evidence-Based Services Module (EBS) of the Juvenile Justice Information System (JJIS).
According to the director of clinical services, Youth Opportunity Investments provides staff
incentives to decrease the likelihood of turnover in implementation of the Standardized Program
Evaluation Protocol services.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of the documentation for Moral Reconation Training (MRT) for the annual review period
of August 1, 2022 through July 31, 2023 showed there were thirty-two (32) internal fidelity checks
completed. The dates of the internal fidelity checks were: 8/10/2022, 8/10/2022, 9/2/2022,
9/5/2022, 9/5/2022, 9/12/2022, 9/19/2022, 10/5/2022, 10/10/2022, 10/12/2022, 10/19/2022,
10/19/2022, 11/2/2022, 11/7/2022, 11/7/2022, 2/21/2023, 2/9/2023, 2/3/2023, 3/31/2023,
3/15/2023, 4/21/2023, 5/30/2023, 5/26/2023, 5/24/2023, 5/5/2023, 6/28/2023, 6/17/2023,
7/26/2023, 7/22/2023, 7/1/2023, 7/21/2023, and 7/1/2023. The former clinical director, the former
assistant clinical director, current assistant clinical director, the lead therapist, and a therapist
conducted all monthly internal fidelity checks for the annual review period. The program was
missing fidelity checks for the month of September 2022 for one therapist; October 2022 for one
therapist; February 2023, March 2023, and June 2023 for the assistant clinical director; April 2023
for the lead therapist and designated mental health clinician authority (DMHCA); and May 2023
for a therapist. The program did not provide any internal fidelity checks for the months of
December 2022 and January 2023.
According to the director of clinical services, the clinical director or another trained facilitator
conducts the monthly internal fidelity checks for the primary service.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The documentation reviewed for Moral Reconation Training (MRT) for the annual review period of
August 1, 2022 through July 31, 2023 showed there were no identified corrective actions noted
during the monthly internal fidelity checks.
According to the director of clinical services, all facilitator’s who do not provide adequate
facilitation of Standardized Program Evaluation Protocol services will receive progressive
discipline to include retraining, coaching, and formal written write-up.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of the documentation for Moral Reconation Training (MRT) for the annual review period
of August 1, 2022 through July 31, 2023 showed the evaluations have an area to document if the
facilitator has been trained in all facets of the position. The program provided annual evaluations
for five therapists, one lead therapist, one designated mental health clinician authority (DMHCA),
and one former clinical director. The program also provided a ninety-day (90) evaluation for one
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
therapist and the current assistant clinical director. The other three facilitators were not employed
with the program long enough to receive an evaluation. The facilitator’s annual performance
evaluations used very similar language when documenting the facilitator’s skill delivering the
primary service. Six of the eight annual performance evaluations were written by the former
clinical director. One of the two ninety-day (90) evaluations mentioned the services the therapist
facilitates and contained detailed, specific information about the therapists’ delivery of the primary
service. The other ninety-day (90) evaluation did not mention the evidence-based services the
staff facilitates.
According to the director of clinical services, all facilitators are evaluated yearly on group services
provided. Evaluations include implementation and youth engagement.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 15 weeks for this type of service. Of the 36 youth in the
sample, 100% (36 of 36) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 45 contact hours for this type of service. Of the 36 youth in the
sample, 78% (28 of 36) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 92%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 3 youth
Moderate-High = 14 youth
High = 16 youth
Total Youth in Sample = 36 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 44%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 16 youth
Total Youth in Sample = 36 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 92% (33 of 36) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 44% (16 of 36) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 23 25 92%
Totals 84 100 84%
This SPEP report evaluates Moral Reconation Therapy, an intervention delivered at Deep Creek
Youth Academy.
38 youth completed the program during the review period. Of those youth, 36 were included in the
sample.
The program scored Medium for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 36 total youth sampled, all
36 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 36
received at least the recommended weeks of service. Youth in the sample completed between 22
and 63 weeks of service, with an average of 34 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program earned 6 points for Amount of Service: Contact Hours. Of the 36 total youth
sampled, all 36 included dosage in the EBS Module. Of those youth with correct dosage, 28
received at least the recommended hours of service. Youth in the sample completed between 28
and 124 hours of service, with an average of 75 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
RECOMMENDATION(S):
Deep Creek Youth Academy can optimize their SPEP Quality of Service Delivery score by
ensuring facilitator turnover does not impact groups, fidelity monitoring is completed every month
for each facilitator, and each facilitator’s performance evaluation includes discussion of their skill
in delivering this primary service.
Deep Creek Youth Academy can optimize their SPEP Amount of Service score by ensuring
dosage for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

file_name: FY2023-24 Deep Creek YA MRT.pdf

service_type: Moral Reconation Therapy