summary: SPEP - FY17-18

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Miami Youth Academy
TrueCore Behavioral Solutions, LLC
(Contract Provider)
10855 SW 84th St
Miami, Florida 33173
Primary Service: Thinking for a Change
SPEP Review Date(s): February 27‐ March 2, 2018
Florida Department of Juvenile Justice
Report Date(s): March 5, 2018
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Miami Youth Academy MQI Program Code:
Provider Name: TrueCore Behavioral Solutions, LLC. Contract Number:
Location: Miami County / Circuit: 11 Number of Beds:
Review Date(s): February 27- March 2, 2018 Lead Reviewer Code:
Persons Interviewed
Program Director Corporate QI/QA staff # Program Supervisors
DJJ Monitor # Case Managers # Youth
DHA or designee # Clinical Staff # Other (listed by title):
DMHCA or designee # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks # Personnel Records
Fidelity Monitoring Documents Program Schedules 5 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts # Youth Records (Closed)
Staff Evaluations Table of Organization # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports # Health Records
Contract Scope of Services # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change. The program was awarded 30
points because the Primary Service is identified as a Group 5 Service. The specific Sub-
Component Service Type identified is Cognitive Behavioral Therapy. The Primary Service was
identified as this type of service as it is intended to correct faulty cognitions and perceptions and
provides skills individuals can use to monitor thought patterns and correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The Standardized Program Evaluation Protocol (SPEP) review encompasses twelve months of
records. The program is currently utilizing the Thinking for a Change (T4C) 3.1 curriculum. The
program provided documentation of two youth specialist II being trained in the primary service on
January 13, 2017 and were facilitating groups.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the evidenced-based curriculum of Thinking for a Change (T4C) 3.1 to
qualifying youth admitted to the program. T4C is comprised of 25 lessons that build upon each
other and contains appendices that can be used to craft an aftercare program to meet ongoing
cognitive behavioral needs of the group. Not all lessons can be completed in one session, so a
typical delivery cycle may take thirty sessions. Sessions should last between one and two hours.
Ideally, the curriculum is delivered two times per week, with a minimum recommended dosage of
once per week and a maximum of three times per week. Participants must be granted time to
complete mandatory homework between each lesson.
T4C is intended for groups of eight to twelve and should be delivered only by trained facilitators.
Due to its integrated structure, T4C is a closed group, meaning members need to start at the
beginning of a cycle, and may not join the group mid-stream. The curriculum provides a script and
instructions for twenty-five lessons. The curriculum includes implementation guidelines, identifies
the order the lessons are to be delivered, and detailed instructions for each lesson.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
This writer observed a Thinking for a Change (T4C) session on March 1, 2018. The facilitator was
comfortable presenting the lesson to the youth and had knowledge of the content presented. The
classroom was comfortable, spacious and conducive for learning. All the youth participated in the
activities and role plays. The facilitator used affirmations throughout the lesson. The homework
assigned from the previous lesson was reviewed at the beginning of the class and new homework
was assigned at the end. The facilitator used hand outs and a white board with markers very
effectively and followed the lesson as outlined in the manual. Technical assistance specialist
recommends using audio/visual equipment to display the power point slides or print out the slides
to give to the youth and to use chart paper when needed.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The SPEP review included records for the last twelve months. The program completed three
cycles of Thinking for a Change (T4C) during the review period. The first cycle of T4C started on
October 18, 2016 and ended on March 11, 2017. Seven youth started in the group, but six youth
were successfully discharged from the program before group ended. The second cycle started on
October 15, 2016 and was completed on April 29, 2017. Six youth started the group, four
successfully completed, one youth was discharged from the program and one youth was
transferred to another program. The third cycle started on July 26, 2017 and was completed on
December 24, 2017. Nine youth started the group, one successfully completed, and eight youth
were discharged from the program. At the time of the review, the program was conducting two
T4C groups. One started on January 28, 2018 and the other group started on January 25, 2018.
Sessions occur twice a week for one hour each day. The Standardized Program Evaluation
Protocol (SPEP) Youth Missing Dosage Form was completed and reflects one youth with missing
dosages of the primary service T4C. The youth was at the program for approximately two months
but was transferred to a different program. Youth are chosen to participate in T4C based on their
needs as assessed by the Residential Positive Achievement Change Tool (RPACT) and a Mental
Health/Substance Abuse evaluation.
The reviewed documentation and an interview with the regional compliance manager confirmed
there has not been a gap in service which adversely affected T4C group during the last twelve
months. In the event of facilitator turnover, another trained staff will assume the role of the
facilitator for the group. It should be noted that many youths are not completing the full dose of the
curriculum because they are being released before the cohort has ended.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
This writer reviewed twelve months of internal fidelity monitoring reports. Documentation validated
staff members who performed internal fidelity monitoring were trained by a qualified trainer. The
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
internal fidelity monitoring reports indicated one facilitator was monitored twice on March 24, 2017
and on April 29, 2017. A second facilitator was monitored on March 29, 2017 and April 28, 2017.
No further internal fidelity monitoring was conducted for the primary service T4C.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 0 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
This writer reviewed documentation and verified the program has a process for the application of
corrective actions based on the results of the fidelity monitoring conducted. The available internal
fidelity monitoring forms can record the observer’s feedback and recommendations, and ideas for
improvement in the delivery of the primary service of T4C. The program did not conduct internal
fidelity monitoring on a regular basis to be able to provide feedback and corrective action.
The program can earn 1 point if any corrective action needed is identified, applied, and
demonstrated.
At the time of the review, the program did not meet criteria to earn a score of 1.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
This writer reviewed of two semi-annual evaluations of the two identified staff trained to facilitate
and/or fidelity monitor Thinking for a Change groups. An interview with the regional compliance
manager indicated staff were recently trained and were not facilitating groups at the time of the
evaluation and therefore were not acknowledged on the semi-annual evaluation.
The program conducts standardized program evaluation protocol (SPEP) review monthly
meetings which started November 2, 2017. The meeting covers attendance, review of the SPEP
tracker, data entry into the evidence-based services (EBS) system in JJIS, internal fidelity checks,
identifying facilitator training needs, coverage planning for services, announcements, questions or
concerns.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates the target duration of 15 weeks for this type of service. Of the 15 youth in the
sample, 40% (6 of 15) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 45 contact hours for this type of service. Of the 15 youth in the
sample, 0% (0 of 15) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 1 youth
Moderate-High = 5 youth
High = 9 youth
Total Youth in Sample = 15 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 60%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 9 youth
Total Youth in Sample = 15 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (15 of 15) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 60% (9 of 15) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 4 10 40%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 25 25 100%
Totals 84 100 84%
This SPEP report evaluates Thinking for a Change, an intervention delivered at Miami Youth
Academy.
The program scored High for Quality of Service Delivery.
The program earned 4 points for Amount of Service: Duration. Of the 15 total youth sampled, only
9 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 6
received at least the recommended weeks of service. Youth in the sample completed between 6
and 21 weeks of service, with an average of 15 weeks.
The program earned 0 points for Amount of Service: Contact Hours. Of the 15 total youth
sampled, all 15 included dosage in the EBS Module. Of those youth, 0 received at least the
recommended hours of service. Youth in the sample completed between 4 and 25 hours of
service, with an average of 13 hours.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered just prior to the youths' admission.
RECOMMENDATION(S):
Miami Youth Academy can optimize their SPEP Quality of Service Delivery score by ensuring
fidelity monitoring is completed every month for each facilitator, and by ensuring any corrective
action necessary is identified, applied, and demonstrated.
Miami Youth Academy can optimize their SPEP Amount of Service score by ensuring that dosage
for all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10


state: FL

facility_name: Miami Youth Academy

summary: SPEP - FY18-19

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Miami Youth Academy
TrueCore Behavioral Solutions, LLC
(Contract Provider)
10855 SW 84th Street
Miami, Florida 33173
Primary Service: Impact of Crime
SPEP Review Date(s): November 6-8, 2018
Florida Department of Juvenile Justice
Report Date(s): November 13, 2018
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime. The program was awarded 15 points
because the Primary Service is identified as a Group 3 Service. The specific Sub-Component
Service Type identified is Social Skills Training. The Primary Service was identified as this type of
service as it focuses on developing social skills required for an individual to interact in a positive
way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The Standardized Program Evaluation Protocol (SPEP) review encompasses 12 months of
records examinations. According to training records maintained in the Department's learning
management system (SkillPro), all facilitators and all staff providing internal fidelity monitoring
were trained in the primary service by a certified master trainer. Those employees included:
youth care worker, youth care worker supervisor, and assistant facility administrator.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the commercially produced curriculum Impact of Crime: Addressing the
Harm to Victims and the Community 2.0. The curriculum was updated in 2011. It is highly detailed
curriculum providing a facilitator's manual, and student workbook. The facilitator's manual
provides lessons plans, classroom management ideas, and tips for engaging youth discussions.
The student manual provides reproducible worksheets to help facilitate group discussions, and
review concepts learned during the group sessions. Homework materials are also utilized in the
student manual to re-enforce learned concepts. Impact of Crime is a closed group meaning once
a group commences, new youth cannot be added to the group.
Impact of Crime is based on the belief that crime is more than just a legal definition, but rather
crime affects the victim, the offender, their families, and the community. The harm caused brings
with it the moral responsibility to all involved. The focus of Impact of Crime is to help offenders
understand the harm they have created, and then assisting them in taking personal accountability
for their actions. Together, the victim, offender, and the community arrive at a viable solution to
the repair the harm caused and the offender. Once the harm is repaired, the offender
subsequently reintegrates back into the community. Facilitators are encouraged to bring in victim
impact speakers throughout the course of the curriculum. The curriculum also allows for the
planning of a community service project as a capstone to the primary service. Youth are able to
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
provide community service in the community, or on site at the program to help to heal some of the
harm done by delinquent acts.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
During the annual review, the Technical Assistance Specialist observed the primary service. The
group consisted of ten (10) youth and was conducted in a classroom setting. The group was
facilitated by a youth care worker supervisor. An additional direct care staff was present in the
classroom to provide supervision. The facilitator provided the observer a copy of the lesson plan.
The facilitator followed the script but did often break away from the script to ask the youth for their
insights on what was read, and to check for understanding. Two of the youth appeared to easily
stray off task during the hour group session. The facilitator was able to re-focus the youth's
attention without taking away from the rest of the group.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
In reviewing documentation, and an informal interview with clinical director and case management
supervisor, there was no facilitator turnover during the review period which affected delivery of the
primary service. In the event of a facilitator leaving the program, another facilitator trained in the
primary service would assume facilitator duties.
A review of documentation provided and dosage available in the Evidence-Based Services (EBS)
module of the Juvenile Justice Information System (JJIS) revealed four cohorts of Impact of Crime
were initiated during the review period. The first cohort ended in February 2018. The completed
cohort ran from March to June 2018. A third cohort was initiated in June 2018; however, the
program could not provide copies of sign in sheets, but dosage was documented in the EBS. A
fourth cohort was just started in October 2018, approximately two weeks before the annual
compliance review.
The program can earn 2 points if physical documentation supports continuous service to youth
during the review period.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program provided documentation of internal fidelity monitoring taking place monthly during
the previous year.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a procedure in place to address corrective actions for facilitators, if needed. In
informal discussion with the clinical director and case management supervisor, in case of a need
for corrective action for facilitators, a conference is held with the supervisor and facilitator. A
corrective action plan is developed cooperatively between the parties, and additional internal
fidelity monitoring takes place to ensure the improvement of the facilitator's skill set.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The annual performance evaluations were reviewed, and it was documented the facilitators were
providing groups in the primary service. The evaluations also rated the skill of the facilitators in
service delivery.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 16 weeks for this type of service. Of the 15 youth in the
sample, 67% (10 of 15) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates a target of 24 contact hours for this type of service. Of the 15 youth in the
sample, 20% (3 of 15) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 0 youth
Moderate-High = 5 youth
High = 10 youth
Total Youth in Sample = 15 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 67%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 10 youth
Total Youth in Sample = 15 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (15 of 15) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 67% (10 of 15) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 2 10 20%
Risk Level of Youth Served 25 25 100%
Totals 73 85 86%
This SPEP report evaluates Impact of Crime, an intervention delivered at Miami Youth Academy.
31 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 6 points for Amount of Service: Duration. Of the 15 total youth sampled, only
13 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 10
received at least the recommended weeks of service. Youth in the sample completed between 13
and 26 weeks of service, with an average of 19 weeks.
The program earned 2 points for Amount of Service: Contact Hours. Of the 15 total youth
sampled, all 15 included dosage in the EBS Module. Of those youth, 3 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
recommended hours of service. Youth in the sample completed between 12 and 38 hours of
service, with an average of 20 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Impact of Crime is an intervention suitable for all youth. The Department recommends providing
this service to every youth in the program.
Miami Youth Academy can optimize their SPEP Quality of Service Delivery score by ensuring
physical documentation supports continuous service to youth during the review period.
Miami Youth Academy can optimize their SPEP Amount of Service score by ensuring that dosage
for all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

source: DJJ

report_type: SPEP

program_name_raw: Miami Youth Academy - TrueCore

program_status_note: Closed

service_type: Impact of Crime


summary: SPEP - FY18-19

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Miami Youth Academy
TrueCore Behavioral Solutions, LLC
(Contract Provider)
10855 SW 84th Street
Miami, Florida 33173
Primary Service: Pathways to Self-Discovery and Change
SPEP Review Date(s): November 6-9, 2018
Florida Department of Juvenile Justice
Report Date(s): November 13, 2018
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Pathways to Self-Discovery and Change.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A Standardized Program Evaluation Protocol (SPEP) review includes a twelve month examination
of records. The program is utilizing the curriculum Pathways to Self-Discovery and Change. The
curriculum is facilitated twice a week by two master's level therapists and one certified addictions
professional. A review of documentation confirmed all three received training by master trainer.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program is utilizing a commerical published curriculum which is highly organized. It is
designed to be delivered in 32 sessions. The service has a facilitator's guide for tips and ideas on
how to faciliate discussions, as well as a participant guide to reinforce learned concepts and ideas
brought about by the youth's discussions and self discovery of thoughts and ideas.
According to the curriculum, Pathways to Self-Discovery and Change (PDSC) "provides the
specific tools necessary for improving evaluation and treatment of at-risk youth, a particularly
vulnerable patient population in the justice system. Using an adolescent-focused format, this
protocol identifies psychological, biological, and social factors that contribute to the onset of
adolescent deviance, and establishes guidelines for delivery of a 32-session treatment curriculum
designed to rehabilitate both male and female adolescents with co-occurring substance abuse
and criminal conduct. Now in its Second Edition, this guide provides treatment practitioners,
program evaluators, and youth services administrators with the most up to date, comprehensive,
and accessible information for the treatment and rehabilitation of juvenile justice clients. It is built
on theoretical and research advances in the treatment and rehabilitation of juvenile justice clients,
as well as feedback over the past seven years from PSDC counselees, treatment providers, and
program administrators."
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of the primary service was not observed during the review period as groups were being
held outside of the time the review team was on site. This curriculum was not able to be
observed; therefore, this indicator rates as non-applicable.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
According to documentation provided by the program, there has been no facilitator turnover
during the previous twelve months that adversly affected the delivery of the primary service. In
the event of facilitator turnover, the licensed and trained staff would cover any groups until a new
facilitator could be hired and trained in delivery of the primary service.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program provided documentation of internal fidelity monitoring taking place monthly for each
facilitator. The program's director of mental health conducted all internal monitorings.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a policy and procedure in place for corrective action plans based on results of
internal fidelity monitoring. According to documentation provided by the program, no need of
corrective actions were noted for facilitators in the primary service.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of the facilitators' last evaluations noted the facilitators' work in the primary service.
They were evaluated on their skills in the delivery of the service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 24 weeks for this type of service. Of the 31 youth in the
sample, 71% (22 of 31) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 40 contact hours for this type of service. Of the 31 youth in the
sample, 3% (1 of 31) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 1 youth
Moderate-High = 10 youth
High = 20 youth
Total Youth in Sample = 31 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 65%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 20 youth
Total Youth in Sample = 31 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (31 of 31) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 65% (20 of 31) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 25 25 100%
Totals 81 95 85%
This SPEP report evaluates Pathways to Self-Discovery and Change, an intervention delivered at
Miami Youth Academy.
31 youth completed the program during the review period. Of those youth, none had previously
received this intervention in another setting.
The program scored High for Quality of Service Delivery.
The program earned 6 points for Amount of Service: Duration. Of the 31 total youth sampled, only
29 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 22
received at least the recommended weeks of service. Youth in the sample completed between 18
and 50 weeks of service, with an average of 31 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
The program earned 0 points for Amount of Service: Contact Hours. Of the 31 total youth
sampled, only 30 included dosage in the EBS Module. Of those youth with correct dosage, 1
received at least the recommended hours of service. Youth in the sample completed between 5
and 53 hours of service, with an average of 15 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Miami Youth Academy can maintain their SPEP Quality of Service Delivery score by continuing
the practices in place at the time of this review.
Miami Youth Academy can optimize their SPEP Amount of Service score by ensuring that dosage
for all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8

service_type: Pathways to Self‐Discovery and Change


summary: SPEP - FY18-19

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Miami Youth Academy
TrueCore Behavioral Solutions, LLC
(Contract Provider)
10855 SW 84th Street
Miami, Florida 33173
Primary Service: Thinking for a Change
SPEP Review Date(s): November 6-9, 2018
Florida Department of Juvenile Justice
Report Date(s): November 13, 2018
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change. The program was awarded 30
points because the Primary Service is identified as a Group 5 Service. The specific Sub-
Component Service Type identified is Cognitive Behavioral Therapy. The Primary Service was
identified as this type of service as it is intended to correct faulty cognitions and perceptions and
provides skills individuals can use to monitor thought patterns and correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The Standardized Program Evaluation Protocol (SPEP) review encompasses twelve months of
records examinations. The program is currently utilizing the Thinking for a Change (T4C) 3.1
curriculum. T4C utilizes a problem-solving program with both cognitive restructuring, and social
skills interventions bringing the identification of thinking, beliefs, attitudes and values to behavior.
Effort is focused on helping youth become aware of their thoughts, feelings and beliefs in
particular circumstances. All staff either facilitating groups or providing internal fidelity monitoring
were trained in the primary service by a master trainer. The trained staff members include: three
youth care workers, one youth care worker supervisor, and one assistant facility administrator,
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program maintains the manuals for Thinking for a Change (T4C) 3.1. The program utilizes a
highly detailed, commercially produced manual that provides lessons plans, facilitator notes, ideas
to facilitate the curriculum, and long-term planning schedules. Groups are conducted as closed
groups, meaning once a group commenced, no new youth can be added to the group.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Annual review staff was able to observe a group during the annual review. The group consisted
of ten youth in a classroom setting. The group was facilitated by two youth care workers. The
facilitator read the script to the youth, but also was able to stop from the script and add clarity and
check for understanding by the youth. No behavior issues were observed during the group. The
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
facilitators role-played the new skill, and each youth was given the opportunity in pairs to practice
the skill. Homework was also assigned at the conclusion of the group.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
According to documentation provided by the program, there was one gap in service for greater
than one month due to facilitator turnover. In accordance with the program's policy, another
trained staff member took over the group and the youth completed all required sessions.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
According to documentation provided by the program, internal fidelity monitoring was occurring on
a monthly basis for each facilitator.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a policy and procedure in place for corrective actions. If during the monthly
internal fidelity monitoring, a facilitator is found to be in need of skill improvement, a plan is put
into place and follow up monitoring confirms the improvement in facilitator skills.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of annual performance evaluations for the staff providing the primary service revealed
the evaluations documented the staff as facilitators of the primary service. Additionally, the
evaluations rated the staff in their skill as facilitators in the primary service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 15 weeks for this type of service. Of the 30 youth in the
sample, 63% (19 of 30) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 45 contact hours for this type of service. Of the 30 youth in the
sample, 0% (0 of 30) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 1 youth
Moderate-High = 10 youth
High = 19 youth
Total Youth in Sample = 30 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 63%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 19 youth
Total Youth in Sample = 30 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (30 of 30) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 63% (19 of 30) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 25 25 100%
Totals 86 100 86%
This SPEP report evaluates Thinking for a Change, an intervention delivered at Miami Youth
Academy.
31 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 6 points for Amount of Service: Duration. Of the 30 total youth sampled, only
21 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 19
received at least the recommended weeks of service. Youth in the sample completed between 13
and 39 weeks of service, with an average of 23 weeks.
The program earned 0 points for Amount of Service: Contact Hours. Of the 30 total youth
sampled, all 30 included dosage in the EBS Module. Of those youth, 0 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
recommended hours of service. Youth in the sample completed between 4 and 25 hours of
service, with an average of 16 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Miami Youth Academy can optimize their SPEP Quality of Service Delivery score by ensuring
facilitator turnover does not impact groups.
Miami Youth Academy can optimize their SPEP Amount of Service score by ensuring that dosage
for all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8

service_type: Thinking for a Change


summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Miami Youth Academy
TrueCore Behavioral Solutions, LLC
(Contract Provider)
10855 SW 84th Street
Miami‐Dade, Florida 33173
Primary Service: Impact of Crime
SPEP Review Date(s): November 19‐20 , 2019
Florida Department of Juvenile Justice
Report Date(s): November 25, 2019
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
 Service Type: service type points are awarded based on the group category that the
service falls within
 Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
 Amount of Service: or “dosage” refers to the amount and duration of services that were
received
 Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime. The program was awarded 15 points
because the Primary Service is identified as a Group 3 Service. The specific Sub-Component
Service Type identified is Social Skills Training. The Primary Service was identified as this type of
service as it focuses on developing social skills required for an individual to interact in a positive
way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The SPEP review encompasses twelve (12) months of records examinations. According to
documentation provided by the program, the program currently has two staff members who are
trained in the primary service, Impact of Crime.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the curriculum Impact of Crime: Addressing the Harm to Victims and the
Community. The curriculum was updated in 2009. It is highly detailed providing a facilitator's
manual, and student workbook. The facilitator's manual provides lessons plans, classroom
management ideas, and tips for engaging youth discussions. The student manual provides
reproducible worksheets to help facilitate group discussions, and review concepts learned during
the group sessions. Homework materials are also utilized in the student manual to re-enforce
learned concepts. Impact of Crime is a closed group meaning once a group commences, new
youth cannot be added to the group.
Impact of Crime is based on the belief that crime is more than just a legal definition, but rather
crime affects the victim, the offender, their families, and the community. The harm caused brings
with it the moral responsibility to all involved. The focus of Impact of Crime is to help offenders
understand the harm they have created, and then assisting them in taking personal accountability
for their actions. Together, the victim, offender, and the community arrive at a viable solution to
the repair the harm caused and the offender. Once the harm is repaired, the offender
subsequently reintegrates back into the community. Facilitators are encouraged to bring in victim
impact speakers throughout the course of the curriculum.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A cohort of the primary service had just been completed, and a new cohort is being planned, but
has not yet begun. At this time, no sessions of the primary service were being facilitated,
therefore, a group session could not be observed. The Clinical Director did state and provide
documentation in the Evidence-Based Services module of the Juvenile Justice Information
System a new cohort was beginning Sunday, November 24, 2019 with ten youth.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
In reviewing documentation and informal interview with clinical director, it was determined no
facilitator turnover occurred during the last year which affected delivery of Impact of Crime (IOC).
IOC groups were being facilitated in eleven of twelve months during the review period. In the
event of a facilitator leaving during a cohort, The assistant facility administrator would fill in until
new staff could be identified and trained.
According to documentation provided by the program, three cohorts of the primary service were
completed during the last twelve months.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program provided documentation of internal fidelity monitoring taking place monthly for each
facilitator. The internal fidelity monitoring was completed by someone trained in Impact of Crime,
and the facilitator and monitor both signed the form to show the results had been reviewed with
the facilitator.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
During the review period, no corrective action plans were initated. An informal conversation with
clinical director did reveal that in the event of a need for improvement of the facilitator skills, the
clinical director and facilitator would sit down and work to create a plan for improvement.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
According to an informal interview with the facility administrator, the current annual employee
evaluations are in progress and not currently completed. The previous annual performance
summaries did show that facilitators were rated on their skills in delivery of Impact of Crime. The
company's annual employee evaluations have a section dedicated to rate the skills of employees
facilitating groups. Company practice is not only to rate the employee, but also list the name of
group(s) the employee is facilitating.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service ‐ Duration
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates the target duration of 16 weeks for this type of service. Of the 19 youth in the
sample, 42% (8 of 19) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates a target of 24 contact hours for this type of service. Of the 19 youth in the
sample, 53% (10 of 19) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 17 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 68%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 95%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 7 youth
Moderate-High = 7 youth
High = 4 youth
Total Youth in Sample = 19 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 21%
High Score: 5 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 38%
Table 2
High = 4 youth
Total Youth in Sample = 19 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 95% (18 of 19) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 21% (4 of 19) youth scored High-Risk to reoffend, for a score of 5 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 4 10 40%
Amount of Service: Contact Hours 4 10 40%
Risk Level of Youth Served 17 25 68%
Totals 65 85 76%
This SPEP report evaluates Impact of Crime, an intervention delivered at Miami Youth Academy.
28 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 4 points for Amount of Service: Duration. Of the 19 total youth sampled, only
14 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 8
received at least the recommended weeks of service. Youth in the sample completed between 13
and 22 weeks of service, with an average of 16 weeks.
The program earned 4 points for Amount of Service: Contact Hours. Of the 19 total youth
sampled, all 19 included dosage in the EBS Module. Of those youth, 10 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
recommended hours of service. Youth in the sample completed between 9 and 26 hours of
service, with an average of 19 hours.
The program was awarded 17 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 86%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Impact of Crime is an intervention suitable for all youth. The Department recommends providing
this service to every youth in the program.
Miami Youth Academy can maintain their SPEP Quality of Service Delivery score by continuing
the practices in place at the time of this review.
Miami Youth Academy can optimize their SPEP Amount of Service score by ensuring that dosage
for all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9


summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Miami Youth Academy
TrueCore Behavioral Solutions, LLC
(Contract Provider)
10855 SW 84th Street
Miami‐Dade, Florida 33173
Primary Service: Pathways to Self‐Discovery and Change
SPEP Review Date(s): November 19‐20, 2019
Florida Department of Juvenile Justice
Report Date(s): November 25, 2019
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
 Service Type: service type points are awarded based on the group category that the
service falls within
 Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
 Amount of Service: or “dosage” refers to the amount and duration of services that were
received
 Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Pathways to Self-Discovery and Change.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The Standardize Program Evaluation Protocal (SPEP) review involves a review of records and
documents over a twelve month period. The program is currently utilizing the primary service,
Pathways Self Discovery and Change. The focus of the primary service is substance abuse. The
curriculum strives to help youth learn to make different, and hopefully better, choices about
substances in their lives. The curriculum is facilitated by master and doctoral level therapists at
the program. The program provided documentation of the facilitators being trained by a doctor's
level therapist familiar with the curriculum, and having had experience in facilitating as well. The
facilitators did practice teaching a session of the curriculum to a master trainer, and
documentation showed the facilitators completed the training successfully.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes a highly detailed, commercially produced curriculum Pathways Self
Discovery and Change which provides lessons plans, student hand-outs, facilitator notes, ideas to
facilitate the curriculum, and long-term planning schedules. The curriculum has both a student,
and facilitator guide. The program identifies the curriculum as the most sophisticated primary
service utilized in the program. It is imparative for facilitators to be prepared for each group
session, to achieve maximum outcomes. The curriculum is at times difficult to facilitate as
facilitators must refer back and forth between facilitator and student books for delivery of content.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Pathways Self Discovery and Change is delivered on Thursdays and Sundays. A group session
of the primary service was not conducted during the time of the annual review. The primary
service will be observed during future quarterly reviews.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
Documentation provided by the program showed no gaps in the delivery of Pathways Self
Discovery and Change caused by facilitator turnover. An informal conversation with the clinical
director revealed that if a facilitator leaves the program, another trained therapist or the clinical
director would step in and facilitate until a new therapist is hired.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of records provided by the program showed internal fidelity monitoring was being
completed during some months during the last year. A total of seven of the twelve months had
internal fidelity monitoring completed.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
A review of the internal fidelity monitoring revealed no corrective actions were noted nor taken.
An informal interview with the clinical director revealed the program has a policy for instituting
corrective actions to facilitators who need to improve their skills.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The annual performance evaluations of the facilitators of Pathways Self Discovery and Change
were reviewed. The evaluations measured the facilitator's skill in delivery of the primary service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service ‐ Duration
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates the target duration of 24 weeks for this type of service. Of the 28 youth in the
sample, 36% (10 of 28) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates a target of 40 contact hours for this type of service. Of the 28 youth in the
sample, 36% (10 of 28) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 18 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 72%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 89%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 8 youth
Moderate-High = 10 youth
High = 7 youth
Total Youth in Sample = 28 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 25%
High Score: 8 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 62%
Table 2
High = 7 youth
Total Youth in Sample = 28 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 89% (25 of 28) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 25% (7 of 28) youth scored High-Risk to reoffend, for a score of 8 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 2 10 20%
Amount of Service: Contact Hours 2 10 20%
Risk Level of Youth Served 18 25 72%
Totals 72 95 76%
This SPEP report evaluates Pathways to Self-Discovery and Change, an intervention delivered at
Miami Youth Academy.
28 youth completed the program during the review period. Of those youth, 28 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 2 points for Amount of Service: Duration. Of the 28 total youth sampled, only
14 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 10
received at least the recommended weeks of service. Youth in the sample completed between 18
and 45 weeks of service, with an average of 26 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
The program earned 2 points for Amount of Service: Contact Hours. Of the 28 total youth
sampled, only 15 included dosage in the EBS Module. Of those youth with correct dosage, 10
received at least the recommended hours of service. Youth in the sample completed between 28
and 42 hours of service, with an average of 38 hours.
The program was awarded 18 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 83%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Miami Youth Academy can optimize their SPEP Quality of Service Delivery score by ensuring
fidelity monitoring is completed every month for each facilitator.
Miami Youth Academy can optimize their SPEP Amount of Service score by ensuring that dosage
for all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8