summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Marion Youth Academy
Sequel Youth and Family Services
(Contract Provider)
10420 NW Gainesville Road
Ocala, Florida 34482
Primary Service: Skillstreaming the Adolescent
SPEP Review Period: April 1, 2021-March 31, 2022
SPEP Review Conducted: April 6-7, 2021
Florida Department of Juvenile Justice
Report Date: 7/27/22
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Skillstreaming the Adolescent.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A review of documentation for the annual review period of April 1, 2021 through March 31, 2022
showed there were five staff trained to facilitate Skillstreaming the Adolescent. The trained staff
are the clinical director and four therapists. All five trained staff facilitated the primary service
during the annual review period. The program has a written training protocol for the Skillstreaming
service. The training is a four-hour in-service training conducted by the clinical director or regional
clinical director. The training is broken down into three parts. The first part is the introduction to
Skillstreaming and is one hour in length. The second section is the review of materials and is two
hours in length. The third section is the observation by the clinical director of the trained staff
conducting a group session. The clinical director conducted all internal fidelity checks for the
annual review period. All trained staff members can conduct the internal fidelity checks.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilized the Skillstreaming the Adolescent 3rd edition. The manual introduces
Skillstreaming, contains scripts for the facilitators to follow as well as specific instructions for each
skill taught, example role plays, and student exercises. There are no separate student manuals.
The facilitator makes copies of the skill cards for each skill and hands out those copies to the
youth in the group. There are fifty skills taught to the youth.
Skillstreaming is run on Wednesdays as an open group, with groups being run year-round. The
program maintains the attendance sheets and monthly internal fidelity checks in the SPEP binder
titled "Skillstreaming". According to the clinical director, the program decides which youth receives
the intervention by first determining if the youth has previously participated in the completion of
the required dosage of services. If the youth has not, the youth is automatically included in the
next incoming cohort. If the youth has completed the required dosage, but continues to struggle
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
with personal accountability for their actions, they can be readmitted into the next incoming
cohort.
The program designated Skillstreaming the Adolescent as a primary service on 8/16/2021.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Skillstreaming the Adolescent was observed on 4/6/2022 via Microsoft Teams. There
were ten (10) youth present for the group along with the facilitator and one floor staff. The group
began at 3pm and was held in the cafeteria. The facilitator was knowledgeable of the material and
followed the lesson plan. The youth participated during the group. The youth were role playing
scenarios for Standing Up For Your Rights. Several youth’s role plays were not appropriate in that
the youth seemed more confrontational instead of using/demonstrating the skill appropriately. The
facilitator did not redirect the youth or have the youth restart the role play. The facilitator was
prepared for the group. The facilitator asked questions to the youth to ensure the youth
understood the concepts before moving on to another topic. The group ended at 3:55p.
The technical assistance specialist spoke with the clinical director and the facilitator about making
sure the youth were role playing the skill correctly. The technical assistance specialist also told
the facilitator she should have made sure the youth were assigned a skill step to observe to
ensure the youth could identify each step in the skill being presented.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation for the annual review period of April 1, 2021 through March 31, 2022
showed there was turnover of Skillstreaming the Adolescent facilitators. One trained therapist left
the program, which negatively affected delivery of the primary service. The therapist was
facilitating the primary service at the time she left the program. The program transitioned the
youth over to another trained therapist. There were no gaps in service.
The clinical director stated the program implements a co-facilitator guideline in which there are
two facilitators facilitating each group, so in the event one of the facilitators is out for any reason,
the group can continue. Therapists are also trained in the primary service in the event that there
are no case managers available to facilitate a group.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of documentation for the annual review period of April 1, 2021 through March 31, 2022
showed the program completed eighteen (18) fidelity monitorings for facilitators of Skillstreaming
the Adolescent. The program did not have any fidelity monitorings completed for Skillstreaming
during the months of April, May, June, or July 2021 because the service was not designated as a
SPEP service until 8/16/2021. The program was missing internal fidelity monitorings for the
clinical director for the months of November 2021 and March 2022. The clinical director conducted
the monthly internal fidelity checks during the annual review period. The clinical director stated the
program conducts internal fidelity monitoring on a monthly basis. Facilitators trained must be
monitored by another trained and experience facilitator.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
A review of documentation for the annual review period showed there were no corrective actions
noted on the fidelity monitorings of Skillstreaming the Adolescent facilitators.
The program's policy for corrective actions noted on the fidelity monitoring is the director of case
management will sit down with the facilitator to go over the feedback. The director of case
management will provide written or verbal strategies and goals with the facilitator. The director of
case management will increase the fidelity monitoring to weekly, if needed, until the corrective
action is ameliorated.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of documentation for the annual review period of April 1, 2021 through March 31, 2022
showed all evaluations reviewed have an area which addresses the facilitator's skill in delivering
Skillstreaming the Adolescent. Three therapists' annual performance evaluations were reviewed.
All three annual performance evaluations contained specific information documenting the
facilitator’s skill delivering the primary service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates the target duration of 16 weeks for this type of service. Of the 19 youth in the
sample, 53% (10 of 19) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates a target of 24 contact hours for this type of service. Of the 19 youth in the
sample, 47% (9 of 19) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 89%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 2 youth
Moderate-High = 6 youth
High = 9 youth
Total Youth in Sample = 19 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 47%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 9 youth
Total Youth in Sample = 19 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 89% (17 of 19) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 47% (9 of 19) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 4 10 40%
Amount of Service: Contact Hours 4 10 40%
Risk Level of Youth Served 23 25 92%
Totals 71 85 84%
This SPEP report evaluates Skillstreaming the Adolescent, an intervention delivered at Marion
Youth Academy.
50 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 4 points for Amount of Service: Duration. Of the 19 total youth sampled, only
16 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 10
received at least the recommended weeks of service. Youth in the sample completed between 3
and 31 weeks of service, with an average of 16 weeks.
The program earned 4 points for Amount of Service: Contact Hours. Of the 19 total youth
sampled, all 19 included dosage in the EBS Module. Of those youth, 9 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
recommended hours of service. Youth in the sample completed between 3 and 31 hours of
service, with an average of 18 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
RECOMMENDATION(S):
Marion Youth Academy can optimize their SPEP Quality of Service Delivery score by ensuring
facilitator turnover does not impact groups and fidelity monitoring is completed every month for
each facilitator.
Marion Youth Academy can optimize their SPEP Amount of Service score by ensuring dosage for
all youth is recorded accurately in EBS and by ensuring youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2021-22 Marion Skillstreaming.pdf

service_type: Skillstreaming the Adolescent


state: FL

facility_name: Marion Regional Juvenile Detention Center

summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Marion Youth Academy
Sequel Youth and Family Services
(Contract Provider)
10420 NW Gainesville Road
Ocala, Florida 34482
Primary Service: Thinking for a Change
SPEP Review Period: April 1, 2021-March 31, 2022
SPEP Review Conducted: April 6-7, 2021
Florida Department of Juvenile Justice
Report Date: 7/27/22
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
A review of documentation for the annual review period of April 1, 2021 through March 31, 2022
showed there were six staff trained to facilitate Thinking for a Change (T4C)- the director of case
management, clinical director, three case managers, and the transition specialist. Two case
managers and the transition specialist facilitated or co-facilitated a cohort of T4C during the
annual review period. T4C training is four-days, conducted by a qualified trainer. The director of
case management conducted the internal fidelity checks for the annual review period. All trained
staff members can conduct the internal fidelity checks.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the Thinking for a Change 4.0 manual. The manual includes Twenty-five (25)
lessons. Each lesson has a script for the facilitator and instructions for each activity. The manual
requires that the lessons be given in order and provides specific instructions for the
implementation of the program. There are no separate student workbooks. The facilitators make
copies of the current lessons to hand out to the youth.
T4C is delivered a closed group with no more than twelve (12) youth in each cohort. The
curriculum is to be delivered to the youth over a minimum of fifteen (15) weeks for forty-five (45)
hours. The program keeps all SPEP documents in a binder labeled "Thinking for a Change". The
program provides the service on Mondays, Wednesdays, and Fridays for one hour each. All youth
receive Thinking for a Change. According to the clinical director, the program decides which youth
receives the intervention by first determining if the youth has previously participated in the
completion of the required dosage of services. If the youth has not, the youth is automatically
included in the next incoming cohort. If the youth has completed the required dosage, but
continues to struggle with personal accountability for their actions, they can be readmitted into the
next incoming cohort.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Thinking for a Change was observed on 4/6/2022 via Microsoft Teams. Present were
twelve (12) youth, the facilitator, and one floor staff. The group began at 11:32am and was held in
the cafeteria. The facilitator was knowledgeable of the material and followed the lesson plan. The
youth participated during the group. The facilitator went over the second part of chapter one.
Chapter one introduces the youth to Thinking for a Change and explains what the youth will be
doing during the group. The facilitator was prepared for the group. The facilitator ensured the
youth understood the concepts before moving on to another topic. The group ended at 12:15p.
The technical assistance specialist spoke with the clinical director about the group not going for a
full hour. All groups need to be run for at least one hour. The program cannot put one hour into
the Evidence-Based Services module (EBS) if the group was not run for a full hour. The clinical
director stated she will speak with the facilitator and ensure she runs group for the full hour.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation for the annual review period of April 1, 2021 through March 31, 2022
showed there was turnover of Thinking for a Change (T4C) facilitators. One trained case manager
left the program and came back. The turnover did not negatively affect delivery of T4C as the
case manager was not running a cohort at the time he left the program. There were no gaps in
service.
There were four cohorts completed during the annual review period. The dates of the cohorts are:
2/23/2021-6/2/2021, 3/17/2021-7/26/2021, 8/11/2021-11/8/2021, and 11/15/2021-3/30/2022.
The 2/23/2021 cohort began with nine youth and ended with six youth.
The 3/17/2021 cohort began with ten (10) youth and ended with seven youth. There were two
youth added to the cohort on 3/19/2021. Four youth were released during the cohort. The dates
the youth were released were: 4/20/2021, 5/3/2021, 5/21/2021, and 6/3/2021. One youth received
new charges on 3/21/2021.
The 8/11/2021 cohort began with ten (10) youth and ended with five youth. Two youth were added
on 8/12/2021. Three youth refused to attend the group and were dropped from the cohort on
8/30/2021. One youth was released on 9/9/2021, one youth was released on 9/17/2021, one
youth was released on 9/30/2021, and one youth was released on 10/11/2021.
The 11/15/2021 cohort began with twelve (12) youth and ended with seven youth. Three youth
were arrested (two on 1/1/2022 and one on 2/25/2022). One youth was released on 1/5/2022.
Another youth was released, but the attendance sheets did not document the date of release.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The clinical director stated the program implements a co-facilitator guideline in which there are
two facilitators facilitating each group, so in the event one of the facilitators is out for any reason,
the group can continue. Therapists are also trained in the primary service in the event there are
no case managers available to facilitate a group.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of documentation for the annual review period of April 1, 2021 through March 31, 2022
showed the program completed fifteen (15) fidelity monitorings of Thinking for a Change. The
program was not missing any internal fidelity monitorings for the annual review period. The
director of case management conducted the monthly internal fidelity checks during the annual
review period. The clinical director stated the program conducts internal fidelity monitoring on a
monthly basis. Facilitators trained must be monitored by another trained and experience
facilitator.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
A review of documentation for the annual review period showed there were no corrective actions
noted on the fidelity monitoring of Thinking for a Change. The program's policy for corrective
actions noted on the fidelity monitoring is the director of case management will sit down with the
facilitator to go over the feedback. The director of case management will provide written or verbal
strategies and goals with the facilitator. The director of case management will increase the fidelity
monitoring to weekly, if needed, until the corrective action is ameliorated.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of documentation for the annual review period of April 1, 2021 through March 31, 2022
showed all evaluations reviewed have an area which addresses the facilitator's skill in delivering
Thinking for a Change (T4C). Three annual performance evaluations were provided for review.
One of the three annual performance evaluations contained specific information documenting the
facilitator’s skill delivering T4C. The other two evaluations did not contain specific information
regarding the facilitator’s skill delivering the primary service.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates the target duration of 15 weeks for this type of service. Of the 49 youth in the
sample, 43% (21 of 49) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 45 contact hours for this type of service. Of the 49 youth in the
sample, 16% (8 of 49) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 86%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 6 youth
Moderate-High = 18 youth
High = 18 youth
Total Youth in Sample = 49 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 37%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 18 youth
Total Youth in Sample = 49 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 86% (42 of 49) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 37% (18 of 49) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 4 10 40%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 23 25 92%
Totals 82 100 82%
This SPEP report evaluates Thinking for a Change, an intervention delivered at Marion Youth
Academy.
50 youth completed the program during the review period. Of those youth, 49 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 4 points for Amount of Service: Duration. Of the 49 total youth sampled, only
38 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 21
received at least the recommended weeks of service. Youth in the sample completed between 5
and 25 weeks of service, with an average of 16 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 0 points for Amount of Service: Contact Hours. Of the 49 total youth
sampled, only 39 included dosage in the EBS Module. Of those youth with correct dosage, 8
received at least the recommended hours of service. Youth in the sample completed between 9
and 61 hours of service, with an average of 36 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
RECOMMENDATION(S):
Marion Youth Academy can optimize their SPEP Quality of Service Delivery score by ensuring
each facilitator’s performance evaluation includes discussion of their skill in delivering this primary
service.
Marion Youth Academy can optimize their SPEP Amount of Service score by ensuring dosage for
all youth is recorded accurately in EBS and by ensuring youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

source: DJJ

report_type: SPEP

file_name: FY2021-22 Marion T4C.pdf

program_name_raw: Marion Youth Academy

program_status_note: Closed

service_type: Thinking for a Change