summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Broward Youth Treatment Center
Youth Opportunity Investments, LLC
(Contract Provider)
8301 South Palm Drive
Pembroke Pines, Florida 33025
Primary Service: Impact of Crime
SPEP Review Period: November 1, 2020-October 31, 2021
SPEP Review Conducted: November 10, 2021
Florida Department of Juvenile Justice
Report Date: December 15, 2021
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The Standardized Program Evaluation Protocol (SPEP) review encompasses twelve (12) months
of records. Impact of Crime is based on the belief that crime is more than just a legal definition,
but rather crime affects the victim, the offender, their families, and the community. The harm
caused brings with it the moral responsibility to all involved. The focus of Impact of Crime is to
help offenders understand the harm they have created, and then assisting them in taking personal
accountability for their actions. Together, the victim, offender, and the community arrive at a viable
solution to the repair the harm caused and the offender. Once the harm is repaired, the offender
subsequently reintegrates back into the community. Facilitators are encouraged to bring in victim
impact speakers throughout the course of the curriculum.
Currently, the program has four staff members trained in Impact of Crime. A case manager and
transitional services manager are facilitating groups. The director of case management and group
leader are also trained and provide monthly internal fidelity monitoring. They also act as back up
facilitators if a facilitator cannot complete a cohort.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes curriculum Impact of Crime: Addressing the Harm to Victims and the
Community, published in 2009. It is highly detailed, providing a facilitator's manual and student
workbook. The facilitator's manual provides lesson plans, classroom management ideas, and tips
for engaging youth discussions. The student manual provides reproducible worksheets to help
facilitate group discussions, and review concepts learned during the group sessions. Homework
materials are also utilized in the student manual to reinforce learned concepts. The primary
service is facilitated as a closed group, meaning once a cohort begins, no new youth can be
added to the group.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
During the annual review, a group session of Impact of Crime was observed. The facilitator did
not read a script word for word, but generalized the information and added her own words to
provide meaning to covered concepts. The youth were generally engaged and participated in the
discussion. No behavioral issues were observed during the group.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There were no gaps in service delivery of Impact of Crime (IOC) during the past year. In reviewing
documentation and an informal interview with both the director of case management and the
facility administrator, a case manager was dismissed from their position in the middle of a cohort.
This was the cohort being delivered from March 2, 2021 to June 17, 2021. The director of case
management, who is trained in IOC, took over facilitation and completed the cohort.
The program has completed three cohorts and a two are on-going at the time of the review. The
first cohort started October 6, 2020 and ended February 18, 2021. It began with ten (10) youth
and ended with eight youth.
Two cohorts started on March 2, 2021. One started with seven youth and ended on June 17,
2021, with four youth. The second started with eight youth and ended on June 24, 2021, with five
youth.
A fourth cohort started on July 15, 2021 and was on going at the time of the review. It started with
ten (10) youth and currently has five youth. A fifth cohort is also on going. It began October 5,
2021 with eight youth and has eight youth at time of review.
According to the Case Management Director, all but one youth who did not complete the service
were released prior to the end of group. In each case, the youth were provided the service on an
accelerated, individual schedule to receive all material prior to discharge.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
In reviewing twelve (12) months of records, the program has provided internal fidelity monitoring.
Additionally, the internal fidelity monitoring was completed by staff member trained in Impact of
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Crime. The facilitator also signed the form saying feedback was given to the facilitator and
understood by the facilitator.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
In reviewing the twelve months of fidelity monitoring documentation, no need for corrective actions
were documented for facilitators of Impact of Crime. The director of case management stated
during an informal interview a procedure does exist of corrective actions if a facilitator is found to
need additional support.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The annual performance evaluations were reviewed for facilitators of Impact of Crime. Those
evaluations showed all facilitators who delivered the primary service were rated on their
proficiency and given credit on evaluations.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 16 weeks for this type of service. Of the 32 youth in the
sample, 78% (25 of 32) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 24 contact hours for this type of service. Of the 32 youth in the
sample, 94% (30 of 32) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 84%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 3 youth
Moderate-High = 11 youth
High = 13 youth
Total Youth in Sample = 32 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 41%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 13 youth
Total Youth in Sample = 32 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 84% (27 of 32) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 41% (13 of 32) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 20 25 80%
Totals 74 85 87%
This SPEP report evaluates Impact of Crime, an intervention delivered at Broward Youth
Treatment Center.
32 youth completed the program during the review period. Of those youth, all 32 were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 6 points for Amount of Service: Duration. Of the 32 total youth sampled, only
26 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 25
received at least the recommended weeks of service. Youth in the sample completed between 15
and 20 weeks of service, with an average of 18 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 8 points for Amount of Service: Contact Hours. Of the 32 total youth
sampled, only 30 included dosage in the EBS Module. Of those youth with correct dosage, 30
received at least the recommended hours of service. Youth in the sample completed between 30
and 33 hours of service, with an average of 32 hours.
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 93%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Broward Youth Treatment Center can optimize their SPEP Quality of Service Delivery score by
ensuring facilitator turnover does not impact groups.
Broward Youth Treatment Center can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2021-22 BYTC IOC.pdf


state: FL

facility_name: Broward Youth Treatment Center

summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Broward Youth Treatment Center
Youth Opportunity Investments, LLC
(Contract Provider)
8301 South Palm Drive
Pembroke Pines, Florida 33025
Primary Service: Living in Balance
SPEP Review Period: November 1, 2020-October 31, 2021
SPEP Review Conducted: November 10, 2021
Florida Department of Juvenile Justice
Report Date: December 15, 2021
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Living in Balance.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The Standardized Program Evaluation Protocol (SPEP) annual review encompasses twelve
months of records. Living in Balance is a research-based substance abuse treatment intended to
help clients address issues in lifestyle areas which may have been neglected during addiction. It
is specifically oriented for group settings. The manual uses didactic education and instruction,
group process interaction through role plays and discussion, daily relaxation and visualization
exerciese, informational handouts, videotapes, and group-oriented recreational therapy exercises.
According to training records maintained by the program, two therapists are currently facilitating
Living in Balance and regional clinical director is providing internal fidelity monitoring.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
Living in Balance is facilitated three days per week on Friday, Saturday, and Sunday. All youth
entering the program are enrolled in the primary service. Living in Balance has a commercially
produced manual which provides lesson plans, facilitator notes, ideas to facilitate the curriculum,
and long-term planning schedules.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Living in Balance was observed virtually. The group took place in a classroom setting
and was attended by six youth, the facilitator and one direct care specialist. The facilitator led a
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
group discussion and then the youth participated in role play activity. No behavioral issues were
noted during the session, and the youth appeared to be engaged through out.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
According to documentation provided by the program, there was turnover with the facilitators for
Living in Balance. Turnover occurred in both the second and third quarters of the year. When one
therapist leaves the program, another therapist covers the ongoing cohort until new staff are
hired. The program consistently runs two separate cohorts of Living in Balance. Although turnover
occurred, no gap in service occurred during the review period.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
According to documentation provided by the program, monthly internal fidelity monitoring is taking
place for facilitators of Living in Balance. The monitoring is being completed by supervisory staff
trained in the primay service. The program does not currently have an on-site clinical director; the
regional clinical director is assisting with internal fidelity monitoring.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a policy in place to ensure therapeutic services are delivered to the youth in a
manner as dictated by the curriculum. No corrective actions were documented as being needed
for facilitators of Living in Balance. In the event of a need for corrective actions, the clinical
director would conduct a conference with therapist(s) to clearly delinate needed areas of
improvement, and work with therapist(s) to develop a plan to address facilitator issues.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of performance evaluation documents confirmed the facilitators were acknowledged on
their proficiency in delivering Living in Balance. This included feedback and recommendations for
the facilitators.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 24 weeks for this type of service. Of the 32 youth in the
sample, 100% (32 of 32) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 40 contact hours for this type of service. Of the 32 youth in the
sample, 100% (32 of 32) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 84%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 3 youth
Moderate-High = 11 youth
High = 13 youth
Total Youth in Sample = 32 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 41%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 13 youth
Total Youth in Sample = 32 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 84% (27 of 32) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 41% (13 of 32) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 20 25 80%
Totals 90 95 95%
This SPEP report evaluates Living in Balance, an intervention delivered at Broward Youth
Treatment Center.
32 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 32 total youth sampled, all
32 included dosage with end dates in the EBS Module. Of those youth, 32 received at least the
recommended weeks of service. Youth in the sample completed between 25 and 53 weeks of
service, with an average of 34 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 32 total youth
sampled, all 32 included dosage in the EBS Module. Of those youth, 32 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
recommended hours of service. Youth in the sample completed between 77 and 153 hours of
service, with an average of 98 hours.
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 100%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Broward Youth Treatment Center can optimize their SPEP Quality of Service Delivery score by
ensuring facilitator turnover does not impact groups.
Broward Youth Treatment Center can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8

source: DJJ

report_type: SPEP

file_name: FY2021-22 BYTC LIB.pdf

program_name_raw: Broward Youth Treatment Center - YOI

program_status_note: Closed

service_type: Living in Balance


summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Broward Youth Treatment Center
Youth Opportunities Investments, LLC
(Contract Provider)
8301 South Palm Drive
Pembroke Pines, Florida 33025
Primary Service: Impact of Crime
SPEP Review Period: November 1, 2021-October 31, 2022
SPEP Review Conducted: November 2, 2022
Florida Department of Juvenile Justice
Report Date: 2/2/23
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The Standardized Program Evaluation Protocol (SPEP) review encompasses twelve (12) months
of records. Impact of Crime (IOC) is based on the belief that crime is more than just a legal
definition, but rather crime affects the victim, the offender, their families, and the community. The
harm caused brings with it the moral responsibility to all involved. The focus of Impact of Crime is
to help offenders understand the harm they have created, and then assisting them in taking
personal accountability for their actions. Together, the victim, offender, and the community arrive
at a viable solution to the repair the harm caused and the offender. Once the harm is repaired, the
offender subsequently reintegrates back into the community. Facilitators are encouraged to bring
in victim impact speakers throughout the course of the curriculum.
Currently, the program has four staff members trained in IOC. A case manager and transitional
services manager are facilitating groups; the director of case management and group leader are
trained and provide monthly internal fidelity monitoring. They also act as back up facilitators if a
facilitator cannot complete a cohort.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the curriculum Impact of Crime: Addressing the Harm to Victims and the
Community. The curriculum was published in 2009. It is highly detailed, providing a facilitator's
manual and student workbook. The facilitator's manual provides lessons plans, classroom
management ideas, and tips for engaging youth discussions. The student manual provides
reproducible worksheets to help facilitate group discussions, and review concepts learned during
the group sessions. Homework materials are also utilized in the student manual to re-enforce
learned concepts.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Impact of Crime was observed during the review period. The group was held in a
classroom of the program. Ten (10) youth, one direct care staff and the facilitator were present.
The lesson was the first unit in Chapter 6, Conflict Resolution. The facilitator had a copy of the
facilitators guide open as a reference during the group. The youth took turns reading the key
words and the definitions that would be used in the chapter. The facilitator then led a large group
discussion of the youth's views of how to deal with conflict in a non-violent, appropriate manner.
The youth were engaged in the conversation and no disciplinary issues were observed during the
one-hour group.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
According to documentation provided and an informal interview with director of case
management, no facilitator turnover occurred which adversely effected delivery of Impact of Crime
(IOC). A trained facilitator left during the past year; however, it was between groups and did not
affect service delivery.
The program completed three cohorts of IOC and a fourth was ongoing at the time of the annual
review. The first cohort began on October 5, 2021, with ten (10) youth and was completed on
February 17, 2022, with seven youth. Three youth completed the program prior to completion of
the service. The second cohort began March 29, 2022, with ten (10) youth and was completed on
July 14, 2022, with ten (10) youth. A third cohort began on June 2, 2022, with ten (10) youth and
was completed on September 26, 2022, with five youth. Five youth were released prior to
completing the service . A fourth cohort began on August 16, 2022, with ten (10) youth and was
ongoing at the time of the review.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Internal fidelity monitoring took place monthly for facilitators of Impact of Crime. Each person
facilitating group was observed and the monitoring was conducted by staff member trained in the
primary service. Both the director of case management and direct care staff supervisor are trained
in the primary service and provided internal monitoring.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process in place to address corrective actions resulting from monthly internal
fidelity monitoring of Impact of Crime facilitators. A review of documentation revealed no
corrective actions were initated during the review period for facilitators of Impact of Crime. The
facilitators were found to be delivering the primary service in an acceptable manner.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A copy of the annual employee evaluations were provided as part of the review. Each employee
who facilitated Impact of Crime was rated in their effectiveness in service delivery.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 16 weeks for this type of service. Of the 46 youth in the
sample, 72% (33 of 46) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 24 contact hours for this type of service. Of the 46 youth in the
sample, 89% (41 of 46) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 85%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 7 youth
Moderate-High = 18 youth
High = 14 youth
Total Youth in Sample = 46 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 30%
High Score: 10 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 77%
Table 2
High = 14 youth
Total Youth in Sample = 46 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 85% (39 of 46) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 30% (14 of 46) youth scored High-Risk to reoffend, for a score of 10 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 20 25 80%
Totals 74 85 87%
This SPEP report evaluates Impact of Crime, an intervention delivered at Broward Youth
Treatment Center.
47 youth completed the program during the review period. Of those youth, 46 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 6 points for Amount of Service: Duration. Of the 46 total youth sampled, all
46 included dosage with end dates in the EBS Module. Of those youth, 33 received at least the
recommended weeks of service. Youth in the sample completed between 12 and 20 weeks of
service, with an average of 16 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 8 points for Amount of Service: Contact Hours. Of the 46 total youth
sampled, all 46 included dosage in the EBS Module. Of those youth, 41 received at least the
recommended hours of service. Youth in the sample completed between 22 and 35 hours of
service, with an average of 31 hours.
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 93%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Broward Youth Treatment Center can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Broward Youth Treatment Center can optimize their SPEP Amount of Service score by ensuring
dosage for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2022-23 BYTC IOC.pdf

service_type: Impact of Crime


summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Broward Youth Treament Center
Youth Opportunities Investments, LLC
(Contract Provider)
8301 South Palm Drive
Pebroke Pines, Florida 33025
Primary Service: Living in Balance
SPEP Review Period: November 1, 2021-October 31, 2022
SPEP Review Conducted: November 2, 2022
Florida Department of Juvenile Justice
Report Date: 2/2/23
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Living in Balance.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The SPEP annual review encompasses twelve (12) months of records. According to training
records maintained by the program, three therapists and the clinical director are trained in Living
in Balance. The three therapists facilitate groups and the clinical director provides internal fidelity
monitoring.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
Living in Balance has a commercially-produced manual which includes lessons plans, facilitator
notes, ideas to facilitate the curriculum, and long-term planning schedules. Living in Balance is a
research-based substance abuse treatment intended to help clients address issues in lifestyle
areas which may have been neglected during addiction. It is specifically oriented for group
settings. The manual uses didactic education and instruction, group process interaction through
role plays and discussion, daily relaxation and visualization exercise, informational handouts,
videotapes, and group-oriented recreational therapy exercises.
Living in Balance in an open group, meaning the youth are added when they arrive at the
program. Groups are held on Friday, Saturday, and Sunday each week.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Living in Balance was observed during the annual review. The group consisted of ten
(10) youth, one direct care staff and the facilitator. The facilitator led a group discussion about the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
influence substances have in thinking errors. The youth also participated in a role play to practice
how to make prosocial choices. The youth were engaged throughout the group and no behavioral
issues were observed.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
According to documentation provided by the program, no facilitator turnover occurred which
adversely affected delivery of Living in Balance. The groups are co-facilitated with trained staff so
if one person leaves, the other trained facilitator can complete the cohort with out delay.
From October 2021 until September 2022, the program had two cohorts of the primary service
running, but due to population growth, since September 2022, three cohorts are facilitated on the
designated days.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has a process in place to conduct monthly internal fidelity monitoring of facilitators of
Living in Balance. Internal fidelity monitoring took place on a monthly basis. Each person
facilitating group was observed and the monitoring was conducted by a staff member trained in
the primary service.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process in place to address corrective actions discovered during monthly
internal fidelity monitoring. A review of documentation revealed no corrective actions were initated
during the review period. The facilitators were found to be delivering Living in Balance in an
acceptable manner.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A copy of the annual employee evaluations were provided as part of the review. Each employee
who facilitated Living in Balance was rated in their effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 24 weeks for this type of service. Of the 46 youth in the
sample, 100% (46 of 46) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 40 contact hours for this type of service. Of the 46 youth in the
sample, 100% (46 of 46) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 85%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 7 youth
Moderate-High = 18 youth
High = 14 youth
Total Youth in Sample = 46 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 30%
High Score: 10 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 77%
Table 2
High = 14 youth
Total Youth in Sample = 46 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 85% (39 of 46) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 30% (14 of 46) youth scored High-Risk to reoffend, for a score of 10 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 20 25 80%
Totals 90 95 95%
This SPEP report evaluates Living in Balance, an intervention delivered at Broward Youth
Treament Center.
47 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 46 total youth sampled, all
46 included dosage with end dates in the EBS Module. Of those youth, 46 received at least the
recommended weeks of service. Youth in the sample completed between 26 and 68 weeks of
service, with an average of 36 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 46 total youth
sampled, all 46 included dosage in the EBS Module. Of those youth, 46 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
recommended hours of service. Youth in the sample completed between 75 and 193 hours of
service, with an average of 104 hours.
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 100%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Broward Youth Treament Center can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Broward Youth Treament Center can optimize their SPEP Amount of Service score by ensuring
dosage for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8

file_name: FY2022-23 BYTC LIB.pdf


summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Broward Youth Treatment Center
Youth Opportunities Investments, LLC
(Contract Provider)
8301 South Palm Drive
Pembroke, Florida 33025
Primary Service: Thinking for a Change
SPEP Review Period: November 1, 2021-October 31, 2022
SPEP Review Conducted: November 2, 2022
Florida Department of Juvenile Justice
Report Date: 2/2/23
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The Standardized Program Evaluation Protocol (SPEP) review encompasses twelve (12) months
of records. The program is currently utilizing the Thinking for a Change (T4C) 4.0 curriculum. T4C
is an evidence-based practice, utilizing a problem-solving program with both cognitive
restructuring and social skills interventions, bringing the identification of thinking, beliefs, attitudes,
and values to behavior. Effort is focused on helping youth become aware of their thoughts,
feelings, and beliefs in particular circumstances.
The program showed documentation of three staff members being trained in the primary service.
One case manager, one direct care staff supervisor, and the director of case management are all
trained in T4C. The case manager is responsible for facilitating the service and the director of
case management or direct care staff supervisor provides internal fidelity monitoring.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes Thinking for a Change (T4C), version 4.0. The curriculum is a highly
detailed, commercially produced manual which provides lesson plans, facilitator notes, ideas to
facilitate the curriculum, and long-term planning schedules. The manual is maintained online.
Facilitators are given access to the materials once they have successfully completed training. T4C
groups are conducted as closed groups, meaning once a group commences, no new youth may
be added.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Thinking for a Change was observed during the review period. The group consisted
of one facilitator, one direct care staff, and ten (10) youth and was held in a classroom. The
faciltator began the group with "check ins" on the youth's feelings/attitudes of the day. The group
then discussed elements of the cognitive iceberg, and how our behaviors are driven by not only
thoughts and feelings, but attitudes and beliefs. The group also then conducted role plays to show
how their thinking errors leads to poor behavior choices, and verbalized what other behavior
options they could chose. The facilitator followed the lesson plan and was knowledgeable of the
material.
Basic Score: 0 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
According to documentation provided and an informal interview with the director of case
management, the program did lose their primary Thinking for a Change (T4C) facilitator during the
review period. No other facilitator was available to deliver T4C and a gap in service delivery
occurred. The last group was completed on April 2, 2022 and no new cohort began until October
12, 2022.
The program provided documentation of three cohorts of T4C being delivered during the review
period. Two cohorts ran simultaneously beginning December 8, 2021, with ten (10) youth, and
were completed on April 4, 2022, with seven youth. Three youth were discharged from the
program prior to completing the service.
A third cohort began on October 12, 2022, with ten (10) youth and was ongoing at the time of this
annual review.
The program can earn 2 points if there are no unnecessary gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Internal fidelity monitoring took place on a monthly basis for Thinking for a Change (T4C)
facilitators. Internal fidelity monitoring was conducted each month groups were held; the
monitoring was conducted by staff member trained in T4C.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process in place to address corrective action needs identified during monthly
internal fidelity monitoring. A review of documentation revealed no corrective actions were initated
during the review period for facilitators of Thinking for a Change. The facilitators were found to be
delivering the primary service in an acceptable manner.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A copy of annual employee evaluations were provided as part of the review. Each employee who
facilitated Thinking for a Change was rated in their effectiveness in service delivery.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 15 weeks for this type of service. Of the 18 youth in the
sample, 89% (16 of 18) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 45 contact hours for this type of service. Of the 18 youth in the
sample, 83% (15 of 18) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 94%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 1 youth
Moderate-High = 8 youth
High = 8 youth
Total Youth in Sample = 18 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 44%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 8 youth
Total Youth in Sample = 18 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 94% (17 of 18) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 44% (8 of 18) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 23 25 92%
Totals 94 100 94%
This SPEP report evaluates Thinking for a Change, an intervention delivered at Broward Youth
Treatment Center.
47 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 18 total youth sampled, all
18 included dosage with end dates in the EBS Module. Of those youth, 16 received at least the
recommended weeks of service. Youth in the sample completed between 12 and 17 weeks of
service, with an average of 16 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 18 total youth
sampled, all 18 included dosage in the EBS Module. Of those youth, 15 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
recommended hours of service. Youth in the sample completed between 30 and 47 hours of
service, with an average of 44 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
RECOMMENDATION(S):
Broward Youth Treatment Center can optimize their SPEP Quality of Service Delivery score by
ensuring there are no unnecessary gaps in service.
Broward Youth Treatment Center can optimize their SPEP Amount of Service score by ensuring
dosage for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2022-23 BYTC T4C.pdf

service_type: Thinking for a Change