summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Broward Youth Treatment Center
Youth Opportunity Investments, LLC
(Contract Provider)
8301 South Palm Drive, Bldg. 2
Pembroke Pines, Florida 33025
Primary Service: Impact of Crime
SPEP Review Date(s): November 5‐6, 2019
Florida Department of Juvenile Justice
Report Date(s): November 12, 2019
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
 Service Type: service type points are awarded based on the group category that the
service falls within
 Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
 Amount of Service: or “dosage” refers to the amount and duration of services that were
received
 Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime. The program was awarded 15 points
because the Primary Service is identified as a Group 3 Service. The specific Sub-Component
Service Type identified is Social Skills Training. The Primary Service was identified as this type of
service as it focuses on developing social skills required for an individual to interact in a positive
way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The SPEP review encompasses twelve (12) months of records examinations. All facilitators
delivering Impact of Crime were trained by a qualified master trainer prior to delivering services to
youth. Those trained facilitators included three case managers and one direct care supervisor.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the commercially produced curriculum Impact of Crime: Addressing the
Harm to Victims and the Community. The curriculum was updated in 2009. It is highly detailed
providing a facilitator's manual, and student workbook. The facilitator's manual provides lessons
plans, classroom management ideas, and tips for engaging youth discussions. The student
manual provides reproducible worksheets to help facilitate group discussions, and review
concepts learned during the group sessions. Homework materials are also utilized in the student
manual to reinforce learned concepts. Impact of Crime is a closed group, meaning once a group
commences, new youth cannot be added to the group.
Impact of Crime is based on the belief that crime is more than just a legal definition, but rather
crime affects the victim, the offender, their families, and the community. The harm caused brings
with it the moral responsibility to all involved. The focus of Impact of Crime is to help offenders
understand the harm they have created, and then assisting them in taking personal accountability
for their actions. Together, the victim, offender, and the community arrive at a viable solution to
the repair the harm caused and the offender. Once the harm is repaired, the offender
subsequently reintegrates back into the community. Facilitators are encouraged to bring in victim
impact speakers throughout the course of the curriculum.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
During the annual review, a group session of Impact of Crime was observed. The facilitator
provided a copy of the lesson plan to the observer. The facilitator did follow the general outline of
the lesson. She did not read a script word for word, but generalized the information and added
her own words to provide meaning to covered concepts. The youth were generally engaged and
participated in the discussion. No behavioral issues were observed during the group.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
In reviewing documentation and an informal interview with director of case management, there
was no facilitator turnover during the review period which adversely affected delivery of Impact of
Crime (IOC). Most of the cohorts have two facilitators, and if one leaves, the co-facilitator would
continue to deliver the group. A new facilitator would be hired and trained before the beginning of
a new cohort of IOC. If only one facilitator is delivering the primary service and leaves the
program, the director of case management would take over and complete the cohort.
According to documentation provided by the program, a total of four cohorts of IOC were
completed during the last year. Two cohorts were also in session while review team was on site.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program completed four cohorts of Impact of Crime (IOC) during the review period. In all four
competed cohorts, internal fidelity monitoring took place monthly without exception. Two cohorts
of the primary service have been running for the last two months. In each of the current cohorts,
internal fidelity monitoring took place monthly. All internal fidelity was completed by an individual
trained in IOC.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
In reviewing the twelve months of fidelity monitoring documentation, no need for corrective actions
were documented. The director of case management stated during an informal interview a policy
and procedure does exist of corrective actions if a facilitator is found to be in need of additional
support.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The annual performance evaluations of facilitators of Impact of Crime (IOC) were reviewed.
Those evaluations showed all facilitators who delivered IOC were rated on their facilitator
proficency, and given credit on evaluations.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service ‐ Duration
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates the target duration of 16 weeks for this type of service. Of the 65 youth in the
sample, 23% (15 of 65) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates a target of 24 contact hours for this type of service. Of the 65 youth in the
sample, 52% (34 of 65) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 15 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 60%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 80%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 7 youth
Moderate-High = 26 youth
High = 19 youth
Total Youth in Sample = 65 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 29%
High Score: 8 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 62%
Table 2
High = 19 youth
Total Youth in Sample = 65 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 80% (52 of 65) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 29% (19 of 65) youth scored High-Risk to reoffend, for a score of 8 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 2 10 20%
Amount of Service: Contact Hours 4 10 40%
Risk Level of Youth Served 15 25 60%
Totals 61 85 72%
This SPEP report evaluates Impact of Crime, an intervention delivered at Broward Youth
Treatment Center.
65 youth completed the program during the review period. Of those youth, all 65 were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 2 points for Amount of Service: Duration. Of the 65 total youth sampled, only
34 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 15
received at least the recommended weeks of service. Youth in the sample completed between 13
and 18 weeks of service, with an average of 16 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 4 points for Amount of Service: Contact Hours. Of the 65 total youth
sampled, only 34 included dosage in the EBS Module. Of those youth with correct dosage, 34
received at least the recommended hours of service. Youth in the sample completed between 28
and 38 hours of service, with an average of 33 hours.
The program was awarded 15 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 84%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Broward Youth Treatment Center can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Broward Youth Treatment Center can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9


state: FL

facility_name: Broward Youth Treatment Center

summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Broward Youth Treatment Center
Youth Opportunity Investments, LLC
(Contract Provider)
8301 South Palm Drive, Bldg. 2
Pembroke Pines, Florida 33025
Primary Service: Living In Balance
SPEP Review Date(s): November 5‐6, 2019
Florida Department of Juvenile Justice
Report Date(s): November 12, 2019
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
 Service Type: service type points are awarded based on the group category that the
service falls within
 Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
 Amount of Service: or “dosage” refers to the amount and duration of services that were
received
 Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Living in Balance. The program was awarded 25 points
because the Primary Service is identified as a Group 4 Service. The specific Sub-Component
Service Type identified is Group Counseling. The Primary Service was identified as this type of
service as it focuses on psychological or interpersonal problems or issues faced by an individual
and involves a group of youths interacting with each other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The SPEP review encompasses twelve months of records examinations. According to training
records maintained by the clinical director, three master's level therapists are trained and
facilitating the Living in Balance on Friday, Saturday and Sunday each week. There are three
separate cohorts ongoing at the same time, and the therapists rotate who covers various groups
during the weekends. The program provided documentation of the therapists being trained in the
service by clinical director prior to delivering services to the youth.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
Living in Balance has a commercially produced manual which includes a highly detailed manual
which provides lessons plans, facilitator notes, ideas to facilitate the curriculum, and long-term
planning schedules. Living in Balance is a researched based substance abuse treatment
intended to help clients address issues in lifestyle areas that may have been neglected during
addiction. It is specifically oriented for group settings. The manual uses didactic education and
instruction, group process interaction through role plays and discussion, daily relaxation and
visualization exercises, informational handouts, videotapes, and group-oriented recreational
therapy exercises. The primary service is facilitated by master's level clinical staff.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Living in Balance groups were not being conducted while the reviewer was on-site. Future
quarterly SPEP reviews will be scheduled so the primary service can be observed and the
program may earn credit in up-coming annual reviews.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation and an informal interview with the clinical director confirmed no
facilitator turnover has affected delivery of Living in Balance. In the event of a therapist leaving
the program, the clinical director would step into rotation to help cover groups until a new therapist
could be hired and trained.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
According to documentation provided by the program, monthly internal fidelity monitoring of Living
in Balance is taking place without exception. The monitoring is being completed by supervisor
staff trained in the primary service.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a policy in place to ensure therapeutic services are delivered to the youth in a
manner as dictated by the curriculum. In the event of need for corrective actions, the clinical
director would conduct a conference with therapist(s) to clearly delineate needed areas of
improvement, and work with therapist(s) to develop a plan to address facilitator issues.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of performance evaluation documents confirmed the facilitators were acknowledged on
the annual performance evaluation in delivering Living in Balance. This included feedback and
recommendations of the facilitator’s proficiency in delivering the primary service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service ‐ Duration
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates the target duration of 24 weeks for this type of service. Of the 8 youth in the
sample, 38% (3 of 8) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 40 contact hours for this type of service. Of the 8 youth in the
sample, 100% (8 of 8) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 75%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 1 youth
Moderate-High = 2 youth
High = 3 youth
Total Youth in Sample = 8 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 38%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 3 youth
Total Youth in Sample = 8 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 75% (6 of 8) youth scored Moderate to High-Risk to reoffend, for a score of
7 points.
Of the SPEP sample, 38% (3 of 8) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 2 10 20%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 20 25 80%
Totals 82 95 86%
This SPEP report evaluates Living in Balance, an intervention delivered at Broward Youth
Treatment Center.
65 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 2 points for Amount of Service: Duration. Of the 8 total youth sampled, 3
included dosage with end dates in the EBS Module. Of those youth with correct dosage, 3
received at least the recommended weeks of service. Youth in the sample completed between 25
and 30 weeks of service, with an average of 27 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 8 total youth
sampled, all 8 included dosage in the EBS Module. Of those youth, 8 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
recommended hours of service. Youth in the sample completed between 73 and 135 hours of
service, with an average of 91 hours.
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 92%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Broward Youth Treatment Center can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Broward Youth Treatment Center can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8

source: DJJ

report_type: SPEP

program_name_raw: Broward Youth Treatment Center - YOI

program_status_note: Closed

service_type: Living in Balance


summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Broward Youth Treatment Center
Youth Opportunity Investments, LLC
(Contract Provider)
8301 South Palm Drive, Bldg. 2
Pembroke Pines, Florida 33025
Primary Service: Thinking for a Change
SPEP Review Date(s): November 5‐6, 2019
Florida Department of Juvenile Justice
Report Date(s): November 12, 2019
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
 Service Type: service type points are awarded based on the group category that the
service falls within
 Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
 Amount of Service: or “dosage” refers to the amount and duration of services that were
received
 Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change. The program was awarded 30
points because the Primary Service is identified as a Group 5 Service. The specific Sub-
Component Service Type identified is Cognitive Behavioral Therapy. The Primary Service was
identified as this type of service as it is intended to correct faulty cognitions and perceptions and
provides skills individuals can use to monitor thought patterns and correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The Standardized Program Evaluation Protocol (SPEP) review encompasses twelve months of
records examinations. All facilitators were trained in Thinking for a Change as documented by the
department's learning management system (SkillPro) training records. Those trained in the
service included: clinical director, two case managers, and two direct care specialists.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
A review of documents and records revealed the program uses the evidence-based curriculum
Thinking for a Change (T4C) 4.0. The program utilizes a highly detailed, commercially produced
manual that provides lessons plans, facilitator notes, ideas to facilitate the curriculum, and long-
term planning schedules. The manual is maintained on-line. Facilitators are given access to the
materials once they have successfully completed train-the-trainer training. Groups are conducted
as closed groups, meaning once a group commenced, no new youth can be added to the group.
T4C utilizes a problem-solving program with both cognitive restructuring, and social skills
interventions bringing the identification of thinking, beliefs, attitudes and values to behavior. Effort
is focused on helping youth become aware of their thoughts, feelings and beliefs in particular
circumstances.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of the primary service could not be observed during this review period due to
scheduling conflict. A group will be observed during a quarterly review.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation and an informal interview with the director of case management
indicated the program has had facilitators leave the program. When a facilitator left, the co-
facilitator completed the cohort, and the program worked to hire and train new facilitator. If no one
was co-facilitating, another trained facilitator would step in and complete the cohort. During the
reviewed period, no gaps in delivery of the primary service were apparent.
According to documentation provided by the program, five cohorts of the primary service have
been completed during the review period. Additionally, a cohort is currently on-going.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
According to documentation provided, internal fidelity monitoring took place on a monthly basis.
The fidelity monitoring was completed by another trained facilitator in the primary service.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a policy for corrective action plans for facilitators of Thinking for a Change. In
the event a facilitator is in need of help with service delivery, the director of case management will
sit with the facilitator and develop a plan to improve the facilitator's skills.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of annual evaluations did document the facilitators were given credit for their group
facilitation. The facilitators' skills in the delivery of the service were also evaluated by supervisors.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 15 weeks for this type of service. Of the 44 youth in the
sample, 16% (7 of 44) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 45 contact hours for this type of service. Of the 44 youth in the
sample, 100% (44 of 44) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 17 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 68%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 84%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 6 youth
Moderate-High = 18 youth
High = 13 youth
Total Youth in Sample = 44 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 30%
High Score: 10 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 77%
Table 2
High = 13 youth
Total Youth in Sample = 44 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 84% (37 of 44) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 30% (13 of 44) youth scored High-Risk to reoffend, for a score of 10 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 17 25 68%
Totals 82 100 82%
This SPEP report evaluates Thinking for a Change, an intervention delivered at Broward Youth
Treatment Center.
65 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 44 total youth sampled, all
44 included dosage with end dates in the EBS Module. Of those youth, 7 received at least the
recommended weeks of service. Youth in the sample completed between 13 and 16 weeks of
service, with an average of 14 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 44 total youth
sampled, all 44 included dosage in the EBS Module. Of those youth, 44 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
recommended hours of service. Youth in the sample completed between 46 and 55 hours of
service, with an average of 48 hours.
The program was awarded 17 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 90%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Broward Youth Treatment Center can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Broward Youth Treatment Center can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8

service_type: Thinking for a Change


summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Broward Youth Treatment Center
Youth Opportunity Investments, LLC
(Contract Provider)
8301 South Palm Drive, Bldg. 2
Pemproke Pines, Florida 33025
Primary Service: Impact of Crime
SPEP Review Period: November 1, 2019-October 31, 2020
SPEP Review Conducted: November 19, 2019
Florida Department of Juvenile Justice
Report Date: December 30, 2020
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The Standardized Program Evaluation Protocol (SPEP) review encompasses twelve (12) months
of SPEP records. The program utilizes Impact of Crime as a primary service. According to
documentation provided by the program, all staff facilitating Impact of Crime were trained as
required prior to delivering services to youth.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the curriculum Impact of Crime: Addressing the Harm to Victims and the
Community. The curriculum was updated in 2009. It is highly detailed, providing a facilitator's
manual and student workbook. The facilitator's manual provides lessons plans, classroom
management ideas, and tips for engaging youth discussions. The student manual provides
reproducible worksheets to help facilitate group discussions, and review concepts learned during
the group sessions. Homework materials are also utilized in the student manual to re-enforce
learned concepts.
Impact of Crime is based on the belief that crime is more than just a legal definition, but rather
crime affects the victim, the offender, their families, and the community. The harm caused brings
with it the moral responsibility to all involved. The focus of Impact of Crime is to help offenders
understand the harm they have created, and then assisting them in taking personal accountability
for their actions. Together, the victim, offender, and the community arrive at a viable solution to
the repair the harm caused and the offender. Once the harm is repaired, the offender
subsequently reintegrates back into the community. Facilitators are encouraged to bring in victim
impact speakers throughout the course of the curriculum. Impact of crime operates as a closed
group, meaning once the cohort begins, new youth cannot be added to the cohort. The youth
must wait for a new cohort to commence.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
During the annual review, a group session of Impact of Crime was observed. The facilitator did
not read a script word for word, but generalized the information and added her own words to
provide meaning to covered concepts. The youth were generally engaged, and participated in the
discussion. No behavioral issues were observed during the group.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
In reviewing documentation, there was no facilitator turnover during the review period which
affected delivery of Impact of Crime (IOC). If a facilitator leaves, other trained staff would continue
to deliver the group. A new facilitator would be hired and trained before the beginning of a new
cohort of IOC. If only one facilitator is delivering the primary service and leaves the program, the
director of case management would take over and complete the cohort.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
According to documentation provided by the program, internal fidelity monitoring took place. In
each of the cohorts, internal fidelity monitoring took place monthly without exception. All internal
fidelity was completed by an individual trained in Impact of Crime.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
No need for corrective actions were documented for Impact of Crime. The director of case
management stated during an informal interview a policy and procedure exists to address
corrective actions if a facilitator is found to be in need of additional support.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The annual performance evaluations of facilitators of Impact of Crime (IOC) were reviewed. Those
evaluations showed all facilitators who delivered IOC were rated on their facilitator proficency and
given credit on evaluations.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates the target duration of 16 weeks for this type of service. Of the 42 youth in the
sample, 55% (23 of 42) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 24 contact hours for this type of service. Of the 42 youth in the
sample, 62% (26 of 42) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 86%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 5 youth
Moderate-High = 13 youth
High = 18 youth
Total Youth in Sample = 42 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 43%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 18 youth
Total Youth in Sample = 42 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 86% (36 of 42) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 43% (18 of 42) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 4 10 40%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 23 25 92%
Totals 73 85 86%
This SPEP report evaluates Impact of Crime, an intervention delivered at Broward Youth
Treatment Center.
45 youth completed the program during the review period. Of those youth, 42 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 4 points for Amount of Service: Duration. Of the 42 total youth sampled, only
31 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 23
received at least the recommended weeks of service. Youth in the sample completed between 9
and 18 weeks of service, with an average of 15 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 6 points for Amount of Service: Contact Hours. Of the 42 total youth
sampled, only 31 included dosage in the EBS Module. Of those youth with correct dosage, 26
received at least the recommended hours of service. Youth in the sample completed between 19
and 35 hours of service, with an average of 31 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
RECOMMENDATION(S):
Broward Youth Treatment Center can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Broward Youth Treatment Center can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

service_type: Impact of Crime


summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Broward Youth Treatment Center
Youth Opportunity Investments, LLC
(Contract Provider)
8301 South Palm Drive, Bldg. 2
Pembroke Pines, Florida 33025
Primary Service: Living in Balance
SPEP Review Period: November 1, 2019-October 31,2020
SPEP Review Conducted: November 19, 2020
Florida Department of Juvenile Justice
Report Date: December 30, 2020
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Living in Balance.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The Standardized Program Evaluation Protocol (SPEP) annual review encompasses twelve
months of SPEP records. According to training records maintained by the program, masters level
therapists are facilitating Living in Balance (LIB). The program provided documentation of the
therapists being trained in the service by clinical director prior to delivering services to youth.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
Living in Balance (LIB) has a commercially produced manual which includes a highly detailed,
commercially-produced manual providing lesson plans, facilitator notes, ideas to facilitate the
curriculum, and long-term planning schedules.
LIB is a researched-based substance abuse treatment intended to help clients address issues in
lifestyle areas which may have been neglected during addiction and is specifically oriented for
group settings. The manual uses didactic education and instruction, group process interaction
through role plays and discussion, daily relaxation and visualization exerciese, informational
handouts, videotapes, and group-oriented recreational therapy exercises.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Living in Balance was observed virtually. The facilitator was knowledgable of the
material and able to engage the youth in group discussion. No behavioral issues were noted
during the group.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
According to documentation provided for review, no facilitator turnover has affected delivery of
Living in Balance. In the event of a therapist leaving the program, the clinical director would step
into rotation to help cover groups until a new therapist could be hired and trained. This group is
run year-round, with documentation confirming there were no gaps in service.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
According to documentation provided by the program, monthly internal fidelity monitoring is taking
place for Living in Balance failitators, without exception. The monitoring is being completed by
supervisory staff trained in the primay service.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
No corrective actions were noted nor taken during the review period for Living in Balance. The
program has a policy in place to ensure therapeutic services are delivered to the youth in a
manner as dictated by the curriculum. In the event of need for corrective actions, the clinical
director would conduct a conference with therapist(s) to clearly delinate needed areas of
improvement, and work with therapist(s) to develop a plan to address facilitator issues.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of performance evaluation documents confirmed the facilitators were acknowledged in
delivering Living in Balance. This included feedback and recommendations of the facilitator’s
proficiency in delivering the primary service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 24 weeks for this type of service. Of the 45 youth in the
sample, 96% (44 of 45) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 40 contact hours for this type of service. Of the 45 youth in the
sample, 100% (45 of 45) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 87%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 5 youth
Moderate-High = 13 youth
High = 21 youth
Total Youth in Sample = 45 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 47%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 21 youth
Total Youth in Sample = 45 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 87% (39 of 45) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 47% (21 of 45) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 23 25 92%
Totals 91 95 96%
This SPEP report evaluates Living in Balance, an intervention delivered at Broward Youth
Treatment Center.
45 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 45 total youth sampled, all
45 included dosage with end dates in the EBS Module. Of those youth, 44 received at least the
recommended weeks of service. Youth in the sample completed between 17 and 44 weeks of
service, with an average of 29 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 45 total youth
sampled, all 45 included dosage in the EBS Module. Of those youth, 45 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
recommended hours of service. Youth in the sample completed between 50 and 126 hours of
service, with an average of 86 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
RECOMMENDATION(S):
Broward Youth Treatment Center can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Broward Youth Treatment Center can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8


summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Broward Youth Treatment Center
Youth Opportunity Investments, LLC
(Contract Provider)
8301 South Palm Drive, Bldg. 2
Pembroke Pines, Florida 33025
Primary Service: Thinking for a Change
SPEP Review Period: November 1, 2019-October 31, 2020
SPEP Review Conducted: November 19, 2020
Florida Department of Juvenile Justice
Report Date: December 30, 2020
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The Standardized Program Evaluation Protocol (SPEP) review encompasses twelve months of
records. The program is currently utilizing the Thinking for a Change (T4C) 4.0 curriculum. All
facilitators were trained in the primary service by a qualified trainer prior to delivering services to
youth.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
A review of documents and records revealed the use of the evidence-based curriculum Thinking
for a Change (T4C) 4.0. The program utilizes a highly detailed, commercially produced manual
which provides lesson plans, facilitator notes, ideas to facilitate the curriculum, and long-term
planning schedules. The manual is maintained online. Facilitators are given access to the
materials once they have successfully completed train-the-trainer training. Groups are conducted
as closed groups, meaning once a group commenced, no new youth can be added to the group.
T4C utilizes a problem-solving program with both cognitive restructuring, and social skills
interventions bringing the identification of thinking, beliefs, attitudes and values to behavior. Effort
is focused on helping youth become aware of their thoughts, feelings and beliefs in particular
circumstances.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
A session of Thinking for a Change was observed. The group was conducted in one of the
classrooms. The facilitator followed the script, but did not read it word for word. No behavioral
issues were observed during the group, and the youth participated in the group activities which
included reading, group discussion, and role plays.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation confirmed there has been no turnover of facilitators which adversely
affected delivery of Thinking for a Change (T4C). If a facilitator leaves, the co-facilitator completed
the cohort, and the program works to hire and train new facilitator. If no one was co-facilitating,
another trained facilitator would step in and complete the cohort. During the reviewed period, no
gaps in delivery of the primary service were apparent. According to documentation provided by
the program, five cohorts of T4C have been completed during the review period.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
According to documentation provided by the program, internal fidelity monitoring of Thinking for a
Change took place on a monthly basis without exception. The fidelity monitoring was completed
by another trained facilitator in the primary service.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
No corrective actions were noted nor taken during the review period. The program has a policy for
corrective action plans for facilitators of the primary service. In the event a facilitator is in need of
help with service delivery, the director of case management will sit with the facilitator and develop
a plan to improve the facilitator's skills.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of annual evaluations documented all facilitators were given credit for their group
facilitation of Thinking for a Change. The facilitators' skills in the delivery of the service was also
evaluated by supervisors.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 15 weeks for this type of service. Of the 21 youth in the
sample, 100% (21 of 21) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 45 contact hours for this type of service. Of the 21 youth in the
sample, 95% (20 of 21) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 95%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 4 youth
Moderate-High = 5 youth
High = 11 youth
Total Youth in Sample = 21 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 52%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 11 youth
Total Youth in Sample = 21 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 95% (20 of 21) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 52% (11 of 21) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 25 25 100%
Totals 98 100 98%
This SPEP report evaluates Thinking for a Change, an intervention delivered at Broward Youth
Treatment Center.
45 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 21 total youth sampled, all
21 included dosage with end dates in the EBS Module. Of those youth, 21 received at least the
recommended weeks of service. Youth in the sample completed between 15 and 16 weeks of
service, with an average of 16 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 21 total youth
sampled, all 21 included dosage in the EBS Module. Of those youth, 20 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
recommended hours of service. Youth in the sample completed between 44 and 55 hours of
service, with an average of 49 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
RECOMMENDATION(S):
Broward Youth Treatment Center can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Broward Youth Treatment Center can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8


summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Broward Youth Treament Center
Youth Opportunity Investments, LLC
(Contract Provider)
8301 South Palm Drive
Pembroke Pines, Florida 33025
Primary Service: Thinking for a Change
SPEP Review Period: November 1, 2020-October 31, 2021
SPEP Review Conducted: November 10, 2021
Florida Department of Juvenile Justice
Report Date: December 16, 2021
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The Standardized Program Evaluation Protocol (SPEP) review encompasses twelve months of
records. The program is currently utilizing the Thinking for a Change (T4C) 4.0 curriculum. T4C is
an evidence-based practice which utilizes a problem-solving program with both cognitive
restructuring and social skills interventions, bringing the identification of thinking, beliefs, attitudes,
and values to behavior. Effort is focused on helping youth become aware of their thoughts,
feelings, and beliefs in particular circumstances.
The program showed documentation of three staff members being trained in T4C- two case
managers and the director of case management. The case managers are responsible for
facilitating the primary service and director of case management provides internal fidelity
monitoring.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
A review of documents confirmed the program uses the evidence-based curriculum Thinking for a
Change (T4C) 4.0. The program utilizes a highly detailed, commercially produced manual which
provides lesson plans, facilitator notes, ideas to facilitate the curriculum, and long-term planning
schedules. The manual is maintained online. Facilitators are given access to the materials once
they have successfully completed train-the-trainer training. All youth assigned to the program
complete T4C. Groups are conducted as closed groups, meaning once a group commenced, no
new youth can be added.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Thinking for a Change was observed. The group session took place in a classroom
with nine youth in attendance. The facilitator modeled the role play, then each member was able
to practice the new skill. There were no behavior issues observed during the group. The facilitator
did not read the script verbatim but did have the facilitator's guide open on the desk to ensure all
points of the lesson were covered.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There were no gaps in delivery of Thinking for a Change (T4C) during the past year. In reviewing
documentation and an informal interview with both the director of case management and the
facility administrator, a case manager was dismissed from their position in the middle of a cohort
being delivered from March 1, 2021 to June 14, 2021. The director of case management, who is
trained in the primary service, took over facilitation and completed the cohort.
The program provided documentation of four cohorts of T4C being completed during the review
period. The first cohort began on October 5, 2020 and was completed on January 29, 2021. The
cohort began with twelve (12) youth and ended with eight youth.
Two cohorts ran simultaneously, beginning March 1, 2021 and completing on June 14, 2021. The
cohorts started with eleven (11) and nine youth and ended with eight and six youth, respectively.
A fourth cohort began on July 19, 2021 with ten (10) youth and ended on November 3, 2021 with
six youth.
According to the Case Management Director, all but one youth who did not complete the service
were released prior to the end of group. In each case, the youth were provided the service on an
accelerated, individual schedule to receive all material prior to discharge.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
According to documentation provided by the program, internal fidelity monitoring took place on a
monthly basis for facilitators of Thinking for a Change. The fidelity monitoring was completed by a
staff member trained in the primary service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a procedure for corrective action plans for facilitators of Thinking for a Change.
In the event a facilitator is in need of help with service delivery, the director of case management
will sit with the facilitator and develop a plan to improve the facilitator's skills.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of annual evaluations documented the facilitators were given credit for their group
facilitation of Thinking for a Change. The facilitators' skills in the delivery of the service was also
evaluated by supervisors.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 15 weeks for this type of service. Of the 30 youth in the
sample, 87% (26 of 30) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 45 contact hours for this type of service. Of the 30 youth in the
sample, 83% (25 of 30) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 83%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 3 youth
Moderate-High = 10 youth
High = 12 youth
Total Youth in Sample = 30 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 40%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 12 youth
Total Youth in Sample = 30 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 83% (25 of 30) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 40% (12 of 30) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 20 25 80%
Totals 91 100 91%
This SPEP report evaluates Thinking for a Change, an intervention delivered at Broward Youth
Treament Center.
32 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 30 total youth sampled, only
26 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 26
received at least the recommended weeks of service. Youth in the sample completed between 15
and 17 weeks of service, with an average of 16 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 30 total youth
sampled, all 30 included dosage in the EBS Module. Of those youth, 25 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
recommended hours of service. Youth in the sample completed between 41 and 46 hours of
service, with an average of 45 hours.
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 96%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Broward Youth Treament Center can optimize their SPEP Quality of Service Delivery score by
ensuring facilitator turnover does not impact groups.
Broward Youth Treament Center can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2021-22 BYTC T4C.pdf