summary: SPEP - FY18-19

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Polk Halfway House
TrueCore Behavioral Solutions, LLC
(Contract Provider)
2145 Bob Phillips Road
Bartwo, Florida 33830
Primary Service: Thinking for a Change
SPEP Review Date(s): 4/2/19‐4/5/19
Florida Department of Juvenile Justice
Report Date(s): 4/9/19
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
 Service Type: service type points are awarded based on the group category that the
service falls within
 Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
 Amount of Service: or “dosage” refers to the amount and duration of services that were
received
 Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change. The program was awarded 30
points because the Primary Service is identified as a Group 5 Service. The specific Sub-
Component Service Type identified is Cognitive Behavioral Therapy. The Primary Service was
identified as this type of service as it is intended to correct faulty cognitions and perceptions and
provides skills individuals can use to monitor thought patterns and correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program provided the facilitator training protocol. The program's protocol was not updated to
reflect the change in primary services for the program. The protocol listed the case manager and
transition services manager as the facilitators of primary services, however the therapists and
transition services manager conducts the current primary services.
The program provided documentation for four staff members formally trained in Thinking for a
Change (T4C) curriculum. The director of clinical services, two therapists, and the transition
services manager are all formally trained. All trained staff are still employed at the program.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program provided the protocol for service delivery for review. The protocol stated each
primary service will be facilitated in accordance with its manual. The primary service will be
conducted as indicated in the manual/protocol; this includes adherence to the session plan, group
size, population, frequency, needed materials, duration, and interventions. The faciltitator of the
primary service will specify the topic covered, interventions utililized, duration of service, youth
present, and progress for each session on the sign-in sheet.
The program provided the Thinking for a Change (T4C) 3.1 manual for review. There is not a
separate manual for youth, but the program has all of the worksheets available for the group. The
manual provides a script and examples to use while facilitating the curriculum. The manual also
assists the facilitators with how the groups should be led, how to word questions and various
topics, and how to prompt the youth for answers. T4C group is held on Tuesdays and Thursdays
from 3:30pm-4:30pm. All youth are eligible to receive T4C group as a part of the programming at
Polk Halfway House.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
The sign-in sheets for T4C are submitted to the director of clinical services for review and then
kept in a binder labeld "PHH SPEP INFORMATION". Fidelity monitoring is comleted monthly for
each facilitator. These are reviewed with the facilitator by the reviewer and then submitted to the
facility administrator/designee for review.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Thinking for a Change (T4C) group was observed on Thursday, April 4, 2019 from 2:30pm-
3:30pm. A copy of the lesson and slides were provided to the observer for reference. A formally
trained therapist facilitated this group. There were eight youth in attendance and one direct care
staff present for group. The director of clinincal services was also present during this group,
conducting internal fidelity monitoring.
The group was continuing their lesson from Tuesday and conducting role plays involving Jim's
situation from the curriculum. The director of clinical services and therapist conducted an example
role play to show expectations for the youth. The youth were able to review the scenario and
thinking report. The group worked together to establish new thoughts, attitudes, and beliefs. The
youth conducted the role play. Three groups of youth were able to conduct their role plays during
this group session. The therapist conducted feedback and asking the questions in the manual.
The program followed the protocol for service delivery and followed the provided lesson as
intended.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program provided their protocol for facilitator turnover for review. The protocol states in the
event of facilitator turnover, the co-facilitator will assume the role of the facilitator for the group. If
a co-facilitator is not present, a new qualified facilitator for the primary service will take over
conducting the service. One therapist was transferred to a different TrueCore program. This did
not disrupt the Thinking for a Change groups being held.
Two full cohorts have been completed since T4C became a primary service in July of 2018. The
cohorts were held from August 21, 2018 to December 28, 2018 and September 18, 2018 to
February 6, 2019. There is currently a group, running which started on February 28, 2019.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program's written protocol for internal fidelity monitoring was provided for review. The
protocol states the designated staffs (facility administrator, designated mental health clinical
authority, therapists, and regional clinical director) are responsible for conducting fidelity
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
monitoring of each faciltator of a primary service on a monthly basis. The staff conducting the
fidelity monitoring has received formal training by a qualified trainer in the primary service. Fidelity
monitoring is documented on a fidelity monitoring tool developed specificallly for that primary
service. Internal fidelity monitoring is conducted randomly each month for each facilitator by other
trained staff.
The program provided all internal fidelity monitoring forms for the three facilitators, with the
exception of August 2018 and March 2019. All other internal fidelity monitoring forms were
provided.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program provided the protocol for the corrective action process. When a deficiency is
identified, the supervisor will start with a coaching session with the employee to discuss the
deficiency noted. If necessary, due to the severity of the deficiency or a repeated pattern, the
supervisor will complete a corrective action or the employee is placed on a performance
improvement plan which is reviewed to ensure the employee is making the necessary
adjustments.
All twenty-eight internal fidelity monitor forms were reviewed for corrective actions. None of the
twenty-eight internal fidelity monitoring forms indicated corrective action was needed for the
faciltiators of Thinking for a Change.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program provided the protocol for yearly performance evaluation of service delivery for
review. The protocol stated the annual evaluation of performance will be conducted on all the
facilitators of each primary service. The performance evaluation is comprised of feedback from
the monthly fidelity monitoring tools and an assesment of the faciltaitor's skill in delivering the
primary service. The evaluation is completed by a qualified facilitator and reviewed by the
facilitator's supervisor.
All facilitators of Thinking for a Change curriculum received a yearly performance evaluation. The
yearly performance evaluations had a section for group faciltiation and had all curricula the
facilitator had been trained in listed.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service ‐ Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 15 weeks for this type of service. Of the 17 youth in the
sample, 100% (17 of 17) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 45 contact hours for this type of service. Of the 17 youth in the
sample, 0% (0 of 17) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 88%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 1 youth
Moderate-High = 8 youth
High = 6 youth
Total Youth in Sample = 17 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 35%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 6 youth
Total Youth in Sample = 17 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 88% (15 of 17) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 35% (6 of 17) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 23 25 92%
Totals 88 100 88%
This SPEP report evaluates Thinking for a Change, an intervention delivered at Polk Halfway
House.
30 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 17 total youth sampled, all
17 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 17
received at least the recommended weeks of service. Youth in the sample completed between 15
and 21 weeks of service, with an average of 19 weeks.
The program earned 0 points for Amount of Service: Contact Hours. Of the 17 total youth
sampled, all 17 included dosage in the EBS Module. Of those youth with correct dosage, 0
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
received at least the recommended hours of service. Youth in the sample completed between 36
and 42 hours of service, with an average of 39 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Polk Halfway House can optimize their SPEP Quality of Service Delivery score by ensuring fidelity
monitoring is completed every month for each facilitator.
Polk Halfway House can optimize their SPEP Amount of Service score by ensuring that dosage
for all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

service_type: Thinking for a Change


state: FL

facility_name: Polk Halfway House

summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Polk Halfway House
TrueCore Behavioral Solutions, LLC
(Contract Provider)
2145 Bob Phillips Road
Bartow, Florida 33830
Primary Service: Impact of Crime
SPEP Review Date(s): 04/27/20
Florida Department of Juvenile Justice
Report Date(s): 05/01/20
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There were four staff formally trained in the Impact of Crime curriculum, one Clinical Director, one
Recreational Therapist, one Youth Care Worker II and one Director of Case Management. Three
facilitated the group and the Director of Case Management provided fidelity monitoring.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The manual provided for review was Impact of Crime-Addressing the Harm. The manual contains
a script for the facilitator. It provides lessons and includes implementation guidelines, identifies the
order the lessons should be delivered in and includes detailed instructions for the delivery of each
lesson.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Impact of Crime group was unable to be observed due to the COVID -19 pandemic;
therefore, this indicator rates as non-applicable.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There has been no facilitator turnover in the past twelve (12) months which adversely affected the
delivery of Impact of Crime to the youth. There was a gap in service from 11/16/19-01/05/20, due
to having less than 10 youth to run the closed group.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation for fidelity monitoring of Impact of Crime was reviewed. The program
completed internal fidelity monitoring for each facilitator month groups were being held.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the Director of Case Management verified the program has a process for
corrective actions based on the fidelity monitoring conducted. The program uses internal fidelity
monitoring/corrective action checklists specific to Impact of Crime. A review of documentation
provided and an interview with the Director of Case Management verified there was no corrective
action noted nor taken during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The performance evaluations of all trained facilitators found entries of the staff's performance in
the delivery of the Impact of Crime curriculum. These included comments on any strengths or
weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates the target duration of 16 weeks for this type of service. Of the 16 youth in the
sample, 56% (9 of 16) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 16 youth in the
sample, 100% (16 of 16) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 4 youth
Moderate-High = 3 youth
High = 9 youth
Total Youth in Sample = 16 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 56%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 9 youth
Total Youth in Sample = 16 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (16 of 16) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 56% (9 of 16) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 4 10 40%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 25 25 100%
Totals 79 85 93%
This SPEP report evaluates Impact of Crime, an intervention delivered at Polk Halfway House.
31 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 4 points for Amount of Service: Duration. Of the 16 total youth sampled, all
16 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 9
received at least the recommended weeks of service. Youth in the sample completed between 10
and 19 weeks of service, with an average of 16 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 16 total youth
sampled, all 16 included dosage in the EBS Module. Of those youth, 16 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
recommended hours of service. Youth in the sample completed between 19 and 34 hours of
service, with an average of 31 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Impact of Crime is an intervention suitable for all youth. The Department recommends providing
this service to every youth in the program.
Polk Halfway House can maintain their SPEP Quality of Service Delivery score by continuing the
practices in place at the time of this review.
Polk Halfway House can optimize their SPEP Amount of Service score by ensuring that dosage
for all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8

source: DJJ

report_type: SPEP

program_name_raw: Polk Halfway House

program_status_note: Closed

service_type: Impact of Crime


summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Polk Halfway House
TrueCore Behavioral Solutions, LLC
(Contract Provider)
2145 Bob Phillips Road
Bartow, Florida 33830
Primary Service: Thinking for a Change
SPEP Review Date(s): 05/01/20
Florida Department of Juvenile Justice
Report Date(s): 05/01/20
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There were four staff trained in the Thinking for a Change curriculum, one Assistant Facility
Administrator, one Director of Case Management, one Clinical Director, and one Therapist. Three
of these staff members facilitated group and one provided fidelity montoring.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The manual provided to for review was Thinking for a Change 3.1. The manual contains a script
for the facilitator for the groups. It provides 25 lessons and includes implementation guidelines,
identifies the order the lessons should be delivered in and includes detailed instructions for the
delivery of each lesson.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Thinking for a Change was unable to be observed due to the COVID-19 pandemic;
therfore, this indicator rates as non-applicable.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There has been some facilitator turnover in the past twelve (12) months; however, this turnover
did not adversely affect the delivery of Thinking for a Change (T4C). A review of documentation
provided and an interview with the Clinical Director provided proof there was a gap from 08/07/19-
01/05/20. According to the Clinical Director, the gap was due to not having enough new youth to
start this closed group.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation for fidelity monitoring of Thinking for a Change was reviewed. The program
completed internal fidelity monitoring for each facilitator monthly while groups were being held.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the Clinical Director verified the program has a process for corrective actions
based on the fidelity monitoring conducted. The program uses internal fidelity monitoring /
corrective action checklists specific to Thinking for a Change. A review of documentation provided
and an interview with the Clinical Director verified there was no corrective action noted nor taken
during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The performance evaluations of all trained facilitators found entries of the staff's performance in
the delivery of the Thinking for a Change curriculum. These included comments on any strengths
or weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 15 weeks for this type of service. Of the 12 youth in the
sample, 92% (11 of 12) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates a target of 45 contact hours for this type of service. Of the 12 youth in the
sample, 58% (7 of 12) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 1 youth
Moderate-High = 2 youth
High = 9 youth
Total Youth in Sample = 12 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 75%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 9 youth
Total Youth in Sample = 12 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (12 of 12) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 75% (9 of 12) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 4 10 40%
Risk Level of Youth Served 25 25 100%
Totals 92 100 92%
This SPEP report evaluates Thinking for a Change, an intervention delivered at Polk Halfway
House.
31 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 12 total youth sampled, all
12 included dosage with end dates in the EBS Module. Of those youth, 11 received at least the
recommended weeks of service. Youth in the sample completed between 5 and 23 weeks of
service, with an average of 21 weeks.
The program earned 4 points for Amount of Service: Contact Hours. Of the 12 total youth
sampled, all 12 included dosage in the EBS Module. Of those youth, 7 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
recommended hours of service. Youth in the sample completed between 8 and 47 hours of
service, with an average of 42 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Polk Halfway House can maintain their SPEP Quality of Service Delivery score by continuing the
processes in place at the time of this review.
Polk Halfway House can optimize their SPEP Amount of Service score by ensuring that dosage
for all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8


summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Polk Halfway House
TrueCore Behavioral Services, LLC
(Contract Provider)
2145 Bob Phillips Road
Bartow, Florida 33830
Primary Service: Impact of Crime
SPEP Review Period: 04/01/20-03/31/21
SPEP Review Conducted: 04/08/21
Florida Department of Juvenile Justice
Report Date: 07/01/2021
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There were three staff formally trained in the Impact of Crime (IOC) curriculum, one Director of
Case Management, one Recreational Therapist, and one Therapist. Two facilitated the group and
the Director of Case Management provided fidelity monitoring.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The manual provided to this reviewer was Impact of Crime-Addressing the Harm. The manual
contains a script for the facilitator for the groups. It provides lessons and includes implementation
guidelines, identifies the order the lessons should be delivered and includes detailed instructions
for the delivery of each lesson.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The Impact of Crime group was observed on Tuesday 04/13/21 from 11:20-12:30 via Zoom.
There were six youth in the group. The facilitator provided a copy of the lesson, Chapter One
What Does Justice Mean? The facilitator followed the lesson plan according to the manual, did
not read it verbatim and used her own words to convey the meaning of the lesson to the youth.
She was very easy to follow and adhered to the lesson plan in every aspect.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There has been no facilitator turnover in the past twelve (12) months which adversely affected the
delivery of Impact of Crime to the youth. There was a gap in service from 11/14/20-12/29/20. This
was due to having a low census county and having to wait on enough youth to run this closed
group and does not qualify as an unnecessary gap in services.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation for fidelity monitoring of Impact of Crime was reviewed. The program
completed monitoring for each facilitator month except for the month of 11/20.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the Director of Case Management and the Clinical Director verified the program
has a process for corrective actions based on the fidelity monitoring conducted. The program
uses internal fidelity monitoring/corrective action checklists specific to Impact of Crime. A review
of documentation provided and the interviews with the above staff verified there was no corrective
action taken during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The performance evaluations of all trained facilitators included evaluations of the staff's
performance in the delivery of the Impact of Crime curriculum. These included comments on any
strengths or weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 16 weeks for this type of service. Of the 25 youth in the
sample, 16% (4 of 25) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 24 contact hours for this type of service. Of the 25 youth in the
sample, 84% (21 of 25) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 88%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 3 youth
Moderate-High = 9 youth
High = 10 youth
Total Youth in Sample = 25 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 40%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 10 youth
Total Youth in Sample = 25 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 88% (22 of 25) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 40% (10 of 25) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 23 25 92%
Totals 70 85 82%
This SPEP report evaluates Impact of Crime, an intervention delivered at Polk Halfway House.
26 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 25 total youth sampled, all
25 included dosage with end dates in the EBS Module. Of those youth, 4 received at least the
recommended weeks of service. Youth in the sample completed between 8 and 16 weeks of
service, with an average of 13 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 25 total youth
sampled, all 25 included dosage in the EBS Module. Of those youth, 21 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
recommended hours of service. Youth in the sample completed between 20 and 34 hours of
service, with an average of 29 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
RECOMMENDATION(S):
Impact of Crime is an intervention suitable for all youth. The Department recommends providing
this service to every youth in the program.
Polk Halfway House can optimize their SPEP Quality of Service Delivery score by ensuring fidelity
monitoring is completed every month for each facilitator.
Polk Halfway House can optimize their SPEP Amount of Service score by ensuring that dosage
for all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8


summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Polk Halfway House
TrueCore Behavioral Services
(Contract Provider)
2145 Bob Phillips Road
Bartow, Florida 33830
Primary Service: Thinking for a Change
SPEP Review Period: 04/01/20-03/31/21
SPEP Review Conducted: 04/08/21
Florida Department of Juvenile Justice
Report Date: 07/01/2021
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There were three staff trained in the Thinking for a Change curriculum, two Therapists and one
Director of Case Management. Two Therapists facilitated groups and the Director of Case
Management provided fidelity monitoring. One Therapist left employment during the review
period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The manual provided for review was Thinking for a Change 3.1. The manual contains a script for
the facilitator for the groups. It provides twenty-five (25) lessons and includes implementation
guidelines, identifies the order the lessons should be delivered and includes detailed instructions
for the delivery of each lesson.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The Thinking for a Change group was observed via Zoom on 04/06/21. The facilitator provided a
copy of the lesson in advance for this reviewer, "Lesson 7, Paying Attention to Our Thinking". She
followed the lesson plan according to the manual, did not read the lesson verbatim, and used her
own words in an easy to understand manner. The facilitator adhered to the lesson plan in every
aspect.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There had been some facilitator turnover in the past twelve (12) months as a Therapist who was a
Thinking for a Change facilitator PRN left; however, the turnover did not adversely affect the
delivery of the Thinking for a Change (T4C) intervention to the youth. A review of the
documentation provided and an interview with the Director of Case Management provided proof of
gaps in the service of the T4C intervention to the youth during the review period from 05/26/20-
06/22/20 and from 12/09/20-03/15/21.
The program can earn 2 points if there are no unnecessary gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation for fidelity monitoring of Thinking for a Change was reviewed. The program
completed monitoring for each facilitator each month groups were held, except for 11/20.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the Director of Case Management verified the program has a process for
corrective actions based on the fidelity monitoring conducted. The program uses internal fidelity
monitoring/corrective action checklists specific to Thinking for a Change. A review of
documentation provided and an interview with the Director of Case Management verified
there were no corrective actions taken during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The performance evaluations of all trained facilitators found entries evaluating the staff's
performance in the delivery of the Thinking for a Change curriculum. These included comments
on any strengths or weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 15 weeks for this type of service. Of the 11 youth in the
sample, 100% (11 of 11) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates a target of 45 contact hours for this type of service. Of the 11 youth in the
sample, 36% (4 of 11) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 91%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 2 youth
Moderate-High = 4 youth
High = 4 youth
Total Youth in Sample = 11 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 36%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 4 youth
Total Youth in Sample = 11 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 91% (10 of 11) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 36% (4 of 11) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 2 10 20%
Risk Level of Youth Served 23 25 92%
Totals 90 100 90%
This SPEP report evaluates Thinking for a Change, an intervention delivered at Polk Halfway
House.
26 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 11 total youth sampled, all
11 included dosage with end dates in the EBS Module. Of those youth, 11 received at least the
recommended weeks of service. Youth in the sample completed between 16 and 25 weeks of
service, with an average of 20 weeks.
The program earned 2 points for Amount of Service: Contact Hours. Of the 11 total youth
sampled, all 11 included dosage in the EBS Module. Of those youth, 4 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
recommended hours of service. Youth in the sample completed between 34 and 45 hours of
service, with an average of 42 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
RECOMMENDATION(S):
Polk Halfway House can optimize their SPEP Quality of Service Delivery score by ensuring there
are no unnecessary gaps in service and fidelity monitoring is completed every month for each
facilitator.
Polk Halfway House can optimize their SPEP Amount of Service score by ensuring that dosage
for all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8


summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Polk Halfway House
Truecore Behavioral Solutions, LLC
(Contract Provider)
2145 Bob Phillips Road
Bartow, Florida 33830
Primary Service: Impact of Crime
SPEP Review Period: 04/01/21-03/31/22
SPEP Review Conducted: 04/05/22
Florida Department of Juvenile Justice
Report Date: 7/7/22
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The Impact of Crime (IOC) intervention is being reviewed. There is a formal training protocol for
this intervention which staff must attend facilitated by a certified Impact of Crime trainer. The
training consists of reviewing and learning the curriculum and conducting mock groups. During the
review period from 04/01/21 through 03/31/22, there were eight staff formally trained in the IOC
curriculum- four current staff members and four previous staff members. Current trained staff
consists of one Transition Services Manager, one Youth Care Worker II, one Assistant Facility
Administrator and one Clinical Director. One Case Manager, one Youth Care Worker II, one
Recreational Therapist and one Clinical Director all left employment during the review period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the manual "Impact of Crime-Addressing the Harm". There are facilitator and
student manuals. The manual provides a script and instructions for the lessons. The curriculum
also includes implementation guidelines, identifies the order the lessons should be delivered and
includes detailed instructions for the delivery of each lesson. All youth receive the Impact of Crime
intervention as a part of their required programming for Restorative Justice.
The program maintains all SPEP documentation in a tabbed binder separated into sections
organized by intervention. The IOC group is held on Tuesdays and Thursdays, 3:30-4:30 p.m.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Impact of Crime was observed via Microsoft Teams on Tuesday 04/05/22 from 3:30
to 4:30 p.m. There were two staff and nine youth present in the group. The facilitator provided the
reviewer a copy of Lesson One, Introduction to Impact of Crime. The facilitator followed the lesson
plan according to the manual, did not read the lesson verbatim but rather used her own words to
convey the meaning of the discussed topic. She was easy for the youth to follow. The facilitator
adhered to the lesson plan in every aspect.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of the program's documentation provided and an interview with the Clinical Director
confirmed that the program has had some turnover during the review period. The program lost
four staff members during the year. The program's policy regarding anticipated turnover is
addressed by having administrative or other trained staff step in and facilitate the group if
necessary.
The program completed four cohorts of Impact of Crime during the review period with between
five and nine youth in each group. There is one group currently ongoing which began on 12/17/21.
The cohorts ran 01/19/21-04/06/21, 04/08/21-08/18/21, 06/09/21-10/06/21, and 09/08/21-
01/21/22.
One group lost a facilitator during a cohort. The group kept consistently running with no gap in
services after transferring the group to another facilitator.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has a process to perform internal fidelity monitoring for facilitators of Impact of Crime
(IOC). The monitoring was completed by staff who were all formally trained in the curriculum. The
program uses internal fidelity monitoring checklists specific to IOC. The program completed
checklists for all facilitators for each month they facilitated groups. All completed fidelity checklists
are maintained in the SPEP binder under a tab entitled "Fidelity Monitoring for IOC". Each tab had
a copy of the fidelity checklist completed on the facilitators who perfomed groups during the
specific month.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process by which corrective action is acknowledged and demonstrated based
on the fidelity/monitoring of Impact of Crime. The program uses internal fidelity montoring /
corrective action checklists specific to the curriculum. Each completed checklist includes feedback
specifying either the facilitator did not require any corrective action or, if the facilitator did require
corrective action, it would include recommendations for future improvement. The following
month's checklist would then include comments the previous month's recommendations were
applied and demonstrated.
An interview with the Clinical Director and a review of the documentation provided verified the
program created a six-step corrective action plan during the review period to ensure the program
meets the required amount of service-duration and/or the amount of service meets the contact
hours. It appears the program has corrected the problem area for which the Corrective Action was
taken.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of employee records for the four current trained facilitators found all but one staff
received an evaluation within the last year. The Case Manager is new and had not been at the
facility for one year as of the review date. The performance evaluation for the other three staff
found entries of the staff's procifiency in delivering Impact of Crime. These included comments on
any strengths or weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 16 weeks for this type of service. Of the 27 youth in the
sample, 67% (18 of 27) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 24 contact hours for this type of service. Of the 27 youth in the
sample, 96% (26 of 27) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 81%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 3 youth
Moderate-High = 7 youth
High = 12 youth
Total Youth in Sample = 27 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 44%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 12 youth
Total Youth in Sample = 27 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 81% (22 of 27) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 44% (12 of 27) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 20 25 80%
Totals 74 85 87%
This SPEP report evaluates Impact of Crime, an intervention delivered at Polk Halfway House.
27 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 6 points for Amount of Service: Duration. Of the 27 total youth sampled, all
27 included dosage with end dates in the EBS Module. Of those youth, 18 received at least the
recommended weeks of service. Youth in the sample completed between 7 and 20 weeks of
service, with an average of 17 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 27 total youth
sampled, all 27 included dosage in the EBS Module. Of those youth, 26 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
recommended hours of service. Youth in the sample completed between 10 and 35 hours of
service, with an average of 31 hours.
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 93%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Polk Halfway House can optimize their SPEP Quality of Service Delivery score by ensuring
facilitator turnover does not impact groups.
Polk Halfway House can optimize their SPEP Amount of Service score by ensuring dosage for all
youth is recorded accurately in EBS and by ensuring youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2021-22 Polk IOC.pdf