summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Polk Halfway House
Truecore Behavioral Solutions, LLC
(Contract Provider)
2145 Bob Phillips Road
Bartow, Florida 33830
Primary Service: Thinking for A Change
SPEP Review Period: 04/01/21-03/31/22
SPEP Review Conducted: 04/05/22
Florida Department of Juvenile Justice
Report Date: 7/7/22
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The Thinking for A Change (T4C) intervention is being reviewed. There is a formal training
protocol (five days/twenty (20) hours) facilitated a certified Thinking for a Change trainer. The
training consists of reviewing and learning the curriculum manual and conducting mock groups.
During the review period of 04/01/21 through 03/31/22, there were five staff formally trained in the
T4C curriculum. At the time of the annual review, there were three staff trained- one Clinical
Director, one Assistant Facility Administrator and one Youth Care Worker. Two trained Therapists
left employment at Polk Halfway House.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the manual Thinking for A Change, version 3.1. There is a facilitator manual
which provides a script and instructions for twenty-five (25) lessons. The curriculum also includes
implementation guidelines, identifies the order the lessons should be delivered and includes
detailed instructions for the delivery of each lesson. All youth receive the Thinking for a Change
intervention as a part of their programming.
The program maintains all SPEP documentation in a tabbed binder separated into sections and
organized by intervention. Groups are held on Mondays and Fridays from 3:15-4:15 p.m.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
A session of Thinking for A Change was observed via Microsoft Teams on Monday, 04/04/22 from
3:15 to 4:15 p.m. There were one staff member and eight youth present in the group. The
facilitator provided the reviewer a copy of Lesson 17, Problem Solving-Stop and Think. The
facilitator followed the lesson plan according to the manual, did not read the lesson verbatim but
rather used her own words to convey the meaning of the discussed topic. She was very easy for
the youth to follow. The facilitator adhered to the lesson plan in every aspect.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of the documentation provided and an interview with the Clinical Director confirmed the
program has had some turnover during the review period. There were no gaps in the delivery of
Thinking for a Change (T4C). The program's policy regarding anticipated turnover is addressed by
having administrative or other trained staff step in and facilitate the group if necessary. Two
trained Therapists left employment at Polk Halfway House during the review period; however,
there was no disruption in service of the intervention to the youth. When the staff left, the other
trained staff in T4C took over facilitation of the group.
The program has completed four cohorts of Thinking for a Change each group with between six
and eight youth. There is one ongoing group at the time of the annual review, which started on
01/06/22. The completed cohorts were 03/16/21-08/19/21, 05/21/21-09/16/21, 08/03/21-11/14/21
and 10/17/21-02/2/22.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has an in-house process to perform internal fidelity monitoring for facilitators of
Thinking for a Change (T4C). The monitoring was completed by both the Clinical Director and the
Assistant Facility Administrator who are both formally trained in the service. The program uses
internal fidelity monitoring checklists specific to T4C to document the monitoring.
The program had completed checklists for all facilitators for each month they facilitated groups.
The completed fidelity checklists are maintained in the SPEP binder under a tab entitled "Fidelity
Monitoring for T4C". Each tab had a copy of the fidelity checklist completed on the facilitators who
perfomed groups during the specific month.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The program has a process by which corrective action is acknowledged and demonstrated based
on the fidelity/monitoring of Thinking for a Change. The program uses internal fidelity montoring /
corrective action checklists specific to Thinking for A Change. Each completed checklist includes
feedback specifying either the facilitator did not require any corrective action or, if the facilitator
did require corrective action, it would include recommendations for future improvement. The
following month's checklist would then include comments the previous month's recommendations
were applied and demonstrated.
An interview with the Clinical Director and a review of the documentation provided verified the
program created a six-step corrective action plan during the review period. This was to ensure the
program meets the required amount of service-duration and/or the amount of service-contact
hours. It appears the program has been able to correct the problem areas.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of employee records for the three current trained facilitators found these staff received
an evaluation within the last year. The performance evaluations for these staff found entries of the
staff's proficiency in delivering Thinking for A Change. These included comments on any
strengths or weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 15 weeks for this type of service. Of the 25 youth in the
sample, 88% (22 of 25) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 45 contact hours for this type of service. Of the 25 youth in the
sample, 99% (22 of 25) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 80%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 3 youth
Moderate-High = 6 youth
High = 11 youth
Total Youth in Sample = 25 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 44%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 11 youth
Total Youth in Sample = 25 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 80% (20 of 25) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 44% (11 of 25) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 20 25 80%
Totals 91 100 91%
This SPEP report evaluates Thinking for a Change, an intervention delivered at Polk Halfway
House.
27 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 25 total youth sampled, all
25 included dosage with end dates in the EBS Module. Of those youth, 22 received at least the
recommended weeks of service. Youth in the sample completed between 9 and 25 weeks of
service, with an average of 19 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 25 total youth
sampled, all 25 included dosage in the EBS Module. Of those youth, 22 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
recommended hours of service. Youth in the sample completed between 14 and 47 hours of
service, with an average of 42 hours.
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 96%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Polk Halfway House can optimize their SPEP Quality of Service Delivery score by ensuring
facilitator turnover does not impact groups.
Polk Halfway House can optimize their SPEP Amount of Service score by ensuring dosage for all
youth is recorded accurately in EBS and by ensuring youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9


state: FL

facility_name: Polk Halfway House

summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Polk Halfway House
TrueCore Behavioral Solutions, LLC
(Contract Provider)
2145 Bob Phillips Road
Bartow, Florida 33830
Primary Service: Impact of Crime
SPEP Review Period: 04/01/22-03/31/23
SPEP Review Conducted: 04/12/23
Florida Department of Juvenile Justice
Report Date: 9/19/23
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The Impact of Crime (IOC) intervention is being reviewed. There is a formal training protocol for
this intervention which staff must attend, facilitated by either Department of Juvenile Justice
certified trainers or a corporate certified Impact of Crime trainer. The training consists of reviewing
and learning the curriculum manual and conducting mock groups. During the review period from
04/01/22 through 03/31/23, there were five staff formally trained in the IOC curriculum- one
Recreational Therapist, one Case Manager, one Transition Services Manager, one Clinical
Director and one Assistant Facility Administrator (AFA). One Case Manager, one Transition
Services Manager and one Recreational Therapist facilitated group during the review period and
the Clinical Director and AFA provided fidelity monitoring. The Transition Services Manager, the
Case Manager, and the AFA are no longer employed at Polk Halfway House.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the brand name manual "Impact of Crime-Addressing the Harm". There are
facilitator and student manuals. The manual provides a script and instructions for the lessons. The
curriculum also includes implementation guidelines, identifies the order the lessons should be
delivered and includes detailed instructions for the delivery of each lesson. All youth receive the
Impact of Crime (IOC) intervention as a part of their required programming for Restorative Justice.
IOC is a closed group, meaning once a group is started, no other youth can be added to that
ongoing group and new youth would have to be placed in next cohort.
The program maintains all SPEP documentation in a tabbed binder separated into sections
organized by intervention. The IOC group is held on Wednesdays and Saturdays from 3:15-4:15.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Impact of Crime was observed via Microsoft Teams on Wednesday 04/12/23 from
3:15 p.m. to 4:15 p.m. There were two staff and seven youth present in the group. The facilitator
provided a copy of lesson "Personal Accountability" to the reviewer. The facilitator followed the
lesson plan according to the manual, did not read the lesson verbatim but rather used her own
words to convey the meaning of the discussed topic. She was easy for the youth to follow. The
facilitator adhered to the lesson plan in every aspect.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of the documentation provided and an interview with the Clinical Director confirmed the
program has had some turnover during the review period; however, this turnover did not
adversely affect delivery of Impact of Crime (IOC). The program lost one Case Manager, one
Transition Services Manager, and the Assistant Facility Administrator (AFA) during the year. The
Case Manager left after the completion of the group she was running. The Transition Services
Manager only facilitated the group once and then she left and the Case Manager took over that
group. The AFA was only providing fidelity monitoring as needed. The program's policy regarding
anticipated turnover is addressed by having administrative or other trained staff step in and
facilitate the group if necessary. The program completed four cohorts of Impact of Crime during
the review period. The first cohort ran from 12/17/21 through 04/11/22 with eight youth. The
second cohort ran from 04/05/22 through 07/21/22 with seven youth. The third cohort ran from
08/19/22 through 12/09/22 with 11 youth. The fourth cohort ran from 11/02/22 through 02/22/23
with nine youth. There is a cohort ongoing at the time of the annual review which began on
03/22/23 with seven youth.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has an in-house process to perform internal fidelity monitoring for facilitators of
Impact of Crime (IOC). The monitoring was completed by staff who were all formally trained in the
curriculum. The program uses internal fidelity monitoring checklists specific to IOC to document
the monitoring. The program completed checklists for all facilitators for each month they
completed IOC groups. All completed fidelity checklists are maintained in the SPEP binder under
a tab entitled "Fidelity Monitoring for IOC". Each tab had a copy of the fidelity checklist completed
on the facilitators who perfomed groups during the specific month.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process by which corrective action is acknowledged and demonstrated based
on the fidelity/monitoring of Impact of Crime (IOC). The program uses internal fidelity
montoring/corrective action checklists specific to the IOC curriculum. Each completed checklist
includes feedback specifying either the facilitator did not require any corrective action or, if the
facilitator did require corrective action, it would include recommendations for future improvement.
The following month's checklist would then include comments that the previous month's
recommendations were applied and demonstrated. An interview with the Clinical Director and a
review of the documentation confirmed no corrective actions had been noted nor taken during the
review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of employee records for the Recreational Therapist and the Clinical Director who are the
current trained facilitators found that both received a performance evaluation within the last year.
The evaluations for both staff found entries of the staff's skill in delivering Impact of Crime. These
included comments on any strengths or weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 16 weeks for this type of service. Of the 24 youth in the
sample, 100% (24 of 24) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 24 youth in the
sample, 100% (24 of 24) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 18 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 72%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 88%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 4 youth
Moderate-High = 10 youth
High = 7 youth
Total Youth in Sample = 24 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 29%
High Score: 8 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 62%
Table 2
High = 7 youth
Total Youth in Sample = 24 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 88% (21 of 24) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 29% (7 of 24) youth scored High-Risk to reoffend, for a score of 8 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 18 25 72%
Totals 78 85 92%
This SPEP report evaluates Impact of Crime, an intervention delivered at Polk Halfway House.
24 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 24 total youth sampled, all
24 included dosage with end dates in the EBS Module. Of those youth, 24 received at least the
recommended weeks of service. Youth in the sample completed between 16 and 19 weeks of
service, with an average of 17 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 24 total youth
sampled, all 24 included dosage in the EBS Module. Of those youth, 24 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
recommended hours of service. Youth in the sample completed between 31 and 33 hours of
service, with an average of 32 hours.
The program was awarded 18 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 100%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Polk Halfway House can maintain their SPEP Quality of Service Delivery score by continuing the
practices in place at the time of this review.
Polk Halfway House can optimize their SPEP Amount of Service score by ensuring dosage for all
youth is recorded accurately in EBS and by ensuring youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

source: DJJ

report_type: SPEP

file_name: FY2022-23 Polk IOC.pdf

program_name_raw: Polk Halfway House

program_status_note: Closed

service_type: Impact of Crime


summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Polk Halfway House
TrueCore Behavioral Solutions, LLC
(Contract Provider)
2145 Bob Phillips Road
Bartow, Florida 33830
Primary Service: Thinking for A Change
SPEP Review Period: 04/01/22-03/31/23
SPEP Review Conducted: 04/12/23
Florida Department of Juvenile Justice
Report Date: 9/19/23
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The Thinking for A Change (T4C) intervention is being reviewed. There is a formal training
protocol (five days-twenty (20) hours) for this intervention which staff must attend qualified trainer.
The training consists of reviewing and learning the curriculum manual and conducting mock
groups. During the review period from 04/01/22 through 03/31/23, there were two staff formally
trained in the T4C curriculum- one Therapist and one Clinical Director.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the brand name manual Thinking for A Change, version 3.1. There is a
facilitator manual only. The manual provides a script and instructions for twenty-five (25) lessons.
The curriculum also includes implementation guidelines, identifies the order the lessons should be
delivered and includes detailed instructions for the delivery of each lesson. The intervention is
held on Tuesdays, Thursdays and Sundays from 3:15 to 4:15. Records for Thinking for a Change
are kept in a SPEP binder seperated by the intervention and also by indicators pursuant to the
SPEP Guide. All youth receive the Thinking for a Change intervention as a part of their
programming. Thinking for a Change is a closed group meaning once the group is started, no new
youth may be added.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Thinking for a Change was observed via Microsoft Teams on Thursday, 04/13/22
from 3:15 p.m. to 4:15 p.m. There was one staff member and six youth present in the group. The
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
facilitator provided a copy of Lesson 16, Introduction to Problem Solving, to this reviewer. The
facilitator followed the lesson plan according to the manual, did not read the lesson verbatim but
rather used her own words to convey the meaning of the discussed topic. She was very easy for
the youth to follow and ensured each grasped the concept being taught before moving on to the
next concept. The facilitator adhered to the lesson plan in every aspect.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation provided and an interview with the Clinical Director confirmed the
program has had some turnover during the review period. The program lost three Therapists and
one Transition Services Manager during that time. One was facilitating a Thinking for a Change
(T4C) group at the time of their departure. The group continued and facilitation was transitioned to
the Clinical Director. Another Therapist was hired and trained in the curriculum and she is now
also facilitating the group and providing fidelity monitoring. There was no gap in the services of
T4C to the youth. The program's policy regarding anticipated turnover is addressed by having
administrative or other trained staff step in and facilitate the group if necessary. There were four
cohorts of T4C completed during the review period. There is one ongoing group which began on
02/16/23. The completed cohorts ran 01/16/22-05/06/22, 04/15/22-08/18/22, 08/11/22-12/21/22,
and 09/13/22-01/09/23.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has an in-house process to perform internal fidelity monitoring for facilitators of
Thinking for a Change (T4C). The monitoring was completed by the Regional Compliance
Director, three Therapists and one Transition Services Manager. Since that time, the program has
lost three Therapists and one Transition Services Manager. Monitoring continues to be done by
the remaining new Therapist, The Regional Compliance Manager and the Clinical Director. The
program uses internal fidelity monitoring checklists specific to T4C to document the monitoring.
The program had completed checklists for all facilitators for each month they completed T4C
groups except for the month of 01/23. The completed fidelity checklists are maintained in the
SPEP binder under a tab entitled "Fidelity Monitoring for T4C". Each tab had a copy of the fidelity
checklist completed on the facilitators who perfomed groups during the specific month.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process by which corrective action is acknowledged and demonstrated based
on the fidelity/monitoring of Thinking for a Change (T4C). The program uses internal fidelity
montoring/corrective action checklists specific to T4C. Each completed checklist includes
feedback specifying either the facilitator did not require any corrective action or, if the facilitator
did require corrective action, it would include recommendations for future improvement. The
following month's checklist would then include comments that the previous month's
recommendations were applied and demonstrated.
An interview with the Clinical Director and a review of the documentation provided verified the
program did not take any corrective actions during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of employee records for the current Therapist confirmed a ninety (90)-day evaluation
was completed for her on 02/03/23 and the annual evaluation for the Clinical Director was
completed on 12/06/22. The performance evaluations for these staff found entries of the staff's
skill in delivering Thinking for a Change. These included comments on any strengths or
weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 15 weeks for this type of service. Of the 24 youth in the
sample, 100% (24 of 24) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 45 contact hours for this type of service. Of the 24 youth in the
sample, 100% (24 of 24) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 18 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 72%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 88%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 4 youth
Moderate-High = 10 youth
High = 7 youth
Total Youth in Sample = 24 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 29%
High Score: 8 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 62%
Table 2
High = 7 youth
Total Youth in Sample = 24 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 88% (21 of 24) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 29% (7 of 24) youth scored High-Risk to reoffend, for a score of 8 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 18 25 72%
Totals 93 100 93%
This SPEP report evaluates Thinking for A Change, an intervention delivered at Polk Halfway
House.
24 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 24 total youth sampled, all
24 included dosage with end dates in the EBS Module. Of those youth, 24 received at least the
recommended weeks of service. Youth in the sample completed between 16 and 20 weeks of
service, with an average of 18 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 24 total youth
sampled, all 24 included dosage in the EBS Module. Of those youth, 24 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
recommended hours of service. Youth in the sample completed between 45 and 53 hours of
service, with an average of 46 hours.
The program was awarded 18 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 100%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Polk Halfway House can optimize their SPEP Quality of Service Delivery score by ensuring
facilitator turnover does not impact groups and fidelity monitoring is completed every month for
each facilitator.
Polk Halfway House can optimize their SPEP Amount of Service score by ensuring dosage for all
youth is recorded accurately in EBS and by ensuring youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2022-23 Polk T4C.pdf

service_type: Thinking for a Change