summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Tampa Residential Facility
TrueCore Behavioral Solutions, LLC
(Contract Provider)
9508 East Columbus Drive
Tampa, Florida 33619‐7715
Primary Service: Thinking for a Change
SPEP Review Date(s): 07/30/19‐08/02/19
Florida Department of Juvenile Justice
Report Date(s): 08/04/19
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
 Service Type: service type points are awarded based on the group category that the
service falls within
 Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
 Amount of Service: or “dosage” refers to the amount and duration of services that were
received
 Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change. The program was awarded 30
points because the Primary Service is identified as a Group 5 Service. The specific Sub-
Component Service Type identified is Cognitive Behavioral Therapy. The Primary Service was
identified as this type of service as it is intended to correct faulty cognitions and perceptions and
provides skills individuals can use to monitor thought patterns and correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 6 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There were seven staff formally trained in the Thinking for a Change (T4C) curriculum, four are no
longer employed. Three received training from a qualified trainer who was formally trained in the
T4C curriculum. Two consistently facilitated groups during the review period, one Assistant
Facility Administrator and one Master Control Technician.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The manual provided to this reviewer was Thinking for a Change 3.1. The manual contains a
script for the facilitator for the groups. It provides 25 lessons and includes implementation
guidelines, identifies the order the lessons should be delivered in and includes detailed
instructions for the delivery of each lesson.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The Thinking for a Change group was observed on Thursday from 5:00-6:00 pm. The facilitator
provided a copy of the lesson to the observer from the program manual. The facilitators followed
the lesson according to the manual, did not read the lesson verbatim and used her own words to
convey the lesson to the group. Both facilitator and co-facilitator were easy to follow and adhered
to the lesson plan in every aspect.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 0 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation provided and an interview with the Regional Compliance Manager
confirmed there has been facilitator turnover in the past twelve (12) months which adversely
affected services to the youth. There was a gap in services to the youth from 03/07/19-07/30/19.
The program, in the event of turnover, would normally utilize other trained staff to facilitate the
curriculum; however, there were no other trained staff to utilize at that time.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation for fidelity monitoring of Thinking for a Change was reviewed. The program
was missing fidelity monitoring for the months of 10/18,11/18, 04/19, 05/19, 06/19, and 07/19.
This was partially due to a gap in services to the youth during the months of 04/19-07/19.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the Regional Compliance Manager verified the program has a process for
corrective actions based on the fidelity monitoring conducted. The program uses internal fidelity
monitoring/corrective action checklists specific to Thinking for a Change. A review of
documentation provided and an interview with the Regional Compliance Manager verified there
was no corrective action noted nor taken.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The performance evaluations of two trained facilitators found evidence the performance
evaluations were completed, however, they did not include assessment of the facilitator
specifically on facilitation of Thinking for a Change.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service ‐ Duration
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates the target duration of 15 weeks for this type of service. Of the 56 youth in the
sample, 25% (14 of 56) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates a target of 45 contact hours for this type of service. Of the 56 youth in the
sample, 21% (12 of 56) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 84%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 7youth
Moderate-High = 19youth
High = 21youth
Total Youth in Sample = 56youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 38%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 21youth
Total Youth in Sample = 56youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 84% (47 of 56) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 38% (21 of 56) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 2 10 20%
Amount of Service: Contact Hours 2 10 20%
Risk Level of Youth Served 20 25 80%
Totals 69 100 69%
This SPEP report evaluates Thinking for a Change, an intervention delivered at Tampa
Residential Facility.
62 youth completed the program during the review period. Of those youth, 56 were included in
the sample.
The program scored Medium for Quality of Service Delivery.
The program earned 2 points for Amount of Service: Duration. Of the 56 total youth sampled, only
14 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 14
received at least the recommended weeks of service. Youth in the sample completed between 18
and 31 weeks of service, with an average of 25 weeks.
The program earned 2 points for Amount of Service: Contact Hours. Of the 56 total youth
sampled, only 14 included dosage in the EBS Module. Of those youth with correct dosage, 12
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
received at least the recommended hours of service. Youth in the sample completed between 28
and 47 hours of service, with an average of 44 hours.
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 74%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Tampa Residential Facility can optimize their SPEP Quality of Service Delivery score by ensuring
facilitator turnover does not impact groups; by ensuring fidelity monitoring is completed every
month for each facilitator; and by ensuring facilitators' performance evaluations include discussion
of their skill in delivering this primary service.
Tampa Residential Facility can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9


state: FL

facility_name: Tampa Residential Facility

summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Tampa Residential Facility
TrueCore Behavioral Solutions, LLC
(Contract Provider)
9508 East Columbus Drive
Tampa, Florida 33619-7715
Primary Service: Impact of Crime
SPEP Review Period: 11/01/19-10/31/20
SPEP Review Conducted: 11/16/20
Florida Department of Juvenile Justice
Report Date: 12/30/20
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There were eight staff trained in the Impact of Crime (IOC) curriculum- one Assistant Facility
Administrator, two Directors of Case Management, one Transition Services Manager, three Case
Managers, and one Master Control Technician. Four staff facilitated the Impact of Crime group
during the review period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The manual provided for review was Impact of Crime-Addressing the Harm. The manual contains
a script for the facilitator for the groups. It provides lessons and includes implementation
guidelines, identifies the order the lessons should be delivered in and includes detailed
instructions for the delivery of each lesson.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The Impact of Crime group was observed on 11/16/20 from 11:30-12:30. The facilitator provided a
copy of the lesson to the observer from the program manual. The facilitator followed the lesson
according to the manual, did not read the lesson verbatim and used his own words to convey the
lesson to the group. He was easy to follow and adhered to the lesson plan in every aspect.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation provided and an interview with the Clinical Director revealed there has
been no facilitator turnover during the review period which adversely affected the Impact of Crime
group. There were no gaps in the service of Impact of Crime to the youth.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation for fidelity monitoring of Impact of Crime (IOC) was reviewed. The program
completed fidelity monitoring for the IOC group for each month during the review period. There
was no missed monitoring. The monitoring was completed by the Director of Case Management.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the Director of Case Management and the documentation provided verified the
program has a process for corrective actions based on the fidelity monitoring conducted. The
program uses internal fidelity monitoring/corrective action checklists specific to Impact of Crime.
The program had no corrective action during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The performance evaluations of the trained facilitators were reviewed. The evaluations were all
completed and contained specific information and feedback regarding the staff's delivery of
Impact of Crime.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates the target duration of 16 weeks for this type of service. Of the 42 youth in the
sample, 43% (18 of 42) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates a target of 24 contact hours for this type of service. Of the 42 youth in the
sample, 57% (24 of 42) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 79%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 3 youth
Moderate-High = 11 youth
High = 19 youth
Total Youth in Sample = 42 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 45%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 19 youth
Total Youth in Sample = 42 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 79% (33 of 42) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 45% (19 of 42) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 4 10 40%
Amount of Service: Contact Hours 4 10 40%
Risk Level of Youth Served 20 25 80%
Totals 68 85 80%
This SPEP report evaluates Impact of Crime, an intervention delivered at Tampa Residential
Facility .
46 youth completed the program during the review period. Of those youth, 42 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 4 points for Amount of Service: Duration. Of the 42 total youth sampled, only
30 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 18
received at least the recommended weeks of service. Youth in the sample completed between 4
and 30 weeks of service, with an average of 15 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
The program earned 4 points for Amount of Service: Contact Hours. Of the 42 total youth
sampled, only 31 included dosage in the EBS Module. Of those youth with correct dosage, 24
received at least the recommended hours of service. Youth in the sample completed between 5
and 32 hours of service, with an average of 25 hours.
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 86%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Tampa Residential Facility can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Tampa Residential Facility can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8

source: DJJ

report_type: SPEP

program_name_raw: Tampa Residential Facility

program_status_note: Closed

service_type: Impact of Crime


summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Tampa Residential Facility
TrueCore Behavioral Solutions, LLC
(Contract Provider)
9508 East Columbus Drive
Tampa, Florida 33619-7715
Primary Service: Pathways to Self-Discovery and Change
SPEP Review Period: 11/01/19-10/31/20
SPEP Review Conducted: 11/16/20
Florida Department of Juvenile Justice
Report Date: 12/30/20
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Pathways to Self-Discovery and Change.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There were seven staff trained in the Pathways to Self-Discovery and Change curriculum, one
Director of Clinical Services, one Assistant Clinical Director and six Therapists. Six Therapists
facilitated the Pathways curriculum during the review period, with the Director of Clinical Services
and Assistant Clinical Director providing fidelity monitoring.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The manual provided for review was Pathways to Self-Discovery and Change. The manual
contains a script for the facilitator for the groups. It provides 32 lessons and includes
implementation guidelines, identifies the order the lessons should be delivered in and includes
detailed instructions for the delivery of each lesson.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The Pathways to Self-Discovery and Change group was observed on 11/16/20 from 2:30-3:30
p.m. The facilitator provided a copy of the lesson to the observer from the program manual. The
facilitator followed the lesson according to the manual, did not read the lesson verbatim and used
her own words to convey the lesson to the group. She was easy to follow and adhered to the
lesson plan in every aspect.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation provided and an interview with the Assistant Clinical Director
confirmed there has been no facilitator turnover during the review period which adversely affected
the Pathways to Self-Discovery and Change group. There were no gaps in services to the youth
during the review period.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation for fidelity monitoring was reviewed. The program completed fidelity
monitoring for Pathways to Self-Discovery and Change for each facilitator monthly during the
review period. There was no missed monitoring. The monitoring was completed by the Director of
Clinical Services and the Assistant Clinical Director.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the Assistant Clinical Director and a review of documentation provided verified
the program has a process for corrective actions based on the fidelity monitoring conducted. The
program uses internal fidelity monitoring/corrective action checklists specific to Pathways to Self-
Discovery and Change. The program had no corrective action during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The performance evaluations of the trained facilitators were reviewed. The evaluations were all
completed and contained specific information and feedback regarding the staff's delivery of
Pathways to Self-Discovery and Change.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 24 weeks for this type of service. Of the 41 youth in the
sample, 80% (33 of 41) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 40 contact hours for this type of service. Of the 41 youth in the
sample, 93% (38 of 41) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 78%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 2 youth
Moderate-High = 13 youth
High = 17 youth
Total Youth in Sample = 41 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 41%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 17 youth
Total Youth in Sample = 41 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 78% (32 of 41) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 41% (17 of 41) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 20 25 80%
Totals 86 95 91%
This SPEP report evaluates Pathways to Self-Discovery and Change, an intervention delivered at
Tampa Residential Facility.
46 youth completed the program during the review period. Of those youth, 41 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 41 total youth sampled, only
39 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 33
received at least the recommended weeks of service. Youth in the sample completed between 7
and 28 weeks of service, with an average of 24 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
The program earned 8 points for Amount of Service: Contact Hours. Of the 41 total youth
sampled, only 40 included dosage in the EBS Module. Of those youth with correct dosage, 38
received at least the recommended hours of service. Youth in the sample completed between 9
and 47 hours of service, with an average of 43 hours.
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 96%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Tampa Residential Facility can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Tampa Residential Facility can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8

service_type: Pathways to Self‐Discovery and Change


summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Tampa Residential Facility
TrueCore Behavioral Solutions, LLC
(Contract Provider)
9508 East Columbus Drive
Tampa, Florida 33619-7715
Primary Service: Thinking for a Change
SPEP Review Period: 11/01/19-10/31/20
SPEP Review Conducted: 11/16/20
Florida Department of Juvenile Justice
Report Date: 12/30/20
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There were seven staff trained in the Thinking for a Change curriculum- two Directors of Case
Management, one Assistant Facility Administrator, three Case Managers and one Master Control
Technician. Four staff facilitated Thinking for a Change during the review period. One Director of
Case Management took another position and is no longer employed at Tampa Residential
Facility.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The manual provided for review was Thinking for a Change 3.1. The manual contains a script for
the facilitator for the groups. It provides 25 lessons and includes implementation guidelines,
identifies the order the lessons should be delivered in and includes detailed instructions for the
delivery of each lesson.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The Thinking for a Change group was observed on 11/16/20 from 1:00-2:00 p.m. The facilitator
provided a copy of the lesson to the observer from the program manual. The facilitator followed
the lesson according to the manual, did not read the lesson verbatim and used her own words to
convey the lesson to the group. The facilitator was easy to follow and adhered to the lesson plan
in every aspect.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation provided and an interview with the Assistant Clinical Director
confirmed there has been no facilitator turnover during the review period which adversely affected
the Thinking for a Change group. There were no gaps in services to the youth.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation for fidelity monitoring was reviewed. The program completed fidelity
monitoring for Thinking for a Change (T4C) each month for each facilitator during the review
period. There was no missed monitoring. The monitoring was completed by the Director of Case
Management, Assistant Facility Administrator and a Case Manager all trained in the T4C
curriculum.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the Assistant Clinical Director and documentation provided verified the program
has a process for corrective actions based on the fidelity monitoring conducted. The program
uses internal fidelity monitoring/corrective action checklists specific to Thinking for a Change. The
program had no corrective action during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The performance evaluations of the trained facilitators were reviewed. The evaluations were all
completed and contained specific information and feedback regarding the staff's delivery of
Thinking for a Change.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates the target duration of 15 weeks for this type of service. Of the 45 youth in the
sample, 49% (22 of 45) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates a target of 45 contact hours for this type of service. Of the 45 youth in the
sample, 29% (13 of 45) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 80%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 3 youth
Moderate-High = 14 youth
High = 19 youth
Total Youth in Sample = 45 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 42%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 19 youth
Total Youth in Sample = 45 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 80% (36 of 45) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 42% (19 of 45) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 4 10 40%
Amount of Service: Contact Hours 2 10 20%
Risk Level of Youth Served 20 25 80%
Totals 81 100 81%
This SPEP report evaluates Thinking for a Change, an intervention delivered at Tampa
Residential Facility.
46 youth completed the program during the review period. Of those youth, 45 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 4 points for Amount of Service: Duration. Of the 45 total youth sampled, only
31 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 22
received at least the recommended weeks of service. Youth in the sample completed between 3
and 25 weeks of service, with an average of 18 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
The program earned 2 points for Amount of Service: Contact Hours. Of the 45 total youth
sampled, only 31 included dosage in the EBS Module. Of those youth with correct dosage, 13
received at least the recommended hours of service. Youth in the sample completed between 3
and 58 hours of service, with an average of 37 hours.
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 86%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Tampa Residential Facility can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Tampa Residential Facility can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8

service_type: Thinking for a Change


summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Tampa Juvenile Residential Facility
TrueCore Behavioral Solutions, LLC
(Contract Provider)
9508 East Columbus Drive
Tampa, Florida 33619
Primary Service: Thinking for A Change
SPEP Review Period: 11/01/20-10/31/21
SPEP Review Conducted: 11/09/21
Florida Department of Juvenile Justice
Report Date: 12/3/21
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The Thinking for A Change (T4C) intervention is being reviewed. The program maintains all SPEP
documentation in a tabbed binder separated into sections, then organized by intervention. There
is a formal training protocol (twenty hours over five days) for this intervention which staff must
attend. The training consists of reviewing and learning the curriculum manual and conducting
mock groups. During the review period from 11/01/20-10/31/21, there were six staff formally
trained in the T4C curriculum- one Director Case Management, one Transition Service Manager,
and four Case Managers. Four staff facilitated during the review period- one Director Case
Management, and three Case Managers. One Case Manager left during the review period but not
during a cohort of T4C.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the brand name manual Thinking for A Change, version 3.1.There is a
facilitator manual only. The manual provides a script and instructions for twenty-five lessons. The
curriculum also includes implementation guidelines, identifies the order the lessons should be
delivered and includes detailed instructions for the delivery of each lesson.
All youth receive the Thinking for a Change intervention as a part of their programming. Groups
are held on Mondays, Wednesdays and Fridays from 11:30 to 12:30 p.m.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Thinking for A Change was observed via Zoom on Wednesday 11/10/21 from 11:30
to 12:30 p.m. There was one staff and seven youth present in the group. The facilitator provided a
copy of the lesson, Problem Solving. The facilitator followed the lesson plan according to the
manual, did not read the lesson verbatim but rather used her own words to convey the meaning of
the discussed topic. She was very easy for the youth to follow. The facilitator adhered to the
lesson plan in every aspect.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of the program's documentation provided and an interview with the Director of Case
Management confirmed the program has had some turnover of facilitators for Thinking for a
Change (T4C) during the review period. The program lost one Case Manager after a cohort was
completed. The turnover did not adversely affect the services of T4C to the youth. The program's
policy regarding anticipated turnover is addressed by having administrative or other trained staff
step in and facilitate the group if necessary.
There were four cohorts of T4C completed during the review period and one group is currently
ongoing which started on 08/30/21.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has an in-house process to perform internal fidelity monitoring. The monitoring was
completed by the Director of Case Management who is formally trained in Thinking for a Change
(T4C). The program uses internal fidelity monitoring checklists specific to T4C to document the
monitoring. The program had completed checklists for all facilitators for each month they
completed their T4C groups. All completed fidelity checklists are maintained in the SPEP binder
under a tab entitled "Fidelity Monitoring for T4C". Each tab had a copy of the fidelity checklist
completed on the facilitator(s) who perfomed groups during the specific month.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process by which corrective action is acknowledged and demonstrated based
on the fidelity/monitoring of the primary service. The program uses internal fidelity
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
montoring/corrective action checklists specific to the Thinking for A Change curriculum. Each
completed checklist includes feedback specifying either the facilitator did not require any
corrective action or, if the facilitator did require corrective action, it would include
recommendations for future improvement. The following month's checklist would then include
comments the previous month's recommendations were applied and demonstrated. However, an
interview with the Director of Case Management and a review of the documentation provided
verified the program did not take any corrective action during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of employee records for the six current trained facilitators found these staff had received
an evaluation within the last year. The performance evaluations for these staff found entries of the
staff's proficiency in delivering Thinking for A Change. These included comments on any
strengths or weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 15 weeks for this type of service. Of the 29 youth in the
sample, 93% (27 of 29) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 45 contact hours for this type of service. Of the 29 youth in the
sample, 83% (24 of 29) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 93%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 4 youth
Moderate-High = 7 youth
High = 16 youth
Total Youth in Sample = 29 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 55%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 16 youth
Total Youth in Sample = 29 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 93% (27 of 29) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 55% (16 of 29) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 23 25 92%
Totals 94 100 94%
This SPEP report evaluates Thinking for a Change, an intervention delivered at Tampa Juvenile
Residential Facility.
30 youth completed the program during the review period. Of those youth, 29 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 29 total youth sampled, only
27 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 27
received at least the recommended weeks of service. Youth in the sample completed between 16
and 33 weeks of service, with an average of 21 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 8 points for Amount of Service: Contact Hours. Of the 29 total youth
sampled, only 27 included dosage in the EBS Module. Of those youth with correct dosage, 24
received at least the recommended hours of service. Youth in the sample completed between 32
and 72 hours of service, with an average of 45 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
RECOMMENDATION(S):
Tampa Juvenile Residential Facility can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Tampa Juvenile Residential Facility can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2021-22 Tampa T4C.pdf


summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Tampa Juvenile Residential Facility
TrueCore Behavioral Solutions, LLC
(Contract Provider)
9508 East Columbus Drive
Tampa, Florida 33619
Primary Service: Impact of Crime
SPEP Review Period: 11/01/20-10/31/21
SPEP Review Conducted: 11/09/21
Florida Department of Juvenile Justice
Report Date: 11/29/21
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The Impact of Crime (IOC) intervention is being reviewed. The program maintains all SPEP
documentation in a tabbed binder which is separated into sections and organized by intervention.
There is a twenty-eight hour formal training protocol by the Department of Juvenile Justice. The
training consists of reviewing and learning the manual and conducting mock groups. During the
review period from 11/01/20-10/31/21, there were four staff formally trained in the IOC curriculum-
one Transtion Service Manager and three Case Managers. All four staff facilitated during the
review period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the brand name manual "Impact of Crime-Addressing the Harm". The manual
provides a script and instructions for the lessons. The curriculum also includes implementation
guidelines, identifies the order the lessons should be delivered and includes detailed instructions
for the delivery of each lesson. All youth receive the Impact of Crime intervention as a part of their
programming. The IOC group is held on Tuesdays and Thursdays from 11:20-12:20 p.m.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Impact of Crime was observed on Tuesday 11/09/21 from 11:20-12:20 p.m. There
were two staff and seven youth present in the group. The facilitator provided a copy of the lesson,
Why Youth Join Gangs. The facilitator followed the lesson plan according to the manual, did not
read the lesson verbatim but rather used his own words to convey the meaning of the discussed
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
topic. He was very easy for the youth to follow. The facilitator adhered to the lesson plan in every
aspect.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of the program's documentation and an interview with the Director of Case Management
confirmed the program has had no facilitator turnover during the review period. The program's
policy regarding anticipated turnover is addressed by ensuring there are enough staff trained in
Impact of Crime (IOC) they could take over and facilitate the group in the case of a staff member
leaving.
There were three cohorts of IOC completed during the review period and one group is currently
ongoing which started on 08/31/21.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has an in-house process to perform internal fidelity monitoring. The monitoring was
completed by the Director of Case Management who is formally trained in Impact of Crime (IOC).
The program uses internal fidelity monitoring checklists specific to IOC to document the
monitoring. The program had completed checklists for all facilitators for each month they
facilitated IOC groups. All completed fidelity checklists are maintained in the SPEP binder under a
tab entitled "Fidelity Monitoring for IOC". Each tab had a copy of the fidelity checklist completed
on the facilitator(s) who performed groups during the specific month.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process by which corrective action is acknowledged and demonstrated based
on the fidelity/monitoring of Impact of Crime. The program uses internal fidelity
montoring/corrective action checklists specific to the Impact of Crime curriculum. Each completed
checklist includes feedback specifying either the facilitator did not require any corrective action or
if the facilitator did require corrective action, it would include recommendations for future
improvement. The following month's checklist would then include comments the previous month's
recommendations were applied and demonstrated.
An interview with the Director of Case Management and a review of the documentation provided
verified the program did not take any corrective action during the review period.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of employee records for the four current trained facilitators found these staff received an
evaluation within the last year. The Performance Evaluation for these staff found entries of the
staff's ability in delivering Impact of Crime. These included comments on any strengths or
weaknesses of which the facilitator delivering the curriculum should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 16 weeks for this type of service. Of the 29 youth in the
sample, 90% (26 of 29) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 24 contact hours for this type of service. Of the 29 youth in the
sample, 90% (26 of 29) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 93%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 4 youth
Moderate-High = 7 youth
High = 16 youth
Total Youth in Sample = 29 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 55%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 16 youth
Total Youth in Sample = 29 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 93% (27 of 29) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 55% (16 of 29) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 23 25 92%
Totals 79 85 93%
This SPEP report evaluates Impact of Crime, an intervention delivered at Tampa Juvenile
Residential Facility.
30 youth completed the program during the review period. Of those youth, 29 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 29 total youth sampled, only
27 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 26
received at least the recommended weeks of service. Youth in the sample completed between 5
and 25 weeks of service, with an average of 18 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 8 points for Amount of Service: Contact Hours. Of the 29 total youth
sampled, only 27 included dosage in the EBS Module. Of those youth with correct dosage, 26
received at least the recommended hours of service. Youth in the sample completed between 4
and 40 hours of service, with an average of 31 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
RECOMMENDATION(S):
Tampa Juvenile Residential Facility can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Tampa Juvenile Residential Facility can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2021-22 Tampa IOC.pdf