state: FL

facility_name: Tampa Residential Facility

summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Tampa Residential Facility
TrueCore Behavioral Solutions
(Contract Provider)
9508 East Colubus Drive
Tampa , Florida 33619-7715
Primary Service: Impact of Crime
SPEP Review Period: 11/1/2022-10/31/2023
SPEP Review Conducted: 11/7/2023
Florida Department of Juvenile Justice
Report Date: 6/19/2024
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The intervention being reviewed is Impact of Crime (IOC). The Department of Juvenile Justice
provides the formal training for the intervention. There were ten (10) staff formally trained in the
IOC curriculum during this annual review period: one director of case management (DCM), two
transition service managers (TSM), and seven Case Managers (CM). Nine of the trained staff
facilitated the IOC groups. The DCM conducted all fidelity monitoring.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program has brand name manuals for the delivery of Impact of Crime curriculum. The manual
contains worksheets for students and suggestions for facilitators. It includes a script,
implementation guidelines, identifies the order the lessons should be delivered, and includes
detailed instructions for each lesson.
Impact of Crime is a closed group and is held on Tuesday and Thursday. The program maintains
their records in a Standardized Program Evaluation Protocol (SPEP) binder. All youth at Tampa
Residential Facility receive Impact of Crime.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Impact of Crime was observed on Thursday 7/6/2023 from 11:25am-12:22pm. The
facilitator provided a copy of the lesson plan from manual prior to the observation. The lesson for
the day was Chapter Two: Owning One’s Action. In attendance were one facilitator, one co-
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
facilitator, and ten (10) youth. The youth were receptive to the information being delivered. The
session began with a review of the last lesson and the youth all appeared eager to participate.
There was a minor interruption by a few of the youth which did not last long, as the facilitators
redirected them to get back on task. The facilitators were knowledgeable of the material which
was demonstrated by the way they used their own words in the delivery of the lesson. The
facilitators were encouraged to incorporate more visual aids and to assign tasks to their more
active youth.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation and an interview with the director of case management confirmed one
case manager left the program during the annual review period. This turnover did not adversely
affect the delivery of Impact of Crime. The program's policy regarding turnover is for anticipated
turnover to be addressed by ensuring any potentially affected groups are co-facilitated, if possible,
at the outset and that all staff are trained in these curriculums. There were no gaps in services
during the annual review period.
There were six cohorts completed during the annual review period. The first cohort started prior to
the review. The dates of the first cohort were 8/16/2022 – 12/13/2022, which started with ten (10)
youth and ended with nine youth. One youth was arrested on 11/29/22 and dropped from the
group after missing three consecutive sessions.
The second cohort was conducted 12/6/2022 – 3/23/2023, which started with ten (10) youth and
ended with nine youth. One youth was arrested on 1/10/2023 and dropped from the cohort for
missing three consecutive sessions.
The third cohort was conducted 1/24/2023 – 5/16/2023, which started with ten (10) youth and
ended with ten (10) youth.
The fourth cohort was conducted 3/28/2023 – 7/13/2023, which started with ten (10) youth and
ended with ten (10) youth.
The fifth cohort was conducted 5/23/2023 – 9/7/2023, which started with ten (10) youth and ended
with ten (10) youth.
The sixth cohort was conducted 6/13/2023 – 10/5/2023, which started with ten (10) youth and
ended with ten (10) youth.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has an internal process to conduct fidelity monitoring for facilitators of Impact of
Crime (IOC) by utilizing a fidelity checklist. The director of case management preformed all fidelity
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
monitoring for this annual review period. The program completed twenty-six (26) fidelity
monitorings. Fidelity monitoring for IOC was conducted each month of the review period.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the director of case management verified that the program has a process for
corrective actions based on fidelity monitoring conducted. The program uses internal fidelity
monitoring/corrective action checklists specific to Impact of Crime. Each fidelity monitoring
checklist includes a section for feedback specifying the facilitator did not require any corrective
action or included recommendations for future improvement. The following month's checklist
would include comments the previous month's recommendations were applied and demonstrated.
There were no corrective actions noted nor taken for facilitators of Impact of Crime.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of annual evaluations of facilitators for the Impact of Crime curriculum found an
evaluation of staff’s performance in delivering the primary service. These included comments on
any strengths or weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 16 weeks for this type of service. Of the 36 youth in the
sample, 100% (36 of 36) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 36 youth in the
sample, 100% (36 of 36) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 83%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 7 youth
Moderate-High = 10 youth
High = 13 youth
Total Youth in Sample = 36 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 36%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 13 youth
Total Youth in Sample = 36 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 83% (30 of 36) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 36% (13 of 36) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 20 25 80%
Totals 80 85 94%
This SPEP report evaluates Impact of Crime, an intervention delivered at Tampa Residential
Facility.
46 youth completed the program during the review period. Of those youth, 36 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 36 total youth sampled, all
36 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 36
received at least the recommended weeks of service. Youth in the sample completed between 16
and 18 weeks of service, with an average of 17 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 10 points for Amount of Service: Contact Hours. Of the 36 total youth
sampled, all 36 included dosage in the EBS Module. Of those youth with correct dosage, 36
received at least the recommended hours of service. Youth in the sample completed between 31
and 32 hours of service, with an average of 32 hours.
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 100%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Tampa Residential Facility can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Tampa Residential Facility can optimize their SPEP Amount of Service score by ensuring dosage
for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted dosage
of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

source: DJJ

report_type: SPEP

file_name: FY2023-24 Tampa IOC.pdf

program_name_raw: Tampa Residential Facility

program_status_note: Closed

service_type: Impact of Crime


summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Tampa Residential Facility
TrueCore Behavioral Solutions
(Contract Provider)
9508 East Colubus Drive
Tampa , Florida 33619-7715
Primary Service: Pathways to Self-Discovery and Change
SPEP Review Period: 11/1/2022-10/31/2023
SPEP Review Conducted: 11/7/2023
Florida Department of Juvenile Justice
Report Date: 6/20/2024
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Pathways to Self Discovery and Change .
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The intervention being reviewed is Pathways to Self-Discovery and Change (Pathways). There
were ten (10) therapists formally trained in the Pathways curriculum during this annual review
period: one clinical director, three assistant clinical directors, and six therapists. Two assistant
clinical directors and two therapists delivered Pathways during the annual review. The clinical
director and an assistant clinical director conducted fidelity monitoring.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The manual provided for review was Pathways to Self-Discovery and Change. The manual
contains a script for the facilitator for the groups. It provides thirty-two (32) lessons and includes
implementation guidelines, identifies the order the lessons should be delivered in and includes
detailed instructions for the delivery of each lesson.
Pathways is an open group which addresses both the mental health and substance abuse
aspects of therapy and is held on Tuesdays and Thursdays. The program maintains their records
in a Standardized Program Evaluation Protocol (SPEP) binder. All youth at Tampa Residential
Facility receive Pathways.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Pathways to Self-Discovery and Change was observed on 7/6/2023 which started at
1:35 p.m.–2:30 p.m. There were nine youth, one therapist and three staff in attendance. The
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
lesson for the session was "Power of Thoughts". The youth were actively engaged in this session,
as they all participated and seemed to enjoy the discussions. Each youth took turns writing on the
chart paper board and reading, and they also participated in a role play activity. The facilitator was
very knowledgeable and held the attention of the group.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation and an interview with the clinical director confirmed there was turnover
during this annual review; however, this turnover did not adversely affect the delivery of
Pathways. The program's policy regarding turnover is for anticipated turnover to be addressed by
ensuring any potentially affected groups are co-facilitated, if possible, at the outset and that all
therapists are trained in the curriculum.
There were seven cohorts of Pathways in total. The first cohort started prior to the annual review
period. The dates of the first cohort were 5/31/2022-11/17/2022. The Mental Health (MH) group
started with four youth and ended with four youth, and the Substance Abuse (SA) group started
with three youth and ended with two youth (a youth was arrested on 11/15/22).
The second cohort also started prior to the annual review period. The dates were 9/13/2022-
2/28/2023. The SA group started with eight youth and ended with eight youth. The MH group
started with two youth and ended with one youth (a youth was released on 1/24/23).
The third cohort was delivered 11/17/2022-5/2/2023. The MH group started with three youth and
ended with two youth (a youth was discharged on 1/24/23). The SA group started with seven (7)
youth and ended with four youth (two youth were discharged on 2/7/23 and one youth was
discharged on 3/28/23).
The fourth cohort was delivered 1/10/2023-6/27/2023. The MH group started with two youth and
ended with two youth. The SA group started with eight youth and ended with eight youth.
The fifth cohort was delivered 3/2/2023-8/17/2023. The SA group started with nine youth and
ended with nine youth. The MH group started with one youth and ended with one youth.
The sixth cohort was delivered 5/4/2023-10/24/2023. The SA group started with eight youth and
ended with eight youth. The MH group started with two youth and ended with two youth.
The seventh cohort started on 6/29/23 and is currently ongoing.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation for internal fidelity monitoring Pathways to Self-Discovery and Change
facilitators was reviewed. The program completed thirty-three (33) internal fidelity monitorings for
the review period. The program completed internal fidelity monitoring for Pathways each month
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
for each facilitator. The monitoring was completed by the clinical director and assistant clinical
director.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the clinical director verified that the program has a process for corrective actions
based on fidelity monitoring conducted. The program uses internal fidelity monitoring/corrective
action checklists specific to Pathways to Self-Discovery and Change.
There were no corrective actions taken by the program.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The performance evaluations of the trained facilitators were reviewed for the annual review
period. The evaluations were all completed and contained specific information and feedback
regarding the staff's delivery of Pathways to Self-Discovery and Change.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 24 weeks for this type of service. Of the 36 youth in the
sample, 89% (32 of 36) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 40 contact hours for this type of service. Of the 36 youth in the
sample, 89% (32 of 36) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 83%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 7 youth
Moderate-High = 10 youth
High = 13 youth
Total Youth in Sample = 36 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 36%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 13 youth
Total Youth in Sample = 36 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 83% (30 of 36) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 36% (13 of 36) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 20 25 80%
Totals 86 95 91%
This SPEP report evaluates Pathways to Self-Discovery and Change, an intervention delivered at
Tampa Residential Facility.
46 youth completed the program during the review period. Of those youth, 36 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 36 total youth sampled, only
35 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 32
received at least the recommended weeks of service. Youth in the sample completed between 10
and 50 weeks of service, with an average of 25 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 8 points for Amount of Service: Contact Hours. Of the 36 total youth
sampled, only 35 included dosage in the EBS Module. Of those youth with correct dosage, 32
received at least the recommended hours of service. Youth in the sample completed between 18
and 94 hours of service, with an average of 47 hours.
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 96%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Tampa Residential Facility can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Tampa Residential Facility can optimize their SPEP Amount of Service score by ensuring dosage
for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted dosage
of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2023-24 Tampa Pathways.pdf

service_type: Pathways to Self‐Discovery and Change


summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Tampa Residential Facility
TrueCore Behavioral Solutions
(Contract Provider)
9508 East Colubus Drive
Tampa , Florida 33619-7715
Primary Service: Thinking for a Change
SPEP Review Period: 11/1/2022-10/31/2023
SPEP Review Conducted: 11/7/2023
Florida Department of Juvenile Justice
Report Date: 6/20/2024
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The intervention being reviewed is Thinking for a Change (T4C). The Department of Juvenile
Justice provides the formal training for the intervention. There were seven staff formally trained in
the T4C curriculum for this annual review period: one director of case management (DCM), three
(3) transition service managers (TSM), and three (3) case managers (CM). All TSMs and case
managers facilitated the primary service. The DCM conducted all fidelity monitoring.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The manual reviewed was Thinking for a Change v3.1. The manual contains worksheets for
students and suggestions for facilitators. It includes a script, implementation guidelines, identifies
the order the lessons should be delivered, and includes detailed instructions for the delivery of
each lesson.
Thinking for a Change is a closed group and is held on Mondays and Wednesdays. The program
maintains their records in a Standardized Program Evaluation Protocol (SPEP) binder. All youth at
Tampa Residential Facility receive Thinking for a Change.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Technical assistance specialist did not observe the primary service during this annual review
period.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
Documentation and an interview with the director of case management confirmed that one case
manager resigned on 2/15/2023; however, this turnover did not adversely affect the services of
T4C because the co-facilitator continued facilitating the group. The program's policy regarding
turnover is for anticipated turnover to be addressed by ensuring any potentially affected groups
are co-facilitated, if possible, at the outset and that all staff are trained in these curricula.
There were seven (7) cohorts facilitated during this annual review period. The first cohort was
delivered 8/15/2022-11/9/2022, which started with ten (10) youth and ended with ten (10) youth.
Aftercare for this cohort ran from 11/11/2022-11/28/2022.
The second cohort was delivered 12/5/2022-2/27/2023, which started with ten (10) youth and
ended with nine (9) youth (one youth was arrested on 1/27/23). Aftercare for this cohort ran from
3/1/2023-3/17/2023.
The third cohort was delivered 1/25/2023-4/21/2023, which started with ten (10) youth and ended
with ten (10) youth. Aftercare for this cohort ran from 4/21/2023-5/8/2023.
The fourth cohort was delivered 3/28/2023-6/16/2023, which started with ten (10) youth and
ended with ten (10) youth. Aftercare for this cohort ran from 6/21/2023-7/12/2023.
The fifth cohort was delivered 5/22/2023-8/18/2023, which started with ten (10) youth and ended
with ten (10) youth. Aftercare for this cohort ran from 8/21/2023-9/11/2023.
The sixth cohort was delivered 7/17/2023-10/9/2023, which started with ten (10) youth and ended
with ten (10) youth. Aftercare for this cohort ran from 10/11/2023-11/3/2023.
The seventh cohort began on 10/2/2023 and is currently ongoing (as of 10/31/2023). There were
no gaps during this annual review period.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The director of case management performed all fidelity monitoring for the review period. The
program completed twenty-five (25) internal fidelity monitorings for the facilitators of Thinking for a
Change (T4C).
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
An interview with the director of case management verified that the program has a process for
corrective actions based on fidelity monitoring conducted. The program uses internal fidelity
monitoring/corrective action checklists specific to Thinking for a Change. Each fidelity monitoring
checklist includes a section for feedback specifying the facilitator did not require any corrective
action or included recommendations for future improvement. The following month's checklist
included comments that the previous month's recommendations were applied and demonstrated.
There were no corrective actions taken by the program for Thinking for a Change during this
review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Technical assistance specialist reviewed the performance evaluations of the trained facilitators.
There were two (2) annual evaluations completed for two (2) of the case managers which
contained information and feedback regarding the staff’s delivery of Thinking for a Change. The
other facilitators’ annual evaluations are due by the end of November 2023.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates the target duration of 15 weeks for this type of service. Of the 36 youth in the
sample, 33% (12 of 36) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 45 contact hours for this type of service. Of the 36 youth in the
sample, 72% (26 of 36) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 83%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 7 youth
Moderate-High = 10 youth
High = 13 youth
Total Youth in Sample = 36 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 36%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 13 youth
Total Youth in Sample = 36 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 83% (20 of 36) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 36% (13 of 36) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 2 10 20%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 20 25 80%
Totals 83 100 83%
This SPEP report evaluates Thinking for a Change, an intervention delivered at Tampa
Residential Facility.
46 youth completed the program during the review period. Of those youth, 36 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 2 points for Amount of Service: Duration. Of the 36 total youth sampled, only
35 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 12
received at least the recommended weeks of service. Youth in the sample completed between 12
and 17 weeks of service, with an average of 14 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 6 points for Amount of Service: Contact Hours. Of the 36 total youth
sampled, all 36 included dosage in the EBS Module. Of those youth with correct dosage, 26
received at least the recommended hours of service. Youth in the sample completed between 11
and 54 hours of service, with an average of 44 hours.
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 88%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Tampa Residential Facility can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Tampa Residential Facility can optimize their SPEP Amount of Service score by ensuring dosage
for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted dosage
of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2023-24 Tampa T4C.pdf

service_type: Thinking for a Change