summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Hillsborough Girls Academy
9506 East Columbus Drive
(Contract Provider)
TrueCore Behavioral Solutions LLC
Tamp, Florida 33619
Primary Service: Impact of Crime
SPEP Review Period: 08/01/20-07/31/21
SPEP Review Conducted: 08/10/21
Florida Department of Juvenile Justice
Report Date: 10/22/2021
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There were five staff members formally trained in the Impact of Crime curriculum- one Facility
Administrator, one Assistant Facility Administrator, one Director of Case Management, one
Transition Services Manager and one Recreational Therapist. Four of these staff facilitated the
Impact of Crime curriculum during the review dates. Each received training from a qualified
trainer who was formally trained in the Impact of Crime curriculum.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes Impact of Crime-Addressing the Harm manual for this group. The curriculum
provides the script for the facilitator, the lessons, an implementation guide, the order the lessons
are to delivered in and instructions for the delivery of each lesson.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
An observation of the Impact of Crime group was completed on Wednesday, 08/11/21, via Zoom
from 11:30-12:30 The facilitator followed the lesson plan, did not read verbatim, explained the
lesson her own words and was easy for the youth to understand. The facilitator adhered to the
lesson plan in every aspect.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 0 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of the documentation provided and an interview with the Regional Director of Clinical
Services confirmed there was no facilitator turnover which adversely affected the services of
Impact of Crime to the youth. There were, however, gaps in services from 08/01/20-08/27/20 and
from 01/01/21-05/23/21. The Regional Director of Clinical Services explained lengthy gap had
occurred because the former Clinical Director did not realize Impact of Crime group had to be run
even if Thinking for a Change was also running at the same time.
The program can earn 2 points if there are no unnecessary gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Internal fidelity monitoring was conducted for Impact of Crime each month services were held
during the review period.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program uses internal fidelity monitoring checklists specific to the Impact of Crime curriculum.
A review of the documentation provided and an interview with the Regional Director of Clinical
Services confirmed there was no corrective action taken during this review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of all facilitators' performance evaluations found each included comments on any
strengths and weaknesses of which the Impact of Crime facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 16 weeks for this type of service. Of the 10 youth in the
sample, 60% (6 of 10) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 24 contact hours for this type of service. Of the 10 youth in the
sample, 70% (7 of 10) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 15 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 60%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 90%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 1 youth
Moderate-High = 6 youth
High = 2 youth
Total Youth in Sample = 10 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 20%
High Score: 5 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 38%
Table 2
High = 2 youth
Total Youth in Sample = 10 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 90% (9 of 10) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 20% (2 of 10) youth scored High-Risk to reoffend, for a score of 5 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 15 25 60%
Totals 67 85 79%
This SPEP report evaluates Impact of Crime, an intervention delivered at Hillsborough Girls
Academy.
22 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 6 points for Amount of Service: Duration. Of the 10 total youth sampled, all
10 included dosage with end dates in the EBS Module. Of those youth, 6 received at least the
recommended weeks of service. Youth in the sample completed between 11 and 19 weeks of
service, with an average of 16 weeks.
The program earned 6 points for Amount of Service: Contact Hours. Of the 10 total youth
sampled, all 10 included dosage in the EBS Module. Of those youth, 7 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
recommended hours of service. Youth in the sample completed between 17 and 32 hours of
service, with an average of 28 hours.
The program was awarded 15 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 91%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Impact of Crime is an intervention suitable for all youth. The Department recommends providing
this service to every youth in the program.
Hillsborough Girls Academy can optimize their SPEP Quality of Service Delivery score by
ensuring there are no unnecessary gaps in service.
Hillsborough Girls Academy can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8

file_name: FY2021-22 HGA IOC.pdf


state: FL

facility_name: Hillsborough Juvenile Detention Center West

summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Hillsborough Girls Academy
9506 East Columbus Drive
(Contract Provider)
Truecore Behavioral Solutions LLC
Tamp, Florida 33619
Primary Service: Thinking for a Change
SPEP Review Period: 08/01/20-07/31/21
SPEP Review Conducted: 08/10/21
Florida Department of Juvenile Justice
Report Date: 10/25/21
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There were two staff trained to facilitate the Thinking for a Change curriculum- one Assistant
Facility Administrator and one Youth Care Worker II. Both received training from a qualified trainer
who was formally trained in the Thinking for a Change curriculum.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the Thinking for a Change version 3.1 manual. The curriculum provides a
script for the facilitator, twenty five lessons, an implementation guide, the order the lessons are to
delivered in and instructions for the delivery of each lesson.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
An observation of Thinkling for a Change was completed on Wednesday 08/11/21 from 2:30-3:30.
The facilitator followed the lesson plan, did not read verbatim, explained the lesson her own
words, and was easy for the youth to understand. The facilitator adhered to the lesson plan in
every aspect.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 0 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation provided and an interview with the Regional Director of Clinical
Services confirmed there was no facilitator turnover which adversely affected the services of
Thinking for a Change to the youth. There was a gap in service from 08/01/20 to 02/15/21.
There was one cohort of Thinking for a Change completed during the dates of 02/16/21-06/15/21
and an ongoing cohort which started on 06/17/21.
The program can earn 2 points if there are no unnecessary gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Internal fidelity monitoring was completed monthly for the facilitator of Thinking for a Change
during each month the service was held during the review period.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program uses internal fidelity monitoring checklists specific to Thinking for a Change. A
review of documentation provided and an interview with the Regional Director of Clinical Services
confirmed there was no corrective action taken during this review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of the facilitators evaluations was completed. The employee performance evaluations
found both tboth facilitators' prior year evaluations had been completed and included conments on
any strengths or weaknesses of which each facfilitator should be aware. This year's evaluation for
both are not due until 10/26/21.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 15 weeks for this type of service. Of the 14 youth in the
sample, 86% (12 of 14) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates a target of 45 contact hours for this type of service. Of the 14 youth in the
sample, 57% (8 of 14) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 79%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 4 youth
Moderate-High = 2 youth
High = 5 youth
Total Youth in Sample = 14 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 36%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 5 youth
Total Youth in Sample = 14 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 79% (11 of 14) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 36% (5 of 14) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 4 10 40%
Risk Level of Youth Served 20 25 80%
Totals 87 100 87%
This SPEP report evaluates Thinking for a Change, an intervention delivered at Hillsborough Girls
Academy.
22 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 14 total youth sampled, all
14 included dosage with end dates in the EBS Module. Of those youth, 12 received at least the
recommended weeks of service. Youth in the sample completed between 10 and 24 weeks of
service, with an average of 17 weeks.
The program earned 4 points for Amount of Service: Contact Hours. Of the 14 total youth
sampled, all 14 included dosage in the EBS Module. Of those youth, 8 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
recommended hours of service. Youth in the sample completed between 19 and 45 hours of
service, with an average of 39 hours.
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 92%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Hillsborough Girls Academy can optimize their SPEP Quality of Service Delivery score by
ensuring there are no unnecessary gaps in service.
Hillsborough Girls Academy can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8

source: DJJ

report_type: SPEP

file_name: FY2021-22 HGA T4C.pdf

program_name_raw: Hillsborough Girls Academy

program_status_note: Closed

service_type: Thinking for a Change


summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Hillsborough Girls Academy
TrueCore Behavioral Solutions, LLC
(Contract Provider)
9506 East Columbus Drive
Tampa , Florida 33619
Primary Service: Female Healthy Relationships
SPEP Review Period: 08/01/21-07/31/22
SPEP Review Conducted: 08/25/22
Florida Department of Juvenile Justice
Report Date: 1/6/23
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Female Healthy Relationships.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
Female Healthy Relationships is designated as a primary service at the program. This bundled
service includes both the The Teen Relationships Workbook and Voices: A Program of Self-
Discovery and Empowerment for Girls interventions. The program has formal training protocols for
both curricula. Facilitators for Teen Relationships are trained by the Director of Clinical Services
or other licensed mental health representative. The training is a comprehensive overview of the
course coupled with opportunities to practice the skills of engagement as outlined by the
curriculum requirements. Facilitators for Voices attend a three-day training conducted by the
Regional Compliance Manager who maintains a Train the Trainer status for Voices. The training
is a comprehensive overview of the course coupled with activities and mock group presentations
which award the Therapists an opportunity to learn and practice the skills of engagement as
outlined by the curriculum requirements.
There were three staff trained in the Voices curriculum- two non-licensed Therapists and one prior
Director of Clinical Services. The prior Director of Clinical Service's last day at the program was
08/03/22. The new Director of Clinical Services is currently in training. There were also four non-
licensed Therapists trained in the Teen Relationships curriculum. Two of those Therapists have
left the program. Fidelity monitoring for this service was provided by the prior Director of Clinical
Services or her trained designee.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program has brand name manuals for the delivery of Female Healthy Relationships. The The
Teen Relationships Workbook manual is divided into five sections and one review section. The
sections are related to healthy relationships and abuse. The manual contains worksheets for
students and suggestions for facilitators. It includes a script, implementation guidelines, identifies
the order the lessons should be delivered, and includes detailed instructions for each lesson.The
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Voices manual is divided into eighteen (18) lessons with specific interventions and activities, and
utilizes a facilitator manual and an associated workbook. The lessons are further divided into the
modules of Self, Others, Healthy Living, and Journey Ahead. The lessons are outlined with detail
with some flexibility for idiosyncratic group dynamics. Teen Relationships is an open ongoing
group. Voices is a closed group. Both groups are held on Tuesdays and Thursdays from 2:30-
3:40 p.m. The program maintains their records for both interventions in a SPEP binder tabbed by
intervention and then seperated into the different SPEP indicator sections. All youth at
Hillsborough Girls Academy receive Female Healthy Relationships.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Voices was observed virtually via Microsoft Teams on Thursday, 08/25/22, from
2:30p to 3:40p. The facilitator provided a copy of the lesson plan from the manual prior to the
observation. The lesson for the day was Lesson 13, Our Bodies. The facilitator followed the
lesson according to the manual, did not read the lesson verbatim, used her own words to convey
the topic and did so in an easy to follow manner. The facilitator adhered to the lesson plan in
every aspect.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The reviewed documentation and an interview with the Assistant Facility Administrator confirmed
a total of two Unlicensed Therapists and the prior Director of Clinical Services left during the
review period. However, this turnover did not adversely affect the delivery of Female Healthy
Relationships. One Unlicensed Therapist left after her cohort of Voices had been completed. The
other Therapist and Director of Clinical Services were not facilitating group when they left the
program. The program's policy regarding turnover to ensure any potentially affected groups are
co-facilitated, if possible, at the outset and that all staff are trained in the service.
Female Healthy Relationships is a bundled service, consisting of a closed group component
(Voices) and an open group component (The Teen Relationships Workbook). Voices had two
completed cohorts and one ongoing cohort during the review period. There were three cycles
completed for the Voices intervention. They ran from 08/31/21 to 10/28/21, 03/02/22-05/03/22,
and group ongoing at the time of the annual review began 7/14/22. There were 3 -7 youth in each
cohort of Voices. During the gaps in the Voices cohorts, all youth continued to receive the open
component of the group.
The Teen Relationships Workbook component completed two cohorts which both ran from
08/04/21-12/15/21. There were two ongoing cohorts at the time of the annual review, which
started on 12/29/21 and 12/22/21. There are 8 youth in each current group.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Twelve months of internal fidelity monitoring reports were reviewed for facilitators of Female
Healthy Relationships. The program has an internal process to conduct fidelity monitoring specific
to the service. The Director of Clinical Services or their trained designee performs all fidelity
monitoring. The Director of Clinical Services and clinical staff were all formally trained in Female
Healthy Relationships. Fidelity monitoring was completed monthly for each facilitator for Teen
Relationships during the review period. Fidelity monitoring for Voices was not completed for the
months of 08/21, 09/21, and 05/22.
The program can earn 2 points fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet the riteria to earn a score of 2, however, they
were able to earn a score of 1 for this indicator.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the Assistant Facility Administrator verified the program has a process for
corrective actions based on fidelity monitoring conducted for facilitators of Female Healthy
Relationships. The program uses internal fidelity monitoring/corrective action checklists specific to
the service. Each fidelity monitoring checklist includes a section for feedback specifying the
facilitator did not require any corrective action or included recommendations for future
improvement. The following month's checklist should include comments the previous month's
recommendations were applied and demonstrated.
There were no corrective actions noted nor taken during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of three facilitators for Female Healthy Relationships found evaluations were completed
in 2021 for two. The other facilitator has not been at the program long enough for an annual
evaluation. The performance evaluations included entries for the evaluation of the staff's
performance in delivering Female Healthy Relationships. These included comments on any
strengths or weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 24 weeks for this type of service. Of the 16 youth in the
sample, 100% (16 of 16) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 40 contact hours for this type of service. Of the 16 youth in the
sample, 100% (16 of 16) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 10 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 40%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 81%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 4 youth
Moderate-High = 6 youth
High = 3 youth
Total Youth in Sample = 16 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 19%
High Score: 3 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 23%
Table 2
High = 3 youth
Total Youth in Sample = 16 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 81% (13 of 16) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 19% (3 of 16) youth scored High-Risk to reoffend, for a score of 3 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 10 25 40%
Totals 80 95 84%
This SPEP report evaluates Female Healthy Relationships, an intervention delivered at
Hillsborough Girls Academy.
16 youth completed the program during the review period. Of those youth, all 16 were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 16 total youth sampled, all
16 included dosage with end dates in the EBS Module. Of those youth, 16 received at least the
recommended weeks of service. Youth in the sample completed between 41 and 78 weeks of
service, with an average of 58 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 10 points for Amount of Service: Contact Hours. Of the 16 total youth
sampled, all 16 included dosage in the EBS Module. Of those youth, 16 received at least the
recommended hours of service. Youth in the sample completed between 44 and 122 hours of
service, with an average of 80 hours.
The program was awarded 10 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 100%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Hillsborough Girls Academy can optimize their SPEP Quality of Service Delivery score by
ensuring fidelity monitoring is completed every month for each facilitator.
Hillsborough Girls Academy can optimize their SPEP Amount of Service score by ensuring
dosage for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2022-23 HGA FHR.pdf

service_type: Female Healthy Relationships


summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Hillsborough Girls Academy
TrueCore Behavioral Solutions, LLC
(Contract Provider)
9506 E Columbus Dr
Tampa, Florida 33619
Primary Service: Impact of Crime
SPEP Review Period: 08/01/21-07/31/22
SPEP Review Conducted: 08/25/22
Florida Department of Juvenile Justice
Report Date: 1/6/23
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The Department of Juvenile Justice provides formal training for Impact of Crime (IOC). There
were four staff formally trained in the curriculum during the review period- one Assistant Facility
Administrator (AFA), one Director of Case Management (DCM), one Recreational Therapist (RT)
and one Transition Service Manager (TSM). The TSM and DCM facilitated and the AFA
conducted fidelity monitoring during this annual review period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program has to appropriate manuals for the delivery of Impact of Crime. The manual contains
worksheets for students and suggestions for facilitators. It includes a script, implementation
guidelines, identifies the order the lessons should be delivered, and includes detailed instructions
for each lesson.
Impact of Crime is a closed group and is held on Mondays and Wednesday from 3:30 to 4:30pm.
The program maintains their records in a SPEP binder. All youth at Hillsborough Girls Academy
receive Impact of Crime.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Impact of Crime was observed on Wednesday, 08/31/22, from 3:50 to 4:40pm. The
facilitator provided a copy of the lesson plan from the manual prior to the observation. The lesson
for the day was Chapter 4, Consequences of Making Decisions. The facilitator followed the lesson
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
according to the manual, she did not read the lesson verbatim, instead she used her own words in
combination with the manual to convey the topic. The facilitator was diligent in controlling the
group and keeping them engaged. The facilitator adhered to the lesson plan in every aspect.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation and an interview with the Assistant Facility Administrator confirmed
there was turnover of Impact of Crime facilitators during the review period; however, this did not
adversely affect the delivery of services. The program's policy regarding turnover is to ensure any
potentially affected groups are co-facilitated, if possible, at the outset and that all staff are trained
in the curriculum.
There were four cohorts run during the review period. The first ran from 08/02/21 through
10/04/21 and started and ended with nine youth. A second cohort ran from 09/13/21 through
02/16/22, which started with six youth and ended with two youth; four youth were discharged prior
to completing the service. There was a gap between 12/15/21 and 01/10/22, where the facilitator
was out due to sick leave.
A third cohort ran from 01/25/22 through 05/29/22; it started with ten (10) youth and ended with
eight youth. The cohort ongoing at the time of the annual review started 07/12/22.
The program can earn 2 points if there are no unnecessary gaps in service.
At the time of the review, the program (did/did not) meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of internal fidelity monitoring reports for Impact of Crime was conducted for the review
period of 08/01/21-07/31/22. The program has an internal process to conduct fidelity monitoring.
The Assistant Facility Administrator performed all fidelity monitoring. Fidelity monitoring for IOC
was conducted each month except 06/22.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program (did/did not) meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
An interview with the Assistant Facility Administrator verified the program has a process for
corrective actions based on fidelity monitoring conducted on facilitators of Impact of Crime (IOC).
The program uses internal fidelity monitoring/corrective action checklists specific to IOC. Each
fidelity monitoring checklist includes a section for feedback specifying the facilitator did not require
any corrective action or included recommendations for future improvement. The following month's
checklist included comments the previous month's recommendations were applied and
demonstrated. There were no corrective actions noted nor taken by the program for IOC.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of the two primary facilitators for the Impact of Crime (IOC) curriculum found both had an
evaluation completed in 2021. The performance evaluations included an entry for the evaluation
of both staff’s performance in delivering IOC. These included comments on any strengths or
weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 16 weeks for this type of service. Of the 14 youth in the
sample, 79% (11 of 14) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 24 contact hours for this type of service. Of the 14 youth in the
sample, 79% (11 of 14) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 15 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 60%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 86%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 3 youth
Moderate-High = 6 youth
High = 3 youth
Total Youth in Sample = 14 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 21%
High Score: 5 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 38%
Table 2
High = 3 youth
Total Youth in Sample = 14 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 86% (12 of 14) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 21% (3 of 14) youth scored High-Risk to reoffend, for a score of 5 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 15 25 60%
Totals 68 85 80%
This SPEP report evaluates Impact of Crime, an intervention delivered at Hillsborough Girls
Academy.
16 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 6 points for Amount of Service: Duration. Of the 14 total youth sampled, all
14 included dosage with end dates in the EBS Module. Of those youth, 11 received at least the
recommended weeks of service. Youth in the sample completed between 4 and 28 weeks of
service, with an average of 18 weeks.
The program earned 6 points for Amount of Service: Contact Hours. Of the 14 total youth
sampled, all 14 included dosage in the EBS Module. Of those youth, 11 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
recommended hours of service. Youth in the sample completed between 6 and 59 hours of
service, with an average of 31 hours.
The program was awarded 15 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 92%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Impact of Crime is an intervention suitable for all youth. The Department recommends providing
this service to every youth in the program.
Hillsborough Girls Academy can optimize their SPEP Quality of Service Delivery score by
ensuring there are no unnecessary gaps in service and fidelity monitoring is completed every
month for each facilitator.
Hillsborough Girls Academy can optimize their SPEP Amount of Service score by ensuring
dosage for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2022-23 HGA IOC.pdf

service_type: Impact of Crime


summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Hillsborough Girls Academy
TrueCore Behavioral Solutions, LLC
(Contract Provider)
9506 E Columbus Dr
Tampa, Florida 33619
Primary Service: Thinking for a Change
SPEP Review Period: 08/01/21-07/31/22
SPEP Review Conducted: 08/25/22
Florida Department of Juvenile Justice
Report Date: 1/6/23
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program has assigned Thinking for a Change (T4C) as a primary service. There were four
staff formally trained in the Thinking for a Change curriculum during the review period- one
Assistant Facility Administrator (AFA), one Director of Case Management (DCM), one
Recreational Therapist (RT) and one Youth Care Worker II (YCW II). The RT and DCM facilitated
groups, and the AFA conducted fidelity monitoring during this annual review period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program has brand name manuals for the delivery of Thinking for a Change. The manual
contains worksheets for students and suggestions for facilitators. It includes a script,
implementation guidelines, identifies the order the lessons should be delivered, and includes
detailed instructions for each lesson.
Thinking for a Change is a closed group and is held on Mondays and Wednesday from 4:30 to
5:30pm. The program maintains their records in a SPEP binder. All youth at Hillsborough Girls
Academy receive Thinking for a Change.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Thinking for a Change was observed on Wednesday, 08/31/22, from 5:05 to 6pm.
The facilitator provided a copy of the lesson plan from the manual prior to the observation. The
lesson for the day was Lesson 2, Social Skill: Active Listening. The facilitator followed the lesson
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
according to the manual, he did not read the lesson verbatim instead he used his own words in
combination with the manual to convey the topic. The method employed had the youth very
engaged and allowed him to keep control of the group. The facilitator adhered to the lesson plan
in every aspect.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation and an interview with the Assistant Facility Administrator confirmed
two Unlicensed Therapists and the prior Director of Clinical Services left during the review period.
However, this turnover did not adversely affect the services of Thinking for a Change. The
program's policy regarding turnover is to ensure any potentially affected groups are co-facilitated,
if possible, at the outset and that all staff are trained in the curriculum.
There were two cohorts completed during the annual review period. The first cohort began
09/08/21 and ended 12/16/21, and started and ended with ten (10) youth. Aftercare ran for this
group 12/12/21-01/18/22. The second cohort ran 03/02/22-06/26/22; it started with ten (10) youth
and ended with six youth. Aftercare ran for this second cohort 06/6/22-08/24/22. There were no
gaps in service.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of internal fidelity monitoring reports for Thinking for a Change (T4C) was conducted for
08/01/21-07/31/22. The program has an internal process to conduct fidelity monitoring. The
Assistant Facility Administrator performed all fidelity monitoring for the annual review period.
Fidelity monitoring for T4C was conducted each month groups were held, without exception.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the Assistant Facility Administrator verified the program has a process for
corrective actions based on fidelity monitoring conducted. The program uses internal fidelity
monitoring/corrective action checklists specific to Thinking for a Change. Each fidelity monitoring
checklist includes a section for feedback specifying the facilitator did not require any corrective
action or included recommendations for future improvement. The following month's checklist
would include comments the previous month's recommendations were applied and demonstrated.
There were no corrective actions noted nor taken during the review period.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of two facilitators for the Thinking for a Change (T4C) curriculum found both had an
evaluation completed in 2021. The performance evaluations completed were reviewed and an
entry was found for the evaluation of both staff’s performances in delivering T4C. These included
comments on any strengths or weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 15 weeks for this type of service. Of the 13 youth in the
sample, 92% (12 of 13) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 45 contact hours for this type of service. Of the 13 youth in the
sample, 62% (8 of 13) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 10 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 40%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 77%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 4 youth
Moderate-High = 4 youth
High = 2 youth
Total Youth in Sample = 13 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 15%
High Score: 3 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 23%
Table 2
High = 2 youth
Total Youth in Sample = 13 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 77% (10 of 13) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 15% (2 of 13) youth scored High-Risk to reoffend, for a score of 3 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 10 25 40%
Totals 79 100 79%
This SPEP report evaluates Thinking for a Change, an intervention delivered at Hillsborough Girls
Academy.
16 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 13 total youth sampled, all
13 included dosage with end dates in the EBS Module. Of those youth, 12 received at least the
recommended weeks of service. Youth in the sample completed between 13 and 27 weeks of
service, with an average of 21 weeks.
The program earned 6 points for Amount of Service: Contact Hours. Of the 13 total youth
sampled, all 13 included dosage in the EBS Module. Of those youth, 8 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
recommended hours of service. Youth in the sample completed between 21 and 46 hours of
service, with an average of 43 hours.
The program was awarded 10 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 94%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Hillsborough Girls Academy can maintain their SPEP Quality of Service Delivery score by
continuing the processes in place at the time of this review.
Hillsborough Girls Academy can optimize their SPEP Amount of Service score by ensuring
dosage for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2022-23 HGA T4C.pdf