summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Hillsborough Girls Academy
Truecore Behavorial Solutions, LLC
(Contract Provider)
9506 East Columbus Drive
Tampa, Florida 33619
Primary Service: Female Healthy Relationships
SPEP Review Date(s): 07/16/19‐07/19/19
Florida Department of Juvenile Justice
Report Date(s): 07/22/19
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
 Service Type: service type points are awarded based on the group category that the
service falls within
 Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
 Amount of Service: or “dosage” refers to the amount and duration of services that were
received
 Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Female Healthy Relationships. The program was
awarded 25 points because the Primary Service is identified as a Group 4 Service. The specific
Sub-Component Service Type identified is Group Counseling. The Primary Service was identified
as this type of service as it focuses on psychological or interpersonal problems or issues faced by
an individual and involves a group of youths interacting with each other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There were eleven staff members formally trained in the Female Healthy Relationships bundled
primary service, which consists of the VOICES and The Teen Relationships Workbook curricula.
One Director of Clinical Services, eight Therapists (two of those are PRN), one Assistant Facility
Director and one Assistant Director Clinical Services. Five of these staff facilitated this primary
service during the review period. All staff received training from a qualified trainer who was
formally trained in the Female Healthy Relationships bundle prior to delivering services to youth.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
All youth admitted to Hillsborough Girls Academy are assessed by staff as to their individualized
needs for evidence-based services. Based on various criteria, the youth are placed into program
services. The program utilizes the VOICES and The Teen Relationships Workbook bundled
curriculum, Female Healthy Relationships. The manual contains a script for the facilitator for the
groups, provides lessons and includes implementation guidelines, identifies the order the lessons
should be delivered in and includes detailed instructions for the delivery of each lesson.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The Female Health Relationships group, specifically a VOICES session, was observed on
Thursday 07/18/19 from 2:30-3:30 pm. The facilitator provided a copy of the lesson plan from the
program manual. The facilitator followed the lesson according to the manual and did not read the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
script verbatim, using her own words to convey the meaning to the youth. The facilitator adhered
to the lesson plan in every aspect.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation provided and an interview with the Clinical Director confirmed there
has been some facilitator turnover in the past twelve (12) months; however, the turnover did not
adversely affect services to the youth. There were no gaps in services of the Female Healthy
Relationships curriculum to the youth.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation for fidelity monitoring was reviewed. Monitoring was performed for all twelve
(12) months of the review period. This was done by the Clinical Director, Assistant Clinical
Director and a Therapist, all trained by a facilitator trained specifically in the curriculum.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the Clinical Director verified the program has a process for corrective actions
based on the fidelity monitoring conducted. The program uses internal fidelity monitoring /
corrective action checklists specific to the Female Healthy Relationships curriculum. A review of
the documentation provided and an interview with the Clinical Director verified there was no
corrective action noted nor taken during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The performance evaluations of all trained facilitators revealed entries on all evaluations of the
staff’s performance in delivering the Female Healthy Relationships curriculum. These included
comments on any strengths or weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service ‐ Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 24 weeks for this type of service. Of the 25 youth in the
sample, 100% (25 of 25) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 40 contact hours for this type of service. Of the 25 youth in the
sample, 100% (25 of 25) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 10 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 40%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 72%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 4 youth
Moderate-High = 8 youth
High = 6 youth
Total Youth in Sample = 25 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 24%
High Score: 5 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 38%
Table 2
High = 6 youth
Total Youth in Sample = 25 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 72% (18 of 25) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 24% (6 of 25) youth scored High-Risk to reoffend, for a score of 5 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 10 25 40%
Totals 80 95 84%
This SPEP report evaluates Female Healthy Relationships, an intervention delivered at
Hillsborough Girls Academy.
25 youth completed the program during the review period. Of those youth, all 25 were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 25 total youth sampled, all
25 included dosage with end dates in the EBS Module. Of those youth, all 25 received at least the
recommended weeks of service. Youth in the sample completed between 35 and 79 weeks of
service, with an average of 53 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
The program earned 10 points for Amount of Service: Contact Hours. Of the 25 total youth
sampled, all 25 included dosage in the EBS Module. Of those youth, all 25 received at least the
recommended hours of service. Youth in the sample completed between 57 and 108 hours of
service, with an average of 73 hours.
The program was awarded 10 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 100%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Hillsborough Girls Academy can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Hillsborough Girls Academy can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8


state: FL

facility_name: Hillsborough Juvenile Detention Center West

summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Hillsborough Girls Academy
Truecore Behavorial Solutions, LLC
(Contract Provider)
9506 East Columbus Drive
Tampa, Florida 33619
Primary Service: Impact of Crime
SPEP Review Date(s): 07/16/19‐07/19/19
Florida Department of Juvenile Justice
Report Date(s): 07/22/19
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
 Service Type: service type points are awarded based on the group category that the
service falls within
 Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
 Amount of Service: or “dosage” refers to the amount and duration of services that were
received
 Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime. The program was awarded 15 points
because the Primary Service is identified as a Group 3 Service. The specific Sub-Component
Service Type identified is Social Skills Training. The Primary Service was identified as this type of
service as it focuses on developing social skills required for an individual to interact in a positive
way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There were five staff members formally trained in the Impact of Crime (IOC) curriculum, one
Director Clinical Services, one Assistant Facility Administrator, one Recreational Therapist, one
Youth Care Worker and one Transition Manager. Out of the five trained, two facilitated group
during the review period. All five of the staff received training from a qualified trainer who was
formally trained in the Impact of Crime curriculum prior to delivering services to youth.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
All youth admitted to Hillsborough Girls Academy are assessed by staff as to their individualized
needs for evidence-based services. Based on various criteria the youth are placed into program
services. The program utilizes the Impact of Crime-Addressing the Harm curriculum. The manual
contains a script for the facilitator for the groups, provides lessons and includes implementation
guidelines, identifies the order the lessons should be delivered in and includes detailed
instructions for the delivery of each
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The Impact of Crime group was not running during the review period; therefore, the curriculum
was not able to be observed. This indicator rates as non-applicable. The last group had just
ended on 06/27/19.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation provided and an interview with the Clinical Director confirmed there
was no staff turnover during the review period that adversely affected the group. There were two
small gaps in service of less than one month; however, the gaps were due to the facility having to
wait on enough new youth to arrive to enroll and start a new Impact of Crime group.
A review of the Juvenile Justice Information System confirmed an average daily population of 19
youth.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation for fidelity monitoring was reviewed. The program completed fidelity for all
months the group was run during the review period. The formally trained staff monitored each
other. Fidelity monitoring was missing for the months of November 2018 through February 2019.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the Clinical Director verified the program has a process for corrective actions
based on the fidelity monitoring conducted. The program uses internal fidelity monitoring /
corrective action checklists specific to the Impact of Crime curriculum. A review of the
documentation provided and an interview with the Clinical Director verified there was no corrective
action noted nor taken during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The performance evaluation of all trained facilitators found entries on all evaluations of the staff’s
performance in delivering Impact of Crime. These included comments on any strengths or
weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service ‐ Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 16 weeks for this type of service. Of the 19 youth in the
sample, 89% (17 of 19) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 24 contact hours for this type of service. Of the 19 youth in the
sample, 89% (17 of 19) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 17 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 68%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 79%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 2 youth
Moderate-High = 7 youth
High = 6 youth
Total Youth in Sample = 19 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 32%
High Score: 10 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 77%
Table 2
High = 6 youth
Total Youth in Sample = 19 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 79% (15 of 19) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 32% (6 of 19) youth scored High-Risk to reoffend, for a score of 10 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 17 25 68%
Totals 73 85 86%
This SPEP report evaluates Impact of Crime, an intervention delivered at Hillsborough Girls
Academy.
25 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 19 total youth sampled, all
19 included dosage with end dates in the EBS Module. Of those youth, 17 received at least the
recommended weeks of service. Youth in the sample completed between 7 and 18 weeks of
service, with an average of 16 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 19 total youth
sampled, all 19 included dosage in the EBS Module. Of those youth, 17 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
recommended hours of service. Youth in the sample completed between 7 and 34 hours of
service, with an average of 31 hours.
The program was awarded 17 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 95%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Impact of Crime is an intervention suitable for all youth. The Department recommends providing
this service to every youth in the program.
Hillsborough Girls Academy can optimize their SPEP Quality of Service Delivery score by
ensuring fidelity monitoring is completed every month for each facilitator.
Hillsborough Girls Academy can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8

source: DJJ

report_type: SPEP

program_name_raw: Hillsborough Girls Academy

program_status_note: Closed

service_type: Impact of Crime


summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Hillsborough Girls Academy
Truecore Behavorial Solutions, LLC
(Contract Provider)
9506 East Columbus Drive
Tampa, Florida 33619
Primary Service: Thinking for a Change
SPEP Review Date(s): 07/16/19-07/19/19
Florida Department of Juvenile Justice
Report Date(s): 07/22/19
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change. The program was awarded 30
points because the Primary Service is identified as a Group 5 Service. The specific Sub-
Component Service Type identified is Cognitive Behavioral Therapy. The Primary Service was
identified as this type of service as it is intended to correct faulty cognitions and perceptions and
provides skills individuals can use to monitor thought patterns and correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There were five staff members formally trained in the Thinking for a Change (T4C) curriculum.
Two facilitators conducted group during the review period. All five of the staff received training
from a qualified trainer who was formally trained in the T4C curriculum prior to delivering services
to youth.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
All youth admitted to Hillsborough Girls Academy are assessed by staff as to their individualized
needs for evidence-based services. Based on various criteria, the youth are placed into program
services. The program utilizes the Thinking for a Change curriculum, version 3.1. The manual
contains a script for the facilitator for the groups, provides 25 lessons and includes
implementation guidelines, identifies the order the lessons should be delivered in and includes
detailed instructions for the delivery of each lesson.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Thinking for a Change was observed on Thursday, 07/18/19. This was the first
session of a new group. The facilitator provided a copy of the lesson plan from the program
manual. The facilitator followed the lesson according to the manual and read the lesson using her
own words to convey the meaning to the youth. The facilitator adhered to the lesson plan in every
aspect.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 0 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation provided and an interview with the Clinical Director confirmed there
has been no staff turnover which adversely affected services of Thinking for a Change to the
youth. There were gaps in service in excess of six months, from 08/10/18-10/24/18 and from
02/27/19 to 7/17/19. The Clinical Director explained the gap was due to having to wait for enough
new youth to begin a new Thinking for a Change group. The program, in the case of turnover,
would utilize other trained staff to facilitate the curriculum.
A review of the Juvenile Justice Information System confirmed an average daily population of 19
youth.
The program can earn 2 points if there are no unnecessary gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation for fidelity monitoring was reviewed. The formally trained staff monitored each
other. Fidelity monitoring was conducted three (10/18, 1/19, and 2/19) of the five months the
group was held during the review period.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the Clinical Director verified the program has a process for corrective actions
based on the fidelity monitoring conducted. The program uses internal fidelity monitoring /
corrective action checklists specific to Thinking for a Change. A review of documentation provided
and an interview with Clinical Director verified there was no corrective action noted nor taken
during the review period.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The performance evaluations of all trained facilitators found entries on all evaluations of the staff’s
performance in delivering the Thinking for a Change curriculum. These included comments on
any strengths or weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 15 weeks for this type of service. Of the 9 youth in the
sample, 89% (8 of 9) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 45 contact hours for this type of service. Of the 9 youth in the
sample, 100% (9 of 9) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 5 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 20%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 56%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 2 youth
Moderate-High = 2 youth
High = 1 youth
Total Youth in Sample = 9 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 11%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 1 youth
Total Youth in Sample = 9 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 56% (5 of 9) youth scored Moderate to High-Risk to reoffend, for a score of
5 points.
Of the SPEP sample, 11% (1 of 9) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 5 25 20%
Totals 68 100 68%
This SPEP report evaluates Thinking for a Change, an intervention delivered at Hillsborough Girls
Academy.
25 youth completed the program during the review period.
The program scored Medium for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 9 total youth sampled, all 9
included dosage with end dates in the EBS Module. Of those youth, 8 received at least the
recommended weeks of service. Youth in the sample completed between 2 and 21 weeks of
service, with an average of 17 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 10 points for Amount of Service: Contact Hours. Of the 9 total youth
sampled, all 9 included dosage in the EBS Module. Of those youth, 9 received at least the
recommended hours of service. Youth in the sample completed 45 hours of service.
The program was awarded 5 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 88%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Hillsborough Girls Academy can optimize their SPEP Quality of Service Delivery score by
ensuring there are no unnecessary gaps in service and by ensuring fidelity monitoring is
completed every month for each facilitator.
Hillsborough Girls Academy can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

service_type: Thinking for a Change


summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Hillsborough Girls Academy
TrueCore Behavioral Solutions, LLC
(Contract Provider)
9506 East Columbus Drive
Tamp, Florida 33619
Primary Service: Female Healthy Relationships
SPEP Review Period: 8/1/2019 - 7/31/2020
SPEP Review Conducted: 8/5/2020
Florida Department of Juvenile Justice
Report Date: 10/28/2020
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Female Healthy Relationships.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There were six staff members formally trained in the Female Healthy Relationships Bundled
curriculum- one Director Clincal Services and five Therapists (one PRN). Two facilitators
conducted group during the review period. All six of the staff received training from a qualified
trainer who was formally trained in the Female Healthy Relationships bundled service, consisting
of Voices- A Program of Self-Discovery and Change and The Teen Relationships Workbook
curricula.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The Female Healthy Relationships bundle consists of two curricula - The Teen Relationships
Workbook and Voices- A Program of Self-Discovery and Change. Both manuals and
accompanying protocols were provided for review. Both curricula provide a script for the facilitator,
the lessons, an implementation guide, the order the lessons are to taught in and instructions for
the delivery of each lesson.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
An observation of The Teen Relationships Workbook portion of Female Healthy Relationships
service was completed via Facetime on Wednesay 09/16/20 from 2:30-3:30 p.m. The facilitator
followed the lesson plan, did not read verbatim and explained the lesson his her own words and
was easy for the youth to understand. The facilitator adhered to the lesson plan in every aspect.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
Accoring to a review of the documentation provided and an interview with the Director of Clinical
Services, there was no facilitator turnover which adversely affected the services of Female
Healthy Relationships to the youth. Additionally, there were no gaps in services.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Internal fidelity monitoring was completed for Impact of Crime by the Director of Clincal Services
for each facilitator during all months of the review period. There were no missing internal fidelity
monitoring forms.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program uses internal fidelity monitoring checklists specific to Female Healthy Relationships
(Voices and Teen Relationships). According to a review of the documentation provided as well as
an interview with the Director of Clinical Services, there were no corrective actions noted nor
taken during this review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of all facilitators' employee performance evaluations found an entry for evaluation of the
staff's performance in delivering Female Healthy Relationships. These included comments on any
strengths or weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 24 weeks for this type of service. Of the 18 youth in the
sample, 100% (18 of 18) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 40 contact hours for this type of service. Of the 18 youth in the
sample, 100% (18 of 18) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 83%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 2 youth
Moderate-High = 6 youth
High = 7 youth
Total Youth in Sample = 18 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 39%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 7 youth
Total Youth in Sample = 18 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 83% (15 of 18) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 39% (7 of 18) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 20 25 80%
Totals 90 95 95%
This SPEP report evaluates Female Healthy Relationships, an intervention delivered at
Hillsborough Girls Academy.
18 youth completed the program during the review period. Of those youth, all 18 were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 18 total youth sampled, all
18 included dosage with end dates in the EBS Module. Of those youth, 18 received at least the
recommended weeks of service. Youth in the sample completed between 27 and 73 weeks of
service, with an average of 52 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
The program earned 10 points for Amount of Service: Contact Hours. Of the 18 total youth
sampled, all 18 included dosage in the EBS Module. Of those youth, 18 received at least the
recommended hours of service. Youth in the sample completed between 44 and 92 hours of
service, with an average of 68 hours.
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 100%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Hillsborough Girls Academy can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Hillsborough Girls Academy can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8

service_type: Female Healthy Relationships


summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Hillsborough Girls Academy
TrueCore Behavioral Solutions, LLC
(Contract Provider)
9506 East Columbus Drive
Tamp, Florida 33619
Primary Service: Impact of Crime
SPEP Review Period: 8/1/2019 - 7/31/2020
SPEP Review Conducted: 8/5/2020
Florida Department of Juvenile Justice
Report Date: 10/28/2020
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There were four staff members formally trained in the Impact of Crime (IOC) curriculum- one
Recreational Therapist, one Transition Services Manager, one Assistant Facility Adminstrator and
one Youth Care Worker. One facilitator conducted group during the review period. All four of the
staff received training from a qualified trainer who was formally trained in the Impact of Crime
curriculum.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
All youth admitted to Hillsborough Girls Academy are assessed by staff as to their need for
evidence-based services. Based on various criteria the youth are placed into program services.
The program utilizes the Impact of Crime-Addressing the Harm curriculum. The manual contains a
script for the facilitator for the groups, provides lessons and includes implementation guidelines,
identifies the order the lessons should be delivered in and includes detailed instructions for the
delivery of each
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The Impact of Crime group was not observed during this review; groups were not being held at
the time of the annual SPEP review; therefore this indicator rates as non-applicable.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 0 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of the documentation provided revealed gaps in service from 08/01/9-10/15/19 and from
02/06/20-07/31/20. The Director of Clinical Services stated the gap from 08/01/19-10/15/19 was
due initially to enrollment numbers; however, following that, they also experienced staff turnover
issues which adversely affected the services of Impact of Crime to the youth.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation for fidelity monitoring of Impact of Crime was reviewed. Fidelity monitoring
was provided for each month groups were held.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the Director of Clinical Services verified the program has a process for
corrective actions based on the fidelity monitoring conducted. The program uses internal fidelity
monitoring/corrective action checklists specific to the Impact of Crime curriculum. A review of the
documentation provided and an interview with the Director of Clinical Services verified there were
no corrective actions noted nor taken during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The performance evaluations of all trained facilitators found entries of the staff's performance and
delivery of the Impact of Crime curriculum. These included comments on any strengths or
weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 16 weeks for this type of service. Of the 10 youth in the
sample, 100% (10 of 10) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 10 youth in the
sample, 100% (10 of 10) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 18 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 72%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 70%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 0 youth
Moderate-High = 3 youth
High = 4 youth
Total Youth in Sample = 10 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 40%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 4 youth
Total Youth in Sample = 10 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 70% (7 of 10) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 40% (4 of 10) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 18 25 72%
Totals 68 85 80%
This SPEP report evaluates Impact of Crime, an intervention delivered at Hillsborough Girls
Academy.
18 youth completed the program during the review period.
The program scored Medium for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 10 total youth sampled, all
10 included dosage with end dates in the EBS Module. Of those youth, 10 received at least the
recommended weeks of service. Youth in the sample completed between 16 and 19 weeks of
service, with an average of 17 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 10 total youth
sampled, all 10 included dosage in the EBS Module. Of those youth, 10 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
recommended hours of service. Youth in the sample completed between 31 and 32 hours of
service, with an average of 32 hours.
The program was awarded 18 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 88%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Impact of Crime is an intervention suitable for all youth. The Department recommends providing
this service to every youth in the program.
Hillsborough Girls Academy can optimize their SPEP Quality of Service Delivery score by
ensuring facilitator turnover does not impact groups.
Hillsborough Girls Academy can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8


summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Hillsborough Girls Academy
TrueCore Behavioral Solutions, LLC
(Contract Provider)
9506 East Columbus Drive
Tamp, Florida 33619
Primary Service: Thinking for a Change
SPEP Review Period: 8/1/2019 - 7/31/2020
SPEP Review Conducted: 8/5/2020
Florida Department of Juvenile Justice
Report Date: 10/28/2020
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There were four staff members formally trained in the Thinking for a Change curriculum- one
Recreational Therapist, one Director of Clinical Services, one Director Case Management and
one Transition Services Manager. Two facilitated group during the review period. All four of the
staff received training from a qualified trainer who was formally trained in the Thinking for a
Change curriculum.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
All youth admitted to Hillsborough Girls Academy are assessed by staff as to their need for
evidence-based services. Based on various criteria, the youth are placed into program services.
The program utilizes the Thinking for a Change, version 3.1 curriculum. The manual contains a
script for the facilitator for the groups, provides 25 lessons and includes implementation
guidelines, identifies the order the lessons should be delivered in and includes detailed
instructions for the delivery of each
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The Thinking for a Change group was not observed during this review. The group had just ended
on 07/15/20 and was not running at the time of the annual review; therefore, this indicator rates as
non-applicable.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of the documentation provided revealed gaps in services from 11/22/19-03/12/20 and
from 07/16/20-07/31/20. The Director of Clinical Services stated the gaps were due to low
enrollment numbers and not having enough new youth to start a new group.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation for fidelity monitoring of Thinking for a Change was reviewed. Fidelity
monitoring was missing for the months of December 2019 through February 2020. Of these
months, none were applicable, as groups were not running at that time.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the Director of Clinical Services verified the program has a process for
corrective actions based on the fidelity monitoring conducted. The program uses internal fidelity
monitoring/corrective action checklists specific to the Thinking for a Change curriculum. A review
of the documentation provided and an interview with the Director of Clinical Services verified there
were no corrective actions noted nor taken during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The performance evaluations of all trained facilitators found entries of the staff's performance and
delivery of the Thinking for a Change curriculum. These included comments on any strengths or
weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 15 weeks for this type of service. Of the 8 youth in the
sample, 100% (8 of 8) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 45 contact hours for this type of service. Of the 8 youth in the
sample, 88% (7 of 8) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 2 youth
Moderate-High = 3 youth
High = 3 youth
Total Youth in Sample = 8 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 38%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 3 youth
Total Youth in Sample = 8 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (8 of 8) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 38% (3 of 8) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 25 25 100%
Totals 98 100 98%
This SPEP report evaluates Thinking for a Change, an intervention delivered at Hillsborough Girls
Academy.
18 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 8 total youth sampled, all 8
included dosage with end dates in the EBS Module. Of those youth, 8 received at least the
recommended weeks of service. Youth in the sample completed between 16 and 19 weeks of
service, with an average of 19 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 8 total youth sampled,
all 8 included dosage in the EBS Module. Of those youth, 7 received at least the recommended
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
hours of service. Youth in the sample completed between 41 and 45 hours of service, with an
average of 45 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
RECOMMENDATION(S):
Hillsborough Girls Academy can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Hillsborough Girls Academy can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8


summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Hillsborough Girls Academy
9506 East Columbus Drive
(Contract Provider)
TrueCore Behavioral Solutions LLC
Tamp, Florida 33619
Primary Service: Female Healthy Relationships
SPEP Review Period: 08/01/20-07/31/21
SPEP Review Conducted: 8/10/21
Florida Department of Juvenile Justice
Report Date: 10/6/21
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Female Healthy Relationships.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes Female Healthy Relationships as a primary service. This service is a bundle
of Voices: A Program of Self-Discovery and Empowerment for Girls and The Teen Relationships
Workbook. One PRN Therapist, two full-time Therapists and one Clinical Director are trained in
the Female Healthy Relationships service. All trained staff received training from a qualified
trainer before delivering services to youth.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the Voices and Teen Relationships manuals for the Female Healthy
Relationships Bundle. Both curricula provide the script for the facilitator, the lessons, an
implementation guide, the order the lessons are to be delivered in and instructions for the delivery
of each lesson.
Teen Relationships is an open/ongoing group with no completed cohorts. Voices cohorts were
from 02/09/21-04/15/21 and 04/20/21-07/06/21. All youth are placed in the Female Healthy
Relationships groups.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Female Healthy Relationships, specifically the Teen Relationships Workbook
component, was completed via Zoom on Wednesday 08/11/21 from 2:30-3:30. The facilitator
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
followed the lesson plan, did not read verbatim, explained the lesson her own words and was
easy for the youth to understand. The facilitator adhered to the lesson plan in every aspect.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation provided and an interview with the Regional Director of Clinical
Services confirmed there was no facilitator turnover which adversely affected the services of
Female Healthy Relationships to the youth. This is an open and ongoing group and has run
consistently during the review dates and is running now. Voices had gaps in services from
08/02/20-08/10/20, 10/09/20-02/28/21, 04/16/21-04/19/21 and 07/21/07/31/21. The longest gap
was due to low enrollment numbers and not having enough new youth to start a new group until
02/29/21. The last Voices group ended on 07/06/21. The facility is again waiting on enough new
youth to start a new group. All successful completers were placed in Female Healthy
Relationships during their stay, regardless of gaps in service for the Voices component.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Internal fidelity monitoring was completed monthly for each facilitator by the prior Clinical Director
for Female Healthy Relationships. There was no missing monitoring.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program uses internal fidelity monitoring checklists specific to Female Healthy Relationships
(Voices and Teen Relationships). A review of the documentation provided and an interview with
the Director of Clinical Services confirmed there was no corrective action taken during this review
period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Annual performance evaluations for the facilitators of Female Healthy Relationships were
reviewed. The full-time and PRN Therapists, prior year evaluations had been completed and
included comments on any strengths or weaknesses of which each facilitator should be aware.
This year's evaluation for both are not due till 11/02/21. The other Therapist has not been
employed at HGA long enough for a yearly evaluation to be done.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 24 weeks for this type of service. Of the 22 youth in the
sample, 100% (22 of 22) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 40 contact hours for this type of service. Of the 22 youth in the
sample, 91% (20 of 22) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 91%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 5 youth
Moderate-High = 7 youth
High = 8 youth
Total Youth in Sample = 22 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 36%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 8 youth
Total Youth in Sample = 22 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 91% (20 of 22) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 36% (8 of 22) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 23 25 92%
Totals 91 95 96%
This SPEP report evaluates Female Healthy Relationships, an intervention delivered at
Hillsborough Girls Academy.
22 youth completed the program during the review period. Of those youth, all 22 were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 22 total youth sampled, all
22 included dosage with end dates in the EBS Module. Of those youth, 22 received at least the
recommended weeks of service. Youth in the sample completed between 26 and 79 weeks of
service, with an average of 51 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
The program earned 8 points for Amount of Service: Contact Hours. Of the 22 total youth
sampled, all 22 included dosage in the EBS Module. Of those youth, 20 received at least the
recommended hours of service. Youth in the sample completed between 24 and 75 hours of
service, with an average of 51 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
RECOMMENDATION(S):
Hillsborough Girls Academy can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Hillsborough Girls Academy can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8

file_name: FY2021-22 HGA FHR.pdf