summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Hillsborough Girls Academy
TrueCore Behavioral Solutions LLC
(Contract Provider)
9506 East Columbus Drive
Tampa, Florida 33619
Primary Service: Aggression Replacement Training
SPEP Review Period: 7/1/2023 - 6/30/2024
SPEP Review Conducted: 8/1/2024
Florida Department of Juvenile Justice
Report Date: 9/30/2025
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Aggression Replacement Training.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The documentation shows the program uses Aggression Replacement Training (ART). Facilitator
training consists of a three-day course provided by a qualified instructor. The documentation
showed there were four staff members trained in the primary service during this review period:
three therapists and one PRN therapist (former clinical director). The new clinical director will be
scheduled for the training. Two therapists facilitated Aggression Replacement Training..
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uaes the manual Aggression Replacement Training. This service is to be provided
three times each week over the course of ten (10) weeks to help youth develop social skill
competence, anger control, and moral reasoning skills. The program maintains all SPEP
documentation (attendance sheets and internal fidelity checks) in a separate binder labeled
Aggression Replacement Training. All youth admitted to the program are required to participate in
this primary service. Aggression Replacement Training is a closed group with no more than ten
(10) youth assigned to a cohort.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
An observation was conducted on August 1,2024 via Teams. The group started at 3:40pm and
ended at 4:30pm. There were six youth and one therapist in attendance. The group started with
the youth reviewing the homework from the prior class. The facilitator was knowledgeable of the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
material and made sure that the topics were understood. The facilitator used handouts and role
play to keep the youth engaged. The facilitator also used her own words to convey the lesson and
highlight pertinent points.
Basic Score: 0 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
According to the documentation for the annual review period of July 1, 2023 through June 30,
2024, the program has experienced facilitator turnover. There have been gaps in service during
this review period caused by a lack of trained facilitators. One therapist left the program on
4/24/24, the former clinical director took on a PRN therapist position with the program. There were
no cohorts completed during this review period for the months of 11/2023, 12/2023, 4/2024, and
5/2024.
One cohort started with the new contract which began 8/7/23-10/13/23. This group started with
eight youth and ended with seven youth; one youth was dropped from the group for non-
compliance.
The second cohort began on 1/22/24-3/29/24 and started and ended with eight youth.
The third cohort began on 6/23/24-and is currently ongoing as of 6/30/24.
An informal interview with the therapist standing in for the clinical director confirmed that the
program will replace one facilitator with another qualified facilitator should there be any staff
turnover in the middle of a cohort.
The program can earn 2 points if there are no unnecessary gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of the documentation for the annual review period showed there were missing monthly
internal fidelity monitoring checks. An informal interview with the therapist (former regional clinical
director) standing in for the clinical director confirmed that there were missing fidelity monitorings
due to a transitioning period in the program. The documentation shows the former clinical director
and a therapist conducted the monthly internal fidelity monitoring checks for this primary service.
Internal fidelity monitoring checks were completed on 8/23/23, 8/25/23, 8/28/23, 9/6/23, 9/8/23,
9/18/23, 10/9/23, 10/9/23, 10/11/23, 10/13/23, 2/21/24, 2/26/24, 3/4/24, and 3/27/24.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
A review of the documentation showed there were no corrective actions identified for the
facilitators of Aggression Replacement Training during the review period. During an informal
interview with the therapist (former regional clinical director) standing in for the clinical director, it
was explained that if a need for corrective action is found, the facilitator will receive progressive
discipline to include coaching, retraining, and formal write-up(s).
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
There were three annual preformance evaluations completed for the review period of July 1, 2023
to June 30, 2024. The former clinical director and two therapist had annual evaluations on
November 16, 2023.
Pursuant to the facility's Standard Program Evaluation Protocol procedure, an annual evaluation
of performance will be conducted on all the facilitators of each primary service. The performance
evaluation is comprised of feedback from the fidelity monitoring tools as well as an assessment of
the facilitator’s skill in delivering the primary service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Due to the Juvenile Justice Information System (JJIS) outage, this report does not contain data for
the period of March 2024 through June 2024. Data contained in this report is representative from
the period of July 2023 through February 2024.
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 10 weeks for this type of service. Of the 9 youth in the
sample, 89% (8 of 9) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Due to the Juvenile Justice Information System (JJIS) outage, this report does not contain data for
the period of March 2024 through June 2024. Data contained in this report is representative from
the period of July 2023 through February 2024.
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 30 contact hours for this type of service. Of the 9 youth in the
sample, 89% (8 of 9) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 7 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 28%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 78%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 2 youth
Moderate-High = 5 youth
High = 0 youth
Total Youth in Sample = 9 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 0%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 0 youth
Total Youth in Sample = 9 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 78% (7 of 9) youth scored Moderate to High-Risk to reoffend, for a score of
7 points.
Of the SPEP sample, 0% (0 of 9) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 7 25 28%
Totals 68 100 68%
This SPEP report evaluates Aggression Replacement Training, an intervention delivered at
Hillsborough Girls Academy.
Due to the Juvenile Justice Information System (JJIS) outage, this report does not contain data for
the period of March 2024 through June 2024. Data contained in this report is representative from
the period of July 2023 through February 2024.
21 youth completed the program during the review period. Of those youth, 9 were included in the
sample.
The program scored Medium for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 9 total youth sampled, only
9 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 8
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
received at least the recommended weeks of service. Youth in the sample completed between 5
and 10 weeks of service, with an average of 9.44 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 9 total youth sampled,
only 9 included dosage in the EBS Module. Of those youth with correct dosage, 8 received at
least the recommended hours of service. Youth in the sample completed between 13 and 30
hours of service, with an average of 28.11 hours.
The targeted number of weeks for this specific intervention is 10 weeks; the targeted number of
hours for this specific intervention is 30 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 5 services (15 weeks; 45
hours).
The program was awarded 7 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 86%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Hillsborough Girls Academy can optimize their SPEP Quality of Service Delivery score by
ensuring there are no unnecessary gaps in service and fidelity monitoring is completed every
month for each facilitator.
Hillsborough Girls Academy can optimize their SPEP Amount of Service score by ensuring
dosage for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

report_type: SPEP

file_name: FY2023-24 Hillsborough Girls Academy ART.pdf

program_name_raw: Hillsborough Girls Academy

program_status_note: Closed

service_type: Aggression Replacement Training


state: FL

facility_name: Hillsborough Juvenile Detention Center West

summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Hillsborough Girls Academy
TrueCore Behavioral Solutions LLC
(Contract Provider)
9506 East Columbus Drive
Tampa, Florida 33619
Primary Service: Cannabis Youth Treatment
SPEP Review Period: 7/1/2023 - 6/30/2024
SPEP Review Conducted: 8/1/2024
Florida Department of Juvenile Justice
Report Date: 9/30/2025
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Cannabis Youth Treatment.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 5 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 25%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 3 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program uses Cannabis Youth Treatment (CYT) as a primary service. The prorgam's clinical
staff are trained to conduct CYT groups. There were no cohorts during this annual review period
due to there being no SAOS youth admitted since the start of the program’s new contract.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the appropriate Cannabis Youth Treatment (CYT) manual. The manual
includes the following: implementation information, facilitator requirements, session requirements,
script, and worksheets for the lessons being provided. Cannabis Youth Treatment is designed for
youth ages 12-18 with a diagnosis of cannabis abuse or dependence. The youth must be
experiencing emotional, physical, legal, social, and/or academic problems associated with
marijuana use to be considered an appropriate candidate to receive this service.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Cannabis Youth Treatment was not able to be observed during this review period; therefore, this
indicator rates as non‐applicable.
d. Facilitator Turnover Basic Score: 0 Point(s)
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program has not run the primary service due to there being no SAOS youth admitted since
the start of the program’s new contract. The program has a protocol in the event of facilitator
turnover for Cannabis Youth Treatment (CYT) groups. In the event of facilitator turnover, the co-
facilitator will assume the role of the facilitator for the group. If a co-facilitator is not present, a new
qualified facilitator for the primary service will take over conducting the service.
The program can earn 2 points if there are no unnecessary gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 0 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has not run the primary service due to there being no SAOS youth admitted since
the start of the program’s new contract.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 0 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
It is unknown if there were any corrective actions needed since the program has not provided the
service and has not completed any internal fidelity monitorings.
Pursuant to the facility's Standard Program Evaluation Protocol procedure, based on the results of
the monthly fidelity monitoring, corrective action may be needed. The first stage of addressing
areas identified as needing improvement through fidelity monitoring is by providing coaching and
supervision to the facilitator on the issue by a qualified facilitator in the primary service. The
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report
additional coaching and/or supervision will be documented by the person conducting the fidelity
monitoring in a note and placed in the clinical supervision file. If during additional fidelity
monitoring the previously identified area addressed continues to be an issue, the next stage is
completing a written warning and placing the facilitator on a Performance Improvement Plan. The
Performance Improvement Plan will state the issue identified, the tasks the facilitator needs to
complete to correct the issue, and a target date for completion. The plan will then be reviewed
with the facilitator by the qualified facilitator conducting the monitoring, and the facilitator’s
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
supervisor. In the event an issue identified during the primary service is detrimental to the youth’s
safety and/or treatment, the matter will be taken up directly with the program’s Human Resource
Manager and be addressed in accordance with the employee handbook.
The program can earn 1 point if any corrective action needed is identified, applied, and
demonstrated.
At the time of the review, the program did not meet criteria to earn a score of 1.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Pursuant to the facility's Standard Program Evaluation Protocol procedure, an annual evaluation
of performance will be conducted on all the facilitators of each primary service. The performance
evaluation is comprised of feedback from the fidelity monitoring tools as well as an assessment of
the facilitator’s skill in delivering the primary service.
The program can earn 1 point if any corrective action needed is identified, applied, and
demonstrated.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
There was no data entered into the Juvenile Justice Information System (JJIS) Evidence Based
Module (EBS) for the primary service for the annual review period. There was no data available to
enter into the report.
Basic Score: <---Select from Drop-Down---> Points
Program Optimization Score: 10 Points
Program Optimization Percentage: %
Research indicates the target duration of weeks for this type of service. Of the
youth in the sample, % ( of ) reached at least the indicated target duration.
Further explanation is detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
There was no data entered into the Juvenile Justice Information System (JJIS) Evidence Based
Module (EBS) for the primary service for the annual review period. There was no data available to
enter into the report.
Basic Score: <---Select from Drop-Down---> Points
Program Optimization Score: 10 Points
Program Optimization Percentage: %
Research indicates a target of contact hours for this type of service. Of the youth in
the sample, % ( of ) reached the indicated target contact hours. Further
explanation is detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: Points
Program Optimization Score: 25 Points
Program Optimization Percentage: %
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: %
Moderate to High Score: <---Select from Drop-Down---> Points
Program Optimization Score: 12 Points
Program Optimization Percentage: %
Table 1
Moderate = youth
Moderate-High = youth
High = youth
Total Youth in Sample = youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: %
High Score: <---Select from Drop-Down---> Points
Program Optimization Score: 13 Points
Program Optimization Percentage: %
Table 2
High = youth
Total Youth in Sample = youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, % ( of ) youth scored Moderate to High-Risk to reoffend,
for a score of <---Select Points---> points.
Of the SPEP sample, % ( of ) youth scored High-Risk to reoffend, for a score of
<---Select Points---> points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 5 20 25%
Amount of Service: Duration 10 %
Amount of Service: Contact Hours 10 %
Risk Level of Youth Served 25 %
Totals %
This SPEP report evaluates Cannabis Youth Treatment, an intervention delivered at Hillsborough
Girls Academy.
There was no data entered into the Juvenile Justice Information System (JJIS) Evidence Based
Module (EBS) for the primary service for the annual review period. There was no data available to
enter into the report.
21 youth completed the program during the review period. Of those youth, 0 were included in the
sample.
The program scored Low for Quality of Service Delivery.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned XXXX points for Amount of Service: Duration. Of the XXXX total youth
sampled, only XXXX included dosage with end dates in the EBS Module. Of those youth with
correct dosage, XXXX received at least the recommended weeks of service. Youth in the sample
completed between XXXX and XXXX weeks of service, with an average of XXXX weeks.
The program earned XXXX points for Amount of Service: Contact Hours. Of the XXXX total youth
sampled, only XXXX included dosage in the EBS Module. Of those youth with correct dosage,
XXXX received at least the recommended hours of service. Youth in the sample completed
between XXXX and XXXX hours of service, with an average of XXXX hours.
The program earned 0 points for Amount of Service: Duration and Amount of Service: Contact
Hours. There was no data entered into the Juvenile Justice Information system (JJIS) Evidence-
Based Services (EBS) module for any youths. With no JJIS data to review, the SPEP score was
reduced. The program must enter dosage data into JJIS in order to receive any score in this
area.
The targeted number of weeks for this specific intervention is 12 weeks; the targeted number of
hours for this specific intervention is 12 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 5 services (15 weeks; 45
hours).
The program was awarded XXXX available points for Risk Level of Youth Served. This is
calculated using data from the Community Assessment Tool (CAT). This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
RECOMMENDATION(S):
Hillsborough Girls Academy can optimize their SPEP Quality of Service Delivery score by
ensuring there are no unnecessary gaps in service, fidelity monitoring is completed every month
for each facilitator, any corrective action needed is identified, applied, and demonstrated, and
each facilitator’s performance evaluation includes discussion of their skill in delivering this primary
service.
Hillsborough Girls Academy can optimize their SPEP Amount of Service score by ensuring
dosage for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

source: DJJ

file_name: FY2023-24 Hillsborough Girls Academy CYT.pdf

service_type: Cannabis Youth Treatment


summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Hillsborough Girls Academy
TrueCore Behavioral Solutions LLC
(Contract Provider)
9506 East Columbus Drive
Tampa, Florida 33619
Primary Service: Impact of Crime
SPEP Review Period: 7/1/2023 - 6/30/2024
SPEP Review Conducted: 8/1/2024
Florida Department of Juvenile Justice
Report Date: 9/30/2025
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The documentation shows the program uses Impact of Crime as a primary service. Facilitator
training consists of a three day 24-hour training provided by the Department of Juvenile Justice.
This training is available to new facilitators both in-person and via Microsoft Teams. The
documentation showed that there were five staff members trained in the primary service during
this review period: one assistant facility administrator (former), one therapist, one director of case
management, one transitional service manager, and one case manager. One case manager
facilitated Impact of Crime during this review period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the manual titled Impact of Crime: Addressing the Harm. The manual includes
seven chapters and provides a script for the facilitator to follow. The curriculum includes a student
guide with exercises for the participants to complete that are aimed at reinforcing the material
presented during group sessions. The program does not assign a student guide to each individual
participant; rather the facilitator makes copies of the exercises and provides one to each youth
during the lesson. The director of case management indicated that the program conducts Impact
of Crime on Tuesdays and Thursdays from 1:15pm to 2:15pm. The program maintains all SPEP
documentation (attendance sheets and internal fidelity checks) in a separate binder labeled
Impact of Crime. All youth admitted to the program are required to participate in this primary
service. Impact of Crime is a closed group with no more than twelve (12) youth assigned to a
cohort.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
An observation of Impact of Crime was conducted in person on August 18, 2023. The group
started at 2:27pm and ended 3:30pm with one facilitator and six youth. The topic covered was
Chater Five: “The Ripple Effect of Crime.” The facilitator was able to keep the youth engaged by
using a video that highlighted the topic of “Poor Decision Making”. The facilitator did read from the
manual however she made sure that the youth had a clear understanding of what was being
covered before moving to the next topic.
Basic Score: 0 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
According to the documentation for the annual review period of July 1, 2023, through June 30,
2024, the program has experienced facilitator turnover. This turnover did result in a gap in service
lasting more than thirty days which affected the delivery of Impact of Crime to the youth. One
facilitator left the program in September 2023, and another facilitator left in February 2024.
One cohort began 6/12/23-9/6/23. This group started with ten youth and ended with eight youth.
One youth was discharged from the program on 7/3/23 and one youth was arrested on 7/5/23.
The second cohort began on 10/30/23- 2/15/24 and started with eight youth and ended with six
youth. One youth was dropped for three refusals on 11/13/23 and one youth was arrested on
11/27/23.
The third cohort began on 3/12/24-and is currently ongoing as of 6/30/24. This group started with
eight youth; one youth was arrested on 5/8/24.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation showed that there were missing monthly internal fidelity monitoring checks.
An informal interview with the therapist (former regional clinical director) standing in for the clinical
director confirmed that there were missing fidelity monitorings for the months of 7/23,12/23, 5/24,
and 6/24 due to a transition period within the program. The documentation shows that the former
assistant facility administrator and director of case management conducted the monthly internal
fidelity monitoring checks for Impact of Crime. Internal fidelity monitoring checks were completed
on 8/18/23, 9/1/23, 10/30/23, 11/15/23, 1/2/24, 2/15/24, 3/19/24, and 4/23/24.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
A review of the documentation showed there were no corrective actions identified for the facilitator
of Impact of Crime given during the review period. During an informal interview with the therapist
(former regional clinical director) standing in for the clinical director and the director of case
management, it was explained if a need for corrective action is found, the facilitator will receive
progressive discipline to include coaching, retraining, and formal write-up(s).
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
There were three annual preformance evaluations completed for the review period of July 1, 2023
to June 30, 2024. The director of case management, and both former case mangers had annual
evaluations on November 16, 2023.
Pursuant to the facility's Standard Program Evaluation Protocol procedure, an annual evaluation
of performance will be conducted on all the facilitators of each primary service. The performance
evaluation is comprised of feedback from the fidelity monitoring tools as well as an assessment of
the facilitator’s skill in delivering the primary service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 16 weeks for this type of service. Of the 21 youth in the
sample, 62% (13 of 21) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 24 contact hours for this type of service. Of the 21 youth in the
sample, 90% (19 of 21) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 5 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 20%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 67%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 4 youth
Moderate-High = 8 youth
High = 2 youth
Total Youth in Sample = 21 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 10%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 2 youth
Total Youth in Sample = 21 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 67% (14 of 21) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 10% (2 of 21) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 5 25 20%
Totals 49 85 58%
This SPEP report evaluates Impact of Crime, an intervention delivered at Hillsborough Girls
Academy.
21 youth completed the program during the review period. Of those youth, 21 were included in the
sample.
The program scored Medium for Quality of Service Delivery.
The program earned 6 points for Amount of Service: Duration. Of the 21 total youth sampled, only
19 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 13
received at least the recommended weeks of service. Youth in the sample completed between 13
and 29 weeks of service, with an average of 17 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 8 points for Amount of Service: Contact Hours. Of the 21 total youth
sampled, only 19 included dosage in the EBS Module. Of those youth with correct dosage, 19
received at least the recommended hours of service. Youth in the sample completed between 32
and 59 hours of service, with an average of 35 hours.
The program was awarded 5 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 81%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Hillsborough Girls Academy can optimize their SPEP Quality of Service Delivery score by
ensuring facilitator turnover does not impact groups and fidelity monitoring is completed every
month for each facilitator.
Hillsborough Girls Academy can optimize their SPEP Amount of Service score by ensuring
dosage for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2023-24 Hillsborough Girls Academy IOC.pdf

service_type: Impact of Crime