summary: SPEP - FY15-16

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Tampa Residential Facility
G4S Youth Services, LLC
(Contract Provider)
9508 East Columbus Drive
Tampa, Florida 33619
Primary Service: Pathways to Self‐Discovery and Change
SPEP Review Date(s): March 8-11, 2016
Florida Department of Juvenile Justice
Report Date(s): 4/3/2017
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Tampa Residential Facility MQI Program Code: 1281
Provider Name: G4S Youth Services, LLC Contract Number: 10098
Location: Hillsborough County / Circuit: 13 Number of Beds: 60
Review Date(s): March 8-11, 2016 Lead Reviewer Code: 119
Persons Interviewed
Program Director Corporate QI/QA staff 3 # Program Supervisors
DJJ Monitor 1 # Case Managers 7 # Youth
DHA or designee 1 # Clinical Staff 1 # Other (listed by title): Lead
DMHCA or designee 2 # Healthcare Staff Teacher
Documents Reviewed
Written Protocol/Manual Logbooks 7 # Personnel Records
Fidelity Monitoring Documents Program Schedules 7 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 5 # Youth Records (Closed)
Staff Evaluations Table of Organization 7 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports 7 # Health Records
Contract Scope of Services 7 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Pathways to Self-Discovery and Change. The program
was awarded 25 points because the Primary Service is identified as a Group 4 Service. The
specific Sub-Component Service Type identified is Group Counseling. The Primary Service was
identified as this type of service as it focuses on psychological or interpersonal problems or issues
faced by an individual and involves a group of youths interacting with each other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program has three facilitators for the delivery of the Pathways curriculum. The program has
two additional staff who conduct fidelity monitoring. There is documentation to reflect all identified
staff have been trained by a qualified trainer to facilitate groups.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes a written manual for the provision of this primary service. There are a total of
thirty-two lessons for this curriculum.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A Pathways to Self-Discovery group was observed by the review team. The facilitator followed the
lesson plan outlined by the facilitator and student guide, and the facilitator completed all
objectives for the session. The facilitator adhered to the lesson plan in every aspect.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Reviewed documentation and interviews with the corporate clinical services director and the
clinical director confirmed there has been no turnover adversely affecting the youth during the
past twelve months. This service ran June 2015 to September 2015. A new cycle started in
January 2016. The program does not schedule to run this service more often.
BasicScore: 2Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has conducted monthly fidelity monitoring for this primary service by trained
facilitators. The assigned fidelity monitor completes an observation checklist for the session they
observe. Each checklist includes a section for feedback and recommendations for facilitator
improvement.
BasicScore: 1Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has conducted monthly fidelity monitoring for this primary service by trained
facilitators. The assigned fidelity monitor completes an observation checklist for the session they
observe. This checklist includes an evaluation of the session and any corrective action required
based on that observation. The checklist is signed by the fidelity monitor and the facilitator(s)
providing the service. There has not been a need for corrective action based of fidelity monitoring
during this review cycle.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program identified three staff who provide this primary service. All reviewed performance
evaluations included an evaluation of their skills in delivering this primary service. Each evaluation
included comments on any strengths and weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 24 weeks for this type of service. Of the 8 youth in the
sample, 0% (0 of 8) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 40 contact hours for this type of service. Of the 8 youth in the
sample, 0% (0 of 8) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 0 youth
Moderate-High = 3 youth
High = 5 youth
Total Youth in Sample = 8 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 63%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 5 youth
Total Youth in Sample = 8 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (8 of 8) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 63% (5 of 8) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 25 25 100%
Totals 75 95 79%
This SPEP report evaluates Pathways to Self-Discovery and Change, an intervention delivered at
Tampa Residential Facility.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 8 total youth sampled, none
received at least the recommended weeks of service. Youth in the sample completed between 15
and 22 weeks of service, with an average of 16 weeks.
The program earned 0 points for Amount of Service: Contact Hours. Of the 8 total youth sampled,
none received at least the recommended hours of service. All youth in the sample completed 32
hours of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Tampa Residential Facility can maintain their SPEP Quality of Service Delivery score by
continuing the same processes in place at the time of this review.
Tampa Residential Facility can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

service_type: Pathways to Self‐Discovery and Change


state: FL

facility_name: Tampa Residential Facility

summary: SPEP - FY15-16

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Tampa Residential Facility
G4S Youth Services, LLC
(Contract Provider)
9508 East Columbus Drive
Tampa, Florida 33619
Primary Service: Thinking for a Change (T4C)
SPEP Review Date(s): March 8-11, 2016
Florida Department of Juvenile Justice
Report Date(s): 4/3/2017
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Tampa Residential Facility MQI Program Code: 1281
Provider Name: G4S Youth Services, LLC Contract Number: 10098
Location: Hillsborough County / Circuit: 13 Number of Beds: 60
Review Date(s): March 8-11, 2016 Lead Reviewer Code: 119
Persons Interviewed
Program Director Corporate QI/QA staff 3 # Program Supervisors
DJJ Monitor 1 # Case Managers 7 # Youth
DHA or designee 1 # Clinical Staff 1 # Other (listed by title): Lead
DMHCA or designee 2 # Healthcare Staff Teacher
Documents Reviewed
Written Protocol/Manual Logbooks 7 # Personnel Records
Fidelity Monitoring Documents Program Schedules 7 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 5 # Youth Records (Closed)
Staff Evaluations Table of Organization 7 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports 7 # Health Records
Contract Scope of Services 7 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change (T4C). The program was awarded
30 points because the Primary Service is identified as a Group 5 Service. The specific Sub-
Component Service Type identified is Cognitive Behavioral Therapy. The Primary Service was
identified as this type of service as it is intended to correct faulty cognitions and perceptions and
provides skills individuals can use to monitor thought patterns and correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program has three facilitators for the delivery of the Thinking for a Change curriculum. Two
additional staff conduct fidelity monitoring. Documentation reflects all staff have been trained by a
qualified trainer.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program is using the Thinking for a Change (T4C) curriculum for all youth admitted to the
program. The program contains twenty-five lessons.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
This curriculum was not able to be observed; therefore, this indicator rates as non-applicable. The
last cycle ended on March 7, 2016. The program is not currently conducting a cycle of T4C and is
in between sessions.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Reviewed documentation and interviews with the corporate clinical services director and the
clinical director confirmed there has been no turnover adversely affecting the youth during the
past twelve months. The program has a policy and procedures in place to address gaps in service
delivery. There were no gaps in service delivery longer than one month during the review period.
BasicScore: 2Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has conducted monthly fidelity monitoring for this primary service by trained
facilitators. The assigned fidelity monitor completes an observation checklist for the session they
observe. Each checklist includes a section for feedback and recommendations for facilitator
improvement.
BasicScore: 1Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has conducted monthly fidelity monitoring for this primary service by trained
facilitators. The assigned fidelity monitor completes an observation checklist for the session they
observe. This checklist includes an evaluation of the session and any corrective action required
based on the observation. The checklist is signed by the fidelity monitor and the facilitator(s)
providing the service. There has not been a need for corrective action based of fidelity monitoring
this review cycle.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program identified three staff who provide this primary service. All three performance
evaluations reviewed included an evaluation of their skill in delivering this primary service. These
included comments on any strengths and weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service ‐ Duration
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates the target duration of 15 weeks for this type of service. Of the 7 youth in the
sample, 57% (4 of 7) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 45 contact hours for this type of service. Of the 7 youth in the
sample, 0% (0 of 7) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 0 youth
Moderate-High = 1 youth
High = 6 youth
Total Youth in Sample = 7 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 86%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 6 youth
Total Youth in Sample = 7 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (7 of 7) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 86% (6 of 7) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 4 10 40%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 25 25 100%
Totals 84 100 84%
This SPEP report evaluates Thinking for a Change, an intervention delivered at Tampa
Residential Facility.
The program scored High for Quality of Service Delivery.
The program earned 4 points for Amount of Service: Duration. Of the 7 total youth sampled, 4
received at least the recommended weeks of service. Youth in the sample completed between 13
and 22 weeks of service, with an average of 18 weeks. This calculation combines the weeks
earned in Thinking For a Change with the weeks earned while completing Thinking For A Change
Aftercare sessions.
The program earned 0 points for Amount of Service: Contact Hours. Of the 7 total youth sampled,
none received at least the recommended hours of service. Youth in the sample completed
between 25 and 43 hours of service, with an average of 34 hours. This calculation combines the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
hours earned in Thinking For a Change with the hours earned while completing Thinking For A
Change Aftercare sessions.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Tampa Residential Facility can maintain their SPEP Quality of Service Delivery score by
continuing the same practices in place at the time of this review.
Tampa Residential Facility can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

source: DJJ

report_type: SPEP

program_name_raw: Tampa Residential Facility

program_status_note: Closed

service_type: Thinking for a Change


summary: SPEP - FY16-17

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Tampa Residential Facility
G4S Youth Services, LLC
(Contract Provider)
9508 East Columbus Drive
Tampa, Florida 33619
Primary Service: The Council For Boys and Young Men (Boys Council)
SPEP Review Date(s): March 14‐17, 2017
Florida Department of Juvenile Justice
Report Date(s): 6/21/2017
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Tampa Residential Facility MQI Program Code: 1281
Provider Name: G4S Youth Services, LLC Contract Number: 10098
Location: Hillsborough County / Circuit: 13 Number of Beds: 60
Review Date(s): March 14-17, 2017 Lead Reviewer Code: 119
Persons Interviewed
Program Director Corporate QI/QA staff 3 # Program Supervisors
DJJ Monitor 1 # Case Managers 7 # Youth
DHA or designee 1 # Clinical Staff # Other (listed by title):
DMHCA or designee 2 # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 7 # Personnel Records
Fidelity Monitoring Documents Program Schedules 7 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 5 # Youth Records (Closed)
Staff Evaluations Table of Organization 7 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports 7 # Health Records
Contract Scope of Services 7 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is The Council For Boys and Young Men (Boys Council).
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
Reviewed documentation reflected five therapists completed the training certification by the One
Circle Foundation are all currently employed by the program. The program has identified one
facilitator for the delivery of The Council curriculum. Four additional staff have also been trained;
however, they serve as substitutes or fidelity monitors. There is documentation to reflect that the
identified staff have been trained by a qualified trainer to facilitate groups. All staff members
identified have facilitated groups during the last twelve months.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses The Council for Boys and Young Men curriculum guide, "Living a Legacy: A
Rite of Passage" for all applicable youth. The curriculum contains the order of the lessons to be
delivered, lesson plans for each session, delivery guidelines, and a script to follow. The facilitator
guide has lesson plans with instructions for conducting each group session.
The manual also provides suggested and various activities in which the facilitator may choose
based on the group dynamics.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
This delinquency intervention was observed during this annual compliance review by the MQI
team. The facilitator provided a copy of the lesson plan from the program manual for the observer
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
to review. The facilitator followed the design of the program to include an opening ritual, theme
introduction, warm-up activity, check-in, activity, reflection, and closing ritual. The facilitator
adhered to the lesson plan in every aspect.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
Reviewed documentation and interviews with the facility administrator and regional clinical
director confirmed there had not been any gaps in service delivery nor any turnover of facilitators
over the past year. The program has a policy and procedures in place to address gaps in service
delivery.
BasicScore: 2Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Fidelity monitoring was conducted in all applicable months for the facilitator of the primary service
by other mental health staff trained in the provision of the curriculum, and used a form designed
by the publisher. These signed documents are maintained in a Standardized Program Evaluation
Protocol (SPEP) binder along with sign-in sheets. Each checklist includes a section for feedback
and recommendations for facilitator improvement.
BasicScore: 1Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program’s fidelity monitoring reports did not reveal any indication corrective action had been
required. An interview with the clinical director and the primary facilitator revealed no corrective
action had been required in the last twelve months.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
An evaluation of the facilitator’s skill in delivering this primary service was addressed in the staff’s
annual performance evaluation. This included an evaluation of the facilitator’s skill and quality of
work and also comments on any strengths or weaknesses.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 24 weeks for this type of service. Of the 17 youth in the
sample, 0% (0 of 17) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 40 contact hours for this type of service. Of the 17 youth in the
sample, 0% (0 of 17) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 0 youth
Moderate-High = 3 youth
High = 14 youth
Total Youth in Sample = 17 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 82%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 14 youth
Total Youth in Sample = 17 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (17 of 17) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 82% (14 of 17) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 25 25 100%
Totals 75 95 79%
This SPEP report evaluates The Council For Boys and Young Men (Boys Council), an
intervention delivered at Tampa Residential Facility.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 17 total youth sampled,
none received at least the recommended weeks of service. Youth in the sample completed
between 8 and 23 weeks of service, with an average of 20 weeks.
The program earned 0 points for Amount of Service: Contact Hours. Of the 17 total youth
sampled, none received at least the recommended hours of service. Youth in the sample
completed between 12 and 36 hours of service, with an average of 31 hours.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Tampa Residential Facility can optimize their SPEP Amount of Service score by ensuring that
youth receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

service_type: The Council for Boys and Young Men


summary: SPEP - FY16-17

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Tampa Residential Facility
G4S Youth Services, LLC
(Contract Provider)
9508 East Columbus Drive
Tampa, Florida 33619
Primary Service: Impact of Crime (IOC)
SPEP Review Date(s): March 14‐17, 2017
Florida Department of Juvenile Justice
Report Date(s): 6/21/2017
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Tampa Residential Facility MQI Program Code: 1281
Provider Name: G4S Youth Services, LLC Contract Number: 10098
Location: Hillsborough County / Circuit: 13 Number of Beds: 60
Review Date(s): March 14-17, 2017 Lead Reviewer Code: 119
Persons Interviewed
Program Director Corporate QI/QA staff 3 # Program Supervisors
DJJ Monitor 1 # Case Managers 7 # Youth
DHA or designee 1 # Clinical Staff 1 # Other (listed by title): Lead
DMHCA or designee 2 # Healthcare Staff Teacher
Documents Reviewed
Written Protocol/Manual Logbooks 7 # Personnel Records
Fidelity Monitoring Documents Program Schedules 7 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 3 # Youth Records (Closed)
Staff Evaluations Table of Organization 7 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports 7 # Health Records
Contract Scope of Services 7 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime (IOC). The program was awarded 15
points because the Primary Service is identified as a Group 3 Service. The specific Sub-
Component Service Type identified is Social Skills Training. The Primary Service was identified
as this type of service as it focuses on developing social skills required for an individual to interact
in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
During the last year, the program has had four different staff members facilitate Impact of Crime
(IOC). All three cycles of IOC were co-facilitated by two of the trained facilitators; however, all four
of the facilitators are still working at the program. A review of the facilitators’ training files
contained documentation showing each facilitator had been trained by a qualified trainer in the
IOC curriculum.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program has a written manual outlining the Impact of Crime (IOC) curriculum. The program is
currently using the manual entitled ‘Impact of Crime: Addressing the Harm’ published in
September 2011, Version 2.0. A review of the manual indicated the curriculum consists of seven
chapters and takes approximately twenty-four sessions to facilitate. The manual was detailed and
provided an explanation as to how each session was to be delivered and contained worksheets
for the youth to complete during each session. The manual also provides suggested and various
activities in which the facilitator may choose based on the group dynamics.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The primary service was co-facilitated by two qualified and trained staff. The group was held
every Monday and Wednesday from 11:20 a.m. to 12:20 p.m. and was observed by a regional
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
monitor. The regional monitor received a copy of the lesson plan being presented to the group
prior to the class observation. The monitor observed the co-facilitators using their own words to
convey the meaning of the topic under discussion and did not read the lesson verbatim from the
manual. The facilitators took turns presenting the information to the group and were both easy to
follow. Both facilitators engaged the youth and had various youth read and answer the exercises.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program conducted two full cycles of Impact of Crime (IOC) and was currently presenting
their third cycle since the last annual compliance review. None of the groups had been affected by
facilitator turnover. The first IOC group started on April 19, 2016 and ended on August 16, 2016.
The program ran another primary service group from November 14, 2016 through January 5,
2017. The program began their current IOC group on February 6, 2017. An interview with the
regional director of clinical services indicated any gap from primary service to primary service was
due to the facility waiting for new youth to enter the facility and preparing them to enter a new
group.
BasicScore: 2Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
There were four different staff members who performed fidelity monitoring for the Impact of Crime
(IOC) curriculum since the last annual compliance review. A review of all four staff member’s
training files indicated they had been trained by a qualified trainer in the primary service. A review
of the previous twelve months of IOC fidelity adherence checklist forms indicated fidelity
monitoring was completed monthly for all facilitators who provided the primary service. The
internal fidelity adherence checklist evaluated the facilitator regarding their presentation of the
material, as wells as adherence to the manual.
BasicScore: 1Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the regional director of clinical services revealed the program has a process for
corrective action based on fidelity monitoring of the primary services. A review of the internal
fidelity adherence checklists indicated corrective action had not been required since the last
annual review.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1Point
Intervention Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program conducts employee evaluations ninety-days after hire and then annually based on
the employee’s hire date. The program identified four staff who provide this primary service. All
four performance evaluations reviewed included an evaluation of their skills in delivering this
primary service. These included comments on any strengths and weaknesses of which the
facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 16 weeks for this type of service. Of the 10 youth in the
sample, 60% (6 of 10) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 10 youth in the
sample, 100% (10 of 10) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 0 youth
Moderate-High = 2 youth
High = 8 youth
Total Youth in Sample = 10 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 80%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 8 youth
Total Youth in Sample = 10 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (10 of 10) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 80% (8 of 10) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 25 25 100%
Totals 81 85 95%
This SPEP report evaluates Impact of Crime (IOC), an intervention delivered at Tampa
Residential Facility.
The program scored High for Quality of Service Delivery.
The program earned 6 points for Amount of Service: Duration. Of the 10 total youth sampled, 6
received at least the recommended weeks of service. Youth in the sample completed between 11
and 17 weeks of service, with an average of 15 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 10 total youth
sampled, all received at least the recommended hours of service. Youth in the sample completed
between 24 and 28 hours of service, with an average of 26 hours.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Tampa Residential Facility can optimize their SPEP Amount of Service score by ensuring that all
youth receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

service_type: Impact of Crime


summary: SPEP - FY16-17

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Tampa Residential Facility
G4S Youth Services, LLC
(Contract Provider)
9508 East Columbus Drive
Tampa, Florida 33619
Primary Service: LifeSkills Training (LST)
SPEP Review Date(s): March 14‐17, 2017
Florida Department of Juvenile Justice
Report Date(s): 6/21/2017
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Tampa Residential Facility MQI Program Code: 1281
Provider Name: G4S Youth Services, LLC Contract Number: 10098
Location: Hillsborough County / Circuit: 13 Number of Beds: 60
Review Date(s): March 14-17, 2017 Lead Reviewer Code: 119
Persons Interviewed
Program Director Corporate QI/QA staff 3 # Program Supervisors
DJJ Monitor 1 # Case Managers 7 # Youth
DHA or designee 1 # Clinical Staff 1 # Other (listed by title): Lead
DMHCA or designee 2 # Healthcare Staff Teacher
Documents Reviewed
Written Protocol/Manual Logbooks 10 # Personnel Records
Fidelity Monitoring Documents Program Schedules 15 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 5 # Youth Records (Closed)
Staff Evaluations Table of Organization 7 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports 7 # Health Records
Contract Scope of Services 7 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is LifeSkills Training (LST). The program was awarded 15
points because the Primary Service is identified as a Group 3 Service. The specific Sub-
Component Service Type identified is Social Skills Training. The Primary Service was identified
as this type of service as it focuses on developing social skills required for an individual to interact
in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program identified five staff as facilitators for the delivery of Life Skills Training. There is
documentation to reflect that the identified staff have been trained by a qualified trainer to facilitate
groups. All five staff members have facilitated groups during the last twelve months.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program has the brand-name manual for Life Skills Training (LST) and also provides the
associated LST student workbooks/manuals to each youth. A review of the facilitator manual,
bound in a three-ring binder with tabbed sections, showcased the LST curriculum with the
provision of instructions as to how the service is to be provided, the order the lessons are to be
presented, as well as, highly detailed documentation of all of the eighteen lesson plans of the
curriculum. The manual outlines each lesson plan providing a script, lesson plan vocabulary to
introduce to the youth, materials needed for the lesson, specific steps in the delivery of the lesson,
and a review of the goals of the lesson. There are also DVD’s in front pocket of the manual to be
used in specific lesson plans.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
During this review of the SPEP for Life Skills Training (LST), one group session was observed by
a member of the annual compliance review team. Prior to the group session, the facilitators
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
ensured the review team member was provided with a copy of the lesson plan and briefly
reviewed the goals of the session. There were ten youth present in the observed session. Each
youth seemed aware of the group process, expectations of the facilitator, and the format of the
lessons. Each youth was provided their own workbook to follow along with. The facilitator
presented the lesson’s vocabulary and encouraged the youth to engage in discussions, as stated
on the lesson plan. At no time did the facilitator read the lesson verbatim, instead, they chose
words which could be easily understood by the youth and often reiterated content to help these
youth, all with intellectual deficits, comprehend the gist of the lesson. Though the facilitator
skipped over two specific exercises specified in the lesson plan, they covered all of the lesson’s
goals and objectives, choosing to present the required information in a different way, for example,
foregoing a group exercise, keeping in mind the needs and abilities of the youth in the group.
When asked about the skipped exercises, the facilitator indicated LST training stressed the
importance of addressing the lesson objectives and did not require each and every lesson plan be
presented as written.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There has been no facilitator turnover during the past twelve months. The program has a policy
and procedures in place to ensure certified Life Skills Training (LST) facilitators are not assigned
to group cycles if there is no intention to maintain employment through to the end of the cycle.
Additionally, if, for some reason, a certified LST facilitator leaves the program unexpectedly, there
is a plan to replace the departing facilitator with a second trained facilitator on staff, who is familiar
to the youth. Over the past year, there were extended gaps in the provision of LST services. The
primary reason for these extended gap was reported to be the lack of enough youth in the
program to effectively begin a new LST cycle (the program prefers ten youth) during the gap
period.
BasicScore: 2Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has an in-house process to perform monthly internal fidelity monitoring of the
delivery of Life Skills Training (LST) services by a certified LST facilitator. Program practice
requires a certified LST facilitator to attend and observe one randomly selected group session
each month. During the monitored sessions, an internal LST fidelity checklist is completed. The
program provided documentation indicating the required monthly fidelity monitoring for this
primary service was consistently conducted from May 2016 through March 2017 for each of the
program’s five trained facilitators who provided LST services during this time period.
BasicScore: 1Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program’s monthly fidelity monitoring for Life Skills Training (LST) service delivery provides
the basis for the in-house corrective action process. First, each completed fidelity monitoring
requires the monitor to document their observations and to offer feedback and recommendations
for the facilitator to enhance his/her delivery of the service. Second, the monitor, who is also a
trained LST facilitator and the immediate supervisor of the LST facilitators, discusses the
feedback and recommendations, if any, with the facilitator and, if warranted, documents a
corrective action plan. Lastly, ongoing fidelity monitoring is conducted to determine if the needed
correction has been made. Improvements or additional corrective action recommendations are
then documented on the fidelity monitoring checklist. None of the reviewed monthly fidelity
monitoring sheets for any of the certified facilitators documented any need for a corrective action.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program conducts employee evaluations ninety-days after hire and annually in October. The
program identified two staff who provide this primary service. Both performance evaluations
reviewed included an evaluation of their skill in delivering this primary service. These included
comments on any strengths and weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 16 weeks for this type of service. Of the 10 youth in the
sample, 0% (0 of 10) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 24 contact hours for this type of service. Of the 10 youth in the
sample, 0% (0 of 10) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 90%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 0 youth
Moderate-High = 4 youth
High = 5 youth
Total Youth in Sample = 10 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 50%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 5 youth
Total Youth in Sample = 10 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 90% (9 of 10) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 50% (5 of 10) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 23 25 92%
Totals 63 85 74%
This SPEP report evaluates LifeSkills Training (LST), an intervention delivered at Tampa
Residential Facility.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 10 total youth sampled,
none received at least the recommended weeks of service. Youth in the sample completed
between 9 and 10 weeks of service.
The program earned 0 points for Amount of Service: Contact Hours. Of the 10 total youth
sampled, none received at least the recommended hours of service. Youth in the sample
completed between 17 and 18 hours of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Tampa Residential Facility can optimize their SPEP Amount of Service score by ensuring that
youth receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

service_type: LifeSkills Training


summary: SPEP - FY16-17

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Tampa Residential Facility
G4S Youth Services, LLC
(Contract Provider)
9508 East Columbus Drive
Tampa, Florida 33619
Primary Service: Criminal Conduct and Substance Abuse Treatment for
Adolescents: Pathways to Self‐Discovery and Change (Pathways)
SPEP Review Date(s): March 14‐17, 2017
Florida Department of Juvenile Justice
Report Date(s): 6/21/2017
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Tampa Residential Facility MQI Program Code: 1281
Provider Name: G4S Youth Services, LLC Contract Number: 10098
Location: Hillsborough County / Circuit: 13 Number of Beds: 60
Review Date(s): March 14-17, 2017 Lead Reviewer Code: 119
Persons Interviewed
Program Director Corporate QI/QA staff 3 # Program Supervisors
DJJ Monitor 1 # Case Managers 7 # Youth
DHA or designee 1 # Clinical Staff 1 # Other (listed by title): Lead
DMHCA or designee 2 # Healthcare Staff Teacher
Documents Reviewed
Written Protocol/Manual Logbooks 7 # Personnel Records
Fidelity Monitoring Documents Program Schedules 7 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 5 # Youth Records (Closed)
Staff Evaluations Table of Organization 7 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports 7 # Health Records
Contract Scope of Services 7 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Criminal Conduct and Substance Abuse Treatment for
Adolescents: Pathways to Self-Discovery and Change (Pathways). The program was awarded 25
points because the Primary Service is identified as a Group 4 Service. The specific Sub-
Component Service Type identified is Group Counseling. The Primary Service was identified as
this type of service as it focuses on psychological or interpersonal problems or issues faced by an
individual and involves a group of youths interacting with each other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program identified five staff as facilitators for the delivery of the Pathways curriculum. The
program has five staff trained, of which two staff conduct fidelity monitoring. There is
documentation to reflect all identified staff have been trained by a qualified trainer in order to
facilitate groups. All staff members have facilitated groups during the last twelve months.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes a written manual for the provision of this primary service. There are a total of
thirty-two lessons for this curriculum.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A Pathways to Self-Discovery group was observed during the annual compliance review by the
review team. The facilitator followed the lesson plan outlined by the facilitator and student guide,
and the facilitator completed all objectives for the session. The facilitator adhered to the lesson
plan in every aspect.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
Reviewed documentation and interviews with the corporate clinical services director and the
clinical director confirmed there has been no facilitator turnover during the past twelve months.
The program has a policy and procedures in place to address facilitator turnover and gaps in
service delivery. There have been no gaps in service delivery longer than one week.
BasicScore: 2Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has conducted monthly fidelity monitoring for this primary service by trained
facilitators. The assigned fidelity monitor completes an observation checklist for each session they
observe. Each checklist includes a section for feedback and recommendations for facilitator
improvement.
BasicScore: 1Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has conducted monthly fidelity monitoring for this primary service by trained
facilitators. The assigned fidelity monitor completes an observation checklist for each session they
observe. This checklist includes an evaluation of the session and any corrective action required
based on that observation. The checklist is signed by the fidelity monitor and the facilitator(s)
providing the service. There had not been a need for corrective action based of fidelity monitoring
this review cycle.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program identified five staff who provide this primary service. All five performance evaluations
reviewed included an evaluation of their skills in delivering this primary service. These included
comments on any strengths and weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 24 weeks for this type of service. Of the 10 youth in the
sample, 0% (0 of 10) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 40 contact hours for this type of service. Of the 10 youth in the
sample, 80% (8 of 10) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 90%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 0 youth
Moderate-High = 4 youth
High = 5 youth
Total Youth in Sample = 10 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 50%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 5 youth
Total Youth in Sample = 10 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 90% (9 of 10) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 50% (5 of 10) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 23 25 92%
Totals 81 95 85%
This SPEP report evaluates Criminal Conduct and Substance Abuse Treatment for
Adolescents: Pathways to Self-Discovery and Change (Pathways), an intervention delivered at
Tampa Residential Facility.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 10 total youth sampled,
none received at least the recommended weeks of service. Youth in the sample completed
between 12 and 21 weeks of service, with an average of 20 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 10 total youth
sampled, 8 received at least the recommended hours of service. Youth in the sample completed
between 32 and 42 hours of service, with an average of 40 hours.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Tampa Residential Facility can optimize their SPEP Amount of Service score by ensuring that
youth receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9