state: FL

summary: PREA - Year 3 Cycle 4

source: DJJ

report_type: PREA

pdf_url: https://www.djj.state.fl.us/content/download/1298182/file/Florida_Sothwest Regional Juvenile Detention Center_Year3Cycle4.pdf?version=1


facility_name: Charles Britt Academy - Sequel

summary: SPEP - FY16-17

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Charles Britt Academy
Sequel Youth and Family Services
(Contract Provider)
3001 26th Street South
St. Petersburg, Florida 33712
Primary Service: Impact of Crime (IOC)
SPEP Review Date(s): May 16‐19, 2017
Florida Department of Juvenile Justice
Report Date(s): 11/21/2017
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Charles Britt Academy MQI Program Code: 1279
Provider Name: Sequel Youth and Family Services Contract Number: 10092
Location: Pinellas County / Circuit: 6 Number of Beds: 28
Review Date(s): May 16-19, 2017 Lead Reviewer Code: 97
Persons Interviewed
Program Director Corporate QI/QA staff 3 # Program Supervisors
DJJ Monitor 2 # Case Managers 5 # Youth
DHA or designee 3 # Clinical Staff # Other (listed by title):
DMHCA or designee 2 # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 10 # Personnel Records
Fidelity Monitoring Documents Program Schedules 10 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 5 # Youth Records (Closed)
Staff Evaluations Table of Organization 3 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports 5 # Health Records
Contract Scope of Services 5 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime (IOC). The program was awarded 15
points because the Primary Service is identified as a Group 3 Service. The specific Sub-
Component Service Type identified is Social Skills Training. The Primary Service was identified
as this type of service as it focuses on developing social skills required for an individual to interact
in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program currently has three staff trained to provide Impact of Crime (IOC). The program
maintains all information regarding IOC in a binder. There was documentation in the binder to
support all staff received facilitator training in March 2017. All of the IOC groups were facilitated
by trained staff during the annual compliance review period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the Impact of Crime (IOC) curriculum which includes facilitator and student
guides. The curriculum is manualized and provides the frequency and duration for the service,
with specific topics to be covered in each lesson. The facilitator guide contains lesson plans with
instructions for conducting each group session.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
This curriculum was not able to be observed; therefore, this indicator rates as non-applicable.
d. Facilitator Turnover Basic Score: 1 Point(s)
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program had a trained facilitator for Impact of Crime (IOC), who started a cohort in January
2017; the facilitator resigned after conducting one IOC group, and there were no other trained
staff. Three staff were trained in the curricula in March 2017; a new cohort was started April 2,
2017.
The program can earn 2 points by minimizing facilitator turnover.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has documented fidelity monitoring for observations of the Impact of Crime (IOC)
curriculum, using a monitoring tool specific to the primary service. The fidelity monitoring was
completed by the program’s designated mental health clinician authority, who has been trained in
the IOC curriculum. The fidelity monitoring was completed each month the primary service was
provided.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program provides fidelity monitoring, which includes coaching and supervision by a qualified
facilitator in the primary service. The fidelity monitoring sheets completed on the Impact of Crime
groups for the past year were reviewed. There were notes of encouragement and suggestions;
however, there was no documentation on any reviewed monitoring sheets to suggest any
corrective action was required.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
There was an annual performance evaluation completed on the two staff who facilitate the Impact
of Crime groups. The evaluation forms did not contain specific information regarding the staff’s
delivery of the primary service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
The program can earn 1 point by ensuring that facilitators' performance evaluations contain an
assessment of their delivery of IOC.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 16 weeks for this type of service. Of the 0 youth in the
sample, 0% (0 of 0) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 24 contact hours for this type of service. Of the 0 youth in the
sample, 0% (0 of 0) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 0 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 0%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 0%
Moderate to High Score: 0 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 0%
Table 1
Moderate = 0 youth
Moderate-High = 0 youth
High = 0 youth
Total Youth in Sample = 0 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 0%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 0 youth
Total Youth in Sample = 0 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 0% (0 of 0) youth scored Moderate to High-Risk to reoffend, for a score of 0
points.
Of the SPEP sample, 0% (0 of 0) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 0 25 0%
Totals 30 85 35%
This SPEP report evaluates Impact of Crime (IOC), an intervention delivered at Charles Britt
Academy.
The program scored Medium for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration and Amount of Service: Contact
Hours. There was no data entered into the Juvenile Justice Information System (JJIS) Evidence-
Based Services Module (EBS) Module for any sampled youth. The program must enter dosage
data into JJIS in order to receive any score in this area.
The program was awarded 0 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered just prior to the youths' admission.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
RECOMMENDATION(S):
Charles Britt Academy can optimize their SPEP Quality of Service Delivery score by minimizing
facilitator turnover and by ensuring facilitators' performance evaluations contain an assessment of
their delivery of IOC.
Charles Britt Academy can optimize their SPEP Amount of Service score by ensuring that dosage
is recorded in the EBS Module and that youth receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

report_type: SPEP

program_name_raw: Charles Britt Academy - Sequel

program_status_note: Closed

service_type: Impact of Crime


summary: SPEP - FY16-17

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Charles Britt Academy
Sequel Youth and Family Services
(Contract Provider)
3001 26th Street South
St. Petersburg, Florida 33712
Primary Service: Living in Balance (LIB)
SPEP Review Date(s): May 16‐19, 2017
Florida Department of Juvenile Justice
Report Date(s): 11/21/2017
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Charles Britt Academy MQI Program Code: 1279
Provider Name: Sequel Youth and Family Services Contract Number: 10092
Location: Pinellas County / Circuit: 6 Number of Beds: 28
Review Date(s): May 16-19, 2017 Lead Reviewer Code: 97
Persons Interviewed
Program Director Corporate QI/QA staff 3 # Program Supervisors
DJJ Monitor 2 # Case Managers 5 # Youth
DHA or designee 3 # Clinical Staff # Other (listed by title):
DMHCA or designee 2 # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 10 # Personnel Records
Fidelity Monitoring Documents Program Schedules 10 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 5 # Youth Records (Closed)
Staff Evaluations Table of Organization 5 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports 5 # Health Records
Contract Scope of Services 5 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Living in Balance (LIB). The program was awarded 25
points because the Primary Service is identified as a Group 4 Service. The specific Sub-
Component Service Type identified is Group Counseling. The Primary Service was identified as
this type of service as it focuses on psychological or interpersonal problems or issues faced by an
individual and involves a group of youths interacting with each other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program currently has three staff trained to provide Living in Balance. The program maintains
all information regarding Living in Balance in a binder. There was documentation in the binder to
support all three staff received facilitator training. All of the Living in Balance groups were
facilitated by trained staff during the annual compliance review period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the Living in Balance curriculum which includes facilitator and student guides.
The curriculum is manualized and provides the frequency and duration for the service, with
specific topics to be covered in each lesson. The facilitator guide contains lesson plans with
instructions for conducting each group session.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Living in Balance was observed during the annual compliance review. A copy of the
chapter of the manual was presented to the reviewer prior to the observation. The group was
facilitated by a trained facilitator, who is a clinician. The group started with a review of the
information from the previous group. The session focused on relapse decisions and triggers. The
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
youth were encouraged to participate, and their comments were well regarded by the facilitator.
The lesson was provided as prescribed by the manual, in all aspects. The youth were comfortable
with the lesson and the format.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program has three trained facilitators for Living in Balance. The current cohort started March
30, 2017. There were no gaps in service of the primary service due to facilitator turnover.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has documented fidelity monitoring for observations of the Living in Balance
curriculum, using a monitoring tool specific to the primary service. The fidelity monitoring was
completed by the program’s designated mental health clinician authority, who has been trained in
the curriculum. The fidelity monitoring was completed each month the primary service was
provided.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program provides fidelity monitoring, which includes coaching and supervision by a qualified
facilitator in the primary service. The fidelity monitoring sheets completed on the Living in Balance
for the review period were reviewed. There were notes of encouragement and suggestions;
however, there was no documentation on any reviewed monitoring sheets to suggest any
corrective action was required.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
There was an annual performance evaluation completed on the three staff who facilitate the Living
in Balance groups. The evaluation forms did not contain specific information regarding the staff’s
delivery of the primary service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
The program can earn 1 point by ensuring that facilitators' performance evaluations include an
assessment of their skill in the delivery of LIB.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 24 weeks for this type of service. Of the 3 youth in the
sample, 0% (0 of 3) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 40 contact hours for this type of service. Of the 3 youth in the
sample, 0% (0 of 3) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 22 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 88%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 2 youth
Moderate-High = 0 youth
High = 1 youth
Total Youth in Sample = 3 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 33%
High Score: 10 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 77%
Table 2
High = 1 youth
Total Youth in Sample = 3 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (3 of 3) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 33% (1 of 3) youth scored High-Risk to reoffend, for a score of 10 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 22 25 88%
Totals 72 95 76%
This SPEP report evaluates Living in Balance (LIB), an intervention delivered at Charles Britt
Academy.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. None received at least the
recommend weeks of service, with sample youth completing an average of 5 weeks of service.
The program earned 0 points for Amount of Service: Contact Hours. None received at least the
recommended hours of service, with sample youth completing an average of 13 hours of service.
The program was awarded 22 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered just prior to the youths' admission.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
RECOMMENDATION(S):
Charles Britt Academy can optimize their SPEP Quality of Service Delivery score by ensuring that
facilitators' performance evaluations include an assessment of their skill in the delivery of LIB.
Charles Britt Academy can optimize their SPEP Amount of Service score by ensuring that youth
receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

service_type: Living in Balance


summary: SPEP - FY16-17

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Charles Britt Academy
Sequel Youth and Family Services
(Contract Provider)
3001 26th Street South
St. Petersburg, Florida 33712
Primary Service: Thinking For A Change (T4C)
SPEP Review Date(s): May 16‐19, 2017
Florida Department of Juvenile Justice
Report Date(s): 11/21/2017
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Charles Britt Academy MQI Program Code: 1279
Provider Name: Sequel Youth and Family Services Contract Number: 10092
Location: Pinellas County / Circuit: 6 Number of Beds: 28
Review Date(s): May 16-19, 2017 Lead Reviewer Code: 97
Persons Interviewed
Program Director Corporate QI/QA staff 3 # Program Supervisors
DJJ Monitor 2 # Case Managers 5 # Youth
DHA or designee 3 # Clinical Staff # Other (listed by title):
DMHCA or designee 2 # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 10 # Personnel Records
Fidelity Monitoring Documents Program Schedules 10 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 5 # Youth Records (Closed)
Staff Evaluations Table of Organization 3 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports 5 # Health Records
Contract Scope of Services 5 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking For A Change (T4C). The program was
awarded 30 points because the Primary Service is identified as a Group 5 Service. The specific
Sub-Component Service Type identified is Cognitive Behavioral Therapy. The Primary Service
was identified as this type of service as it is intended to correct faulty cognitions and perceptions
and provides skills individuals can use to monitor thought patterns and correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program currently has two staff trained to provide Thinking For a Change (T4C). The program
maintains all information regarding T4C in a binder. There was documentation in the binder to
support both staff received facilitator training. All of the T4C groups were facilitated by trained staff
during the review period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the Thinking For A Change (T4C) curriculum which includes facilitator and
student guides. The curriculum is manualized and provides the frequency and duration for the
service, with specific topics to be covered in each lesson. The facilitator guide contains lesson
plans with instructions for conducting each group session.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
This curriculum was not able to be observed; therefore, this indicator rates as non-applicable.
d. Facilitator Turnover Basic Score: 2 Point(s)
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program has two trained facilitators for T4C. The first cohort started January 11, 2017, and
was completed in May 2017. The second cohort was scheduled to start the week following the
annual compliance review. There were no gaps in service of the primary service due to facilitator
turnover.
BasicScore: 2Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has documented fidelity monitoring for observations of the Thinking For A Change
(T4C) curriculum, using a monitoring tool specific to the primary service. The fidelity monitoring
was completed by the program’s designated mental health clinician authority, who has been
trained in the T4C curriculum. The fidelity monitoring was completed each month the primary
service was provided.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program provides fidelity monitoring, which includes coaching and supervision by a qualified
facilitator in the primary service. The fidelity monitoring sheets completed on Thinking For A
Change (T4C) for the review period were reviewed. There were notes of encouragement and
suggestions; however, there was no documentation on any reviewed monitoring sheets to
suggest any corrective action was required.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
There was an annual performance evaluation completed on the two staff who facilitate the
Thinking For A Change groups. The evaluation forms did not contain specific information
regarding the staff’s delivery of the primary service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
The program can earn 1 point by ensuring that facilitators' performance evaluations include an
assessment of their skill in the delivery of T4C.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 15 weeks for this type of service. Of the 0 youth in the
sample, 0% (0 of 0) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 45 contact hours for this type of service. Of the 0 youth in the
sample, 0% (0 of 0) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 0 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 0%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 0%
Moderate to High Score: 0 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 0%
Table 1
Moderate = 0 youth
Moderate-High = 0 youth
High = 0 youth
Total Youth in Sample = 0 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 0%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 0 youth
Total Youth in Sample = 0 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 0% (0 of 0) youth scored Moderate to High-Risk to reoffend, for a score of 0
points.
Of the SPEP sample, 0% (0 of 0) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 0 25 0%
Totals 55 100 55%
This SPEP report evaluates Thinking For A Change (T4C), an intervention delivered at Charles
Britt Academy.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration and Amount of Service: Contact
Hours. There was no data entered into the Juvenile Justice Information System (JJIS) Evidence-
Based Services Module (EBS) Module for any sampled youth. The program must enter dosage
data into JJIS in order to receive any score in this area.
The program was awarded 0 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered just prior to the youths' admission.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
RECOMMENDATION(S):
Charles Britt Academy can optimize their SPEP Quality of Service Delivery score by ensuring
facilitators' performance evaluations contain an assessment of their delivery of T4C.
Charles Britt Academy can optimize their SPEP Amount of Service score by ensuring that data for
all youth is entered in the EBS Module of JJIS and that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

service_type: Thinking for a Change


summary: SPEP - FY17-18

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Charles Britt Academy
Sequel Youth and Family Services
(Contract Provider)
3001 26th Ave South
St. Petersburg, Florida 33712
Primary Service: Impact of Crime
SPEP Review Date(s): February 6 ‐ 9, 2018
Florida Department of Juvenile Justice
Report Date(s): February 7, 2018
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Charles Britt Academy MQI Program Code: 1279
Provider Name: Sequel Youth and Family Services Contract Number: 10092
Location: Pinellas County / Circuit: 6 Number of Beds: 28
Review Date(s): February 6 - 9, 2018 Lead Reviewer Code: 140
Persons Interviewed
Program Director Corporate QI/QA staff # Program Supervisors
DJJ Monitor 1 # Case Managers 5 # Youth
DHA or designee 1 # Clinical Staff # Other (listed by title):
DMHCA or designee # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 10 # Personnel Records
Fidelity Monitoring Documents Program Schedules 10 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 3 # Youth Records (Closed)
Staff Evaluations Table of Organization 5 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports 5 # Health Records
Contract Scope of Services 5 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime. The program was awarded 15 points
because the Primary Service is identified as a Group 3 Service. The specific Sub-Component
Service Type identified is Social Skills Training. The Primary Service was identified as this type of
service as it focuses on developing social skills required for an individual to interact in a positive
way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program currently has two facilitators trained in the Impact of Crime (IOC) curriculum. The
program had a third facilitator who provided IOC curriculum to the youth in the program over the
last year; however, the facilitator is no longer employed by the program. One of the facilitators at
the program is the designated mental health clinician authority, and the other facilitator is a youth
care worker. The facilitator who is no longer with the program held the position of a youth care
worker. Two of the facilitators conducted groups since the last standardized program evaluation
protocol review. Each facilitator maintains a copy of the group sign-in sheets in a binder entitled
IOC. A review of the facilitators training records contained documentation each facilitator was
trained to facilitate IOC groups. A qualified trainer provided training for all facilitators in the primary
service.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the manual entitled ‘Impact of Crime: addressing the harm’, distributed by
residential services with the Florida Department of Juvenile Justice. The program is currently
using volume two, which was published in September 2011. The program’s manual is very detail
oriented and explains how each specific session is to be delivered, and there is an outline for
each session and section. An interview with the program’s designated mental health clinician
authority indicated all youth admitted to the program are placed in the Impact of Crime (IOC)
groups, and should complete the curriculum prior to being released from the program. Since the
last standardized program evaluation protocol (SPEP) review the program has conducted two full
cycles of IOC and a third was started; however, it was stopped due to the facilitator resigning from
the program. The SPEP missing dosage form was not required for this primary service.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Impact of Crime groups were not currently being conducted due to facilitator turnover. The
primary service could not be observed during the annual compliance review; therefore, the
indicator rates as non-applicable.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program conducted two full cycles of Impact of Crime (IOC) and started a third group;
however, the third group was not completed because the facilitator resigned from the program,
and there was not another trained facilitator to take over the group. Reviewed documentation
confirmed there was no facilitator turnover during the two full cycles of IOC, and the same
facilitator conducted each cycle of with no change in facilitators. There was approximately a
month between the start of the first and second cycle of IOC; however, the second cycle of IOC
was cancelled due to the facilitator resigning. Another facilitator was trained and the third cycle of
IOC was started within a month of the cancelled group. The third cycle of IOC finished the day
before the reviewer was on-site to review the primary service.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The team reviewed internal fidelity monitoring since the last standardized program evaluation
protocol review. The program has an internal process to conduct internal fidelity monitoring for all
facilitators of Impact of Crime (IOC) monthly, if the group is being conducted during the month.
The program uses a form entitled IOC fidelity adherence checklist to conduct fidelity monitoring. A
review of the program’s documentation revealed there were two facilitators of the primary service
who required monthly fidelity monitoring. Each facilitator received fidelity monitoring each month
they facilitated a group, except for one facilitator who did not receiving fidelity monitoring during a
two-month period. The program’s designated mental health clinician authority (DMHCA)
conducted all IOC fidelity monitoring activities for the two facilitators. The DMHCA is also a trained
facilitator of IOC and has been trained to conducted fidelity monitoring.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process by which corrective action is applied and demonstrated based on the
fidelity monitoring of the primary service. The program uses an internal fidelity adherence
checklist to specify if a facilitator needs corrective action or improvement. A review of the
completed checklists since the last standardized program evaluation protocol review indicated
there was no corrective action or improvement needed for the facilitators who conducted the
Impact of Crime groups.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of employee records for the three facilitators who conducted the Impact of Crime groups
since the last annual review, found only one facilitator had an evaluation conducted by their
supervisor. The evaluation addressed the facilitators skills in delivering the primary service. The
other facilitator who is currently employed by the facility has not been with the program long
enough to have an evaluation completed. The third facilitator is not currently employed with the
company and was not employed long enough to have an evaluation completed.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 16 weeks for this type of service. Of the 16 youth in the
sample, 0% (0 of 16) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates a target of 24 contact hours for this type of service. Of the 16 youth in the
sample, 56% (9 of 16) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 2 youth
Moderate-High = 4 youth
High = 10 youth
Total Youth in Sample = 16 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 63%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 10 youth
Total Youth in Sample = 16 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (16 of 16) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 63% (10 of 16) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 4 10 40%
Risk Level of Youth Served 25 25 100%
Totals 59 85 69%
This SPEP report evaluates Impact of Crime, an intervention delivered at Charles Britt Academy.
The program scored Medium for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 16 total youth sampled, only
10 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 0
received at least the recommended weeks of service. Youth in the sample completed between 11
and 14 weeks of service, with an average of 11 weeks.
The program earned 4 points for Amount of Service: Contact Hours. Of the 16 total youth
sampled, all 16 included dosage in the EBS Module. Of those youth, 9 received at least the
recommended hours of service. Youth in the sample completed between 4 and 24 hours of
service, with an average of 16 hours.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered just prior to the youths' admission.
RECOMMENDATION(S):
Charles Britt Academy can optimize their SPEP Quality of Service Delivery score by ensuring
facilitator turnover does not impact groups, and ensuring fidelity monitoring is completed every
month for each facilitator
Charles Britt Academy can optimize their SPEP Amount of Service score by ensuring that dosage
for all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10


summary: SPEP - FY17-18

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Charles Britt Academy
Sequel Youth and Family Services
(Contract Provider)
3001 26th Ave South
St. Petersburg, Florida 33712
Primary Service: Living in Balance
SPEP Review Date(s): February 6 ‐ 9, 2018
Florida Department of Juvenile Justice
Report Date(s): February 8, 2018
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Charles Britt Academy MQI Program Code: 1279
Provider Name: Sequel Youth and Family Services Contract Number: 10092
Location: Pinellas County / Circuit: 6 Number of Beds: 28
Review Date(s): February 6 - 9, 2018 Lead Reviewer Code: 140
Persons Interviewed
Program Director Corporate QI/QA staff # Program Supervisors
DJJ Monitor 1 # Case Managers 5 # Youth
DHA or designee 1 # Clinical Staff # Other (listed by title):
DMHCA or designee # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 10 # Personnel Records
Fidelity Monitoring Documents Program Schedules 10 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 3 # Youth Records (Closed)
Staff Evaluations Table of Organization 5 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports 5 # Health Records
Contract Scope of Services 5 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Living in Balance. The program was awarded 25 points
because the Primary Service is identified as a Group 4 Service. The specific Sub-Component
Service Type identified is Group Counseling. The Primary Service was identified as this type of
service as it focuses on psychological or interpersonal problems or issues faced by an individual
and involves a group of youths interacting with each other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program currently has three facilitators trained in the Living in Balance (LIB) curriculum;
however, two prior employees facilitated LIB since the last standardized program evaluation
protocol review for a total of five facilitators. One of the facilitators at the program is the
designated mental health clinician authority, and the other two facilitators are therapists. The
program had five staff facilitate LIB groups since the last standardized program evaluation
protocol review. The program maintains the group sign-in sheets in a binder entitled LIB. A review
of the facilitators training records contained documentation all five facilitators received traning to
facilitate LIB groups. A qualified trainer provided training for all facilitators in the primary service.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the manual entitled ‘Living in Balance - moving from a life of addiction to a life
of recovery', published by Jeffery Hoffman, Mim J. Landry and Barry Caudill. The program has a
facilitator guide and a student guide. The program’s facilitator manual is detail oriented and
explains how each specific session is to be delivered, and contains an outline for each session.
An interview with the program’s designated mental health clinician authority indicated all youth
admitted to the program are placed into Living in Balance (LIB) groups, and should complete the
curriculum prior to being released from the program. Since the last standardized program
evaluation protocol (SPEP) review, the program has conducted two full cycles of LIB and is
currently in the process of conducting a third cycle. The SPEP missing dosage form was not
required for the primary service.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The primary service was currently being conducted on Wednesday and Thursday afternoons.
Prior to observation of the group, the facilitator provided a copy of the lesson plan from the
manual to the observer. The lesson plan outlined the lesson being covered in the class being
observed, which allowed the observer to ensure the facilitator was adhering to the written
protocol. The reviewer was able to observe the group which occurred on Thursday afternoon from
12:30 p.m. to 1:30 p.m. The lesson being covered was session nine: skills for reducing stress.
There was one facilitator, one direct care staff member, and thirteen youth in the group room.
Reviewer observed the facilitator does not read the lesson verbatim, but uses their own words to
convey the meaning of the topic and does so in an easy to follow manner. The group sat in chairs
facing around the walls of the group room, but facing the white board in the front of the room. The
facilitator engaged the youth to participate in the lesson and the youth actively participated in the
group by giving examples of relaxtion activites they like to do. The youth even walked to the front
of the room and wrote their relaxation activity on the white board. The facilitator positively
reinforced the youth’s answers and redirected the youth when necessary. The facilitator was
respectful and encouraging to the youth. During the group the facilitator conducted a relaxation
excerise with the youth. The facilitator had the youth do deep breathing and talked in a low, soft
voice, and encouraged the youth to relax and de-stress. After the exercise the facilitator went over
the purpose of the relaxation exercise and encouraged the youth to use the skill to relax, de-
stress, help their mood and help them focus. At the end of the group, several youth thanked the
facilitator for conducting a great group.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program conducted two full cycles of Living In Balance (LIB) and is currently in the process of
conducting a third cycle. Reviewed documentation confirmed during one of the cycles a facilitator
resigned and the group was taken over by another trained facilitator. An interview with the
designated mental health clinician (DMHCA) indicated when the program is aware staff who are
faciliating a group will no longer be faciliating the group, the DMHCA notifies the youth in the
group of the change, and introduces the new faciliator to the group prior to the first faciliator
leaving. There was less than a month gap between each cycle of LIB.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The team reviewed internal fidelity monitoring since the last standardized program evaluation
protocol review. The program has an internal process for conducting internal fidelity monitoring for
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
all facilitators of Living in Balance (LIB) monthly, if the group is being conducted during the
particular month. The program uses a form entitled LIB fidelity adherence checklist to conduct
fidelity monitoring. A review of the program’s documentation revealed there were five facilitators of
the primary service who required monthly fidelity monitoring. Four of the five facilitators received
fidelity monitoring on a monthly basis from the designated mental health clinician authority
(DMHCA) when they facilitated a group. The DMHCA faciliated a group when they had to take
over for a facilitator who left the program and received fidelity monitoring during the month they
faciliated the group. The DMHCA received fidelity monitoring from another trained LIB facilitator.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process by which corrective action is applied and demonstrated based on the
fidelity monitoring of the primary service. The program uses an internal fidelity adherence
checklist to specify if a facilitator needs corrective action or improvement. A review of the
completed checklists since the last standardized program evaluation protocol review indicated
there was no corrective action or improvement needed for the facilitators who conducted the
Living in Balance groups.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of employee records for the five facilitators who conducted Living in Balance (LIB)
groups since the last review of the primary service found two facilitators had an evaluation
conducted by their supervisor. The evaluations addressed each facilitators skills in delivering the
primary service. Two of the faciliators are no longer employeed by the program and were not
employed long enough to be evaluated on how they delivered the primary service. One faciliator
was recently trained in the primary service and has yet to receive an evaluation since they were
trained.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 24 weeks for this type of service. Of the 18 youth in the
sample, 0% (0 of 18) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 40 contact hours for this type of service. Of the 18 youth in the
sample, 94% (17 of 18) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 2 youth
Moderate-High = 4 youth
High = 12 youth
Total Youth in Sample = 18 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 67%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 12 youth
Total Youth in Sample = 18 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (18 of 18) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 67% (12 of 18) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 25 25 100%
Totals 83 95 87%
This SPEP report evaluates Living in Balance, an intervention delivered at Charles Britt Academy.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 18 total youth sampled, only
13 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 0
received at least the recommended weeks of service. Youth in the sample completed between 5
and 21 weeks of service, with an average of 10 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 18 total youth
sampled, all 18 included dosage in the EBS Module. Of those youth, 17 received at least the
recommended hours of service. Youth in the sample completed between 38 and 69 hours of
service, with an average of 51 hours.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered just prior to the youths' admission.
RECOMMENDATION(S):
Charles Britt Academy can maintain their SPEP Quality of Service Delivery score by continuing
the practices in place at the time of this review.
Charles Britt Academy can optimize their SPEP Amount of Service score by ensuring that dosage
for all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10


summary: SPEP - FY17-18

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Charles Britt Academy
Sequel Youth and Family Services
(Contract Provider)
3001 26th Ave South
St. Petersburg, Florida 33712
Primary Service: Thinking for a Change
SPEP Review Date(s): February 6 - 9, 2018
Florida Department of Juvenile Justice
Report Date(s): February 7, 2018
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Charles Britt Academy MQI Program Code: 1279
Provider Name: Sequel Youth and Family Services Contract Number: 10092
Location: Pinellas County / Circuit: 6 Number of Beds: 28
Review Date(s): February 6 - 9, 2018 Lead Reviewer Code: 140
Persons Interviewed
Program Director Corporate QI/QA staff # Program Supervisors
DJJ Monitor 1 # Case Managers 5 # Youth
DHA or designee 1 # Clinical Staff # Other (listed by title):
DMHCA or designee # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 10 # Personnel Records
Fidelity Monitoring Documents Program Schedules 10 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 3 # Youth Records (Closed)
Staff Evaluations Table of Organization 5 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports 5 # Health Records
Contract Scope of Services 5 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change. The program was awarded 30
points because the Primary Service is identified as a Group 5 Service. The specific Sub-
Component Service Type identified is Cognitive Behavioral Therapy. The Primary Service was
identified as this type of service as it is intended to correct faulty cognitions and perceptions and
provides skills individuals can use to monitor thought patterns and correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program currently has three facilitators trained in the Thinking for a Change (T4C) curriculum.
One of the facilitators at the program is the designated mental health clinician authority, and the
other two facilitators are youth care workers. All three of the facilitators conducted groups since
the last standardized program evaluation protocol review. Each facilitator maintains a copy of the
group sign-in sheets in a binder entitled T4C. A review of the facilitators training records contained
documentation each facilitator was trained to facilitate T4C groups. A qualified trainer provided
training for all facilitators in the primary service.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the manual entitled ‘Thinking for a Change', put out by the national institute of
corrections. The program is currently using version four of the manual. The program’s manual is
detail oriented and explains how each specific session is to be delivered, and there is an outline
for each session and section. An interview with the program’s designated mental health clinician
authority indicated all youth admitted to the program are placed into Thinking for a Change (T4C)
groups, and should complete the curriculum prior to being released from the program. Since the
last standardized program evaluation protocol (SPEP) review the program has conducted two full
cycles of T4C and is currently in the process of conducting a third cycle. The SPEP missing
dosage form was not required for the primary service.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The primary service was currently being conducted on Tuesday and Saturday afternoons. The
primary service could not be observed during the annual compliance review due to the facilitator
having a family emergency and not being able to facilitate the group while the review team was
on-site.
This curriculum was not able to be observed; therefore, this indicator rates as non-applicable
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program conducted two full cycles of Thinking for a Change (T4C) and is currently in the
procecss of conducting a third cycle. Reviewed documentation confirmed there was no facilitator
turnover during any of the groups previously facilitated or the current group. The same facilitator
conducted each cycle of T4C with no change in facilitators or the addition of a co-facilitator. There
was approximately a two month gap between the end of the first cycle and beginning of the
second cycle of T4C. There was approximately a three month gap between the end of the second
cycle and beginning of the third cycle of T4C. An interview with the designated mental health
clinician (DMHCA) indicated the gaps in service were to allow other services to be provided to the
youth and to ensure the program had enough new youth to start a group. The DMHCA further
indicated if the program is aware staff who are faciliating a group can no longer faciliate the group,
the DMHCA notify the youth in the group, and will attempt to introduce a new faciliator to the
group prior to the faciliator leaving.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The team reviewed internal fidelity monitoring since the last standardized program evaluation
protocol review. The program has an internal process for conducting internal fidelity monitoring for
all facilitators of Thinking for a Change (T4C) monthly, if the group is being conducted during the
month. The program uses a form entitled T4C fidelity adherence checklist to conduct fidelity
monitoring. A review of the program’s documentation revealed there were three facilitators of the
primary service who required monthly fidelity monitoring. Two of the three facilitators received
fidelity monitoring monthly from the designated mental health clinician authority (DMHCA) when
they facilitated a group during the month. The third facilitator was the DMHCA and they received
fidelity monitoring two out of the four months they facilitated T4C groups. The DMHCA received
fidelity monitoring from another trained T4C facilitator.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process by which corrective action is acknowledged and demonstrated based
on the fidelity monitoring of the primary service. The program uses an internal fidelity adherence
checklist to specify if a facilitator needs corrective action or improvement. A review of the
completed checklists since the last standardized program evaluation protocol review indicated
there was no corrective action or improvement needed for the facilitators who conducted the
Thinking for a Change groups.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of employee records for the three facilitators who conducted Thinking for a Change
(T4C) groups since the last annual compliance review, found two facilitators had an evaluation
conducted by their supervisor. The evaluations addressed the facilitators skills in delivering the
primary service. The third facilitator has not been with the program long enough to have an
evaluation completed by their supervsior.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service - Duration
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates the target duration of 15 weeks for this type of service. Of the 9 youth in the
sample, 22% (2 of 9) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 45 contact hours for this type of service. Of the 9 youth in the
sample, 0% (0 of 9) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 1 youth
Moderate-High = 1 youth
High = 7 youth
Total Youth in Sample = 9 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 78%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 7 youth
Total Youth in Sample = 9 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (9 of 9) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 78% (7 of 9) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 2 10 20%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 25 25 100%
Totals 82 100 82%
This SPEP report evaluates Thinking for a Change, an intervention delivered at Charles Britt
Academy.
The program scored High for Quality of Service Delivery.
The program earned 2 points for Amount of Service: Duration. Of the 9 total youth sampled, only
7 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 2
received at least the recommended weeks of service. Youth in the sample completed between 14
and 15 weeks of service, with an average of 14 weeks.
The program earned 0 points for Amount of Service: Contact Hours. Of the 9 total youth sampled,
all 9 included dosage in the EBS Module. Of those youth, 0 received at least the recommended
hours of service. Youth in the sample completed between 25 and 37 hours of service, with an
average of 29 hours.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered just prior to the youths' admission.
RECOMMENDATION(S):
Charles Britt Academy can maintain their SPEP Quality of Service Delivery score by continuing
the practices in place at the time of this review.
Charles Britt Academy can optimize their SPEP Amount of Service score by ensuring that dosage
for all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10