summary: SPEP - FY16-17

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Lake Academy
G4S Youth Services, LLC
(Contract Provider)
9504 East Columbus Drive
Tampa, Florida 33619
Primary Service: Thinking for a Change
SPEP Review Date(s): January 31‐February 3, 2017
Florida Department of Juvenile Justice
Report Date(s): 7/21/2017
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Lake Academy QI Program Code: 1069
Provider Name: G4S Youth Services, LLC Contract Number: R2104
Location: Hillsborough County / Circuit: 13 Number of Beds: 48
Review Date(s): January 31-February 2, 2017 Lead Reviewer Code: 140
Persons Interviewed
Program Director Corporate QI/QA staff 2 # Program Supervisors
DJJ Monitor 2 # Case Managers 5 # Youth
DHA or designee 2 # Clinical Staff # Other (listed by title):
DMHCA or designee 2 # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 7 # Personnel Records
Fidelity Monitoring Documents Program Schedules 7 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 7 # Youth Records (Closed)
Staff Evaluations Table of Organization 7 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports 7 # Health Records
Contract Scope of Services 7 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change. The program was awarded 30
points because the Primary Service is identified as a Group 5 Service. The specific Sub-
Component Service Type identified is Cognitive Behavioral Therapy. The Primary Service was
identified as this type of service as it is intended to correct faulty cognitions and perceptions and
provides skills individuals can use to monitor thought patterns and correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program had four facilitators trained in the Thinking for a Change curriculum during the
review period. All four facilitators conducted group in the last twelve-month period. A review of all
four training files contained documentation that all received formal training in the Thinking for a
Change curriculum. A qualified trainer trained all four facilitators in the primary service.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
All youth admitted at Lake Academy are assessed by staff as needing evidence based services.
Based on various criteria, they are placed into program services. The program is utilizing the
Thinking for a Change curriculum, Version 3.1. The manual details the primary service and
explains how each specific session is to be delivered. It includes a script, gives outlines for each
group session, and provides specific details on the delivery of each of the 25 lessons.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The primary service is usually held each Thursday and Saturday from 6:00 pm to 7:00 pm.
Thinking for a Change was observed on Thursday, 02/02/17. The lesson facilitated was Lesson
13, Social Skill-Apologizing. The facilitator adhered to the written protocol, did not read from the
script verbatim and explained the lesson in his own words which was understandable to the youth.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The reviewed documentation and the Clinical Director confirmed there was staff turnover in the
last twelve months, however, that turnover did not adversely affect the youth due to proper
transitioning of the facilitators between groups. There were no gaps in service. The program has a
policy for facilitator turnover. If a facilitator is going to be leaving the program or will be on leave,
they explain to the youth why another facilitator will be taking over the class. Another qualified
facilitator from the staff would conduct the class.
BasicScore: 2Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Internal fidelity monitoring reports were reviewed. The program has an internal process to conduct
internal fidelity monitoring specific to Thinking for a Change. The LMHC and Shift Supervisor now
perform all fidelity monitoring. Both are also trained in the Thinking for a Change curriculum. The
fidelity monitors have received certification in the primary service and fidelity monitoring. The
program utilizes fidelity adherence checklists to document fidelity monitoring. Fidelity monitoring
has been conducted monthly on the staff members who facilitated the Thinking for a Change
class within the last twelve months.
BasicScore: 1Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with Clinical Director verified that the program has a process for corrective actions
based on fidelity monitoring conducted. The program uses internal fidelity monitoring/correction
action checklists specific to Thinking for a Change. Each monitoring checklist includes a section
for feedback and recommendations for facilitator improvement. There were no corrective actions
pertaining to the Thinking for a Change service in the last 12 months.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Performance evaluations were completed on three of the staff who facilitated the Thinking for a
Change intervention in the past twelve months. One facilitator was not due for an evaluation as of
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
the review date. The evaluations were reviewed. No entries were found that were specific to
facilitating the Thinking for a Change intervention.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 15 weeks for this type of service. Of the 11 youth in the
sample, 82% (9 of 11) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 45 contact hours for this type of service. Of the 11 youth in the
sample, 0% (0 of 11) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 0 youth
Moderate-High = 6 youth
High = 5 youth
Total Youth in Sample = 11 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 45%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 5 youth
Total Youth in Sample = 11 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (11 of 11) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 45% (5 of 11) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 25 25 100%
Totals 83 100 83%
This SPEP report evaluates Thinking for a Change, an intervention delivered at Lake Academy.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 11 total youth sampled, all
included dosage with end dates in the EBS Module. Of those youth with correct dosage, 9
received at least the recommended weeks of service. Youth in the sample completed between 12
and 41 weeks of service, with an average of 21 weeks.
The program earned 0 points for Amount of Service: Contact Hours. Of the 11 total youth
sampled, all included dosage in the EBS Module. Of those youth with correct dosage, none
received at least the recommended hours of service. Youth in the sample completed between 24
and 41 hours of service, with an average of 33 hours.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Lake Academy can optimize their SPEP Quality of Service Delivery score by ensuring each
facilitator’s performance evaluation includes discussion of their skill in delivering this primary
service.
Lake Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10


state: FL

facility_name: Lake Academy

summary: SPEP - FY16-17

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Lake Academy
G4S Youth Services, LLC
(Contract Provider)
9504 East Columbus Drive
Tampa, Florida 33619
Primary Service: VOICES
SPEP Review Date(s): January 31‐February 3, 2017
Florida Department of Juvenile Justice
Report Date(s): 7/21/2017
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Lake Academy QI Program Code: 1069
Provider Name: G4S Youth Services, LLC Contract Number: R2104
Location: Hillsborough County / Circuit: 13 Number of Beds: 48
Review Date(s): January 31-February 2, 2017 Lead Reviewer Code: 140
Persons Interviewed
Program Director Corporate QI/QA staff 2 # Program Supervisors
DJJ Monitor 2 # Case Managers 5 # Youth
DHA or designee 2 # Clinical Staff # Other (listed by title):
DMHCA or designee 2 # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 7 # Personnel Records
Fidelity Monitoring Documents Program Schedules 7 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 7 # Youth Records (Closed)
Staff Evaluations Table of Organization 7 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports 7 # Health Records
Contract Scope of Services 7 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is VOICES. The program was awarded 25 points because
the Primary Service is identified as a Group 4 Service. The specific Sub-Component Service
Type identified is Group Counseling. The Primary Service was identified as this type of service as
it focuses on psychological or interpersonal problems or issues faced by an individual and
involves a group of youths interacting with each other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program has nine facilitators trained in the VOICES curriculum. Three facilitators actually
conducted group in the last twelve-month period. The last group ended on 11/17/16. A review of
all three facilitator’s training files contained documentation that all received formal training in the
VOICES curriculum from a qualified trainer.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
All youth admitted at Lake Academy are assessed by staff as needing evidence based services.
Based on various criteria, they are placed into program services. The program is utilizing the
VOICES curriculum “A Program of Self-Discovery and Empowerment for Girls”. The manual
details the primary service and explains how each specific session is to be delivered. It includes a
script, gives outlines for each of the 18 group sessions, and provides specific details on the
delivery of those sessions.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
This curriculum was not able to be observed; therefore, this indicator rates as non-applicable.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The reviewed documentation and the Clinical Director confirmed there was staff turnover in the
last twelve months, however, that turnover did not adversely affect the youth due to proper
transition of facilitators in between groups. There were no gaps in service. The program has a
policy for facilitator turnover. If a facilitator is going to be leaving the program or will be on leave,
they explain to the youth why another facilitator will be taking over the class. Another qualified
facilitator from the trained staff would then conduct that class.
BasicScore: 2Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Internal fidelity monitoring reports were reviewed. The program has an internal process to conduct
internal fidelity monitoring specific to the VOICES curriculum. The LMHC performs all monitoring.
He is also trained in the VOICES curriculum and is the only staff member who performs
monitoring for this service. The fidelity monitor has received certification in the primary service
and fidelity monitoring. The program utilizes fidelity adherence checklists to document the
monitoring. Fidelity monitoring has been conducted monthly on the staff members who facilitated
the group within the last twelve months.
BasicScore: 1Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the Clinical Director verified that the program has a process for corrective
actions based on fidelity monitoring. The program uses internal fidelity monitoring/correction
action checklists specific to VOICES. Each monitoring checklist includes a section for feedback
and recommendations for facilitator improvement. There were no corrective actions pertaining to
VOICES in the last 12 months.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Performance evaluations were completed on two of the three trained staff members who have
facilitated the VOICES intervention in the past twelve months. The evaluations were reviewed and
were found to include entries specific to the staff member facilitating the VOICES curriculum. The
third facilitator was terminated from employment at Lake Academy on 11/17/16. An evaluation
was started and does make mention of her facilitating the VOICES curriculum, however, the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
evaluation for this staff member was never finalized by the Administration due to the employee’s
termination.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 24 weeks for this type of service. Of the 30 youth in the
sample, 0% (0 of 30) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 40 contact hours for this type of service. Of the 30 youth in the
sample, 3% (1 of 30) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 0 youth
Moderate-High = 10 youth
High = 20 youth
Total Youth in Sample = 30 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 67%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 20 youth
Total Youth in Sample = 30 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (30 of 30) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 67% (20 of 30) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 25 25 100%
Totals 75 95 79%
This SPEP report evaluates VOICES, an intervention delivered at Lake Academy.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 30 total youth sampled, all
included dosage with end dates in the EBS Module. Of those youth with correct dosage, none
received at least the recommended weeks of service. Youth in the sample completed between 2
and 16 weeks of service, with an average of 9 weeks.
The program earned 0 points for Amount of Service: Contact Hours. Of the 30 total youth
sampled, all included dosage in the EBS Module. Of those youth with correct dosage, only 1
received at least the recommended hours of service. Youth in the sample completed between 7.5
and 48 hours of service, with an average of 26 hours.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
Lake Academy, a G4S-operated facility, acknowledged the SPEP dosage targets, but respectfully
declined to adjust the VOICES curriculum specifications to accommodate increased SPEP
targeted dosage levels over those dosage levels specified in the VOICES curriculum.
RECOMMENDATION(S):
Lake Academy can maintain their SPEP Quality of Service Delivery score by continuing the
processes in place at the time of this review.
Lake Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

source: DJJ

report_type: SPEP

program_name_raw: Lake Academy

program_status_note: Closed

service_type: VOICES: A Program of Self-Exploration and Empowerment


summary: SPEP - FY17-18

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Lake Academy
TrueCore Behavioral Solutions, LLC
(Contract Provider)
9504 East Columbus Drive
Tampa, Florida 33619
Primary Service: Impact of Crime
SPEP Review Date(s): February 27‐March 2, 2018
Florida Department of Juvenile Justice
Report Date(s): March 14, 2018
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Lake Academy MQI Program Code: 1069
Provider Name: TrueCore Behavioral Solutions Contract Number: R2104
Location: Hillsborough County / Circuit: 13 Number of Beds: 50
Review Date(s): February 27-March 2, 2018 Lead Reviewer Code: 97
Persons Interviewed
Program Director Corporate QI/QA staff 2 # Program Supervisors
DJJ Monitor 2 # Case Managers 7 # Youth
DHA or designee 3 # Clinical Staff # Other (listed by title):
DMHCA or designee 3 # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 14 # Personnel Records
Fidelity Monitoring Documents Program Schedules 14 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 3 # Youth Records (Closed)
Staff Evaluations Table of Organization 7 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports 7 # Health Records
Contract Scope of Services 7 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime. The program was awarded 15 points
because the Primary Service is identified as a Group 3 Service. The specific Sub-Component
Service Type identified is Social Skills Training. The Primary Service was identified as this type of
service as it focuses on developing social skills required for an individual to interact in a positive
way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program currently has six staff trained to provide the Impact of Crime (IOC) curriculum. The
program maintains all information regarding IOC in a binder. There was documentation in the
binder to support that each staff received facilitator training. All of the training was provided by a
qualified trainer. The IOC groups were facilitated by the six staff for the past twelve months.
When possible, the provider will make all efforts to have a co-facilitator for the primary service in
order to enhance the continuity of the service due to unexpected absence and provide more
individualized instruction as needed during the service. The co-facilitator will have completed
formalized training for the primary service from a qualified trainer.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the Impact of Crime (IOC) curriculum, which includes facilitator and student
guides. The curriculum is manualized and provides the frequency and duration for the service,
with specific topics to be covered in each lesson. The facilitator guide contains lesson plans with
instructions for conducting each group session.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
During the annual compliance review, a session of Impact of Crime was observed. A copy of the
lesson plan was provided to the observer. The group was facilitated by a youth care worker, who
has been trained to provide the intervention. The facilitator opened the group with a check-in,
which was facilitated by a youth. The values for the group were reviewed by the youth. The
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
facilitator then moved on to IOC lesson 3, which included a scenario involving a crime. The youth
discussed primary and secondary victims in the scenario which had been presented. The youth
also discussed what the impact of the crime was in the scenario. The youth were able to role play,
as well, and discussed their homework. The lesson followed the manual. The facilitator was able
to get all of the youth involved in the discussion.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There has been no turnover of facilitators, nor were there any gaps in service during this review
period. In the event of staff turnover, there are several staff employed by the program who could
facilitate groups.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has documented fidelity monitoring for observations of the Impact of Crime
curriculum, using a monitoring tool for manualized curriculum. The fidelity monitoring was
completed by the program’s clinical director and a DJJ regional monitor, both of whom have been
trained in the curriculum. The form included a list of the chapters and objectives completed during
the review, and three sections, such as the mechanics of the lesson, experiential content and
overall impressions. The fidelity monitoring was completed each month the primary service was
provided for the past twelve months.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program provides fidelity monitoring; the fidelity monitoring sheets completed on the Impact
of Crime groups for the past year were reviewed. There was a suggestion to have the group's
rules posted to help the youth remember rules. The group rules were noted to be posted in the
room the group was held. There was no other documentation requiring any further corrective
action.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Six staff are trained to provide the Impact of Crime curricula. There was an annual performance
evaluation completed on each staff. The evaluation forms did not contain specific information
regarding the staff’s delivery of the primary service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
3. Amount of Service ‐ Duration
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates the target duration of 16 weeks for this type of service. Of the 16 youth in the
sample, 44% (7 of 16) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 24 contact hours for this type of service. Of the 16 youth in the
sample, 94% (15 of 16) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 94%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 0 youth
Moderate-High = 0 youth
High = 15 youth
Total Youth in Sample = 16 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 94%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 15 youth
Total Youth in Sample = 16 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 94% (15 of 16) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 94% (15 of 16) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 4 10 40%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 23 25 92%
Totals 75 85 88%
This SPEP report evaluates Impact of Crime, an intervention delivered at Lake Academy.
The program scored High for Quality of Service Delivery.
The program earned 4 points for Amount of Service: Duration. Of the 16 total youth sampled, all
16 included dosage with end dates in the EBS Module. Of those youth, 7 received at least the
recommended weeks of service. Youth in the sample completed between 8 and 19 weeks of
service, with an average of 13 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 16 total youth
sampled, all 16 included dosage in the EBS Module. Of those youth, 15 received at least the
recommended hours of service. Youth in the sample completed between 16 and 28 hours of
service, with an average of 25 hours.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered just prior to the youths' admission.
RECOMMENDATION(S):
Lake Academy can optimize their SPEP Quality of Service Delivery score by continuing the
practices in place at the time of this review.
Lake Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

service_type: Impact of Crime


summary: SPEP - FY17-18

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Lake Academy
TrueCore Behavioral Solutions
(Contract Provider)
9504 East Columbus Drive
Tampa, Florida 33619
Primary Service: Individual Counseling
SPEP Review Date(s): February 27‐March 2, 2018
Florida Department of Juvenile Justice
Report Date(s): March 14, 2018
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Lake Academy MQI Program Code: 1069
Provider Name: TrueCore Behavioral Solutions Contract Number: R2104
Location: Hillsborough County / Circuit: 13 Number of Beds: 50
Review Date(s): February 27-March 2, 2018 Lead Reviewer Code: 97
Persons Interviewed
Program Director Corporate QI/QA staff 2 # Program Supervisors
DJJ Monitor 2 # Case Managers 7 # Youth
DHA or designee 3 # Clinical Staff # Other (listed by title):
DMHCA or designee 3 # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 14 # Personnel Records
Fidelity Monitoring Documents Program Schedules 14 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 3 # Youth Records (Closed)
Staff Evaluations Table of Organization 7 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports 7 # Health Records
Contract Scope of Services 7 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
1. Primary Service and Supplemental Service Types POS: 10 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Individual Counseling. The program was awarded 5
points because the Primary Service is identified as a Group 1 Service. The specific Sub-
Component Service Type identified is Individual Counseling. The Primary Service was identified
as this type of service as it includes any range of treatment techniques that focus on
psychological or interpersonal problems or issues faced by an individual that involves a one-on-
one relationship with a therapist or counselor.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There were six clinicians to provide individual counseling to youth in the program. The program
maintains all information regarding individual counseling in the youth's mental health/substance
abuse record. There was documentation to support that each therapist had the requisite
education and experience to provide counseling sessions. Individual counseling was facilitated by
the six clinicians for the past twelve months. For this primary service, which does not have a
formalized curriculum, the facilitator receives training from a licensed clinician. The facilitator
receives additional training as needed which is documented during their clinical supervision.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
There is no written manual for individual counseling. The program has a written protocol for
individual counseling. The frequency, duration and intervention in which the therapist will conduct
individual counseling are stated on the youth's individualized treatment plan. This plan is
reviewed and signed by a licensed clinician.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
This was not observed during the annual compliance review, due to the nature of individual
counseling; therefore, this indicator rates non-applicable.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There has been no turnover of facilitators, nor were there any gaps in service during this review
period. In the event of staff turnover, there are several therapists employed by the program who
could facilitate individual counseling.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has documented fidelity monitoring for observations of the individual counseling.
The fidelity monitoring was completed by the program’s clinical director. The fidelity monitoring is
completed through a review of therapy notes to verify provision of effective and appropriate
interventions, which are in line with the youth's treatment plan. The clinical director reviews
trackers to ensure compliance with frequency and duration. There was documentation of the
clinical director conducting this review. The fidelity monitoring was completed each month and the
primary service was provided for the past twelve months.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program provides fidelity monitoring. There was no documentation on any reviewed
monitoring sheets to suggest any corrective action was required.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Six therapists provide individual counseling to the youth in the program. There was an annual
performance evaluation completed on each staff. The evaluation forms did not contain specific
information regarding the staff’s delivery of the primary service.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service ‐ Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 25 weeks for this type of service. Of the 35 youth in the
sample, 89% (31 of 35) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 30 contact hours for this type of service. Of the 35 youth in the
sample, 74% (26 of 35) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 97%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 2 youth
Moderate-High = 2 youth
High = 30 youth
Total Youth in Sample = 35 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 86%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 30 youth
Total Youth in Sample = 35 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 97% (34 of 35) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 86% (30 of 35) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 10 10 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 25 25 100%
Totals 69 75 92%
This SPEP report evaluates Individual Counseling, an intervention delivered at Lake Academy.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 35 total youth sampled, all
35 included dosage with end dates in the EBS Module. Of those youth, 31 received at least the
recommended weeks of service. Youth in the sample completed between 19 and 64 weeks of
service, with an average of 38 weeks.
The program earned 6 points for Amount of Service: Contact Hours. Of the 35 total youth
sampled, all 35 included dosage in the EBS Module. Of those youth, 26 received at least the
recommended hours of service. Youth in the sample completed between 12 and 66 hours of
service, with an average of 36 hours.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered just prior to the youths' admission.
RECOMMENDATION(S):
Lake Academy can maintain their SPEP Quality of Service Delivery score by continuing the
practices in place at the time of this review.
Lake Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

service_type: Individual Counseling


summary: SPEP - FY17-18

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Lake Academy
TrueCore Behavioral Solutions
(Contract Provider)
9504 East Columbus Drive
Tampa, Florida 33619
Primary Service: Thinking for a Change
SPEP Review Date(s): February 27‐March 2, 2018
Florida Department of Juvenile Justice
Report Date(s): March 14, 2018
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Lake Academy MQI Program Code: 1069
Provider Name: TrueCore Behavioral Solutions Contract Number: R2104
Location: Hillsborough County / Circuit: 13 Number of Beds: 50
Review Date(s): February 27-March 2, 2018 Lead Reviewer Code: 97
Persons Interviewed
Program Director Corporate QI/QA staff 2 # Program Supervisors
DJJ Monitor 2 # Case Managers 7 # Youth
DHA or designee 3 # Clinical Staff # Other (listed by title):
DMHCA or designee 3 # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 14 # Personnel Records
Fidelity Monitoring Documents Program Schedules 14 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 3 # Youth Records (Closed)
Staff Evaluations Table of Organization 7 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports 7 # Health Records
Contract Scope of Services 7 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change. The program was awarded 30
points because the Primary Service is identified as a Group 5 Service. The specific Sub-
Component Service Type identified is Cognitive Behavioral Therapy. The Primary Service was
identified as this type of service as it is intended to correct faulty cognitions and perceptions and
provides skills individuals can use to monitor thought patterns and correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program currently has three staff trained to provide the Thinking for a Change (T4C)
curriculum. The program maintains all information regarding T4C in a binder. There was
documentation in the binder to support that each of the three staff received facilitator training. The
facilitators completed a formalized training for the primary service from a qualified trainer. The
T4C groups were facilitated by the three staff for the past twelve months. When possible, the
provider will make all efforts to have a co-facilitator for the primary service to enhance the
continuity of the service due to unexpected absence and provide more individualized instruction
as needed during the service. The co-facilitator will have completed formalized training for the
primary service from a qualified trainer.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the Thinking for a Change (T4C) curriculum, which includes facilitator and
student guides. The curriculum is manualized and provides the frequency and duration for the
service, with specific topics to be covered in each lesson. The facilitator guide contains lesson
plans with instructions for conducting each group session.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
During the annual compliance review, a session of Thinking for a Change was observed. A copy
of the lesson plan was provided to the observer. The group was facilitated by the program's
restorative justice counselor, who has been trained to provide the intervention. The facilitator
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
opened the group with a check-in of each youth. There was a sheet of paper depicting many
types of feelings, including aggressive, blissful, grieving, interested, and sad. Each youth selected
one of the emotions as their check-in. The facilitator then used an exercise in which an emotion
was written on a small piece of paper and placed in a container. Each youth selected a piece of
paper and either discussed or acted out the emotion on the paper. Some of the youth initially
opted out of the exercise; however, the facilitator was patient with the youth, and all eventually
participated. The facilitator then moved on to the T4C lesson, which was lesson 5, ‘Knowing your
Feelings'. The youth discussed their homework and the social skill. The lesson followed the
manual. The youth worked in groups of two to act out scenarios based on their feelings. The
facilitator was able to get all of the youth involved in the discussion.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There has been no turnover of facilitators, nor were there any gaps in service during this review
period. In the event of staff turnover, there are several staff employed by the program who could
facilitate groups.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has documented fidelity monitoring for observations of the Thinking for a Change
curriculum, using a monitoring tool for the manualized curriculum. The fidelity monitoring was
completed by the program’s clinical director and the Department of Juvenile Justice Technical
Assistance Specialist, both of whom have been trained in the curriculum. The form included an
integrity checklist, which included a check of the facilitator's ability to keep the youth engaged, use
of non-judgmental behavior and statements, reinforcement of group rules and preparation for the
group. The fidelity monitoring was completed each month the primary service was provided for the
past twelve months.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program provides fidelity monitoring. Fidelity monitoring sheets that were completed on the
Thinking for a Change groups for the past year were reviewed. There was a suggestion to have
the groups rules posted to help the youth remember rules. The group rules were noted to be
posted in the room the group was held. There was no other documentation requiring any further
corrective action.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Three staff are trained to provide the Thinking for a Change curricula. There was an annual
performance evaluation completed on each staff. The evaluation forms did not contain specific
information regarding the staff’s delivery of the primary service.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
3. Amount of Service ‐ Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 15 weeks for this type of service. Of the 18 youth in the
sample, 83% (15 of 18) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 45 contact hours for this type of service. Of the 18 youth in the
sample, 61% (11 of 18) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 2 youth
Moderate-High = 2 youth
High = 14 youth
Total Youth in Sample = 18 youth
Percentage of Youth with High-Risk Level to Reoffend: 78%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 14 youth
Total Youth in Sample = 18 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (18 of 18) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 78% (14 of 18) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 25 25 100%
Totals 94 100 94%
This SPEP report evaluates Thinking for a Change, an intervention delivered at Lake Academy.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 18 total youth sampled, all
18 included dosage with end dates in the EBS Module. Of those youth, 15 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
recommended weeks of service. Youth in the sample completed between 11 and 20 weeks of
service, with an average of 18 weeks.
The program earned 25 points for Amount of Service: Contact Hours. Of the 18 total youth
sampled, all 18 included dosage in the EBS Module. Of those youth, 11 received at least the
recommended hours of service. Youth in the sample completed between 27 and 51 hours of
service, with an average of 40 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered just prior to the youths' admission.
RECOMMENDATION(S):
Lake Academy can maintain their SPEP Quality of Service Delivery score by continuing the
practices in place at the time of this review.
Lake Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

service_type: Thinking for a Change


summary: SPEP - FY17-18

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Lake Academy
TrueCore Behavioral Solutions
(Contract Provider)
9504 East Columbus Drive
Tampa, Florida 33619
Primary Service: VOICES
SPEP Review Date(s): February 27‐March 2, 2018
Florida Department of Juvenile Justice
Report Date(s): March 14, 2018
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Lake Academy MQI Program Code: 1069
Provider Name: TrueCore Behavioral Solutions Contract Number: R2104
Location: Hillsborough County / Circuit: 13 Number of Beds: 50
Review Date(s): February 27-March 2, 2018 Lead Reviewer Code: 97
Persons Interviewed
Program Director Corporate QI/QA staff 2 # Program Supervisors
DJJ Monitor 2 # Case Managers 7 # Youth
DHA or designee 3 # Clinical Staff # Other (listed by title):
DMHCA or designee 3 # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 14 # Personnel Records
Fidelity Monitoring Documents Program Schedules 14 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 3 # Youth Records (Closed)
Staff Evaluations Table of Organization 7 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports 7 # Health Records
Contract Scope of Services 7 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is VOICES. The program was awarded 25 points because
the Primary Service is identified as a Group 4 Service. The specific Sub-Component Service
Type identified is Group Counseling. The Primary Service was identified as this type of service as
it focuses on psychological or interpersonal problems or issues faced by an individual and
involves a group of youths interacting with each other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There were eight staff trained to provide the VOICES curriculum. The program maintains all
information regarding VOICES in a binder. There was documentation in the binder to support that
each staff received facilitator training. Each facilitator completed a formalized training for the
primary service from a qualified trainer. The VOICES groups were facilitated by eight staff for the
past twelve months. When possible, the provider will make all efforts to have a co-facilitator for
the primary service in order to enhance the continuity of the service due to unexpected absence
and to provide a more individualized instruction as needed during the service. The co-facilitator
will have completed formalized training for the primary service from a qualified trainer.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the VOICES curriculum, which includes facilitator and student guides. The
curriculum is manualized and provides the frequency and duration for the service, with specific
topics to be covered in each lesson. The facilitator guide contains lesson plans with instructions
for conducting each group session.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
During the annual compliance review, a session of VOICES was observed. A copy of the lesson
plan was provided to the observer. The group was facilitated by a therapist who has been trained
to provide the intervention. The facilitator opened the group with a check-in of the youth. The
values for the group were reviewed by the youth. The facilitator then moved on to the VOICES
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
lesson, which was session 16, 'Spirituality'. This lesson had the youth discuss the differences
between religion and spirituality. The youth worked on activity 2, which was entitled 'Finding the
Wise Woman'. The lesson followed the manual. The facilitator was able to get all of the youth
involved in the discussion.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There has been no turnover of facilitators, nor were there any gaps in service during this review
period. In the event of staff turnover, there are several staff employed by the program who could
facilitate groups.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has documented fidelity monitoring for observations of the VOICES curriculum,
using a monitoring tool for the curriculum. The fidelity monitoring was completed by the program’s
clinical director, who has been trained in the curriculum. The form, which was specific to the
lesson observed, included scores on the session content, which rated how the activity was
presented, and whether the session closing included reflective questions. There were ratings on
the facilitator's skill, which included being positive and non-judgmental, ensured rules were
maintained and group members were allowed time to speak and ask questions. Finally, there
were scores for fundamental items, which included whether the structure was adhered to, all of
the required activities were conducted and whether the space was comfortable. The rated items
were tallied for each section. The fidelity monitoring was completed each month the primary
service was provided for the past twelve months.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program provides fidelity monitoring. The fidelity monitoring sheets completed on the
VOICES groups for the past year were reviewed. There was no documentation on any reviewed
monitoring sheets to suggest any corrective action was required.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Eight staff are trained to provide the VOICES curricula. There was an annual performance
evaluation completed on each staff. The evaluation forms did not contain specific information
regarding the staff’s delivery of the primary service.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 24 weeks for this type of service. Of the 35 youth in the
sample, 0% (0 of 35) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 40 contact hours for this type of service. Of the 35 youth in the
sample, 0% (0 of 35) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 97%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Table 1
Moderate = 2 youth
Moderate-High = 2 youth
High = 30 youth
Total Youth in Sample = 35 youth
Percentage of Youth with High-Risk Level to Reoffend: 86%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 30 youth
Total Youth in Sample = 35 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 97% (34 of 35) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 86% (30 of 35) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 25 25 100%
Totals 75 95 79%
This SPEP report evaluates VOICES, an intervention delivered at Lake Academy.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 35 total youth sampled, all
35 included dosage with end dates in the EBS Module. Of those youth, 0 received at least the
recommended weeks of service. Youth in the sample completed between 3 and 12 weeks of
service, with an average of 8 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 0 points for Amount of Service: Contact Hours. Of the 35 total youth
sampled, all 35 included dosage in the EBS Module. Of those youth, 0 received at least the
recommended hours of service. Youth in the sample completed between 9 and 29 hours of
service, with an average of 26 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered just prior to the youths' admission.
RECOMMENDATION(S):
Lake Academy can maintain their SPEP Quality of Service Delivery score by continuing the
practices in place at the time of this review.
Lake Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Lake Academy, a facility operated by TrueCore Behavioral Solutions, acknowledged the SPEP
dosage targets, but respectfully declined to adjust the VOICES curriculum specifications to
accommodate increased SPEP targeted dosage levels over those dosage levels specified in the
VOICES curriculum. .
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9