state: FL

summary: SPEP - FY15-16

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Lake Academy
G4S Youth Services, LLC
(Contract Provider)
9504 East Columbus Drive
Tampa, Florida 33619
Primary Service: Individual Counseling
SPEP Review Date(s): January 5‐8, 2016
Florida Department of Juvenile Justice
Report Date(s): 8/31/2016
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Lake Academy MQI Program Code: 1069
Provider Name: G4S Youth Services, LLC Contract Number: R2104
Location: Hillsborough County / Circuit: 13 Number of Beds: 48
Review Date(s): January 5-8, 2016 Lead Reviewer Code: 140
Persons Interviewed
Program Director Corporate QI/QA staff 2 # Program Supervisors
DJJ Monitor 1 # Case Managers 5 # Youth
DHA or designee 1 # Clinical Staff # Other (listed by title):
DMHCA or designee 1 # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 8 # Personnel Records
Fidelity Monitoring Documents Program Schedules 7 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 5 # Youth Records (Closed)
Staff Evaluations Table of Organization 7 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports 10 # Health Records
Contract Scope of Services 12 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
1. Primary Service and Supplemental Service Types POS: 10 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Individual Counseling. The program was awarded 5
points because the Primary Service is identified as a Group 1 Service. The specific Sub-
Component Service Type identified is Individual Counseling. The Primary Service was identified
as this type of service as it includes any range of treatment techniques that focus on
psychological or interpersonal problems or issues faced by an individual that involves a one-on-
one relationship with a therapist or counselor.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes their therapists to provide individual counseling to the youth in their program.
The program has a clinical director, who is a licensed mental health counselor and is responsible
for the coordination and implementation of individual counseling. The program also has a lead
therapist, who is a licensed clinical social worker and provides individual counseling to youth. The
program also utilizes three master level therapists who have received on-the job training specific
to therapeutic services and provides individual therapy to the youth in the program. All training
was documented in the Department's Learning Management System (SkillPro) or in the
therapists’ individual training files.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program has a written comprehensive plan for mental health and substance abuse services
which defines the protocol for individual counseling. Each youth is provided with individual therapy
at a minimum of one session per week for at least fifty minutes. A review of seven mental health
records revealed documentation of weekly individual progress notes for each of the reviewed
youth.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Individual counseling sessions are not observed as part of the evaluation process for SPEP
services. Interviews with the clinical director, therapists and youth validated individual counseling
is being conducted on a weekly basis. Furthermore, a review of seven mental health records
revealed documentation of weekly individual sessions for each of the reviewed youth.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
During the last twelve months the program hired two new master level therapists to provide
individual counseling to the youth. Documentation supported there were no gaps in the primary
service when the turnover occurred. Progress notes supported other therapists at the program
were able to provide the primary service to the youth while the turnover was occurring.
The program can earn 2 points by minimizing facilitator turnover.
At the time of the review, the program did not meet criteria to earn a score of 2.
BasicScore: 2Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Individual counseling fidelity monitoring occurs by the clinical directors, or lead therapist reviewing
each youth’s weekly mental health progress notes and signing them. Furthermore, fidelity
monitoring is completed during the weekly clinical supervision meetings conducted between the
Clinical Director and all therapists. A review of the Clinical Directors documentation of weekly
supervision meetings revealed therapists are conducting individual counseling with youth at the
program.
BasicScore: 1Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process by which corrective action is applied and demonstrated, which is
outlined in the program comprehensive plan for mental health and substance abuse services.
There has not been any corrective action necessary during the last twelve months.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Performance evaluations were completed on all five therapists during the last twelve months. The
evaluations included a section for job specific criteria in relation to clinical services. The
evaluations addressed the therapists’ ability to conduct screenings, assessments, evaluations,
counseling sessions, and their ability to complete a mental health/substance abuse discharge
plan.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 25 weeks for this type of service. Of the 7 youth in the
sample, 100% (7 of 7) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 30 contact hours for this type of service. Of the 7 youth in the
sample, 71% (5 of 7) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 13 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 52%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 71%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 0 youth
Moderate-High = 3 youth
High = 2 youth
Total Youth in Sample = 7 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 29%
High Score: 8 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 62%
Table 2
High = 2 youth
Total Youth in Sample = 7 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 71% (5 of 7) youth scored Moderate to High-Risk to reoffend, for a score of
5 points.
Of the SPEP sample, 29% (2 of 7) youth scored High-Risk to reoffend, for a score of 8 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 10 10 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 13 25 52%
Totals 59 75 79%
This SPEP report evaluates Individual Counseling, an intervention delivered at Lake Academy.
The program scored High for Quality of Service Delivery. This score can be optimized by
minimizing facilitator turnover.
The program earned 10 points for Amount of Service: Duration. Of the 7 total youth sampled, all
received at least the recommended weeks of service. Youth in the sample completed between 25
and 46 weeks of service, with an average of 38 weeks.
The program earned 6 points for Amount of Service: Contact Hours. Of the 7 total youth sampled,
5 received at least the recommended hours of service. Youth in the sample completed between
25 and 48 hours of service, with an average of 38 hours.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program was awarded 13 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Lake Academy can optimize their SPEP Quality of Service Delivery score by minimizing facilitator
turnover.
Lake Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

source: DJJ

report_type: SPEP

program_name_raw: Lake Academy

program_status_note: Closed

service_type: Individual Counseling


facility_name: Lake Academy

summary: SPEP - FY15-16

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Lake Academy
G4S Youth Services, LLC
(Contract Provider)
9504 East Columbus Drive
Tampa, Florida 33619
Primary Service: Impact of Crime
SPEP Review Date(s): January 5‐8, 2016
Florida Department of Juvenile Justice
Report Date(s): 8/31/2016
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Lake Academy MQI Program Code: 1069
Provider Name: G4S Youth Services, LLC Contract Number: R2104
Location: Hillsborough County / Circuit: 13 Number of Beds: 48
Review Date(s): January 5-8, 2016 Lead Reviewer Code: 140
Persons Interviewed
Program Director Corporate QI/QA staff 2 # Program Supervisors
DJJ Monitor 1 # Case Managers 5 # Youth
DHA or designee 1 # Clinical Staff # Other (listed by title):
DMHCA or designee 1 # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 8 # Personnel Records
Fidelity Monitoring Documents Program Schedules 7 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 5 # Youth Records (Closed)
Staff Evaluations Table of Organization 7 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports 10 # Health Records
Contract Scope of Services 12 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime. The program was awarded 15 points
because the Primary Service is identified as a Group 3 Service. The specific Sub-Component
Service Type identified is Social Skills Training. The Primary Service was identified as this type of
service as it focuses on developing social skills required for an individual to interact in a positive
way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program has five facilitators trained in the Impact of Crime (IOC) curriculum. Four of the five
facilitators conducted groups during the review period. All five training files contained
documentation of certification in the IOC curriculum. The training received by all facilitators was
conducted by a qualified trainer.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
IThe program is using Impact of Crime (IOC) curriculum for all youth admitted to the program who
are identified as in need of the service via the programs risk and needs assessment completed
during a youth’s admission process. The program maintains a detailed manual which explains
how each specific session is delivered and there is an outline for each session and section.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The facilitator provided a copy of the lesson plan from the program manual, which was to be
covered in the class being observed. The lesson plan being covered was Chapter 5 – Ripple
Effect of Crime. As observed the facilitator does not read the lesson verbatim but uses her own
words to convey the meaning of the topic under discussion and does so in an easy to follow
manner. The facilitator engaged the youth to participate in the lesson and the youth actively
participated in the group by giving examples of dating violence. The youth participated in three
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
separate activities. The facilitator assigned homework for the youth to complete for the next class.
The facilitator adhered to the lesson plan in every aspect but did not complete the lesson by the
end of the session.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
During the annual compliance review documentation and interviews with the Facility Administrator
confirmed there was no turnover adversely affecting the youth during the last year. There were no
gaps in service delivery; however, there was a gap of approximately a month and a half between
the start and end dates of each group to allow for enough youth to enter the facility and be ready
to start a new closed group.
BasicScore: 1Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The director of case management performed all fidelity monitoring for Impact of Crime (IOC)
during the last review period and she has been certified to teach the primary service. Internal
fidelity adherence checklists are utilized monthly by the fidelity monitor to capture specifics of the
primary service. During the annual compliance review, there was a review of twelve months of
fidelity monitoring checklists. There was no fidelity monitoring conducted during the months of
May or August for the primary service or the facilitators.
The program can earn 2 points by ensuring that internal fidelity monitoring is conducted at least
monthly for all facilitators.
At the time of the review, the program did not meet criteria to earn a score of 2.
BasicScore: 1Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process by which corrective action is applied and demonstrated based on the
fidelity monitoring of Impact of Crime (IOC). Each fidelity monitoring checklist includes a section
for feedback and recommendations for facilitator improvement. Each reviewed checklist included
feedback which indicated the facilitator did not require any corrective action or recommendations
for future improvements.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1Point
Intervention Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Performance evaluations were completed on all four facilitators delivering Impact of Crime
interventions. The evaluations addressed the skills and quality of the facilitators’ ability to teach
the primary service. Each evaluation had comments from the facilitators’ supervisor in regards to
their ability to provide the services to youth.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates the target duration of 16 weeks for this type of service. Of the 2 youth in the
sample, 50% (1 of 2) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates a target of 24 contact hours for this type of service. Of the 2 youth in the
sample, 50% (1 of 2) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 5 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 20%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 50%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 0 youth
Moderate-High = 1 youth
High = 0 youth
Total Youth in Sample = 2 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 0%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 0 youth
Total Youth in Sample = 2 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 50% (1 of 2) youth scored Moderate to High-Risk to reoffend, for a score of
5 points.
Of the SPEP sample, 0% (0 of 2) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 4 10 40%
Amount of Service: Contact Hours 4 10 40%
Risk Level of Youth Served 5 25 20%
Totals 53 85 62%
This SPEP report evaluates Impact of Crime (IOC), an intervention delivered at Lake Academy.
The program scored High for Quality of Service Delivery. This score can be optimized by
ensuring that internal fidelity monitoring is conducted at least monthly for all facilitators.
The program earned 4 points for Amount of Service: Duration. Of the 2 total youth sampled, only
1 included dosage with end dates in the EBS Module. That youth completed 16 weeks of service.
The program earned 4 points for Amount of Service: Contact Hours. Of the 2 total youth sampled,
only one reached the targeted dosage. Youth earned 23 and 24 weeks of service.
The program was awarded 5 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Lake Academy can optimize their SPEP Quality of Service Delivery score by ensuring that internal
fidelity monitoring is conducted at least monthly for all facilitators.
Lake Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

service_type: Impact of Crime


summary: SPEP - FY15-16

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Lake Academy
G4S Youth Services, LLC
(Contract Provider)
9504 East Columbus Drive
Tampa, Florida 33619
Primary Service: Thinking For a Change
SPEP Review Date(s): January 5‐8, 2016
Florida Department of Juvenile Justice
Report Date(s): 8/31/16
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Lake Academy MQI Program Code: 1069
Provider Name: G4S Youth Services, LLC Contract Number: R2104
Location: Hillsborough County / Circuit: 13 Number of Beds: 48
Review Date(s): January 5-8, 2016 Lead Reviewer Code: 140
Persons Interviewed
Program Director Corporate QI/QA staff 2 # Program Supervisors
DJJ Monitor 1 # Case Managers 5 # Youth
DHA or designee 1 # Clinical Staff # Other (listed by title):
DMHCA or designee 1 # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 8 # Personnel Records
Fidelity Monitoring Documents Program Schedules 7 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 5 # Youth Records (Closed)
Staff Evaluations Table of Organization 7 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports 10 # Health Records
Contract Scope of Services 12 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change (T4C). The program was awarded
30 points because the Primary Service is identified as a Group 5 Service. The specific Sub-
Component Service Type identified is Cognitive Behavioral Therapy. The Primary Service was
identified as this type of service as it is intended to correct faulty cognitions and perceptions and
provides skills individuals can use to monitor thought patterns and correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program has six facilitators trained in the Thinking for a Change (T4C) curriculum. Five of the
six facilitators conducted groups during the last twelve months. All six training files contained
documentation of certification in the T4C curriculum. The training received by all facilitators was
conducted by a qualified trainer.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program is using Thinking for a Change (T4C) curriculum for all youth admitted to the
program who are identified by the risk and needs assessment to be in need of the curriculum. The
program utilizes the National Institute of Corrections Thinking for a Change Manual. The manual
is very detailed and explains how each specific session is delivered and there is an outline for
each section.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The facilitator provided the reviewer with a copy of the lesson plan from the programs manual
prior to the start of the class. The lesson being covered was Lesson 13- Social Skill: Apologizing.
Observation of the facilitator verified the lesson is not read verbatim but is in her own words to
convey the meaning of the topic under discussion and does so in an easy to follow manner. The
facilitator engaged the youth to participate in the lesson and all youth actively participated in the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
group, which included role play. The facilitator adhered to the lesson plan in every aspect but did
not complete the lesson by the end of the session.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The reviewed documentation confirmed there was no turnover which adversely affect the youth
during the past twelve months. There have been no gaps in service delivery; however, there was
a gap of approximately a month and a half between the March 2015 class and the May 2015
class, and between the September class and November class. The program advised the gap was
due to the program waiting for new youth to enter the facility and be ready to start a new closed
group.
BasicScore: 1Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
There was twelve months of internal fidelity monitoring reports reviewed for compliance with
fidelity monitoring. The Director of Case Management was the only staff member who performed
fidelity monitoring for the Thinking for a Change curriculum. The fidelity monitor has been certified
to teach the primary service. Internal fidelity adherence checklists are utilized monthly by the
fidelity monitor to capture specifics of the primary service. There was no documentation fidelity
monitoring was conducted during the months of May or June for the primary service or facilitators.
The program can earn 2 points if fidelity monitoring is conducted at least monthly for all
facilitators.
At the time of the review, the program did not meet criteria to earn a score of 2.
BasicScore: 1Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process by which corrective action is applied and demonstrated based on the
fidelity monitoring of the primary service. The program utilizes internal fidelity monitoring
corrective action checklists to specify if a facilitator is in need of corrective action or improvement.
A review of the completed checklists for the last twelve months indicated there was no corrective
action or improvement needed for any of the facilitators of T4C.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1Point
Intervention Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Performance evaluations were completed on four of the five facilitators delivering Thinking for a
Change interventions. Each performance evaluation contained an entry for the evaluation of the
facilitators’ abilities and skills to deliver the primary service. The reason one facilitator did not have
an evaluation was because she is currently in the first ninety days of hire and the program does
not complete evaluations until an employee reaches ninety days and then annually.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates the target duration of 15 weeks for this type of service. Of the 5 youth in the
sample, 20% (1 of 5) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 45 contact hours for this type of service. Of the 5 youth in the
sample, 0% (0 of 5) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 80%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 0 youth
Moderate-High = 2 youth
High = 2 youth
Total Youth in Sample = 5 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 40%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 2 youth
Total Youth in Sample = 5 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 80% (4 of 5) youth scored Moderate to High-Risk to reoffend, for a score of
7 points.
Of the SPEP sample, 40% (2 of 5) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 2 10 20%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 20 25 80%
Totals 77 100 77%
This SPEP report evaluates Thinking for a Change (T4C), an intervention delivered at Lake
Academy.
The program scored High for Quality of Service Delivery. This score can be optimized by ensuring
that fidelity monitoring is conducted at least monthly for all facilitators.
The program earned 2 points for Amount of Service: Duration. Of the 5 total youth sampled, only
4 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 1
received at least the recommended weeks of service. Youth in the sample completed between 10
and 16 weeks of service, with an average of 13 weeks.
The program earned 0 points for Amount of Service: Contact Hours. Of the 5 total youth sampled,
0 received at least the recommended hours of service. Youth in the sample completed between 9
and 31 hours of service, with an average of 22 hours.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Lake Academy can optimize their SPEP Quality of Service Delivery score by ensuring that fidelity
monitoring is conducted at least monthly for all facilitators.
Lake Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

service_type: Thinking for a Change


summary: SPEP - FY15-16

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Lake Academy
G4S Youth Services, LLC
(Contract Provider)
9504 East Columbus Drive
Tampa, Florida 33619
Primary Service: VOICES
SPEP Review Date(s): January 5‐8, 2016
Florida Department of Juvenile Justice
Report Date(s): 8/31/16
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Lake Academy MQI Program Code: 1069
Provider Name: G4S Youth Services, LLC Contract Number: R2104
Location: Hillsborough County / Circuit: 13 Number of Beds: 48
Review Date(s): January 5-8, 2016 Lead Reviewer Code: 140
Persons Interviewed
Program Director Corporate QI/QA staff 2 # Program Supervisors
DJJ Monitor 1 # Case Managers 5 # Youth
DHA or designee 1 # Clinical Staff # Other (listed by title):
DMHCA or designee 1 # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 8 # Personnel Records
Fidelity Monitoring Documents Program Schedules 7 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 5 # Youth Records (Closed)
Staff Evaluations Table of Organization 7 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports 10 # Health Records
Contract Scope of Services 12 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is VOICES. The program was awarded 25 points because
the Primary Service is identified as a Group 4 Service. The specific Sub-Component Service
Type identified is Group Counseling. The Primary Service was identified as this type of service as
it focuses on psychological or interpersonal problems or issues faced by an individual and
involves a group of youths interacting with each other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program has five facilitators trained in the VOICES curriculum and all facilitators are current
therapists at the program. Three of the six facilitators conducted groups since the last annual
compliance review. All five training files contained documentation of certification in the VOICES
curriculum. The training received by all facilitators was conducted by a qualified trainer.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program is using the 2004 VOICES curriculum developed by Stephanie S. Covington.
VOICES curriculum is provided to all the girls in the program. The program’s manual is on-site,
detailed and explains how each session is to be delivered. The program maintains a VOICES
teacher’s guide, as well as, a student guide.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The facilitator provided the reviewer with a copy of the lesson plan from the programs manual
prior to the start of the class. The lesson plan being covered was Lesson 3: Breaking the Silence
and Lesson 4: The World Girls Live In. Reviewer observed the facilitator does not read the lesson
verbatim but uses her own words to convey the meaning of the topic under discussion and does
so in an easy to follow manner. The facilitator engaged the youth to participate in the lesson and
the youth completed three activities during the session. The facilitator adhered to the lesson plan
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
in every aspect and was able to keep the class on track even when there was outside distractions
from adjacent rooms.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There have been no gaps in service delivery and no turnover of facilitators since the last annual
review. There was a gap of approximately a month between the start and end dates of each group
to allow for enough youth to enter the facility and be ready to start a new group.
BasicScore: 1Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
There were three staff members performing fidelity monitoring for the VOICES curriculum and the
fidelity monitors have been certified to teach the primary service. There were two facilitators of the
VOICES group since the last annual compliance review, and fidelity monitoring was conducted on
both facilitators during the times they taught the VOICES curriculum. Internal fidelity adherence
checklists are utilized monthly by the fidelity monitors to capture specifics of the primary service.
There was no fidelity monitoring conducted during the month of March for the primary service.
The program can earn 2 points by ensuring that internal fidelity monitoring is conducted at least
monthly for all facilitators.
At the time of the review, the program did not meet criteria to earn a score of 2.
BasicScore: 1Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process by which corrective action is applied and demonstrated based on the
fidelity monitoring of the primary service. A review of the documentation indicated corrective
action was not required.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Performance evaluations were completed on the two facilitators who delivered the VOICES
curriculum during the review period. Each facilitator was evaluated on their ability to demonstrate
an appropriate level of skill when delivering the primary service, as well as, their ability to provide
quality services to the youth being served.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 24 weeks for this type of service. Of the 7 youth in the
sample, 0% (0 of 7) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 40 contact hours for this type of service. Of the 7 youth in the
sample, 0% (0 of 7) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 13 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 52%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 71%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 0 youth
Moderate-High = 3 youth
High = 2 youth
Total Youth in Sample = 7 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 29%
High Score: 8 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 62%
Table 2
High = 2 youth
Total Youth in Sample = 7 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 71% (5 of 7) youth scored Moderate to High-Risk to reoffend, for a score of
5 points.
Of the SPEP sample, 29% (2 of 7) youth scored High-Risk to reoffend, for a score of 8 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 13 25 52%
Totals 63 95 66%
This SPEP report evaluates VOICES, an intervention delivered at Lake Academy.
The program scored High for Quality of Service Delivery. This score can be optimized by
ensuring that internal fidelity monitoring is conducted at least monthly for all facilitators.
The program earned 0 points for Amount of Service: Duration. Of the 7 total youth sampled, none
received at least the recommended weeks of service. Youth in the sample completed between 8
and 9 weeks of service. The program earned 0 points for Amount of Service: Contact Hours. Of
the 7 total youth sampled, none received at least the recommended hours of service. All youth
completed 27 hours of service. Lake Academy, a G4S-operated facility, acknowledged the SPEP
dosage targets, but respectfully declined to adjust the VOICES curriculum specifications to
accommodate increased SPEP targeted dosage levels over those dosage levels specified in the
VOICES curriculum.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program was awarded 13 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Lake Academy can optimize their SPEP Quality of Service Delivery score by ensuring that internal
fidelity monitoring is conducted at least monthly for all facilitators.
Lake Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

service_type: VOICES: A Program of Self-Exploration and Empowerment


summary: SPEP - FY16-17

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Lake Academy
G4S Youth Services, LLC
(Contract Provider)
9504 East Columbus Drive
Tampa, Florida 33619
Primary Service: Impact of Crime
SPEP Review Date(s): January 31‐February 3, 2017
Florida Department of Juvenile Justice
Report Date(s): 7/21/2017
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Lake Academy QI Program Code: 1069
Provider Name: G4S Youth Services, LLC Contract Number: R2104
Location: Hillsborough County / Circuit: 13 Number of Beds: 48
Review Date(s): January 31-February 2, 2017 Lead Reviewer Code: 140
Persons Interviewed
Program Director Corporate QI/QA staff 2 # Program Supervisors
DJJ Monitor 2 # Case Managers 5 # Youth
DHA or designee 2 # Clinical Staff # Other (listed by title):
DMHCA or designee 2 # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 7 # Personnel Records
Fidelity Monitoring Documents Program Schedules 7 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 7 # Youth Records (Closed)
Staff Evaluations Table of Organization 7 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports 7 # Health Records
Contract Scope of Services 7 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime. The program was awarded 15 points
because the Primary Service is identified as a Group 3 Service. The specific Sub-Component
Service Type identified is Social Skills Training. The Primary Service was identified as this type of
service as it focuses on developing social skills required for an individual to interact in a positive
way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program has two facilitators trained in the Impact of Crime curriculum. Two facilitators
conducted group in the last twelve-month period. A review of both facilitator’s training files
contained documentation that both received formal training in the curriculum. A qualified trainer
trained both facilitators in the primary service.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
All youth admitted at Lake Academy are assessed by staff as needing evidence based services.
Based on various criteria, they are placed into program services. The program is utilizing the
Impact of Crime curriculum and the manual details the primary service and explains how each
specific session is to be delivered. It includes a script, gives outlines for each group session, and
specific details on the delivery of the lessons.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The primary service is held each Friday and Saturday from 6:00 pm to 7:00 pm and was observed
on Friday, 02/03/17. The topic covered was Chapter 4, Impaired Driving and Drugs and Alcohol.
The facilitator provided the reviewer with a copy of the lesson, adhered to the written protocol, did
not read from the script verbatim and explained the lesson in his own words which was
understandable to the youth.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The reviewed documentation and the Clinical Director confirmed there was staff turnover in the
last twelve months, however, that turnover did not adversely affect the youth due to proper
transitioning of the facilitators between groups. There were no gaps in service. The program has
a policy for facilitator turnover. If a facilitator is going to be leaving the program or will be on leave,
they explain to the youth why another facilitator will be taking over the class. Another qualified
facilitator from the trained staff would conduct that class.
BasicScore: 2Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Internal fidelity monitoring reports were reviewed. The program has an internal process to conduct
internal fidelity monitoring specific to Impact of Crime. The LMHC performs all monitoring. He is
also trained in the Impact of Crime curriculum and is the only staff member who performs
monitoring for the Impact of Crime group. The fidelity monitor has received certification in the
primary service and fidelity monitoring. The program utilizes fidelity adherence checklists to
document the monitoring. Fidelity monitoring has been conducted monthly on the staff members
conducting the Impact of Crime classes within the last twelve months.
BasicScore: 1Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the Clinical Director verified that the program has a process for corrective
actions based on fidelity monitoring conducted. The program uses internal fidelity
monitoring/correction action checklists specific to Impact of Crime. Each monitoring checklist
includes a section for feedback and recommendations for facilitator improvement. There were no
corrective actions noted pertaining to the Impact of Crime service in the last 12 months.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Performance evaluations were completed on the trained staff who have facilitated the Impact of
Crime intervention in the past twelve months. The evaluations were reviewed and were found to
include no entries specific to Impact of Crime.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 16 weeks for this type of service. Of the 17 youth in the
sample, 6% (1 of 17) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 24 contact hours for this type of service. Of the 17 youth in the
sample, 76% (13 of 17) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 0 youth
Moderate-High = 4 youth
High = 13 youth
Total Youth in Sample = 17 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 76%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 13 youth
Total Youth in Sample = 17 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (17 of 17) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 76% (13 of 17) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 25 25 100%
Totals 71 85 84%
This SPEP report evaluates Impact of Crime, an intervention delivered at Lake Academy.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 17 total youth sampled, all
included dosage with end dates in the EBS Module. Of those youth with correct dosage, none
received at least the recommended weeks of service. Youth in the sample completed between 6
and 16 weeks of service, with an average of 13 weeks.
The program earned 6 points for Amount of Service: Contact Hours. Of the 17 total youth
sampled, all included dosage in the EBS Module. Of those youth with correct dosage, 13 received
at least the recommended hours of service. Youth in the sample completed between 7 and 27
hours of service, with an average of 22 hours.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Lake Academy can optimize their SPEP Quality of Service Delivery score by ensuring facilitators
are assessed specifically related to IOC on their performance evaluations.
Lake Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10


summary: SPEP - FY16-17

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Lake Academy
G4S Youth Services, LLC
(Contract Provider)
9504 East Columbus Drive
Tampa, Florida 33619
Primary Service: Individual Counseling
SPEP Review Date(s): January 31‐February 3, 2017
Florida Department of Juvenile Justice
Report Date(s): 7/21/2017
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Lake Academy QI Program Code: 1069
Provider Name: G4S, LLC Contract Number: R2104
Location: Hillsborough County / Circuit: 13 Number of Beds: 48
Review Date(s): January 31-February 2, 2017 Lead Reviewer Code: 140
Persons Interviewed
Program Director Corporate QI/QA staff 2 # Program Supervisors
DJJ Monitor 2 # Case Managers 5 # Youth
DHA or designee 2 # Clinical Staff # Other (listed by title):
DMHCA or designee 2 # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 7 # Personnel Records
Fidelity Monitoring Documents Program Schedules 7 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 7 # Youth Records (Closed)
Staff Evaluations Table of Organization 7 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports 7 # Health Records
Contract Scope of Services 7 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
1. Primary Service and Supplemental Service Types POS: 10 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Individual Counseling. The program was awarded 5
points because the Primary Service is identified as a Group 1 Service. The specific Sub-
Component Service Type identified is Individual Counseling. The Primary Service was identified
as this type of service as it includes any range of treatment techniques that focus on
psychological or interpersonal problems or issues faced by an individual that involves a one-on-
one relationship with a therapist or counselor.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program therapists provide individual counseling to the youth. They have a Clinical Director
who is a Licensed Mental Health Counselor and is responsible for the implementation and
coordination of individual counseling. The staff also includes a lead therapist who is a licensed
clinical social worker who provides individual counseling to the youth. The program also has five
masters level therapists who have received on the job training specific to therapeutic services.
They provide individual therapy to the youth in the program. All training was documented in the
Department’s Learning Management System (SkillPro) or in the therapist’s individual training files.
All licensed mental health staff and masters level therapists are required to complete CEUs in
order to maintain their individual licenses as well as training through G4S.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program has a written comprehensive plan for mental health and substance abuse services
which defines the protocol for individual counseling. Each youth is provided with individual therapy
at a minimum of one session per week for at least fifty minutes. A review of seven mental health
records revealed documentation of weekly individual progress notes for each of the reviewed
youth. A treatment plan is customized to meet the needs of the individual youth because all have
different issues. Therapists at Lake Academy primarily offer Cognitive Behavioral Therapy. The
program maintains some records such as training records and evaluations in the SPEP binder,
however, all confidential counseling records are maintained in the individual youth’s file.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Individual counseling sessions are not observed as part of the evaluation process for SPEP
services. Interviews with the Clinical Director, therapists, and youth validated individual counseling
is being conducted on a weekly basis. Furthermore, a review of seven mental health records
revealed documentation of weekly individual sessions for each of the reviewed youth.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
During the last twelve months, the program hired two new masterS level therapists to provide
individual counseling to the youth. Documentation supported there were not gaps in the primary
service when the turnover occurred. Progress notes supported other therapists at the program
were able to provide the primary service to the youth while the turnover was occurring.
The program can earn 2 points if turnover does not impact youth.
At the time of the review, the program did not meet criteria to earn a score of 2.
BasicScore: 2Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Individual counseling fidelity monitoring occurs by the Clinical Directors or lead therapist. They are
responsible for reviewing each youth’s weekly mental health progress notes and approving and
signing them. Furthermore, fidelity monitoring is completed during the weekly clinical supervision
meetings. The Clinical Director's documentation of weekly supervision meetings revealed
therapists are conducting individual counseling with youth at the program with fidelity.
BasicScore: 1Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process by which corrective action is applied and demonstrated, which is
outlined in the program's comprehensive plan for mental health and substance abuse services.
There has been no corrective action necessary during the last twelve months.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1Point
Intervention Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Performance evaluations were completed on all five therapists during the last twelve months. The
evaluations included a section for job specific criteria in relation to clinical services. All
evaluations addressed the therapist’s ability to conduct screenings, assessments, evaluations,
counseling sessions, and their ability to complete a mental health/substance abuse discharge
plan. Note, only three of the five specifically used the name of Individual Counseling.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 25 weeks for this type of service. Of the 30 youth in the
sample, 80% (24 of 30) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 30 contact hours for this type of service. Of the 30 youth in the
sample, 70% (21 of 30) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 0 youth
Moderate-High = 10 youth
High = 20 youth
Total Youth in Sample = 30 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 67%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 20 youth
Total Youth in Sample = 30 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (30 of 30) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 67% (20 of 30) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 10 10 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 25 25 100%
Totals 69 75 92%
This SPEP report evaluates Individual Counseling, an intervention delivered at Lake Academy.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 30 total youth sampled, all
included dosage with end dates in the EBS Module. Of those youth with correct dosage, 24
received at least the recommended weeks of service. Youth in the sample completed between 13
and 94 weeks of service, with an average of 45 weeks.
The program earned 6 points for Amount of Service: Contact Hours. Of the 30 total youth
sampled, all included dosage in the EBS Module. Of those youth with correct dosage, 21 received
at least the recommended hours of service. Youth in the sample completed between 12 and 85
hours of service, with an average of 43 hours.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Lake Academy can optimize their SPEP Quality of Service Delivery score by minimizing the
impact of therapist turnover.
Lake Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10