summary: SPEP - FY18-19

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Lake Academy
Truecore Behavioral Solutions, LLC
(Contract Provider)
9504 East Columbus Drive
Tampa, Florida 33619
Primary Service: Impact of Crime
SPEP Review Date(s): 03/12/19-03-15-19
Florida Department of Juvenile Justice
Report Date(s): 03/14/19
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime. The program was awarded 15 points
because the Primary Service is identified as a Group 3 Service. The specific Sub-Component
Service Type identified is Social Skills Training. The Primary Service was identified as this type of
service as it focuses on developing social skills required for an individual to interact in a positive
way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There were four staff trained in the Impact of Crime curriculum, one Case Manager and three
Youth Care Workers. Two of the facilitators have been consistently facilitating this group during
the review period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The manual provided to this reviewer was Impact of Crime, Addressing the Harm. The manual
contains a script for the facilitator for the groups. It provides lessons and includes implementation
guidelines, identifies the order the lessons should be delivered in and includes detailed
instructions for the delivery of each lesson.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The Impact of Crime group was observed on Tuesday, 03/12/19 from 5:00-6:00 p.m. The
facilitator provided a copy of the lsson plan for the observer from the program manual. The
facilitator followed the lesson according to the manual, did not read the lesson verbatim used their
own words to convey the top and was easy for the youth to follow. The facilitator adhered to the
lesson plan in every aspect.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There has been no facilitator turnover in the past twelve (12) months that adversly affected the
delivery of Impact of Crime (IOC) to the youth. There were no gaps in services of IOC to the
youth.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation for fidelity monitoring was reviewed. The program was missing monitoring for
the months of July, August, and September 2018.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the Regional Clinical Director verified the program has a process for corrective
actions based on the fidelity monitoring conducted. The program uses internal fidelity
monitoring/corrective action checklists specific to Impact of Crime. A review of documentation
provided and an interview with the Clinical Director verified there was no corrective action taken
during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The performance evaluation of all trained facilitators found entries on the evaluations of the staff's
performance in the delivery of Impact of Crime. These included comments on any strengths or
weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates the target duration of 16 weeks for this type of service. Of the 44 youth in the
sample, 34% (15 of 44) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 24 contact hours for this type of service. Of the 44 youth in the
sample, 93% (41 of 44) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 98%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 3 youth
Moderate-High = 9 youth
High = 31 youth
Total Youth in Sample = 44 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 70%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 31 youth
Total Youth in Sample = 44 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 98% (43 of 44) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 70% (31 of 44) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 2 10 20%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 25 25 100%
Totals 75 85 88%
This SPEP report evaluates Impact of Crime, an intervention delivered at Lake Academy.
57 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 2 points for Amount of Service: Duration. Of the 44 total youth sampled, all
44 included dosage with end dates in the EBS Module. Of those youth, 15 received at least the
recommended weeks of service. Youth in the sample completed between 4 and 23 weeks of
service, with an average of 14 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 44 total youth
sampled, all 44 included dosage in the EBS Module. Of those youth, 41 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
recommended hours of service. Youth in the sample completed between 6 and 32 hours of
service, with an average of 30 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Impact of Crime is an intervention suitable for all youth. The Department recommends providing
this service to every youth in the program.
Lake Academy can optimize their SPEP Quality of Service Delivery score by ensuring fidelity
monitoring is completed every month for each facilitator.
Lake Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8


state: FL

facility_name: Lake Academy

summary: SPEP - FY18-19

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Lake Academy
Truecore Behavioral Solutions, LLC
(Contract Provider)
9504 East Columbus Drive
Tampa, Florida 33619
Primary Service: Individual Counseling
SPEP Review Date(s): 03/12/19-03-15-19
Florida Department of Juvenile Justice
Report Date(s): 03/15/19
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 10 Points
1. Primary Service and Supplemental Service Types POS: 10 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Individual Counseling. The program was awarded 5
points because the Primary Service is identified as a Group 1 Service. The specific Sub-
Component Service Type identified is Individual Counseling. The Primary Service was identified
as this type of service as it includes any range of treatment techniques that focus on
psychological or interpersonal problems or issues faced by an individual that involves a one-on-
one relationship with a therapist or counselor.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There were nine clinicians trained to provide Individual Counseling to the youth in the program.
The program maintains all information regarding individual counseling in the youth's mental
health/substance abuse record. There was adequate documentation to support that each
Therapist had the requisite education and experience to provide counseling sessions. All nine
clinicians provided counseling during the review period of the last twelve (12) months. This
primary service does not have a formalized curriculum. The facilitators receive additional training
as needed which is documented during their clinical supervision.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
There is no written curriculum/manual for Individual Counseling. The program has a written
protocol for this intervention. The dosage is stated on the youth's individualized treatment plan.
This plan is then reviewed and signed by a licensed clinician.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
This was no observed during the review due to the nature of individual counseling. Therefore, this
indicator was rated as non-applicable.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There has been some facilitator turnover in the past twelve (12) months, however the turnover did
not adversely affect Individual Counseling to the youth. There were no gaps in service during the
review period. In the event of turnover, the facility has other therapists employed by the program
who would assist in the delivery of Individual Counseling to the youth.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has documented fidelity monitoring for observations of the Individual Counseling.
The fidelity monitoring was completed by the program's Clinical Director. It is completed through a
review of therapy notes to verify provision of effective and appropriate interventions which are in
line with the youth's treatment plan. The Clinical Director reviews and ensures compliance with
dosage. There was documentation of the Clinical Director conducting this review. The fidelity
monitoring was completed each month and the primary service was provided for the past twelve
(12) months, without exception.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the Regional Clinical Director verified the program has a process for corrective
actions based on the fidelity monitoring conducted. A review of documentation provided and an
interview with the Regional Clinical Director verified there was no corrective action taken during
the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The performance evaluations of all nine therapists found entries on the evaluations of the staff's
performance in the delivery of Individual Counseling. These included comments on any strengths
or weaknesses of which the clinician should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 25 weeks for this type of service. Of the 57 youth in the
sample, 88% (50 of 57) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 30 contact hours for this type of service. Of the 57 youth in the
sample, 74% (42 of 57) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 95%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 3 youth
Moderate-High = 11 youth
High = 40 youth
Total Youth in Sample = 57 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 70%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 40 youth
Total Youth in Sample = 57 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 95% (54 of 57) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 70% (40 of 57) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 10 10 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 25 25 100%
Totals 69 75 92%
This SPEP report evaluates Individual Counseling, an intervention delivered at Lake Academy.
57 youth completed the program during the review period. Of those youth, each was eligible to
receive this service while at the program.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 57 total youth sampled, only
52 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 50
received at least the recommended weeks of service. Youth in the sample completed between 7
and 76 weeks of service, with an average of 40 weeks.
The program earned 6 points for Amount of Service: Contact Hours. Of the 57 total youth
sampled, only 56 included dosage in the EBS Module. Of those youth with correct dosage, 42
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
received at least the recommended hours of service. Youth in the sample completed between 6
and 72 hours of service, with an average of 37 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Lake Academy can maintain their SPEP Quality of Service Delivery score by continuing the
practices in place at the time of this review.
Lake Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8

source: DJJ

report_type: SPEP

program_name_raw: Lake Academy

program_status_note: Closed

service_type: Individual Counseling


summary: SPEP - FY18-19

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Lake Academy
Truecore Behavioral Solutions, LLC
(Contract Provider)
9504 East Columbus Drive
Tampa, Florida 33619
Primary Service: Thinking for a Change
SPEP Review Date(s): 03/12/19-03-15-19
Florida Department of Juvenile Justice
Report Date(s): 03/14/19
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change. The program was awarded 30
points because the Primary Service is identified as a Group 5 Service. The specific Sub-
Component Service Type identified is Cognitive Behavioral Therapy. The Primary Service was
identified as this type of service as it is intended to correct faulty cognitions and perceptions and
provides skills individuals can use to monitor thought patterns and correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There were four staff trained in the Thinking for a Change curriculum, one Assistant Facility
Administrator, two Youth Care Workers and one Recreational Therapist. Three of the facilitators
have been consistently facilitating this group during the review period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The manual provided to this reviewer was Thinking for a Change 3.1. The manual contains a
script for the facilitator for the groups. It provides 25 lessons and includes implementation
guidelines, identifies the order the lessons should be delivered in and includes detailed
instructions for the delivery of each lesson.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The Thinking for a Change group had just completed a regular cycle on 02/21/19 and an Aftercare
group on 03/09/19. The group could not be observed, however instead of receiving a non-
applicable rating for this indicator, no points were deducted as the group had just completed just
prior to the review and completed an Aftercare group on 03/09/19.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There has been no facilitator turnover in the past twelve (12) months which adversely affected the
delivery of Thinking for a Change to the youth. The Thinking for a Change Group appeared to
have had a short gap between cohorts; however, per the Regional Clinical Director, this was
caused by the wait for enough new youth to start this closed group. Once enough new youth were
admitted, a new group was started.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation for fidelity monitoring was reviewed. The program was missing monitoring for
the months of June, September, November, and December 2018.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the Regional Clinical Director verified the program has a process for corrective
actions based on the fidelity monitoring conducted. The program uses internal fidelity monitoring /
corrective action checklists specific to Thinking for a Change. A review of documentation provided
and an interview with the Clinical Director verified there was no corrective action taken during the
review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The performance evaluation of two of the three trained facilitators found entries on the evaluations
of the staff's performance in delivering Thinking for a Change. These included comments on any
strengths or weaknesses of which the facilitator should be aware. One of the Youth Care Workers
had not yet been employed for one year and is awaiting her evaluation to be completed. TrueCore
no longer does Probationary Evaluations. This employee is rated as non-applicable for this
indicator.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 15 weeks for this type of service. Of the 12 youth in the
sample, 75% (9 of 12) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates a target of 45 contact hours for this type of service. Of the 12 youth in the
sample, 33% (4 of 12) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 83%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 0 youth
Moderate-High = 2 youth
High = 8 youth
Total Youth in Sample = 12 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 67%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 8 youth
Total Youth in Sample = 12 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 83% (10 of 12) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 67% (8 of 12) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 2 10 20%
Risk Level of Youth Served 20 25 80%
Totals 83 100 83%
This SPEP report evaluates Thinking for a Change, an intervention delivered at Lake Academy.
57 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 6 points for Amount of Service: Duration. Of the 12 total youth sampled, all
12 included dosage with end dates in the EBS Module. Of those youth, 9 received at least the
recommended weeks of service. Youth in the sample completed between 12 and 28 weeks of
service, with an average of 22 weeks.
The program earned 2 points for Amount of Service: Contact Hours. Of the 12 total youth
sampled, all 12 included dosage in the EBS Module. Of those youth, 4 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
recommended hours of service. Youth in the sample completed between 24 and 48 hours of
service, with an average of 39 hours.
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 88%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Lake Academy can optimize their SPEP Quality of Service Delivery score by ensuring fidelity
monitoring is completed every month for each facilitator.
Lake Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8

service_type: Thinking for a Change


summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Lake Academy
TrueCore Behavioral Solutions, LLC
(Contract Provider)
9504 East Columbus Drive
Tampa, Florida 33619
Primary Service: Impact of Crime
SPEP Review Date(s): 04/10/20
Florida Department of Juvenile Justice
Report Date(s): 04/13/20
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There were six staff trained in the Impact of Crime (IOC) curriculum - one Facility Administrator,
one Director of Case Management, one Regional Coordinator, one Restorative Justice Counselor,
one Recreational Therapist, and one Clinicial Director. Three of these staff facilitated IOC during
the review period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The manual provided to this reviewer was Impact of Crime; Addressing the Harm. The manual
contains a script for the facilitator for the groups. It provides lessons and includes implementation
guidelines, identifies the order the lessons should be delivered in and includes detailed
instructions for the delivery of each lesson.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The Impact of Crime group was not observed during the review period. The review had to be done
offsite due to the COVID-19 related travel restrictions. This curriculum was not able to be
observed; therefore, this indicator rates as non-applicable.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There has been no facilitator turnover in the past 12 months which adversely affected the delivery
of Impact of Crime to the youth. There were no gaps in services of Impact of Crime to the youth.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation for fidelity monitoring was reviewed. The program was missing monitoring for
the month of 09/19. All other months were conducted for each individual facilitating groups.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the Clinical Director verified the program has a process for corrective actions
based on the fidelity monitoring conducted. The program uses internal fidelity monitoring /
corrective action checklists specific to Impact of Crime. A review of documentation provided and
an interview with the Clinical Director verified there was no corrective action noted nor taken
during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The performance evaluations of all trained facilitators found entries of the staff's performance in
the delivery of Impact of Crime. These included comments on any strengths or weaknesses of
which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 16 weeks for this type of service. Of the 29 youth in the
sample, 0% (1 of 29) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 24 contact hours for this type of service. Of the 29 youth in the
sample, 76% (22 of 29) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 97%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 5 youth
Moderate-High = 6 youth
High = 17 youth
Total Youth in Sample = 29 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 59%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 17 youth
Total Youth in Sample = 29 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 97% (2 of 29) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 59% (17 of 29) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 25 25 100%
Totals 71 85 84%
This SPEP report evaluates Impact of Crime, an intervention delivered at Lake Academy.
43 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 30 total youth sampled, all
30 included dosage with end dates in the EBS Module. Of those youth, 1 received at least the
recommended weeks of service. Youth in the sample completed between 7 and 18 weeks of
service, with an average of 12 weeks.
The program earned 6 points for Amount of Service: Contact Hours. Of the 30 total youth
sampled, all 30 included dosage in the EBS Module. Of those youth, 23 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
recommended hours of service. Youth in the sample completed between 14 and 32 hours of
service, with an average of 28 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Impact of Crime is an intervention suitable for all youth. The Department recommends providing
this service to every youth in the program.
Lake Academy can optimize their SPEP Quality of Service Delivery score by ensuring fidelity
monitoring is completed every month for each facilitator.
Lake Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8

service_type: Impact of Crime


summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Lake Academy
TrueCore Behavioral Solutions, LLC
(Contract Provider)
9504 East Columbus Drive
Tampa, Florida 33619
Primary Service: Individual Counseling
SPEP Review Date(s): 04/10/20
Florida Department of Juvenile Justice
Report Date(s): 04/13/20
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 10 Points
1. Primary Service and Supplemental Service Types POS: 10 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Individual Counseling.
The program was awarded 5 points because the Primary Service is identified as a Group 1
Service. The specific Sub-Component Service Type identified is Individual Counseling. The
Primary Service was identified as this type of service as it includes any range of treatment
techniques that focus on psychological or interpersonal problems or issues faced by an individual
that involves a one-on-one relationship with a therapist or counselor.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There were seventeen clinicians trained to provide Individual Counseling to the youth in the
program, six of which provided counseling during the review period. The program mantains all
information regarding individual counsleing in the youth's mental health/substance abuse record.
There was adequate documentation to support each Therapist had the requisite education and
experience to provide counseling sessions. This primary service does not have a formalized
curriculum. The facilitators receive additional training as needed which is documented during their
clinical supervision.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
There is no formal written curriculum/manual for Individual Counseling; however, the program has
a written protocol for this intervention. The dosage is stated on the youth's individualized
treatment plan. This plan is then reviewed and signed by a licensed clinician.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Due to the nature of individual counseling, no observation was completed; therefore, this indiciator
was rated as non-applicable.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There has been some facilitator turnover during the review period; however, the turnover did not
adversely affect Individual Counseling to the youth. There were no gaps in service during the
review period. In the event of turnover, the facility has other Therapists employed by the program
who would assist in the delivery of Individual Counseling to the youth.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has documented monthly fidelity monitoring for observations of Individual
Counseling. The fidelity monitoring was completed by the program's Clinical Director. It is
completed through a review of therapy notes to verify provision of effective and appropriate
interventions which are in line with the youth's treatment plan. The Clinical Director reviews and
ensures compliance with dosage. There was documentation of the Clinical Director conducting
this review. The fidelity monitoring was completed each month and the primary service was
provided without interruption for the past twelve months.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the Clinical Director verified the program has a process for corrective actions
based on the fidelity monitoring conducted. A review of documentation provided and an interview
with the Clinical Director verified there was no corrective action noted nor taken during the review
period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The performance evaluations of all nine Therapists found entries of the staff's performance in the
delivery of Individual Counseling. These included comments on any strengths or weaknesses of
which the clinician should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 25 weeks for this type of service. Of the 42 youth in the
sample, 93% (39 of 42) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 30 contact hours for this type of service. Of the 42 youth in the
sample, 90% (38 of 42) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 98%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 5 youth
Moderate-High = 9 youth
High = 27 youth
Total Youth in Sample = 42 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 64%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 27 youth
Total Youth in Sample = 42 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 98% (41 of 42) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 64% (27 of 42) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 10 10 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 25 25 100%
Totals 71 75 95%
This SPEP report evaluates Individual Counseling, an intervention delivered at Lake Academy.
43 youth completed the program during the review period. Of those youth, all 42 were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 42 total youth sampled, only
40 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 39
received at least the recommended weeks of service. Youth in the sample completed between 16
and 69 weeks of service, with an average of 43 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 42 total youth
sampled, all 42 included dosage in the EBS Module. Of those youth, 38 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
recommended hours of service. Youth in the sample completed between 13 and 55 hours of
service, with an average of 36 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Lake Academy can maintain their SPEP Quality of Service Delivery score by continuing the
practices in place at the time of this review.
Lake Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8


summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Lake Academy
TrueCore Behavioral Solutions, LLC
(Contract Provider)
9504 East Columbus Drive
Tampa, Florida 33619
Primary Service: Thinking for a Change
SPEP Review Date(s): 04/10/20
Florida Department of Juvenile Justice
Report Date(s): 04/13/20
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Thinking for a Change.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There were four staff trained in the Thinking for a Change (T4C) curriculum - one Regional
Compliance Manager, one Clinical Director, one Restorative Justice Counselor, and one
Recreational Therapist. Three of these staff facilitated groups consistently during the review
period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The manual provided for review was Thinking for a Change 3.1. The manual contains a script for
the facilitator for the groups. It provides 25 lessons and includes implementation guidelines,
identifies the order the lessons should be delivered in and includes detailed instructions for the
delivery of each lesson.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The Thinking for a Change group was not observed during this review. The review had to be done
off-site due to the COVID-19 related travel restrictions; therefore, this indicator rates as non-
applicable.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There has been no facilitator turnover during the review period which adversely affected the
delivery of Thinking for a Change to the youth. The Thinking for a Change Group appeared to
have had a gap from 03/12/19 to 04/15/19 and from 08/15/19 to 02/03/20. According to the
Clinical Director, the gaps were caused by the wait for enough new youth to start this closed
group. Once enough new youth were admitted, a new group was started.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation for fidelity monitoring was reviewed. The program was missing monitoring
from the months of April through August, 2019.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the Clinical Director verified the program has a process for corrective actions
based on the fidelity monitoring conducted. The program uses internal fidelity monitoring /
corrective action checklists specific to Thinking for a Change. A review of documentation provided
and an interview with the Clinical Director verified there was no corrective action noted nor taken
during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The performance evaluations of the facilitators found entries the staff's performance in delivering
Thinking for a Change. These included comments on any strengths or weaknesses of which the
facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 15 weeks for this type of service. Of the 13 youth in the
sample, 100% (13 of 13) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 45 contact hours for this type of service. Of the 13 youth in the
sample, 100% (13 of 13) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 0 youth
Moderate-High = 3 youth
High = 10 youth
Total Youth in Sample = 13 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 77%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 10 youth
Total Youth in Sample = 13 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (13 of 13) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 77% (10 of 13) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 25 25 100%
Totals 100 100 100%
This SPEP report evaluates Thinking for a Change, an intervention delivered at Lake Academy.
43 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 13 total youth sampled, all
13 included dosage with end dates in the EBS Module. Of those youth, 13 received at least the
recommended weeks of service. Youth in the sample completed between 17 and 20 weeks of
service, with an average of 19 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 13 total youth
sampled, all 13 included dosage in the EBS Module. Of those youth, 13 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
recommended hours of service. Youth in the sample completed between 46 and 49 hours of
service, with an average of 47 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Lake Academy can optimize their SPEP Quality of Service Delivery score by ensuring fidelity
monitoring is completed every month for each facilitator.
Lake Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8


summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Lake Academy
TrueCore Behavioral Solutions, LLC
(Contract Provider)
9504 East Columbus Drive
Tampa, Florida 33619
Primary Service: Impact of Crime
SPEP Review Date(s): 03/15/20-03/14/21
SPEP Review Conducted: 5/5/21
Florida Department of Juvenile Justice
Report Date: 6/17/21
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There were four staff trained in the Impact of Crime (IOC) curriculum- one Director of Case
Management, one Facility Administrator, one Restorative Justice Counselor, and one Clinical
Director. Three of these staff facilitated IOC during the review period and the Clinical Director
provided Fidelity Monitoring
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The manual provided for review was Impact of Crime; Addressing the Harm. The manual contains
a script for the facilitator for the groups. It provides lessons and includes implementation
guidelines, identifies the order the lessons should be delivered in and includes detailed
instructions for the delivery of each lesson.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The Impact of Crime group was observed via Microsoft Teams on Wednesday 04/07/21, 6:00-
7:00 p.m. The facilitator followed the lesson plan according to the manual, did not read verbatim
and explained the lesson in her own words. The facilitator adhered to the lesson plan in every
aspect.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There was some facilitator turnover during the review period which adversely affected the
services of Impact of Crime to the youth. The facility has also experienced positive Covid19
testing, quarantine periods relating to positive testing, along with low census counts which have
caused gaps in services during the review period. However, given all these issues, they were still
able to complete 3 cohorts of Impact of Crime and have an ongoing group which started on
03/15/21. The gaps were from 04/04/20-04/25/20, 05/25/20-06/25/20, 08/13/20-09/08/20 and
12/12/20-03/14/21
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation for fidelity monitoring of Impact of Crime was reviewed. The program
completed fidelity monitoring for the months services were provided. They were missing
monitoring for the months of 03/20, 04/20, 05/20, 06/20, and 09/20.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
An interview with the Clinical Director verified the program has a process for corrective actions
based on the fidelity monitoring conducted. The program uses internal fidelity monitoring /
corrective action checklists specific to Impact of Crime. A review of documentation provided and
an interview with the Clinical Director verified there was no corrective action taken during the
review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The performance evaluations of all trained facilitators found entries on the evaluations of the
staff's performance in the delivery of Impact of Crime. These included comments on any strengths
or weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates the target duration of 16 weeks for this type of service. Of the 12 youth in the
sample, 50% (6 of 12) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 24 contact hours for this type of service. Of the 12 youth in the
sample, 92% (11 of 12) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 7 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 28%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 75%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 3 youth
Moderate-High = 6 youth
High = 0 youth
Total Youth in Sample = 12 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 0%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 0 youth
Total Youth in Sample = 12 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 75% (9 of 12) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 0% (0 of 12) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 4 10 40%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 7 25 28%
Totals 59 85 69%
This SPEP report evaluates Impact of Crime, an intervention delivered at Lake Academy.
30 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 4 points for Amount of Service: Duration. Of the 12 total youth sampled, all
12 included dosage with end dates in the EBS Module. Of those youth, 6 received at least the
recommended weeks of service. Youth in the sample completed between 6 and 16 weeks of
service, with an average of 14 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 12 total youth
sampled, all 12 included dosage in the EBS Module. Of those youth, 11 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
recommended hours of service. Youth in the sample completed between 15 and 33 hours of
service, with an average of 30 hours.
The program was awarded 7 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 91%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Impact of Crime is an intervention suitable for all youth. The Department recommends providing
this service to every youth in the program.
Lake Academy can optimize their SPEP Quality of Service Delivery score by ensuring fidelity
monitoring is completed every month for each facilitator.
Lake Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8