summary: SPEP - FY18-19

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Escambia Boys Base
AMIkids, Inc.
(Contract Provider)
Bldg 3780 Corry Station
Pensacola, Florida 32511
Primary Service: Cannabis Youth Treatment
SPEP Review Date(s): 1/29/19‐2/1/19
Florida Department of Juvenile Justice
Report Date(s): 2/1/19
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
 Service Type: service type points are awarded based on the group category that the
service falls within
 Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
 Amount of Service: or “dosage” refers to the amount and duration of services that were
received
 Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Cannabis Youth Treatment. The program was awarded
30 points because the Primary Service is identified as a Group 5 Service. The specific Sub-
Component Service Type identified is Cognitive Behavioral Therapy. The Primary Service was
identified as this type of service as it is intended to correct faulty cognitions and perceptions and
provides skills individuals can use to monitor thought patterns and correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The documentation provided by the program reflected the training protocol included all therapists
to be trained as facilitators in Cannabis Youth Treatment (CYT), and four therapists were formally
trained in CYT. One therapist left during the review period. When another therapist was hired,
they were formally trained in the curriculum and are running group.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program maintains the appropriate Cannabis Youth Treatment (CYT) manual. The manual
was copied, put in a binder with a copy of the manual in the front sleeve, and provided to the staff
during formal training. The program facilitates a combination of the curriculum; Motivational
Enhancement Therapy/Cognitive Behavioral Therapy (MET/CBT5 and CBT7). This combination is
allowed per the instructor’s manual. The manual includes the following: implementation
information, facilitator requirements, session requirements, script, and worksheets for the lessons
being provided. CYT is designed for youth ages 12-18 with a diagnosis of cannabis abuse or
dependence. The youth must be experiencing emotional, physical, legal, social, and/or academic
problems associated with marijuana use. CYT is currently held on Thursdays from 1:45pm-
3:00pm.
The facility maintains a black binder with a paper in front with the title MET/CBT printed on it.
The first section contains the schedule of when groups are currently held. The second section
contains all the attendance sheets separated by a blank sheet of green paper per cohort. The
attendance sheets included the type of group, location of group, date, time, group topic,
facilitator/credentials, youth names and signatures, comment section, and facilitator signature and
date. If a youth was absent, the majority of the attendance sheets had a reason listed why the
youth was absent. All of the attendance sheets had signatures completed.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
The third section includes the internal fidelity monitoring sheets for the full review period. The
fourth section contains the internal Associated Marine Institute (AMI) Standardized Program
Evaluation Protocol (SPEP) documents that are created to send to AMI headquarters on a
monthly basis. These are a snapshot of the group for each month and the program sends this
report to AMI headquarters. These reports include, sessions per month, primary facilitator and
training date, youth enrolled at the beginning of the month and number of youth enrolled at the
close of the month, along with an explanation for any enrollment changes.
The next section of the report contains information for each indicator of SPEP, i.e., Facilitator
Training, Primary Service Manual, Observed adherence to Primary Service Manual, Facilitator
Turnover, Internal Fidelity, Corrective Actions, and Evaluation of Facilitator Skill. Each of these
sections has a scoring and comment section for each indicator. The last section of the binder had
certifications for the facilitators. The section contained education information and training
certificates for CYT facilitators.
The binder was very organized and easy to find all information needed to complete the review.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
This reviewer observed Cannabis Youth Treatment (CYT) on Thursday from 1:45pm - 3:00pm.
The group was held in the classroom with nine youth in attendance, one facilitator, and one staff
member. The group began with check-ins and all the youth participated. The check-in was
innovative and interesting to see. The facilitator created a poster board with three houses drawn:
a straw house, stick house, brick house and the face of a pig. The question on the poster board
was "How are you living?"
The facilitator checked homework of all the youth and reviewed it with the youth that completed it.
The majority of the class completed and submitted their homework. The group was discussing
how to control anger and worked through anger management techniques. The youth were asked
what was a trigger for them during the last week and what was their physical and emotional
response. The facilitator went by the curriculum and lesson as designed. Three groups completed
role plays and were appropriate.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program did have one therapist and the recreational therapist leave during the review period.
This did not adversly affect the group. When the two staff left, there were two other clinical staff
trained in the Cannabis Youth Treatment (CYT) curriculum who were able to facilitate the class.
There was a gap in service between July 19, 2018 and October, 11, 2018, but this was not
directly related to facilitator turnover.
The program can earn 2 points if there are no unnecessary gaps in service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program's process for internal fidelity monitoring is for the Designated Mental Health Clinical
Authority to conduct internal fidelity monitoring for Cannabis Youth Treatment. There were three
facilitators monitored for fidelity during the review period. Out of the twleve month review period,
five months of internal fidelity monitoring was missing.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program's corrective action process is the Designated Mental Health Clinical Authority
(DMHCA) conducts weekly supervision with the facilitators. If any issues or corrective actions are
noted, the DMHCA will address them during the weekly supervisions. Upon review of the internal
fidelity documents no corrective actions or issues were noted during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Three therapists had received a yearly performance evaluation by the time of the annual review.
These documents were made available for review. The program provided yearly performance
evaluations for all current Cannabis Youth Treatment (CYT) facilitators who have been employed
for a year or longer. The evaluations include a section for group facilitation. There were no issues
on the performance evaluations.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 12 weeks for this type of service. Of the 45 youth in the
sample, 4% (2 of 45) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 12 contact hours for this type of service. Of the 45 youth in the
sample, 93% (42 of 45) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 78%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 3 youth
Moderate-High = 14 youth
High = 18 youth
Total Youth in Sample = 45 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 40%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 18 youth
Total Youth in Sample = 45 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 78% (35 of 45) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 40% (18 of 45) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 20 25 80%
Totals 83 100 83%
This SPEP report evaluates Cannabis Youth Treatment, an intervention delivered at Escambia
Boys Base.
50 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The targeted number of weeks for this specific intervention is 12 weeks; the targeted number of
hours for this specific intervention is 12 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 5 services (15 weeks; 45
hours).
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 0 points for Amount of Service: Duration. Of the 45 total youth sampled, all
45 included dosage with end dates in the EBS Module. Of those youth, 2 received at least the
recommended weeks of service. Youth in the sample completed between 6 and 19 weeks of
service, with an average of 9 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 45 total youth
sampled, all 45 included dosage in the EBS Module. Of those youth, 42 received at least the
recommended hours of service. Youth in the sample completed between 11 and 29 hours of
service, with an average of 15 hours.
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 88%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Escambia Boys Base can optimize their SPEP Quality of Service Delivery score by ensuring there
are no unnecessary gaps in service, and by ensuring fidelity monitoring is completed every month
for each facilitator.
Escambia Boys Base can optimize their SPEP Amount of Service score by ensuring that dosage
for all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9


state: FL

facility_name: Escambia Boys Base

summary: SPEP - FY18-19

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Escambia Boys Base
AMIkids, Inc.
(Contract Provider)
Bldg 3780 Corry Station
Pensacola, Florida 32511
Primary Service: Impact of Crime
SPEP Review Date(s): 1/29/19‐2/1/19
Florida Department of Juvenile Justice
Report Date(s): 2/1/19
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
 Service Type: service type points are awarded based on the group category that the
service falls within
 Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
 Amount of Service: or “dosage” refers to the amount and duration of services that were
received
 Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime. The program was awarded 15 points
because the Primary Service is identified as a Group 3 Service. The specific Sub-Component
Service Type identified is Social Skills Training. The Primary Service was identified as this type of
service as it focuses on developing social skills required for an individual to interact in a positive
way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program's documentation reflected the program had a training protocol and three staff were
formally trained in the Impact of Crime (IOC) curriculum. One clinical case manager, a shift
supervisor, and the recreational therapist have been trained to facilitate the curriculum. The
recreational therapist left the program during the review period.
The program's protocol is to have case management staff trained to facilitate IOC. There is
currently only one clinical case manager employed at the facility. Once more staff are hired they
will be trained to facilitate IOC.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program maintains the appropriate manual for Impact of Crime (IOC). There are four sections
to the IOC manual. The manual includes all needed worksheets and facilitator forms to conduct
IOC. The facility maintains a black binder with the title “Impact of Crime”. The binder was
separated into five sections. The first section contains the schedule of when groups are currently
held. IOC is currently held on Mondays and Wednesays from 12:00pm-1:00pm.
The second section contains all the attendance sheets separated by a blank sheet of green paper
per cohort. The attendance sheets are those created by the Department of Juvenile Justice and
include the program name, group sessions date, group session time, facilitator one and two. The
second section includes all of the lessons for IOC with checkboxes to indicate what lesson is
being held on the date. The third section has all youth names and DJJID numbers typed, a space
for the youth signature, and checkboxes to mark if homework was completed. The last section is
to indicate if a youth was dropped and a reason why, number of youth attending group, and if
there were any supportive lessons. The last section is for facilitators to sign and date.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
The third section of the binder includes the Department of Juvenile Justice internal fidelity
monitoring sheets for the review period.
The fourth section contains the internal Associated Marine Institute (AMI) fidelity monitoring
documents that are created to send to AMI headquarters on a monthly basis. These are a
snapshot of the group for each month and the program sends this report to AMI headquarters.
These reports include, sessions per month, primary facilitator and training date, youth enrolled at
the beginning of the month and number of youth enrolled at the close of the month, along with an
explanation for any enrollment changes. The next section of the report contains information for
each indicator of SPEP, i.e., Facilitator Training, Primary Service Manual, Observed adherence to
Primary Service Manual, Facilitator Turnover, Internal Fidelity, Corrective Actions, and Evaluation
of Facilitator Skill. Each of these sections has a scoring and comment section for each indicator.
The last section of the binder had certifications for the facilitators. The section contained
education information and training certificates for IOC facilitators. The binder was organized and
easy to find all information needed to complete the review.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
This reviewer observed the Impact of Crime (IOC) group on Wednesday, January 30, 2019 from
12:00pm-1:00pm. There were nine youth who participated in the IOC group. The group was being
introduced to chapter five. The facilitator conducted a short review of what they discussed during
the last meeting, what can interfere with making good decisions.
The facilitator is consistent in how the group is conducted. The facilitator is comfortable with the
material and does not rely on the manual. The facilitator discussed the ripple effect, along with
cause and effect. The youth also began going through the definitions for chapter five. The
faciltator consistently tied in previous information from other chapters. All youth were involved in
the group and participated. This faciltiator has been consistent over the past two review periods.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The reviewed documentation confirmed turnover of one trained facilitator, the recreational
therapist, in the Impact of Crime (IOC) curriculum. This turnover did not adversely affect the
group. The recreational therapist was co-facilitating IOC when the employment ended. The other
facilitator of the group continued on with the group with no interruptions.
There was one gap in service between October 19, 2018 and December 3, 2018. This reviewer
met with the current facilitator to ask about the gap in service. It was reported the gap was due to
all youth completing IOC. The facilitator had to wait for enough new youth to come in to the facility
to begin a new group. This was confirmed by the reviewer.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program's protocol is for the shift supervisor to conduct internal fidelity monitoring for Impact
of Crime (IOC). Twelve months of documentation was reviewed. Two months of documentation
were missing for internal fidelity monitoring.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program's protocol for corrective actions is for the Designated Mental Health Clinical
Authority (DMHCA) to meet with the clinical case manager once a week during supervisions. If
there are any corrective actions or issues noted it will be addressed during weekly supervisions.
Twelve months of internal fidelity monitoring forms were reviewed and there were no issues or
corrective actions noted.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of the clinical case manager's yearly performance evaluation found an entry for
evaluation of staff's performance in group facilitation. There were no concerns noted. The
recreational therapist was not at the facility long enough to receive an evaluation.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service ‐ Duration
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates the target duration of 16 weeks for this type of service. Of the 50 youth in the
sample, 36% (18 of 50) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 24 contact hours for this type of service. Of the 50 youth in the
sample, 88% (44 of 50) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 80%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 5 youth
Moderate-High = 15 youth
High = 20 youth
Total Youth in Sample = 50 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 40%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 20 youth
Total Youth in Sample = 50 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 80% (40 of 50) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 40% (20 of 50) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 2 10 20%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 20 25 80%
Totals 70 85 82%
This SPEP report evaluates Impact of Crime, an intervention delivered at Escambia Boys Base.
50 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 2 points for Amount of Service: Duration. Of the 50 total youth sampled, all
50 included dosage with end dates in the EBS Module. Of those youth, 18 received at least the
recommended weeks of service. Youth in the sample completed between 10 and 16 weeks of
service, with an average of 14 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 50 total youth
sampled,all 50 included dosage in the EBS Module. Of those youth, 44 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
recommended hours of service. Youth in the sample completed between 12 and 37 hours of
service, with an average of 27 hours.
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 88%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Escambia Boys Base can optimize their SPEP Quality of Service Delivery score by ensuring
fidelity monitoring is completed every month for each facilitator.
Escambia Boys Base can optimize their SPEP Amount of Service score by ensuring that dosage
for all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

source: DJJ

report_type: SPEP

program_name_raw: Escambia Boys Base

program_status_note: Open

service_type: Impact of Crime


summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Escambia Boys Base
AMIkids, Inc
(Contract Provider)
640 Roberts Road, Building 3780 Corry Station
Pensacola, Florida 32511
Primary Service: Aggression Replacement Training
SPEP Review Date(s): January 2019-February 2020
Florida Department of Juvenile Justice
Report Date(s): 4/22/2020
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Aggression Replacement Training.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program provided documentation of training in Aggression Replacement Training (ART).
There were three therapists, the designated mental health clinician authority (DMHCA), the
transition coordinator, and a previous clinical case manager formally trained in ART during the
review period. The transition coordinator did not have a certificate for completion of training;
however, there was documentation of the formal training on the AMIKids Standardized Program
Evaluation Protocol fidelity report. The transition coordinator provided documentation of the
request for the training certificate to this reviewer.
There are two facilitators who have consistently facilitated ART during the review period,
according to a review of the sign-in sheets.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program provided the Aggression Replacement Training (ART) manual for review. This
manual was the third edition-revised and expanded by authors Barry Glick and John Gibbs. The
manual was organized by each lesson: skill streaming, moral reasoning, and anger control. Each
lesson is divided by weeks. The book designates what the skill is for each lesson, lists the
objectives that should be met, materials that are needed, and then leads into the lesson. There is
a general script to follow and a procedure for each lesson to use as a guide. The skill steps are
broken into sub-steps for the facilitator to follow and steps the youth need to complete. There are
no student manuals for ART.
Aggression Replacement Training is held three days a week on Mondays, Wednesdays, and
Fridays from 2:00 pm to 3:00 pm. The group runs for thirty sessions over ten weeks.
Documentation for ART is kept in a binder in the designated mental health clinician authority's
(DMHCA) office. The binder is separated by group schedule, internal fidelity monitoring forms
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
separated by month, attendance sheets separated by cohort, and training records. The binder is
neatly organized and all information was able to easily be located. All groups were held for one
full hour and all sessions were provided in order per a review of the sign-in sheets. All five
completed cohorts met the required dosage. The groups averaged ten youth per group, with the
lowest being seven youth in the current group. The program's census is low at this time. The
youth are chosen for ART based upon past and current aggression issues and their Residential
Assessment for Youth results.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Aggression Replacement Training was observed on Wednesday, February 19, 2020 from
2:45pm-3:45pm. There were two facilitators and six youth attending group. One youth was out
taking a test for a forklift certificate. One facilitator shared the youth missing would have an
individual session conducted on a different day. The group lesson was Anger Control week 2 -
Hassle Log. All of the youth were engaged throughout the group. The facilitator did a great job
introducing the hassle log and explaining each of the components. The facilitator gave great
directions for the hassle logs and allowed the youth to provide examples and ask questions.
The facilitator and co-facilitator demonstrated the new skill to the group and provided a good
example for their role play. The youth were able to choose their role play partner and their
situations. The facilitators gave adequate time for the youth to prepare and conducted time
checks to see if the youth needed more time or not. Feedback was provided appropriately to each
youth throughout group. The youth who were not conducting a role play were given specific skills
to look for in each role play.
There were no behavioral issues during the group. Motivational Interviewing skills were used
throughout group. The youth were polite throughout group. It appeared they felt comfortable and
asked questions and asked for clarification when needed. One staff member sat in group and
engaged in group with the youth, to include assisting with role plays when needed.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
Three employees trained in Aggression Replacement Training (ART) left during the twelve month
review period, two clinical case managers and the designated mental health clinician authority;
none of the turnover affected the delivery of ART to youth. The clinical case managers who left
did not facilitate ART groups. According to a review of the sign-in sheets, there were no gaps in
service during the review period.
There have been five completed cohorts during the twelve month review period: February 2019 -
April 2019, April 2019 - June 2019, June 26, 2019 - August 30, 2019, September 2, 2019 -
November 8, 2019, and November 11, 2019 - January 10, 2020. A sixth cohort was started on
February 10, 2020.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Aggression replacement training (ART) requires each facilitator be monitored three times per
month for each part of ART; skillstreaming, anger control, and moral reasoning. The designated
mental health clinician authority (DMHCA) conducts internal fidleity monitoring for ART. When the
DMHCA initially started, she was unaware this was the requirement for fideilty monitoring.
Therefore internal fidlety monitoring was missing for three months of the review period. This
reviewer spoke with the DMHCA about this.
For the rest of the review period, approximately nine months, all internal fidelity monitoring logs
were reviewed for both facilitators of ART.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 0 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program's policy is for the designated mental health clinician authority (DMCHA) to conduct
interal fidelity monitoring. If there are issues or a corrective action is given the DMHCA meets with
facilitator after group and during weekly supervisions to address the issue. The facilitator is
observed again to ensure the issues has been corrected. If the issue is not corrected further
action is taken.
There were some low scores marked on the reviewed internal fidelity monitoring forms, but no
further information was provided on the comment sections or corrective action information. There
were no follow up notes or information provided on any other internal fidelity monitoring forms.
The facilitators receiving the low scores were not identified on the forms. There were no specific
corrective actions noted during this review period. Per an interview with the DMHCA there have
not been issues during Aggression Replacement Training fidelity.
The program can earn 1 point if any corrective action needed is identified, applied, and
demonstrated.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include evaluation
of skill in delivering the intervention/service.
The designated mental health clinician authority (DMCHA) conducts annual evaluations for
facilitators. The new DMHCA was hired in July 2019 and the annual evaluations for the two
Aggression Replacement Training (ART) facilitators were due in June of 2019. The evaluations
were not located on this review date of February 19, 2020.
The program's policy is for the DMHCA to conduct the annual evaluations, provide them to the
program director and then it is filed into the employee's personnel file. The annual evaluations
were not located in the program director's office or the personnel files. The annual evaluations
were requested for review by Friday, February 21, 2020. This reviewer visited the program on
Monday, February 24, 2020. During this visit the DMHCA provided the annual evaluations for both
facilitators of ART.
One evaluation was completed on July 8, 2019 and the other evaluation was completed on
February 24, 2020. The evaluation for the transition coordinator does not include an area for
group facilitation. This employee will need to be evaluated annually for group facilitation. The
therapist's annual evaluation included a section for group facilitation.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service - Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 10 weeks for this type of service. Of the 35 youth in the
sample, 71% (25 of 35) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 30 contact hours for this type of service. Of the 35 youth in the
sample, 74% (26 of 35) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 86%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 3 youth
Moderate-High = 13 youth
High = 14 youth
Total Youth in Sample = 35 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 40%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 14 youth
Total Youth in Sample = 35 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 86% (30 of 35) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 40% (14 of 35) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 23 25 92%
Totals 80 100 80%
This SPEP report evaluates Aggression Replacement Training, an intervention delivered at
Escambia Boys Base.
43 youth completed the program during the review period.
The program scored Medium for Quality of Service Delivery.
The targeted number of weeks for this specific intervention is 10 weeks; the targeted number of
hours for this specific intervention is 30 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 5 services (15 weeks; 45
hours).
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program earned 6 points for Amount of Service: Duration. Of the 35 total youth sampled, only
30 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 25
received at least the recommended weeks of service. Youth in the sample completed between 5
and 26 weeks of service, with an average of 10 weeks.
The program earned 6 points for Amount of Service: Contact Hours. Of the 35 total youth
sampled, all 35 included dosage in the EBS Module. Of those youth, 26 received at least the
recommended hours of service. Youth in the sample completed between 11 and 75 hours of
service, with an average of 31 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Escambia Boys Base can optimize their SPEP Quality of Service Delivery score by ensuring
fidelity monitoring is completed every month for each facilitator, by ensuring any corrective action
needed is identified, applied, and demonstrated, and by ensuring each facilitator’s performance
evaluation includes discussion of their skill in delivering this primary service.
Escambia Boys Base can optimize their SPEP Amount of Service score by ensuring that dosage
for all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

service_type: Aggression Replacement Training


summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Escambia Boys Base
AMIkids, Inc
(Contract Provider)
640 Roberts Road, Building 3780 Corry Station
Pensacola, Florida 32511
Primary Service: Cannabis Youth Treatment
SPEP Review Date(s): January 2019-February 2020
Florida Department of Juvenile Justice
Report Date(s): 4/22/2020
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Cannabis Youth Treatment.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program provided training documentation for Cannabis Youth Treatment (CYT). Therapists
are trained to conduct this group. The previous and current designated mental health clinician
authority (DMHCA), a previous clinical case manager, and two therapists were formally trained to
conduct CYT during this review period. The former clinincal case manager facilitated CYT groups.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The manual was copied, put in a binder with a copy of the manual in the front sleeve, and
provided to the staff during formal training. The program facilitates a combination of the
curriculum; Motivational Enhancement Therapy/Cognitive Behavioral Therapy (MET/CBT5 and
CBT7). This combination is allowed per the instructor’s manual. The manual includes
implementation information, facilitator requirements, session requirements, script, and worksheets
for the lessons being provided.
CYT is designed for youth ages 12-18 with a diagnosis of cannabis abuse or dependence. The
youth must be experiencing emotional, physical, legal, social, and/or academic problems
associated with marijuana use. CYT can be held as a closed or open group. The program
conducts CYT as a closed group. The groups are held on Tuesdays from 1:45 pm-3:00 pm for
twelve sessions. The first two sessions are conducted as individual counseling sessions to
prepare and evaluate the participants for group.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The Cannabis Youth Treatment group was not observed during this review. A group will be
observed during a quarterly review.
This curriculum was not able to be observed; therefore, this indicator rates as non-applicable.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program's protocol is for another trained therapist to facilitate Cannabis Youth Treatment
group if a facilitator should be out or leave employment. There has been one trained employee
who left during the twelve month review period. The trained clinical case manager left
employment in July 2019. This employee did not consistently conduct Cannabis Youth Treatment
groups and there was no impact to facilitation of groups.
There was a one month and five day gap between two cohorts. Per the designated mental health
clinical authority there was a gap in service due to waiting for enough youth to begin a cohort. The
gap was not due to facilitator turnover. Per review of the sign-in sheets there were no other gaps
in service. There have been five cohorts completed during the twelve month review.
The first group was held December 2018 through February 2019. The group started with ten
youth. One youth left the program and went to detention in late December. This youth did not
return to the program. The second group was held March 2019 through May 2019. The group
started with nine youth and completed with nine youth. There were some youth who missed a
session due to being on an outing with the program. Individual sessions were conducted and
documented to make up those sessions.
The third group started on June 12, 2019 through August 6, 2019. There was a gap in time
between the second and third group of one month and five days. Ten youth started this group and
all ten youth completed this group. The fourth group was held from August 2019 through October
2019. The group started and ended with seven youth. The fifth cohort was held from November
2019 through January 2020. This group started with seven youth and ended with six youth. One
youth was released from the program prior to completing this group. A sixth cohort was started on
February 4, 2020 and is continuing to meet at this time.
All documentation of the first two individual sessions were provided for review. Two youth
completed a second cohort due to failing random drug screens. The youth had previously
completed a cohort and tested positive after this. The program and youth decided the youth would
retake CYT to further address substance abuse issues.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program's protocol is for the two trained therapists to rotate completing internal fidelity
monitoring. The designated mental health clinician authority (DMHCA) currently conducts
Cannabis Youth Treatment (CYT) on a regular basis at the program. Two out of twelve internal
fidelity monitoring forms were missing for the review period.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program's protocol is for the two trained therapists to conduct internal fidelity monitoring for
Cannabis Youth Treatment (CYT) groups. According to the designated mental health clinical
authority, the two therapists have been facilitating CYT longer and are confident in conducting
fidelity monitoring. If there is an issue, the fidelity monitor will address it with the facilitator and
observe again. If the issue continues it would be addressed with the program director since the
main facilitator is the designated mental health clinician authority (DMHCA).
This reviewer met with the program director on Monday, February 24, 2020. It was suggested a
formal policy and procedure be created regarding internal fidelity monitoring for the DMHCA given
the DMHCA's subordinates are conducting internal fidelity for CYT groups. There is some concern
if there are significant issues the program director would need to address the issues with the
facilitator.
A review of twelve months of fidelity monitoring forms revealed there were no corrective actions
noted.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include evaluation
of skill in delivering the intervention/service.
The annual performance evaluations include a specific section for group facilitation and facilitators
are scored in three areas: group facilitation skills, group documentation, and knowledge of
evidenced based interventions. The designated mental health clinician authority (DMHCA) had an
annual performance evaluation completed.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 12 weeks for this type of service. Of the 32 youth in the
sample, 3% (1 of 32) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 12 contact hours for this type of service. Of the 32 youth in the
sample, 100% (32 of 32) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 91%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 5 youth
Moderate-High = 12 youth
High = 12 youth
Total Youth in Sample = 32 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 38%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 12 youth
Total Youth in Sample = 32 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 91% (29 of 32) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 38% (12 of 32) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 23 25 92%
Totals 88 100 88%
This SPEP report evaluates Cannabis Youth Treatment, an intervention delivered at Escambia
Boys Base.
43 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The targeted number of weeks for this specific intervention is 12 weeks; the targeted number of
hours for this specific intervention is 12 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 5 services (15 weeks; 45
hours).
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
For a time during the review period, the program used the start date for groups in the Evidence-
Based Services Module (EBS) at session three, thus reducing the documented number of weeks
served for included youth. The service was provided for the twelve (12) weeks and twelve (12)
hours, as designed by the developer, but was not documented appropriately in the EBS module.
This error has been identified and corrected.
The program earned 0 points for Amount of Service: Duration. Of the 32 total youth sampled, all
32 included dosage with end dates in the EBS Module. Of those youth, 1 received at least the
recommended weeks of service. Youth in the sample completed between 5 and 20 weeks of
service, with an average of 9 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 32 total youth
sampled, all 32 included dosage in the EBS Module. Of those youth, 32 received at least the
recommended hours of service. Youth in the sample completed between 14.5 and 29 hours of
service, with an average of 15 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Escambia Boys Base can optimize their SPEP Quality of Service Delivery score by ensuring
fidelity monitoring is completed every month for each facilitator.
Escambia Boys Base can optimize their SPEP Amount of Service score by ensuring that dosage
for all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

service_type: Cannabis Youth Treatment


summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Escambia Boys Base
AMIkids, Inc
(Contract Provider)
640 Roberts Road, Building 3780 Corry Station
Pensacola, Florida 32511
Primary Service: Impact of Crime
SPEP Review Date(s): January 2019-February 2020
Florida Department of Juvenile Justice
Report Date(s): 4/22/2020
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
Clinical case management staff, the activities coordinator, and direct care staff are trained to
conduct Impact of Crime (IOC). There have been three staff trained to conduct IOC during the
twelve month review period: activities coordinator, director of operations, and the previous clinical
case manager. All training documentation was provided by the program to be reviewed.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program maintains the appropriate manual for Impact of Crime (IOC). There are four sections
to the IOC manual. The manual includes all needed worksheets and facilitator forms to conduct
IOC.
The facility maintains a black binder with the title “Impact of Crime”. The binder was separated
into five sections. The first section contains the schedule of when groups are currently held. IOC
is currently held on Mondays and Wednesays from 12:00pm-1:00pm.
The second section contains all the attendance sheets separated by a blank sheet of green paper
per cohort. The attendance sheets are those created by the Department of Juvenile Justice and
include the program name, group sessions date, group session time, facilitator one and two. The
second section includes all of the lessons for IOC with checkboxes to indicate what lesson is
being held on the date. The third section has all youth names and DJJ ID numbers typed, a space
for the youth signature, and checkboxes to mark if homework was completed. The last section is
to indicate if a youth was dropped and a reason why, number of youth attending group, and if
there were any supportive lessons. The last section is for facilitators to sign and date.
The third section includes the Department of Juvenile Justice internal fidelity monitoring sheets for
the review period. The fourth section contains the internal Associated Marine Institute (AMI)
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
fidelity monitoring documents created to send to AMI headquarters on a monthly basis. These
reports include sessions per month, primary facilitator and training date, youth enrolled at the
beginning of the month and number of youth enrolled at the close of the month, along with an
explanation for any enrollment changes. The next section of the report contains information for
each indicator of Standardized Program Evaluation Protocol (SPEP), i.e., Facilitator Training,
Primary Service Manual, Observed adherence to Primary Service Manual, Facilitator Turnover,
Internal Fidelity, Corrective Actions, and Evaluation of Facilitator Skill. Each of these sections
have a scoring and comment section for each indicator.
The last section of the binder contains certifications for the facilitators. The section contained
education information and training certificates for IOC facilitators.
Impact of Crime is held as a closed group, meaning once the first lesson has ended no youth can
be added to the group. Group is scheduled to be held for one full hour twice a week for sixteen
weeks.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Impact of Crime group was observed on Wednesday, February 19, 2020 from 12:00pm to
1:00pm. Eleven youth were in attendance on this date. The youth were starting the lesson on
what can interfere with decision making. During the group four youth left due to having to take
their forklift certificaton test.
There were no behavioral issues during the group. The youth were all engaged in the group and
discussion. The facilitator consistently used motivational interviewing skills to engage the youth in
conversation. There was one staff member who participated in group and was helpful throughout
group. The staff member also co-acted with one of the youth during role plays. The facilitator did
an excellent job affirming, reflecting, and summarizing throughout group to all of the youth. One
upgrade would be to provide specific feedback on role-plays and allow the youth to provide
feedback as well.
The group was held for the full hour and all youth participated. The four youth who left will need to
complete the rest of the lesson they missed due to missing the majority of the group time.
Basic Score: 0 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program's protocol is for three staff members to be trained in the Impact of Crime (IOC)
curriculum to be able to facilitate in the event a facilitator would leave or be unable to continue
with group for an extended period of time.
The previous facilitator left employment in September 2019, during a cohort. The IOC trained shift
supervisor took over the group. This staff member was promoted to director of operations during
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
the review period. In October, the activities coordinator began co-facilitating with the director of
operations.
There was one gap in service, but it was not due to facilitator turnover. Group was not held on a
regular basis due to only one facilitator being available to conduct group. During this review period
there was gap in service between January 2019 and March 2019. There was no explanation
provided as to why there was a gap during this time. The current designated mental health clinical
authority (DMHCA) was not in this position at that time and was unaware as to why there was a
gap. The previous facilitator is no longer employed at the program as well.
One cohort ran from March 25, 2019 to July 8, 2019. The previous clinical case manager (CCM)
conducted the full cohort. Twelve youth began the group and ten youth ended the group. Two
youth were discharged from the program on June 19, 2019 and May 13, 2019. These youth did
not complete IOC group prior to being discharged from the program. These groups were held
anywhere from an hour and fifteen minutes to two hours. This specific group only completed
twenty-three hours out of the required thirty-two hours.
The second cohort began on August 5, 2019 through November 20, 2019. The previous clinical
case manager (CCM) began the group and ran it until the CCM left employment in September
2019. A seasoned facilitator took over the group and co-facilitated with a newly trained staff.
Twelve youth began the group and nine youth completed the group. Three youth were released
from the program before completing IOC. Two attendance sheets had the same date, August 19,
2019. During the month of October the groups were held more consistently. The group completed
thirty-three hours.
A third cohort was started on November 25, 2019. There are twelve youth in the group and are
still continuing to meet at this time.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
All internal fidelity monitoring forms were located in the Standardized Program Evaluation Protocol
binder for review.
Upon review the fidelity monitoring forms eleven of twelve internal fidelity monitoring forms were
reviewed. The form for June 2019 was conducted by the facilitator of the group and will not count
as an internal fidelity monitoring. The director of operations typically conducts internal fidelity
monitoring for the program.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The director of operations conducts fidelity monitoring for the Impact of Crime groups. If there is
an issue identified during group the director of operations will address it with the faciltiator. The
designated mental health clinician authority (DMHCA) will also address any issues during weekly
supervisions with the facilitator. If the issue persists further action will be taken.
Upon review of the internal fidelity monitoring forms there were no corrective actions noted.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include evaluation
of skill in delivering the intervention/service.
The annual performance evaluations include a specific section for group facilitation and facilitators
are scored in three areas: group facilitation skills, group documentation, and knowledge of
evidenced based interventions.
The current facilitator of Impact of Crime has not been employed for a full year at the time of this
review and had not received an annual review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service - Duration
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates the target duration of 16 weeks for this type of service. Of the 42 youth in the
sample, 29% (12 of 42) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 24 contact hours for this type of service. Of the 42 youth in the
sample, 69% (29 of 42) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 20 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 80%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 83%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 4 youth
Moderate-High = 14 youth
High = 17 youth
Total Youth in Sample = 42 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 40%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 17 youth
Total Youth in Sample = 43 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 84% (36 of 43) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 40% (17 of 43) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 2 10 20%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 20 25 80%
Totals 58 85 68%
This SPEP report evaluates Impact of Crime, an intervention delivered at Escambia Boys Base.
43 youth completed the program during the review period. Of those youth, 42 were included in
the sample.
The program scored Medium for Quality of Service Delivery.
The program earned 2 points for Amount of Service: Duration. Of the 42 total youth sampled, only
24 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 12
received at least the recommended weeks of service. Youth in the sample completed between 7
and 16 weeks of service, with an average of 14 weeks.
The program earned 6 points for Amount of Service: Contact Hours. Of the 42 total youth
sampled, only 34 included dosage in the EBS Module. Of those youth with correct dosage, 29
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
received at least the recommended hours of service. Youth in the sample completed between 13
and 32 hours of service, with an average of 26 hours.
The program was awarded 20 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 74%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Escambia Boys Base can optimize their SPEP Quality of Service Delivery score by ensuring
facilitator turnover does not impact groups, and by ensuring fidelity monitoring is completed every
month for each facilitator.
Escambia Boys Base can optimize their SPEP Amount of Service score by ensuring that dosage
for all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10