summary: SPEP - FY17-18

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Brevard Group Treatment Home
Aspire Health Partners
(Contract Provider)
3905 Grissom Parkway
Cocoa, Florida 32926
Primary Service: Impact of Crime
SPEP Review Date(s): June 12‐14, 2018
Florida Department of Juvenile Justice
Report Date(s): June 14, 2018
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Brevard Group Treatment Home MQI Program Code: 571
Provider Name: Aspire Health Services Contract Number: R2109
Location: Brevard County / Circuit: 18 Number of Beds: 30
Review Date(s): June 12-14, 2018 Lead Reviewer Code: 155
Persons Interviewed
Program Director Corporate QI/QA staff # Program Supervisors
DJJ Monitor # Case Managers # Youth
DHA or designee 4 # Clinical Staff # Other (listed by title):
DMHCA or designee # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 4 # Personnel Records
Fidelity Monitoring Documents Program Schedules 4 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts # Youth Records (Closed)
Staff Evaluations Table of Organization # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports # Health Records
Contract Scope of Services # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime. The program was awarded 15 points
because the Primary Service is identified as a Group 3 Service. The specific Sub-Component
Service Type identified is Social Skills Training. The Primary Service was identified as this type of
service as it focuses on developing social skills required for an individual to interact in a positive
way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program currently has four staff trained in the Impact of Crime (IOC) curriculum. Each staff
received formal training, in IOC, from a qualified trainer. Of the four staff, one is the program's
recreational therapist, one is the transition specialist, one is the program manager, and one is the
program's senior program/training manager. Of the four staff, three facilitated the group over the
last twelve months. The program maintains a binder containing all four primary services the
program utilizes. The binder contains each cohort's sign-in sheets, facilitator's certificates and
monthly fidelity monitoring for the last twelve months.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The service delivery manual for Impact of Crime (IOC) was reviewed. The program utilizes the
Impact of Crime Addressing the Harm, 2nd edition manual for their delivery of service, which was
published in 2011 by the Department's Residential Services. The manual includes the facilitator
manual, as well as the student handbook. The facilitator section includes Introduction, seven
chapters for the group sessions, and the Appendix. Each of the chapters are utilized in a group
session and includes the following sections: Objective, Words to Know, Definitions, Introduction,
Student Exercises, Facilitator Notes and Discussion Points. The student workbook includes the
following sections: Objective, Words to Know, Definitions, Introduction, and Student Exercises.
Each youth admitted to the program is scheduled to complete the IOC curriculum.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
This curriculum was not able to be observed; therefore, this indicator rates as non-applicable.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program facilitated four cohorts during the last twelve months, utilizing three trained staff. The
group is scheduled Wednesday and Fridays, each for a one hour session. Of the four cohorts,
three trained staff facilitated the groups. There has been no turnover during the last twelve
months. The program has four staff trained for this primary service. According to the clinical
manager if they had a facilitator leave, the program would utilize the additional trained staff to
continue the cohort. There were no gaps in service over the last twelve months.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program's process in conducting internal fidelity monitoring consist of a staff member, who is
trained in the curriculum, conducting the fidelity monitoring one time per month, per cohort. Three
staff, trained in the service delivery, facilitated the Impact of Crime (IOC) cohorts during the review
period. Of the twelve months, there were groups facilitated each month, with the exception of
October 2017. Of the eleven months, each facilitator who facilitated a group had a fidelity
monitoring, with the exception of two facilitators in one month. During the month of May 2018
there were two facilitators, and neither had a fidelity monitoring conducted. The program manager
and the senior program and training manager, both facilitated and conducted the fidelity
monitoring for this service delivery. Both staff were trained in the curriculum by a qualified trainer.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has an in-house process for corrective action pertaining to the primary service,
which consist of documenting any corrective action needed on the fidelity monitoring sheet.
During the last twelve months, there was no corrective action notated during any fidelity
monitorings. Through interview with the clinical manager, it was determined if corrective action is
required, the clinical manager would utilize the provider's corrective action form. This form
includes the date the corrective action was issued, the staff name, any prior disciplinary actions,
details of the current incident, current infraction, corrective action required, supervisor’s follow-up
actions and the employee's comments. Both the supervisor and employee are expected to sign
the corrective action form.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program's policy is to conduct an annual performance evaluation on each staff member. Of
the four staff trained in this service delivery, one staff has only been employed for five months;
therefore, has not received his annual performance evaluation. This individual did have a 90-day
evaluation; however, it was received one day prior to him facilitating his first group. The additional
three staff received their performance evaluation and each contained specific information about
their delivery of Impact of Crime (IOC).
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 16 weeks for this type of service. Of the 7 youth in the
sample, 0% (0 of 7) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 7 youth in the
sample, 100% (7 of 7) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 1 youth
Moderate-High = 3 youth
High = 3 youth
Total Youth in Sample = 7 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 43%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 3 youth
Total Youth in Sample = 7 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (7 of 7) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 43% (3 of 7) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 25 25 100%
Totals 75 85 88%
This SPEP report evaluates Impact of Crime, an intervention delivered at Brevard Group
Treatment Home.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 7 total youth sampled, only
6 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 0
received at least the recommended weeks of service. Youth in the sample completed 12 weeks of
service.
The program earned 10 points for Amount of Service: Contact Hours. Of the 7 total youth
sampled, all 7 included dosage in the EBS Module. Of those youth, 7 received at least the
recommended hours of service. Youth in the sample completed between 27 and 36 hours of
service, with an average of 28 hours.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered just prior to the youths' admission.
RECOMMENDATION(S):
Brevard Group Treatment Home can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Brevard Group Treatment Home can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10


state: FL

facility_name: Brevard Juvenile Detention Center

summary: SPEP - FY17-18

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Brevard Group Treatment Home
Aspire Health Partners
(Contract Provider)
3905 Grissom Parkway
Cocoa, Florida 32926
Primary Service: Life Skills Training
SPEP Review Date(s): June 12‐14, 2018
Florida Department of Juvenile Justice
Report Date(s): June 15, 2018
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Brevard Group Treatment Home MQI Program Code: 571
Provider Name: Aspire Health Services Contract Number: R2109
Location: Brevard County / Circuit: 18 Number of Beds: 30
Review Date(s): June 12-14, 2018 Lead Reviewer Code: 155
Persons Interviewed
Program Director Corporate QI/QA staff # Program Supervisors
DJJ Monitor 1 # Case Managers # Youth
DHA or designee 1 # Clinical Staff # Other (listed by title):
DMHCA or designee # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 2 # Personnel Records
Fidelity Monitoring Documents Program Schedules 2 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts # Youth Records (Closed)
Staff Evaluations Table of Organization # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports # Health Records
Contract Scope of Services # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is LifeSkills Training. The program was awarded 15 points
because the Primary Service is identified as a Group 3 Service. The specific Sub-Component
Service Type identified is Social Skills Training. The Primary Service was identified as this type of
service as it focuses on developing social skills required for an individual to interact in a positive
way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program currently has one staff trained in the LifeSkills Training (LST) curriculum. The
facilitator received formal training in LST from a qualified trainer. The program's case manager is
the facilitator for this primary service. Only this staff facilitated the group over the last twelve
months. The program maintains a binder containing all four primary services the program utilizes.
The binder contains each cohort's sign-in sheets, facilitator's certificates and monthly fidelity
monitoring for the last twelve months.
Basic Score: 1 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The primary service manual for LifeSkills Training was reviewed. The program utilizes the
LifeSkills Training Promoting Health and Personal Development Teacher's Manual 1. The
facilitator's manual has fifteen sections which the facilitator adheres to during each of the fifteen
sessions. The manual contains additional sections to include an introduction, implementation
guidelines, background information for the teachers and references. The case manager indicated
the primary service has a student manual; however, when a staff member left, over a year ago,
the student manual was never returned. They program is not utilizing a student handbook, or the
materials from the student handbook for their cohorts. Each youth admitted to the program is
assessed to determine if they are appropriate for this primary service. All youth who are
diagnosed with a history of substance abuse are assigned to this primary service.
The program can earn 2 points if the manual specifically details the delivery of this primary
service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The primary service is delivered on Tuesday and Thursdays for one hour a session. The service
delivery was observed during the review process. The facilitator provided the observer with a copy
of the lesson being facilitated. The facilitator followed the lesson, while adding real life stories. The
facilitator started the group by having a youth assist in completing the sign-in sheet. The facilitator
began the lesson, while engaging the youth during the entire session. There were no specific
exercises during this session; however, the youth interacted with the facilitator consistently
throughout hour-long session.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program facilitated continues cohorts during the last twelve months, utilizing one trained staff.
The group is scheduled Tuesday and Thursdays, each for one hour session. Of the cohorts
reviewed for the last twelve months, the one trained staff facilitated each group. There has been
no turnover during the last twelve months. The program utilizes a behavior health tech, trained in
this primary service, from another program, under the same provider, Aspire. If there should be a
turnover in staff, the program would inquire as to utilizing the behavior health tech temporarily.
During the month of July 2017, there were no groups held, which was due to the facilitator’s
schedule.
The program can earn 2 points if there are no unnecessary gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The behavior health tech, from Oasis, who was trained by a qualified trainer in the primary
service, conducted the fidelity monitoring for this service delivery. Oasis is a program overseen by
the same provider, Aspire. The program's process in conducting internal fidelity monitoring
consists of a staff member, who is trained in the curriculum, conducting the fidelity monitoring one
time per month, per cohort. One staff, trained in the primary service, facilitated the LST cohorts
during the review period. Of the twelve months, there were groups facilitated each month, with the
exception of July. Of those eleven months, a fidelity monitoring was completed each month with
the exception of May 2018, when there were four groups held. The program's policy has been to
conduct fidelity monitoring one time per month, per cohort, rather than one fidelity monitoring per
month, per facilitator, which would suffice for this intervention as there was only one facilitator.
This requirement was discussed with the clinical manager.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has an in-house process for corrective action, pertaining to the primary service,
which consists of documenting any corrective action needed on the fidelity monitoring sheet.
During the last twelve months, there was no corrective action noted during any fidelity
monitorings. Through an interview with the clinical manager, it was determined if there is a
corrective action required, the clinical manager would utilize the provider's corrective action form.
This form includes the date the corrective action was issued, the staff name, any prior disciplinary
actions, details of the current incident, current infraction, corrective action required, supervisor’s
follow-up actions and the employee's comments. Both the supervisor and employee are expected
to sign the corrective action form.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program's policy is to conduct an annual performance evaluation on each staff member. The
one staff trained in this primary service received their performance evaluation, which contained
specific information about their delivery of LST.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 16 weeks for this type of service. Of the 14 youth in the
sample, 0% (0 of 14) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 24 contact hours for this type of service. Of the 14 youth in the
sample, 0% (0 of 14) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 0 youth
Moderate-High = 5 youth
High = 9 youth
Total Youth in Sample = 14 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 64%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 9 youth
Total Youth in Sample = 14 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (14 of 14) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 64% (9 of 14) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 25 25 100%
Totals 55 85 65%
This SPEP report evaluates LifeSkills Training, an intervention delivered at Brevard Group
Treatment Home.
The program scored Medium for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 14 total youth sampled, all
14 included dosage with end dates in the EBS Module. Of those youth, 0 received at least the
recommended weeks of service. Youth in the sample completed between 7 and 14 weeks of
service, with an average of 11 weeks.
The program earned 0 points for Amount of Service: Contact Hours. Of the 14 total youth
sampled,all 14 included dosage in the EBS Module. Of those youth, 0 received at least the
recommended hours of service. Youth in the sample completed between 15 and 16 hours of
service, with an average of 15 hours.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered just prior to the youths' admission.
RECOMMENDATION(S):
Brevard Group Treatment Home can optimize their SPEP Quality of Service Delivery score by
ensuring the manual for this service is present and specifically details the delivery of this primary
service, by ensuring there are no unnecessary gaps in service, and by ensuring fidelity monitoring
is completed every month for each facilitator.
Brevard Group Treatment Home can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

source: DJJ

report_type: SPEP

program_name_raw: Brevard Group Treatment Home

program_status_note: Closed

service_type: LifeSkills Training


summary: SPEP - FY17-18

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Brevard Group Treatment Home
Aspire Health Partners
(Contract Provider)
3905 Grissom Parkway
Cocoa, Florida 32926
Primary Service: Moral Reconation Therapy
SPEP Review Date(s): June 12‐14, 2018
Florida Department of Juvenile Justice
Report Date(s): June 14, 2018
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Brevard Group Treatment Home MQI Program Code: 571
Provider Name: Aspire Health Services Contract Number: R2109
Location: Brevard County / Circuit: 18 Number of Beds: 30
Review Date(s): June 12-14, 2018 Lead Reviewer Code: 155
Persons Interviewed
Program Director Corporate QI/QA staff # Program Supervisors
DJJ Monitor 1 # Case Managers # Youth
DHA or designee 1 # Clinical Staff # Other (listed by title):
DMHCA or designee # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 2 # Personnel Records
Fidelity Monitoring Documents Program Schedules 2 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts # Youth Records (Closed)
Staff Evaluations Table of Organization # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports # Health Records
Contract Scope of Services # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Moral Reconation Therapy. The program was awarded
30 points because the Primary Service is identified as a Group 5 Service. The specific Sub-
Component Service Type identified is Cognitive Behavioral Therapy. The Primary Service was
identified as this type of service as it is intended to correct faulty cognitions and perceptions and
provides skills individuals can use to monitor thought patterns and correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program currently has two staff trained in the Moral Reconation Therapy (MRT) curriculum.
Each staff received formal training in MRT from a qualified trainer. Of the two staff, one is the
program's clinical manager and the other is the program's case manager. Of the two staff, both
facilitated the group over the last twelve months. The program maintains a binder containing all
four primary services the program utilizes. The binder contains each cohort's sign-in sheets,
facilitator's certificates and monthly fidelity monitoring for the last twelve months.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The primary service manual for Moral Reconation Therapy (MRT) was reviewed. The program
utilizes the Juvenile MRT How to Escape Your Prison youth workbook for the youth and the Moral
Reconation Therapy Facilitator's Handbook. The facilitator's handbook includes the sixteen steps,
which the facilitator adheres to during each of the sixteen sessions. The youth workbook includes
the sixteen steps and each section includes youth activities and instructions for each step. Each
youth admitted to the program is scheduled to complete the MRT curriculum.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The primary service is delivered on Monday and Fridays for one hour a session. The primary
service was observed during the review process. The facilitator provided the observer with a copy
of the lesson being facilitated. The facilitator followed the lesson, while adding real life stories. The
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
facilitator recapped the last few sessions, then began reviewing the current lesson. The facilitator
kept the youth engaged throughout the hour-long session.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program facilitated continuous cohorts during the last twelve months, utilizing two trained
staff. The group is scheduled Monday and Fridays, each for a one hour session. Of the cohorts
reviewed for the last twelve months, two trained staff facilitated the groups. There has been no
turnover during the last twelve months. The program has two staff trained for this service delivery.
According to the clinical manager, if they had a facilitator leave, the program would utilize the
additional trained staff to continue the cohort. During the last twelve months the clinical director
was the primary facilitator, while the case manager only filled in when the clinical manager was
not available. There were no gaps in service over the last twelve months.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The clinical manager and case manager both facilitated the groups; however, only one staff
conducted the fidelity monitoring for this primary service. Both staff were trained in the curriculum
by a qualified trainer. The program's process in conducting internal fidelity monitoring consists of
a staff member who is trained in the curriculum conducting the fidelity monitoring one time per
month, per cohort. Two staff, trained in the service delivery facilitated the Moral Reconation
Therapy (MRT) cohorts during the twelve month review period. Of the twelve months, there were
groups facilitated each month. A fidelity monitoring was completed each month a group was
facilitated; however, there were two facilitators during the months of March and April 2018. The
program's policy has been to conduct fidelity monitoring one time per month, per cohort, rather
than one fidelity monitoring per month, per facilitator. Of these two months, only one facilitator had
fidelity monitoring conducted, while the other did not. The facilitator who did not have the fidelity
monitoring during their groups, was covering for the facilitator who regularly ran the cohorts, and
only facilitated six groups for the entire twelve months. The requirement to conduct the fidelity
monitoring for each facilitator, per month, was discussed with the clinical monitor while on site for
the review process.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has an in-house process for corrective action, pertaining to the primary service,
which consists of documenting any corrective action needed on the fidelity monitoring sheet.
During the last twelve months, there was no corrective action noted during any fidelity
monitorings. Through interview with the clinical manager, it was determined if there is a corrective
action required, the clinical manager would utilize the provider's corrective action form. This form
includes the date the corrective action was issued, the staff name, any prior disciplinary actions,
details of the current incident, current infraction, corrective action required, supervisor’s follow-up
actions and the employee's comments. Both the supervisor and employee are expected to sign
the corrective action form.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program's policy is to conduct an annual performance evaluation on each staff member. Of
the two staff trained in this service delivery, both received their performance evaluation and each
contained specific information about their delivery of Moral Reconation Therapy (MRT).
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 15 weeks for this type of service. Of the 18 youth in the
sample, 61% (11 of 18) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 45 contact hours for this type of service. Of the 18 youth in the
sample, 0% (0 of 18) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 89%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 1 youth
Moderate-High = 6 youth
High = 9 youth
Total Youth in Sample = 18 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 50%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 9 youth
Total Youth in Sample = 18 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 89% (16 of 18) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 50% (9 of 18) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 23 25 92%
Totals 84 100 84%
This SPEP report evaluates Moral Reconation Therapy, an intervention delivered at Brevard
Group Treatment Home.
The program scored High for Quality of Service Delivery.
The program earned 6 points for Amount of Service: Duration. Of the 18 total youth sampled, only
16 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 11
received at least the recommended weeks of service. Youth in the sample completed between 9
and 39 weeks of service, with an average of 27 weeks.
The program earned 0 points for Amount of Service: Contact Hours. Of the 18 total youth
sampled, all 18 included dosage in the EBS Module. Of those youth, 0 received at least the
recommended hours of service. Youth in the sample completed between 3 and 29 hours of
service, with an average of 19 hours.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered just prior to the youths' admission.
RECOMMENDATION(S):
Brevard Group Treatment Home can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Brevard Group Treatment Home can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

service_type: Moral Reconation Therapy


summary: SPEP - FY18-19

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Brevard Group Treatment Home
Aspire Health Partners
(Contract Provider)
3905 Grissom Parkway
Cocoa, Florida 32926
Primary Service: Impact of Crime
SPEP Review Date(s): February 5 ‐ February 8, 2019
Florida Department of Juvenile Justice
Report Date(s): February 11, 2019
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
 Service Type: service type points are awarded based on the group category that the
service falls within
 Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
 Amount of Service: or “dosage” refers to the amount and duration of services that were
received
 Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime. The program was awarded 15 points
because the Primary Service is identified as a Group 3 Service. The specific Sub-Component
Service Type identified is Social Skills Training. The Primary Service was identified as this type of
service as it focuses on developing social skills required for an individual to interact in a positive
way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program’s documentation reflected the program had a training protocol and there were three
staff who received formal training from a qualified trainer to become a group facilitator and/or
conduct internal fidelity monitoring for Impact of Crime within the twelve month review period. At
the time of the review, the program had one transition coordinator and one recreation therapist
trained on April 6, 2018, and one program manager trained on November 11, 2016.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program’s facilitator utilizes the Impact of Crime, Addressing the Harm curriculum. The
program has a manual for the delivery of services and a workbook for the students to use. A copy
of the manual and workbook were provided for the team to reference during the review. The
curriculum provides a script and instructions for seven chapters which requires a minimum of
thirty-two contact hours to facilitate. The curriculum includes implementation guidelines, identifies
the order the lessons are to be delivered, and provides detailed instructions for the delivery of
each lesson.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
An observation of and Impact of Crime group was conducted on February 5, 2019. The facilitator
was knowledgeable of the curriculum, was comfortable facilitating group and had good classroom
management skills. Youth participated in the discussion and group activities. The group was held
in the cafeteria, which was large enough to conduct group; however, there was a lot of distracting
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
noise coming from the kitchen. The program director stated group is usually held in the
conference room but the Monitoring and Quality Improvement team was utilizing the room during
the review. The observer's recommendation is for the facilitator to use chart paper and markers
during group.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The reviewed documentation, and an interview with the Designated Mental Health Clinician
Authority (DMHCA), confirmed there has been no turnover which has adversely affected services
to the youth, nor have there been any gaps in service during the review period.
The SPEP review included twelve months of records, which indicated three cohorts of Impact of
Crime (IOC) were completed and one group was in progress. The first cohort of IOC started on
May 23, 2018 and ended on July 18, 2018. Nine youth completed group. The group ended after
twenty-one sessions, not the required thirty-two sessions. The second cohort started on July 17,
2018 and was completed on September 13, 2018 with ten youth completing group. The group
ended after eighteen sessions. The third cohort started on September 19, 2018 and was
completed on January 1, 2019 with ten youth completing group. The group ended after nineteen
sessions. Youth participate in reparation activities such as community service projects which
include writing letters to victims. The cohort in progress started on January 29, 2019 with ten
youth participating. Youth are chosen to participate in IOC based on their needs as assessed by
the Residential Positive Achievement Change Tool (R-PACT) and a Mental Health/Substance
Abuse evaluation.
The program’s activity schedule verified the IOC groups were provided; however, group sign-in
sheets indicated the program did not meet the thirty-two sessions required by the curriculum
developer. The deficiencies were discussed with the department during the SPEP call on January
3, 2019 and technical assistance was provided on January 11, 2019. The program implemented
internal corrective action to ensure future restorative justice groups comply with the SPEP
guidelines.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Internal fidelity monitoring reports were review for the review period. Documentation validated
staff members who performed internal fidelity monitoring were trained by a qualified trainer. The
internal fidelity monitoring reports indicated facilitators were monitored consistently during the
review period for the primary service of Impact of Crime.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
Documentation provided verified the program has a process for the application of corrective
actions based on the results of the fidelity monitoring conducted. An interview with the Designated
Mental Health Clinician Authority confirmed the program had any applicable issues requiring
corrective actions. The completed Impact of Crime groups did not meet the thirty-two session
SPEP requirement. Technical assistance was provided by the department and the program
implemented internal corrective action to ensure future restorative justice groups comply with the
SPEP guidelines.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of two performance evaluations of the staff who facilitated Impact of Crime indicated
both facilitators were acknowledged for their understanding of the program's treatment model and
delinquency intervention.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 16 weeks for this type of service. Of the 28 youth in the
sample, 0% (0 of 28) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 24 contact hours for this type of service. Of the 28 youth in the
sample, 96% (27 of 28) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 2 youth
Moderate-High = 13 youth
High = 13 youth
Total Youth in Sample = 28 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 46%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 13 youth
Total Youth in Sample = 28 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (28 of 28) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 46% (13 of 28) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 25 25 100%
Totals 73 85 86%
This SPEP report evaluates Impact of Crime, an intervention delivered at Brevard Group
Treatment Home.
35 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 28 total youth sampled, all
28 included dosage with end dates in the EBS Module. Of those youth, 0 received at least the
recommended weeks of service. Youth in the sample completed between 8 and 15 weeks of
service, with an average of 12 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 28 total youth
sampled, all 28 included dosage in the EBS Module. Of those youth, 27 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
recommended hours of service. Youth in the sample completed 23 and 29 hours of service, with
an average of 25 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Impact of Crime is an intervention suitable for all youth. The Department recommends providing
this service to every youth in the program.
Brevard Group Treatment Home can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Brevard Group Treatment Home can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

service_type: Impact of Crime


summary: SPEP - FY18-19

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Brevard Group Treatment Home
Aspire Health Partners
(Contract Provider)
3905 Grissom Parkway
Cocoa, Florida 32926
Primary Service: Botvin LifeSkills Training
SPEP Review Date(s): February 5 ‐ February 8, 2019
Florida Department of Juvenile Justice
Report Date(s): February 11, 2019
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
 Service Type: service type points are awarded based on the group category that the
service falls within
 Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
 Amount of Service: or “dosage” refers to the amount and duration of services that were
received
 Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is LifeSkills Training. The program was awarded 15 points
because the Primary Service is identified as a Group 3 Service. The specific Sub-Component
Service Type identified is Social Skills Training. The Primary Service was identified as this type of
service as it focuses on developing social skills required for an individual to interact in a positive
way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program’s documentation reflected the program had a training protocol and there were two
staff who received formal training from a qualified trainer to become group facilitators and/or
conduct internal fidelity monitoring for LifeSkills Training during the review period. Both staff were
trained on August 5, 2011.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program used the Botvin LifeSkills Training (LST) middle school curriculum. LST is a
research-validated substance abuse prevention program proven to reduce the risks of alcohol,
tobacco, drug abuse, and violence by targeting the major social and psychological factors that
promote the initiation of substance use and other risky behaviors. This comprehensive program
provides adolescents and young teens with the confidence and skills necessary to successfully
handle challenging situations.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
An observation of LifeSkills Training (LST) could not be conducted, therefore it is rated as not
applicable. LST group was not in progress at the time of the review because the program did not
have enough qualified youth to start a group.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The reviewed documentation and an interview with the Designated Mental Health Clinician
Authority (DMHCA) confirmed there were gaps in service from July 12, 2018 to September 25,
2018 and from November 20, 2018 till February 8, 2019. The DMHCA stated there were not
enough eligible youth needing the service during the gap periods. There was no facilitator
turnover during the review period.
The SPEP review encompasses twelve months of records and examinations. The program used
the LifeSkills Training (LST) middle school curriculum. The program conducted two cohorts of LST
during the review period. The first cohort started on May 22, 2018 and was completed on July 5,
2018. Nine youth started group and seven youth successfully completed. The second cohort
started on September 25, 2018 and was completed on November 20, 2018 with eight youth
successfully completing group. Youth are chosen to participate in LST based on their needs as
assessed by the Residential Positive Achievement Change Tool (R-PACT) and a Mental
Health/Substance Abuse evaluation.
The program can earn 2 points if there are no unnecessary gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Twelve months of internal fidelity monitoring reports were reviewed. Documentation validated staff
members who performed internal fidelity monitoring were trained by a qualified trainer. The
internal fidelity monitoring reports indicated the facilitator was monitored each month groups were
held.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
Documentation reviewed verified the program has a process for the application of corrective
actions based on the results of the fidelity monitoring conducted. An interview with the Designated
Mental Health Clinician Authority confirmed the program has not had any applicable issues
requiring corrective actions. The completed internal fidelity monitoring forms recorded the
observer’s feedback and recommendations, and ideas for improvement in the delivery of the
primary service LifeSkills Training.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of one performance evaluation of the staff who facilitated LifeSkills Training during the
review period indicated the facilitator was acknowledged for understanding of the program's
treatment model and delinquency intervention.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 15 weeks for this type of service. Of the 35 youth in the
sample, 9% (3 of 35) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 15 contact hours for this type of service. Of the 35 youth in the
sample, 60% (21 of 35) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 97%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 2youth
Moderate-High = 15youth
High = 17youth
Total Youth in Sample = 35youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 49%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 17youth
Total Youth in Sample = 35youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 97% (34 of 35) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 49% (17 of 35) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 25 25 100%
Totals 71 85 84%
This SPEP report evaluates LifeSkills Training, an intervention delivered at Brevard Group
Treatment Home.
35 youth completed the program during the review period. Of those youth, none had previously
received this intervention in another setting.
The program scored High for Quality of Service Delivery.
The targeted number of weeks for this specific intervention is 15 weeks; the targeted number of
hours for this specific intervention is 15 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 3 services (16 weeks; 24
hours).
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 0 points for Amount of Service: Duration. Of the 35 total youth sampled, only
21 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 3
received at least the recommended weeks of service. Youth in the sample completed between 8
and 21 weeks of service, with an average of 9 weeks.
The program earned 6 points for Amount of Service: Contact Hours. Of the 35 total youth
sampled, only 21 included dosage in the EBS Module. Of those youth with correct dosage, 21
received at least the recommended hours of service. Youth in the sample completed between 13
and 29 hours of service, with an average of 16 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Brevard Group Treatment Home can optimize their SPEP Quality of Service Delivery score by
ensuring there are no unnecessary gaps in service.
Brevard Group Treatment Home can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9


summary: SPEP - FY18-19

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Brevard Group Treatment Home
Aspire Health Partners
(Contract Provider)
3905 Grissom Parkway
Cocoa, Florida 32926
Primary Service: Moral Reconation Therapy
SPEP Review Date(s): February 5 ‐ February 8, 2019
Florida Department of Juvenile Justice
Report Date(s): February 11, 2019
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
 Service Type: service type points are awarded based on the group category that the
service falls within
 Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
 Amount of Service: or “dosage” refers to the amount and duration of services that were
received
 Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Moral Reconation Therapy. The program was awarded
30 points because the Primary Service is identified as a Group 5 Service. The specific Sub-
Component Service Type identified is Cognitive Behavioral Therapy. The Primary Service was
identified as this type of service as it is intended to correct faulty cognitions and perceptions and
provides skills individuals can use to monitor thought patterns and correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program’s documentation reflected the program had a training protocol and there were two
staff who received formal training from a qualified trainer to become group facilitators and/or
conduct internal fidelity monitoring for Moral Reconation Therapy within the last twelve months.
The Designated Mental Health Clinician Authority was trained on April 3, 2015, and the case
manager supervisor was trained on August 8, 2013.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
Moral Reconation Therapy (MRT) is a systematic treatment approach that seeks to decrease
recidivism, or the tendency of a convicted criminal to re-offend, among juvenile and adult
offenders by increasing moral reasoning. A cognitive-behavioral type of treatment approach, MRT
combines elements from a variety of psychological models to address clients’ ego, moral, social,
and positive behavioral growth, and research has shown that this type of therapy can increase
moral reasoning in adult drug and alcohol as well as juvenile offenders. In addiction recovery
treatment, MRT takes the form of individual and group counseling, and uses prescribed
homework assignments and structured group exercises to address seven basic treatment issues.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
An observation of Moral Reconation Therapy (MRT) could not be conducted therefore it is rated
as not applicable. The program is reviewing for a possible change in service delivery.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The reviewed documentation and interview with the Designated Mental Health Clinician Authority
confirmed there were no gaps in service or facilitator turnover during the review period. The
SPEP review encompasses twelve months of records and examinations. The program used Moral
Reconation Therapy (MRT) curriculum. The program conducted three cohorts of MRT during the
review period. The first cohort started on May 25, 2018 and was completed on July 30, 2018 with
ten youth completing group. The second cohort started on July 16, 2018, 2018 and was
completed on October 8, 2018 with ten youth completing group. The third cohort started on
October 8, 2018 and ended on January 14, 2019 with nine youth completing group. Youth are
chosen to participate in MRT based on their needs as assessed by the Residential Positive
Achievement Change Tool (R-PACT) and a Mental Health/Substance Abuse evaluation.
The program’s activity schedule verified the MRT groups were provided; however, group sign-in
sheets indicated the program did not meet the forty-five hours required by the SPEP guidelines.
The deficiencies were discussed with the department during the SPEP call on January 3, 2019.
The program implemented internal corrective action to ensure future groups comply with the
SPEP guidelines. The program is reviewing for a possible change in service delivery.
BasicScore: 2Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Internal fidelity monitoring reports for the last twelve months were reviewed. Documentation
validated staff members who performed internal fidelity monitoring were trained by a qualified
trainer. The internal fidelity monitoring reports indicated the facilitator was monitored on a
consistent basis during the review period.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
Documentation reviewed verified the program has a process for the application of corrective
actions based on the results of the fidelity monitoring conducted. An interview with the Designated
Mental Health Clinician Authority confirmed the program had applicable issues requiring
corrective actions. The completed Moral Reconation Therapy groups did not meet the forty-five
hour SPEP requirement. Technical assistance was provided by the department and the program
implemented internal corrective action to ensure future mental health groups comply with the
SPEP guidelines.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of one performance evaluation of the staff who facilitated Moral Reconation Therapy
since the last review period indicated the facilitator was acknowledged for understanding of the
program's treatment model and delinquency intervention.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 15 weeks for this type of service. Of the 35 youth in the
sample, 9% (3 of 35) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 45 contact hours for this type of service. Of the 35 youth in the
sample, 0% (0 of 35) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 97%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 2 youth
Moderate-High = 15 youth
High = 17 youth
Total Youth in Sample = 35 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 49%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 17 youth
Total Youth in Sample = 35 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 97% (34 of 35) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 49% (17 of 35) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 25 25 100%
Totals 80 100 80%
This SPEP report evaluates Moral Reconation Therapy, an intervention delivered at Brevard
Group Treatment Home.
35 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 35 total youth sampled, only
30 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 3
received at least the recommended weeks of service. Youth in the sample completed between 10
and 54 weeks of service, with an average of 14 weeks.
The program earned 0 points for Amount of Service: Contact Hours. Of the 35 total youth
sampled, all 35 included dosage in the EBS Module. Of those youth with correct dosage, 0
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
received at least the recommended hours of service. Youth in the sample completed between 3
and 29 hours of service, with an average of 19 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Brevard Group Treatment Home can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Brevard Group Treatment Home can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9