summary: PREA - Year 1 Cycle 4

raw_content:
Program: Brevard Regional Juvenile Detetion Center
Report Type: PREA
Fiscal Year:
Status:
PDF URL: https://www.djj.state.fl.us/content/download/766181/file/Florida_Brevard Regional Juvenile Detetion Center_year1cycle4.pdf?version=1

cycle: Year 1 Cycle 4


state: FL

facility_name: Brevard Juvenile Detention Center

summary: SPEP - FY15-16

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Brevard Group Treatment Home
Aspire Health Partners, Inc.
(Contract Provider)
3905 Grissom Parkway
Cocoa, Florida 32926
Primary Service: Impact of Crime
SPEP Review Date(s): May 10‐13, 2016
Florida Department of Juvenile Justice
Report Date(s): 5/1/2017
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Brevard Group Treatment Home QI Program Code: 571
Provider Name: Aspire Health Partners, Inc Contract Number: R2109
Location: Brevard County / Circuit: 18 Number of Beds: 30
Review Date(s): May 10-13, 2016 Lead Reviewer Code: 77
Persons Interviewed
Program Director 1 # Case Managers 4 # Other (listed by title): Chief
DJJ Monitor 1 # Clinical Staff Development Officer, Director of
DHA or designee 1 # Healthcare Staff Community Relations, Corporate
DMHCA or designee # Program Supervisors Regional Nurse, Corporate Food
Corporate QI/QA staff 5 # Youth Service Manager
Documents Reviewed
Written Protocol/Manual Logbooks 10 # Personnel Records
Fidelity Monitoring Documents Program Schedules 10 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 5 # Youth Records (Closed)
Staff Evaluations Table of Organization 5 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports 5 # Health Records
Contract Scope of Services 5 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime. The program was awarded 15 points
because the Primary Service is identified as a Group 3 Service. The specific Sub-Component
Service Type identified is Social Skills Training. The Primary Service was identified as this type of
service as it focuses on developing social skills required for an individual to interact in a positive
way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 6 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
One staff member received formal training by a qualified trainer to become a group facilitator in
Impact of Crime in August 2013. Another staff member received formal training by a qualified
trainer in April 2015. A supervisory staff member received formal training by a qualified trainer in
November 2015. Only two staff members have conducted IOC in the last ten months. The
supervisory staff has completed fidelity monitoring of IOC in the last ten months.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program has the manual for delivery for Impact of Crime (IOC) and provided a copy of the
manual to the reviewer. The curriculum was revised and updated in 2009. The revised Impact of
Crime is a victim impact/restorative justice curriculum consisting of seven chapters taking
approximately twenty-four sessions to facilitate. The manual includes a script, implementation
guidelines, identifies the order the lessons should be delivered and provides detailed instructions
for the delivery of each lesson.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
This curriculum was not able to be observed; therefore this indicator rates as non-applicable.
d. Facilitator Turnover Basic Score: 1 Point(s)
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The reviewed documentation and interviews with staff revealed there has been no turnover in the
middle of cohorts. The program completed one cycle of IOC inJuly 2015, and two additional
cycles by March 2016. The program began a fourth cycle which was ongoing at the time of the
review. There was a gap in service delivery between the two completed cycles from November
18, 2015 until January 4, 2016.
The program can earn 2 points if there are no gaps in service of more than two weeks.
At the time of the review, the program did not meet criteria to earn a score of 2.
BasicScore: 1Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The team reviewed ten months of fidelity monitoring reports. The program did not have a specific
monitoring protocol until December 31, 2015. The program had sporadic fidelity monitoring and
one fidelity monitoring was completed in June 2015 and one fidelity monitoring was completed in
December 2015. On December 31, 2015, the program revised its protocol to include monthly
fidelity monitoring in IOC using a form specifically designed for the intervention. Documentation
was reviewed that confirmed fidelity monitoring was completed the months of January and
February 2016 by a trained facilitator in IOC using a form specifically designed for IOC. There
was constructive feedback found on each of the fidelity monitoring tools. There were no monthly
fidelity monitoring completed since February 2016.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
BasicScore: 1Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The fidelity monitoring in June and December 2015 did not describe a need for corrective action.
The fidelity monitoring in 2016 provided constructive feedback to the facilitator delivering the
intervention, but none of the feedback required further corrective action. There was no corrective
action documented for a particular facilitator during the review period.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
There are three trained facilitators in the IOC curriculum. One staff has been delivering the
curriculum for several years and was due to receive an annual evaluation. Her evaluation did not
specifically rate her performance in the delivery of the IOC curriculum. The new facilitator received
an annual evaluation last year, but had not been delivering the service for a complete cohort and
was not evaluated on delivery of the intervention. The supervisory staff trained in IOC has not
been with the agency long enough to require an annual evaluation. The ninety-day evaluation was
completed prior to his receiving and delivering the IOC curriculum.
The program can earn 1 point if facilitators are evaluated on their skill in delivering this primary
service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 16 weeks for this type of service. Of the 11 youth in the
sample, 0% (0 of 11) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 11 youth in the
sample, 100% (11 of 11) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 0 youth
Moderate-High = 2 youth
High = 9 youth
Total Youth in Sample = 11 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 82%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 9 youth
Total Youth in Sample = 11 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (11 of 11) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 82% (9 of 11) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 25 25 100%
Totals 65 85 76%
This SPEP report evaluates Impact of Crime, an intervention delivered at Brevard Group
Treatment Home.
The program scored Medium for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 11 total youth sampled,
none received at least the recommended weeks of service. Youth in the sample completed
between 12 and 13 weeks of service.
The program earned 10 points for Amount of Service: Contact Hours. Of the 11 total youth
sampled, all received at least the recommended hours of service. Youth in the sample completed
24 hours of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Brevard Group Treatment Home can optimize their SPEP Quality of Service Delivery score by
minimizing gaps in service, conducting fidelity monitoring monthly, and including an assessment
of facilitator skill in delivery of IOC in performance evaluations.
Brevard Group Treatment Home can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

source: DJJ

report_type: SPEP

program_name_raw: Brevard Group Treatment Home

program_status_note: Closed

service_type: Impact of Crime


summary: SPEP - FY15-16

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Brevard Group Treatment Home
Aspire Health Partners, Inc.
(Contract Provider)
3905 Grissom Parkway
Cocoa, Florida 32926
Primary Service: LifeSkills Training
SPEP Review Date(s): May 10‐13, 2016
Florida Department of Juvenile Justice
Report Date(s): 5/1/2017
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Brevard Group Treatment Home QI Program Code: 571
Provider Name: Aspire Health Partners, Inc Contract Number: R2109
Location: Brevard County / Circuit: 18 Number of Beds: 30
Review Date(s): May 10-13, 2016 Lead Reviewer Code: 77
Persons Interviewed
Program Director 1 # Case Managers 4 # Other (listed by title): Chief
DJJ Monitor 1 # Clinical Staff Development Officer, Director of
DHA or designee 1 # Healthcare Staff Community Relations, Corporate
DMHCA or designee # Program Supervisors Regional Nurse, Corporate Food
Corporate QI/QA staff 5 # Youth Service Manager
Documents Reviewed
Written Protocol/Manual Logbooks 10 # Personnel Records
Fidelity Monitoring Documents Program Schedules 10 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 5 # Youth Records (Closed)
Staff Evaluations Table of Organization 5 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports 5 # Health Records
Contract Scope of Services 5 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is LifeSkills Training. The program was awarded 15 points
because the Primary Service is identified as a Group 3 Service. The specific Sub-Component
Service Type identified is Social Skills Training. The Primary Service was identified as this type of
service as it focuses on developing social skills required for an individual to interact in a positive
way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The documentation reviewed reflected two staff received formal training from a qualified trainer in
LifeSkills Training (LST). One staff member was trained in August 2011 and the other staff
received training in February 2016. The first staff has been delivering LST for three years and the
new staff began co-facilitating after being trained.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program provides the LST curriculum to all youth admitted, one cohort at a time. The program
has the Middle School Manual and Student Guides on-site. The manual contains the order of the
lessons to be delivered, lesson plans for each session, and implementation guidelines.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The team observed a session of LST during the review. The facilitator provided a copy of the
lesson plan, but it only had every other page. The master lesson was printed on both sides of the
page. So the team was only able to compare the presentation on the pages delivered. The
facilitator followed the lesson reading verbatim. The facilitator added an icebreaker game as the
lesson suggested a board game to introduce the idea of making decisions.
d. Facilitator Turnover Basic Score: 2 Point(s)
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The reviewed documentation and interviews with the clinical manager and the corporate
compliance manager revealed there has been no turnover in facilitators during the review period.
One new facilitator has been added into the mix of delivering the LST curriculum during the review
period.
BasicScore: 1Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program had no documented internal fidelity monitoring from June until December 2015. The
program reviewed their fidelity monitoring practice and rewrote the facility operating procedure on
fidelity monitoring which was signed December 31, 2015. Since then, the LST delivery received
two monthly fidelity monitoring by a trained facilitator including February 18, 2016 and March 31,
2016. Both internal fidelity monitoring events were documented on a tool specifically designed for
the LST intervention.
The program can earn 2 points if fidelity monitoring is completed each month for every facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
BasicScore: 1Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program had no documented fidelity monitoring from June until December 2015. Since then,
the LST delivery received two monthly fidelity monitoring by a trained facilitator. The trained
facilitator providing the fidelity monitoring provided constructive feedback to the facilitator
delivering the intervention, but none of the feedback required further corrective action.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
There are two trained facilitators in the LST curriculum. One staff has been delivering the
curriculum for several years and her annual evaluation specifically rated her performance in the
delivery of the LST curriculum. The new facilitator received an annual evaluation last year, but
had not been trained in the delivering the service of LST at that time.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 16 weeks for this type of service. Of the 11 youth in the
sample, 73% (8 of 11) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 24 contact hours for this type of service. Of the 11 youth in the
sample, 0% (0 of 11) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 3 youth
Moderate-High = 3 youth
High = 5 youth
Total Youth in Sample = 11 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 45%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 5 youth
Total Youth in Sample = 11 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (11 of 11) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 45% (5 of 11) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 25 25 100%
Totals 71 85 84%
This SPEP report evaluates LifeSkills Training, an intervention delivered at Brevard Group
Treatment Home.
The program scored High for Quality of Service Delivery.
The program earned 6 points for Amount of Service: Duration. Of the 11 total youth sampled, 8
received at least the recommended weeks of service. Youth in the sample completed between 14
and 16 weeks of service.
The program earned 0 points for Amount of Service: Contact Hours. Of the 11 total youth
sampled, none received at least the recommended hours of service. All youth received 15 hours
of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Brevard Group Treatment Home can optimize their SPEP Quality of Service Delivery score by
ensuring fidelity monitoring is completed monthly for each facilitator.
Brevard Group Treatment Home can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

service_type: LifeSkills Training


summary: SPEP - FY15-16

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Brevard Group Treatment Home
Aspire Health Partners, Inc.
(Contract Provider)
3905 Grissom Parkway
Cocoa, Florida 32926
Primary Service: Moral Reconation Therapy
SPEP Review Date(s): May 10‐13, 2016
Florida Department of Juvenile Justice
Report Date(s): 5/1/2017
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Brevard Group Treatment Home QI Program Code: 571
Provider Name: Aspire Health Partners, Inc Contract Number: R2109
Location: Brevard County / Circuit: 18 Number of Beds: 30
Review Date(s): May 10-13, 2016 Lead Reviewer Code: 77
Persons Interviewed
Program Director 1 # Case Managers 4 # Other (listed by title): Chief
DJJ Monitor 1 # Clinical Staff Development Officer, Director of
DHA or designee 1 # Healthcare Staff Community Relations, Corporate
DMHCA or designee # Program Supervisors Regional Nurse, Corporate Food
Corporate QI/QA staff 5 # Youth Service Manager
Documents Reviewed
Written Protocol/Manual Logbooks 10 # Personnel Records
Fidelity Monitoring Documents Program Schedules 10 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 5 # Youth Records (Closed)
Staff Evaluations Table of Organization 5 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports 5 # Health Records
Contract Scope of Services 5 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Moral Reconation Therapy. The program was awarded
30 points because the Primary Service is identified as a Group 5 Service. The specific Sub-
Component Service Type identified is Cognitive Behavioral Therapy. The Primary Service was
identified as this type of service as it is intended to correct faulty cognitions and perceptions and
provides skills individuals can use to monitor thought patterns and correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 6 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program's documentation indicated a training protocol was revised in December 31, 2015 to
reflect all facilitators will receive formal training. At the time of this review the program had three
staff who received formal training in the Moral Reconation Therapy (MRT) curriculum. One of the
three trained staff does not deliver the curriculum on a regular basis, but co-facilitates on an
occasional basis.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the published manual and workbooks for the Moral Reconation Therapy (MRT)
curriculum. The program is using Juvenile MRT: How to Escape Your Prison, A Moral Reconation
Workbook. Each youth has his own workbook in which all work is written and graded for
progression to the next level. The lessons are not scripted; each lesson has a plan and process
that allows for youth to work on their own speed and motivation level. The schedule is laid out for
the facilitator to follow with a standard process to guide each lesson and to guide each youth
through the twelve steps.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Moral Reconation Therapy (MRT) was observed during the review. The session was
run by the clinical manager for eight youth on Thursday afternoon. The youth drafted the agenda
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
for the session to include who would present and in what order. Three youth presented to the
group as a whole and asked for feedback and the group gave verbal feedback then voted on
whether the presenting youth should be allowed to progress to the next level. Each youth’s
presentation represented homework he had completed and previously processed with the
facilitator during an individual session.
The reviewer received a copy of the lesson plan prior to the lesson. The facilitator followed the
design of the lesson, providing encouragement, and affirmation, as well as redirection when
needed.
Basic Score: 0 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There was considerable turnover in the program impacting delivery of the Moral Reconation
Therapy (MRT) during the review period. The clinical manager and another clinician trained in
MRT resigned within a six-month period in 2015, each leaving groups. One trained clinician
continued delivering MRT to one group of youth with a one month gap in service. In February
2016, a new clinical manager came aboard who had been trained in MRT and the two clinicians
began delivery to two groups of eight to ten youth each. The turnover meant that only one third of
the youth received the service for most of the year.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
BasicScore: 1Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The team reviewed ten months of fidelity monitoring reports. The program did not have a specific
internal fidelity monitoring protocol until December 31, 2015. The program had one monitoring in
June 2015, prior to December 2015. Documentation was reviewed during the review week that
confirmed fidelity monitoring was completed each month from January to April 2016 by a trained
facilitator in MRT using a form specifically designed for the intervention. There was constructive
feedback found on each of the fidelity monitoring tools.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
BasicScore: 1Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
In all completed fidelity monitoring events the trained facilitator provided constructive feedback to
the facilitator delivering the intervention, but none of the feedback required further corrective
action.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
There were three trained facilitators for MRT receiving evaluations during the review period. The
clinical manager has been delivering the curriculum for several years at another program and her
annual evaluation included her performance in delivery of MRT. Another clinician delivering the
MRT curriculum received a ninety-day performance review after receiving the MRT training and
delivering the service; however the evaluation did not include her performance in the delivery of
MRT. The third staff received an annual performance evaluation; however her performance in the
delivery of the MRT was not included.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates the target duration of 15 weeks for this type of service. Of the 20 youth in the
sample, 25% (5 of 20) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 45 contact hours for this type of service. Of the 20 youth in the
sample, 5% (1 of 20) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 4 youth
Moderate-High = 4 youth
High = 12 youth
Total Youth in Sample = 20 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 60%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 12 youth
Total Youth in Sample = 20 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (20 of 20) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 60% (12 of 20) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 2 10 20%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 25 25 100%
Totals 72 100 72%
This SPEP report evaluates Moral Reconation Therapy, an intervention delivered at Brevard
Group Treatment Home.
The program scored Medium for Quality of Service Delivery.
The program earned 2 points for Amount of Service: Duration. Of the 20 total youth sampled, only
8 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 5
received at least the recommended weeks of service. Youth in the sample completed between 8
and 40 weeks of service, with an average of 23 weeks.
The program earned 0 points for Amount of Service: Contact Hours. Of the 20 total youth
sampled, one received at least the recommended hours of service. Youth in the sample
completed between 6 and 51 hours of service, with an average of 22 hours.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
Brevard Group Treatment Home can optimize their SPEP Quality of Service Delivery score by
planning and minimizing the impact of facilitator turnover, completed fidelity monitoring monthly,
and ensuring each facilitator’s performance evaluation includes discussion of their skill in
delivering this primary service.
Brevard Group Treatment Home can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

service_type: Moral Reconation Therapy


summary: SPEP - FY16-17

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Brevard Group Treatment Home
Aspire Health Partners, Inc.
(Contract Provider)
3905 Grissom Parkway
Cocoa, Florida 32926
Primary Service: Impact of Crime (IOC)
SPEP Review Date(s): April 11‐14, 2017
Florida Department of Juvenile Justice
Report Date(s): 11/28/2017
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Brevard Group Treatment Home MQI Program Code: 571
Provider Name: Aspire Health Partners, Inc. Contract Number: R2109
Location: Brevard County / Circuit: 18 Number of Beds: 30
Review Date(s): April 11-14, 2017 Lead Reviewer Code: 77
Persons Interviewed
Program Director Corporate QI/QA staff 2 # Program Supervisors
DJJ Monitor 1 # Case Managers 5 # Youth
DHA or designee 1 # Clinical Staff 2 # Other (listed by title): Senior
DMHCA or designee 2 # Healthcare Staff Director, Vice President
Documents Reviewed
Written Protocol/Manual Logbooks 8 # Personnel Records
Fidelity Monitoring Documents Program Schedules 5 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 5 # Youth Records (Closed)
Staff Evaluations Table of Organization 5 # Youth Records (Open)
Accreditation Reports Youth Handbook 5 # Other:
Contract Monitoring Reports 5 # Health Records
Contract Scope of Services 5 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime (IOC). The program was awarded 15
points because the Primary Service is identified as a Group 3 Service. The specific Sub-
Component Service Type identified is Social Skills Training. The Primary Service was identified
as this type of service as it focuses on developing social skills required for an individual to interact
in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program documentation indicated a training protocol was revised in December 31, 2015, to
reflect all facilitators will receive formal training in the evidence based intervention and be
evaluated through fidelity monthly monitoring by a facilitator who also received formal training in
the intervention. One staff member received formal training by a qualified trainer to become a
group facilitator in Impact of Crime August 2013. Another staff member received formal training by
a qualified trainer to become a group facilitator in Impact of Crime November 18, 2016. Only one
staff member has conducted IOC in the last eleven months.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program has a manual of delivery for Impact of Crime (IOC) and provided a copy of the
manual to the reviewer. The curriculum was revised and updated in 2009. The revised Impact of
Crime is a victim impact/restorative justice curriculum consisting of seven chapters taking
approximately twenty-four sessions to facilitate. The manual includes a script, implementation
guidelines, identifies the order the lessons should be delivered and detailed instructions for the
delivery of each lesson.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
This curriculum was not able to be observed; therefore, this indicator rates as non-applicable.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The reviewed documentation and interviews with staff revealed there has been no turnover in the
middle of cohorts. One primary facilitator began delivering the intervention at the beginning of a
new cohort and continued delivery of all the following sessions and cohorts to the present.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The team reviewed ten months of fidelity monitoring reports. The program has a specific
monitoring protocol including monthly monitoring will be conducted by a facilitator trained in the
intervention using a form specifically designed for the intervention. Documentation was reviewed
during the review week confirming fidelity monitoring was completed the months of August to
December 2016 by a trained facilitator in IOC using a form specifically designed for IOC. There
was constructive feedback found on each of the fidelity monitoring tools.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program consistently completed monthly fidelity monitoring in IOC from August until
December 2016. None of these monitoring events documented a need for corrective action.
The program reviewed their fidelity monitoring practice and rewrote the facility operating
procedure on fidelity monitoring which was signed December 31, 2015. The trained facilitator
providing the fidelity monitoring provided constructive feedback to the facilitator delivering the
intervention, but none of the feedback required further corrective action. The corrective action
plan for the curriculum as a whole was reviewed monthly with the clinical manager and completed
March 15, 2016. There was no corrective action documented for a particular facilitator during the
review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
There are two trained facilitators in the IOC curriculum. One staff has been delivering the
curriculum for several years and her evaluation rated her performance in the delivery of the IOC
curriculum. The new facilitator received an annual evaluation last year, but had not been
delivering the service and was not evaluated on delivery of the intervention. The annual
evaluation was not reviewed of the supervisory staff trained in IOC from corporate, who provides
fidelity monitoring.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 16 weeks for this type of service. Of the 8 youth in the
sample, 0% (0 of 8) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 24 contact hours for this type of service. Of the 8 youth in the
sample, 0% (0 of 8) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 0 youth
Moderate-High = 3 youth
High = 5 youth
Total Youth in Sample = 8 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 63%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 5 youth
Total Youth in Sample = 8 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (8 of 8) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 63% (5 of 8) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 25 25 100%
Totals 65 85 76%
This SPEP report evaluates Impact of Crime (IOC), an intervention delivered at Brevard Group
Treatment Home.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. 0% received at least the
recommend weeks of service, as sampled youth did not contain an end date. Without an end
date, it was impossible to ascertain service weeks. The program earned 0 points for Amount of
Service: Contact Hours. 0% received at least the recommended hours of service, with sample
youth completing an average of 17 hours of service.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
itself has no control over youths' C-PACT risk level because the scored assessment was
administered just prior to the youths' admission.
RECOMMENDATION(S):
Brevard Group Treatment Home can optimize their SPEP Amount of Service score by ensuring
data is entered in the EBS Module accurately and that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10


summary: SPEP - FY16-17

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Brevard Group Treatment Home
Aspire Health Partners, Inc.
(Contract Provider)
3905 Grissom Parkway
Cocoa, Florida 32926
Primary Service: LifeSkills Training (LST)
SPEP Review Date(s): April 11‐14, 2017
Florida Department of Juvenile Justice
Report Date(s): 11/28/2017
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Brevard Group Treatment Home MQI Program Code: 571
Provider Name: Aspire Health Partners, Inc. Contract Number: R2109
Location: Brevard County / Circuit: 18 Number of Beds: 30
Review Date(s): April 11-14, 2017 Lead Reviewer Code: 77
Persons Interviewed
Program Director Corporate QI/QA staff 2 # Program Supervisors
DJJ Monitor 1 # Case Managers 5 # Youth
DHA or designee 1 # Clinical Staff 2 # Other (listed by title): Senior
DMHCA or designee 2 # Healthcare Staff Director, Vice President
Documents Reviewed
Written Protocol/Manual Logbooks 8 # Personnel Records
Fidelity Monitoring Documents Program Schedules 5 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 5 # Youth Records (Closed)
Staff Evaluations Table of Organization 5 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports 5 # Health Records
Contract Scope of Services 5 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is LifeSkills Training (LST). The program was awarded 15
points because the Primary Service is identified as a Group 3 Service. The specific Sub-
Component Service Type identified is Social Skills Training. The Primary Service was identified
as this type of service as it focuses on developing social skills required for an individual to interact
in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program documentation indicated a training protocol was revised in December 31, 2015 to
reflect all facilitators will receive formal training in the evidence based intervention and be
evaluated through fidelity monthly monitoring by a facilitator who also received formal training in
the intervention.
The documentation reviewed reflected the program training protocol was completed for two staff
who received formal training from a qualified trainer in LifeSkills Training. One staff member was
trained in August 2011 and the other staff received training February 2016. The first staff has
been delivering LST for three years and the new staff began co-facilitating and in August 2016
began delivery of his first cohort. The staff on the corporate level providing fidelity monitoring of
the intervention received training by a qualified trainer.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program provides the evidence-based curriculum LifeSkills Training to all youth admitted, one
cohort at a time. The program has on-site the LifeSkills Training Middle School Teachers Manual
and LifeSkills Training Student Guides for each youth. The manual contains the order of the
lessons to be delivered, lesson plans for each session, and implementation guidelines.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
This curriculum was not able to be observed; therefore, this indicator rates as non-applicable.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The reviewed documentation and interviews with the clinical manager and the corporate senior
director revealed there has been no turnover in delivery of the LifeSkills Training (LST)
intervention during the eleven months of the review period.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The LST facilitators received monthly fidelity monitoring by a trained facilitator during both
cohorts. There was no fidelity monitoring found for the month of October 2016. All internal fidelity
monitoring events were documented on a tool specifically designed for the LifeSkills Training
intervention.
The program can earn 2 points by ensuring that internal fidelity monitoring is conducted monthly.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The LifeSkills Training intervention received eight total monthly fidelity monitoring events by a
trained facilitator. The trained facilitator providing the fidelity monitoring provided constructive
feedback to the facilitator delivering the intervention, but none of the feedback required further
corrective action. The corrective action plan for the curriculum as a whole was reviewed monthly
with the clinical manager and completed March 15, 2016. There was no corrective action
documented for a particular facilitator during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1Point
Intervention Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
There are two trained facilitators in the LifeSkills Training (LST) curriculum on the campus. One
staff has been delivering the curriculum for several years and her annual evaluation specifically
rated her performance in the delivery of the LST curriculum. The new facilitator received an
annual evaluation last year, but had not been trained in delivering the service or actually delivered
the service. The annual evaluation of the corporate staff providing fidelity monitoring was not
reviewed.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 16 weeks for this type of service. Of the 16 youth in the
sample, 0% (0 of 16) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 24 contact hours for this type of service. Of the 16 youth in the
sample, 0% (0 of 16) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 1 youth
Moderate-High = 4 youth
High = 11 youth
Total Youth in Sample = 16 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 69%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 11 youth
Total Youth in Sample = 16 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (16 of 16) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 69% (11 of 16) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 25 25 100%
Totals 65 85 76%
This SPEP report evaluates LifeSkills Training (LST), an intervention delivered at Brevard Group
Treatment Home.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. 0% received at least the
recommend weeks of service and most sampled youth did not have a service end date. The
program earned 0 points for Amount of Service: Contact Hours. 0% received at least the
recommended hours of service, with sample youth completing an average of 10 hours of service.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered just prior to the youths' admission.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
RECOMMENDATION(S):
Brevard Group Treatment Home can optimize their SPEP Quality of Service Delivery score by
ensuring that internal fidelity monitoring is conducted monthly.
Brevard Group Treatment Home can optimize their SPEP Amount of Service score by ensuring
that data is entered correctly into the EBS MOdule and by ensuring youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10


summary: SPEP - FY16-17

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Brevard Group Treatment Home
Aspire Health Partners, Inc.
(Contract Provider)
3905 Grissom Parkway
Cocoa, Florida 32926
Primary Service: Moral Reconation Therapy (MRT)
SPEP Review Date(s): April 11‐14, 2017
Florida Department of Juvenile Justice
Report Date(s): 11/28/2017
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Brevard Group Treatment Home MQI Program Code: 571
Provider Name: Aspire Health Partners, Inc. Contract Number: R2109
Location: Brevard County / Circuit: 18 Number of Beds: 30
Review Date(s): April 11-14, 2017 Lead Reviewer Code: 77
Persons Interviewed
Program Director Corporate QI/QA staff 2 # Program Supervisors
DJJ Monitor 1 # Case Managers 5 # Youth
DHA or designee 1 # Clinical Staff 2 # Other (listed by title): Senior
DMHCA or designee 2 # Healthcare Staff Director, Vice President
Documents Reviewed
Written Protocol/Manual Logbooks 8 # Personnel Records
Fidelity Monitoring Documents Program Schedules 5 # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts 5 # Youth Records (Closed)
Staff Evaluations Table of Organization 5 # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports 5 # Health Records
Contract Scope of Services 5 # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Moral Reconation Therapy (MRT). The program was
awarded 30 points because the Primary Service is identified as a Group 5 Service. The specific
Sub-Component Service Type identified is Cognitive Behavioral Therapy. The Primary Service
was identified as this type of service as it is intended to correct faulty cognitions and perceptions
and provides skills individuals can use to monitor thought patterns and correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program documentation indicated a training protocol was revised in December 31, 2015 to
reflect all facilitators will receive formal training in the evidence based intervention and be
evaluated through fidelity monthly monitoring by a facilitator who also received formal training in
the intervention. The program currently has three staff who received formal training in the MRT
curriculum. One of the three trained staff does not deliver the curriculum on a regular basis, but
co-facilitates on an occasional basis. The other staff holds the position of senior director and is the
direct report over the program director. This person although not actively delivering the
intervention, provides fidelity monitoring monthly.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the published manual and workbooks for the MRT curriculum. The program is
using the Juvenile MRT: How to Escape Your Prison, A Moral Reconation Workbook. Each youth
has his own workbook in which all work is written and graded for progression to the next level.
The curriculum has a design for each session to include presentations of youth at various levels,
with a protocol for response from facilitator and the group in order to progress to the next level.
The lessons are not scripted so the facilitator reads or recites the lesson word for word. Each
lesson has a plan and process allowing youth to work at their own speed and motivation
maintaining the open group. The schedule is laid out for the facilitator to follow with a standard
process to guide each lesson and to guide each youth through the twelve steps.
c. Observed Adherence to the Manual/Protocol Basic Score: 1 Point(s)
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The team observed the delivery of MRT by the clinical manager, who facilitated the intervention to
ten youth on Thursday morning. In addition to the facilitator, a direct care staff was in the group.
The facilitator reviewed who was presenting today and the step on which each individual was
working. The reviewer received a copy of the lesson plan prior to the lesson and briefly discussed
the progression of the session. The facilitator essentially followed the design of the lesson,
providing encouragement, and affirmation as well as redirection when needed.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There was considerable turnover in the program delivery of the MRT curriculum. The clinical
manager trained in MRT resigned following the last annual compliance review. Additionally, all
three of the other clinicians resigned and were replaced.
At the time of the annual compliance review, a new clinical manager who had been trained in
MRT came aboard in August 2016. Only one cohort of youth has received the intervention, and
they have not completed. The gap in delivery of service meant only ten of the youth received the
service beginning this year.
The program can earn 2 points by minimizing turnover and gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The team reviewed eleven months of fidelity monitoring reports. The program provided monthly
fidelity monitoring in MRT using a form the program specifically designed for the intervention.
Documentation was reviewed during the review week confirming fidelity monitoring was
completed each month from January to March 2017 by a trained facilitator in MRT. There was
comprehensive feedback found on each of the fidelity monitoring tools.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program had three internal fidelity monitoring events in MRT. The trained facilitator providing
the fidelity monitoring providing comprehensive feedback to the facilitator delivering the
intervention. There was no corrective action documented for a particular facilitator during the
review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
There are three trained facilitators in the MRT curriculum. The clinical manager has been
delivering the curriculum for several years at another program and her annual evaluation included
her performance in delivery of MRT.
The second staff received an annual performance evaluation; however, her performance in the
delivery of the MRT was not included since she does not deliver the intervention. The third staff
trained MRT does not deliver the intervention, but only provides fidelity monitoring for the
intervention.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 15 weeks for this type of service. Of the 7 youth in the
sample, 0% (0 of 7) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 45 contact hours for this type of service. Of the 7 youth in the
sample, 0% (0 of 7) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 1 youth
Moderate-High = 3 youth
High = 3 youth
Total Youth in Sample = 7 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 43%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 3 youth
Total Youth in Sample = 7 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (7 of 7) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 43% (3 of 7) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 25 25 100%
Totals 80 100 80%
This SPEP report evaluates Moral Reconation Therapy (MRT), an intervention delivered at
Brevard Group Treatment Home.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. No sampled youth included an end
date, so it was impossible to ascertain the number of intervention weeks received. The program
earned 0 points for Amount of Service: Contact Hours. 0% received at least the recommended
hours of service, with sample youth completing an average of 9 hours of service.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered just prior to the youths' admission.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
RECOMMENDATION(S):
Brevard Group Treatment Home can optimize their SPEP Quality of Service Delivery score by
minimizing facilitator turnover and gaps in service.
Brevard Group Treatment Home can optimize their SPEP Amount of Service score by ensuring
that youth receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10