summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Brevard Group Treatment Home
Aspire Health Partners, Inc.
(Contract Provider)
3905 Grissom Parkway
Cocoa , Florida 32926
Primary Service: LifeSkills Training
SPEP Review Period: February 1, 2021-January 31, 2022
SPEP Review Conducted: February 8, 2022
Florida Department of Juvenile Justice
Report Date: 3/24/2022
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is LifeSkills Training.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The annual SPEP (Standardized Program Evaluation Protocol) review is a twelve month
evaluation of all related documents and records. LifeSkills Training (LST) is an evidence-based
substance abuse prevention program which aims to deter alcohol, tobacco, drug abuse, and
violence among teenagers by teaching them the necessary skills to resist social peer pressures to
engage in alcohol and substance use, increasing their understanding of the consequences of
substance abuse, and promoting individual self-esteem and confidence.
The program provided evidence of two case managers and clinical director being trained in LST.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the commercially-produced curriculum LifeSkills Training from Botvin. The
curriculum contains reproduceable student worksheets, as well as facilitator notes, classroom
management ideas and helpful instructions. The curiculum is specifically designed for middle
school students. The group functions as a closed group, meaning once a cohort has started, no
new youth can be added until a new cohort begins.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
During the review period, a session of LifeSkills Training was observed. The group session took
place in a cafe with six youth present. The facilitator reviewed the previous lesson, and then
introduced the new topic for the day's group on assertiveness. The group completed individual
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
worksheets, and shared their answers within the group. This led to a large group discussion about
various degrees of assertiveness among the youth. There were no behavior issues observed. The
facilitator did not read the script verbatim, but did have the manual open on the desk to ensure all
points of the lesson were covered.
Basic Score: 0 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program had a facilitator leave the program in July 2021 and an new facilitator was hired.
The cohort continued in late August. The gap in service, according to the documentation provided,
was approximately six weeks.
The program provided documentation of three cohorts taking place during the review period. The
first cohort began on January 16, 2021 with eleven youth, and ended on April 26, 2021 with seven
youth. Four youth were discharged from the program. A second cohort began on May 15, 2021
with seven youth and was completed on October 1, 2021 with six youth. One youth was
discharged from the program. A third cohort started on October 15, 2021 with six youth and was
on going with six youth at the time of the review.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
According to documentation provided by the program, internal fidelity monitoring took place twelve
of the twelve months LifeSkills Training was facilitated. The fidelity monitoring was completed by
another trained facilitator in the primary service.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a policy for corrective action plans for facilitators of LifeSkills Training. In the
event a facilitator is in need of help with service delivery, the director of case management will sit
with the facilitator and develop a plan to improve the facilitator's skills.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of annual evaluations documented the facilitators were given credit for their group
facilitation. The facilitators' skills in the delivery of LifeSkills Training was also evaluated by
supervisors.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 15 weeks for this type of service. Of the 12 youth in the
sample, 67% (8 of 12) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 15 contact hours for this type of service. Of the 12 youth in the
sample, 75% (9 of 12) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 0 youth
Moderate-High = 3 youth
High = 9 youth
Total Youth in Sample = 12 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 75%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 9 youth
Total Youth in Sample = 12 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (12 of 12) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 75% (9 of 12) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 25 25 100%
Totals 77 85 91%
This SPEP report evaluates LifeSkills Training, an intervention delivered at Brevard Group
Treatment Home .
12 youth completed the program during the review period. Of those youth, all 12 were included in
the sample.
The program scored High for Quality of Service Delivery.
The targeted number of weeks for this specific intervention is 15 weeks; the targeted number of
hours for this specific intervention is 15 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 3 services (16 weeks; 24
hours).
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 6 points for Amount of Service: Duration. Of the 12 total youth sampled, all
12 included dosage with end dates in the EBS Module. Of those youth, 8 received at least the
recommended weeks of service. Youth in the sample completed between 4 and 15 weeks of
service, with an average of 13 weeks.
The program earned 6 points for Amount of Service: Contact Hours. Of the 12 total youth
sampled, all 12 included dosage in the EBS Module. Of those youth, 9 received at least the
recommended hours of service. Youth in the sample completed between 2 and 16 hours of
service, with an average of 14 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
RECOMMENDATION(S):
Brevard Group Treatment Home can optimize their SPEP Quality of Service Delivery score by
ensuring facilitator turnover does not impact groups
Brevard Group Treatment Home can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2021-22 BGTH LST.pdf


state: FL

facility_name: Brevard Juvenile Detention Center

summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Brevard Group Treatment Home
Aspire Health Partners, Inc.
(Contract Provider)
3905 Grissom Parkway
Cocoa, Florida 32926
Primary Service: Moral Reconation Therapy
SPEP Review Period: February 1, 2021-January 31, 2022
SPEP Review Conducted: February 8, 2022
Florida Department of Juvenile Justice
Report Date: 3/24/2022
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Moral Reconation Therapy.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The Standardized Program Evaluation Protocol (SPEP) review encompasses twelve months of
SPEP records. Moral Reconation Therapy (MRT) was initially designed as a treatment program
for adults residing in prison-based substance abuse therapeutic communities. MRT is designed to
help participants reexamine their choices and develop cognitive structures that will improve
decision-making skills. The curriculum for juveniles, entitled “Juvenile MRT- How to Escape Your
Prison” is appropriate for both juvenile offenders and juvenile substance abusers, and is
appropriate for general treatment and substance abuse treatment. Based on cognitive-behavioral
therapy principles, characteristics guiding MRT practice include: 1) self-assessment of attitudes,
beliefs, defense mechanisms, and behavior, 2) current relationship assessment, 3) positive
reinforcement, 4) alteration of self-concept in the positive direction, 5) changes in gratification
stimuli, and 6) enhance moral reasoning. The program's clinical director and therapist are trained
in this primary service.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes a commercially-produced curriculum consisting of both student
worksheets/activities and facilitator guide book. The student edition contains reproduceable
activities which reinforce various concepts discussed in group. The facilitator's edition provides
helpful notes in facilitating the ideas, as well as ways to spark youth engagement.
Moral Reconation Therapy (MRT) is a cognitive-behavioral therapy which runs in an open group
setting, meaning youth can be added to the primary service at anytime, without a need to wait for
a new cohort.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Moral Reconation Therapy was observed during the annual review. Four youth and a
direct care staff attended group along with the facilitator. The group was held in a conference
room. The youth were seated around a large conference table, yet worked independently on the
assigned task. Each youth was working on their assigned step, and one youth presented their
step to the rest of the group to receive credit for completing the step. No disciplinary problems
were noted during the group.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
A review of documentation and informal interview with the clinical director confirmed the program
has had no turnover of Moral Reconation Therapy facilitators during the review period. The
program had no gaps in service during the review period, as the group is run year-round.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
According to documentation provided by the program, internal fidelity monitoring for facilitators of
Moral Reconation Therapy took place on a monthly basis and occurred all twelve months. The
fidelity monitoring was completed by another trained facilitator in the primary service.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a policy for corrective actions for facilitators of Moral Reconation Therapy. In the
event a facilitator is in need of help with service delivery, the director of case management will sit
with the facilitator and develop a plan to improve the facilitator's skills.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of annual evaluations documented the facilitators were given credit for their group
facilitation. The facilitators' skills in the delivery of Moral Reconation Therapy was also evaluated
by supervisors.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 15 weeks for this type of service. Of the 11 youth in the
sample, 73% (8 of 11) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates a target of 45 contact hours for this type of service. Of the 11 youth in the
sample, 36% (4 of 11) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 0 youth
Moderate-High = 3 youth
High = 8 youth
Total Youth in Sample = 11 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 73%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 8 youth
Total Youth in Sample = 11 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (11 of 11) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 73% (8 of 11) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 2 10 20%
Risk Level of Youth Served 25 25 100%
Totals 88 100 88%
This SPEP report evaluates Moral Reconation Therapy, an intervention delivered at Brevard
Group Treatment Home .
12 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 6 points for Amount of Service: Duration. Of the 11 total youth sampled, all
11 included dosage with end dates in the EBS Module. Of those youth, 8 received at least the
recommended weeks of service. Youth in the sample completed between 7 and 18 weeks of
service, with an average of 15 weeks.
The program earned 2 points for Amount of Service: Contact Hours. Of the 11 total youth
sampled, all 11 included dosage in the EBS Module. Of those youth, 4 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
recommended hours of service. Youth in the sample completed between 14 and 48 hours of
service, with an average of 39 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
RECOMMENDATION(S):
Brevard Group Treatment Home can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Brevard Group Treatment Home can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8

source: DJJ

report_type: SPEP

file_name: FY2021-22 BGTH MRT.pdf

program_name_raw: Brevard Group Treatment Home

program_status_note: Closed

service_type: Moral Reconation Therapy