state: FL

facility_name: JoAnn Bridges Academy

summary: SPEP - FY16-17

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
JoAnn Bridges Academy
Rite of Passage
(Contract Provider)
950 SW Greenville Road
Greenville, Florida 32331
Primary Service: Aggression Replacement Training (ART)
SPEP Review Date(s): May 16-19, 2017
Florida Department of Juvenile Justice
Report Date(s): 1/16/2018
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: JoAnn Bridges Academy MQI Program Code: 832
Provider Name: Rite of Passage Contract Number: 10361
Location: Madison County / Circuit: 3 Number of Beds: 28
Review Date(s): May 16-19, 2017 Lead Reviewer Code: 141
Persons Interviewed
Program Director Corporate QI/QA staff # Program Supervisors
DJJ Monitor # Case Managers # Youth
DHA or designee 1 # Clinical Staff # Other (listed by title):
DMHCA or designee # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 3 # Personnel Records
Fidelity Monitoring Documents Program Schedules # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts # Youth Records (Closed)
Staff Evaluations Table of Organization # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports # Health Records
Contract Scope of Services # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Aggression Replacement Training (ART). The program
was awarded 30 points because the Primary Service is identified as a Group 5 Service. The
specific Sub-Component Service Type identified is Cognitive Behavioral Therapy. The Primary
Service was identified as this type of service as it is intended to correct faulty cognitions and
perceptions and provides skills individuals can use to monitor thought patterns and correct
behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 5 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There were four staff that facilitated Aggression Replacement Training (ART) during the review
period. All four staff were trained by the previous clinical director. The previous clinical director
had been formally trained in ART on 9/12/2014. One facilitator was trained December 2015 and is
no longer employed at the facility. Another facilitator was trained on October 24, 2016. The last
two facilitators were trained March 31, 2017. All training certificates were provided during the
review.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
This program provided the Aggression Replacement Training (ART) manual for this reviewer. This
manual was the third edition-revised and expanded by authors Barry Glick and John Gibbs. The
manual was organized by each lesson: skill streaming, moral reasoning, and anger control. Each
lesson is divided by weeks. The book designates what the skill is for each lesson, lists the
objectives that should be met, materials that are needed, and then leads into the lesson. There is
a general script to follow and a procedure for each lesson to use as a guide. The skill steps are
broken into sub-steps for the facilitator to follow and steps the youth needs to complete. There are
no student manuals for ART.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Aggression Replacement Training group was not observed during this review.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The facility’s policy regarding turnover is to have all therapeutic managers and the clinical director
trained in Aggression Replacement Training (ART). In the event of staff turnover or a facilitator
absence, another facilitator is able to facilitate the group without disruption. The reviewed
documentation and interviews with the program director confirmed that no staff turnover adversely
effected group.
Basic Score: 0 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
This reviewer met with the program director regarding the internal fidelity forms. The program
director reported that she had requested the SPEP books to review from the clinical director.
According to the program director the clinical director left employment and did not provide the
requested documentation. The internal fidelity monitoring forms have not been located since the
termination of employment in 2017.
The program procedure for internal fidelity monitoring is for the new clinical director to complete
internal fidelity monitoring once trained in Aggression Replacement Training curriculum. This
reviewer met with the new clinical director and discussed that the current facilitators can monitor
each other once time per month.
The program can earn 2 points by ensuring internal fidelity monitoring is conducted at least
monthly for all facilitators.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 0 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
No internal fidelity monitoring has been completed, therefore no corrective actions were
completed.
The program can earn 1 point by ensuring there is a procedure in place for corrective action
based on fidelity monitoring.
At the time of the review, the program did not meet criteria to earn a score of 1.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Two of the Aggression Replacement Training facilitators have received yearly performance
evaluations. The previous clinical director received her yearly performance evaluation prior to
ending employment. The two other facilitators have not been employed at the facility long enough
to have had a yearly evaluation.
All three yearly performance evaluations were provided to this reviewer. There is not a specific
section on the yearly performance evaluations specifically for group facilitation.
The program can earn 1 point by ensuring all facilitators' performance evaluations includes an
assessment of their delivery of ART.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 10 weeks for this type of service. Of the 14 youth in the
sample, 100% (14 of 14) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 30 contact hours for this type of service. Of the 14 youth in the
sample, 100% (14 of 14) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 10 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 40%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 93%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 1 youth
Moderate-High = 10 youth
High = 2 youth
Total Youth in Sample = 14 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 14%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 2 youth
Total Youth in Sample = 14 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 93% (13 of 14) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 14% (2 of 14) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 10 25 40%
Totals 75 100 75%
This SPEP report evaluates Aggression Replacement Training (ART), an intervention delivered at
JoAnn Bridges Academy.
The program scored Medium for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 14 total youth sampled, all
14 received at least the recommended weeks of service. All youth in the sample completed
approximately 10 weeks of service.
The program earned 10 points for Amount of Service: Contact Hours. Of the 14 total youth
sampled, all 14 received at least the recommended hours of service. All youth in the sample
completed 30 hours of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program was awarded 10 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered just prior to the youths' admission.
RECOMMENDATION(S):
JoAnn Bridges Academy can optimize their SPEP Quality of Service Delivery score by ensuring
internal fidelity monitoring is conducted at least monthly for all facilitators, by ensuring there is a
process for corrective action based on fidelity monitoring, and by ensuring all facilitators'
performance evaluations includes an assessment of their delivery of ART.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

source: DJJ

report_type: SPEP

program_name_raw: JoAnn Bridges Academy

program_status_note: Open

service_type: Aggression Replacement Training


summary: SPEP - FY16-17

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
JoAnn Bridges Academy
Rite of Passage
(Contract Provider)
950 SW Greenville Road
Greenville, Florida 32331
Primary Service: Dialectical Behavioral Therapy (DBT)
SPEP Review Date(s): May 16-19, 2017
Florida Department of Juvenile Justice
Report Date(s): 1/16/2018
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: JoAnn Bridges Academy MQI Program Code: 832
Provider Name: Rite of Passage Contract Number: 10361
Location: Madison County / Circuit: 3 Number of Beds: 28
Review Date(s): May 16-19, 2017 Lead Reviewer Code: 141
Persons Interviewed
Program Director Corporate QI/QA staff # Program Supervisors
DJJ Monitor # Case Managers # Youth
DHA or designee 1 # Clinical Staff # Other (listed by title):
DMHCA or designee # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 3 # Personnel Records
Fidelity Monitoring Documents Program Schedules # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts # Youth Records (Closed)
Staff Evaluations Table of Organization # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports # Health Records
Contract Scope of Services # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Dialectical Behavioral Therapy (DBT). The program was
awarded 30 points because the Primary Service is identified as a Group 5 Service. The specific
Sub-Component Service Type identified is Cognitive Behavioral Therapy. The Primary Service
was identified as this type of service as it is intended to correct faulty cognitions and perceptions
and provides skills individuals can use to monitor thought patterns and correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 5 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 25%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 3 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program’s protocol is to have all therapeutic managers and clinical director trained in
facilitation of Dialectical Behavioral Therapy. During the review period there were five staff trained.
Basic Score: 0 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program provided a manual titled for Dialectical Behavioral Therapy (DBT) and it is divided
into five sections. The manual gives the theories and research for DBT only. The lessons are
divided by the five sections and the handouts are grouped by each section. The manual does not
contain a script, lesson plan, or outline for the lessons. In previous years, the program had the
DBT notebook on site, as well as the student guide book.
The program can earn 2 points by ensuring the manual contains the order of lessons and detailed
lesson plans for each session.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Dialectical Behavioral Therapy group was not observed during this review.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The facility’s policy regarding turnover is to have all therapeutic managers and the clinical director
trained in Dialectical Behavioral Therapy. In the event of staff turnover or a facilitator absence,
another facilitator is able to facilitate the group without disruption.
The reviewed documentation and interviews with the program director confirmed that no staff
turnover adversely effected group.
Basic Score: 0 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
This reviewer met with the program director regarding the internal fidelity forms. The program
director reported that she had requested the SPEP books to review from the clinical director.
According to the program director the clinical director left employment and did not provide the
requested documentation. The internal fidelity monitoring forms have not been located since the
termination of employment in 2017.
This reviewer met with the new clinical director. The plan will be for each of the current facilitators
to conduct internal fidelity monitoring each month until the clinical director is trained in Dialectical
Behavioral Therapy. At that time the clinical director will complete all internal fidelity monitoring for
this group.
The program can earn 2 points by ensuring internal fidelity monitoring is conducted at least
monthly for all facilitators.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 0 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
No internal fidelity monitoring has been completed, therefore no corrective actions were
completed.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
The program can earn 1 point by ensuring there is a plan in place for corrective action based on
fidelity monitoring.
At the time of the review, the program did not meet criteria to earn a score of 1.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The two Dialectical Behavioral Therapy facilitators have received yearly performance evaluations.
The previous clinical director received her yearly performance evaluation prior to ending
employment.
All three yearly performance evaluations were provided to this reviewer. There is not a specific
section on the yearly performance evaluations specifically for group facilitation.
The program can earn 1 point by ensuring all facilitators' performance evaluations contains an
assessment of their delivery of DBT.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service - Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 15 weeks for this type of service. Of the 5 youth in the
sample, 0% (0 of 5) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 45 contact hours for this type of service. Of the 5 youth in the
sample, 100% (5 of 5) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 12 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 48%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 0 youth
Moderate-High = 5 youth
High = 0 youth
Total Youth in Sample = 5 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 0%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 0 youth
Total Youth in Sample = 5 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (5 of 5) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 0% (0 of 5) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 5 20 25%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 12 25 48%
Totals 62 100 62%
This SPEP report evaluates Dialectical Behavioral Therapy (DBT), an intervention delivered at
JoAnn Bridges Academy.
The program scored Low for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 5 total youth sampled, none
included dosage with end dates in the EBS Module. Without end dates, it was impossible to
calculate the number of weeks youth attended.
The program earned 10 points for Amount of Service: Contact Hours. Of the 5 total youth
sampled, all 5 received at least the recommended hours of service. All youth completed exactly
52 hours of service.
The program was awarded 12 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered just prior to the youths' admission.
RECOMMENDATION(S):
JoAnn Bridges Academy can optimize their SPEP Quality of Service Delivery score by ensuring
the manual contains detailed lesson plans, by ensuring internal fidelity monitoring is conducted at
least monthly for all facilitators, by ensuring there is a process for corrective action based on
fidelity monitoring, and by ensuring all facilitators' performance evaluations includes an
assessment of their delivery of DBT.
JoAnn Bridges Academy can optimize their SPEP Amount of Service score by ensuring that data
is entered correctly in the EBS Module of JJIS and by ensuring all youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

service_type: Dialectical Behavior Therapy


summary: SPEP - FY16-17

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
JoAnn Bridges Academy
Rite of Passage
(Contract Provider)
950 SW Greenville Road
Greenville, Florida 32331
Primary Service: VOICES
SPEP Review Date(s): May 16-19, 2017
Florida Department of Juvenile Justice
Report Date(s): 1/16/2018
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: JoAnn Bridges Academy MQI Program Code: 832
Provider Name: Rite of Passage Contract Number: 10361
Location: Madison County / Circuit: 3 Number of Beds: 28
Review Date(s): 5/16/17-5/19/17 Lead Reviewer Code: 141
Persons Interviewed
Program Director Corporate QI/QA staff # Program Supervisors
DJJ Monitor # Case Managers # Youth
DHA or designee 1 # Clinical Staff # Other (listed by title):
DMHCA or designee # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 3 # Personnel Records
Fidelity Monitoring Documents Program Schedules # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts # Youth Records (Closed)
Staff Evaluations Table of Organization # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports # Health Records
Contract Scope of Services # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is VOICES. The program was awarded 25 points because
the Primary Service is identified as a Group 4 Service. The specific Sub-Component Service
Type identified is Group Counseling. The Primary Service was identified as this type of service as
it focuses on psychological or interpersonal problems or issues faced by an individual and
involves a group of youths interacting with each other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 5 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program’s protocol is to have all therapeutic managers and clinical director trained in
facilitation of this curriculum. During the review period there were five staff trained. Two of these
staff are still currently employed at the facility and continue to facilitate VOICES. The training
dates for these two staff are March 14, 2013 and October 4, 2016.
The program provided all training certificates for prior facilitators. All facilitators were trained by a
formal trainer.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program provided a manual titled VOICES Facilitator Guide for this review. This curriculum
also provides a participant journal for the youth to complete during the time they are in group. The
manual provides detailed information for each lesson, to include implementation guidelines. All
needed worksheets and exercises were also included in the manual. There was a guide and
specific outline for each lesson.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
VOICES group was not observed during this review.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The facility’s policy regarding turnover is to have all therapeutic managers and the clinical director
trained in VOICES. In the event of staff turnover or a facilitator absence, another facilitator is able
to facilitate the group without disruption. The reviewed documentation and interviews with the
program director confirmed that no staff turnover adversely effected group.
Basic Score: 0 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
This reviewer met with the program director regarding the internal fidelity forms. The program
director reported that she had requested the SPEP books to review from the clinical director.
According to the program director the clinical director left employment and did not provide the
requested documentation. The internal fidelity monitoring forms have not been located since the
termination of employment in 2017.
Due to the change in primary services in March of 2017, internal fidelity monitoring will not be
taking place for this curriculum.
The program can earn 2 points by ensuring internal fidelity monitoring is conducted at least
monthly for all facilitators.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 0 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
No internal fidelity monitoring has been completed, therefore no corrective actions were
completed.
The program can earn 1 point by ensuring there is a plan for corrective action based on fidelity
monitoring.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The two VOICES facilitators have received yearly performance evaluations. The previous clinical
director received her yearly performance evaluation prior to ending employment.
All three yearly performance evaluations were provided to this reviewer. There is not a specific
section on the yearly performance evaluations specifically for group facilitation.
The program can earn 1 point by ensuring all facilitators' performance evaluations contains an
assessment of their delivery of VOICES.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service - Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 24 weeks for this type of service. Of the 21 youth in the
sample, 5% (1 of 21) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 40 contact hours for this type of service. Of the 21 youth in the
sample, 86% (18 of 21) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 12 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 48%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 95%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 1 youth
Moderate-High = 16 youth
High = 3 youth
Total Youth in Sample = 21 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 14%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 3 youth
Total Youth in Sample = 21 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 95% (20 of 21) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 14% (3 of 21) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 12 25 48%
Totals 60 95 63%
This SPEP report evaluates VOICES, an intervention delivered at JoAnn Bridges Academy.
The program scored Medium for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 21 total youth sampled, only
16 included dosage with end dates in the EBS Module. Of those youth with end dates, one
received at least the recommended weeks of service. Youth in the sample completed between 13
and 25 weeks of service, with an average of 19 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 21 total youth
sampled, 18 received at least the recommended hours of service. Youth in the sample completed
between 26 and 54 hours of service, with an average of 44 hours.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program was awarded 12 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered just prior to the youths' admission.
RECOMMENDATION(S):
JoAnn Bridges Academy can optimize their SPEP Quality of Service Delivery score by ensuring
internal fidelity monitoring is conducted at least monthly for all facilitators, by ensuring there is a
process for corrective action based on fidelity monitoring, and by ensuring all facilitators'
performance evaluations includes an assessment of their delivery of VOICES.
JoAnn Bridges Academy can optimize their SPEP Amount of Service score by ensuring that
dosage is entered correctly in the EBS Module of JJIS and by ensuring youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

service_type: VOICES: A Program of Self-Exploration and Empowerment


summary: SPEP - FY17-18

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Joann Bridges Academy
Rite of Passage
(Contract Provider)
950 SW Greenville Hills Road
Greenville, Florida 32331
Primary Service: Aggression Replacement Training (ART)
SPEP Review Date(s): April 17‐18, 2018
Florida Department of Juvenile Justice
Report Date(s): April 18, 2018
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Joann Bridges Academy MQI Program Code: 832
Provider Name: Rite of Passage Contract Number: 10361
Location: Madison County / Circuit: 3 Number of Beds: 28
Review Date(s): April 17-18, 2018 Lead Reviewer Code: 116
Persons Interviewed
Program Director Corporate QI/QA staff # Program Supervisors
DJJ Monitor 0 # Case Managers 5 # Youth
DHA or designee 1 # Clinical Staff # Other (listed by title):
DMHCA or designee # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks # Personnel Records
Fidelity Monitoring Documents Program Schedules # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts # Youth Records (Closed)
Staff Evaluations Table of Organization # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports # Health Records
Contract Scope of Services # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Aggression Replacement Training. The program was
awarded 30 points because the Primary Service is identified as a Group 5 Service. The specific
Sub-Component Service Type identified is Cognitive Behavioral Therapy. The Primary Service
was identified as this type of service as it is intended to correct faulty cognitions and perceptions
and provides skills individuals can use to monitor thought patterns and correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program had five staff members who have each received formal training from a qualified
trainer to become a group facilitator in the primary service Aggression Replacement Training
(ART). One staff member was trained on ART in August 2017, two staff members trained March
2017, one in November 2016, and one was trained in January 2014. An interview with the case
manager supervisor validated each facilitator successfully completed ART curriculum before
facilitating groups. The ART facilitators received twenty-one hours of training on the ART
curriculum.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
This program provided the Aggression Replacement Training (ART) manual for this reviewer. This
manual was the third edition-revised and expanded by authors Barry Glick and John Gibbs. The
manual was organized by each lesson: skill streaming, moral reasoning, and anger control. Each
lesson is divided by weeks. The book designates what the skill is for each lesson, lists the
objectives that should be met, materials that are needed, and then leads into the lesson. There is
a general script to follow and a procedure for each lesson to use as a guide. The skill steps are
broken into sub-steps for the facilitator to follow and steps the youth needs to complete. There are
no student manuals for ART.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
This reviewer observed an Aggression Replacement Training (ART) session during the annual
compliance review. The facilitator demonstrated knowledge of the content and was comfortable
facilitating a group of youth. The group discussion addressed making accusations and the
facilitator followed the lesson plan. The facilitator used her own words to teach the lesson. The
facilitator challenged thinking errors in a respectful way. Individuals were praised for positive
answers. All group members were involved in the discussion and there were no behavior issues
during group.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The facility’s policy regarding turnover is to have all therapeutic managers and the clinical director
trained in Aggression Replacement Training (ART). In the event of staff turnover or a facilitator
absence, another facilitator is able to facilitate the group without disruption. The reviewed
documentation and interviews with the program director confirmed that no staff turnover adversely
effected group.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program is required to conduct fidelity monitoring. The program provided documentation of
fidelity monitoring for Aggression Replacement Training (ART) for the months of June 2017
through March 2018. The fidelity monitoring for ART was completed by the case managers.
Documentation in training files validated staff members who performed fidelity monitoring were
trained by a qualified trainer. Staff whom are trained in the primary service ART conduct internal
fidelity monitoring at the program by observing another facilitator's group for the entire group
session at least once a week. The program utlizes an ART specific fidelity adnerence checklist.
On the checlist, there is a corrective action follow-up component, which incorporated any
applicable recommendations identified during fidelity montitoring. These signed documents are
maintained in a Standardized Program Evalutation Protocol (SPEP) binder along with the ART
sign-in sheets.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
Three fidelity reports reviewed did not indicate the need for any corrective actions. An interview
with the case manager supervisor found a process for qualified staff to develop a corrective action
based on the fidelity monitoring observations, if applicable. Reviewed documentation found fidelity
monitoring was conducted weekly by trained staff. There were consistent critiques and
recommendations, when applicable, where the fidelity monitor provided constructive feedback to
the facilitator. The facilitator’s signature on the fidelity monitoring checklist document
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
acknowledged the constructive feedback and improvements. The program has a practice in place
to monitor this corrective action during the following fidelity monitoring session.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Three of the Aggression Replacement Training facilitators have received yearly performance
evaluations. The other facilitator has not been employed at the facility long enough to have had a
yearly evaluation.
Each yearly performance evaluation was provided to this reviewer. There is not a specific section
on the yearly performance evaluations specifically for group facilitation. However, the program
provided an updated evaluation template, which will include a section on the performance
evalutation specifically for group facilitation for future evaluations.
The program can earn 1 point by ensuring all facilitators' performance evaluations includes an
assessment of their delivery of ART.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates the target duration of 10 weeks for this type of service. Of the 17 youth in the
sample, 35% (6 of 17) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 30 contact hours for this type of service. Of the 17 youth in the
sample, 100% (17 of 17) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 94%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 2 youth
Moderate-High = 6 youth
High = 8 youth
Total Youth in Sample = 17 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 47%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 8 youth
Total Youth in Sample = 17 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 94% (16 of 17) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 47% (8 of 17) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 2 10 20%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 23 25 92%
Totals 90 100 90%
This SPEP report evaluates Aggression Replacement Training, an intervention delivered at JoAnn
Bridges Academy.
The program scored High for Quality of Service Delivery.
The targeted number of weeks for this specific intervention is 10 weeks; the targeted number of
hours for this specific intervention is 30 hours. After additional review and discussion with the
Peabody Research Insitute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 5 Services (15 weeks; 45
hours)
The program earned 2 points for Amount of Service: Duration. Of the 17 total youth sampled, only
6 included dosage with end dates in the EBS Module. Of those youth with correct dosage, all 6
received at least the recommended weeks of service. Youth in the sample completed 14 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program earned 10 points for Amount of Service: Contact Hours. Of the 17 total youth
sampled, all 17 included dosage in the EBS Module. Of those youth, all 17 received at least the
recommended hours of service. Youth in the sample completed between 30 and 38 hours of
service, with an average of 33 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered just prior to the youths' admission.
RECOMMENDATION(S):
JoAnn Bridges Academy can maintain their SPEP Quality of Service Delivery score by continuing
the practices in place at the time of this review.
JoAnn Bridges Academy can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10


summary: SPEP - FY17-18

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Joann Bridges Academy
Rite of Passage
(Contract Provider)
950 SW Greenville Hills Rd
Greenville, Florida 32331
Primary Service: Dialectical Behvaioral Therapy (DBT)
SPEP Review Date(s): April 19, 2018
Florida Department of Juvenile Justice
Report Date(s): April 18, 2018
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: Joann Bridges Academy MQI Program Code: 832
Provider Name: Right of Passage Contract Number: 10361
Location: Madison County / Circuit: 3 Number of Beds: 28
Review Date(s): April 19, 2018 Lead Reviewer Code: 116
Persons Interviewed
Program Director Corporate QI/QA staff # Program Supervisors
DJJ Monitor 2 # Case Managers # Youth
DHA or designee 1 # Clinical Staff # Other (listed by title):
DMHCA or designee # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 1 # Personnel Records
Fidelity Monitoring Documents Program Schedules # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts # Youth Records (Closed)
Staff Evaluations Table of Organization # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports # Health Records
Contract Scope of Services # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Dialectical Behavior Therapy . The program was awarded
30 points because the Primary Service is identified as a Group 5 Service. The specific Sub-
Component Service Type identified is Cognitive Behavioral Therapy. The Primary Service was
identified as this type of service as it is intended to correct faulty cognitions and perceptions and
provides skills individuals can use to monitor thought patterns and correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program’s practice is to have all therapeutic managers trained to facilitate Dialectic
Behavioral Therapy (DBT). There are currently three staff trained at this residential program. Two
staff members hold the position of therapeutic managers. The third facilitator is the on-site
Licensed Mental Health Counselor (LMHC), and was trained in 2005. The LMHC is also
responsible for internal fidelity monitoring. One therapeutic manager completed training in August
of 2013 and the second completed her training in October of 2016. Two additional therapeutic
managers are pending a training class in Ft. Lauderdale, FL. The program does not have staff
members facilitating or monitoring DBT groups who have not been trained.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The manual this program utilizes was available for review. The manual was a second edition-
Dialectic Behavioral Therapy Skills Training Handbook and Worksheets by author Marshall M.
Linehan. The manual is organized by lessons and corresponding handouts and worksheets. All
lessons are divided by week. The manual designates what the skill is for each lesson, lists the
objectives, and materials needed. Each lesson provides a script to follow and procedure to use as
a guide. The skill steps are broken into sub-steps for the facilitator to follow and steps youth are
required to complete. There are no student manuals for DBT.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The program completed a Dialectical Behavioral Therapy Group (DBT) in March 2018 and is
currently rotating through Aggression Replacement Training (ART). Therefore, this is not
applicable for this review.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
Facility turnover has not been a factor adversely affecting Dialectical Behavioral Therapy (DBT) at
the program. The program’s practices are to send new therapeutic managers, as needed, to Ft.
Lauderdale when DBT training is offered. There are currently two therapeutic managers pending
training for DBT in Ft. Lauderdale, Florida.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The process for internal fidelity monitoring is for one staff member to complete all fidelity
monitoring for Dialectical Behavioral Therapy (DBT). The staff member conducting the internal
fidelity monitoring is trained in DBT. Internal fidelity monitoring reports have been consistently
completed from June 2017 through March 2018. The internal fidelity monitoring reports reflect
they are being completed by the program’s Licensed Mental Health Counselor (LMHC).
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The process for corrective actions is for the person completing the fidelity monitoring to notate any
issues they see on the internal fidelity monitoring form. The facilitator has the opportunity to
respond, in writing, on the same form. Both monitor and facilitator sign and date the sheet. Issues
will be monitored over the next several group meetings to ensure the issue is resolved. If the
issue is not resolved, documentation will continue to be completed on the subsequent monitoring
forms. There have been no instances for a need for corrective action since the last review.
However, there are critiques and suggestions documented on the fidelity monitoring forms. In
three of these instances, it is noted the facilitator refused to sign the fidelity monitoring form.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Three of the Dialectical Behavioral Therapy facilitators have received yearly performance
evaluations. On the current performance evaluation template used, there is not an evaluated area
for group facilitation. During the annual compliance review, the program provided an updated
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
performance evaluation template with an area designated for group facilitation evaluation to be
used in the future.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates the target duration of 15 weeks for this type of service. Of the 11 youth in the
sample, 36% (4 of 11) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates a target of 45 contact hours for this type of service. Of the 11 youth in the
sample, 36% (4 of 11) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 18 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 72%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 91%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 1 youth
Moderate-High = 6 youth
High = 3 youth
Total Youth in Sample = 11 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 27%
High Score: 8 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 62%
Table 2
High = 3 youth
Total Youth in Sample = 11 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 91% (10 of 11) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 27% (3 of 11) youth scored High-Risk to reoffend, for a score of 8 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 2 10 20%
Amount of Service: Contact Hours 2 10 20%
Risk Level of Youth Served 18 25 72%
Totals 77 100 77%
This SPEP report evaluates Dialectical Behavior Therapy, an intervention delivered at JoAnn
Bridges Academy.
The program scored High for Quality of Service Delivery.
The program earned 2 points for Amount of Service: Duration. Of the 11 total youth sampled, only
8 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 4
received at least the recommended weeks of service. Youth in the sample completed between 8
and 17 weeks of service, with an average of 12 weeks.
The program earned 2 points for Amount of Service: Contact Hours. Of the 11 total youth
sampled, all 11 included dosage in the EBS Module. 4 received at least the recommended hours
of service. Youth in the sample completed between 9 and 104 hours of service, with an average
of 43 hours.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program was awarded 18 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered just prior to the youths' admission.
RECOMMENDATION(S):
JoAnn Bridges Academy can maintain their SPEP Quality of Service Delivery score by continuing
the practices in place at the time of this review.
JoAnn Bridges Academy can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10


summary: SPEP - FY17-18

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
JoAnn Bridges Academy
Rite of Passage
(Contract Provider)
950 SW Greenville Road
Greenville, Florida 32331
Primary Service: Voices
SPEP Review Date(s): April 17‐18, 2018
Florida Department of Juvenile Justice
Report Date(s): April 18, 2018
Introduction
The Standardized Program Evaluation Protocol (SPEP) is an assessment tool derived from
meta-analytic research on the effectiveness of juvenile justice interventions. The tool is
designed to compare existing intervention services, as implemented in the field, to the
characteristics of the most effective intervention services found in the research.
The SPEP scoring system allows service providers to identify specific areas in which program
improvements can be made to their existing Primary Services. These improvements can be
expected to increase the effectiveness of those Primary Services in the reduction of recidivism
for youth receiving the Primary Service. A separate SPEP evaluation is conducted, at the time
of the program’s Quality Improvement Review, for each Primary Service provided by the
program.
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Program Name: JoAnn Bridges Academy MQI Program Code: 832
Provider Name: Rite of Passage Contract Number: 10361
Location: Madison County / Circuit: 3 Number of Beds: 28
Review Date(s): April 17-18, 2018 Lead Reviewer Code: 116
Persons Interviewed
Program Director Corporate QI/QA staff # Program Supervisors
DJJ Monitor # Case Managers # Youth
DHA or designee 1 # Clinical Staff # Other (listed by title):
DMHCA or designee # Healthcare Staff
Documents Reviewed
Written Protocol/Manual Logbooks 3 # Personnel Records
Fidelity Monitoring Documents Program Schedules # Training Records/CORE
Internal Corrective Action Reports Supplemental Contracts # Youth Records (Closed)
Staff Evaluations Table of Organization # Youth Records (Open)
Accreditation Reports Youth Handbook # Other:
Contract Monitoring Reports # Health Records
Contract Scope of Services # MH/SA Records
Observations During Review
Group/Session of Primary Service(s)
Program Activities
Recreation
Social Skill Modeling by Staff
Staff Interactions with Youth
Staff Supervision of Youth
Transition/Exit Conferences
Treatment Team Meetings
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is VOICES. The program was awarded 25 points because
the Primary Service is identified as a Group 4 Service. The specific Sub-Component Service
Type identified is Group Counseling. The Primary Service was identified as this type of service as
it focuses on psychological or interpersonal problems or issues faced by an individual and
involves a group of youths interacting with each other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated by the Bureau of Monitoring and Quality Improvement while
on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program’s protocol is to have all therapeutic managers and clinical director trained in
facilitation of this curriculum. During the review period there were four staff trained. Three of these
staff are still currently employed at the facility and continue to facilitate VOICES. The training
dates for these three staff are March 14, 2013, October 4, 2016, January 30, 2018.
The program provided all training certificates for prior facilitators. All facilitators were trained by a
formal trainer.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program provided a manual titled VOICES Facilitator Guide for this review. This curriculum
also provides a participant journal for the youth to complete during the time they are in group. The
manual provides detailed information for each lesson, to include implementation guidelines. All
needed worksheets and exercises were also included in the manual. There was a guide and
specific outline for each lesson.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A group of the primary service VOICES could not be observed during the annual compliance
review week; therefore, this indicator rates as non-applicable.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The facility’s policy regarding turnover is to have all therapeutic managers and the clinical director
trained in VOICES. In the event of staff turnover or a facilitator absence, another facilitator is able
to facilitate the group without disruption. The reviewed documentation and interviews with the
program director confirmed that no staff turnover adversely effected group.
BasicScore: 2Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program is required to conduct fidelity monitoring. The program provided documentation of
fidelity monitoring for VOICES for the months of December 2017 through March 2018. The fidelity
monitoring for VOICES was completed by the case managers reviewing other case managers.
Documentation in training files validated staff members who performed fidelity monitoring were
trained by a qualified trainer. Staff whom are trained in the primary service VOICES conduct
internal fidelity monitoring at the program by observing another facilitator's group for the entire
group session at least once a week. The program utlizes an VOICES specific fidelity adherence
checklist. On the checlist, there is a corrective action follow-up component, which incorporated
any applicable recommendations identified during fidelity montitoring. These signed documents
are maintained in a Standardized Program Evalutation Protocol (SPEP) binder along with the
VOICES sign-in sheets.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The fidelity reports reviewed did not indicate the need for any corrective actions. An interview with
the case manager supervisor found a process for qualified staff to develop a corrective action
based on the fidelity monitoring observations, if applicable. Reviewed documentation found fidelity
monitoring was conducted monthly by trained staff. There were consistent critiques and
recommendations, when applicable, when the fidelity monitor provided constructive feedback to
the facilitator. The facilitator’s signature on the fidelity monitoring checklist document
acknowledged the constructive feedback and improvements. The program has a practice in place
to monitor this corrective action during the following fidelity monitoring session.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Two of the VOICES facilitators have received yearly performance evaluations. The other facilitator
has not been employed at the facility long enough to have had a yearly evaluation.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Both yearly performance evaluations were provided to this reviewer. There is not a specific
section on the yearly performance evaluations specifically for group facilitation. However, the
program provided an updated evaluation template, which will include a section on the
performance evalutation specifically for group facilitation for future evaluations.
The program can earn 1 point by ensuring all facilitators' performance evaluations includes an
assessment of their delivery of VOICES.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service ‐ Duration
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates the target duration of 24 weeks for this type of service. Of the 10 youth in the
sample, 40% (4 of 10) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates a target of 40 contact hours for this type of service. Of the 10 youth in the
sample, 40% (4 of 10) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 2 youth
Moderate-High = 4 youth
High = 4 youth
Total Youth in Sample = 10 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 40%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 4 youth
Total Youth in Sample = 10 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (10 of 10) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 40% (4 of 10) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 4 10 40%
Amount of Service: Contact Hours 4 10 40%
Risk Level of Youth Served 25 25 100%
Totals 83 95 87%
This SPEP report evaluates VOICES, an intervention delivered at JoAnn Bridges Academy.
The program scored High for Quality of Service Delivery.
The program earned 4 points for Amount of Service: Duration. Of the 10 total youth sampled, all
10 included dosage with end dates in the EBS Module. Of those youth, 4 received at least the
recommended weeks of service. Youth in the sample completed between 11 and 31 weeks of
service, with an average of 21 weeks.
The program earned 4 points for Amount of Service: Contact Hours. Of the 10 total youth
sampled, all 10 included dosage in the EBS Module. Of those youth, 4 received at least the
recommended hours of service. Youth in the sample completed between 22 and 52 hours of
service, with an average of 37 hours.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered just prior to the youths' admission.
RECOMMENDATION(S):
JoAnn Bridges Academy can maintain their SPEP Quality of Service Delivery score by continuing
the practices in place at the time of this review.
JoAnn Bridges Academy can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10