summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
JoAnn Bridges Academy
Rite of Passage
(Contract Provider)
950 SW Greenville Hills Road
Greenville , Florida 32331
Primary Service: Impact of Crime
SPEP Review Period: April 1, 2021 - March 31, 2022
SPEP Review Conducted: April 12, 2022
Florida Department of Juvenile Justice
Report Date: 7/20/22
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes the Impact of Crime (IOC) curriculum as a primary service. The coach
counselors are trained to conduct IOC groups. There were eight trained staff during the annual
review period; one shift supervisor (6/23/2017), one director of student services (8/14/2009), and
six coach counselors (1/2/2014, three were trained 4/9/2021, 11/6/2020, and 1/12/2021) trained in
the IOC curriculum.
All training records were provided for review and are maintained in the SPEP binder for the IOC
curriculum. Of the eight staff, four were the main faciltiators during the annual review period. The
shift supervisor conducts internal fidelity monitoring observations for IOC groups.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program recently received the updated Impact of Crime (IOC) manual for facilitators. Prior to
this, the program had the most recent currciulum to utilize for the IOC groups. The manual has
guidelines for how Impact of Crime should be delivered and a workbook for the students. The
curriculum provides a script and instructions for seven chapters, which requires a minimum of
thirty-two (32) contact hours to facilitate. The curriculum includes implementation guidelines,
identifies the order the lessons are to be delivered, and provides detailed instructions for the
delivery of each lesson.
Impact of Crime groups are held on Sundays and Tuesdays in the evenings. All sign-in sheets
were provided for review. This review confrimed the groups are being held on the scheduled
dates and times.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Documentation for this curriculum is maintained in a binder kept in the shift supervisor's office.
The binder includes all sign-in sheets organized by cohorts, internal fidelity monitoring forms, and
all training records.
All youth are eligible to attend IOC groups while in the program. Impact of Crime groups are held
as closed groups. Youth may not enter into a cohort once the first lesson has been completed.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A virtual observation was conducted on Tuesday, February 8, 2022 from 5:30pm to 6:30pm.
There were four youth, a facilitator, and one staff present. The facilitator provided the observer a
copy of the lesson to follow, which was Chapter four, exercise ten. This exercise is for a victim
speaker, however the program was not allowing visitors at the time of this group, therefore a video
was chosen. The facilitator was well prepared; she had a video ready to play for the group and
was comfortable teaching the lesson. There were no behavioral issues during the group. The
youth were quiet during the video and had a good discussion regarding the video and relating it to
the lesson. The facilitator was able to utilize her own words and followed up with the youth to
ensure they understood the concepts before moving on.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program's policy to address turnover is for another trained faciltiator to conduct Impact of
Crime (IOC) groups, if not being co-facilitated, to continue with group facilitation. The program did
experience turnover with Impact of Crime (IOC) facilitators. Three trained coach counselors left
during the annual review period; one in March 2021 and two in June 2021. One coach counselor
left during a group she was facilitating. This group was not being co-facilitated. In this situation
another trained facilitator, who was not conducting another group at this time, continued
facilitation with this group.
All sign-in sheets were provided for the annual review period. A review of the sign-in sheets
confirmed there were no gaps in service during the annual review period. The program completed
five cohorts of the IOC curriculum during the annual review period.
The first cohort started January 6, 2021 and ended on May 4, 2021. The group started with six
youth and five youth successfully completed . One youth was released from the program early
and did not successfully complete the group.
The second cohort started April 20, 2021 and ended on June 27, 2021. The group started with
seven youth and all seven youth successfully completed. This cohort had shortened weeks. This
reviewer asked the director of student services why the groups were being held for two hours and
it was reported it was due to identifying some youth had release dates ending before the class
ending date. This situation was discussed during last year's annual SPEP call with the program.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The third cohort was started on July 6, 2021 and ended on September 21, 2021. The group
started with seven youth and ended with seven youth successfully completing. Twelve of these
groups were held for two hours in length and one group was held for one and a half hours. This
group will be short on weeks as well.
The fourth cohort started September 28, 2021 and ended on January 23, 2022. The group started
with four youth and four youth successfully completed.
The fifth cohort started December 5, 2021 and ended on March 29, 2022. The group started with
four youth and four youth successfully completed.
A new cohort started on April 5, 2022 with four youth. These sign-in sheets were reviewed. The
group is held on Sundays and Tuesdays from 4:30pm to 5:30pm.
The program can earn 2 points if facilitator turnover does not impact groups.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The shift supervisor conducts the internal fidelity monitoring observations for the Impact of Crime
(IOC) groups. There were four main facilitators for IOC groups during the annual review period.
Twenty-one (21) out of twenty-two (22) internal fidelity monitoring observations were completed
during the twelve month review period. This is based upon when facilitators conducted cohorts.
One facilitator was missing one month of observation.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The shift supervisor or director of student services addresses any significant issues or corrective
actions for the facilitators for the Impact of Crime (IOC) groups. The internal fidelity forms have a
section to document issues with facilitators. A review of the internal fidelity forms confirmed there
were no documented significant issues or corrective actions with the IOC facilitators during the
annual review period.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program's annual performance evaluation includes an addendum for group facilitation. Three
facilitators and the shift supervisor had their annual performance evaluations completed at the
time of this review. All four evaluations included the addendums for group facilitation. All
addendums included the specific curriculum being evaluated, including Impact of Crime.
Three of the facilitators had left employment and had not received an annual performance
evaluation.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service - Duration
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates the target duration of 16 weeks for this type of service. Of the 24 youth in the
sample, 42% (10 of 24) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 24 youth in the
sample, 100% (24 of 24) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 7 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 28%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 83%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 7 youth
Moderate-High = 12 youth
High = 1 youth
Total Youth in Sample = 24 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 4%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 1 youth
Total Youth in Sample = 24 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 83% (20 of 24) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 4% (1 of 24) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 4 10 40%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 7 25 28%
Totals 61 85 72%
This SPEP report evaluates Impact of Crime, an intervention delivered at JoAnn Bridges
Academy.
25 youth completed the program during the review period. Of those youth, 24 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 4 points for Amount of Service: Duration. Of the 24 total youth sampled, all
24 included dosage with end dates in the EBS Module. Of those youth, 10 received at least the
recommended weeks of service. Youth in the sample completed between 10 and 18 weeks of
service, with an average of 14 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program earned 10 points for Amount of Service: Contact Hours. Of the 24 total youth
sampled, all 24 included dosage in the EBS Module. Of those youth, 24 received at least the
recommended hours of service. Youth in the sample completed between 32 and 56 hours of
service, with an average of 39 hours.
The program was awarded 7 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 93%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
JoAnn Bridges Academy can optimize their SPEP Quality of Service Delivery score by ensuring
facilitator turnover does not impact groups and fidelity monitoring is completed every month for
each facilitator.
JoAnn Bridges Academy can optimize their SPEP Amount of Service score by ensuring dosage
for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted dosage
of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

file_name: FY2021-22 JBA IOC.pdf


state: FL

facility_name: JoAnn Bridges Academy

summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
JoAnn Bridges Academy
Rite of Passage
(Contract Provider)
950 SW Greenville Hills Road
Greenville, Florida 32331
Primary Service: Aggression Replacement Training
SPEP Review Period: April 1, 2022 - March 31, 2023
SPEP Review Conducted: May 10, 2023
Florida Department of Juvenile Justice
Report Date: 9/22/2023
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Aggression Replacement Training.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes Aggression Replacement Training (ART) as a primary service for all youth in
the program. There is a formal training protocol for this service. Therapy staff are formally trained
to facilitate ART groups. At the time of the annual review, four staff were trained- three therapists
and the clinical director. There are two primary facilitators for ART; the clinical director conducts
internal fidelity monitoring.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program provided the Aggression Replacement Training (ART), 3rd edition, manual for
review. The manual was organized by each lesson: skill streaming, moral reasoning, and anger
control. Each lesson is divided by weeks, for a total of ten (10) weeks and thirty (30) hours.
Aggression Replacement Training does not have separate manuals or workbooks for the youth.
ART groups are held on Mondays, Wednesdays, and Fridays from 9:30am to 10:30am.
Documentation for the ART groups is maintained in a binder kept in the clinical director's office.
The binder includes all sign-in sheets organized by cohort and all internal fidelity monitoring
forms. All youth are eligible to attend ART groups while in the program. Aggression Replacement
Training groups are closed, meaning once the first lesson has been completed, no youth may be
added to the group.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
An observation of Aggression Replacement Training (ART) occurred on November 16, 2022.
There were eight youth, one facilitator, the clinical director, and one coach counselor in
attendance. The group was on lesson week 26, Rehearsal of Full Sequence. The facilitator had a
copy of the lesson to follow but used her own words throughout the lesson. The facilitator has
conducted multiple ART groups and is comfortable with the material. The group reviewed each
part of the anger control cycle. All youth participated in role-plays. The facilitator and youth
provided appropriate feedback and ensured the youth understood the steps. The lesson lasted
one full hour.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program's protocol in the event of facilitator turnover for Aggression Replacement Training
(ART) is for another trained staff to continue with group facilitation. The ART groups are not co-
facilitated at the program. There were no gaps in service during the review period.
The program completed four cohorts during the review period, and one was ongoing at the time of
the annual review.
The first cohort began April 11, 2022, and was completed on June 17, 2022. The group began
and ended with four youth. The second cohort started June 27, 2022, and ended on September
2, 2022. The group started and ended with seven youth. The third cohort began on September 19,
2022 and ended on November 25, 2022. The group began and ended with eight youth.
The fourth cohort began December 28, 2022 with six youth and ended March 10, 2023 with four
youth. Two youth were released just before finishing the service: one finished lesson 28 and one
finished lesson 29. The cohort transitioned to a new facilitator at lesson five.
A fifth cohort began on March 27, 2022 with seven youth and was ongoing at the time of this
review.
A review of the sign-in sheets revealed three extra sessions are held after the completion of the
last lesson. This clinical director indicated the facilitators for ART hold three extra sessions, two
for review of the curriculum, answer questions, etc. The last session the youth take a test for the
clinical staff to review and ensure the youth have fully understood the curriculum.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The clinical director conducts internal fidelity monitoring for Aggression Replacement Training
(ART) facilitators. Each facilitator is observed three times a month, once for each section of ART.
Two main facilitators conducted ART groups during the review period. The other two trained staff
will fill in to facilitate ART groups when the main facilitator is not present.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
All internal fidelity monitoring forms were provided for review; there were no missing internal
fidelity monitoring observations for the review period.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The clinical director addresses any significant issues or corrective actions for the facilitators of
Aggression Replacement Training (ART). The internal fidelity forms provide a section to document
any significant issues or corrective actions for the facilitators in ART. All internal fidelity monitoring
forms include feedback from the facilitator and the observer.
A review of the fidelity forms confirmed there were no corrective actions noted nor taken during
the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Annual performance evaluations include an addendum for an evaluation of facilitator skill in
service delivery. Each facilitator eligible for an annual evaluation was rated on their skill in group
facilitation, including Aggression Replacement Training.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 10 weeks for this type of service. Of the 13 youth in the
sample, 100% (13 of 13) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 30 contact hours for this type of service. Of the 13 youth in the
sample, 100% (13 of 13) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 5 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 20%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 54%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 3 youth
Moderate-High = 4 youth
High = 0 youth
Total Youth in Sample = 13 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 0%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 0 youth
Total Youth in Sample = 13 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 54% (7 of 13) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 0% (0 of 13) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 5 25 20%
Totals 80 100 80%
This SPEP report evaluates Aggression Replacement Training, an intervention delivered at JoAnn
Bridges Academy.
13 youth completed the program during the review period. Of those youth, all 13 were included in
the sample.
The program scored High for Quality of Service Delivery.
The targeted number of weeks for this specific intervention is 10 weeks; the targeted number of
hours for this specific intervention is 30 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 5 services (15 weeks; 45
hours).
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 10 points for Amount of Service: Duration. Of the 13 total youth sampled, all
13 included dosage with end dates in the EBS Module. Of those youth, 13 received at least the
recommended weeks of service. Youth in the sample completed between 10 and 11 weeks of
service, with an average of 10 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 13 total youth
sampled, all 13 included dosage in the EBS Module. Of those youth, 13 received at least the
recommended hours of service. Youth in the sample completed 30 hours of service.
The program was awarded 5 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 100%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
JoAnn Bridges Academy can maintain their SPEP Quality of Service Delivery score by continuing
the practices in place at the time of this review.
JoAnn Bridges Academy can optimize their SPEP Amount of Service score by ensuring dosage
for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted dosage
of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

source: DJJ

report_type: SPEP

file_name: FY2022-23 JBA ART.pdf

program_name_raw: JoAnn Bridges Academy

program_status_note: Open

service_type: Aggression Replacement Training


summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
JoAnn Bridges Academy
Rite of Passage
(Contract Provider)
950 SW Greenville Hills Road
Greenville, Florida 32331
Primary Service: Impact of Crime
SPEP Review Period: April 1, 2022 - March 31, 2023
SPEP Review Conducted: May 10, 2023
Florida Department of Juvenile Justice
Report Date: 9/22/2023
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes Impact of Crime (IOC) as a primary service. The program trains coach
counselors to conduct IOC groups. There are five trained staff currently in the program: one
director of student services, one shift supervisor, and three coach counselors. The three coach
counselors have facilitated groups during the review period. The director of student services or
the shift supervisor conduct internal fidelity monitoring for IOC facilitators.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the updated Impact of Crime (IOC) manual for facilitators. The manual
includes guidelines for how Impact of Crime should be delivered and a workbook for the students.
The curriculum provides a script and instructions for seven chapters, which requires a minimum of
thirty-two (32) contact hours to facilitate. The curriculum includes implementation guidelines,
identifies the order the lessons are to be delivered, and provides detailed instructions for the
delivery of each lesson.
Documentation for this curriculum is maintained in a binder kept in the shift supervisor's office.
The binder includes all sign-in sheets organized by cohorts, internal fidelity monitoring forms, and
all training records. Impact of Crime groups are held on Sundays and Tuesdays in the evenings.
All youth are eligible to attend IOC groups while in the program. Impact of Crime groups are held
as closed groups, meaning youth may not enter a cohort once the first lesson has been
completed.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
An observation of Impact of Crime (IOC) was conducted on Tuesday, January 31, 2023. There
were four youth and one facilitator in attendance. The youth were on chapter four and completed
exercises one and two. The youth completed the role play of Damien's Dilemma. The facilitator
provided feedback and led a discussion with the group about the situation. All youth participated
in the discussions for all the exercises.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program's policy for facilitator turnover is for another trained staff member to continue with
the group. The program did not experience turnover during the review period. Five cohorts were
completed, and one was ongoing at the time of this annual review.
The first cohort began on April 5, 2022 with four youth and ended on July 12, 2022 with four youth
started the group. The second cohort began on May 17, 2022 with four youth and ended on
October 11, 2022 with four youth. The third cohort began July 10, 2022 with five youth and ended
on November 1, 2022 with five youth.
The fourth cohort was started on September 20, 2022 with six youth and ended February 12,
2023 with five youth. One youth who started the group was transferred from another program and
discharged on December 19, 2022. There was a gap in service from December 27, 2022 to
January 22, 2023. The gap was due to the facilitators and youth contracting COVID19 and does
not qualify as an unnecessary gap in service as the program followed department guidance for
service delivery.
The fifth cohort was started on November 29, 2022 with five youth and was completed April 4,
2023 with three youth. Two youth were transferred out of the program prior to completing the
service. There was a gap in service from January 1, 2023 to January 22, 2023. This was also due
to the staff and youth being ill. As with the previously listed cohort, this is not an unnecessary gap
in service.
The sixth cohort start February 7, 2023 with five youth and was ongoing at the time of this review
with all five youth.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The director of student services or the shift supervisor conducts internal fidelity monitoring for the
Impact of Crime groups. There were three main facilitators during the review period. The shift
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
supervisor did conduct some groups to assist if a facilitator was out. The shift supervisor also
conducts some groups to stay current in the curriculum. When this happens, the director of
student services observes those groups.
A review of documentation confirmed each facilitator was monitored each month they delivered
the service, with only one exception. One internal fidelity monitoring form missing for the month of
October for one facilitator.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program's policy is for the director of student services to address any significant issues or
corrective actions for the facilitators of Impact of Crime (IOC). There is a section to document
issues or corrective actions on the internal fidelity form. All fidelity forms were provided for review
and there were no significant issues or corrective actions noted during the review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program's annual performance evaluation includes an addendum for group facilitation. Each
eligible facilitator had their annual performance evaluations completed at the time of this review,
which included the addendum for group facilitation and included Impact of Crime.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 16 weeks for this type of service. Of the 13 youth in the
sample, 92% (12 of 13) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 13 youth in the
sample, 100% (13 of 13) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 5 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 20%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 54%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 3 youth
Moderate-High = 4 youth
High = 0 youth
Total Youth in Sample = 13 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 0%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 0 youth
Total Youth in Sample = 13 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 54% (7 of 13) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 0% (0 of 13) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 5 25 20%
Totals 63 85 74%
This SPEP report evaluates Impact of Crime, an intervention delivered at JoAnn Bridges
Academy.
13 youth completed the program during the review period. Of those youth, 13 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 13 total youth sampled, all
13 included dosage with end dates in the EBS Module. Of those youth, 12 received at least the
recommended weeks of service. Youth in the sample completed between 13 and 22 weeks of
service, with an average of 17 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 10 points for Amount of Service: Contact Hours. Of the 13 total youth
sampled, all 13 included dosage in the EBS Module. Of those youth, 13 received at least the
recommended hours of service. Youth in the sample completed between 26 and 32 hours of
service, with an average of 32 hours.
The program was awarded 5 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 98%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
JoAnn Bridges Academy can optimize their SPEP Quality of Service Delivery score by ensuring
fidelity monitoring is completed every month for each facilitator.
JoAnn Bridges Academy can optimize their SPEP Amount of Service score by ensuring dosage
for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted dosage
of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2022-23 JBA IOC.pdf

service_type: Impact of Crime


summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
JoAnn Bridges Academy
Rite of Passage
(Contract Provider)
950 SW Greenville Hills Road
Greenville, Florida 32331
Primary Service: Seeking Safety
SPEP Review Period: April 1, 2022 - March 31, 2023
SPEP Review Conducted: May 10, 2023
Florida Department of Juvenile Justice
Report Date: 9/22/2023
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Seeking Safety.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes Seeking Safety as a primary service. The therapeutic staff are trained in
Seeking Safety by the regional clinical director. The clinical director provided the training outline
for Seeking Safety. All training records were provided for review. During the review period, there
were four trained staff: one clinical director and three therapeutic managers. One therapeutic
manager is no longer employed at the program. The clinical director conducts internal fidelity
monitoring.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the current Seeking Safety manual. The manual includes all lessons and
materials needed to conduct the Seeking Safety groups.
Seeking Safety groups are held on Tuesdays and Thursdays from 9:30am to 10:30am.
Documentation for Seeking Safety groups is maintained in a binder kept in the clinical director's
office. The binder includes all sign-in sheets and fidelity monitoring forms. All youth are eligible to
attend Seeking Safety groups while in the program. Seeking Safety is conducted as an open
group, meaning they are continuously held, and youth may enter the group upon their arrival.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
An observation of Seeking Safety was conducted on November 22, 2022 from 8:30am to 9:30am.
In attendance were thirteen (13) youth, the clinical director, and one coach counselor. The group
was on the lesson Creating Meaning. The facilitator was comfortable with the lesson and the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
youth. She did adhere to the manual, but was able to put the lesson into her own words. The
facilitator ensured the youth understood the concepts before moving on.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program's protocol in the event of facilitator turnover for Seeking Safety facilitators, is for
another trained facilitator to continue with the group. Seeking Safety is held as an open group and
is run year-round. The program experience turnover during the review period; however, this
turnover did not adversely affect service delivery. All trained facilitators rotate facilitation of the
groups weekly, so youth are able to build rapport with each.
There was one gap in service of ten (10) days in January, due to staff and youth contracting
COVID19. Services were provided outside of the group setting, in accordance with guidance
provided by the department. This does not qualify as an unnecessary gap in service.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The clinical director conducts internal fidelity monitoring observations for Seeking Safety groups.
All internal fidelity monitoring forms were provided for review. Internal fidelity monitoring was
missing for the Clinical Director for the months of January and March 2023. The Clinical Director
was covering groups for facilitators out on sick leave and did not have access to another trained
facilitator eligible to conduct internal fidelity monitoring.
The fidelity form for July 28, 2022 did not have any numbers circled or highlighted. The rest of the
form was completed and signed by the facilitator and the observer.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The clinical director addresses any significant issues or corrective actions for facilitators of
Seeking Safety. The internal fidelity monitoring forms include a section to document any
significant issues or corrective actions. All fidelity forms included feedback for the facilitators.
A review of the fidelity monitoring forms confirmed there were no corrective actions noted nor
taken during the review period.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Annual performance evaluations include an addendum for an evaluation of facilitator skill in
service delivery. Each facilitator eligible for an annual evaluation was rated on their skill in group
facilitation, including Seeking Safety.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 24 weeks for this type of service. Of the 13 youth in the
sample, 0% (0 of 13) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 40 contact hours for this type of service. Of the 13 youth in the
sample, 92% (12 of 13) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 5 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 20%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 54%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 3 youth
Moderate-High = 4 youth
High = 0 youth
Total Youth in Sample = 13 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 0%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 0 youth
Total Youth in Sample = 13 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 54% (7 of 13) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 0% (0 of 13) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 5 25 20%
Totals 63 95 66%
This SPEP report evaluates Seeking Safety, an intervention delivered at JoAnn Bridges Academy.
13 youth completed the program during the review period. Of those youth, all 13 were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 13 total youth sampled, all
13 included dosage with end dates in the EBS Module. Of those youth, 0 received at least the
recommended weeks of service. Youth in the sample completed between 14 and 20 weeks of
service, with an average of 17 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 13 total youth
sampled, all 13 included dosage in the EBS Module. Of those youth, 12 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
recommended hours of service. Youth in the sample completed between 31 and 48 hours of
service, with an average of 41 hours.
The program was awarded 5 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 87%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
JoAnn Bridges Academy can optimize their SPEP Quality of Service Delivery score by ensuring
fidelity monitoring is completed every month for each facilitator.
JoAnn Bridges Academy can optimize their SPEP Amount of Service score by ensuring dosage
for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted dosage
of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2022-23 JBA SS.pdf

service_type: Seeking Safety