summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
JoAnn Bridges Academy
Rite of Passage, Inc
(Contract Provider)
950 Southwest Greenville Hills Road
Greenville, Florida 32331
Primary Service: Aggression Replacement Training
SPEP Review Period: April 1, 2020 to March 31, 2021
SPEP Review Conducted: April 14, 2021 to April 19, 2021
Florida Department of Juvenile Justice
Report Date: 4/19/2021
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Aggression Replacement Training.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes Aggression Replacement Training (ART) as a primary service. The
program's documentation and contract reflected therapeutic staff are trained to conduct ART
groups.
During the twelve month review period, four staff were trained in the curriculum; one clinical
director and three therapeutic managers. All four staff have conducted groups at some point
during the twelve month review period. According to the clinical director, the therapeutic
managers typically rotate conducting group. However, there is one therapeutic manager who
volunteers to conduct more cohorts. All training records for the four trained staff were provided
and reviewed. All staff were formally trained in the ART curriculum.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program provided the Aggression Replacement Training (ART), 3rd edition, manual for
review. The manual was organized by each lesson: skill streaming, moral reasoning, and anger
control. Each lesson is divided by weeks, for a total of ten weeks and thirty hours.
Aggression Replacement Training does not have separate manuals for the youth. All worksheets
are provided within the facilitator manual.
Aggression Replacement Training is currently held on Mondays, Wednesdays, and Fridays from
9:15am to 10:15am. Documentation for ART is maintained in a binder in the clinical director's
office. A review of the program's contract confirmed all youth are eligible to attend ART groups.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Aggression Replacement Training was observed on Monday, April 5, 2021 from
9:15am to 10:15am. Present for group were four youth, the facilitator, the clinical director, and one
direct care staff. The facilitator used helpful visual aides during group, and the youth used their
"bubble talk" visual aids. The lesson for the day was "Dealing with an Accusation". The youth
were asked to participate in two role-plays. Each youth and the direct care staff volunteered for a
part in the activity. The youth were able to choose topics they were passionate about, which led to
great discussion between each other, the facilitator, and direct care staff. The facilitator was
knowledgeable of the material and followed the lesson plan, trying back each participant's actions
in the role-plays to the lesson at hand.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The reviewed documentation and an informal interview with the clinical director confirmed there
has been no turnover which adversely affected the youth during the past twelve months of the
review period. Should the program experience any turnover, their policy is for another trained staff
to continue with the cohort being held.
The therapeutic managers typically rotate conducting Aggression Replacement Training (ART)
groups to avoid gaps in service and cause minimal disruption to the youth in the groups.
There have been four completed cohorts of ART during the twelve month review period.
The first cohort was started May 4, 2020 and was completed on July 10, 2020. Four youth started
in this group and four youth successfully completed the group. A second cohort was also started
on May 4, 2020 and was completed on July 10, 2020. Two youth started in this group and two
youth successfully completed the group.
A third cohort was started on August 31, 2020 and was completed on November 11, 2020. Four
youth started this group and four youth successfully completed this group. A fourth cohort was
started on November 30, 2020 and was completed on February 5, 2021. Eight youth started this
group and eight youth successfully completed this group.
A fifth cohort was started on February 22, 2021 and is continuing to meet. Four youth began in
this group and four youth continue in this group at the time of this review.
There was a small gap in service between July 10, 2020 and August 31, 2020. This was due to a
low census in the program. This does not qualify as an unnecessary gap in service. According to
the compliance manager, all youth who were eligible for this curriculum received the group.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has an in-house process to complete internal fidelity monitoring. The clinical director
conducts internal fidelity monitoring three times a month, one fidelity monitoring for each section
of Aggression Repalcement Training (ART) per facilitator. The facilitator also takes an extra step
of providing a "mid-term" and "post-test" to the youth in an effort to self-monitor her proficeiency in
teaching youth the materal.
There is one main facilitator for ART, but three facilitators conducted some of the groups during
the twelve month review period. Twenty-eight out of twenty-eight internal fidelity monitoring forms
were provided for review. There were no missing forms based upon when groups were held.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program maintains a process for any issues or corrective actions which may occurr during
fidelity monitoring. The clinical director addresses these issues or corrective actions with the
facilitator of the group and the facilitator is monitored for a second time. The issue is also
documented on the internal fidelity monitoring form.
A review of the internal fidelity monitoring forms showed no corrective actions were noted during
the twelve month review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of each facilitators' employee performance evaluations found documentation of group
facilitation performance in the evidence-based curriculum of Aggression Replacement Training
(ART). This section included comments on any strengths or weaknesses of which the facilitator
should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 10 weeks for this type of service. Of the 19 youth in the
sample, 100% (19 of 19) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 30 contact hours for this type of service. Of the 19 youth in the
sample, 100% (19 of 19) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 5 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 8%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 58%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 2 youth
Moderate-High = 8 youth
High = 1 youth
Total Youth in Sample = 19 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 5%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 1 youth
Total Youth in Sample = 19 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 58% (11 of 19) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 5% (1 of 19) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 5 25 8%
Totals 80 100 80%
This SPEP report evaluates Aggression Replacement Training, an intervention delivered at JoAnn
Bridges Academy.
19 youth completed the program during the review period. Of those youth, all 19 were included in
the sample.
The program scored High for Quality of Service Delivery.
The targeted number of weeks for this specific intervention is 10 weeks; the targeted number of
hours for this specific intervention is 30 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 5 services (15 weeks; 45
hours).
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 10 points for Amount of Service: Duration. Of the 19 total youth sampled, all
19 included dosage with end dates in the EBS Module. Of those youth, 19 received at least the
recommended weeks of service. Youth in the sample completed between 10 and 13 weeks of
service, with an average of 10 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 19 total youth
sampled, all 19 included dosage in the EBS Module. Of those youth with correct dosage, 19
received at least the recommended hours of service. Youth in the sample completed 30 hours of
service.
The program was awarded 5 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 100%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
JoAnn Bridges Academy can maintain their SPEP Quality of Service Delivery score by continuing
the practices in place at the time of this review.
JoAnn Bridges Academy can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9


state: FL

facility_name: JoAnn Bridges Academy

summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
JoAnn Bridges Academy
Rite of Passage, Inc
(Contract Provider)
950 Southwest Greenville Hills Road
Greenville, Florida 32331
Primary Service: Impact of Crime
SPEP Review Period: April 1, 2020 to March 31, 2021
SPEP Review Conducted: April 14, 2021 to April 19, 2021
Florida Department of Juvenile Justice
Report Date: 4/19/2021
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes the Impact of Crime (IOC) curriculum as a primary service. The program's
policy has documentation the coach counselors are trained to conducted IOC groups.
There have been four total staff trained during this twelve month review period; three coach
counselors and one supervisor. All three coach counselors have conducted IOC cohorts during
the twelve month review period. All training records were provided for review and there were no
issues. Staff was formally trained in the IOC curriculum.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the Impact of Crime; Addressing the Harm curriculum, version 2.0, updated
September 2011. The program has a manual detailing how Impact of Crime should be delivered,
and a workbook for the students. The curriculum provides a script and instructions for seven
chapters, which requires a minimum of thirty-two contact hours to facilitate. The curriculum
includes implementation guidelines, identifies the order the lessons are to be delivered, and
provides detailed instructions for the delivery of each lesson. Documentation for this service is
maintained in a binder specific to this intervention. Included in this binder are sign in sheets,
fidelity monitoring checklists, and staff training records. The binder is maintained in the director of
student services' office.
Impact of Crime groups are held Sundays and Tuesdays. A review of the program's contract
confirmed all youth are eligible to attend Impact of Crime groups.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Impact of Crime was observed on Sunday, April 11, 2021 from 6:30pm to 7:30pm.
Present for group were five youth and the facilitator. The group was working through chapter six,
Managing Conflict, and took turns recapping the previous lesson and presenting their homework.
The facilitator played Dr. King's "I have a dream" speech and the youth discussed how the speech
is still relevant today. The group then began Student Activity three, which the youth were assigned
to complete as homework.
The facilitator was observed utilizing the most current (2020) version of Impact of Crime. She was
knowledgeable of the material and followed the lesson plan. The youth were distracted by the
observer's presence and the facilitator worked to ensure the youth remained on task and
understood the material.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The director of student services provided information there was turnover during this twelve month
review period for Impact of Crime (IOC) facilitators. One trained coach counselor left; however,
this did not disrupt the groups. The coach counselor was able to complete the cohort prior to
leaving employment.
In the event an employee leaves during a cohort, another trained facilitator would continue with
the group. The youth are familiar with all coach counselors and this would minimize group
disruption and avoid gaps in service.
The program completed three cohorts of IOC during the twelve month review period. According to
the director of student services, any youth who is scheduled to leave prior to an IOC group being
completed will receive one-to-one sessions in order for the youth to be able to complete the
required hours for IOC sessions.
The first cohort started on January 26, 2020 and was completed on July 5, 2020. Seven youth
began the group and four youth completed the group. Three youth were released prior to
completion. One-to-one session sign-in sheets were not provided for the three youth who were
released prior to completion.
The second cohort started on May 24, 2020 and was completed on September 20, 2020. This
group started and ended with eight youth.
The third cohort started October 22, 2020 and was completed on February 21, 2021. This group
started with five youth and ended with four youth. One youth was released prior to the group
being completed. The program provided one-to-one sessions with this youth starting December
22, 2020 through January 5, 2021, the youth's release date.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
A fourth cohort was started on January 6, 2021 and continues to meet at this time. Six youth
started in this group and there are currently five youth in the group at the time of this review. One
youth was released early. The program provided one-to-one sessions with this youth prior to the
release date.
There were no significant gaps in service during this review period.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program provided all twelve months of internal fidelity monitoring forms for this review. The
staff supervisor conducts all internal fidelity monitoring for the Impact of Crime (IOC) curriculum.
The program's policy is for each faciltator to be monitorined internally one time a month.
There were three coach counselors who conducted IOC groups during the twelve month review
period. Twenty-four out of twenty-four internal fidelity monitoring forms were provided for review.
All internal fidelity monitoring forms provided feedback for the facilitator of the group.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program's policy is for the staff supervisor to address any issues or corrective actions for
facilitators of the Impact of Crime (IOC) curriculum. The internal fidelity monitoring forms have a
place to document any issues or corrective actions noted during a group observation.
A review of the internal fidelity monitoring forms for the twelve month review period showed there
were no issues or corrective actions noted for the IOC facilitators.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of three facilitators' employee performance evaluations found documentation of group
facilitation performance in the evidence-based curriculum of Impact of Crime. This section
included comments on any strengths or weaknesses of which the facilitator should be aware.
Two out of the three facilitators received an annual performance evaluation. One facilitator left
employment prior to an annual performance evaluation being conducted. The supervisor also
received an annual performance evaluation and it was provided for review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 16 weeks for this type of service. Of the 19 youth in the
sample, 63% (12 of 19) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 24 contact hours for this type of service. Of the 19 youth in the
sample, 89% (17 of 19) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 5 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 8%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 58%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 2 youth
Moderate-High = 8 youth
High = 1 youth
Total Youth in Sample = 19 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 5%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 1 youth
Total Youth in Sample = 19 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 58% (11 of 19) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 5% (1 of 19) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 5 25 8%
Totals 59 85 69%
This SPEP report evaluates Impact of Crime, an intervention delivered at JoAnn Bridges
Academy.
19 youth completed the program during the review period. Of those youth, all 19 were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 6 points for Amount of Service: Duration. Of the 19 total youth sampled, all
19 included dosage with end dates in the EBS Module. Of those youth, 12 received at least the
recommended weeks of service. Youth in the sample completed between 11 and 24 weeks of
service, with an average of 17 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 8 points for Amount of Service: Contact Hours. Of the 19 total youth
sampled, all 19 included dosage in the EBS Module. Of those youth, 17 received at least the
recommended hours of service. Youth in the sample completed between 20 and 34 hours of
service, with an average of 31 hours.
The program was awarded 5 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 93%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
JoAnn Bridges Academy can maintain their SPEP Quality of Service Delivery score by continuing
the practices in place at the time of this review.
JoAnn Bridges Academy can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

source: DJJ

report_type: SPEP

program_name_raw: JoAnn Bridges Academy

program_status_note: Open

service_type: Impact of Crime


summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
JoAnn Bridges Academy
Rite of Passage, Inc
(Contract Provider)
950 Southwest Greenville Hills Road
Greenville, Florida 32331
Primary Service: Seeking Safety
SPEP Review Period: April 1, 2020 to March 31, 2021
SPEP Review Conducted: April 14, 2021 to April 19, 2021
Florida Department of Juvenile Justice
Report Date: 4/19/2021
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Seeking Safety.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes Seeking Safety as a primary service. The program's documentation for
training protocol reflected the therapy staff are trained to facilitate Seeking Safety groups. There
were four trained staff members during the twelve month review period; one clinical director and
three therapeutic managers. During the twelve month review period, three therapeutic managers
conducted Seeking Safety groups. The clinical director is able to fill in as a facilitator as needed.
The clinical director provided training documentation for the therapy staff to review. The clinical
director also provided weekly supervision notes. This documentation indicated the clinical director
reviews all primary services and conducts ongoing training for Seeking Safety.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The Seeking Safety manual, and all associated documentation, is maintained in an organized
binder. This binder includes an overview of the curriculum, how lessons should be facilitated,
topics for discussions, guidelines and tips for the facilitator. There is not a separate manual for
youth, but rather a section in the facilitator binder for handouts to be copied and given to the
youth. The curriculum is twenty-five sessions which may be delivered in any order.
Seeking Safety is held on Tuesdays, Thursdays, and Saturdays from 9:15am to 10:15am and
10:30am to 11:30am. A review of the program's contract confirmed all youth are eligible to attend
Seeking Safety groups. The groups average six youth in each group. The faciltiators have
documented any reasons a youth is not in group or leaves the group on the sign-in sheets on a
consistent basis.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
An observation of Seeking Safety was conducted on Thursday, April 22, 2021 from 9:15am to
10:23am. There were ten youth present for the group along with two coach counselors and the
facilitator. The group was held in one of the program's classrooms. The facilitator encouraged the
youth to participate in group. Nine of the ten youth were engaged and took turns reading the
material. One youth did not want to actively participate in the group but was not disruptive. The
facilitator was knowledgable of the material and followed the lesson plan. The youth who
participated in group provided good dialogue and answered all the facilitator's questions.
The facilitator was not able to finish the day's lesson during the hour group. The facilitator stated
she would finish the lesson during the next group session as the youth were fully engaged and
willing to learn the material.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The reviewed documentation and an informal interview with the clinical director confirmed there
has been no turnover which adversely affected the youth during the past twelve months of the
review period. Should the program experience any turnover, their policy is for another trained staff
to continue with groups. The therapy staff rotate facilitation of Seeking Safety to minimize group
disruption and avoid gaps in service.
Seeking Safety is held as an open group, meaning the group is continuously held and youth are
able to start the group upon entry to the program. All sign-in sheets were provided for this twelve
month review period. There were no gaps in service during this review period.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program's policy for internal fidelity monitoring is for the clinical director to conduct monitoring
once a month for each facilitator. During the twelve month review period three therapuetic
managers conducted Seeking Safety groups.
Thirty-three fidelity monitoring forms out of thirty-six were provided for review. Three fidelity
monitoring forms were missing from the first and second quarter reviews of this review period.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program's policy is for the clinical director to address any issues or corrective actions
regarding Seeking Safety facilitation. Should there be any issues or corrective actions, it is
documented on the internal fidelilty monitoring forms and the clinical director conducts additional
fidelity monitoring.
Upon a review of the internal fidelity monitoring forms there was no documentation of corrective
actions needed during the twelve month review period. One facilitator did have some constructive
feedback documented during a fidellty monitoring and was observed an additional time within the
same month with no further issues.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
A review of three facilitators' employee performance evaluations found documentation of group
facilitation performance in the evidence-based curriculum of Seeking Safety. This section included
comments on any strengths or weaknesses of which the facilitator should be aware.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates the target duration of 24 weeks for this type of service. Of the 19 youth in the
sample, 63% (12 of 19) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates a target of 40 contact hours for this type of service. Of the 19 youth in the
sample, 58% (11 of 19) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 5 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 8%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 58%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 2 youth
Moderate-High = 8 youth
High = 1 youth
Total Youth in Sample = 19 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 5%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 1 youth
Total Youth in Sample = 19 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 58% (11 of 19) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 5% (1 of 19) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 6 10 60%
Amount of Service: Contact Hours 4 10 40%
Risk Level of Youth Served 5 25 8%
Totals 65 95 68%
This SPEP report evaluates Seeking Safety, an intervention delivered at JoAnn Bridges Academy.
19 youth completed the program during the review period. Of those youth, all 19 were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 6 points for Amount of Service: Duration. Of the 19 total youth sampled, only
18 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 12
received at least the recommended weeks of service. Youth in the sample completed between 10
and 35 weeks of service, with an average of 24 weeks.
The program earned 4 points for Amount of Service: Contact Hours. Of the 19 total youth
sampled, only 18 included dosage in the EBS Module. Of those youth with correct dosage, 11
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
received at least the recommended hours of service. Youth in the sample completed between 25
and 52 hours of service, with an average of 39 hours.
The program was awarded 5 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 89%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
JoAnn Bridges Academy can optimize their SPEP Quality of Service Delivery score by ensuring
fidelity monitoring is completed every month for each facilitator
JoAnn Bridges Academy can optimize their SPEP Amount of Service score by ensuring that
dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

service_type: Seeking Safety


summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
JoAnn Bridges Academy
Rite of Passage
(Contract Provider)
950 SW Greenville Hills Road
Greenville, Florida 32331
Primary Service: Seeking Safety
SPEP Review Period: April 1, 2021 - March 31, 2022
SPEP Review Conducted: April 12, 2022
Florida Department of Juvenile Justice
Report Date: 7/20/22
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Seeking Safety.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes Seeking Safety as a primary service. The program's documentation reflected
there were a total of four staff trained to facilitate the Seeking Safety groups; one clinical director
(9/15/2014) and three therapeutic managers (10/4/2016, 1/30/2018, and 10/11/2018). All training
records were provided for review and all staff were properly trained in the Seeking Safety
curriculum. Three therapeutic managers conducted Seeking Safety groups consistently during the
annual review period. The clinical director conducts group on an as needed basis. The clinical
director stated there is a new therapeutic manager currently in training at the time of this review.
The clinical director conducts all internal fidelity monitoring observations for the Seeking Safety
groups.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the current Seeking Safety manual. The manual includes all lessons and
materials needed to conduct the Seeking Safety groups.
Seeking Safety groups are held on Tuesdays, Thursdays, and Saturdays from 9:15am to
10:15am. A review of sign-in sheets confirmed the groups have been held in accordance with the
program schedule during the annual review period. The documentation for the Seeking Safety
groups is maintained in a binder kept in the clinical director's office. The binder includes all sign-in
sheets and fidelity monitoring forms. All youth are eligible to attend the Seeking Safety groups
while in the program. Seeking Safety is conducted as an open group, meaning they are
continuously held. The youth may enter the group at admission.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Seeking Safety was observed on Thursday, July 8, 2021. Present were seven youth,
one facilitator, and one staff member. The observer was provided a copy of the lesson to follow.
The facilitator adhered to the lesson for the group. The lesson for the day was commitment. the
facilitator has been facilitating Seeking Safety group since January of 2018 when she was trained
and is comfortable facilitating the groups. The facilitator did a good job of including all of the
youth. There were two new youth in group; one was hesitant to participate, but started to
participate toward the end of group time. The group stayed on topic and had good conversations
about the topic.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program's policy in the event of facilitator turnover is for another facilitator to continue with the
Seeking Safety groups. The program has the therapeutic managers rotate facilitation so the youth
are familiar with all of the faciltiators. The program did experience turnover during the annual
review period. Two therapeutic managers left, one in December 2021 and the other in March
2022. One therapeutic manager has been hired and is currently in training. The staff left during
the group due to the groups being open and facilitated on a continual basis.
All sign-in sheets were provided for this review. There were no gaps in service during the annual
review period. There were four to five youth in each group during the annual review period.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The clinical director conducts internal fidelity monitoring observations for the Seeking Safety
groups. All fidelity forms were provided for the annual review period. Thirty-two (32) out of thirty-
two (32) internal fidelity monitoring observations were completed during the annual review period.
There were no missing fidelity observations.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The clinical director addresses any significant issues or corrective actions for the facilitators for
the Seeking Safety groups. The internal fidelity monitoring forms include a section to document
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
any significant issues or corrective actions. A review of the internal fidelity monitoring forms
confirmed there were no significant issues or corrective actions noted during the annual review
period for the Seeking Safety facilitators.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The therapeutic manager who is continuing employment at the program has received her annual
performance evaluation. There is an addendum included with the annual evaluation to include
group faciltiation of Seeking Safety.
The other two facilitators for Seeking Safety are no longer employed at the program.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 24 weeks for this type of service. Of the 24 youth in the
sample, 8% (2 of 24) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 40 contact hours for this type of service. Of the 24 youth in the
sample, 100% (24 of 24) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 7 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 28%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 83%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 7 youth
Moderate-High = 12 youth
High = 1 youth
Total Youth in Sample = 24 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 4%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 1 youth
Total Youth in Sample = 24 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 83% (20 of 24) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 4% (1 of 24) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 7 25 28%
Totals 67 95 71%
This SPEP report evaluates Seeking Safety, an intervention delivered at JoAnn Bridges Academy.
25 youth completed the program during the review period. Of those youth, 24 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 24 total youth sampled, all
24 included dosage with end dates in the EBS Module. Of those youth, 2 received at least the
recommended weeks of service. Youth in the sample completed between 13 and 37 weeks of
service, with an average of 18 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 24 total youth
sampled, all 24 included dosage in the EBS Module. Of those youth, 24 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
recommended hours of service. Youth in the sample completed between 41 and 50 hours of
service, with an average of 44 hours.
The program was awarded 7 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 89%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
JoAnn Bridges Academy can maintain their SPEP Quality of Service Delivery score by continuing
the practices in place at the time of this review.
JoAnn Bridges Academy can optimize their SPEP Amount of Service score by ensuring dosage
for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted dosage
of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2021-22 JBA SS.pdf


summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
JoAnn Bridges Academy
Rite of Passage
(Contract Provider)
950 SW Greenville Hills Road
Greenville, Florida 32331
Primary Service: Aggression Replacement Training
SPEP Review Period: April 1, 2021 - March 31, 2022
SPEP Review Conducted: April 12, 2022
Florida Department of Juvenile Justice
Report Date: 7/20/22
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Aggression Replacement Training.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes Aggression Replacement Training (ART) as a primary service. All
therapeutic managers are trained to conduct ART groups. There were four therapy staff able to
facilitate the ART groups during the annual review period; one clinical director (9/12/2014), and
three therapeutic managers (11/11/2016, 1/30/2018, and 10/18/2018). All training documentation
was provided for review. There is a new therapeutic manager who was in training at the time of
this review. The clinical director conducts fidelity monitoring for the ART groups.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program provided the Aggression Replacement Training (ART), 3rd edition, manual for
review. The manual was organized by each lesson: skill streaming, moral reasoning, and anger
control. Each lesson is divided by weeks, for a total of ten (10) weeks and thirty (30) hours.
Aggression Replacement Training does not have separate manuals or workbooks for the youth.
ART groups are held on Mondays, Wednesdays, and Fridays from 9:15am to 10:15am.
Documentation for the ART groups is maintained in a binder kept in the clinical director's office.
The binder includes all sign-in sheets organized by cohort and all internal fidelity monitoring
forms. All youth are eligible to attend ART groups while in the program. Aggression Replacement
Training groups are closed, meaning once the first lesson has been completed, no youth may be
added to the group.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
An Aggression Replacement Training (ART) group was observed via video on August 11, 2021.
The group was in week three of ART; Anger Control Training, cues and Anger Reducers 1-3, was
being conducted. Five youth, one facilitator, and one staff were present for the group. The
facilitator was comfortable conducting the group. She had great energy with the youth throughout
the entire time. The facilitator reviewed the homework with the youth and did a short review of the
anger control lesson from the previous week. The facilitator discussed the role plays to be
conducted, provided an example role play with the staff in the group, and then each youth
conducted their own role plays while the youth in the audience provided feedback and identified
the steps the group was learning about. All materials were present for the group. There were no
behavioral issues. All of the youth brought their folders with their homework and placed their new
homework in the folder upon leaving group. The facilitator was organized and it was evident she
enjoys facilitating ART groups. She engages the youth and the youth respond very well to her.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program's policy is for another trained facilitator to conduct Aggression Replacement Training
(ART) groups should turnover occur. There was one main facilitator during this annual review
period. The program did experience turnover of two therapeutic managers during the annual
review period. The faciliators left between cohorts and therefore the groups were not adversely
affected.
There were two gaps in service during the annual review period. One was from October 1, 2021
to December 6, 2021 (56 days) and from February 11, 2022 to April 11, 2022 (58 days). The gaps
were due to not enough youth entering the program and a low program census during this time,
according to the clinical director.
The program completed four cohorts of ART during the annual review period.
The first cohort started February 22, 2021, with four youth, and ended on April 30, 2021 with all
four youth successfully completed the ART curriculum.
The second cohort was started May 20, 2021, with seven youth, and ended on July 16, 2021 with
all seven youth successfully completing the group. One youth was moved from this group to
another group after the first session. Another youth had two sessions conducted individually and
then added to this cohort. The sign-in sheets were reviewed for the individual sessions.
The third cohort started July 26, 2021, with five youth, and ended on October 1, 2021 with all five
youth successfully completing the group. One youth was discharged early; however, the last three
sessions were completed individually and those sign-in sheets were reviewed.
The fourth cohort started December 6, 2021, with seven youth, and ended on February 11, 2022,
with all seven youth successfully completing the group.
The clinical director reported a new cohort will begin on April 11, 2022. The program had to wait
for enough youth to enter the program to begin a cohort.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The clinical director conducts all internal fidelity monitoring observations for the facilitators of
Aggression Replacement Training (ART). There were three facilitators for ART groups during the
annual review period, although the majority of cohorts are conducted by one therapeutic manager.
The curriculum dictates at least two groups are observed each month per facilitator. The program
typically observes three per facilitator per month. Based upon when the staff faciltiated the
groups, thirty-eight (38) out of thirty-eight (38) internal fidelity monitoring observations were
conducted during the annual review period.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The clinical director addresses any significant issues or corrective actions for the facilitators of
Aggression Replacement Training (ART) groups. The internal fidelity monitoring forms include a
section to document any signficant issues or corrective actions for the facilitators. All internal
fideitly monitoring forms were provided for review. There were no documented significant issues
or corrective actions noted for the faciltiators during the annual review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The annual performance evaluations include an addendum to include an evaluation of the
services the employee facilitates. One of the facilitators received an annual performance
evaluation for this review period and it included the addendum for group facilitation, including
Aggression Replacement Training.
The other two therapeutic managers left employment during the annual review period.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 10 weeks for this type of service. Of the 24 youth in the
sample, 96% (23 of 24) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 30 contact hours for this type of service. Of the 24 youth in the
sample, 100% (24 of 24) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 7 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 28%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 83%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 7 youth
Moderate-High = 12 youth
High = 1 youth
Total Youth in Sample = 24 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 4%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 1 youth
Total Youth in Sample = 24 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 83% (20 of 24) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 4% (1 of 24) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 7 25 28%
Totals 80 100 80%
This SPEP report evaluates Aggression Replacement Training, an intervention delivered at JoAnn
Bridges Academy.
25 youth completed the program during the review period. Of those youth, 24 were included in the
sample.
The program scored High for Quality of Service Delivery.
The targeted number of weeks for this specific intervention is 10 weeks; the targeted number of
hours for this specific intervention is 30 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 5 services (15 weeks; 45
hours).
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 8 points for Amount of Service: Duration. Of the 24 total youth sampled, all
24 included dosage with end dates in the EBS Module. Of those youth, 23 received at least the
recommended weeks of service. Youth in the sample completed between 9 and 10 weeks of
service, with an average of 10 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 24 total youth
sampled, all 24 included dosage in the EBS Module. Of those youth, 24 received at least the
recommended hours of service. Youth in the sample completed 30 hours of service.
The program was awarded 7 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 98%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
JoAnn Bridges Academy can maintain their SPEP Quality of Service Delivery score by continuing
the practices in place at the time of this review.
JoAnn Bridges Academy can optimize their SPEP Amount of Service score by ensuring dosage
for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted dosage
of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2021-22 JBA ART.pdf

service_type: Aggression Replacement Training