state: FL

facility_name: Oak Grove Academy

summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Oak Grove Academy
Rite of Passage, INC.
(Contract Provider)
11180 NE 38th Street
Jasper, Florida 32052
Primary Service: Anger Replacement Training (ART)
SPEP Review Period: April 1, 2023-March 31, 2024
SPEP Review Conducted: 5/9/2024; 5/10/2024
Florida Department of Juvenile Justice
Report Date: 9/18/2025
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Anger Replacement Training (ART).
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The documentation for the annual review period of April 1, 2023 through March 31, 2024 shows
the program uses Aggression Replacement Training (ART) as their primary service. Aggression
Replacement Training is provided by a qualified trainer over a three-day period. There were six
staff members trained in the primary service. The trained staff members were four therapeutic
managers, the clinical director, and the current director of student services (DSS). Five trained
staff members (four therapeutic managers and the current director of student services) facilitated
the primary service during the annual review period. The director of student services was a
therapeutic manager prior to her promotion to DSS in October 2023. The DSS has not run groups
since the promotion. The clinical director or other trained facilitator conducts the monthly internal
fidelity monitorings. The clinical director and the current director of student services conducted the
monthly internal fidelity checks during the annual review period.
The program has a policy and procedure in place to address facilitator training. The facilitator for
the primary service will have training from a qualified trainer. This training will be documented in
the facilitators on the job training plan. The facilitator will receive additional training on the
curriculum as needed from the clinical director documented within the clinical supervision. All
facilitators will receive training prior to conducting the service.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
A review of the documentation for the annual review period of April 1, 2023 through March 31,
2024 shows the program utilizes the Aggression Replacement Training (ART) manual. Technical
assistance specialist reviewed the manual. The manual is broken down into six sections. The first
section covers Skillstreaming. The second section deals with anger control. The third section
covers moral reasoning. Each of the first three sections has a script for the facilitator to use for
each lesson. Sections four through six are for the facilitator's use. It includes training step guides,
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
fidelity forms and hassle logs, and the program materials. The program provides the students with
copies of handouts for each lesson. The program provides the primary service on Mondays,
Tuesdays, and Wednesdays for forty-five (45) minutes each session. The program’s contract lists
the minimum group length is forty-five (45) minutes. The program keeps their documentation for
the primary service in a binder labeled ART in the clinical director’s office. The binder contains all
attendance sheets, internal fidelity monitoring checklists, training certificates, and group
schedules. All youth who are admitted to the program receive the primary service. The primary
service is held as a closed group.
The program has a policy and procedure in place to address the treatment manual/protocol. The
primary service will be conducted as indicated in the manual/protocol; this includes adherence to
the session plan, group size, population, frequency, needed materials, duration, and interventions.
The facilitator of the primary service will specify the topic covered, interventions utilized, duration
of the service, youth present, and progress for each session on the sign-in sheet.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Technical assistance specialist observed the primary service on 5/9/2024. The group began at
1:35p. There were ten (10) youth, three staff, and one facilitator present for the group. The
facilitator began with the group pledge and check in. The lesson for the day was Alonzo’s
Dilemma. The facilitator had the youth read the scenario, answer the questions on the worksheet,
and then went over each question. The facilitator was knowledgeable of the material and
presented the content in a way the youth could understand. The group ended at 2:30p.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The documentation for Aggression Replacement Training (ART) for the annual review period of
April 1, 2023 through March 31, 2024 shows the program did have facilitator turnover; however,
the turnover did not negatively affect delivery of the primary service. The director of student
services was a therapeutic manager who was facilitating the primary service prior to her
promotion to DSS in October 2023. The DSS has not run groups since the promotion. The DSS
was not running groups at the time of her promotion. There were five cohorts completed and one
currently ongoing. The dates of the cohorts were: 2/12/2023-4/26/2023, 5/8/2023-7/14/2023,
8/1/2023-10/6/2023, 10/17/2023-12/20/2023, 1/2/2024-3/7/2024, and 3/19/2024-current (as of
3/31/2024). The documentation showed there were four trained therapeutic managers and the
current Director of Student Services who facilitated the cohorts during the annual review period.
The cohort which began on 2/12/2023 began with ten (10) youth and ended with ten (10) youth
completing the service. There was a note from the clinical director stating the primary service was
not held during the week of 4/10/2023 due to the primary facilitator being out. Group resumed on
4/20/2023 and was completed on 4/26/2023.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The cohort which began on 5/8/2023 began with ten (10) youth and ended with ten (10) youth
completing the service.
The cohort which began on 8/1/2023 began with eight youth and ended with eight youth
completing the service.
The cohort which began on 10/17/2023 began with eight youth and ended with eight youth
completing the service. The program began running the service for forty-five (45) minutes each
session beginning with the session dated 11/8/2023.
The cohort which began on 1/2/2024 began with ten (10) youth and ended with nine youth
completing the service. The make-up session for a youth on 1/9/2024 was only ran for thirty (30)
minutes. All attendance sheets show the primary service is ran for forty-five (45) minutes each
session. One youth missed group on 2/1/2024 and completed a make-up session on 2/2/2024.
One youth was discharged on 2/12/2024. One youth was off campus on 2/16/2024 and made up
the session on 2/17/2024. Three youth were off campus on 2/29/2024 and made up the session
on 3/1/2024.
The cohort which began on 3/19/2024 began with nine youth and currently has ten (10) youth (as
of 3/31/2024). One youth was added on 3/21/2024. The youth made up lessons one and two on
3/24/2024 and 3/26/2024.
The program has a protocol in place in the event of facilitator turnover. The program will have
another trained facilitator take over the facilitation of the cohort.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of the documentation for Aggression Replacement Training (ART) for the annual review
period of April 1, 2023 through March 31, 2024 shows there were thirty-five (35) internal fidelity
monitoring’s completed. The program completed all internal fidelity monitorings for the annual
review period for each facilitator. The clinical director and the current director of student services
conducted the monthly internal fidelity checks.
The clinical director, regional clinical director, or other designated trained staff member conducts
the monthly fidelity monitoring.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
A review of the documentation for Aggression Replacement Training (ART) for the annual review
period of April 1, 2023 through March 31, 2024 shows were no corrective actions noted on the
completed monthly internal fidelity checks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
The clinical director stated if corrective action is needed it would be documented on the fidelity
monitoring checklist and addressed with staff. The clinical director would monitor the corrective
action to ensure it is corrected. Retraining the staff will occur if necessary.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program has not provided annual evaluations for staff who facilitated the primary service
during this annual review period. The program has employee performance evaluation addendums
to document the facilitator’s skill delivering the primary service. The addendums are attached to
the facilitator’s annual performance evaluation.
The program has a policy and procedure in place for annual staff performance evaluations. The
program’s policy states an annual evaluation of performance will be conducted on all the
facilitators of each primary service. The performance evaluation is comprised of feedback from
the fidelity monitoring tools as well as an assessment of the facilitator’s skill in delivering the
primary service.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 10 weeks for this type of service. Of the 34 youth in the
sample, 94% (32 of 34) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 30 contact hours for this type of service. Of the 34 youth in the
sample, 97% (33 of 34) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 15 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 60%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 82%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 4 youth
Moderate-High = 14 youth
High = 10 youth
Total Youth in Sample = 34 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 29%
High Score: 8 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 62%
Table 2
High = 10 youth
Total Youth in Sample = 34 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 82% (28 of 34) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 29% (10 of 34) youth scored High-Risk to reoffend, for a score of 8 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 15 25 60%
Totals 86 100 86%
This SPEP report evaluates Anger Replacement Training, an intervention delivered at Oak Grove
Academy.
42 youth completed the program during the review period. Of those youth, 34 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 34 total youth sampled, only
34 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 32
received at least the recommended weeks of service. Youth in the sample completed between 6
and 12 weeks of service, with an average of 10 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program earned 8 points for Amount of Service: Contact Hours. Of the 34 total youth
sampled, only 34 included dosage in the EBS Module. Of those youth with correct dosage, 33
received at least the recommended hours of service. Youth in the sample completed between 18
and 33 hours of service, with an average of 30 hours.
The targeted number of weeks for this specific intervention is 10 weeks; the targeted number of
hours for this specific intervention is 30 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 5 services (15 weeks; 45
hours).
The program was awarded 15 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 96%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Oak Grove Academy can optimize their SPEP Quality of Service Delivery score by ensuring each
facilitator’s performance evaluation includes discussion of their skill in delivering this primary
service.
Oak Grove Academy can optimize their SPEP Amount of Service score by ensuring that dosage
for all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

source: DJJ

report_type: SPEP

file_name: FY2023-24 Oak Grove ART.pdf

program_name_raw: Oak Grove Academy

program_status_note: Open

service_type: Aggression Replacement Training


summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Oak Grove Academy
Rite of Passage, INC.
(Contract Provider)
11180 NE 38th Street
Jasper, Florida 32052
Primary Service: Impact of Crime (IOC)
SPEP Review Period: April 1, 2023-March 31, 2024
SPEP Review Conducted: 5/9/2024; 5/10/2024
Florida Department of Juvenile Justice
Report Date: 9/18/2025
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime (IOC).
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The documentation for the annual review period of April 1, 2023 through March 31, 2024 shows
the program uses Impact of Crime Curriculum (IOC) as their primary service. Impact of Crime
training is a three-day, twenty-four (24) hour training via Microsoft Teams which is delivered from
a qualified trainer. There were twelve (12) staff members trained in the primary service. The
trained staff members were three therapeutic service managers, a coach counselor, the
operations specialist, a therapeutic manager, two regional compliance managers, one recreation
manager, the program director, the director of group living, and the director of student services.
Nine trained staff members (three therapeutic service managers, one therapeutic manager, one
recreation manager, one regional compliance manager, the operations specialist, one coach
counselor, and the director of group living) facilitated the primary service during the annual review
period. The regional compliance manager or other trained facilitator conducts the monthly internal
fidelity monitorings. Both regional compliance managers, the director of group living, and the
operations specialist conducted the monthly internal fidelity monitorings for the annual review
period.
The program has a policy and procedure in place to address facilitator training. The facilitator for
the primary service will have training from a qualified trainer. This training will be documented in
the facilitators on the job training plan. The facilitator will receive additional training on the
curriculum as needed from the clinical director documented within the clinical supervision. All
facilitators will receive training prior to conducting the service.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
A review of the documentation for the annual review period of April 1, 2023 through March 31,
2024 shows the program utilizes the Impact of Crime: Addressing the Harm manual. Technical
assistance specialist reviewed the manual. The manual opens with an introduction to the Impact
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
of Crime Curriculum. Then the manual goes over all seven chapters of the service and provides a
script for the facilitator to follow. The manual also has an appendix and a student guide with all the
student exercises. The program makes copies of the student exercises and passes them out to
the youth during each lesson. The program conducts the primary service on Mondays and
Wednesdays. The program keeps all SPEP documentation (attendance sheets, fidelity checks,
and copies of training certificates) in a binder labeled IOC SPEP documentation in the Clinical
Director’s office. All youth admitted to the program are required to participate in the primary
service. The primary service is a closed group with no more than twelve youth in the cohort. The
program provides the primary service on Tuesdays and Thursdays for at least one hour each
session. The primary service is a closed group.
The program has a policy and procedure in place to address the treatment manual/protocol. The
primary service will be conducted as indicated in the manual/protocol; this includes adherence to
the session plan, group size, population, frequency, needed materials, duration, and interventions.
The facilitator of the primary service will specify the topic covered, interventions utilized, duration
of the service, youth present, and progress for each session on the sign-in sheet.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Technical assistance specialist observed the primary service on 5/9/2024. The group began at
2:41p. There were eight youth, one staff, and one facilitator present for the group. The facilitator
began with the group rules and reviewed the previous lesson. The lesson for the day was
Damian’s Dilemma. The facilitator had three youth read a part of the scenario and then went over
the scenario. The facilitator was knowledgeable of the material and presented the content in a
way the youth could understand. The group ended at 3:40p.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The documentation for Impact of Crime (IOC) for the annual review period of April 1, 2023 through
March 31, 2024 shows the program did have facilitator turnover. One case manager left the
program in May 2023 and the recreation therapist left the program in October 2023. One
therapeutic service manager left the program in November 2023 and another therapeutic service
Manager left the program in January 2024. The facilitator turnover negatively affected the delivery
of the primary service since the case manager was in the middle of a cohort at the time she left.
The program brought another trained facilitator in to continue the cohort without any gap in
service. The recreation therapist was co-facilitating a cohort at the time she left the program. The
main facilitator continued with the cohort without a gap in service. Both therapeutic service
managers were in the middle of a cohort at the time they left. The program brought another
trained facilitator in to continue the cohort without any gap in service. There were five cohorts
completed and two currently ongoing during the annual review period. The dates of the cohorts
were: 1/10/2023-5/16/2023, 4/26/2023-8/23/2023, 7/6/2023-11/5/2023, 10/24/2023-2/17/2024,
11/7/2023-2/29/2024, 1/31/2024-current (as of 3/31/2024), and 2/20/2024-current (as of
3/31/2024). Nine trained staff members (three therapeutic service managers, one therapeutic
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
manager, one recreation manager, one regional compliance manager, the operations specialist,
one coach counselor, and the director of group living) facilitated the primary service during the
annual review period. There were no gaps in service.
The cohort which began on 1/10/2023 began with eleven (11) youth and ended with ten (10)
youth. One youth was transferred from the program on 4/16/2023.
The cohort which began on 4/26/2023 began with ten (10) youth and ended with ten (10) youth.
One youth was added to the cohort on 4/29/2023. One youth was discharged on 8/21/2023. There
were two attendance sheets dated 5/10/2023. One attendance sheet shows the group ran from
4:54p-5:54p and was on chapter two objective one. The second attendance sheet dated
5/10/2023 shows the group ran from 4:30p-5:30p and covered chapter two objective two. The
Clinical Director stated it was a mistake in documentation. The facilitator should have put
5/13/2023 on the second attendance sheet instead of 5/10/2023. The therapeutic services
manager who started the cohort as the facilitator left the program in May 2023. The trained
therapeutic manager took over the cohort on 5/31/2023 with no gap in service. The cohort ran
back-to-back on 6/10/2023 and 6/11/2023 and then again on 6/17/2023 and 6/18/2023. The
clinical director stated the facilitator was going to be off, so they tried to account for the time
missed.
The cohort which began on 7/6/2023 began with eight youth and ended with five youth. One youth
was transferred on 7/13/2023. One youth was discharged on 7/27/2023 and another youth was
discharged on 8/25/2023. The facilitator ran the cohort back-to-back on 10/18/2023 and
10/19/2023. The cohort which began on 10/24/2023 began with eight youth and currently has
eight youth.
The cohort which began on 10/24/2023 began with eight youth and ended with five youth. One
youth was discharged on 1/4/2024. There was a note stating the group began with a therapeutic
service manager (TSM) facilitating groups until the TSM left the program in January 2024. The
regional compliance manager took over the group on 1/5/2024. The attendance sheet dated for
2/3/2024 shows the group ran for one and a half hours. There were two youth who completed a
make up session on 2/3/2024 for missing on 1/27/2024. One youth completed the make up
session independently on 2/6/2024. On 2/10/2024, two youth were instructed to complete IOC
work independently. The youth were given worksheets for the remaining objectives for chapter
seven due to their release dates of 2/12/2024 and 2/13/2024.
The cohort which began on 11/7/2023 began with eight youth and ended with ten (10) youth. Two
youth were added on 11/19/2023. There was a note dated 11/29/2023 stating the group began
with a therapeutic service manager (TSM) facilitating until the TSM left the program in November
2023. The regional compliance manager took over the group on 11/28/2023. A note dated
12/8/2023 stated the group was now facilitated by the operations specialist.
The cohort which began on 1/31/2024 began with eight youth and currently has seven youth (as
of 3/31/2024). Attendance sheets dated on 2/9/2024, 2/14/2024, 2/16/2024, and 2/24/2024 show
one youth missed the lesson due to being at work. On 3/14/2024, the clinical director moved the
youth from the cohort and put him into a different cohort so the youth would not miss any lessons
due to his work schedule.
The cohort which began on 2/20/2024 began with eight youth and currently has eight youth (as of
3/31/2024). The cohort began with the regional compliance manager facilitating groups. The
director of group living took over facilitating the cohort on 3/7/2024.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
The program has a protocol in place in the event of facilitator turnover. The program will have
another trained facilitator take over the facilitation of the cohort.
The program can earn 2 points if facilitator turnover does not impact groups and there are no
unnecessary gaps in service..
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
A review of the documentation for Impact of Crime (IOC) for the annual review period of April 1,
2023 through March 31, 2024 shows there were nineteen (19) internal fidelity monitoring’s
completed. Both regional compliance managers, the director of group living, and the operations
specialist conducted the monthly internal fidelity monitorings for the annual review period. The
clinical director, regional compliance manager, or other designated trained staff member conducts
the monthly fidelity monitoring. The program was missing a fidelity monitoring for a case manager
for the month of April 2023.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
A review of the documentation for Impact of Crime (IOC) for the annual review period of April 1,
2023 through March 31, 2024 shows were no corrective actions noted on the completed monthly
internal fidelity checks.
The clinical director stated if corrective action is needed it would be documented on the fidelity
monitoring checklist and addressed with staff. The clinical director would monitor the corrective
action to ensure it is corrected. Retraining the staff will occur if necessary.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program has not provided annual evaluations for staff who facilitated the primary service
during this annual review period. The program has employee performance evaluation addendums
to document the facilitator’s skill delivering the primary service. The addendums are attached to
the facilitator’s annual performance evaluation.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
The program has a policy and procedure in place for annual staff performance evaluations. The
program’s policy states an annual evaluation of performance will be conducted on all the
facilitators of each primary service. The performance evaluation is comprised of feedback from
the fidelity monitoring tools as well as an assessment of the facilitator’s skill in delivering the
primary service.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 16 weeks for this type of service. Of the 42 youth in the
sample, 95% (40 of 42) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 24 contact hours for this type of service. Of the 42 youth in the
sample, 98% (41 of 42) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 15 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 60%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 83%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 5 youth
Moderate-High = 19 youth
High = 11 youth
Total Youth in Sample = 42 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
Percentage of Youth with High-Risk Level to Reoffend: 26%
High Score: 8 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 62%
Table 2
High = 11 youth
Total Youth in Sample = 42 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 83% (35 of 42) youth scored Moderate to High-Risk to reoffend, for a score
of 7 points.
Of the SPEP sample, 26% (11 of 42) youth scored High-Risk to reoffend, for a score of 8 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 15 25 80%
Totals 61 85 72%
This SPEP report evaluates Impact of Crime, an intervention delivered at Oak Grove Academy.
42 youth completed the program during the review period. Of those youth, 42 were included in the
sample.
The program scored Medium for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 42 total youth sampled, only
42 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 42
received at least the recommended weeks of service. Youth in the sample completed between 4
and 20 weeks of service, with an average of 18 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 42 total youth
sampled, only 42 included dosage in the EBS Module. Of those youth with correct dosage, 42
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10
received at least the recommended hours of service. Youth in the sample completed between 8
and 35 hours of service, with an average of 33 hours.
The program was awarded 15 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 84%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Oak Grove Academy can optimize their SPEP Quality of Service Delivery score by ensuring
facilitator turnover does not impact groups, fidelity monitoring is completed every month for each
facilitator, and each facilitator’s performance evaluation includes discussion of their skill in
delivering this primary service.
Oak Grove Academy can optimize their SPEP Amount of Service score by ensuring that dosage
for all youth is recorded accurately in EBS and by ensuring that youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 11

file_name: FY2023-24 Oak Grove IOC.pdf

service_type: Impact of Crime


summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Oak Grove Academy
Rite of Passage, INC.
(Contract Provider)
11180 NE 38th Street
Jasper, Florida 32052
Primary Service: Seeking Safety
SPEP Review Period: April 1, 2023-March 31, 2024
SPEP Review Conducted: 5/9/2024; 5/10/2024
Florida Department of Juvenile Justice
Report Date: 9/18/2025
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Seeking Safety.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The documentation for the annual review period of April 1, 2023 through March 31, 2024 shows
the program uses Seeking Safety as their primary service. Seeking Safety training is conducted
by the clinical director as an in-service training. There were six staff members trained in the
primary service. The trained staff members were four therapeutic managers, the clinical director,
and the current director of student services (DSS). The director of student services (DSS) was a
therapeutic manager prior to being promoted to DSS in October 2023. The DSS does not run
groups since she is no longer working as a therapeutic manager. All six trained staff members
facilitated the primary service during the annual review period. The regional director of clinical
services, clinical director, or other trained facilitator conducts the monthly internal fidelity
monitorings. The clinical director, a therapeutic manager, and the regional director of clinical
services conducted the monthly internal fidelity checks during the annual review period.
The program has a policy and procedure in place to address facilitator training. The facilitator for
the primary service will have training from a qualified trainer. This training will be documented in
the facilitators on the job training plan. The facilitator will receive additional training on the
curriculum as needed from the clinical director documented within the clinical supervision. All
facilitators will receive training prior to conducting the service.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
A review of the documentation for the annual review period of April 1, 2023 through March 31,
2024 shows the program uses the Seeking Safety: A Treatment Manual for PTSD and Substance
Abuse for their primary service manual. Technical assistance specialist reviewed the manual. The
manual is broken down into twenty-five sessions. Each session explains the topic that the
therapist will cover and provides handouts for the youth to complete. The primary service can be
utilized for individual treatment or group treatment. The program provides the primary service on
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Tuesdays and Thursdays for a minimum of forty-five (45) minutes each session. The clinical
director stated the contract states the minimum group length is forty-five (45) minutes. The
primary service is run as an open group. The program keeps all SPEP documentation
(attendance sheets, fidelity checks, and copies of training certificates) in a binder labeled Seeking
Safety SPEP documentation in the clinical director’s office. All youth admitted to the program are
required to participate in the primary service.
The program has a policy and procedure in place to address the treatment manual/protocol. The
primary service will be conducted as indicated in the manual/protocol; this includes adherence to
the session plan, group size, population, frequency, needed materials, duration, and interventions.
The facilitator of the primary service will specify the topic covered, interventions utilized, duration
of the service, youth present, and progress for each session on the sign-in sheet.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Technical assistance specialist observed the primary service on 5/10/2024 via Microsoft Teams.
The group began at 1:34p. There were fifteen (15) youth, two staff, and one facilitator present for
the group. The facilitator began with the opening pledge, reviewed the group norms, and
completed the check in. The lesson for the day was Substance Control. The facilitator was
knowledgeable of the material and presented the content in a way the youth could understand.
The youth were engaged and open during the group. The group ended at 2:22p.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The documentation for Seeking Safety for the annual review period of April 1, 2023 through March
31, 2024 shows the program did have facilitator turnover. The turnover did negatively affect the
primary service. The director of student services was a therapeutic manager who was facilitating
the primary service prior to her promotion to DSS in October 2023. The DSS has not run groups
since the promotion. All six trained staff members facilitated the primary service during the annual
review period. There were no gaps in service. The primary service is an open group ran year-
round. The documentation reviewed showed the program was conducting the primary service
each month.
There were four dates in April 2023 where the program ran supportive counseling sessions in lieu
of the full group. Each of the four supplemental sessions were conducted for twenty (20) minutes
each. Technical assistance specialist informed the clinical director in May 2023 the youth should
receive a full hour of the group instead of only twenty (20) minutes of supportive counseling. The
program did not have any additional supportive counseling sessions noted during the remainder
of the annual review period. The attendance sheet dated 8/31/2023 did not have the facilitator
listed.
Each attendance sheet during October 2023 and on shows the program is now running the
primary service for a minimum of forty-five (45) minutes each session as stated in the contract.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The program has a protocol in place in the event of facilitator turnover. The program will have
another trained facilitator take over the facilitation of the cohort.
The program can earn 2 points if facilitator turnover does not impact groups and there are no
unnecessary gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation for Seeking Safety for the annual review period of April 1, 2023 through March
31, 2024 shows there were twenty-six (26) internal fidelity monitoring’s completed. The clinical
director, a therapeutic manager, and the regional director of clinical services conducted the
monthly internal fidelity checks during the annual review period. The regional director of clinical
services, clinical director, or other trained facilitator conducts the monthly internal fidelity
monitorings. The program completed all internal fidelity monitorings for each month for each
facilitator.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
A review of the documentation for Seeking Safety for the annual review period of April 1, 2023
through March 31, 2024 shows were no corrective actions noted on the completed monthly
internal fidelity checks.
The clinical director stated if corrective action is needed it would be documented on the fidelity
monitoring checklist and addressed with staff. The clinical director would monitor the corrective
action to ensure it is corrected. Retraining the staff will occur if necessary.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program has not provided annual evaluations for staff who facilitated the primary service
during this annual review period. The program has employee performance evaluation addendums
to document the facilitator’s skill delivering the primary service. The addendums are attached to
the facilitator’s annual performance evaluation.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
The program has a policy and procedure in place for annual staff performance evaluations. The
program’s policy states an annual evaluation of performance will be conducted on all the
facilitators of each primary service. The performance evaluation is comprised of feedback from
the fidelity monitoring tools as well as an assessment of the facilitator’s skill in delivering the
primary service.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 24 weeks for this type of service. Of the 40 youth in the
sample, 100% (40 of 40) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 40 contact hours for this type of service. Of the 40 youth in the
sample, 100% (40 of 40) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 18 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 72%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 85%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 5 youth
Moderate-High = 18 youth
High = 11 youth
Total Youth in Sample = 40 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 28%
High Score: 8 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 62%
Table 2
High = 11 youth
Total Youth in Sample = 40 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 85% (34 of 40) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 28% (11 of 40) youth scored High-Risk to reoffend, for a score of 8 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 18 25 72%
Totals 78 95 82%
This SPEP report evaluates Seeking Safety, an intervention delivered at Oak Grove Academy.
42 youth completed the program during the review period. Of those youth, 40 were included in the
sample.
The program scored Medium for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 40 total youth sampled,
only 40 included dosage with end dates in the EBS Module. Of those youth with correct dosage,
40 received at least the recommended weeks of service. Youth in the sample completed between
25 and 77 weeks of service, with an average of 43 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 40 total youth
sampled, only 40 included dosage in the EBS Module. Of those youth with correct dosage, 40
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
received at least the recommended hours of service. Youth in the sample completed between 48
and 99 hours of service, with an average of 59 hours.
The program was awarded 18 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 89%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Oak Grove Academy can optimize their SPEP Quality of Service Delivery score by ensuring
facilitator turnover does not impact groups, fidelity monitoring is completed every month for each
facilitator, and each facilitator’s performance evaluation includes discussion of their skill in
delivering this primary service.
Oak Grove Academy can maintain their SPEP Amount of Service score by continuing to ensure
dosage for all youth is recorded accurately in EBS and by continuing to ensure youth receive the
full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10

file_name: FY2023-24 Oak Grove Seeking Safety.pdf

service_type: Seeking Safety