state: FL

facility_name: Palm Beach Youth Academy

summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Palm Beach Youth Academy
Rite of Passage
(Contract Provider)
9680 Weisman Way
West Palm Beach, Florida 33411
Primary Service: Aggression Replacement Training
SPEP Review Period: 2/1/2023-1/31/2024
SPEP Review Conducted: 2/21/2024-2/22/2024
Florida Department of Juvenile Justice
Report Date: 6/27/2024
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Aggression Replacement Training.
The program was awarded 30 points because the Primary Service is identified as a Group 5
Service. The specific Sub-Component Service Type identified is Cognitive Behavioral Therapy.
The Primary Service was identified as this type of service as it is intended to correct faulty
cognitions and perceptions and provides skills individuals can use to monitor thought patterns and
correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The documentation for the annual review period of February 1, 2023 through January 31, 2024
shows the program utilizes Aggression Replacement Training as a primary service. Facilitator
training consists of a three-day course provided by a qualified instructor. The documentation
showed there were seven staff members trained in the primary service during this review period:
one regional director of clinical services, two clinical directors, four therapeutic managers, and one
trainer. One clinical director and two therapeutic managers facilitated Aggression Replacement
Training.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the manual Aggression Replacement Training. This service is to be provided
three times per week over the course of ten (10) weeks to help youth develop social skill
competence, anger control, and moral reasoning skills. The program maintains all SPEP
documentation (attendance sheets and internal fidelity checks) in a separate binder labeled
“Aggression Replacement Training.” All youth admitted to the program are required to participate
in this primary service. Aggression Replacement Training is a closed group with no more than ten
(10) youth assigned to a cohort.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Technical assistance specialist was unable to observe this service for the annual review period of
February 1, 2023 through January 31, 2024. The program completed a cohort just prior to the
date of the third quarter review and did not have a cohort running at the time of this annual review.
An observation was completed by the previously assigned technical assistance specialist on
August 15, 2024. Notes from the previously assigned technical assistance specialist indicate there
were no issues with the quality of the facilitation of this group.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The documentation provided for the annual review period of February 1, 2023 through January
31, 2024 shows the program has experienced facilitator turnover. Two clinical directors, three
therapeutic managers, and one trainer left the program during this review period. This turnover did
not adversely affect the delivery of the service, as no facilitators left in the middle of a cohort.
There were no gaps in the delivery of this service as a result of a lack of trained staff. Four
cohorts were completed during this annual review period. Two cohorts were on-going at the end
of the annual review period.
The first cohort began on February 28, 2023 with six youth and ended on May 5, 2023 with four
youth. One youth was dropped from the cohort due to arrest and one youth was dropped for
repeated refusals to attend group.
The second cohort began on June 8, 2023 with ten (10) youth and ended on August 18, 2023 with
nine youth. One youth was dropped from the cohort due to repeated refusals to attend group.
The third cohort began on August 1, 2023 with ten (10) youth and ended on October 6, 2023 with
eight youth. One youth was dropped from the cohort due to disruptive behaviors and non-
compliance, and one youth was dropped from the cohort after he was transferred to a different
program.
The fourth cohort began on September 18, 2023 with seven youth and ended on November 27,
2023 with six youth. One youth was dropped from the cohort due to being arrested.
The fifth cohort began on December 5, 2023 with ten (10) youth and was on-going at the time of
this review with ten youth.
The sixth cohort began on December 5, 2023 with ten (10) youth and was on-going at the time of
this review with ten youth.
An informal interview with the regional compliance director confirmed the program will replace one
facilitator with another qualified facilitator should there be any staff turnover in the middle of a
cohort.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The documentation for the annual review period from February 1, 2023 through January 31, 2024
showed thirty-six (36) of the required forty-eight (48) internal fidelity monitoring checks were
completed. Any facilitator trained in Aggression Replacement Training may conduct the internal
fidelity monitoring. Monthly internal fidelity monitoring checks are conducted by the provider’s
regional director of clinical services, two therapeutic managers, two clinical directors (off-site
provider staff), and one director of student services (off-site provider staff).
The program can earn 1 point if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 1.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The documentation showed there was no corrective actions identified for the facilitators of
Aggression Replacement Training given during the annual review period of February 1, 2023
through January 31, 2024. During an informal interview with the regional compliance director, it
was explained that if a need for corrective action is found, the facilitator will receive progressive
discipline to include coaching, retraining, and formal write-up(s).
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program provided an annual evaluation for one Aggression Replacement Training (ART)
facilitator (therapeutic manager) for the annual review period of February 1, 2023 through January
31, 2024. The evaluation provided does not list the specific service the staff member is facilitating,
nor did it address the facilitator’s skill in delivering the primary service.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 10 weeks for this type of service. Of the 14 youth in the
sample, 93% (13 of 14) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 30 contact hours for this type of service. Of the 14 youth in the
sample, 93% (13 of 14) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 18 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 72%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 71%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 3 youth
Moderate-High = 2 youth
High = 5 youth
Total Youth in Sample = 14 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 36%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 5 youth
Total Youth in Sample = 14 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 71% (10 of 14) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 36% (5 of 14) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 18 25 72%
Totals 89 100 89%
This SPEP report evaluates Aggression Replacement Training, an intervention delivered at Palm
Beach Youth Academy.
16 youth completed the program during the review period. Of those youth, 14 were included in the
sample.
The program scored High for Quality of Service Delivery.
The targeted number of weeks for this specific intervention is 10 weeks; the targeted number of
hours for this specific intervention is 30 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 5 services (15 weeks; 45
hours).
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 8 points for Amount of Service: Duration. Of the 14 total youth sampled, all
14 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 13
received at least the recommended weeks of service. Youth in the sample completed between 4
and 16 weeks of service, with an average of 11 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 14 total youth
sampled, all 14 included dosage in the EBS Module. Of those youth with correct dosage, 13
received at least the recommended hours of service. Youth in the sample completed between 14
and 42 hours of service, with an average of 30 hours.
The program was awarded 18 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 96%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Palm Beach Youth Academy can optimize their SPEP Quality of Service Delivery score by
ensuring fidelity monitoring is completed every month for each facilitator and each facilitator’s
performance evaluation includes discussion of their skill in delivering this primary service.
Palm Beach Youth Academy can optimize their SPEP Amount of Service score by ensuring
dosage for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

source: DJJ

report_type: SPEP

file_name: FY2023-24 PBYA ART.pdf

program_name_raw: Palm Beach Youth Academy – Rite of Passage

program_status_note: Closed

service_type: Aggression Replacement Training


summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Palm Beach Youth Academy
Rite of Passage
(Contract Provider)
9680 Weisman Way
West Palm Beach, Florida 33411
Primary Service: Impact of Crime
SPEP Review Period: 2/1/2023-1/31/2024
SPEP Review Conducted: 2/21/2024-2/22/2024
Florida Department of Juvenile Justice
Report Date: 6/27/2024
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The documentation for the annual review period of February 1, 2023 through January 31, 2024
shows the program uses Impact of Crime as a primary service. Facilitator training consists of a
three-day, twenty-four (24) hour training provided by the Department of Juvenile Justice. This
training is available to new facilitators both in-person and via Microsoft Teams. The
documentation shows there were seven staff members trained in the primary service during this
review period: one director of case management, one director of student services, and five case
managers. The director of case management and four case managers facilitated Impact of Crime
during the annual review period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the manual titled Impact of Crime: Addressing the Harm. The manual includes
seven chapters and provides a script for the facilitator to follow. The curriculum includes a student
guide with exercises for the participants to complete to reinforce the material presented during
group sessions. The program does not assign a student guide to each individual participant,
rather the facilitator makes copies of the exercises and provides one to each youth during the
lesson. The program maintains all SPEP documentation (attendance sheets and internal fidelity
checks) in a separate binder labeled Impact of Crime. All youth admitted to the program are
required to participate in this primary service. Impact of Crime is a closed group with no more than
twelve (12) youth assigned to a cohort.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
A technical assistance specialist observed a session of Impact of Crime on-site on February 22,
2024. The group began at 2:45 p.m. and was held in a classroom. A copy of the lesson plan was
not provided to the technical assistance specialist for review. There were seven youth, one
facilitator, and one direct care staff present. The group began with an ice breaker activity and a
brief review of the previous lesson before proceeding to the lesson of the day. The facilitator did
have a copy of the facilitator’s manual available during this session. The technical assistance
specialist observed no behavior management issues. The group ended at 3:47 p.m.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The documentation for the annual review period of February 1, 2023 through January 31, 2024
shows the program has experienced facilitator turnover. This turnover did not adversely affect the
delivery of Impact of Crime. One director of case management, one director of student services,
and two case managers left the program during this annual review period. The two case
managers left in the middle of their cohorts; however, the co-facilitator took over the group
immediately. There were no gaps in service during the annual review period caused by a lack of
trained facilitators. There were two cohorts were completed and two cohorts were ongoing at the
time of this annual review.
The first cohort began on February 22, 2023 with twelve (12) youth and ended on June 13, 2023
with eight youth. Three youth were dropped from the cohort due to being arrested. One youth was
dropped from the cohort due to repeated refusals to attend group.
The second cohort began on June 27, 2023 with twelve (12) youth and ended on November 5,
2023 with seven youth. Five youth were dropped from the cohort due to physical aggression.
The third cohort began on October 19, 2023 with twelve (12) youth and was on going at the end of
the annual review period with nine youth. Three youth were dropped from this cohort due to being
arrested.
The fourth cohort began on December 7, 2023 with twelve (12) youth and was on going at the end
of the annual review period with twelve (12) youth.
An informal interview with the director of case management confirmed the program will replace
one facilitator with another qualified facilitator should there be any staff turnover in the middle of a
cohort.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The documentation for the annual review period from February 1, 2023 through January 31, 2024
showed there were seventeen (17) out of the required seventeen (17) internal fidelity monitoring
checks completed for the facilitators of Impact of Crime. Any facilitator that is trained in Impact of
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Crime can conduct the internal fidelity monitoring. Internal fidelity monitoring checks were
conducted by the one regional compliance director and one director of student services who were
trained in the service.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The documentation shows there was no need for corrective actions identified for the facilitators of
Impact of Crime during the annual review period of February 1, 2023 through January 31, 2024.
During an informal interview with the director of case management, it was explained that if a need
for corrective action is found, the facilitator will receive progressive discipline to include coaching,
retraining, and formal write-up(s).
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Copies of the annual employee evaluations for the Impact of Crime facilitators were not provided
to the technical assistance specialist for the annual review period of February 1, 2023 through
January 31, 2024.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates the target duration of 16 weeks for this type of service. Of the 14 youth in the
sample, 50% (7 of 14) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates a target of 24 contact hours for this type of service. Of the 17 youth in the
sample, 50% (7 of 14) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 18 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 72%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 71%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 3 youth
Moderate-High = 2 youth
High = 5 youth
Total Youth in Sample = 14 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 36%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 5 youth
Total Youth in Sample = 14 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 71% (10 of 14) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 36% (5 of 14) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 4 10 40%
Amount of Service: Contact Hours 4 10 40%
Risk Level of Youth Served 18 25 72%
Totals 66 85 78%
This SPEP report evaluates Impact of Crime, an intervention delivered at Palm Beach Youth
Academy.
16 youth completed the program during the review period. Of those youth, 14 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 4 points for Amount of Service: Duration. Of the 14 total youth sampled, only
8 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 7
received at least the recommended weeks of service. Youth in the sample completed between 9
and 16 weeks of service, with an average of 15 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 4 points for Amount of Service: Contact Hours. Of the 14 total youth
sampled, only 13 included dosage in the EBS Module. Of those youth with correct dosage, 7
received at least the recommended hours of service. Youth in the sample completed between 9
and 32 hours of service, with an average of 23 hours.
The program was awarded 18 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 86%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Palm Beach Youth Academy can optimize their SPEP Quality of Service Delivery score by
ensuring each facilitator’s performance evaluation includes discussion of their skill in delivering
this primary service.
Palm Beach Youth Academy can optimize their SPEP Amount of Service score by ensuring
dosage for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2023-24 PBYA IOC.pdf

service_type: Impact of Crime