summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Crestview Sex Offender Program
Gulf Coast Youth Services
(Contract Provider)
4445 Straightline Road
Crestview, Florida 32539
Primary Service: Impact of Crime
SPEP Review Period: May 1, 2020 - April 30, 2021
SPEP Review Conducted: June 14, 2021
Florida Department of Juvenile Justice
Report Date: 8/9/21
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program's contract documents indicate case management staff is trained to conduct Impact
of Crime groups. There are five total staff trained to facilitate Impact of Crime: four case managers
and the case management supervisor. There are no specific case managers assigned to facilitate
IOC for the youth in the sex offender program. Accodring to the case management supervisor, the
case managers rotate based upon where the need is for the group in the program.
All staff attended a formal training and all documentation was provided for review. The case
management staff rotate faciliting Impact of Crime cohorts throughout the twelve month review
period.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
All records were provided for review for Impact of Crime (IOC). IOC is held as a closed group,
meaning once a group has started, youth cannot enter a group and must wait for the next group to
begin. The groups are held Mondays and Thursdays from 4:00pm to 5:00pm. The program
maintains the current IOC manual. The manual includes a detailed script for the facilitator, all
lesson worksheets, and the student workbook. The student workbook can be copied and provided
to the youth during each cohort.
All sign-in sheets, internal fidelity monitoring forms, and training records are maintained in a
binder in the case management supervisors office. All youth are eligible to attend this curriculum.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Impact of Crime was not observed during this review period and will be rated as non-
applicable. The reviewer did set up an observation; however, no one from the program called in
for the group.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
An interview with the case management supervisor and a review of the program's documentation
showed the program has not experienced any turnover during the twelve month review period.
If the program experiences turnover during a cohort, another trained case manager or the case
management supervisor will continue the group.
The program has completed ten cohorts during the twelve month review period.
The first cohort was started on March 24, 2021 and ended on June 25, 2020. Seven youth started
the group and six youth ended the group successfully.
The second cohort was started on March 24, 2020 and ended on July 9, 2020. Eight youth started
the group and seven youth successfully completed.
The third cohort was started on March 26, 2020 and ended on June 30, 2020. Eight youth started
the group and eight youth successfully completed.
The fourth cohort was started on June 1, 2020 and ended on September 14, 2020. The group
started with five youth and ended successfully with five youth.
The fifth cohort was started August 10, 2020 and ended on September 21, 2020. The reviewer
spoke with the case management supervisor regarding the short time for the group.The case
management supervisor indicated the groups were held for two hours at a time. There were only
three youth in the group and these youth needed to receive the service prior to their release. It
was decided the groups would be held for two hours at a time so the youth could complete the
curriculum prior to release. Three youth started the cohort and three youth successfully completed
the cohort.
The sixth cohort began on September 15, 2020 and ended on December 17, 2020. This group
started with eight youth and eight youth successfully completed. The weeks were short for this
group due to some of the groups being held for an hour-and-a-half or two hours at a time. When
asked about this, it was, again, to ensure all youth in the group were able to complete the
curriculum prior to their release.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The seventh cohort began on November 17, 2020 and was ended on February 23, 2021. Eight
youth started this group and eight youth successfully completed. Four of these sessions were held
for two hours at a time.
The eighth cohort began on December 8, 2020 and ended on March 25, 2021. Eight youth started
the group and seven youth successfully completed. One youth was transferred from the program
during the cohort.
The ninth cohort began on December 28, 2020 and ended on April 7, 2021. Eight youth started
the group and eight youth successfully completed.
The tenth cohort began on January 19, 2021 and ended on May 6, 2021. Eight youth started this
cohort and eight youth successfully completed the group.
Three cohorts are ongoing at the time of this review. One cohort started on March 16, 2021, with
eight youth. A second cohort was started April 6, 2021, with eight youth.
A third cohort was started on May 17, 2021. One youth is doing individual sessions. According to
the case management supervisor and case manager, the youth was transferred in from another
program where he had started the Impact of Crime curriculum. Since groups had already been
started when he was transferred into the program, it was decided to do individual sessions in
order to allow him to successfully complete the group.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The case management supervisor conducts all internal fidelity monitoring for the Impact of Crime
curriculum. All case managers rotate conducting the Impact of Crime groups.
Fourty-five out of fourty-five internal fidelity monitoring forms were provided for the twelve month
review period. There were no missing internal fidelity monitoring forms for the twelve month
review period. All Impact of Crime facilitators were included in the internal fidelity monitoring. The
program is using the correct internal fidelity monitoring forms and the forms include a feedback
section for the facilitator.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The case management supervisor addresses any issues or corrective actions with the case
management supervisor. All internal fidelity monitoring forms were provided for review and there
were no corrective actions noted.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
g. Evaluation of Facilitator Skill Delivering the Basic Score: 0 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
All case management yearly performance evaluations were provided for review.
Each includes a section for delinquency intervention service facilitation. This section lists all
services the facilitator is trained in but does not include discussion of their skill in delivering Impact
of Crime.
The program can earn 1 point if each facilitator’s performance evaluation includes discussion of
their skill in delivering this primary service.
At the time of the review, the program did not meet criteria to earn a score of 1.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
3. Amount of Service - Duration
Basic Score: 4 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 40%
Research indicates the target duration of 16 weeks for this type of service. Of the 7 youth in the
sample, 57% (4 of 7) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 7 youth in the
sample, 100% (7 of 7) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 13 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 52%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 71%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 1 youth
Moderate-High = 2 youth
High = 2 youth
Total Youth in Sample = 7 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
Percentage of Youth with High-Risk Level to Reoffend: 29%
High Score: 8 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 62%
Table 2
High = 2 youth
Total Youth in Sample = 7 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 71% (5 of 7) youth scored Moderate to High-Risk to reoffend, for a score of
5 points.
Of the SPEP sample, 29% (2 of 7) youth scored High-Risk to reoffend, for a score of 8 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 4 10 40%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 13 25 52%
Totals 67 85 79%
This SPEP report evaluates Impact of Crime, an intervention delivered at Crestview Sex Offender
Program.
7 youth completed the program during the review period. Of those youth, all 7 were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 4 points for Amount of Service: Duration. Of the 7 total youth sampled, all 7
included dosage with end dates in the EBS Module. Of those youth, 4 received at least the
recommended weeks of service. Youth in the sample completed between 14 and 17 weeks of
service, with an average of 15 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9
The program earned 10 points for Amount of Service: Contact Hours. Of the 7 total youth
sampled, all 7 included dosage in the EBS Module. Of those youth, 7 received at least the
recommended hours of service. Youth in the sample completed between 29 and 33 hours of
service, with an average of 32 hours.
The program was awarded 13 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Based on the risk level of the youth placed at the program (52% Program Optimization
Percentage), the Department should work to ensure a larger proportion of higher risk youth are
recommended. As the program serves youth with a history of sexual behavior problems, this
includes a multitier strategy of working with stakeholders to achieve that practice, as well as
ensuring appropriate treatment services are available in the community to reduce the reliance on
residential commitment for lower risk youth presenting with sexually-related offenses.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 93%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Crestview Sex Offender Program can optimize their SPEP Quality of Service Delivery score by
ensuring each facilitator’s performance evaluation includes discussion of their skill in delivering
this primary service.
Crestview Sex Offender Program can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 10


state: FL

facility_name: Crestview Youth Academy

summary: SPEP - FY20-21

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Crestview Sex Offender Program
Gulf Coast Youth Services
(Contract Provider)
4445 Straightline Road
Crestview, Florida 32539
Primary Service: Pathways/Roadmaps to Recovery
SPEP Review Period: May 1, 2020- April 30, 2021
SPEP Review Conducted: June 14, 2021
Florida Department of Juvenile Justice
Report Date: 8/9/21
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Pathways/Roadmaps to Recovery.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes Pathways/Roadmaps to Recovery as a primary service for the program. The
program's documentation and contract reflects licensed therapy staff are trained to conduct the
Pathways curriculum. All training documentation was provided for the twelve month review period.
The clinical director conducts training for the therapy staff for the Pathways curriculum. Two
therapy staff and the clinical director are trained to facilitate the Pathways curriculum. There is
one main facilitator for these groups in the program.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program provided the Pathways/Roadmaps to Recovery manual for review. The manual is a
guided workbook the youth work through on their own with a therapist. There are thirteen chapters
with twelve-steps to complete in chapter twelve. There is not a script for the facilitator to follow.
The manual is broken into individual assignments and reading passages.
Pathways is a curriculum designed for sexual offenders. All youth admitted into the program
attend the Pathways group prior to being released. Pathways is provided four days a week,
Monday through Thursday from 2:15pm to 3:15pm and 3:15pm to 4:15pm, based upon current
census.
Attendance sheets are bound by month with all other substance abuse and mental health groups
and maintained in the clinical director's office. The sign in sheets, weekly clinical summaries, and
fidelity monitoring forms are maintained in the clinical director's. Pathways is held as an open
group meaning youth can enter the group at any time.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Pathways/Roadmaps to Recovery was not able to be oberved; therefore this
indicator is rated as non-applicable. The reviewer attempted on three occassions to schedule a
virtual observation, but was unable to complete this due to internet issues at the program.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
An interview with a licensed therapist, who was sitting in for the clinical director on this review
date confirmed the program has not experienced any turnover during the twelve month review
period. Should the program experience turnover, another trained licensed therapist would
continue conducting the Pathways/Roadmaps to Recovery groups. All group sign-in sheets were
provided for review. There were no gaps in service during this review period.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program's policy is for the clinical director to conduct internal fideilty monitoring once a month
for each facilitator. There has been one main facilitator for the Pathways/Roadmaps to Recovery
groups during this twelve month review period.
Internal fidelity monitoring forms are located in a binder maintained in the clinical director's office.
Twelve out of twelve internal fidelity monitoring forms were provided for the review period. There
were no missing internal fidelity monitoring forms for this review period. The internal fidelity
monitoring forms include a section for feedback/corrective actions. All forms included feedback for
the facilitator.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The clinical director addresses any issues or corrective actions for faciltatiors of the
Pathways/Roadmaps to Recovery curriculum. The clinical director conducts weekly supervisons
with therapy staff. Any issue or corrective actions are addressed during those meetings or one-to-
one, when necessary. There were no corrective actions noted on the internal fidelity monitoring
forms for the twelve month review period.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The facilitator for the Pathways/Roadmaps to Recovery curriculum received an annual
performance evaluation at the time of this review. The annual performance evaluation includes a
section for group facilitation and the specific groups the employee is trained to facilitate. This
section also includes feedback for the facilitator.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 24 weeks for this type of service. Of the 7 youth in the
sample, 100% (7 of 7) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 40 contact hours for this type of service. Of the 7 youth in the
sample, 100% (7 of 7) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 13 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 52%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 71%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 1 youth
Moderate-High = 2 youth
High = 2 youth
Total Youth in Sample = 7 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 29%
High Score: 8 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 62%
Table 2
High = 2 youth
Total Youth in Sample = 7 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 71% (5 of 7) youth scored Moderate to High-Risk to reoffend, for a score of
5 points.
Of the SPEP sample, 29% (2 of 7) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 13 25 52%
Totals 83 95 87%
This SPEP report evaluates Pathways/Roadmaps to Recovery, an intervention delivered at
Crestview Sex Offender Program.
7 youth completed the program during the review period. Of those youth, all 7 were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 7 total youth sampled, all 7
included dosage with end dates in the EBS Module. Of those youth, 7 received at least the
recommended weeks of service. Youth in the sample completed between 30 and 49 weeks of
service, with an average of 38 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 10 points for Amount of Service: Contact Hours. Of the 7 total youth
sampled, all 7 included dosage in the EBS Module. Of those youth, 7 received at least the
recommended hours of service. Youth in the sample completed 52 hours of service.
The program was awarded 13 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT) assessment. This score reflects youths'
most recent CAT score prior to placement at the program. The program itself has no control over
youths' CAT risk level because the scored assessment was administered prior to the youths'
admission.
Based on the risk level of the youth placed at the program (52% Program Optimization
Percentage), the Department should work to ensure a larger proportion of higher risk youth are
recommended. As the program serves youth with a history of sexual behavior problems, this
includes a multitier strategy of working with stakeholders to achieve that practice, as well as
ensuring appropriate treatment services are available in the community to reduce the reliance on
residential commitment for lower risk youth presenting with sexually-related offenses.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 100%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Crestview Sex Offender Program can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Crestview Sex Offender Program can optimize their SPEP Amount of Service score by ensuring
that dosage for all youth is recorded accurately in EBS and by ensuring that youth receive the full
targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

source: DJJ

report_type: SPEP

program_name_raw: Crestview Sex Offender Program

program_status_note: Closed

service_type: Pathways/Roadmaps to Recovery


summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Crestview Youth Academy - Secure
Youth Opportunity Investments, LLC
(Contract Provider)
4449 Straightline Road
Crestview, Florida 32539
Primary Service: Seven Challenges
SPEP Review Period: February 14, 2021 - February 13, 2022
SPEP Review Conducted: March 2, 2022
Florida Department of Juvenile Justice
Report Date: 5/31/22
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Seven Challenges.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes Seven Challenges as a primary service for the secured program. The
program's protocol is for all therapy staff to be trained to conduct Seven Challenges groups.
During the twelve month review period, eleven staff were trained to facilitated Seven Challenges.
All training documentation was provided for review and all staff were trained, per program
protocol.
All staff conducted Seven Challenges groups during the twelve month review period at some
point. The program's policy is for the therapists to rotate group facilitation. The clinical director
conducts internal fidelity monitoring monthly.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program has a manual for the delivery of services. The program currently uses the Seven
Challenges manual by Rober Schwebel, PhD. The youth also have workbooks to work through at
their own pace. These workbooks include lesson readings and journaling pages. The facilitator
has a master manual which has all of the lessons.
Seven Challenges groups are held on Thursdays, Saturdays, and Sundays from 2:30pm to
3:30pm. Two groups are held each of the three days of group scheduled. Seven Challenges
groups are held as open groups, meaning the youth can begin the group when they enter the
program and can stay in the group throughout their time in the program until they are completed.
Youth with substance abuse diagnoses are eligible to attend the Seven Challenges groups.
The program maintains records for the Seven Challenges groups in two binders maintained in the
clinical director's office. One binder includes all sign-in sheets for the groups. The second binder
includes all training records and monthly internal fidelity monitoring observation forms.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Seven Challeges was unable to be observed; therefore, this indicator rates as non-
applicable.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program experienced turnover during the twelve month review period. The therapy staff
rotates facilitating the Seven Challenges groups on a regular basis. Seven Challenges is held as
an open group, so there were no cohorts.
In the event of turnover, another trained facilitator conducts Seven Challenges group and the
clinical director adjusts the therapists rotation for facilitation. During the twelve month review
period, seven therapy staff left employment. A review of the sign-in sheets indicated there were
no gaps in service. Seven Challenges groups were held consistently based upon the program
schedule.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The clinical director typically conducts internal fidelity monitoring observations for the Seven
Challenges groups. The program keeps all internal fidelity monitoring forms in a binder maintained
in the clinical director's office. All fidelity forms were provided for review. Thirty-four out of thirty-
nine internal fidelity monitoring observations were provided for review for the twelve month review
period.
Two internal fidelity observations were msising for February 2022 and one in November 2021,
December 2021, and January 2022 for one facilitator. The clinical director was asked about the
missing fidelity observations. It was reported the clinical director has days she has to conduct
group at the same time as the other therapists and is unable to conduct internal fidelity
observations at times.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The clinical director addresses any issues or corrective actions with the facilitators for Seven
Challenges. The clinical director also meets with the therapy staff once a week.
The internal fidelity monitoring forms include areas for the observer to provide feedback to the
facilitator. A review of the internal fidelity monitoring forms confirmed there were no significant
issues or corrective actions noted.
g. Evaluation of Facilitator Skill Delivering the Basic Score: N/A Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The annual performance evaluation document for Youth Opportunity Investments Includes a
section for group facilitation with the specific groups the facilitator conducts.
All therapy staff who facilitate the Seven Challenges groups have been employed less than a year
and have not received an annual performance evaluation. A blank copy was provided for review.
There were no facilitators eligible for an annual evaluation, therefore, this indicator is rated as
non-applicable.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 24 weeks for this type of service. Of the 33 youth in the
sample, 97% (32 of 33) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 40 contact hours for this type of service. Of the 33 youth in the
sample, 100% (33 of 33) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 94%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 8 youth
Moderate-High = 9 youth
High = 14 youth
Total Youth in Sample = 33 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 42%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 14 youth
Total Youth in Sample = 33 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 94% (31 of 33) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 42% (14 of 33) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 23 25 92%
Totals 81 95 85%
This SPEP report evaluates Seven Challenges, an intervention delivered at Crestview Youth
Academy - Secure.
33 youth completed the program during the review period.
The program scored Medium for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 33 total youth sampled, all
33 included dosage with end dates in the EBS Module. Of those youth, 32 received at least the
recommended weeks of service. Youth in the sample completed between 23 and 41 weeks of
service, with an average of 36 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 33 total youth
sampled, all 33 included dosage in the EBS Module. Of those youth, 33 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
recommended hours of service. Youth in the sample completed between 67 and 100 hours of
service, with an average of 98 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
RECOMMENDATION(S):
Crestview Youth Academy - Secure can optimize their SPEP Quality of Service Delivery score by
ensuring fidelity monitoring is completed every month for each facilitator.
Crestview Youth Academy - Secure can optimize their SPEP Amount of Service score by
ensuring that dosage for all youth is recorded accurately in EBS and by ensuring that youth
receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2021-22 CYA-SEC 7C.pdf

program_name_raw: Crestview Youth Academy Secure

program_status_note: Open

service_type: Seven Challenges


summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Crestview Youth Academy - Nonsecure
Youth Opportunity Investments, LLC
(Contract Provider)
4449 Straightline Road
Crestview, Florida 32539
Primary Service: Skillstreaming the Adolescent
SPEP Review Period: February 14, 2021 - February 13, 2022
SPEP Review Conducted: March 2, 2022
Florida Department of Juvenile Justice
Report Date: 5/26/22
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Skillstreaming the Adolescent.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes Skillstreaming the Adolescent as a primary service. The therapy staff are
trained to conduct the Skillstreaming groups. There were thirteen staff trained during the twelve
month review period; ten of the thirteen staff conducted groups at some point during the twelve
month review period. The program rotates the therapy staff for Skillstreaming groups so the youth
are familiar with each facilitator. The clinical director conducts internal fidelity monitoring for the
Skillstreaming groups.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program has a manual for the delivery of services. The program currently uses The Prepare
Curriculum manual by Arnold Goldstein. The program also utilizes the Skillstreaming the
Adolescent skill cards. The manual does not have a specific script for each lesson. It is divided
into exercises for each lesson. Each lesson also contains the objective, materials, procedure,
guided questions and discussion. Once the lesson is completed, the following exercise is
practicing the skill using the identified skill card.
Skillstreaming the Adolescent groups are held on Mondays and Wednesdays from 3:30pm to
4:30pm. Documentation for the groups is maintained in two binders. One binder has the sign-in
sheets for the groups. The second binder has all training documentation and the internal fidelity
observations forms. These binders are kept in the clinical director's office.
All youth are eligible to attend Skillstreaming groups. These groups are held as open, meaning
the youth may enter the group upon admission to the program and can work through the
curriculum throughout their stay.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Skillstreaming the Adolescent was observed remotely on Monday, August 17, 2020
from 3:35pm to 4:35pm. There were six youth in the group. The skill being taught was listening.
One therapist facilitated this group and there was a staff member present. The youth reviewed the
skill steps and then began role plays. One youth was not participating much at the beginning, but
the other youth and the facilitator did a good job encouraging this youth to participate. The youth
did end up participating in a role play and provided feedback. The facilitator did a good job
pausing the role-plays when they needed to correct any behaviors and go through the steps
again. The facilitator also played the telelphone game with the group to further the skill steps for
listening.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program's process, in the event of a turnover, is for other trained staff to continue with the
Skillstreaming groups. The therapists rotate group facilitation on a regular basis so the youth are
familiar with them and there is less of an impact if the facilitator should be out or leave
employment.
The program did experience turnover during the twelve month review period. Nine therapy staff
left employment during the annual review period. The previous clinical director did leave full-time
employment, but is available Pro Re Nata (PRN). The previous clinical director has come in to
assist the new clinical director with training of new staff.
Skillstreaming the Adolescent is held as an open group, meaning the youth can join the group
upon admission to the program and work through the curriculum throughout their time in the
program. All sign-in sheets were provided for review. There were no gaps in service. The group
had, at a minimum, eight youth in each.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program's policy is for the clinical director to conduct internal fidelity monitoring observations
for the facilitators of Skillstreaming the Adolescent groups.
There were thirteen therapists who conducted Skillstreaming groups during the twelve month
review period. Fifty out of sixty-one internal fidelity monitoring observations were completed. One
observation was missing for one facilitator in September, two were missing for October,
November, December, January, and February. This reviewer spoke with the clinical director
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
regarding the missing fidelity observations. It was reported the clinical director is also facilitating
groups at this time and it is difficult for all fidelity observations to be completed monthly.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The clinical director addresses any significant issues or corrective actions with the facilitators of
Skillstreaming the Adolescent. All completed internal fidelity monitoring forms were provided for
review. The forms include a section to document any issues or corrective actions for facilitators.
All forms did include feedback. There were no significant issues or corrective actions noted for the
annual review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: N/A Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The annual performance evaluation includes specific information about facilitation and delivery of
the employee's specific intervention(s).
There are no current facilitators for Skillstreaming the Adolescent groups who have been
employed long enough to receive an annual performance evaluation. A blank performance
evaluation was provided for review.
There were no facilitators eligible for an annual evaluation, therefore, this indicator is rated as
non-applicable.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 16 weeks for this type of service. Of the 26 youth in the
sample, 100% (26 of 26) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 26 youth in the
sample, 100% (26 of 26) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 3 youth
Moderate-High = 8 youth
High = 15 youth
Total Youth in Sample = 26 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 58%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 15 youth
Total Youth in Sample = 26 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (26 of 26) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 58% (15 of 26) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 25 25 100%
Totals 85 85 100%
This SPEP report evaluates Skillstreaming the Adolescent, an intervention delivered at Crestview
Youth Academy - Nonsecure.
26 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 26 total youth sampled, all
26 included dosage with end dates in the EBS Module. Of those youth, 26 received at least the
recommended weeks of service. Youth in the sample completed between 23 and 29 weeks of
service, with an average of 26 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 26 total youth
sampled, all 26 included dosage in the EBS Module. Of those youth, 26 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
recommended hours of service. Youth in the sample completed between 44 and 50 hours of
service, with an average of 49 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
RECOMMENDATION(S):
Crestview Youth Academy - Nonsecure can optimize their SPEP Quality of Service Delivery score
by ensuring fidelity monitoring is completed every month for each facilitator.
Crestview Youth Academy - Nonsecure can optimize their SPEP Amount of Service score by
ensuring that dosage for all youth is recorded accurately in EBS and by ensuring that youth
receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2021-22 CYA-NS Skillstreaming.pdf

program_name_raw: Crestview Youth Academy Nonsecure

service_type: Skillstreaming the Adolescent


summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Crestview Youth Academy - Nonsecure
Youth Opportunity Investments, LLC
(Contract Provider)
4449 Straightline Road
Crestview, Florida 32539
Primary Service: Male Healthy Relationships
SPEP Review Period: February 14, 2021 - February 13, 2022
SPEP Review Conducted: March 2, 2022
Florida Department of Juvenile Justice
Report Date: 5/26/22
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Male Healthy Relationships.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 8 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
Male Healthy Relationships (MHR) is a gender specific curriculum utilized as a primary service in
the program. MHR is a curriculum combined of Young Men's Work and The Teen Relationships
Workbook. Therapy staff are trained to conduct MHR in the program.
All training records were provided for review. There have been thirteen therapy staff trained to
facilitate MHR during the annual review period. All thirteen have conducted MHR groups at some
point during the twelve month review period. The clinical director typically conducts internal fidelity
observations for the MHR groups.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
Male Healthy Relationships (MHR) is a bundled group of The Teen Relationships Workbook and
Young Men's Work and is utilized as a primary service for the program. The program maintains
two manuals the therapists work from; Young Men's Work and Teen Relationships. The Young
Men's Work manual is maintained in a binder in the clinical director's office. It is approximately ten
sessions and includes videos and handouts. The manual is broken into individual sessions that
include the objective, agenda, session outline, and all handouts. The lessons also include a script
to follow. The manual for Teen Relationships is maintained in a binder in the clinical director's
office. It is five main topics broken down into individual lessons. All lessons and handouts are
included. Facilitator guidelines are also included, but no script is provided.
The Teen Relationships Workbook group is held on Tuesdays from 3:30pm to 4:30pm and Young
Men's Work is held on Fridays from 3:30pm to 4:30pm. The records for the intervention are
maintained in two binders kept in the clinical director's office. One binder holds the sign-in sheets
for the groups. The second binder includes the internal fidelity monitoring forms and training
records.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
All youth are eligible to receive this service. The groups are held as open, meaning the youth are
able to work through the curriculum throughout their stay at the program. A review of sign-in
sheets and the program schedule confirmed groups are being held on a consistent basis.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A group observation of Young Men's Work was conducted on Friday, November 5, 2021 from
3:35pm to 4:35pm. There were ten youth in the group, one facilitator, and one staff member. The
lesson being covered for that day was Taking Care of Myself.
This observer was provided a copy of the lesson to follow during the group. The facilitator was
comfortable conducting the lesson and was able to use her own words while using the lesson as a
guide. The youth were well behaved with no behavior issues. The youth volunteered to read
during the group and activities. The youth did have a lot of energy during the group, but the
facilitator did an excellent job bringing the youth back to the topic and stay on topic.
The only suggestion for the facilitator was to review group expectations before each group
meeting.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program's policy, in the event of turnover, is for another trained therapist to continue with
group faciltiation. The program has a schedule of the therapists conducting groups in a regular
rotation so the youth are familiar with the different staff and there is less disruption to the group.
Seven therapy staff left during the twelve month review period. The previous clinical director did
leave full-time employment, but is available pro re nata (PRN). The previous clinical director has
come in to assist the new clinical director with training of new staff.
All sign-in sheets were provided for review. There were no gaps in service during the annual
review period. There were eight youth in each group, on average.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The clinical director conducts internal fidelity monitoring for Male Healthy Relationships. Youth
Opportunity Investments requires each facilitator be observed two times a month, once for Young
Men's Work and once for Teen Relationships.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The fidelity monitoring forms are maintained in a binder kept in the clinical director's office. The
forms are organized by month and by therapist. Forty-nine out of fifty internal fidelity monitoring
observations were completed during the twelve month review period. One facilitator was missing
a fidelity observation for November 2021.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The clinical director addresses any significant issues or corrective actions with the facilitators for
Male Healthy Relationships. The internal fidelity forms include a section to document any issues
or corrective actions for the facilitators.
All forms were provided for the twelve month review period. There were no corrective actions
noted nor taken.
g. Evaluation of Facilitator Skill Delivering the Basic Score: N/A Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The annual performance evaluations for Youth Opportunity Investments include a section for
facilitator skill. There is a space to indicate the curricula the employee facilitates at the program.
All therapy staff trained to facilitate Male Healthy Relationships have been employed less than
one year. Therefore, no annual performance evaluations were completed at the time of the annual
review; therefore, this indicator is rated as non-applicable.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 24 weeks for this type of service. Of the 26 youth in the
sample, 92% (24 of 26) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 40 contact hours for this type of service. Of the 26 youth in the
sample, 100% (26 of 26) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 3 youth
Moderate-High = 8 youth
High = 15 youth
Total Youth in Sample = 26 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 58%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 15 youth
Total Youth in Sample = 26 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (26 of 26) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 58% (15 of 26) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 25 25 100%
Totals 93 95 98%
This SPEP report evaluates Male Healthy Relationships, an intervention delivered at Crestview
Youth Academy - Nonsecure.
26 youth completed the program during the review period. Of those youth, all 26 were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 26 total youth sampled, only
25 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 24
received at least the recommended weeks of service. Youth in the sample completed between 23
and 41 weeks of service, with an average of 29 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 10 points for Amount of Service: Contact Hours. Of the 26 total youth
sampled, all 26 included dosage in the EBS Module. Of those youth, 26 received at least the
recommended hours of service. Youth in the sample completed between 42 and 78 hours of
service, with an average of 56 hours.
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
RECOMMENDATION(S):
Crestview Youth Academy - Nonsecure can optimize their SPEP Quality of Service Delivery score
by ensuring fidelity monitoring is completed every month for each facilitator.
Crestview Youth Academy - Nonsecure can optimize their SPEP Amount of Service score by
ensuring that dosage for all youth is recorded accurately in EBS and by ensuring that youth
receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2021-22 CYA-NS MHR.pdf

service_type: Male Healthy Relationships


summary: SPEP - FY21-22

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Crestview Youth Academy - Secure
Youth Opportunity Investments, LLC
(Contract Provider)
4449 Straightline Road
Crestview, Florida 32539
Primary Service: Impact of Crime
SPEP Review Period: February 14, 2021 - February 13, 2022
SPEP Review Conducted: March 2, 2022
Florida Department of Juvenile Justice
Report Date: 5/31/22
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes Impact of Crime (IOC) as a primary service in the program. The case
management staff are trained to conduct IOC groups. There have been six staff who facilitated
IOC groups during the annual review period. Four of the six staff were trained during the annual
review period.
The director of case management conducts the majority of the IOC facilitator internal fidelity
monitoring observations. If the director of case management is unable to conduct the fidelity
monitoring, the facilitators are able observe one another. The director of case management
reviews the internal fidelity forms.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program has a manual for the delivery of services and provided a copy for this reviewer for
reference during the annual review. The program uses the current Impact of Crime (IOC) manual.
The youth are provided copies of IOC workbooks with all of the needed worksheets. The manual
includes implementation guidelines, identifies the order the lessons should be delivered, includes
detailed instructions for lesson delivery, and all needed worksheets for the youth.
Impact of Crime groups for the secure program are held on Tuesdays and Thursdays from
4:30pm to 5:30pm. The documentation for the IOC groups is maintained in a binder kept in the
director of case management's office. The sign-in sheets are maintained in one binder and the
fidelity monitoring forms and training documentation is maintained in a second binder. Impact of
Crime groups are held as closed groups, meaning no youth can be added to the group once the
first lesson is completed.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
An Impact of Crime (IOC) group was observed on Monday, November 8, 2021 from 2:30pm to
3:30pm. There were seven youth in attendance, one youth care worker, and one facilitator
present during group. The facilitator conducted an ice breaker activity with the youth. A word
search was handed out with a list of words to choose from. The title was "I am thankful for…" and
the youth were to chose two to three words of what they were thankful for and then find the words
within the word search. Then the youth took turns explaining why they chose the words they did.
The youth then took turns writing out the definitions of the vocabulary words from chapter six on
the white board. They were able to put the definitions in their own words. The facilitator allowed
the youth to read what they wrote and provide an example for each word. The youth appeared to
enjoy these activities and all participated in the group activities. There were no behavioral issues
with the youth. The facilitator was confident in teaching the lesson. Exercises one and three were
completed for this chapter. The facilitator was able to summarize the sections and use his own
words to discuss the topics and activities for the youth. The facilitator ensured the youth
understood the concepts prior to moving on in the lesson. The session lasted one full hour.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program's policy in the event of turnover is for another trained facilitator to continue with the
group. The groups are not co-facilitated on a regular basis. The program did experience turnover
during this twelve month review period. Two trained facilitators left employment; however, there
were co-facilitators for both groups and the co-facilitator was able to continue with group
facilitation.
All sign-in sheets were reviewed for the twelve month review period. The program completed
three cohorts during the annual review period. The first cohort was held from December 1, 2020
to March 30, 2021. Ten youth started the group and ten youth successfully completed the group.
The main facilitator of this group left employment during this group and the co-facilitator was able
to continue with the group.
The second cohort started April 15, 2021 and completed on July 30, 2021. The group began with
twelve youth twelve youth successfully completed the group. The main facilitator left employment
during this group, however it was being co-facilitated and the other facilitator was able to complete
this group.
The third cohort started September 10, 2021 and ended on December 28, 2021. The group began
with ten youth and ten youth successfully completed the group. This group was also co-facilitated.
The primary facilitator was put on emergency medical leave. The co-facilitator was able to
continue and successfully complete the group.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
A fourth cohort began on November 9, 2021 and was continuing to be held at the time of this
review. The group started with twelve youth and continues with twelve youth at the time of this
review.
There was one gap in service from July 30, 2021 to September 10, 2021. The director of case
management reported there were not enough youth during this timeframe of forty-two days to
conduct an Impact of Crime group.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The director of case management typically conducts internal fidelity monitoring for the Impact of
Crime groups. If needed, the facilitators can monitor each other. The program utilizes the correct
fidelity monitoring form.
Based upon the group schedules and when groups were held with the facilitators, fifteen out of
fifteen internal fidelity monitoring observations were completed during the annual review period.
The program maintains the internal fidelity observations forms in a binder kept in the director of
case management's office. The program is using the current internal fidelity form for Impact of
Crime.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The director of case management addresses any issues or corrective actions with the facilitators
for the Impact of Crime (IOC). The IOC forms include a section for feedback and to document any
significant issues.
A review of the internal fidelity monitoring forms confirmed there were no significant issues or
corrective actions during the twelve month review period.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program provided an annual performance evaluation form created by the provider. There is a
specific section for group facilitation and it has the specific group the facilitators conduct.
One facilitator for Impact of Crime received an annual performance evaluation during the twleve
month review period. All other facilitators have not been employed long enough to have received
their reviews at the time of the annual review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates the target duration of 16 weeks for this type of service. Of the 33 youth in the
sample, 100% (33 of 33) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 24 contact hours for this type of service. Of the 33 youth in the
sample, 100% (33 of 33) reached the indicated target contact hours. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 94%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 8 youth
Moderate-High = 9 youth
High = 14 youth
Total Youth in Sample = 33 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 42%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 14 youth
Total Youth in Sample = 33 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 94% (31 of 33) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 42% (14 of 33) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 10 10 100%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 23 25 92%
Totals 83 85 98%
This SPEP report evaluates Impact of Crime, an intervention delivered at Crestview Youth
Academy - Secure.
33 youth completed the program during the review period. Of those youth, all 33 were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 10 points for Amount of Service: Duration. Of the 33 total youth sampled, all
33 included dosage with end dates in the EBS Module. Of those youth, 33 received at least the
recommended weeks of service. Youth in the sample completed between 16 and 36 weeks of
service, with an average of 22 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 10 points for Amount of Service: Contact Hours. Of the 33 total youth
sampled, all 33 included dosage in the EBS Module. Of those youth, 33 received at least the
recommended hours of service. Youth in the sample completed between 32 and 33 hours of
service, with an average of 32 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
RECOMMENDATION(S):
Crestview Youth Academy - Secure can maintain their SPEP Quality of Service Delivery score by
continuing the practices in place at the time of this review.
Crestview Youth Academy - Secure can optimize their SPEP Amount of Service score by
ensuring that dosage for all youth is recorded accurately in EBS and by ensuring that youth
receive the full targeted dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2021-22 CYA-SEC IOC.pdf

service_type: Impact of Crime