summary: SPEP - FY22-23

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Twin Oaks Vocational Academy II
Twin Oaks Juvenile Development, Inc.
(Contract Provider)
29841 Liberty Wilderness Road
Sumatra, Florida 32335
Primary Service: Impact of Crime
SPEP Review Period: November 1, 2021 - October 31, 2022
SPEP Review Conducted: December 6, 2022
Florida Department of Juvenile Justice
Report Date: 2/20/23
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes Impact of Crime (IOC) as a primary service. There are a variety of staff
trained to facilitate the IOC groups. At the time of the annual review, ten (10) staff had been
trained to facilitate IOC groups. One therapist and five case managers facilitated IOC groups
during the annual review period. All training documentation confirmed staff were trained through
the Department of Juvenile Justice.
The trained staff observe each other for monthly internal fidelity monitoring.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program utilizes the current version of the Impact of Crime (IOC) manual. The manual
includes detailed lesson plans, a script for the facilitator, facilitator notes, and implementation
information. The manual also includes IOC workbooks for the youth. All facilitators have the
newest version of the manuals.
The program holds IOC groups two days a week for one hour sessions. The days of the week for
the groups are dependent upon the facilitator's schedule at that time. The schedule for groups at
the time of the annual review was Tuesdays and Thursdays from 11:00am to 12:00pm. All youth
in the program are eligible to attend IOC groups and are required to successfully complete the
curriculum before graduating the program. Impact of Crime groups are held as closed groups,
meaning no youth should be added after the first lesson of chapter one.
The program is working on developing a binder to maintain all records for the IOC cohorts. At the
time of the annual review, the sign-in sheets and fidelity monitoring forms are maintained in
binders by the facilitators of the group. The clinical director indicated the binders will be
consolidated into one binder during the first quarter of the 2023-2024 review period.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
An Impact of Crime (IOC) group was observed on Tuesday, December 13, 2022 from 11:00am to
12:00pm. There were two youth in the group with one facilitator. The youth were on the "I Have a
Dream" part of chapter six. The youth reviewed the video of the speech and discussed it. The
youth were quiet and did not discuss a lot from the video. The facilitator did attempt to engage the
youth in discussion. The youth completed their own I have a dream speeches and shared them.
There were no behavioral issues during the group. The facilitator utilized the manual, but was able
to put everything in her own words to relate to the youth.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program's protocol, in the event of facilitator turnover for Impact of Crime (IOC), is for another
trained staff to continue with group facilitation. During the annual review period, the program did
experience facilitator turnover. Three trained staff left employment during the annual review
period; however, their departures did not adversely affect service delivery.
The program completed four cohorts of IOC during the annual review period. The first cohort
started November 8, 2022 and ended on March 1, 2022. The group began with three youth and
three youth successfully completed the group.
The second cohort began January 24, 2022 and ended on June 1, 2022. There was one youth in
the group; the youth successfully completed the service. The sign-in sheets were not completed
correctly. Many of the groups were held sporadically and inconsistently. The majority of the
groups were more than one hour. There was a gap of three weeks during the group, two of which
were due to the youth being in quarantine.
The third cohort started on March 10, 2022 and ended on August 15, 2022. There was one youth
who attended and succesfully completed the group. There were no dates on the sign-in sheets
between April 7, 2022 and April 20, 2022. There was neither a youth name nor signature on the
sign-in sheet for May 2, 2022.
The fourth cohort began on July 21, 2022 and ended on November 3, 2022. Two youth began the
group and two youth successfully completed the group. This cohort was held more consistently at
one hour a session. Starting the week of September 26, 2022 through October 21, 2022, groups
were held four days a week for one hour.
A fifth cohort was started on October 3, 2022 and was ongoing at the time of the review. The
group began with one youth. One youth was added to the group during chapter one, in
accordance with guidance from the department. Groups were being held consistently for one
hour, two times a week.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The issues with the sign-in sheets has been an ongiong issue. The Technical Assistance
Specialist will be meeting with the IOC facilitators to review how to properly complete the sign-in
sheets for IOC.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program's process for Impact of Crime (IOC) internal fidelity observations is for the trained
staff to observe each other on a monthly basis. During the annual review period, there were two
main facilitators for IOC groups. There were three months missing of internal fidelity observations
for one facilitator the annual review period.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The clinical director addresses any significant issues or corrective actions for the facilitators of
Impact of Crime (IOC) groups. All internal fidelity forms were reviewed for any issues and none
were noted. This reviewer also spoke with the clinical director and there were no corrective
actions for the facilitators of IOC.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
There were six facilitators during the twelve month review period. Of the six facilitators, two were
eligible for their annual performance evaluation. The performance evaluations include a specific
section to document what curricula the employee facilitates and how the employee is doing with
facilitation.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 16 weeks for this type of service. Of the 6 youth in the
sample, 83% (5 of 6) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 24 contact hours for this type of service. Of the 6 youth in the
sample, 83% (5 of 6) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 7 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 28%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 83%
Moderate to High Score: 7 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 58%
Table 1
Moderate = 1 youth
Moderate-High = 4 youth
High = 0 youth
Total Youth in Sample = 6 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 0%
High Score: 0 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 0%
Table 2
High = 0 youth
Total Youth in Sample = 6 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 83% (5 of 6) youth scored Moderate to High-Risk to reoffend, for a score of
7 points.
Of the SPEP sample, 0% (0 of 6) youth scored High-Risk to reoffend, for a score of 0 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 7 25 28%
Totals 63 85 74%
This SPEP report evaluates Impact of Crime, an intervention delivered at Twin Oaks Vocational
Academy II.
6 youth completed the program during the review period. Of those youth, all 6 were included in
the sample.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 6 total youth sampled, only
5 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 5
received at least the recommended weeks of service. Youth in the sample completed between 17
and 25 weeks of service, with an average of 19 weeks.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 8 points for Amount of Service: Contact Hours. Of the 6 total youth sampled,
only 5 included dosage in the EBS Module. Of those youth with correct dosage, 5 received at
least the recommended hours of service. Youth in the sample completed 32 hours of service.
The program was awarded 7 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 95%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Twin Oaks Vocational Academy II can optimize their SPEP Quality of Service Delivery score by
ensuring fidelity monitoring is completed every month for each facilitator.
Twin Oaks Vocational Academy II can optimize their SPEP Amount of Service score by ensuring
dosage for all youth is recorded accurately in EBS and by ensuring youth receive the full targeted
dosage of service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2022-23 TOVAII IOC rev23.3.1.pdf

service_type: Impact of Crime


state: FL

facility_name: Twin Oaks Academy II

summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Twin Oaks Academy II
Twin Oaks Juvenile Development, Inc
(Contract Provider)
29841 Liberty Wilderness Road
Sumatra, Florida 32335
Primary Service: Impact of Crime
SPEP Review Period: November 1, 2022 - October 31, 2023
SPEP Review Conducted: December 5, 2023
Florida Department of Juvenile Justice
Report Date: 6/27/2024
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 10 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program utilizes Impact of Crime (IOC) as a primary service for all youth in the program.
Case management staff are trained to facilitate IOC groups. There is a training protocol in place,
delivered by a qualified trainer. During the review period, there were four staff trained to facilitate
IOC: three case managers and one transition specialist. One case manager is no longer
employed at the program.
There are two primary facilitators; other trained case managers conducted monthly internal fidelity
monitoring.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the current version of the Impact of Crime (IOC) manual. The manual includes
detailed lesson plans, a script for the facilitator, facilitator notes, and implementation information.
The manual also includes IOC workbooks for the youth. All facilitators have the newest version of
the manuals.
IOC groups are typically held on Tuesdays and Thursdays. All youth in the program are eligible to
attend IOC groups and are required to successfully complete the curriculum before graduating the
program. Impact of Crime groups are held as closed groups, meaning no youth should be added
after the first lesson of chapter one.
All records for the IOC cohorts are maintained in one binder, organized by facilitator/cohort.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Impact of Crime was observed on December 14, 2023. Nine youth, the facilitator, two
direct care staff, and the clinical director were present for the session. The session was a final
recap and graduation celebration. The youth took turns presenting key concepts and providing
examples of the concepts. All youth were engaged and excited about successfully completing the
service. The facilitator and direct care staff were encouraging and supportive of the youths’
participation.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program's policy in the event of Impact of Crime (IOC) facilitator turnover is for another
trained staff to continue with group facilitation. One trained facilitator left the program during the
review period, but was not facilitating an IOC group at the time of their departure. The IOC groups
are not co-facilitated. All sign-in sheets were provided for review. There were no gaps in services.
Five cohorts of IOC were facilitated during the review period. The first cohort began July 21, 2022
and ended on November 3, 3022. Two youth began the cohort, and two youth successfully
completed the cohort. Groups were held consistently for one hour two times a week until mid-
September. Between September 26, 2022 to October 21, 2022 groups were held four days a
week for one hour. After October 21st the groups went back to two days a week for one hour.
The second cohort began on October 3, 2022 and ended on January 10, 2023. The group began
with one youth and two youth successfully completed the cohort. One youth was admitted to the
program immediately after the curriculum began. One individual session was completed,
according to a review of the sign-in sheets. The group began on October 10, 2022. Sign-in sheets
were not completed correctly for this cohort. There were dates and times missing from various
forms. This reviewer spoke with the clinical director about the issues with the sign-in sheets. The
clinical director addressed the issues with the staff and corrections were made.
The third cohort began on February 13, 2023 and ended on June 1, 2023. Three youth began the
cohort, and two youth successfully completed the cohort.
The fourth cohort began on June 20, 2023 with two youths. While in Chapter 1, three more youth
were enrolled on June 28, 2023, as allowed by the Department. The cohort ended on September
14, 2023 with four youth.
The fifth cohort was started September 19, 2023 with one youth. Two youth were added at the
start of Chapter 2, as allowed by the Department. The cohort was ongoing as of the end of the
review period with all three youth. Groups were held consistently on Tuesdays and Thursdays.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program has a process in place to conduct monthly internal fidelity monitoring. The trained
Impact of Crime (IOC) facilitators monitor each other, or the transition specialist conducts internal
fidelity monitoring observations monthly.
All internal fidelity forms were provided for review. There were no missing internal fidelity forms
missing for the review period.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process in place to address corrective actions identified during monthly
internal fidelity monitoring. The clinical director addresses any significant issues with the
facilitators for Impact of Crime (IOC). A review of the internal fidelity forms confirmed no
significant issues or corrective actions were noted nor taken.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The performance evaluations include a specific section to document what curricula the employee
facilitates and how the employee is doing with facilitation, including Impact of Crime. All eligible
faciltiators received an annual evaluation included feedback on their skill in delivering the service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 16 weeks for this type of service. Of the 6 youth in the
sample, 83% (5 of 6) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates a target of 24 contact hours for this type of service. Of the 6 youth in the
sample, 83% (5 of 6) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 8 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 32%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 67%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 0 youth
Moderate-High = 3 youth
High = 1 youth
Total Youth in Sample = 6 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 17%
High Score: 3 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 23%
Table 2
High = 1 youth
Total Youth in Sample = 6 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 67% (4 of 6) youth scored Moderate to High-Risk to reoffend, for a score of
5 points.
Of the SPEP sample, 17% (1 of 6) youth scored High-Risk to reoffend, for a score of 3 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 8 10 80%
Risk Level of Youth Served 8 25 32%
Totals 64 85 75%
This SPEP report evaluates Impact of Crime, an intervention delivered at Twin Oaks Academy II.
6 youth completed the program during the review period. Of those youth, 6 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 6 total youth sampled, only
5 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 5
received at least the recommended weeks of service. Youth in the sample completed between 16
and 20 weeks of service, with an average of 17 weeks.
The program earned 8 points for Amount of Service: Contact Hours. Of the 6 total youth sampled,
only 5 included dosage in the EBS Module. Of those youth with correct dosage, 5 received at
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
least the recommended hours of service. Youth in the sample completed between 31 and 32
hours of service, with an average of 32 hours.
The program was awarded 8 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 95%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Twin Oaks Academy II can maintain their SPEP Quality of Service Delivery score by continuing
the practices in place at the time of this review.
Twin Oaks Academy II can optimize their SPEP Amount of Service score by ensuring dosage for
all youth is recorded accurately in EBS and by ensuring youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

source: DJJ

report_type: SPEP

file_name: FY2023-24 TOVA IOC.pdf

program_name_raw: Twin Oaks Academy II

program_status_note: Open


summary: SPEP - FY23-24

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Twin Oaks Academy II
Twin Oaks Juvenile Development, Inc
(Contract Provider)
29841 Liberty Wilderness Road
Sumatra, Florida 32335
Primary Service: LifeSkills Training
SPEP Review Period: November 1, 2022 - October 31, 2023
SPEP Review Conducted: December 5, 2023
Florida Department of Juvenile Justice
Report Date: 6/27/2024
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is LifeSkills Training.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated during quarterly and annual reviews.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 6 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program uses LifeSkills Training (LST) as a primary service for all youth. There is a training
protocol delivered by the developers of the curriculum. Therapy staff are trained to conduct LST
groups. At the time of the annual review, there were two trained staff members, a clinical
coordinator and the assistant facility administrator. The assistant facility administrator was on
military leave much of the review period and did not facilitate group or conduct monthly internal
fidelity monitoring. The sole facilitator of LST groups was the clinical coordinator.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program uses the Botvin LifeSkills (LST) manual. The facilitators each have their own manual
and workbooks for the youth. The manual includes a script, lesson plans, and facilitator
guidelines.
LifeSkills Training is scheduled for Mondays and is typically held at 4:30 p.m. The program
maintains sign-in sheets and fidelity monitoring observations in a binder kept in the clinical
director's office. All youth are eligible to receive the LST curriculum. The groups are closed groups
and youth may not be added to the cohort after completion of the first lesson.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of LifeSkills Training was observed on December 14, 2023. Present for group were
seven youth, one facilitator, and the clinical director. The youth were working on Lesson 10,
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Dealing with Anger. The youth worked through the techniques and strategies to deal with anger in
a healthy way. The facilitator was knowledgeable of the material and was able to successfully
encourage the youth to remain engaged in the conversation.
Basic Score: 1 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The program's policy in the event of facilitator turnover for LifeSkills Training (LST) is for another
trained facilitator to continue with the group. The LST groups are not co-facilitated.
The first cohort began on December 5, 2022. Two youth began the group, and three youth were in
the group at the time of the review. There were three individual sessions completed with the youth
added to the cohort. This youth joined the group on December 30, 2022, lesson four.
The second cohort began on February 22, 2023 with three youth. There was a gap in service.
There was only one group conducted in the months of February and March 28, 2023, in which
there were 6 youth in attendance. In May, there were two groups per week during the first two
weeks of the. According to the clinical director, there has been one youth in attendance since May
10, 2022. The other youth have since been released.
A third cohort began on September 27, 2023 and was ongoing as of October 30, 2023. The group
began with six youth and all six remained at the end of the review period. This cohort has been
held in accordance with the approved group sessions exhibit.
The program can earn 2 points if there are no unnecessary gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 0 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program's procedure for conducting internal fidelity monitoring is for the trained staff to
monitor each other for Life Skills Training (LST) groups.
An informal interview with the clinical director revealed one facilitator has been on military leave at
various times during this review period. Internal fidelity monitoring was not conducted February-
October 2023. The secondary trained facilitator is back from military leave and is scheduled to
resume monthly internal fidelity monitoring during December 2023.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 0 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program has a process to address corrective actions identified during monthly fidelity
monitoring of LifeSkills Training (LST). The clinical director addresses any significant issues or
corrective actions with the facilitators for LST. Internal fidelity monitoring was not conducted
February-October 2023.
Internal fidelity monitoring was not conducted for a majority of the review period; therefore, this
indicator is not eligible to receive any points. Monthly internal fidelity monitoring must occur to
score points in this section.
The program can earn 1 points if any corrective action needed is identified, applied, and
demonstrated.
At the time of the review, the program did not meet criteria to earn a score of 1.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
The program provided annual performance evaluations for the facilitators for LifeSkills Training
(LST). The evaluations include a section to document the curricula the staff are trained in.
At the time of the annual review the primary trained facilitator had an evaluation completed.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 15 weeks for this type of service. Of the 6 youth in the
sample, 83% (5 of 6) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 10 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 100%
Research indicates a target of 15 contact hours for this type of service. Of the 6 youth in the
sample, 100% (6 of 6) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 8 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 32%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 67%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 0 youth
Moderate-High = 3 youth
High = 1 youth
Total Youth in Sample = 6 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 17%
High Score: 3 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 23%
Table 2
High = 1 youth
Total Youth in Sample = 6 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 67% (4 of 6) youth scored Moderate to High-Risk to reoffend, for a score of
5 points.
Of the SPEP sample, 17% (1 of 6) youth scored High-Risk to reoffend, for a score of 3 points.
Note: The latest Community Assessment Tool (CAT) prior to the placement date was used in the
derivation of the risk level score. This CAT provides the best indication of the risk to re-offend
level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 10 10 100%
Risk Level of Youth Served 8 25 32%
Totals 56 85 66%
This SPEP report evaluates LifeSkills Training, an intervention delivered at Twin Oaks Academy
II.
6 youth completed the program during the review period. Of those youth, 6 were included in the
sample.
The program scored Medium for Quality of Service Delivery.
The targeted number of weeks for this specific intervention is 15 weeks; the targeted number of
hours for this specific intervention is 15 hours. After additional review and discussion with the
Peabody Research Institute at Vanderbilt University, the Department determined it was within
SPEP standards to deviate from the standard dosage targets for Group 3 services (16 weeks; 24
hours).
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program earned 8 points for Amount of Service: Duration. Of the 6 total youth sampled, all 6
included dosage with end dates in the EBS Module. Of those youth with correct dosage, 5
received at least the recommended weeks of service. Youth in the sample completed between 10
and 33 weeks of service, with an average of 22 weeks.
The program earned 10 points for Amount of Service: Contact Hours. Of the 6 total youth
sampled, all 6 included dosage in the EBS Module. Of those youth with correct dosage, 6
received at least the recommended hours of service. Youth in the sample completed between 15
and 15 hours of service, with an average of 15 hours.
The program was awarded 8 available points for Risk Level of Youth Served. This is calculated
using data from the Community Assessment Tool (CAT). This score reflects youths' most recent
CAT score prior to placement at the program. The program itself has no control over youths' CAT
risk level because the scored assessment was administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 86%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
Twin Oaks Academy II can optimize their SPEP Quality of Service Delivery score by ensuring
there are no unnecessary gaps in service, fidelity monitoring is completed every month for each
facilitator, and any corrective action needed is identified, applied, and demonstrated.
Twin Oaks Academy II can optimize their SPEP Amount of Service score by ensuring dosage for
all youth is recorded accurately in EBS and by ensuring youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

file_name: FY2023-24 TOVA LST.pdf

service_type: LifeSkills Training


facility_name: Home Builders Institute, Inc.

summary: QI Residential - FY23-24

raw_content:
STATE OF FLORIDA
DEPARTMENT OF JUVENILE JUSTICE
BUREAU OF MONITORING AND
QUALITY IMPROVEMENT
PROGRAM REPORT FOR
Home Builders Institute, Inc.
(Contract Provider)
1201 15th Street Northwest, 6th Floor
Washington, D.C. 20005
Review Date(s): FY 2023-2024
PROMOTING CONTINUOUS IMPROVEMENT AND ACCOUNTABILITY
IN JUVENILE JUSTICE PROGRAMS AND SERVICES
Rating Definitions
Ratings were assigned to each indicator by the review team using the following definitions:
No exceptions to the requirements of the indicator; or limited,
Satisfactory unintentional, and/or non-systemic exceptions that do not result in
Compliance reduced or substandard service delivery; or systemic exceptions
with corrective action already applied and demonstrated.
Systemic exceptions to the requirements of the indicator;
Limited exceptions to the requirements of the indicator that result in the
Compliance interruption of service delivery; and/or typically require oversight
by management to address the issues systemically.
The absence of a component(s) essential to the requirements of
Failed
the indicator that typically requires immediate follow-up and
Compliance
response to remediate the issue and ensure service delivery.
Review Team
The Bureau of Monitoring and Quality Improvement wishes to thank the following supplemental
review team members for their participation gathering information for the annual statewide rollup
report, and for promoting continuous improvement and accountability in juvenile justice
programs and services in Florida:
Jennifer Tummino, Office of Accountability and Program Support, Regional Monitor
LeAnn Gruentzel, Office of Accountability and Program Support, Regional Monitor
Ken Phillips, Office Accountability and Program Support, Regional Monitor
Patrick McKinstry, Office of Accountability and Program Support, Regional Monitor
Kelsea Holmes, Office of Accountability and Program Support, Contract Manager
Mahogany Brown, Office of Accountability and Program Support, Contract Manager
Paula Friedrich, Office of Accountability and Program Support, Regional Monitor
Staff and youth records were reviewed for the following service location providers:
• Cypress Creek Juvenile Offender Correctional Center/Treatment Center, Circuit 5
• Everglades Youth Academy, Circuit 19
• Home Builder’s Institute, Inc. (HBI) Corporate, Statewide
• Walton Academy for Growth and Change, Circuit 1
Contract amendment seventeen, which was executed June 17, 2022, removed service locations
at Okaloosa Youth Academy/Crestview Sex Offender Program, Oak Grove Academy, Deep
Creek Academy, and St. Johns Youth Academy.
Florida Department of Juvenile Justice Statewide Home Builders Institute, Inc. Annual Compliance Report
Office of Program Accountability Page 3 of 12 (Revised September 2016)
Methodology used for Monitoring Statewide Contracts
The statewide report serves as an overview of services provided by Home Builder’s Institute
(HBI) throughout the State of Florida. Regions in Florida are divided into north, central, and
south. Each region has assigned regional monitors who are responsible for conducting
supplemental and annual compliance monitoring based on the Monitoring and Prioritization Tool
which is updated quarterly by the Department. The regional monitors enter summaries within
the Department’s Juvenile Justice Information System (JJIS) Program Monitoring and
Management (PMM) system. The monitoring summaries are comprised into the annual
compliance statewide report.
Overview
This statewide report addresses services provided by Home Builders Institute, Inc. (HBI) to
youth located in residential programs throughout the State of Florida during fiscal year 2023-
2024. Contract 10186 between the Department and HBI establishes services which began April
1, 2015, and end on December 31, 2025. Contract 10186 between the Department and HBI
establishes services which began April 1, 2015, and end on June 30, 2024. The HBI program
provides vocational training for youth who are committed to residential programs throughout
Florida and identified as in need of vocational services. The mission of HBI is to change lives by
educating, inspiring, and preparing individuals for careers in the building industry. HBI ensures
all participants graduate from the program with the necessary skills, experience, and credentials
to be successful in the workforce. HBI’s award-winning Pre-Apprenticeship Certificate Training
(PACT) curriculum integrates performance-based learning in the building trades with vocational
and academic skills training and includes life skills, career development, and on-the-job training.
The PACT curriculum aligns closely with Science, Technology, Engineering and Math
(STEM) education, is based on National Association of Home Builders’ Green Building
Standard™ and National Skills Standards, and is one of only three pre-apprenticeship curricula
recognized by the Department of Labor. HBI’s PACT program includes training and certification,
mentorship and customer support, student workbooks, certification assessments, and access to
HBI’s online learning management system (CTEtechWorks®). CTEtechWorks® is a web-based
platform of instructional tools and content to supplement hands-on training. CTEtechWorks®
makes career technical education more accessible to youth who require flexibility and diversity
in learning.
General Description of Services
Home Builders Institute (HBI) shall design, develop, implement, and operate vocational training
overlay services to support residential programs for youth who have been committed to the
Department and assessed utilizing the Department’s risk/needs assessment, the Department’s
Residential Assessment of Youth (RAY), and pre-disposition comprehensive evaluation.
Services are provided in accordance with the contract between HBI and the Department, Florida
Statutes, Florida Administrative Code, and Department policy. Enrolled youth have the
opportunity to receive occupational skills training, which may include exposure to brick or
cement masonry, carpentry, electrical wiring, plumbing, painting, landscaping, and facility
maintenance. Services provided by HBI staff include care and custody, screening, assessment,
counseling, case management, education, vocational services, and job training. Post-residential
job placement and job assistance are provided for youth as part of the transition and aftercare
program components. The program staff work with youth to develop appropriate goals and
objectives which are outlined in each youth’s Individualized Performance Plan. HBI staff also
Florida Department of Juvenile Justice Statewide Home Builders Institute, Inc. Annual Compliance Report
Office of Program Accountability Page 4 of 12 (Revised September 2016)
participate in scheduled residential program led treatment team meetings to discuss youth
progress and goals.
Youth Served
Youth found eligible to receive services and training through the Home Builders Institute (HBI)
vocational programs are required to be between fifteen and nineteen years of age. Both male
and female youth are eligible to participate. Served youth are committed to the Department after
being assessed and classified as a risk to public safety, and in need of vocational training
overlay services. Each residential program is responsible for referring youth to the HBI
vocational program. Participants must function at a minimum of a fourth grade reading and math
level, as determined by the Test for Adult Basic Education (TABE), Bergan’s, Wide Range
Achievement Test (WRAT), or similar testing instrument. Participants are selected through
interviews, risk assessments, and are provided a two-day orientation session.
Screening and Evaluation
The initial fourteen days of the youth’s participation are considered the Situational Assessment
Phase. During this time, the youth’s interest in the construction trade area is affirmed, and
physical limitations and basic skill levels are determined. Applicants are assessed on their
motivation and interests in the building industry. Specific core assessments include vocational
service offerings, goals of the program, guidelines, safety requirements, and processes for
complaints. Other Home Builders Institute (HBI) screenings and assessments include the HBI
Pre-Apprenticeship Certificate Training (PACT) Core Diagnostic Assessment, HBI PACT
Communications Diagnostic, HBI PACT Math Diagnostic, and Realistic, Investigative, Artistic,
Social, Enterprising, and Conventional (RIASEC) Inventory.
Specified Offerings
Home Builders Institute (HBI) shall follow the guidelines of the Florida Department of Education
and Division of Workforce Development. Each HBI site enters into an interagency agreement
with the local school board which addresses educational requirements. The HBI Pre-
Apprenticeship Certificate Training (PACT) for youth shall provide an array of vocational
education and certification curricula designed to meet a wide range of youth interests and
aptitudes.
HBI schedules hands-on instruction approximately seventy-five percent of the time with the
remaining twenty-five percent devoted to related education, soft skills, leadership development,
group mentoring, risk behavior, prevention/intervention, and career development. Vocational
skills training is offered on an open entry basis with an expected duration of three to six months
equating to 520 to 1,040 hours depending on the rate of skills achievement. HBI provides
community restitution as a component of the work-based learning approach. Weekday and
weekend community service projects provide youth with an opportunity to apply their newly
developed skills into real-work situations through Habitat for Humanity, Rebuilding Together,
and other construction and trades projects in the community.
All enrolled youth participate in the development of an individualized plan using the process
required by the hosting residential provider. The plan will be collaboratively developed by the
participant, project staff, and integrated with performance contracts and self-sufficiency goals,
as well as other individualized treatment plans formulated by referring agencies. To the greatest
Florida Department of Juvenile Justice Statewide Home Builders Institute, Inc. Annual Compliance Report
Office of Program Accountability Page 5 of 12 (Revised September 2016)
extent possible the development of social skills will be linked with employability skills
development, community re-entry, family reintegration, and self-sufficiency goals. Additionally,
HBI provides an integrated program life skills training. Life skills adjustment goals are
documented in the individualized/self-sufficiency plan.
Transition / Discharge
Prior to discharge, youth develop a vocational transition plan which includes objectives
regarding the continuation of the youth's technical education and vocational training as they
transition successfully back to the community. The plan is developed with the youth, their family,
and the residential program. The Home Builders Institute (HBI) program supplies appropriate
hand tools for HBI Pre-Apprenticeship Certificate Training (PACT) graduates who have been
placed in industry-related employment, when applicable. The HBI staff coordinate with regional
workforce board services to enhance vocational training certification of youth while in the
program or as part of transition to the youth’s home community. The HBI staff are responsible
for providing and maintaining a regular schedule of contact with the youth and parent/guardian
during residential placement, and for a minimum of six months following the release of the youth
from the commitment program.
Staffing / Personnel
Home Builders Institute (HBI) must ensure the constant presence of sufficient qualified staff to
provide program services to include proper supervision of youth during hours of program
operation or service, transportation, and orientation of youth at all times. Positions outlined
include the operations manager, regional program manager, instructor, regional placement
coordinator, and regional career development coordinator. HBI is responsible for continually
maintaining all licenses, protocols, and certifications which are required by the Department or
another local, state, or federal agency, for the services to be performed or for the position held.
All staff shall renew licenses or certifications pursuant to applicable laws or rules. Staff are
required to adhere to all Occupational Safety and Health Administration (OSHA) requirements,
including training and certifications. HBI staff are required to comply with the Department’s
statewide procedures regarding background screening. Staff employed with HBI work in
collaboration with the applicable residential commitment program staff to ensure required
supervision ratios are maintained. HBI must employ sufficient relief staff to ensure the
operations of the program continue in the event a vacancy may occur in staff positions. No
annual administrative monitoring was completed; therefore, it is unknown if there were any
vacancies for the annual compliance review period.
Staff Training
Home Builders Institute (HBI) staff are trained in accordance with Florida Administrative Code
and contract. Completed trainings are documented in the Department’s Learning Management
System (SkillPro). HBI staff have access to online training opportunities through SkillPro.
Additionally, HBI staff have access to online training opportunities through CTEtechWorks®,
which is an online HBI intranet training site. Pre-service training is aligned with the Department’s
residential services training requirements. Within ninety days of hire, staff providing direct
services are responsible for completing the Department’s two-day Motivational Interviewing and
Stages of Change training. Prior to having contact with youth, newly hired staff must complete
Right Interactions (RI), cardiopulmonary resuscitation (CPR), first-aid, automated external
Florida Department of Juvenile Justice Statewide Home Builders Institute, Inc. Annual Compliance Report
Office of Program Accountability Page 6 of 12 (Revised September 2016)
defibrillator (AED), professionalism and ethics, suicide prevention, emergency procedures, and
Prison Rape Elimination Act (PREA) trainings. Within the first 180-days of employment, newly
hired staff must complete program philosophy, adolescent behavior, confidentiality, infection
control, gangs, safety and security, behavior management, Department of Juvenile Justice –
The Organization, mental health and substance abuse services, sexual harassment,
communication skills, and applicable on-the-job training. Annual in-service training includes
twenty-four mandatory hours of training. Training includes a RI update, applicable CPR, AED,
first-aid training, and professionalism and ethics. Supervisory in-service training includes annual
training in fiscal, leadership, personal accountability, management, and communication skills.
Summary of Services Provided
Annual Administrative Monitoring
A quarterly review for administrative compliance review was completed for the 2023-2024 fiscal
year by the contract manager. No deficiencies were identified. A summary of the findings
included the following:
All required documentation for the contract was reviewed and no issues were noted. The other
required quarterly reviews and an annual review could not be reviewed for this report as they
were not located in the contract file.
Cypress Creek Juvenile Offender Correctional Center/Treatment Center, Circuit 5
Annual and supplemental reviews were completed for the 2023-2024 fiscal year by the regional
monitors with the Bureau of Monitoring and Quality Improvement. Monitoring of the Home
Builder’s Institute (HBI) services at Cypress Creek Juvenile Offender Correctional
Center/Treatment Center included three supplemental and one annual review. There were no
deficiencies identified. A summary of the findings included the following:
The provider provides Occupational Safety and Health Administration (OHSA), HBI Pre-
Apprenticeship Certificate Training (PACT)-Core, HBI PACT Facilities Maintenance for pre-
vocational and vocational services as listed in the contract, to include carpentry, electrical, brick
masonry, landscaping, and painting. All youth who are admitted and complete the program
receive all services offered. The program provides case management services including
developing appropriate goals and objectives for inclusion in each youth’s Individualized
Performance Plan/treatment plan in concert with treatment, education, and medical needs. HBI
staff also participate in bi-weekly treatment team meetings with the program staff. The
program’s schedule runs from Monday through Friday with Class A starting at 7:30 a.m. and
ending at 10:30 a.m., and Class B starting at 10:35 a.m. and ending at 11:30 a.m. At the time of
the program visit, the regional program manager, regional career development coordinator, and
instructor were providing the services at the program.
The provider has a full complement, with three staff. A review of the provider’s staff roster found
one staff was applicable for an initial background screening during the annual compliance
review period. The one applicable staff had a completed and eligible background screening, as
required, prior to contact with youth and confidential youth records. There was one staff
applicable for a five-year rescreening, which was confirmed in the Clearinghouse. There was
one staff applicable for pre-service training. A review of the staff’s record confirmed all pre-
service training was completed, as required, which were documented in the Department’s
Florida Department of Juvenile Justice Statewide Home Builders Institute, Inc. Annual Compliance Report
Office of Program Accountability Page 7 of 12 (Revised September 2016)
Learning Management System (SkillPro). All three staff possess adequate education and
training to perform the duties for which they are assigned, and they meet all applicable
licensing/certification requirements. The staff to youth ratio of one-to-eight is maintained for on-
site activities; the provider does not conduct any off-site activities. The provider staff work
closely with the program staff regarding training, supervision, and tool control. The program staff
provide youth supervision and continuous line of sight during provider activities, while provider
staff conduct the youth training. The program staff also conduct regular youth counts and
searches of youth, not the provider.
The provider works closely with the program’s leadership to ensure proper responses to
emergencies and are on the active emergency call list for Continuity of Operations Plan (COOP)
related activities.
The provider has a tool inventory which includes tools being signed out and in when activities
are completed, as well as all tools being counted four times a day, Monday through Friday. The
tool inventory was reviewed which indicated all tools were present. Observations of the tool
storage area indicated the tools were present. The tools are maintained in the provider work
area in a locked room, which includes areas for tools designated by shadow boards.
A review of the five closed youth records confirmed during the first fourteen days of
participation, which is the situational assessment phase, youth are assessed on their motivation
and interests to include vocational service offerings, goals of the program, guidelines, safety
requirements, process for complaints, HBI PACT Core Diagnostic Assessment, HBI PACT
Communications Diagnostic, HBI PACT Math Diagnostic and the Realistic, Investigative, Artistic,
Social, Enterprising, and Conventional (RIASEC) Inventory.
The program provides hands-on instructions for about seventy-five percent of the time with the
remaining twenty-five percent of the time devoted to related education, soft skills, leadership
development, group mentoring, risk behavior, prevention/intervention, and career development.
The classroom component provides instruction in practical trade skills, care and use of tools,
safety, work habits, fundamental mathematics, and communication skills. Each youth also
receives occupational skills training including exposure to brick masonry, carpentry, electricity,
plumbing, painting, landscaping, and general construction. Upon completion of the
apprenticeship program and release from the facility, youth are placed in employment,
vocational, technical, or school based training, apprenticeship, or other career progressive
tracks, as appropriate.
An individualized plan is incorporated into the performance plan for each youth and for those
youth who are placed in industry-related employment and require supplies, during the follow-up
period, they will be furnished a customized tool kit by the provider. The provider includes social
skills enhancement activities in all components of the HBI PACT curriculum. These services
include interpersonal relations, coping with authority, cooperation, decision-making skills, and
problem solving.
The provider provides the program treatment team staff with progress made on vocational goals
and recommendations for goals, objectives, and activities, which continue during transition, and
when the youth returns to their community. At a minimum, the plans included objectives
regarding the continuation of the youth’s technical education and vocational training as they
transition successfully back to the community. The plans were developed with the youth,
parent/guardian, and treatment team based on the treatment planning process.
Florida Department of Juvenile Justice Statewide Home Builders Institute, Inc. Annual Compliance Report
Office of Program Accountability Page 8 of 12 (Revised September 2016)
A review of five closed records indicated the provider established a regular schedule of contact
with youth and parent/guardian or community groups at least six months after release as part of
the follow-up services. In all five youth records, the provider participated and contributed to the
pre-release planning for each youth.
An observation of the HBI area was conducted on April 1, 2024. A comparison of tools to
inventory log was conducted for five tools on the shadow board, and tools matched. The area is
secured behind a locked fence only accessible to staff. Observations of multiple construction
projects were made, inclusive of a covered porch swing, a ping pong table, two picnic tables,
one of which was disassembled for easy transport, a mailbox for the program is in the early
stages of construction. Youth and staff are required to sign-in/out tools daily and the log is
reviewed by the instructors to ensure accuracy.
Everglades Youth Academy, Circuit 19
Annual and supplemental monitorings were completed for 2023-2024 fiscal year by regional
monitors with the Bureau of Monitoring and Quality Improvement. Monitoring of the Home
Builder’s Institute (HBI) services at Everglades Youth Academy – Maximum Risk and Sex
Offender Program included two supplemental and one annual review. There were no
deficiencies identified. A summary of the findings included the following:
The program had no vacancies during the annual compliance review period in the instructor,
administrative coordinator, or regional career development coordinator positions. There were no
staff applicable for an initial or five-year background rescreening as all program staff are beyond
their first 180 days of hire with the program.
The program requires in-service training to be completed annually to Right Interactions (RI)
Update, cardiopulmonary resuscitation (CPR), first-aid, and professionalism and ethics. Three
staff training records were reviewed. All three reviewed staff completed all required trainings
which were documented in the Department's Learning Management System (SkillPro). The
program maintains pre-service and in-service training plans. The applicable in-service training
plan exceeds contractual the training requirements with a minimum of twenty-four hours
annually to include the facility eight-hour RI update, CPR, AED, and first-aid
recertification/refresher and/or instructor recertification, Prison Rape Elimination Act (PREA),
active shooter preparedness, Child Abuse: Recognition, Prevention and Reporting, civil rights,
emergency procedures, ethics, HIPAA Confidentiality, Human Trafficking, information security
awareness, It’s All About Reporting, OSHA Exposure and Infection Control Policy Review,
Quality Customer Service, Safety training, sexual harassment every two years, suicide
awareness and prevention, and trauma informed care. Additional training on communication,
staff relations, fiscal management, leadership, management, and personal accountability are
required for staff with supervisory responsibilities. All trainings are due for completion by
December 31 each year.
The program is contracted for sixteen youth slots. At the time of the annual compliance review,
the program had a total of thirteen youth participating, with three additional youth
awaiting assessment to fill the remaining three slots. Reviewed documentation indicated there
were three active classes and the youth were working on different projects. Observations and
reviewed documentation validated the program was compliant with the required staff to youth
ratio of one safety and security specialist staff for every five youth. The co-located residential
programs, Everglades Youth Academy and Everglades Youth Academy Sex Offender Program,
Florida Department of Juvenile Justice Statewide Home Builders Institute, Inc. Annual Compliance Report
Office of Program Accountability Page 9 of 12 (Revised September 2016)
are identified as a high-risk and maximum-risk programs; therefore, youth do not participate in
any off-campus activities.
Five active youth records and three closed youth records were reviewed. Reviewed
documentation of five active youth records supported the provider participated in bi-weekly
treatment team meetings with the facility management staff. Each record documented the
completion of a risk assessment, acknowledgment of safety rules, a participation profile, receipt
of a handbook, initial academic assessments, welcome letters, training and credentialing letters,
and follow-up contact documentation. Each reviewed record documented each youth’s progress
and completion of the situational assessment during the first fourteen days of participation,
which assesses each youth on their motivation and interest including the goals of the program,
the vocational service offerings, and elements of the HBI PACT curriculum, which includes
vocational services, self-sufficiency planning, and life skills training.
Walton Academy for Growth and Change, Circuit 1
Annual and supplemental monitorings were completed for the 2023-2024 fiscal year by regional
monitors with the Bureau of Monitoring and Quality Improvement staff. Monitoring of the Home
Builder’s Institute (HBI) services at Walton Academy for Growth and Change included three
supplemental, one verification, and one annual review. There were two deficiencies identified. A
summary of the findings included the following:
One minor deficiency was identified on December 1, 2023, and closed February 13, 2024. The
deficiency was created due to general program cleanliness and maintenance. The issues noted
were grime and stains on drinking fountains, excess sawdust and dust found within the shop
area tables, equipment, eye wash stations, and bathrooms. On February 13, 2024, the first
verification was completed. All program areas were found to be clean and the shop floors
appear to have been swept. The eye wash station, bathrooms, and water fountains were clean.
Shop tools were found to be organized using a shadow board system. The HBI instructor
reported the youth are instructed to stop working and begin cleaning their areas and the shop
approximately fifteen minutes prior to the end of the class period.
One major deficiency was identified during the annual compliance review on June 4, 2024, and
recommended to be closed and continued to be closely monitored on February 6, 2025. The
deficiency was created due to the provider failing to provide regular contacts for a six-month
duration to any of the eight reviewed youth as required. Six contacts were completed timely for
eight youth. Out of the remaining thirty-seven contacts required, twenty-one were not made at
all and sixteen were made late.
The program had no staff vacancies during the annual compliance review period. The list of pre-
service training had been approved by the Department’s regional director which included the
course names, descriptions, objectives, and training hours for any instructor-led training. All staff
possess adequate education and training to perform the duties to which they are assigned and
meet all applicable licensing or certification requirements for their respective disciplines. The
HBI program had sixteen slots available, with nine slots being filled at the time of the annual
compliance review. The appropriate staff-to-youth ratio is maintained for on-site activities. The
residential program is high-risk; therefore, the youth are unable to go off-site for HBI activities.
The HBI staff work closely with the facility staff regrading training, supervision, and tool control.
The youth are in constant sight and sound supervision of staff during program activities. The
staff conduct regular youth counts as outlined by the facility leadership and document counts
and other required documentation according to Department policy. The HBI staff work with
Florida Department of Juvenile Justice Statewide Home Builders Institute, Inc. Annual Compliance Report
Office of Program Accountability Page 10 of 12 (Revised September 2016)
facility leadership to ensure proper response to emergencies and are on the active emergency
call list for COOP related activities and drills. Documentation was reviewed of HBI participating
in a COOP drill in May 2024. Tool inventory is conducted four times daily according to the
perpetual inventory policy.
The HBI program provides carpentry, electrical, plumbing, and landscaping services. All
services are being provided according to the current contract. The program provides case
management services to include developing appropriate goals and objectives for inclusion in the
youth’s Individualized Performance Plan and/or treatment plan. Staff participate in bi-weekly
treatment team meetings with the facility management staff.
The program provides hands-on instructions approximately seventy-five percent of the time with
the remaining twenty-five percent devoted to related education, soft skills, leadership
development, group mentoring, risk behavior, prevention/intervention, and career development.
The classroom component provides instructions in practical trade skills, care and use of tools,
safety, work habits, fundamental mathematics, and communication skills. The youth also
receive occupation skills exposure to carpentry, electricity, plumbing, painting, landscaping, and
general construction. The youth receive restitution as a component of the work-based learning
approach with projects framed in consideration of the impact of crime on the victim, offender,
and the community. When youth complete the pre-apprenticeship program, youth are placed in
employment, vocational, technical, school-based training, apprenticeship, or other career
progressive tracks, as appropriate. The provider demonstrated attempts to access regional
workforce board services or funding to enhance vocational training certification of youth while in
the program or as part of transition to the youth’s home community. The provider ensures job
placement assistance will be provided to all youth completing the program, and in some
instances, to those who leave prior to completion.
An individualized plan is incorporated into the performance plan for each youth. The youth are
provided a supply of appropriate hand tools for HBI PACT graduates who have been placed in
industry-related employment and are supplied at any time during the follow-up period. No youth
were applicable during the annual compliance review period. The provider includes social skills
enhancement activities in all components of the HBI PACT curriculum, which includes
interpersonal relations, coping with authority, cooperation, decision-making, and problem
solving. The youth are provided an integrated program life skills training in conjunction with its
pre-apprenticeship skills and employability skills training. The life skill adjustment goals
documented in the Individualized Performance Plan address topics such as use of community
resources, health and nutrition, transportation, peer and family relationships, parenting skills,
finances, consumerism, budgeting, recreation and leisure time, community integration issues,
and housing.
The provider gives the facility treatment team staff progress made on vocational goals and
recommendations for goals, objectives, and activities which should continue during transition
and when the youth returns to their community. Each reviewed plan included objectives
regarding the continuation of the youth’s technical education and vocational training as they
transition successfully back to the community. The plan is developed with the youth,
parent/guardian, and treatment team based on the treatment planning process.
A review of eight youth records confirmed during the first fourteen days of participation, which is
the situational assessment phase, youth are assessed on their motivation and interests and
interests to include vocational service offerings, goals of the program, guidelines, safety
requirements, process for complaints, HBI PACT Core Diagnostic Assessment, HBI PACT
Florida Department of Juvenile Justice Statewide Home Builders Institute, Inc. Annual Compliance Report
Office of Program Accountability Page 11 of 12 (Revised September 2016)
Communications Diagnostic, HBI PACT Math Diagnostic, and the RIASEC Inventory. Eight
youth records were reviewed for discharge and transition services. The youth and
parent/guardian contacts were not completed timely during the six months after placement
and/or release as part of follow-up services as noted in the major deficiency above. All eight
youth records included documentation the youth was provided a written summary which was
submitted to the management of the residential program outlining the progress of the youth in
the program and pre-release goals. All eight youth records included documentation the provider
participates and contributes to the pre-release planning.
Florida Department of Juvenile Justice Statewide Home Builders Institute, Inc. Annual Compliance Report
Office of Program Accountability Page 12 of 12 (Revised September 2016)

report_type: QI Residential

program_name_raw: Home Builders Institute, Inc.

program_status_note: HBI