summary: SPEP - FY18-19

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Dove Academy
Twin Oaks Juvenile Development, Inc.
(Contract Provider)
5270 Ezell Road
Graceville, Florida 32440
Primary Service: Seven Challenges
SPEP Review Date(s): 12/11/2018-12/14/18
Florida Department of Juvenile Justice
Report Date(s): 12/11/18
Introduction
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Seven Challenges.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program provided documentation of therapeutic staff training for the Seven Challenges
curriculum. There are currently four therapeutic staff trained. Three staff consistently facilitate the
groups. The Designated Mental Health Clinical Authority (DMHCA) was trained in April 2013. The
DMHCA was then trained as a trainer of the curriculum in July 2013. One therapist was trained
December 2016 and the other therapist was trained May 2018. When the program has a
therapist leave or out on extended leave there is a contracted therapist comes and conducts
groups as needed.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program provided three manuals for The Seven Challenges curriculum. The facilitator's
manual has 13 chapters, which covers implementation, each of the seven challenges, counseling
approaches, gender issues, supervision, games for groups, two appendices, four extra
worksheets, group note examples, and treatment goal examples. There is a section in the manual
for group session topics. The topics are able to be copied. The youth read through the topic and
they discuss how the topic applies to their life. The curriculum is open for the therapist to be
creative in how they want to facilitate the group. The Seven Challenges is designed to be an open
discussion for all youth to participate and apply the lesson to their lives.
The second manual provided was an activity book. The activity book provides various counseling
activities for each challenge. Each activity provides materials, instruction, script, how to connect
the lesson, and worksheets for each lesson.
The last manual was the journaling book the youth use during journaling sessions. Seven
Challenges is held as an open group and it is run consistently at the program. As an open group
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
the youth complete the curriculum at their own pace. Seven Challenges is held on Tuesdays and
Thursdays at 10:30am-11:30am and 11:30am-12:30pm.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
The Seven Challenges group was observed on Thursday, December 13, 2018 from 10:30am to
11:30am. There were seven youth in group. Seven Challenges is facilitated as a open group,
meaning the youth begin and end the group at their own pace. The youth have seven sections of
the curriculum to work through at their own pace with their therapist and the facilitator. The youth
completed the sign in sheet and checked in for the day. The youth had a scenario to read titled
"Cutting Back". The youth had an opportunity to read the scenario and discuss the topic. After
reading, the youth were asked probing questions and how it applied to their lives in the past,
currently, and in the future. The youth were open to share experiences and the facilitator ensured
that all of the youth participated in the discussion. The group atmosphere was open and
comfortable. The youth were calm and relaxed. The youth were supportive of one another and
helpful towards others who were struggling. The group lasted one hour and they were able to
work through their questions and self-assessments. There were no behavioral issues during the
group. The staff member who sat in group was appropriate and did not cause a disruption.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There was one therapist who left during the review period. However, the program has a therapist
who is a contracted provider. The therapist is available to come to the program and complete
groups, as needed. There were no gaps in service during the review period.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The Designated Mental Health Clinical Authority (DMHCA) conducts internal fidelity monitoring for
Seven Challenges. The months of internal fidelity monitoring that were missing were January
2018 and October 2018. This reviewer spoke with the DMHCA regarding the missing months.
October was not completed due to the DMHCA being out the majority of the month after
Hurricane Michael. January 2018 was not completed due to the DMHCA being out due to a death
in the family. No other therapist is able to complete the internal fidelity monitoring for this
curriculum. The internal fidelity monitoring forms are kept in a binder labeled "Fidelity
Documents" in the DMHCA's office.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program's corrective action process is for the therapists to meet with the Designated Mental
Health Clinical Authority (DMHCA) on a weekly basis. If there are corrective action issues, it is
indicated on the internal fidelity monitoring form and addressed by the DMHCA with the therapist.
There were no corrective action issues indicated on the internal fidelity monitoring forms
reviewed.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Two of the four therapists have received yearly performance evaluations. One therapist left the
program prior to completing a yearly and the other is a contracted provider. The yearly
performance evaluations were provided for review. There is a section on the yearly evaluations
which address therapeutic skills and details facilitation skills.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 24 weeks for this type of service. Of the 18 youth in the
sample, 0% (0 of 18) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 40 contact hours for this type of service. Of the 18 youth in the
sample, 78% (14 of 18) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 100%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 0 youth
Moderate-High = 7 youth
High = 11 youth
Total Youth in Sample = 18 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 61%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 11 youth
Total Youth in Sample = 18 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 100% (18 of 18) youth scored Moderate to High-Risk to reoffend, for a
score of 12 points.
Of the SPEP sample, 61% (11 of 18) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 25 25 100%
Totals 81 95 85%
This SPEP report evaluates Seven Challenges, an intervention delivered at DOVE Academy.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 18 total youth sampled, 0
included dosage with end dates in the EBS Module. With no JJIS data to review, the SPEP score
was reduced. The program must enter dosage data into JJIS in order to receive any score in this
area.
The program earned 6 points for Amount of Service: Contact Hours. Of the 18 total youth
sampled, all 18 included dosage in the EBS Module. Of those youth with correct dosage, 14
received at least the recommended hours of service. Youth in the sample completed between 2
and 64 hours of service, with an average of 43 hours.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
DOVE Academy can optimize their SPEP Quality of Service Delivery score by ensuring fidelity
monitoring is completed every month for each facilitator.
DOVE Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
The failure to include end dates in the EBS Module is a large factor in the reduced Program
Optimization Percentage for this program. It is imperative end dates are entered for all youth.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9


state: FL

facility_name: DOVE Academy

summary: SPEP - FY18-19

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Dove Academy
Twin Oak Juvenile Development, Inc.
(Contract Provider)
5270 Ezell Road
Graceville, Florida 32440
Primary Service: Impact of Crime
SPEP Review Date(s): 12/11/18-12/14/18
Florida Department of Juvenile Justice
Report Date(s): 12/20/18
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime.
The program was awarded 15 points because the Primary Service is identified as a Group 3
Service. The specific Sub-Component Service Type identified is Social Skills Training. The
Primary Service was identified as this type of service as it focuses on developing social skills
required for an individual to interact in a positive way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 10 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 50%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 7 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
The program's process for Impact of Crime (IOC) facilitator training is to have all case
management staff trained in curriculum. There are currently five staff members formally trained in
the IOC curriculum. Three staff members are case management, one staff member is the
transition specialist, and one is the program director.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program maintains the current Impact of Crime (IOC) facilitator's manual. There are four
sections to the IOC manual. The first section is the facilitator’s guide, which includes an
introduction and implementation section. The first section also includes how the chapters and
lessons are structured and how they should be delivered. There are two parts of the first section
that goes in depth and discusses the importance of victims and considerations when facilitating
IOC to girls. The second section has all seven chapters divided and numbered. Within each
chapter contains the IOC script, lessons, and activities for the youth. Every chapter begins with
objectives, words to know, and definitions. The third section is the appendix. This contains extra
worksheets, list for videos, behavior contracts, and other helpful worksheets to use during group.
The final section is the student guide. This is the specific workbook for the youth. These can be
copied and provided to the youth during group as homework and the worksheets they use during
group.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Impact of Crime (IOC) group was observed on Thursday, December 20, 2018. There are 10 youth
in this particular group. This group started on Tuesday, December 18, 2018. The group reviewed
the rules that were established during the first meeting and were hanging in the group room. The
youth were sitting at various tables in the room. The lesson was review of chapter terms,
introducing stakeholders, discussing victim rights, and a introduction to victim impact.
The youth were rather rowdy during the group; however the facilitator was able to bring them back
for most of the group. The objectives were completed for the day's group lesson.
This reviewer met with the facilitator after the group to discuss suggestions made to help the
group run smoothly. This is the facilitator's first time running a group on her own in a few years.
Overall, the group went well and the facilitator was open to suggestions.
Basic Score: 0 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There has been some faciltiator turnover for this curriculum during the review period. A case
manager left the program in March 2018. Two more case managers were hired since that time
and trained in the Impact of Crime curriculum. Due to the low number of case management staff
during the review period there were gaps in service.
The first gap was between June 7th and August 23rd. The second gap in service occurred
between October 22nd and December 12th. The remaining case management staff and program
director did not begin a group during these gaps. A group was started May 22nd and was ended
early due to the facilitator being out so much; the other trained case manager was unable to take
over the group at that time. Those youth did not receive the IOC curriculum while at the program.
The gaps in service were discussed with the program director. The program's goal is to have all
case management staff trained to step in and cover a group when needed.
The program can earn 2 points if there are no unnecessary gaps in service.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program maintains internal fidelity monitoring records along with the attendance sheets for
the Impact of Crime (IOC) curriculum. These documents are located in a yellow folder in the
program director's office. The program director currently conducts the internal fidelity monitoring
for IOC groups. There were four months of internal fidelity monitoring documents provided for
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
review. Eight months of internal fidelity monitoring groups were not provided or completed during
the review period.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program provided four internal fidelity monitoring forms for review. There were no corrective
actions indicated on the forms.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Two of three case managers have received their yearly performance evaluations. One case
manager has not been here a year and has not received an evaluation. All performance
evaluations have a section for group facilitation. The performance evaluations for case
management were provided for review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 16 weeks for this type of service. Of the 69 youth in the
sample, 1% (1 of 69) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates a target of 24 contact hours for this type of service. Of the 69 youth in the
sample, 28% (19 of 69) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 23 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 92%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 94%
Moderate to High Score: 10 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 83%
Table 1
Moderate = 7 youth
Moderate-High = 24 youth
High = 34 youth
Total Youth in Sample = 69 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 49%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 34 youth
Total Youth in Sample = 69 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 94% (65 of 69) youth scored Moderate to High-Risk to reoffend, for a score
of 10 points.
Of the SPEP sample, 49% (34 of 69) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 10 20 50%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 2 10 20%
Risk Level of Youth Served 23 25 92%
Totals 55 85 65%
This SPEP report evaluates Impact of Crime, an intervention delivered at DOVE Academy.
70 youth completed the program during the review period. Of those youth, 69 had not previously
received this intervention in another setting.
The program scored Medium for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 69 total youth sampled, only
41 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 1
received at least the recommended weeks of service. Youth in the sample completed between 2
and 17 weeks of service, with an average of 9 weeks.
The program earned 2 points for Amount of Service: Contact Hours. Of the 69 total youth
sampled, only 58 included dosage in the EBS Module. Of those youth with correct dosage, 19
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
received at least the recommended hours of service. Youth in the sample completed between 3
and 32 hours of service, with an average of 17 hours.
The program was awarded 23 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
DOVE Academy can optimize their SPEP Quality of Service Delivery score by ensuring there are
no unnecessary gaps in service, and by ensuring fidelity monitoring is completed every month for
each facilitator.
DOVE Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

source: DJJ

report_type: SPEP

program_name_raw: Dove Academy

program_status_note: Open

service_type: Impact of Crime


summary: SPEP - FY18-19

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Dove Academy
Twin Oaks Juvenile Development, Inc
(Contract Provider)
5270 Ezell Road
Graceville, Florida 32440
Primary Service: Moral Reconation Therapy
SPEP Review Date(s): 12/11/18-12/14/18
Florida Department of Juvenile Justice
Report Date(s): 12/14/18
Introduction
This report provides two types of SPEP scores: a Basic Score, equivalent to the number of
points received, and a Program Optimization Score (POS) that is equivalent to the maximum
number of possible points that could be received based on the SPEP domains under the control
of the program. The Basic Score compares the Primary Service being evaluated to other
intervention services found in the research to be effective, regardless of service type. It is meant
as a reference to the expected overall recidivism reduction when compared to other Primary
Services of any Type.
A Program Optimization Percentage (POP) rate is derived from the Basic Score and Program
Optimization Score. The POP rate is a percentage score that indicates where the rate of
effectiveness of the Primary Service is when compared to its potential effectiveness if optimized
to match the characteristics of similar Primary Services found to be most effective in the
research. The POP rate is likely more meaningful to service providers as it represents how
close the program’s Primary Service is to its potential for that Primary Service Type. For
example, a POP rate of 55% would indicate that the program’s Primary Service is operating at
55% of its potential effectiveness for recidivism reduction that has been found for a similar
Primary Service Type with research evidence of effectiveness.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Moral Reconation Therapy. The program was awarded
30 points because the Primary Service is identified as a Group 5 Service. The specific Sub-
Component Service Type identified is Cognitive Behavioral Therapy. The Primary Service was
identified as this type of service as it is intended to correct faulty cognitions and perceptions and
provides skills individuals can use to monitor thought patterns and correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
Three therapists and the Designated Mental Health Clinical Authority (DMHCA) are formally
trained in Moral Reconation Therapy (MRT). The DMHCA was trained in MRT in May 2010 and
was trained as a trainer in February 2012. The other therapists were trained in December 2016,
May 2010, and July 2018 The program provided all training documentation for the staff trained in
MRT. The program has a protocol of training all therapuetic staff in MRT when they are hired. If
the program is low on therapeutic staff, there is a contracted therapist who will come to the
program to conduct groups as needed.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program provided a workbook titled, "Juvenile MRT: How to Escape Your Prison, A Moral
Reconation Therapy Workbook". The book is written by Dr. Gregory Little and Dr. Kenneth
Robinson. For this curriculum, each youth is provided a workbook they will work in throughout
their time in group. There are 12 steps each youth must work through and successfully complete
in order to successfully complete the curriculum. They are given specific tasks they must
complete before they can move to the next step.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
This group was observed on Wednesday, December 12, 2018 from 10:30am-11:30am. Eight
youth attended the group. One youth was out with Department of Transportation during this
group. Moral Reconation Therapy is run as an open group, so the youth are able to complete this
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
group at their own pace. The group completed the topic of "Will I Make a Difference". Four youth
also presented during this group. The lesson was followed according to the manual. The youth
provided constructive criticism to those who presented on this date. The youth were appropriate
and the facilitator conducted the group appropriately.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There has been facilitator turnover during the review period. More therapuetic staff was hired
during the course of the review period. There were no gaps in service for this group. The program
did evacuate during Hurricane Michael from October 9, 2018 through October 15, 2018; however,
a review of attendance sheets confirmed groups continued to be held during that time. There are
currently four therapists trained at the facility. To avoid any gaps in service the program contracts
with a therapist to come in and cover any groups.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program's protocol is for the Designated Mental Health Clinical Authority (DMHCA) to
conduct all internal fidelity monitoring for Moral Reconation Training (MRT) group. There were
eight internal fidelity forms turned in during the review period. Three months were not completed.
This reviewer spoke with the DMHCA about the three months that are missing. For January 2018,
the DMHCA was out most of the month due to a death in her family. During the month of October
2018 the DMHCA was out the majority of the month after Hurricane Michael. The DMHCA is the
only authorized internal fidelity monitor for MRT groups. There was no explanation for November
2018.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The program's process for corrective actions is the Designated Mental Health Clinical Authority
(DMHCA) meets with the therapists on a weekly basis. If there are any issues or corrective
actions indicated during internal fidelity monitoring, the DMHCA will address it with the therapist
during a one-to-one meeting. There were no corrective actions indicated on the internal fidelity
monitoring forms reviewed.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Three of the four therapists have received their yearly performance evaluations. The fourth
therapist is a contract provider and does not receive a performance evaluation from the program.
All yearly performance evaluations include a section for therapeutic skills, which details the group
facilitation skills.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service - Duration
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates the target duration of 15 weeks for this type of service. Of the 29 youth in the
sample, 0% (0 of 29) reached at least the indicated target duration. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 45 contact hours for this type of service. Of the 29 youth in the
sample, 7% (2 of 29) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 25 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 100%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 97%
Moderate to High Score: 12 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 100%
Table 1
Moderate = 3 youth
Moderate-High = 12 youth
High = 13 youth
Total Youth in Sample = 29 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 45%
High Score: 13 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 100%
Table 2
High = 13 youth
Total Youth in Sample = 29 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 97% (28 of 29) youth scored Moderate to High-Risk to reoffend, for a score
of 12 points.
Of the SPEP sample, 45% (13 of 29) youth scored High-Risk to reoffend, for a score of 13 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 0 10 0%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 25 25 100%
Totals 80 100 80%
This SPEP report evaluates Moral Reconation Therapy, an intervention delivered at DOVE
Academy.
The program scored High for Quality of Service Delivery.
The program earned 0 points for Amount of Service: Duration. Of the 29 total youth sampled,
none included dosage with end dates in the EBS Module. With no JJIS data to review, the SPEP
score was reduced. The program must enter dosage data into JJIS in order to receive any score
in this area.
The program earned 0 points for Amount of Service: Contact Hours. Of the 29 total youth
sampled, all 29 included dosage in the EBS Module. Of those youth with correct dosage, 2
received at least the recommended hours of service. Youth in the sample completed between 4
and 50 hours of service, with an average of 25 hours.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
The program was awarded 25 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
RECOMMENDATION(S):
DOVE Academy can optimize their SPEP Quality of Service Delivery score by ensuring fidelity
monitoring is completed every month for each facilitator.
DOVE Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
The failure to include end dates in the EBS Module is a large factor in the reduced Program
Optimization Percentage for this program. It is imperative end dates are entered for all youth.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9

service_type: Moral Reconation Therapy


summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Dove Academy
Twin Oaks Juvenile Development, Inc
(Contract Provider)
5270 Ezell Road
Graceville, Florida 32440
Primary Service: Seven Challenges
SPEP Review Date(s): December 2018 - December 2019
Florida Department of Juvenile Justice
Report Date(s): December 3, 2019
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
• Service Type: service type points are awarded based on the group category that the
service falls within
• Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
• Amount of Service: or “dosage” refers to the amount and duration of services that were
received
• Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 30 Points
1. 1. Primary Service and Supplemental Service Types POS: 30 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Seven Challenges.
The program was awarded 25 points because the Primary Service is identified as a Group 4
Service. The specific Sub-Component Service Type identified is Group Counseling. The Primary
Service was identified as this type of service as it focuses on psychological or interpersonal
problems or issues faced by an individual and involves a group of youths interacting with each
other.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
Three therapists and the designated mental health clinical authority (DMHCA) are currently
trained to facilitate the Seven Challenges curriculum. All training certificates were located in the
training binders. The third therapist was trained in Seven Challenges for non-clinical staff. This
therapist does not facilitate Seven Challenges group.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program provided three manuals for The Seven Challenges curriculum. The facilitator's
manual has 13 chapters, which covers implementation, each of the seven challenges, counseling
approaches, gender issues, supervision, games for groups, two appendices, four extra
worksheets, group note examples, and treatment goal examples. There is a section in the manual
for group session topics. The topics are able to be copied. The youth read through the topic and
they discuss how the topic applies to their life. The curriculum is open for the therapist to be
creative in how they want to facilitate the group. The Seven Challenges is designed to be an open
discussion for all youth to participate and apply the lesson to their lives.
The second manual provided was an activity book. The activity book provides various counseling
activities for each challenge. Each activity provides materials, instruction, script, how to connect
the lesson, and worksheets for each lesson.
The last manual was the journaling book the youth use during journaling sessions. Seven
Challenges is held as an open group and it is run consistently at the program. As an open group
the youth complete the curriculum at their own pace and new youth are added to the group as
they are admitted to the program. Seven challenges is held on Tuesdays and Thursdays from
10:30am-11:30am and 11:30am to 12:30pm.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
A session of Seven Challenges was unable to be observed; due to dangerous weather conditions,
the reviewer was unable to travel to the program. This curriculum was not able to be observed;
therefore, this indicator rates as non-applicable.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
The previous designated mental health clinical authority (DMHCA) left shortly after the annual
review in December 2018. The new DMHCA was already a therapist in the program and was
already trained in the curriculum. Two therapists consistently facilitate Seven Challenges. The
program's plan is for the third trained therapist and/or the DMHCA to take over facilitation should
a therapist be out or leave employment. There were no gaps in service during this review period.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
Internal fidelity monitoring is conducted by the designated mental health clinical authority
(DMHCA) for Seven Challenges. The internal fidelity monitoring forms are kept in a binder in the
DMHCA's office. All internal fidelity monitoring forms were available for review. There were no
missing internal fidelity monitoring forms for the full review period.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
All internal fidelity monitoring forms were reviewed and there were no corrective actions during the
review period. Suggestions were made in regards to facilitation on various fidelity forms; however,
none of them warranted corrective action.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
All three of the current seven challenges facilitators have received yearly performance
evaluations. The evaluations for therapy staff have a specific section for group facilitation.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service - Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 24 weeks for this type of service. Of the 11 youth in the
sample, 82% (9 of 11) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 40 contact hours for this type of service. Of the 11 youth in the
sample, 73% (8 of 11) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 10 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 40%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 45%
Moderate to High Score: 2 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 17%
Table 1
Moderate = 0 youth
Moderate-High = 2 youth
High = 3 youth
Total Youth in Sample = 11 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 27%
High Score: 8 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 62%
Table 2
High = 3 youth
Total Youth in Sample = 11 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 45% (5 of 11) youth scored Moderate to High-Risk to reoffend, for a score
of 2 points.
Of the SPEP sample, 27% (3 of 11) youth scored High-Risk to reoffend, for a score of 8 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 30 30 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 10 25 40%
Totals 74 95 78%
This SPEP report evaluates Seven Challenges, an intervention delivered at DOVE Academy.
54 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 11 total youth sampled, only
10 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 9
received at least the recommended weeks of service. Youth in the sample completed between 18
and 40 weeks of service, with an average of 29 weeks.
The program earned 6 points for Amount of Service: Contact Hours. Of the 11 total youth
sampled, all 11 included dosage in the EBS Module. Of those youth, 8 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
recommended hours of service. Youth in the sample completed between 15 and 58 hours of
service, with an average of 43 hours.
The program was awarded 10 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 94%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
DOVE Academy can maintain their SPEP Quality of Service Delivery score by continuing the
practices in place at the time of this review.
DOVE Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8

service_type: Seven Challenges


summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Dove Academy
Twin Oaks Juvenile Development, Inc
(Contract Provider)
5270 Ezell Road
Graceville, Florida 32440
Primary Service: Impact of Crime
SPEP Review Date(s): December 2018‐December 2019
Florida Department of Juvenile Justice
Report Date(s): December 2019
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
 Service Type: service type points are awarded based on the group category that the
service falls within
 Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
 Amount of Service: or “dosage” refers to the amount and duration of services that were
received
 Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 20 Points
1. Primary Service and Supplemental Service Types POS: 20 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Impact of Crime. The program was awarded 15 points
because the Primary Service is identified as a Group 3 Service. The specific Sub-Component
Service Type identified is Social Skills Training. The Primary Service was identified as this type of
service as it focuses on developing social skills required for an individual to interact in a positive
way with others.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
There were a total of seven staff formally trained to conduct the Impact of Crime (IOC) curriculum
during the annual review period. Four case management staff, one therapist, one shift supervisor,
and the program director.
All training documentation was located in the staff training files and provided for review. Training
records were maintained in binders located in the human resources office. Each employee at the
program has a training binder. All certificates or SkillPro training records were located within the
binders for IOC.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program maintains the current Impact of Crime (IOC) facilitator's manual. There are four
sections to the IOC manual. The first section is the facilitator’s guide, which includes an
introduction and implementation section. The first section also includes how the chapters and
lessons are structured and how they should be delivered. There are two parts of the first section
that goes in depth and discusses the importance of victims and considerations when facilitating
IOC to girls. The second section has all seven chapters divided and numbered. Within each
chapter contains the IOC script, lessons, and activities for the youth. Every chapter begins with
objectives, words to know, and definitions. The third section is the appendix. This contains extra
worksheets, list for videos, behavior contracts, and other helpful worksheets to use during group.
The final section is the student guide. This is the specific workbook for the youth. These can be
copied and provided to the youth during group as homework and the worksheets they use during
group.
Impact of Crime (IOC) groups are held Mondays and Wednesdays from 3:15pm to 4:15pm and
Tuesdays and Thursdays from 3:15pm to 4:15pm. All youth are eligible to attend IOC groups
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
while in the program. All group documentation is maintained in the program's SPEP folder for IOC
located in the program director's office. The sessions were completed for one hour consistently
per a review of the attendance sheets. For youth who attend the Department of Transportation
work outings, individual sessions and separate attendance sheets were completed. The program
has increased the number of youth who complete IOC before their release.
Basic Score: 1 Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Three of the four current facilitators have been observed conducting Impact of Crime group during
the review period. All facilitators adhered to the lesson plans.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
One Impact of Crime (IOC) facilitator left employment during the review period. According to a
review of the attendance records, there were no gaps in service. The prior case manager was
able to complete the cohort that was started prior to leaving. The program has had one additional
staff member trained to conduct IOC. Six cohorts have been conducted during this review period.
The first cohort was held December 18, 2018 to April 9, 2019. The group started with eleven
youth and ended with seven youth. Four youth were released from the program prior to group
completion. The second cohort was held from March 5, 2019 through June 24, 2019. The group
started with nine youth. This group and the second group combined halfway through the lessons.
Both groups were on the same lesson and objective when combined. The group ended with
fifteen youth. The third cohort was held from May 21, 2019 to September 10, 2019. Twelve youth
started the group and ten youth completed the group. One youth was released from the program
early, but was able to complete the group through individual sessions.
The fourth cohort was held from July 31, 2019 and is currently being held. The group has eight
youth attending. The fifth cohort began on Monday, September 23, 2019 with two facilitators.
There are eleven youth in the group. The sixth cohort began on December 5, 2019 with twelve
youth in the group.
Basic Score: 1 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The program director or therapist conducts internal fidelity monitoring for Impact of Crime groups.
All internal fidelity monitoring forms are maintained in a binder with the attendance forms for the
cohorts. Upon review, three months were missing for two facilitators of the curriculum.
This reviewer spoke with the program director regarding the missing forms. At this time two of the
case managers are rotating faciltiation for the cohort which started September 23, 2019.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
Therefore internal fidelity monitoring was completed for the facilitator in group on the date of
monitoring. This reviewer advised all current facilitators must have a fidelity monitoring completed
monthly. The program director will be ensuring they both receive this.
The program can earn 2 points if fidelity monitoring is completed every month for each facilitator.
At the time of the review, the program did not meet criteria to earn a score of 2.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
The process for corrective actions is for the program director to address the issue during a one-to-
one meeting with the facilitator. The facilitator will be monitored again to ensure the issue is
corrected. No corrective actions were indicated on the internal fidelity monitoring forms during this
review period. The observers did make suggestions consistently, but nothing warranted a
corrective action.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
Two out of the four current facilitators have received their annual review. Two are one month
overdue, but are scheduled to be conducted during the week of December 9th. All case
management yearly performance evaluations include a section for the delivery of delinquency
interventions. The shift supervisor did not have a section to review group facilitation. However, the
program director indicated it would be added to the evaluation when completed.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
3. Amount of Service ‐ Duration
Basic Score: 2 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 20%
Research indicates the target duration of 16 weeks for this type of service. Of the 53 youth in the
sample, 34% (18 of 53) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 6 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 60%
Research indicates a target of 24 contact hours for this type of service. Of the 53 youth in the
sample, 60% (32 of 53) reached the indicated target contact hours. Further explanation is detailed
in the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 15 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 60%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 64%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 4 youth
Moderate-High = 12 youth
High = 18 youth
Total Youth in Sample = 53 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
Percentage of Youth with High-Risk Level to Reoffend: 34%
High Score: 10 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 77%
Table 2
High = 18 youth
Total Youth in Sample = 53 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 64% (34 of 53) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 34% (18 of 53) youth scored High-Risk to reoffend, for a score of 10 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 20 20 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 2 10 20%
Amount of Service: Contact Hours 6 10 60%
Risk Level of Youth Served 15 25 60%
Totals 63 85 74%
This SPEP report evaluates Impact of Crime, an intervention delivered at DOVE Academy.
54 youth completed the program during the review period. Of those youth, 53 were included in the
sample.
The program scored High for Quality of Service Delivery.
The program earned 2 points for Amount of Service: Duration. Of the 53 total youth sampled, only
46 included dosage with end dates in the EBS Module. Of those youth with correct dosage, 18
received at least the recommended weeks of service. Youth in the sample completed between 2
and 18 weeks of service, with an average of 14 weeks.
The program earned 4 points for Amount of Service: Contact Hours. Of the 53 total youth
sampled, only 51 included dosage in the EBS Module. Of those youth with correct dosage, 32
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8
received at least the recommended hours of service. Youth in the sample completed between 2
and 32 hours of service, with an average of 25 hours.
The program was awarded 15 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 86%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
DOVE Academy can optimize their SPEP Quality of Service Delivery score by ensuring fidelity
monitoring is completed every month for each facilitator.
DOVE Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 9


summary: SPEP - FY19-20

raw_content:
Standardized Program Evaluation
Protocol [SPEP] Report
Dove Academy
Twin Oaks Juvenile Development, Inc
(Contract Provider)
5270 Ezell Road
Graceville, Florida 32440
Primary Service: Moral Reconation Therapy
SPEP Review Date(s): December 2018‐December 2019
Florida Department of Juvenile Justice
Report Date(s): December 3, 2019
Introduction
The Standardized Program Evaluation Protocol (SPEP) is data-driven rating system for
evaluating program effectiveness. The SPEP was developed by relying on hundreds of studies
that have examined key characteristics of programs that demonstrated their ability to reduce
recidivism. To determine a SPEP score, the program’s services are compared to characteristics
of effective programs in the research of a similar type.
The SPEP score is composed of four principle characteristics that have been shown to predict
reductions in youth recidivism; service type, quality of service, dosage (amount of service), and
youth level of risk.
 Service Type: service type points are awarded based on the group category that the
service falls within
 Quality of Service: quality points are awarded by determining the extent to which the
program is delivered in what research suggests is a high-quality manner
 Amount of Service: or “dosage” refers to the amount and duration of services that were
received
 Risk Level: refers to the level of criminogenic risk for youth involved in the intervention.
The final output of the SPEP review is the Program Optimization Percentage (POP). This rate is
derived from the Basic Score and Program Optimization Score. The POP rate is a percentage
score that indicates how the service rates in effectiveness when compared to the potential
effectiveness if optimized to match the most effective services in the research. This means the
POP rate represents how close the program’s Primary Service is to its potential for that Primary
Service Type. For example, a POP rate of 55% would indicate that the program’s Primary
Service is operating at 55% of its potential effectiveness for recidivism reduction based on the
SPEP research.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 2
Basic Score: 35 Points
1. Primary Service and Supplemental Service Types POS: 35 Points
POP: 100%
There are five Primary Service Types that have been classified into Groups with a maximum
number of points possible for rating purposes. Some Primary Service Types may also have
qualifying Supplemental Service Types that could earn a program an additional 5 points.
The Primary Service for this program is Moral Reconation Therapy. The program was awarded
30 points because the Primary Service is identified as a Group 5 Service. The specific Sub-
Component Service Type identified is Cognitive Behavioral Therapy. The Primary Service was
identified as this type of service as it is intended to correct faulty cognitions and perceptions and
provides skills individuals can use to monitor thought patterns and correct behaviors.
An additional 5 points was awarded based on a Qualifying Supplemental Service. The Qualifying
Supplemental Service was identified as None (automatic 5 points added to score), which was not
demonstrated to have been implemented.
The Primary and Supplemental Service Raw Score is equal to the sum of the Primary Service
points plus the Qualifying Supplemental Service points.
Note: Quality information is evaluated while on-site during the annual compliance review.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 3
Basic Score: 20 Points
2. Overall Quality of Service Delivery Score POS: 20 Points
POP: 100%
The Quality of Service Delivery Score is the sum of the scores for the seven treatment quality
indicators. The Program Optimization Percentage Rating determines the Overall Quality of
Service Level:
Indicator Sum Score 0-3 = Low; Sum Score 4-7 = Medium; Sum Score 8-10 = High.
Sum of all Indicator Scores (a – g below): 9 Points
Overall Quality of Service Delivery Level:
Low (Raw Score = 5)
Medium (Raw Score = 10)
High (Raw Score = 20 Points)
Basic Score: 1 Point(s)
a. Facilitator Training
Maximum Possible Score: 1 Point
All facilitator(s) of the Primary Service must have received formal training specific to the
intervention or model/protocol.
Three therapists and the designated mental health clinical authority (DMHCA) are trained to
conduct Moral Reconation Therapy (MRT). All training records were provided for review and are
located in the program's training records.
Basic Score: 2 Point(s)
b. Treatment Manual/Protocol
Maximum Possible Score: 2 Points
There is a specific written manual/protocol detailing delivery of the Primary Service.
The program provided a workbook titled, "Juvenile MRT: How to Escape Your Prison, A Moral
Reconation Therapy Workbook". The book is written by Dr. Gregory Little and Dr. Kenneth
Robinson. For this curriculum, each youth is provided a workbook they will work in throughout
their time in group. There are 12 steps each youth must work through and successfully complete
in order to successfully complete the curriculum. Each youth must complete all 12 steps of the
curriculum. They are given specific tasks they must complete before they can move to the next
step.
Moral Reconation Therapy is held as an open group, meaning the youth work through the steps
on their own and complete the curriculum at their own pace. Moral Reconation Therapy is held on
Mondays and Wednesdays from 10:30am to 11:30am and 11:30am to 12:30pm. Youth with a
mental health diagnosis are eligible to attend Moral Reconation Therapy.
Basic Score: N/A Point(s)
c. Observed Adherence to the Manual/Protocol
Maximum Possible Score: 1 Point
Upon observation of the Primary Service by the Quality Improvement reviewer, the facilitator of
that service adhered to the written protocol/manual.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 4
A session of Moral Reconation Therapy was unable to be observed; due to dangerous weather
conditions, the reviewer was unable to travel to the program. This curriculum was not able to be
observed; therefore, this indicator rates as non-applicable.
Basic Score: 2 Point(s)
d. Facilitator Turnover
Maximum Possible Score: 2 Points
Measures the extent to which facilitators of the specific intervention/service have changed as
well as gaps in service of that Primary Service.
There has been no turnover in the therapy staff since the last review. Upon review of the
attendance sheets, there were no gaps in service for this review period. In the event of a therapist
being out or leaving employment, the designated mental health clinical authority (DMHCA) will
facilitate group to avoid any gaps in service.
Basic Score: 2 Point(s)
e. Internal Fidelity Monitoring
Maximum Possible Score: 2 Points
The program has a process to monitor the delivery of the intervention to examine how closely
actual implementation matches the model protocol.
The designated mental health clinical authority (DMHCA) conducts internal fidelity monitoring for
Moral Reconation Therapy groups. All fidelity monitoring forms were made available for review
and are kept in a SPEP binder in the DMHCA's office. There were no missing fidelity forms for
any of the current facilitators of group.
Basic Score: 1 Point
f. Corrective Action based on Fidelity Monitoring
Maximum Possible Score: 1 Point
The program has a process by which corrective action is applied and demonstrated based on
the fidelity monitoring of the delinquency intervention/therapeutic service.
One corrective action was noted upon review of the internal fidelity monitoring forms. There was
also a follow up documented during the following month's monitoring. After this corrective action
there were no other corrective actions. The program has a policy in place to address corrective
actions resulting from internal fidelity monitoring, which was followed as prescribed.
g. Evaluation of Facilitator Skill Delivering the Basic Score: 1 Point
Intervention Maximum Possible Score: 1 Point
Performance evaluations of the facilitators of the specific intervention/service include
evaluation of skill in delivering the intervention/service.
All three of the current Moral Reconation Therapy facilitators have received yearly performance
evaluations. the evaluations for therapy staff have a specific section for group facilitation of this
intervention.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 5
3. Amount of Service ‐ Duration
Basic Score: 8 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 80%
Research indicates the target duration of 15 weeks for this type of service. Of the 14 youth in the
sample, 86% (12 of 14) reached at least the indicated target duration. Further explanation is
detailed in the Summary and Recommendations below.
Note: Dosage information (duration) is calculated from the Juvenile Justice Information System
(JJIS) Evidence-Based Services module. Duration is included for the youth in the SPEP sample.
4. Amount of Service – Contact Hours
Basic Score: 0 Points
Program Optimization Score: 10 Points
Program Optimization Percentage: 0%
Research indicates a target of 45 contact hours for this type of service. Of the 14 youth in the
sample, 7% (1 of 14) reached the indicated target contact hours. Further explanation is detailed in
the Summary and Recommendations below.
Note: Dosage information (contact hours) is calculated from the Juvenile Justice Information
System (JJIS) Evidence-Based Services module. Contact hours are included for the youth in the
SPEP sample.
5. Risk Level of Youth Served:
Basic Score: 10 Points
Program Optimization Score: 25 Points
Program Optimization Percentage: 40%
Percentage of Youth with Moderate, Moderate-High, and High-Risk Levels to Reoffend: 50%
Moderate to High Score: 5 Points
Program Optimization Score: 12 Points
Program Optimization Percentage: 42%
Table 1
Moderate = 0 youth
Moderate-High = 4 youth
High = 3 youth
Total Youth in Sample = 14 youth
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 6
Percentage of Youth with High-Risk Level to Reoffend: 21%
High Score: 5 Points
Program Optimization Score: 13 Points
Program Optimization Percentage: 38%
Table 2
High = 3 youth
Total Youth in Sample = 14 youth
The risk level score is compiled by calculating the total percent of the SPEP sample that score
Moderate to High-Risk to reoffend and also the total percent of the SPEP sample that score High-
Risk to reoffend.
Of the SPEP sample, 50% (7 of 14) youth scored Moderate to High-Risk to reoffend, for a score
of 5 points.
Of the SPEP sample, 21% (3 of 14) youth scored High-Risk to reoffend, for a score of 5 points.
Note: The latest Community Positive Achievement Change Tool (C-PACT) prior to the placement
date was used in the derivation of the risk level score. This C-PACT provides the best indication
of the risk to re-offend level of the youth when the youth was first placed in the program.
Summary and Recommendations
Program Program
Category Basic Score Optimization Optimization
Score Percentage
Primary and Supplemental Service Type 35 35 100%
Quality of Service Delivery 20 20 100%
Amount of Service: Duration 8 10 80%
Amount of Service: Contact Hours 0 10 0%
Risk Level of Youth Served 10 25 40%
Totals 73 100 73%
This SPEP report evaluates Moral Reconation Therapy, an intervention delivered at DOVE
Academy.
54 youth completed the program during the review period.
The program scored High for Quality of Service Delivery.
The program earned 8 points for Amount of Service: Duration. Of the 14 total youth sampled, 12
included dosage with end dates in the EBS Module. Of those youth with correct dosage, 12
received at least the recommended weeks of service. Youth in the sample completed between 18
and 40 weeks of service, with an average of 30 weeks.
The program earned 0 points for Amount of Service: Contact Hours. Of the 14 total youth
sampled, all 14 included dosage in the EBS Module. Of those youth, 1 received at least the
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 7
recommended hours of service. Youth in the sample completed between 15 and 46 hours of
service, with an average of 33 hours.
The program was awarded 10 available points for Risk Level of Youth Served. This is calculated
using data from the Community - Positive Achievement Change Tool (C-PACT) assessment. This
score reflects youths' most recent C-PACT score prior to placement at the program. The program
itself has no control over youths' C-PACT risk level because the scored assessment was
administered prior to the youths' admission.
Had the percentage of youth served with Moderate-High and High risk to reoffend been higher,
and this report awarded each 25 of the available points for this section, the Program Optimization
Percentage for this service would increase to 88%. As stated previously, the Risk Level of Youth
Served score is not under the control of the program.
RECOMMENDATION(S):
DOVE Academy can maintain their SPEP Quality of Service Delivery score by continuing the
practices in place at the time of this review.
DOVE Academy can optimize their SPEP Amount of Service score by ensuring that dosage for all
youth is recorded accurately in EBS and by ensuring that youth receive the full targeted dosage of
service.
Department of Juvenile Justice | Standardized Program Evaluation Protocol [SPEP] Report 8