The UCF Court Health Services and Policy Workgroup, an interdisciplinary team led by Dr. Barbara “Basia” Andraka-Christou, received a competitive $150,000 award to evaluate enhancement and evaluation for Florida's Family Dependency Drug Courts. This three-year, $1.5 million project is funded by the Bureau of Justice Administration and was awarded to the Florida Office of the State Courts Administrator. The workgroup’s project evaluation involves examining the implementation process, implementation outcomes, and health service outcomes of evidence-based substance use disorder treatment, parenting engagement interventions and cultural competency practices in five family dependency drug courts across the state of Florida. The research team uses community participatory mixed methods to evaluate the program in four stages. The research approach is grounded in the EPIS Framework, Proctor’s Implementation Outcomes, and Centers for Disease Control and Prevention best practices for program evaluation.
The UCF Health Services and Policy Workgroup is evaluating a three-year, $1.5 million grant funded by the Bureau of Justice Administration and awarded to the Florida Office of the State Courts Administrator (OSCA). The project evaluation involves examining the implementation process, implementation outcomes, and health service outcomes of evidence-based substance use disorder treatment, parenting engagement interventions, and cultural competency practices in five family dependency drug courts across the state of Florida.
The Centers for Disease Control and Prevention recommends two types of evaluations for these kinds of projects: 1) an implementation/process evaluation and 2) an outcome evaluation. (Process evaluations occur before outcome evaluations1.)
Process evaluations include documenting how the intervention was implemented, how people experienced the intervention, the degree to which essential elements were delivered, the context in which interventions were delivered, and the potential impact of contextual factors on outcomes1. For our process evaluation, we will use the Exploration, Implementation, Preparation and Sustainment framework (i.e. “EPIS framework”) to describe the stages of implementation as well as the context of implementation and related barriers or facilitators to implementation2. For process evaluation, we will also use Proctor et al.’s framework to describe implementation measures of fidelity, appropriateness, adoption, acceptability, penetration and cost3. For outcome evaluation, we will utilize best practices recommended by the Centers for Disease Control and Prevention1.
Our research will describe the intervention(s) selected by each court and the implementation context. Data collection for this aim includes documents provided by each court to the Office of the State Courts Administrator (i.e. court action plans), qualitative focus group data from court staff/judges and qualitative interview data with stakeholders from court staff/judges.
We will describe the intervention implementation process using the EPIS framework, which posits that implementation occurs in four stages: exploration, preparation, implementation, and sustainability. Our research team will collect and analyze data at each of these four stages for the duration of the grant project. Identification of the stage of implementation of each court is based on integration of qualitative and quantitative results.
We will identify health outcomes and explore the causal relationship between the intervention implementation and health outcomes. Implementation of an evidence-based intervention is expected to lead to a positive change in health outcomes. Health outcomes cannot be interpreted apart from implementation outcomes. For example, if a health outcome does not change as expected following intervention implementation, it may be due to lack of fidelity of implementation of the intervention, rather than lack of efficacy of the intervention. Therefore, our research team will primarily be providing results regarding health outcomes in the latter stages of the project, after we have collected comprehensive data on implementation outcomes.
Since the project involves courts rather than traditional healthcare settings (e.g. hospitals), we are conceptualizing health outcomes more broadly to not only include client health measures (e.g. relapse rates) but also court program completion and parent-child reunification. Information about expected outcomes for each intervention have been obtained and integrated from three sources: action plans submitted by courts to the Office of the State Courts Administrator, interview and focus group data, and scholarly literature (if available) on the intervention selected. Specific expected health outcomes for each court are available in the results section of this report.
- Centers for Disease Control and Prevention. Types of Evaluation.
- Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. 2011;38(1):4-23.
- Proctor E, Silmere H, Raghavan R, et al. Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda. Adm Policy Mental Health. 2011;38:65-76.
- Proctor EK, Warren G, Landsverk J, et al. Implementation Research in Mental Health Services: an Emerging Science with Conceptual, Methodological, and Training challenges. Adm Policy Ment Health. 2009;36.
The UCF Court Health Service and Policy Workgroup includes interdisciplinary faculty and graduate students with expertise in substance use disorder health services and policy, mental health services, qualitative methods, quantitative methods and mixed methods. Our goal is to help judges, court staff, court participants, and government agencies use evidence-based practices for substance use disorder treatment, including for opioid use disorder.
Contact the UCF Court Health Service and Policy Workgroup at CourtHealthResearch@ucf.edu.