Date of Award
Summer 2024
Document Type
Open Access Dissertation
Department
Psychology
First Advisor
Kimberly Becker
Abstract
Over the past 25 years, there have been increasing calls to advance the application of evidence in routine mental health settings for youth and families. Methods in the field for characterizing evidence application have played a critical role in shaping these efforts, which, to-date, have largely focused on measuring providers’ adherence to practices from evidence-based programs in treatment sessions. While this program-centric paradigm for characterizing evidence application has supported the widespread implementation of evidence-based programs in public mental health settings, it is limited in scope and applicability for characterizing all the ways that evidence may be applied across a service episode to accommodate the variable realities of routine service contexts and address the diverse and complex challenges that youth in these settings present with. This dissertation sought to describe, test, and demonstrate the utility of an observational coding system that follows a knowledge-centric paradigm for characterizing evidence application, with a specific focus on treatment engagement in youth mental health services. The presented coding system was designed to measure interventions at varying levels of granularity and targets of potential focus across different phases of planning, implementation, and evaluation throughout a service episode. The first study described the development of the coding system and evaluated its interrater reliability in measuring steps (i.e., granular therapeutic techniques), practices (i.e., broader clinical procedures), and targets (i.e., problems that may be targeted through intervention) relevant to treatment engagement across different service activities (i.e., treatment, supervision) and levels of analysis (e.g., excerpt-level, event-level). The results of this study demonstrated that the coding system generally measured steps, practices and targets with sufficient reliability. However, specific patterns of variation in the reliability of codes across contexts that were observed in this study may be important to consider when selecting measurement strategies for quality improvement efforts. The second study provided one ‘proof of concept’ of how this coding system could be applied to examine the flexible application of evidence in routine care with the purpose of generating practice-based evidence. Specifically, this study examined patterns of clinical practice associated with a stronger therapeutic relationship in routine mental health care and compared patterns to empirical associations in the interventions literature. Results revealed that providers evidencing a stronger therapeutic relationship more frequently used steps consistent with “soft skills” (e.g., Express empathy, Express optimism) while still using other steps (e.g., Practice skill, Make a specific plan) with a similar frequency compared with providers evidencing a weaker therapeutic relationship. In addition, when comparing practice patterns to empirical associations in the interventions literature, it was found that providers in stronger relationship sessions used significantly more “relationship” practices (i.e., clinical procedures common to interventions that effectively improved the therapeutic relationship in the literature) than providers in weaker relationship sessions at a high level of extensiveness, but not at a low level of extensiveness. Findings from these studies demonstrate that the presented coding system may provide a reliable measurement strategy for advancing our understanding of the many nuanced ways that evidence is applied in routine service contexts.
Rights
© 2024, Eleanor Wu
Recommended Citation
Wu, E.(2024). Measuring Evidence Application in Routine Mental Health Settings: Testing a Coding System and Providing a Proof of Concept. (Doctoral dissertation). Retrieved from https://scholarcommons.sc.edu/etd/7923