An observational study in Mobile Application-Based CBT Worksheet Collection and Paper-Based Worksheet Collection, sponsored by Palo Alto Veterans Institute for Research. Status unknown at 4 sites in United States. Open to participants aged 18 Years and older, including healthy volunteers. Per ClinicalTrials.gov, last updated 2022-11-03.
Sponsored by Palo Alto Veterans Institute for Research · Observational
This project compares two methods of assessing the quality of cognitive behavioral therapy (CBT) that do not involve directly observing sessions: 1) adherence checklists embedded in clinical notes, and 2) rating the quality of worksheets that are completed with therapist guidance during sessions. It also examines whether ratings of worksheets completed on a mobile app are reliable and valid quality measures. This information can inform strategies to monitor and enhance CBT quality, which can ultimately improve the quality of care and clinical outcomes.
Cognitive Behavioral Therapy (CBT) has been demonstrated to be effective for numerous presenting problems, including depression, anxiety, and post-traumatic stress disorder (PTSD). Several large mental health systems have invested heavily in programs to train their clinicians in CBTs, but relatively little attention has been devoted to the monitoring or promotion of CBT quality after training is complete. Identifying strategies to do so can facilitate research and training, and is critical to ensuring consumer access to high quality, evidence-based treatments. The lack of a scalable, effective, and efficient method of monitoring quality is a key barrier to efforts to promote high-quality implementation. Self-report fidelity assessments increase clinician and consumer burden and may not accurately reflect clinician skill or the intensity with which CBT interventions are delivered. Observation and expert ratings are time and resource intensive and unlikely to be feasible or affordable in large systems. To maximize the likelihood of broad implementation once effective strategies to monitor quality are established, it is essential that these strategies are feasible and acceptable in routine care contexts, leveraging information collected during routine care. To date, few monitoring strategies that do not involve observation, client/caregiver reports, or clinician self-reports have been tested. To address this critical implementation challenge, we propose to refine and evaluate a method of monitoring quality that is based on an evaluation of CBT worksheets that are completed in session. Because the worksheets were developed to implement core cognitive and behavioral elements and are embedded in CBTs across diagnostic categories, they may be used to elucidate the clinician's ability to guide the client through CBT interventions in session. Preliminary research with this measure demonstrated high correlations between the measure and observer ratings of clinician competence, associations with subsequent symptom change, and high agreement between raters with differing levels of familiarity with CBT. Completion of the ratings based on worksheets requires only a small fraction of time required for session observation and ratings. This project will compare this novel strategy to observer ratings and adherence checklists that are embedded in clinical notes. Furthermore, it will compare the accuracy of worksheet data collected by mobile app to paper-form worksheets, and assess the feasibility and acceptability of these strategies. Because the core elements of CBT and its worksheets are common across many CBTs, this research has broad implications for monitoring fidelity to CBTs in a variety of mental health and healthcare systems and settings. This research will be conducted by a team of investigators with expertise in CBT, training, implementation, psychotherapy process and outcome research, psychometrics, longitudinal data analysis, mobile technologies and healthcare economics, with input from community partners and end-users. The resulting products have the potential to significantly improve efforts to monitor and ensure ongoing high quality implementation of CBT in routine care settings.
Therapists who provide Cognitive Behavioral Therapy, and their patients who receive CBT for depression, anxiety, or PTSD. Therapists and patients will be recruited from community-based, public mental health (e.g., Boston and Palo Alto VA) and private practice treatment settings.
Inclusion Criteria:
A. Clinician eligibility:
B. Patient eligibility:
Clinician diagnosis of primary PTSD (PTSD-Checklist-5 score of 33 or more) Depressive disorder (e.g., major depressive disorder, dysthymia; PHQ of 10 or above) Or an Anxiety Disorder (Beck Anxiety Disorder score of 22 or above)
Note that if a patient has a score close to the cut-off score, it's up to the therapist digression
Cannot have*:
C. Administrator eligibility:
Although we will be mostly observing routine practice, we will randomize patients into either an App condition or paper condition. The App condition refers to patients completing routine Cognitive Behavioral Therapy (CBT) worksheets on a mobile application during session. Everything else that occurs in treatment sessions will be consistent with routine care practices.
Behavioral: Cognitive Behavioral Therapy
Although we will be mostly observing routine practice, we do randomize patients into either an App condition or paper condition. The paper condition refers to patients completing routine CBT worksheets on paper (the current standard) during session.
Behavioral: Cognitive Behavioral Therapy
Cognitive Behavioral Therapy (CBT) is an evidence-based psychotherapy that treats a variety of disorders. In this study, clinicians will be enrolled who use types of CBT for anxiety disorders, depression and PTSD in their routine clinical practice. CBT is usually a 12-16 session treatment that focuses on intervening on disorders cognitively and behaviorally through the use of worksheets during session and outside of session.
Also known as: Cognitive Processing Therapy
Change in Patient Health Questionnaire (PHQ-9)
9-item self-report measure of depression symptoms with good internal reliability (α = .89) and test-retest reliability (r = .84).
Time frame: Baseline (pre-treatment), Every session (weekly) if primary /target problem up to 16 weeks
Change in Posttraumatic Stress Disorder Checklist (PCL-5)
A 20-item self-report measure that assesses the 20 Diagnostic and Statistical Manual (DSM-5) symptoms of PTSD. Good internal consistency and concurrent validity with the Clinician-assessed PTSD Scale.
Time frame: Baseline (pre-treatment), Every session (weekly) if primary /target problem up to 16 weeks
Change in Beck Anxiety Inventory
21-item self-report instrument for measuring the severity of anxiety symptoms. Good internal consistency (α = .92) and high test-retest reliability (r = .75.).
Time frame: Baseline (pre-treatment), Every session (weekly) if primary /target problem up to 16 weeks
Change in Inventory of Psycho social Functioning-Brief Version
14 items, yields a grand mean and 7 subscale means for functioning in romance, family, friendships, parenting, education, work, and self-care activities (non-applicable categories can be skipped; total score reflects mean of applicable items). Excellent reliability and internal consistency ( α = 0.93); correlates with other measures of quality of life.
Time frame: Completed at baseline, One Month, and post-treatment (week 16)
Evidence-Based Practice Attitudes Scale-50 (EBPAS)
This measure looks at Evidence-based practice attitudes and motivation among clinicians, 1 (worse)-5 (best) total score range
Time frame: Completed at therapist baseline (prior to providing treatment in the study)
Perceived Characteristics of Interventions (PCIS)
Innovation characteristics hypothesized to influence adoption and sustainability
Time frame: Completed at therapist baseline (prior to providing treatment in the study)
Implementation Climate Assessment (ICA)
Brief validated measure of implementation climate ( manager support for EBPs, etc.)
Time frame: Completed at therapist baseline (prior to providing treatment in the study)
This study is status unknown, as verified in Oct 2022. You cannot join it, but the record below documents what was studied.
Get an email when the registry record changes — status, dates, results — or when someone posts here.
Sign in to followQuestions and observations about this study, from anyone following it. Not medical advice, and not a channel to the study team — their contact details are on the registry record.
Sign in to join the discussion. Reading takes no account; posting does. You choose a display name, and a pseudonym is the default.
Nothing here yet. If you are running this trial, taking part in it, or weighing whether to, this is the place to say so.
Palo Alto Veterans Institute for Research