Background: With support from NIDA (#R44-DA12066) we have developed an online curriculum to educate and further clinical skill development in Year 2 to 4 medical students on buprenorphine treatment for opioid use disorders. Objective: To educate medical students about treatment of patients with opioid dependence using buprenorphine and impact clinical skills in working with patients with addictive disorders. Program Description: We are performing a summative evaluation of our curriculum. Each medical student completed a pre/post test effectiveness evaluation of our online curriculum, including pre/post evaluations of clinical skills through a remote live standardized patient (SP) experience. The OSCE style evaluation of the learner's performance during the remote, live SP clinical encounter was assessed by the SP actor and a MA level clinical professional. Measurement was done via: 1) a 9 item Likert scale rating sheet, and 2) a 16 item done/not done checklist. Both modifications were reviewed and approved by two external clinical substance abuse professionals. Program Evaluation Outcome: Preliminary results (n=25) of this evaluation indicate that the modules have a statistically significantly effect on knowledge and intended behavior related to treatment of patients with opioid dependence (p<.001). Average pre-knowledge score was 68% while the average post-test score was 83% (p<0.001). Attitude and self-efficacy also increased significantly for most modules. Students showed the most improvement in the areas of common comorbidities and pharmacology. Student competency was measured in the following areas: drug detection and screening, comorbid conditions, diagnosis and treatment. Competency (p<0.0007) increased quite significantly from pre to post experience. Competency scores increased an average of 44% from pre to post-interview. Conclusions: Based on the preliminary results, this curriculum is an effective tool for educating students on buprenorphine and opioid use disorder. The addition of a standardized patient component pre and post curriculum allows the students an opportunity to better demonstrate an increase in competencies and clinical skill in a more true to life situation.

Background: With support from NIAAA (#4R44AA016724-02) we are developing an online curriculum to educate and enhance clinical skill development in Year 2 to 4 medical students on at risk drinking and alcohol use disorders. Objective: Medical students were surveyed to understand learning preferences regarding the topics covered and the case-based learning environment. Program Description: Recruited through a convenience sampling approach, medical students (n=17) completed an on-line survey regarding their interests and perceived needs. Quantitative analyses of responses to multiple choice and 1-5 point Likert scale questions guided development of a case-based online curriculum. Program Evaluation Outcome: Students felt inadequately prepared [1=not prepared, 5=fully prepared] to address the following: ability to treat alcohol use disorders in patients with other addictions (1.65), ability to treat alcohol use disorders (1.76), ability to treat alcohol use in patients at risk for alcohol use disorders (1.76), ability to triage alcohol use disorder patients to appropriate treatment areas (2.00), recognition and treatment/proper referral of relapse to alcohol use disorders (1.88). Students were most interested in learning more about [1=uninterested, 5=very interested]: brief interventions (3.94), co-morbidities (3.94) and case studies representing a variety of patient experiences (4.41). Students would likely use an optional live remote Standardized Patient learning experience [1=strongly disagree, 5=strongly agree] (3.41) and would like to have the option available to them (3.53). A Web-based case interface (4.12) and online chat discussion with the patient (3.94) were preferred. Students preferred to learn clinical skills in multiple short cases compared to one long case (77%). Conclusions: Medical students identify many deficits in their knowledge and skills especially those skills that would be essential in any practice. They express interest in an experience using a live remote standardized patient which would focus on alcohol intervention skills training.

Objective: We completed a needs and preference analysis of primary care providers to guide development of a live remote (offsite) Standardized Patient experience that would be used for skills training. We specifically wanted to understand enthusiasm for the concept of remote (Internet-based) SP-based learning that would occur via live chat or other real-time communication technology. We also sought preferences regarding such an experience, so the experience could be designed efficiently and be consistent with the target audience s wishes. We specifically sought input regarding an SP experience involving interviewing skills related to recognizing addiction and the risk of potential misuse of controlled substances.

Method: We performed online surveys of convenience samples of 3 groups of practicing physicians (total n=27), residents (n=9), nurse practitioners (n=9), and addiction specialist physicians (n=8) regarding their preference in terms of a remote Standardized Patient experience. Total number of respondents surveyed was 53.

Results or Conclusions: Enthusiasm for learning via a live remote SP experience was strong and consistent across different experience levels of physicians as well as for nurse practitioners. In all groups, 77% or more of respondents expressed willingness to take an online continuation education course or earn CME credits based on a remote Standardized Patient experience. The majority of respondents (50-85%) preferred that Internet-based text chat be used for the remote experience instead of video, telephone, or an asynchronous bboard. Most (55 to 78%) preferred to be able to use a specific time window for the experience compared to setting aside a specific time. The majority also expressed a willingness to be observed by other learners and to observe other learners themselves (50 to 77%). Further investigation of this aspect of live remote SP learning is warranted, since respondents were interested having or serving as an observer. Health care providers are interested and willing to utilize a remote SP experience for clinical skills training. Their needs and preferences can be accommodated efficiently.

Through a convenience sampling approach, we performed 5 detailed needs analysis interviews of PCPs. We also asked interviewees about typical primary care practice to better understand current behavior so that interactive educational materials would be realistic. With respect to typical primary care practice, we found that most of the PCPs screen for current substance abuse, but over half do not screen for past substance abuse. Most participants only ask about substance abuse during the patient interview if the medical history screening question(s) were positive or if there is another red flag. The fact that followup questioning only happens in these situations underscores the need for the medical history questions to be sensitive and evidence-based. Interviewees reviewed the early plans for the website to see if it met their needs and to identify further needs. Interviewees described a strong interest in information on: 1) reimbursement, 2) how to handle resistance from patients, especially when alcohol is the substance, 3) brief intervention, and 4) referral resources. They also noted a lack of a protocol for updating screening info for patients in ongoing care. The planned curriculum is a core course on each of the following: screening, brief intervention, and referral to treatment plus an advanced course addressing variations in patients or settings. The planned educational approach is live, standardized patient interviews via online chat. Support among the interviewees for the planned curriculum including the method of providing online learning was high. All participants said they were likely or very likely to take a course on SBIRT. The need for a curriculum including evidenced-based recommendations and skills training in SBIRT is clear.

With support from NIAAA (#4R44AA016724-02) we are developing an online curriculum to educate and further clinical skill development in Year 2 to 4 medical students on at risk drinking and alcohol disorders. Through a convenience sampling approach, fifteen medical school faculty took an on-line survey to obtain preference information regarding case-based instruction. Almost all of the faculty (n=14) preferred multiple short cases to cover one topic, rather than one long case in a module. In a course with both didactic and case information, half of the faculty (n=7) preferred three cases per one hour course. When presenting the case to students, 66.7% (n=10) preferred that information be general. To enhance clinical skill development, we queried respondents as to their preferences for using a remote live Standardized Patient (SP) interview within the learning experience. Fourteen out of fifteen (93%) felt that an online SP experience would be an interesting way to teach skills and 87% (13/15) would be willing to use the online SP with their students. Faculty preferred that students receive basic patient case information online through the website (80%; 12/15) and an Internet chat (i.e. Google or Skype) was rated high for conversing with the live SP. We queried about potential topics. Respondents showed a clear interest in supplementing their teachings with materials on assessment and screening (80%, n=12), providing brief interventions (87%, n=13), common co-morbidities of alcohol use disorders (80%, n=12), and referring patients for treatment (87%, n=13). Based on their experience with medical students, the respondents felt that medical students would be interested in learning more about the biology behind addiction (80%, n=12), co-morbid diseases associated with long-term alcohol use (67%, n=10), and recognizing abuse in health case professions (80%, n=12). Support for the planned curriculum including the method of providing online learning was high.

Clinical skills assessment exams using Standardized patients (SPs) are an established component of medical school education, including the current USLME Step 2 CS exam™. The authors have developed a novel method for conducting SP encounters using web-based chat technology (e.g. Google® chat). “Remote Live Standardized Patient” (RLSP) interviews are conducted entirely online. The interview requires active learner participation focused on clinical skills, and represents a unique solution to teaching specific skills and affecting behavior. We are not aware of other educators using chat for this purpose. We have been pilot testing chat-based OSCE-style exams1,2 since 2007. In 2009 we have begun using RLSPs to teach skills, as well as evaluate them. The instructional RLSP key components include an interactive SP actor, an EMR like interface, and a “Hats Off” mode, which allows the student learner to “ask a preceptor” or research information during the encounter. Each learner’s performance is evaluated by standardized measures and by the RLSP actor feedback. This interactive, responsive web environment allows the RLSP interview to simulate a broad spectrum of learner/patient encounters, from initial screening through diagnosis and treatment.

Advantages include 1) decreased SP training cost and time commitment, 2) more flexible SP scheduling and recruiting since all encounters occur online, 3) elimination of face-to-face constraints of verisimilitude errors and correlation of actor physical appearance compared to the “patient”, and 4) instant availability of chat conversation transcripts for review and learner feedback. Potential limitations include missing the “whole person” experience afforded by face-to-face encounters, including the ability to assess body language and demonstrate physical exam skills. Since face-to-face SP encounters also suffer from realism limitations (i.e. the actor does not typically have the “patient's” presenting symptoms), we feel that the advantages of an RLSP balance and actually outweigh the drawbacks.

Use of web-based chat or its equivalent may have significant appeal to the current generation of Internet-savvy medical students. The RLSP experience appears to be a viable alternative to the traditional face-to-face SP interview, allows cost savings, and prepares students for Step 2 CS-style exams.

Introduction: To deliver buprenorphine training to a larger number of physicians, we developed an Internet-based buprenorphine training program consisting of 12 online continuing medical education (CME) modules organized into 8 hours of learning. Education could be completed in any order and at the learner's convenience. In total, 1180 learners completed the program between April 2007 and April 2009; 95% were physicians.

Methodology: For each course, learners completed pre- and post-test measures on knowledge about substance abuse and treatment, attitudes toward treating addiction, intended behavior toward addicted patients, self-efficacy in understanding and treating addiction, and satisfaction with the learning experience. For analysis purposes, knowledge post-tests recorded only the learners' first answers to each question. The data reported here was collected from April - October 2008 (n=243).

Results: Data analysis revealed that the mean scores increased from the pre- to post-test in 12/12 courses for all outcomes measured: knowledge, attitude, intended behavior, and self-efficacy (p< .0001 for each outcome). Of learners who completed the optional post-training satisfaction survey (n= 243), 89.7% agreed/strongly agreed that the training program was a valuable learning experience and 95% said that they had colleagues who could benefit from the program. Convenience, organization, and availability of the online program were strong assets.

Conclusion: Online training on buprenphine is well received and effective. The value of online training to other topics and for alternative audiences should be studied further.

Disclaimer/Funding: The training program was developed with funding from NIDA (#R44DA12066) and is supported by ASAM as a DATA-qualifying buprenorphine training program.

Idea: Medical educators interested in online learning have confidence in their ability to effect knowledge and the tools exist to demonstrate success. Yet, online learning is limited by the inability to show that the learning experience effects higher levels in Bloom's Taxonomy. How do we understand if an online course is succeeding beyond its knowledge improvement goal?

Why the idea was necessary: Students took 5 online courses on the topic of addiction that were broken down into Prevalence, Detection and Diagnosis, Comorbidities, PCP Role, and Pharmacology. The courses demonstrated significant changes in knowledge in a very small sample (N=10) with p ranging from .033 to .001 and average knowledge scores improving from 14% to 52%. Self-efficacy too showed statistically significant improvement (p<.001) even with the very small sample. The students rated themselves as being able to effectively screen and detect opioid abuse following the learning experience. There was not enough of a difference in scores to detect a difference in attitudes and intended behavior. Unfortunately the standardized patient interview showed no change in clinical skills.

What was done: This project utilized a remote standardized patient (SP) experience to test, examine, and improve the learning. We planned to use the ratings of the SP experience to identify the weaknesses in student performance and modify the online learning to address them. After the first round of students completed the study, we looked at the data from both the Modified Skills Inventory (a subjective rating tool completed only by the standardized patient) and the Evaluator Rating Sheet (an objective checklist completed by both the standardized patient and an observer).

Evaluation: Results from the Modified Skills Inventory showed general improvement for all participants post-experience. However, the Evaluator Rating Sheet had some problematic areas; clinical areas which were expected to improve post-experience did not improve. Specifically, students consistently failed to screen the standardized patient using a standardized measure, such as the CAGE-AID. Mental status exams were also rarely conducted, and students showed difficulty in screening patients for psychiatric co-morbid conditions. We evaluated our existing curriculum, and found weaknesses in the level of detail paid to these areas. Pages covering these content areas were expanded to further illustrate these teaching points.

Conclusions: A remote SP experience is a useful tool in the development of an online educational intervention. It can ensure that the student is truly mastering skills and integrating learning into their clinical experience framework.