Background: With support from NIAAA (#4R44AA016724-02) we are developing an online curriculum to educate and enhance clinical skill development in Year 2 to 4 medical students on at risk drinking and alcohol use disorders.

Objective: Medical students were surveyed to understand learning preferences regarding the topics covered and the case-based learning environment.

Program Description: Recruited through a convenience sampling approach, medical students (n=17) completed an on-line survey regarding their interests and perceived needs. Quantitative analyses of responses to multiple choice and 1-5 point Likert scale questions guided development of a case-based online curriculum.

Program Evaluation Outcome: Students felt inadequately prepared [1=not prepared, 5=fully prepared] to address the following: ability to treat alcohol use disorders in patients with other addictions (1.65), ability to treat alcohol use disorders (1.76), ability to treat alcohol use in patients at risk for alcohol use disorders (1.76), ability to triage alcohol use disorder patients to appropriate treatment areas (2.00), recognition and treatment/proper referral of relapse to alcohol use disorders (1.88). Students were most interested in learning more about [1=uninterested, 5=very interested]: brief interventions (3.94), co-morbidities (3.94) and case studies representing a variety of patient experiences (4.41).

Students would likely use an optional live remote Standardized Patient learning experience [1=strongly disagree, 5=strongly agree] (3.41) and would like to have the option available to them (3.53). A Web-based case interface (4.12) and online chat discussion with the patient (3.94) were preferred. Students preferred to learn clinical skills in multiple short cases compared to one long case (77%).

Conclusions: Medical students identify many deficits in their knowledge and skills especially those skills that would be essential in any practice. They express interest in an experience using a live remote standardized patient which would focus on alcohol intervention skills training.

Background: With support from NIDA (#R44-DA12066) we have developed an online curriculum to educate and further clinical skill development in Year 2 to 4 medical students on buprenorphine treatment for opioid use disorders.

Objective: To educate medical students about treatment of patients with opioid dependence using buprenorphine and impact clinical skills in working with patients with addictive disorders.

Program Description: We are performing a summative evaluation of our curriculum. Each medical student completed a pre/post test effectiveness evaluation of our online curriculum, including pre/post evaluations of clinical skills through a remote live standardized patient (SP) experience. The OSCE style evaluation of the learner's performance during the remote, live SP clinical encounter was assessed by the SP actor and a MA level clinical professional. Measurement was done via: 1) a 9 item Likert scale rating sheet, and 2) a 16 item done/not done checklist. Both modifications were reviewed and approved by two external clinical substance abuse professionals.

Program Evaluation Outcome: Preliminary results (n=25) of this evaluation indicate that the modules have a statistically significantly effect on knowledge and intended behavior related to treatment of patients with opioid dependence (p<.001). Average pre-knowledge score was 68% while the average post-test score was 83% (p<0.001). Attitude and self-efficacy also increased significantly for most modules. Students showed the most improvement in the areas of common comorbidities and pharmacology. Student competency was measured in the following areas: drug detection and screening, comorbid conditions, diagnosis and treatment. Competency (p<0.0007) increased quite significantly from pre to post experience. Competency scores increased an average of 44% from pre to post-interview.

Conclusions: Based on the preliminary results, this curriculum is an effective tool for educating students on buprenorphine and opioid use disorder. The addition of a standardized patient component pre and post curriculum allows the students an opportunity to better demonstrate an increase in competencies and clinical skill in a more true to life situation.

Idea: Medical educators interested in online learning have confidence in their ability to effect knowledge and the tools exist to demonstrate success. Yet, online learning is limited by the inability to show that the learning experience effects higher levels in Bloom's Taxonomy. How do we understand if an online course is succeeding beyond its knowledge improvement goal?

Why the idea was necessary: Students took 5 online courses on the topic of addiction that were broken down into Prevalence, Detection and Diagnosis, Comorbidities, PCP Role, and Pharmacology. The courses demonstrated significant changes in knowledge in a very small sample (N=10) with p ranging from .033 to .001 and average knowledge scores improving from 14% to 52%. Self-efficacy too showed statistically significant improvement (p<.001) even with the very small sample. The students rated themselves as being able to effectively screen and detect opioid abuse following the learning experience. There was not enough of a difference in scores to detect a difference in attitudes and intended behavior. Unfortunately the standardized patient interview showed no change in clinical skills.

What was done: This project utilized a remote standardized patient (SP) experience to test, examine, and improve the learning. We planned to use the ratings of the SP experience to identify the weaknesses in student performance and modify the online learning to address them. After the first round of students completed the study, we looked at the data from both the Modified Skills Inventory (a subjective rating tool completed only by the standardized patient) and the Evaluator Rating Sheet (an objective checklist completed by both the standardized patient and an observer).

Evaluation: Results from the Modified Skills Inventory showed general improvement for all participants post-experience. However, the Evaluator Rating Sheet had some problematic areas; clinical areas which were expected to improve post-experience did not improve. Specifically, students consistently failed to screen the standardized patient using a standardized measure, such as the CAGE-AID. Mental status exams were also rarely conducted, and students showed difficulty in screening patients for psychiatric co-morbid conditions. We evaluated our existing curriculum, and found weaknesses in the level of detail paid to these areas. Pages covering these content areas were expanded to further illustrate these teaching points.

Conclusions: A remote SP experience is a useful tool in the development of an online educational intervention. It can ensure that the student is truly mastering skills and integrating learning into their clinical experience framework.

Idea: For medical students in the pre-clinical years, Standardized Patients (SPs) can help the learner integrate medical science knowledge into a clinical framework. A remote SP experience, where the encounter occurs via the Internet in comparison to traditional “face-to-face” encounters, could offer many advantages.

Why the Idea was Necessary: A pool of remote SPs could be potentially less expensive, and more available. Also, the remoteness may offer an opportunity for more variety in patients and the anonymity may offer other benefits. Although there is an abundance of experience with the utilization of SPs in a live context, the concept of utilizing remote SPs is novel. There are few standards to guide educators interested in utilizing SPs remotely.

What Needs to be Done: We seek to determine the parameters that would define a successful remote SP encounter. We seek to reach consensus regarding the implementation of a remote SP experience from leaders in medical education innovation.

Evaluation Plan: A modified Delphi approach will be used to gather feedback. After explaining the theory behind the “remote” SP encounter, we will then use iterative rounds of discussion to create the framework on which to build our “remote” SP experience. As an example, we will reach consensus on learner-patient interaction technology as follows: Round 1 will present options for modes of delivery (chat, video, discussion boards, or other technology) and elicit further suggestions. Based on responses, we will ask participants to rate the importance of the mode of delivery in Round 2. Discussions will continue until a general consensus is reached on the optimal mode of delivery of the proposed remote SP experience. Additional consensus will be sought in terms of how to integrate remote SPs into other educational approaches (such as TBL), the value of being able to engage a diverse population of SPs, and the value and potential of anonymity.

Future Development: Once the framework of the remote SP program is developed, we will create, integrate, and evaluate the use of remote SP to impart knowledge and clinical skills training in medical school education via our NIAAA funded project.

Clinical skills assessment exams using Standardized patients (SPs) are an established component of medical school education, including the current USLME Step 2 CS exam™. The authors have developed a novel method for conducting SP encounters using web-based chat technology (e.g. Google® chat). “Remote Live Standardized Patient” (RLSP) interviews are conducted entirely online. The interview requires active learner participation focused on clinical skills, and represents a unique solution to teaching specific skills and affecting behavior. We are not aware of other educators using chat for this purpose. We have been pilot testing chat-based OSCE-style exams1,2 since 2007. In 2009 we have begun using RLSPs to teach skills, as well as evaluate them. The instructional RLSP key components include an interactive SP actor, an EMR like interface, and a “Hats Off” mode, which allows the student learner to “ask a preceptor” or research information during the encounter. Each learner’s performance is evaluated by standardized measures and by the RLSP actor feedback. This interactive, responsive web environment allows the RLSP interview to simulate a broad spectrum of learner/patient encounters, from initial screening through diagnosis and treatment.

Advantages include 1) decreased SP training cost and time commitment, 2) more flexible SP scheduling and recruiting since all encounters occur online, 3) elimination of face-to-face constraints of verisimilitude errors and correlation of actor physical appearance compared to the “patient”, and 4) instant availability of chat conversation transcripts for review and learner feedback. Potential limitations include missing the “whole person” experience afforded by face-to-face encounters, including the ability to assess body language and demonstrate physical exam skills. Since face-to-face SP encounters also suffer from realism limitations (i.e. the actor does not typically have the “patient's” presenting symptoms), we feel that the advantages of an RLSP balance and actually outweigh the drawbacks.

Use of web-based chat or its equivalent may have significant appeal to the current generation of Internet-savvy medical students. The RLSP experience appears to be a viable alternative to the traditional face-to-face SP interview, allows cost savings, and prepares students for Step 2 CS-style exams.

H01 Education–Professional and Lay Online training of primary care providers in pain and addiction T. Tannera, M. Metcalfa, K. Rossiea and M. Coulehana aClinical Tools, Inc, Chapel Hill, NC (173). Available online 2 April 2009. The majority of primary care providers in the pain practice community have not fully incorporated evidence-based and clinical consensus recommendations for preventing, recognizing, and managing addiction in pain patients. Existing interventions based on conferring knowledge or providing prompts are limited in effect. With support from NIDA, we are designing an online training experience for pain management providers that focuses on skills training. We surveyed addiction specialists (n = 9) to better understand the needs and limitations of primary care providers. All addiction specialists identified a need for additional training in: 1) Detecting Substance Use Disorders in Pain Patients at Initial Evaluation, 2) Preventing Substance Use Disorders When Prescribing Opioids to Pain Patients, 3) Managing Addiction Issues in Ongoing Treatment of Chronic Pain Patients, and 4) Preventing and Detecting Diversion of Pain Medications [listed in the respondents' order of priority]. When asked about supplementary disease focused training in addiction issues, they highlighted back pain as the top priority followed by headache, osteoarthritis, and fibromyalgia/neuropathy [tied]. All addiction expert participants expressed interest in participating in features such as an online forum and an “Ask the Expert” forum. A survey of pain specialists (n = 5) ranked the four topics similarly and also identified back pain as the most important topic. A survey of primary care physicians (n = 9) revealed that all would consider taking a course with an Standardized Patient component; the most preferred modality was online chat. Most preferred a short SP experience that would more closely mimic the clinic encounter. With expert and target audience input, we have outlined a skills training experience based on the topics above and which uses virtual (i.e., Internet chat-based) Standardized Patients as well as interactive clinical cases with multiple questions. (This research is funded by NIDA/NIH contract #HHSN271200800012C.)