Background: With support from NIDA (#R44-DA12066) we have developed an online curriculum to educate and further clinical skill development in Year 2 to 4 medical students on buprenorphine treatment for opioid use disorders.

Objective: To educate medical students about treatment of patients with opioid dependence using buprenorphine and impact clinical skills in working with patients with addictive disorders.

Program Description: We are performing a summative evaluation of our curriculum. Each medical student completed a pre/post test effectiveness evaluation of our online curriculum, including pre/post evaluations of clinical skills through a remote live standardized patient (SP) experience. The OSCE style evaluation of the learner's performance during the remote, live SP clinical encounter was assessed by the SP actor and a MA level clinical professional. Measurement was done via: 1) a 9 item Likert scale rating sheet, and 2) a 16 item done/not done checklist. Both modifications were reviewed and approved by two external clinical substance abuse professionals.

Program Evaluation Outcome: Preliminary results (n=25) of this evaluation indicate that the modules have a statistically significantly effect on knowledge and intended behavior related to treatment of patients with opioid dependence (p<.001). Average pre-knowledge score was 68% while the average post-test score was 83% (p<0.001). Attitude and self-efficacy also increased significantly for most modules. Students showed the most improvement in the areas of common comorbidities and pharmacology. Student competency was measured in the following areas: drug detection and screening, comorbid conditions, diagnosis and treatment. Competency (p<0.0007) increased quite significantly from pre to post experience. Competency scores increased an average of 44% from pre to post-interview.

Conclusions: Based on the preliminary results, this curriculum is an effective tool for educating students on buprenorphine and opioid use disorder. The addition of a standardized patient component pre and post curriculum allows the students an opportunity to better demonstrate an increase in competencies and clinical skill in a more true to life situation.

Idea: Medical educators interested in online learning have confidence in their ability to effect knowledge and the tools exist to demonstrate success. Yet, online learning is limited by the inability to show that the learning experience effects higher levels in Bloom's Taxonomy. How do we understand if an online course is succeeding beyond its knowledge improvement goal?

Why the idea was necessary: Students took 5 online courses on the topic of addiction that were broken down into Prevalence, Detection and Diagnosis, Comorbidities, PCP Role, and Pharmacology. The courses demonstrated significant changes in knowledge in a very small sample (N=10) with p ranging from .033 to .001 and average knowledge scores improving from 14% to 52%. Self-efficacy too showed statistically significant improvement (p<.001) even with the very small sample. The students rated themselves as being able to effectively screen and detect opioid abuse following the learning experience. There was not enough of a difference in scores to detect a difference in attitudes and intended behavior. Unfortunately the standardized patient interview showed no change in clinical skills.

What was done: This project utilized a remote standardized patient (SP) experience to test, examine, and improve the learning. We planned to use the ratings of the SP experience to identify the weaknesses in student performance and modify the online learning to address them. After the first round of students completed the study, we looked at the data from both the Modified Skills Inventory (a subjective rating tool completed only by the standardized patient) and the Evaluator Rating Sheet (an objective checklist completed by both the standardized patient and an observer).

Evaluation: Results from the Modified Skills Inventory showed general improvement for all participants post-experience. However, the Evaluator Rating Sheet had some problematic areas; clinical areas which were expected to improve post-experience did not improve. Specifically, students consistently failed to screen the standardized patient using a standardized measure, such as the CAGE-AID. Mental status exams were also rarely conducted, and students showed difficulty in screening patients for psychiatric co-morbid conditions. We evaluated our existing curriculum, and found weaknesses in the level of detail paid to these areas. Pages covering these content areas were expanded to further illustrate these teaching points.

Conclusions: A remote SP experience is a useful tool in the development of an online educational intervention. It can ensure that the student is truly mastering skills and integrating learning into their clinical experience framework.

Introduction: Recent years have seen dramatic advances in the field of medical genetics and a corresponding growth in the development of genetic testing technologies. The National Human Genome Research Institute (NHGRI) and the National Coalition for Health Professional Education in Genetics (NCHPEG) have communicated a great need for physician education about the Ethical, Legal, and Social Implications of genetic testing and counseling. Physicians and health care providers are not sufficiently trained to make decisions about genetic tests, interpret the tests, or counsel patients about the results.

Methods: With funding from NHGRI (R25-HG02266) and NCI (5R44-CA086720), we created a curriculum of web-based modules for medical students on genetic testing and counseling. We strove to prepare future physicians for the common and complex genetic issues they will encounter. A novel curriculum plan incorporated the genetics core competency guidelines from NCHPEG, the American Society of Human Genetics (ASHG), and the Association of American Medical Colleges (AAMC) with input from faculty, medical genetics instructors and instructional design experts. Five “Case Studies” of an hour’s length incorporated “think ahead” and “test yourself” interactive features, graphics, and one to two minute videos of patients. Additional modules reviewed the basics of genetics, clinical genetics, and cancer genetics. The online modules included, hyperlinks to supplemental material, high quality graphics and complete references. Medical school faculty chose the modules best suited to enhance their specific curriculum needs. We also developed and pilot-tested the use of “virtual” simulated patient assessment conducted via Internet based chat both with and without video.

Results: The evaluation reviewed which topic areas were in most demand by medical school faculty, and the overall effectiveness of the modules. Modules with case studies and clear clinical applications were well received by students and faculty alike; whereas more research focused modules were determined to be less relevant to medical students. Effectiveness was assessed using pre/post tests of knowledge, and retrospective surveys of attitude, self-efficacy, and intended behavior. Data from 897 medical students from 8 schools showed a statistically significant effect on knowledge, attitude, intended behavior and self-efficacy related to genetic testing. Students and faculty expressed high levels of satisfaction with the content, relevance, and usability of the course modules. We also implemented the virtual standardized patient assessment protocol and found that students had the necessary hardware and software to complete text as well as audio/video chat sessions. Importantly, they expressed interest in this type of assessment and a desire to use it in future coursework.

Discussion: Web-based supplemental training can be successfully integrated into traditional medical school educational programs to enhance student learning in a specialized medical topic such as genetics. Assessments can be built into the learning experience and performed automatically without placing an additional burden on the institution. Web-based training affects a range of outcomes including knowledge, attitude, and behavior and yields high satisfaction. Future evaluations can include standardized patient assessments via text and audio/video chat and thus assess the impact of web-based training on interpersonal skills development.

Clinical skills assessment exams using Standardized patients (SPs) are an established component of medical school education, including the current USLME Step 2 CS exam™. The authors have developed a novel method for conducting SP encounters using web-based chat technology (e.g. Google® chat). “Remote Live Standardized Patient” (RLSP) interviews are conducted entirely online. The interview requires active learner participation focused on clinical skills, and represents a unique solution to teaching specific skills and affecting behavior. We are not aware of other educators using chat for this purpose. We have been pilot testing chat-based OSCE-style exams1,2 since 2007. In 2009 we have begun using RLSPs to teach skills, as well as evaluate them. The instructional RLSP key components include an interactive SP actor, an EMR like interface, and a “Hats Off” mode, which allows the student learner to “ask a preceptor” or research information during the encounter. Each learner’s performance is evaluated by standardized measures and by the RLSP actor feedback. This interactive, responsive web environment allows the RLSP interview to simulate a broad spectrum of learner/patient encounters, from initial screening through diagnosis and treatment.

Advantages include 1) decreased SP training cost and time commitment, 2) more flexible SP scheduling and recruiting since all encounters occur online, 3) elimination of face-to-face constraints of verisimilitude errors and correlation of actor physical appearance compared to the “patient”, and 4) instant availability of chat conversation transcripts for review and learner feedback. Potential limitations include missing the “whole person” experience afforded by face-to-face encounters, including the ability to assess body language and demonstrate physical exam skills. Since face-to-face SP encounters also suffer from realism limitations (i.e. the actor does not typically have the “patient's” presenting symptoms), we feel that the advantages of an RLSP balance and actually outweigh the drawbacks.

Use of web-based chat or its equivalent may have significant appeal to the current generation of Internet-savvy medical students. The RLSP experience appears to be a viable alternative to the traditional face-to-face SP interview, allows cost savings, and prepares students for Step 2 CS-style exams.

Medical students need to understand the role of practicing physicians in substance abuse treatment. In addition they may not be aware of complicated multi-disciplinary treatment issues, the biological and social basis of addictions, and how new treatments are developed then moved into practice. The topic of detection and treatment of opioid addiction is an excellent one to teach these basic skills and convey essential knowledge. By teaching medical students we can impact future physicians before potential negative attitudes regarding substance dependence have developed or solidified.

With research funding from NIDA/NIH, we are developing outlines for a suite of online educational modules addressing these issues. Curriculum development included medical school professionals and substance abuse experts. The prototype module, "Prevalence, Trends and Impact on Health: Opioid Addiction" was pilot tested with 22 medical students in their 2nd through 4th years of study. Pre/post-assessments of knowledge, attitudes, self-efficacy, and intended behavior showed significant increases following completion of the module. Satisfaction with the module was high and there was an interest in learning more about substance abuse. During the coming year (2008), we will perform a large-scale evaluation of the project using an innovative virtual standardized patient. As the need for physicians to recognize and treat addiction disorders grows, these online modules hold promise for integrating the necessary knowledge, skills, and attitudes into the medical school curriculum.

Opioid dependence and treatment with buprenorphine are topics well suited to teach medical students about the role of practicing physicians in substance abuse treatment, complicated multi-disciplinary treatment issues, complex biological and social basis of addictions, and how new treatments are developed and moved into practice. During Phase I of the grant, we developed a curriculum plan for a suite of modules on these issues as they relate to opioid abuse and dependence with input from a focus group of medical school professionals with experience in substance abuse education. The prototype module entitled “Prevalence, Trends and Impact on Health: Opioid Addiction” was developed and underwent usability and pilot testing with medical students in their 2nd through 4th years of study. Usability testing revealed that students enjoyed the online format, found the module well-organized and easy to navigate, and showed an interest in learning more about Buprenorphine. More in-depth pilot testing assessed the impact of the pilot module on the attitudes, self-efficacy, and intendedbehavior of medical students, as well as changes in knowledge and satisfaction with the module material. A retrospective pre/post design was regarding detecting and treating opioid misuse or abuse significantly increased with the completion of the pilot module.