Getting to the information clinicians need quickly: Pharmacist evaluation of DynaMed and micromedex with watson (DynaMedex) using real-world questions

Introduction

Digital point-of-care information resources are frequently used by clinicians to answer clinical questions. An evidence-based disease management database (DynaMed) was merged with a pharmaceutical knowledge base (Micromedex). We evaluated the ability of the combined solution, DynaMedex, to answer clinical questions.

Methods

Real-world questions were used for testing and were categorized by information type and specialty area. Two pharmacists independently performed 600 searches for 300 questions, using keyword search and Watson Assistant (WA). Search results were evaluated based on whether information was found (yes, no), relevance to the question (relevant, not relevant), difficulty in finding the answer (easy, medium, hard), and quality of the evidence (good, fair, poor).

Results

An answer was found 86.3% of the time using keyword search and 81.0% of the time using WA. In keyword searches, 86.0% of answers were considered relevant and 74.5% in WA. Most answers were easy to find (78.7% in keyword search, 94.4% in WA). The quality of evidence for answers was good, fair, or poor in 62.7%, 36.4%, and 0.9% for keyword search and 50.3%, 47.8%, and 1.9% for WA.

Conclusion

Pharmacists found answers to most clinical questions easily with good quality, evidence-based information and a high agreement rate. This resource could be further improved by recognizing different search terms, standardizing the location of drug and disease information in appropriate sections, providing citations, if available, with the highest quality evidence, and including access to content types that haven’t been incorporated.