A Review of:
Hauser, Susan E., Dina Demner-Fushman, Joshua L. Jacobs, Susanne M. Humphrey, Glenn Ford, and George R. Thoma. “Using Wireless Handheld Computers to Seek Information at the Point of Care: An Evaluation by Clinicians.” Journal of the American Medical Informatics Association 14.6 (Nov./Dec. 2007): 807-15.
Abstract
Objective – To assess the effectiveness of wireless handheld computers (HHCs) for information retrieval in clinical environments and the role of MEDLINE in answering clinical questions at the point of care.
Design – A prospective single-cohort study.
Setting – Teaching rounds in the intensive care units and general medicine wards in two hospitals associated with a university’s school of medicine in the United States.
Subjects – Five internal medicine residents with training in evidence-based practice.
Methods – While accompanying medical teams on teaching rounds for approximately four consecutive weeks, each resident used MD on Tap (an application for handheld computers) on a TreoTM 650 PDA/cell phone to find answers in real time, to questions that were raised by members of the medical teams. Using a special version of MD on Tap, each resident initialized a UserID. Serving as evaluators, the residents described and categorized clinical scenarios and recognized questions. They also formulated search terms, searched MEDLINE and identified citations determined to be useful for answering the questions. An intermediate server collected details of all MEDLINE search query transactions, including system response time, the user (based on UserIDs), citations selected for viewing, the saving of citations to HHC memory, as well as use of the Linkout and Notes features. In addition evaluators submitted daily summaries. These summaries included information on the scenarios, clinical questions, evidence-based methodology (EBM) category, the team member who was the source of the question, the PubMed Identifiers (PMIDs) of relevant citations, and comments. At the end of the data collection period, each evaluator submitted a summary report consisting of a qualitative and quantitative evaluation of his experience using MEDLINE via the handheld device to find relevant evidence based information at the point of care. The report also focused on the usefulness of MD on Tap features, along with suggestions for additional features.
Data analysis encompassed matching the text of daily summaries to transaction records in order to identify sessions (containing a scenario, clinical question, one or more search queries, citation fetches and selected PMIDs). A senior medical librarian/expert indexer reviewed all the citations selected by evaluators and graded each citation as A (useful for answering the question), B (provided a partial answer) or C (not useful for answering the question). Only those graded A were regarded as “relevant.” For the purpose of analysis a session was deemed to be successful “if at least one of the citations selected by the evaluator as relevant was also classified as Relevant” (810) by the expert indexer. Similarly, an individual query was successful “if at least one of the citations among the results of the query was Relevant, that citation was viewed by the evaluator during rounds, and it addressed the clinical question as recorded in the daily summary” (810). Various types of relationships were analyzed including the characteristics of clinical questions vis-a-vis successful sessions, search strategies in relation to successful queries, and the association between MD on Tap features and successful queries. SAS/SUDAAN version 9.1 was used for statistical analysis.
Main Results – Evaluators answered 68% (246 of 363) clinical questions during rounding sessions. They identified 478 “relevant” citations, an average of 1.9 per successful session and 1.3 for each successful question. Session lengths averaged 3 minutes and 41 seconds. Characteristics of the evaluator (training, interest, experience and expertise) were a significant predictor of a session’s success. The significant determinants of query success were “the number of search terms that could be mapped to Medical Subject Headings (MeSH)” (812), the number of citations that were found for a query, and the use of MD on Tap’s auto-spellcheck feature. Narrative comments from the evaluators indicated that using MEDLINE on a HHC at the point of care contributed positively to the practice of evidence -based medicine.
Conclusion – Wireless handheld computers are useful for retrieving information in clinical environments. The application of several MeSH terms in a query facilitates the retrieval of MEDLINE citations that provide answers to clinical questions. The MD on Tap program is a valuable interface to MEDLINE at the point of care.
Preparing entry-level practitioners for evidence-based practice
