UpToDate is the Only Clinical Decision Support Resource Associated with Improved Outcomes

Over the years, more than 80 research studies have demonstrated that the widespread usage of UpToDate is associated with improved patient care and hospital performance.

In 2011, researchers at Harvard University published a compelling study confirming that the use of UpToDate over a three year period was associated with:

  • Improved quality of every condition on the Hospital Quality Alliance Metrics
  • Shorter lengths of stay (372,000 hospital days saved per year)
  • Lower mortality rates (saving 11,500 lives per three year period)

"The data suggests the use of computerized tools such as UpToDate enable better decisions, better outcomes and better care." — Ashish Jha, M.D., M.P.H., Harvard, and Study Author

View Research Studies by Category

Clinical knowledge declines over time

Systematic review: the relationship between clinical experience and quality of health care.

Choudhry NK, Fletcher RH, Soumerai SB. Systematic review: the relationship between clinical experience and quality of health care. Ann Intern Med. 2005; 142:260.

The study examined the belief that physicians with more years in practice have accumulated knowledge and skills that permit them to deliver higher-quality care. However, there was an inverse relationship between the number of years that a physician had been in practice and the quality of care that they provided. The authors concluded that:

  • Physicians who have been in practice longer may be at risk for providing lower-quality care. As a result, they may need quality improvement interventions.
Fletcher Abstract PDF

Clinical questions raised by clinicians at the point of care: a systematic review.

Del Fiol G, Workman TE, Gorman PN. Systematic review: the relationship between clinical experience and quality of health care. JAMA Intern Med. 2014 May;174(5):710-8. doi: 10.1001/jamainternmed.2014.368.

Clinicians frequently raise questions about patient care in their practice. Although they are effective at finding answers to questions they pursue, roughly half of the questions are never pursued. This picture has been fairly stable over time despite the broad availability of online evidence resources that can answer these questions. Technology-based solutions should enable clinicians to track their questions and provide just-in-time access to high-quality evidence in the context of patient care decision making. Opportunities for improvement include the recent adoption of electronic health record systems and maintenance of certification requirements.

  • In 11 studies, 7012 questions were elicited through short interviews with clinicians after each patient visit.
  • The mean frequency of questions raised was 0.57 (95% CI, 0.38-0.77) per patient seen, and clinicians pursued 51% (36%-66%) of questions and found answers to 78% (67%-88%) of those they pursued.
  • Overall, 34% of questions concerned drug treatment, and 24% concerned potential causes of a symptom, physical finding, or diagnostic test finding.
  • Clinicians' lack of time and doubt that a useful answer exists were the main barriers to information seeking.
Read More

Clinical questions are common but most are not answered

Information needs in office practice: are they being met?

Covell, DG, Uman, GC, Manning, PR, Information Needs in Office Practice: Are They Being Met? Ann Intern Med 1985;103(4):596-9.

The study examined the self-reported needs of 47 physicians during a half day of typical office practice.

  • Physicians self-report that they need answers, on average, once per week.
  • Physicians have about 2 questions for every 3 patients seen.
  • Only 30% of clinical questions were answered, usually by another physician or other health care professional.
Covell Abstract PDF

Information seeking in primary care: how physicians choose which clinical questions to pursue and which to leave unanswered.

Gorman PN, Helfand M. Information seeking in primary care: how physicians choose which clinical questions to pursue and which to leave unanswered. Med Decis Making. 1995 Apr-Jun;15(2):113-9.
  • Primary care physicians have many clinical questions while they are seeing patients, but they pursue only about 30% of their questions.
  • Two primary factors determined which questions were pursued: the physician's belief that a definitive answer existed, and the urgency of the patient's problem.
Click here to read more

Information in practice: analysis of questions asked by family doctors regarding patient care.

Ely JW, Osheroff JA, Ebell MH, Bergus GR, Levy BT, Chambliss ML, Evans ER. Analysis of questions asked by family doctors regarding patient care. BMJ. 1999 Aug 7;319(7206):358-61.
  • Family doctors frequently had questions about patient care (about 3 questions per 10 patients or 8 questions per day) but pursued answers to only 40 percent of them. Of those pursued, 80% were answered.
  • An answer was pursued when the problem was perceived as urgent and when a definitive answer was thought to exist.
Click here to read more

Residents' medical information needs in clinic: are they being met?

Green ML, Ciampi MA, Ellis PJ. Residents' medical information needs in clinic: are they being met? Am J Med. 2000 Aug 15;109(3):218-23.
  • Residents identified 2 questions for every 3 patients they saw and pursued answers to 29% of the questions.
  • The primary reason for not pursuing answers was lack of time (60%).
Read More

Answering physicians' clinical questions: obstacles and potential solutions.

Ely JW, Osheroff JA, Chambliss ML, Ebell MH, Rosenbaum ME. Answering physicians' clinical questions: obstacles and potential solutions. J Am Med Inform Assoc. 2005 Mar-Apr;12(2):217-24.

A study of generalist physicians in Iowa investigated the obstacles preventing physicians from answering their patient care questions. Physicians asked 5.5 questions per half-day but pursued answers to just 55% of them. Of the 55% of questions pursued, UpToDate was used the most (41%) out of over 10 information resources consulted during the study. Other resources on the list included Epocrates (25%), MICROMEDEX (15%), and the Sanford Guide to Antimicrobial Therapy (14%).

  • The typical primary care physician has approximately 11 questions a day. While they pursue answers to 55% of questions, only 40% get answered.
  • Answering all questions would change up to 5 management decisions per day.
  • The most common reason for not pursuing an answer was that they doubted that an answer existed.
Ely Abstract PDF

UpToDate answers clinical questions more effectively than other resources

Variable Quality and Readability of Patient-Oriented Websites on Colorectal Cancer Screening

Schreuders EH, Grobbee EJ, Kuipers EJ, Spaander MC, Veldhuyzen van Zanten SJ. Variable Quality and Readability of Patient-Oriented Websites on Colorectal Cancer Screening. Clin Gastroenterol Hepatol. 2016 Jul 9. pii: S1542-3565(16)30376-7. doi: 10.1016/j.cgh.2016.06.029. [Epub ahead of print].

The efficacy of colorectal cancer (CRC) screening is dependent on participation and subsequent adherence to surveillance. The internet is increasingly used for health information and important to support decision-making. We evaluated the accuracy, quality, and readability of online information on CRC screening and surveillance.

Twenty websites were included. The mean Website Accuracy Score was 26 out of 44 (range 9-41). Websites with the highest scores were www.cancer.org, www.bowelcanceraustralia.org and www.uptodate.com.

There is marked variation in quality and readability of websites on CRC screening. Most websites do not address polyp surveillance. The poor correlation between quality and Google ranking suggests that screenees will miss out on high-quality websites using standard search strategies.

Click here to read more

Medical students' use of information resources: Is the digital age dawning?

Peterson MW, Rowat J, Kreiter C, Mandel J. Medical students' use of information resources: is the digital age dawning? Acad Med. 2004 79(1):89-95.

Investigators at the University of Iowa monitored the use of UpToDate by second year students as they transitioned to their clinical years. Students rapidly adopted UpToDate. By the end of their third year:

  • More than 85% of medical students identified electronic sources as their primary resource for medical information (UpToDate 53%, MDConsult 33%). UpToDate was preferred significantly more often than other resources.
  • They used electronic information resources daily and required less than 15 minutes to answer most of their clinical questions.
Peterson Abstract PDF

Multiprogram evaluation of reading habits of primary care internal medicine residents on ambulatory rotations.

Lai CJ, Aagaard E, Brandenburg S, Nadkarni M, Wei HG, Baron R., Multiprogram evaluation of reading habits of primary care internal medicine residents on ambulatory rotations. J Gen Intern Med. 2006 May;21(5):486-9.

Of online sources, 98% of residents reported using UpToDate regularly, 44% used literature search, and 35% used Google.com or other search engines.

Click here to read more

The reading habits of medicine clerks at one medical school: frequency, usefulness, and difficulties.

Leff B, Harper GM, The reading habits of medicine clerks at one medical school: frequency, usefulness, and difficulties. Acad Med. 2006 May;81(5):489-94.

Among medical students at Johns Hopkins, UpToDate was the most commonly used reading source and was rated as the most useful.

Click here to read more

An evaluation of five bedside information products using a user-centered, task-oriented approach.

Campbell R, Ash J. An evaluation of five bedside information products using a user-centered, task-oriented approach. J Med Libr Assoc 2006 Oct; 94(4):435-41, e206-7.
  • Participants successfully answered more clinical questions with UpToDate than with any of the other resource (P < 0.0001).
  • In response to the question, "Overall, did this database satisfy your needs?," UpToDate ranked significantly higher than all other resources (P = 0.006). UpToDate also ranked significantly higher on ease of use (P < 0.0001).
  • Most users (76%) ranked UpToDate to be the best product while none rated it the worst.
Campbell Abstract PDF

Speed, accuracy, and confidence in Google, Ovid, PubMed, and UpToDate: results of a randomised trial.

Robert H Thiele, Nathan C Poiro, David C Scalzo, et al. Postgrad Med J 2010 86: 459-465 doi: 10.1136/pgmj.2010.098053
  • Google was the most popular search tool.
  • Users of Google and UpToDate were more likely than users of PubMed to answer questions correctly.
  • Subjects had the most confidence in UpToDate.
  • Searches with Google and UpToDate were faster than searches with PubMed or Ovid.
Click here to read the PDF

To Compare PubMed Clinical Queries and UpToDate in Teaching Information Mastery to Clinical Residents: A Crossover Randomized Controlled Trial

Sayyah Ensan L, Faghankhani M, Javanbakht A, Ahmadi SF, Baradaran HR. Center for Educational Research in Medical Sciences, Medical Education and Development Center, Tehran University of Medical Sciences, Tehran, Iran PLoS One. 2011;6(8):e23487. Epub 2011 Aug 12.
  • Based on intention-to-treat analysis, participants retrieved the answer of 67 (76%) questions using UpToDate and 38 (43%) questions using PubMed Clinical Queries (P<0.001).
  • The median time to answer retrieval was significantly faster with UpToDate.
  • The satisfaction with the accuracy of retrieved answers, interaction with UpToDate and also overall satisfaction were higher among UpToDate users compared to PubMed Clinical Queries users (P<0.001).
Click here to read more

A comparison of answer retrieval through four evidence-based textbooks (ACP PIER, Essential Evidence Plus, First Consult, and UpToDate): A randomized controlled trial.

Ahmadi SF, Faghankhani M, Javanbakht A, Akbarshahi M, Mirghorbani M, Safarnejad B, Baradaran H. Tehran University of Medical Sciences, Iran Med Teach. 2011;33(9):724-30.
  • The rate of answer retrieval was 86% in UpToDate, 69% in First Consult, 49% in ACP PIER, and 45% in Essential Evidence Plus (p<0.001).
  • The mean time to answer retrieval was significantly faster with UpToDate.
  • UpToDate seems more comprehensive in content and also faster than the other three evidence-based textbooks. Thus, it may be considered as one of the best sources for answering clinicians' questions at the point of care.
Click here to read more

Utility of the electronic information resource UpToDate for clinical decision-making at bedside rounds.

Phua J, MBBS, MRCP, See KC, MBBS, MRCP, Khalizah HJ, MBChB Manc, MRCP, Low SP, MBBS, MRCP, Lim TK, MBBS, FRCPE. Utility of the electronic information resource UpToDate for clinical decision-making at bedside rounds. Singapore Med J 2012; 53(2): 116-120.
  • Each search took a median of three minutes (first quartile: two minutes, third quartile: five minutes).
  • UpToDate led to a change in investigations, diagnosis or management 37% of the time
  • Conclusion: "Incorporating UpToDate searches into daily bedside rounds was feasible and useful in clinical decision-making."
Click here to read more

Survey of user preferences from a comparative trial of UpToDate and ClinicalKey

Michael R. Kronenfeld, MLS, MBA, AHIP, R. Curtis Bay, PhD, William Coombs, MA. Survey of user preferences from a comparative trial of UpToDate and ClinicalKey. J Med Libr Assoc. 2013 April; 101(2): 151–154.

The Journal of the Medical Library Association (JMLA) recently published, and released to the general public, a study conducted by the A. T. Still University of Health on a side-by-side comparison of UpToDate and ClinicalKey. The survey study was completed by 23 faculty members and 292 students from two osteopathic schools of medicine and a physician assistant program.

  • UpToDate scored higher than ClinicalKey on overall satisfaction, particularly by those in clinical rotation or teaching in the clinic who strongly preferred UpToDate.
  • UpToDate was strongly favored for ease of use, efficiency in answering specific questions, organization and depth.
  • UpToDate was the preferred choice by a large margin if the library could only purchase one product.
  • Respondents reported that Clinical key was valued for its library of full texts by those in the didactic or classroom phase of training, but that its use as a point-of-care tool was limited.

Study authors conclude that UpToDate was preferred in the clinical setting. They further concluded that "not only was it heavily used in the clinical setting," but that respondents reported students "were at a competitive disadvantage if they did not have access to it."

The authors noted that ClinicalKey, in advance marketing "implied its use as a point-of-care product for physicians." However, the authors conclude that ClinicalKey's "use as a point-of-care product was limited by the fact that the survey respondents did not find First Consult, which represents the major point-of-care content in ClinicalKey, to be at all comparable to UpToDate."

Click here to read more

The value of library and information services in patient care: results of a multisite study

Marshall JG, Sollenberger J, Easterby-Gannett S, Morgan LK, Klem ML, Cavanaugh SK, Oliver KB, Thompson CA, Romanosky N, Hunter S. The value of library and information services in patient care: Results of a multi-site study. Journal of the Medical Library Association 2013 Jan; 101(1):39-46.

A large-scale, multi-site study on the value and impact of library and information services on patient care was conducted by the National Network of Libraries of Medicine, MidAtlantic region and researchers at the University of North Carolina at Chapel Hill. The respondents were 16,122 clinicians at 56 library sites serving 118 hospitals in urban and rural areas.

Survey respondents were asked to report on a recent incident in which they had sought information for patient care. The study found:

  • UpToDate was the most used CDS tool by physicians and residents, used significantly more than any other CDS tool. Use of UpToDate followed closely behind only research studies
  • Clinicians reported that patient mortality was avoided in 6% of patient care incidents
  • Clinical decisions changed as the result of consulting resources: diagnoses (25%), choice of drugs (33%), other treatment (31%), ordered tests (23%) and patient advice (48%)
  • Clinicians reported that adverse events were avoided including, misdiagnosis, patient mortality, adverse drug reactions, medication errors and ordering of unneeded tests
  • 85% of respondents reported that the information saved them an average of 2.5 hours per incident

Used with permission from the Medical Library Association

Click here to read more

Answering physicians' clinical questions: obstacles and potential solutions.

Ely JW, Osheroff JA, Chambliss ML, Ebell MH, Rosenbaum ME. Answering physicians' clinical questions: obstacles and potential solutions. J Am Med Inform Assoc. 2005 Mar-Apr;12(2):217-24.

A study of generalist physicians in Iowa investigated the obstacles preventing physicians from answering their patient care questions. Physicians asked 5.5 questions per half-day but pursued answers to just 55% of them. Of the 55% of questions pursued, UpToDate was used the most (41%) out of over 10 information resources consulted during the study. Other resources on the list included Epocrates (25%), MICROMEDEX (15%), and the Sanford Guide to Antimicrobial Therapy (14%).

  • The typical primary care physician has approximately 11 questions a day. While they pursue answers to 55% of questions, only 40% get answered.
  • Answering all questions would change up to 5 management decisions per day.
  • The most common reason for not pursuing an answer was that they doubted that an answer existed.
Ely Abstract PDF

UpToDate improves patient care

Effectiveness of a clinical knowledge support system for reducing diagnostic errors in outpatient care in Japan: A retrospective study

Shimizu T., Nemoto T., Tokuda Y. Effectiveness of a clinical knowledge support system for reducing diagnostic errors in outpatient care in Japan: A retrospective study. International Journal of Medical Informatics 109 (2018): 1-4. doi: 10.1016/j.ijmedinf.2017.09.010

To evaluate the effectiveness of UpToDate for reducing diagnostic errors, the study authors retrospectively identified 100 patients who visited an outpatient department in a community-based hospital from July 2014 to June 2015. Half the patients were seen by physicians equipped with UpToDate and half were seen by physicians without UpToDate access. They compared diagnostic error rates between the two groups.

  • The diagnostic error rate for the patients seen by physicians equipped with UpToDate was 2%.
  • The diagnostic error rate for patients seen by physicians not equipped with UpToDate was 24%.
  • Multivariate logistic regression analysis showed that error rate reduction was significantly associated with exposure to UpToDate with an odds ratio of 15.21 (95% CI 1.86–124.36).
  • The results demonstrated the effectiveness of UpToDate for the prevention and reduction of diagnostic error.
Click here to read more

Variable Quality and Readability of Patient-Oriented Websites on Colorectal Cancer Screening

Schreuders EH, Grobbee EJ, Kuipers EJ, Spaander MC, Veldhuyzen van Zanten SJ. Variable Quality and Readability of Patient-Oriented Websites on Colorectal Cancer Screening. Clin Gastroenterol Hepatol. 2016 Jul 9. pii: S1542-3565(16)30376-7. doi: 10.1016/j.cgh.2016.06.029. [Epub ahead of print].

The efficacy of colorectal cancer (CRC) screening is dependent on participation and subsequent adherence to surveillance. The internet is increasingly used for health information and important to support decision-making. We evaluated the accuracy, quality, and readability of online information on CRC screening and surveillance.

Twenty websites were included. The mean Website Accuracy Score was 26 out of 44 (range 9-41). Websites with the highest scores were www.cancer.org, www.bowelcanceraustralia.org and www.uptodate.com.

There is marked variation in quality and readability of websites on CRC screening. Most websites do not address polyp surveillance. The poor correlation between quality and Google ranking suggests that screenees will miss out on high-quality websites using standard search strategies.

Click here to read more

Can an electronic database help busy physicians answer clinical questions?

Blackman D, Cifu A, Levinson W. Can an electronic database help busy physicians answer clinical questions? J Gen Intern med 2002; 17Suppl1:220.

A pilot controlled trial of the impact of UpToDate was conducted at the University of Chicago among 10 physicians in four primary care clinics. Physicians were randomly assigned to use their usual information resources with or without UpToDate. Clinic sessions were monitored by an investigator and data was collected on 678 patient visits over 5 weeks.

  • Among physicians who had access to UpToDate, 89% of questions were answered; 78% of answers changed patient care. UpToDate users answered 79% more of their questions than did physicians who did not have access and 75% more of these answers led to a change in clinical decision making.
Blackman Abstract PDF

Usage of UpToDate at an academic medical center.

Maviglia, SM, Martin, MT, Wang, SJ, et al. Usage of UpToDate at an academic medical center. J Gen Inter Med 2002; 17(Suppl1):204.

An online survey of clinicians at the Massachusetts General Hospital and Brigham and Women's Hospital described the following effects among UpToDate users:

  • 95% reported that UpToDate was integral for making decisions
  • 94% reported that they had changed diagnosis
  • 95% reported that UpToDate led to a change in patient management
  • 97% said UpToDate helps them provide the best care for their patients
  • 90% reported that UpToDate makes them a better doctor
  • 96% reported made them more comfortable with their decisions
Atlanta Abstracts PDF

The impact of evidence on physicians' inpatient treatment decisions.

Lucas BP, Evans AT, Reilly BM, Khodakov YV, Perumal K, Rohr LG, Akamah JA, Alausa TM, Smith CA, Smith JP. The impact of evidence on physicians' inpatient treatment decisions. J Gen Intern Med. 2004 May;19(5 Pt 1):402-9.

The authors examined the impact of evidence-based knowledge (provided mainly from UpToDate) on attending physicians' treatment decisions about hospitalized patients.Before being provided information, most attending physicians believed that they had made an evidence-based choice. After reading the information:

  • Treatment changed in 18% of patients.
  • Most changed decisions were considered to have improved the care of the patient.
Click here to read more

Association of a clinical knowledge support system with improved patient safety, reduced complications and shorter length of stay among Medicare beneficiaries in acute care hospitals in the United States.

Bonis PA, Pickens GT, Rind DM, Foster DA. Association of a clinical knowledge support system with improved patient safety, reduced complications and shorter length of stay among Medicare beneficiaries in acute care hospitals in the United States. Int J Med Inform. 2008 Nov;77(11):745-53.

The authors examined the impact of evidence-based knowledge (provided mainly from UpToDate) on attending physicians' treatment decisions about hospitalized patients.Before being provided information, most attending physicians believed that they had made an evidence-based choice. After reading the information:

  • Demonstrated strong association between hospital quality and efficiency and use of UpToDate.
  • Hospitals with access to UpToDate performed significantly better on risk-adjusted measures of patient safety and complications and had significantly shorter length of stay (by an average 0.167 days per discharge) compared with hospitals without access. These benefits correlated with how frequently UpToDate was used.
Solucient Study Abstract PDF

How doctors make use of online, point-of-care clinical decision support systems: a case study of UpToDate

Addison J, Whitcombe J, Glover SW. How doctors make use of online, point-of-care clinical decision support systems: a case study of UpToDate. Health Information & Libraries Journal, 2012 30, pp. 13–22

Through an online questionnaire, investigators surveyed doctors at healthcare organizations in North West England which subscribe to UpToDate. Respondents were asked to describe a scenario in which they had used UpToDate, and to identify benefits, if any, associated with that scenario.

More than 90% of the 239 respondents who had used UpToDate identified at least one benefit:

  • 57% reported using UpToDate reduced treatment delays
  • 52% reported using UpToDate meant they avoided unnecessary diagnostic tests
  • 48% reported UpToDate reduced delays in diagnosis
  • 39% reported UpToDate changed their treatment decision
  • 28% reported UpToDate reduced the time to discharge
Click here to read more

An evaluation of five bedside information products using a user-centered, task-oriented approach.

Campbell R, Ash J. An evaluation of five bedside information products using a user-centered, task-oriented approach. J Med Libr Assoc 2006 Oct; 94(4):435-41, e206-7.
  • Participants successfully answered more clinical questions with UpToDate than with any of the other resource (P < 0.0001).
  • In response to the question, "Overall, did this database satisfy your needs?," UpToDate ranked significantly higher than all other resources (P = 0.006). UpToDate also ranked significantly higher on ease of use (P < 0.0001).
  • Most users (76%) ranked UpToDate to be the best product while none rated it the worst.
Campbell Abstract PDF

The value of library and information services in patient care: results of a multisite study

Marshall JG, Sollenberger J, Easterby-Gannett S, Morgan LK, Klem ML, Cavanaugh SK, Oliver KB, Thompson CA, Romanosky N, Hunter S. The value of library and information services in patient care: Results of a multi-site study. Journal of the Medical Library Association 2013 Jan; 101(1):39-46.

A large-scale, multi-site study on the value and impact of library and information services on patient care was conducted by the National Network of Libraries of Medicine, MidAtlantic region and researchers at the University of North Carolina at Chapel Hill. The respondents were 16,122 clinicians at 56 library sites serving 118 hospitals in urban and rural areas.

Survey respondents were asked to report on a recent incident in which they had sought information for patient care. The study found:

  • UpToDate was the most used CDS tool by physicians and residents, used significantly more than any other CDS tool. Use of UpToDate followed closely behind only research studies
  • Clinicians reported that patient mortality was avoided in 6% of patient care incidents
  • Clinical decisions changed as the result of consulting resources: diagnoses (25%), choice of drugs (33%), other treatment (31%), ordered tests (23%) and patient advice (48%)
  • Clinicians reported that adverse events were avoided including, misdiagnosis, patient mortality, adverse drug reactions, medication errors and ordering of unneeded tests
  • 85% of respondents reported that the information saved them an average of 2.5 hours per incident

Used with permission from the Medical Library Association

Click here to read more

UpToDate has an important role in medical education

Knowledge seeking behaviors of pre interns and early career doctors in Sri Lanka: a cross sectional study.

Chaturaka Rodrigo, Sachith Maduranga, Milinda Withana, Deepika Fernando and Senaka Rajapakse. BMC Research Notes 2015, 8:610 doi:10.1186/s13104-015-1600-3.
  • UpToDate is one of the most frequently used online resources to obtain general medical information.
  • Early career doctors prefer UpToDate.
Click here to read more

Factors associated with medical knowledge acquisition during internal medicine residency.

McDonald FS, Zeger SL, Kolars JC. Factors associated with medical knowledge acquisition during internal medicine residency. J Gen Intern Med. 2007 Apr 28.

A study of internal medicine residents at the Mayo Clinic examined the impact of learning habits such as conference attendance and use of an electronic knowledge resource (UpToDate) on medical knowledge acquisition as measured by the Internal Medicine In-training Examination (IM-ITE). The IM-ITE assesses the medical knowledge of internal medicine residents during their three-year training program; scores increase with each year of residency reflecting the acquisition of medical knowledge during residency. Performance on the examination correlates with subsequent performance on the American Board of Internal Medicine Certification Examination. The authors correlated how much UpToDate was used by each resident (based upon usage logs) with IM-ITE scores. Using UpToDate for 20 minutes a day was associated with a comparable increase in IM-ITE scores as an entire year of residency. The model was fully adjusted and considered known covariates associated with performance on the IM-ITE, suggesting that use of UpToDate was an independent predictor of performance. A similar degree of benefit was also detected for regular conference attendance. The authors also cited a survey of 18,000 residents in which UpToDate was the most commonly used clinical resource for clinical information. These data (as well as previous studies involving UpToDate) suggest that it has an important role in medical education and in acquisition of medical knowledge.

  • Conference attendance and self-directed reading of an electronic knowledge resource had statistically and educationally significant independent associations with knowledge acquisition that were comparable to the benefit of a year in residency training.
Click here to read more

How residents and interns utilize and perceive the personal digital assistant and UpToDate.

Phua J, Lim TK. How residents and interns utilize and perceive the personal digital assistant and UpToDate. BMC Medical Education 2008, 8:39. doi:10.1186/1472-6920-8-39.

A survey study at National University Hospital in Singapore found that after 5 months of use 60% of respondents reported that UpToDate lead to a change in patient management and 95% would recommend UpToDate to a colleague.

Click here to read more

A multi-institutional survey of internal medicine residents' learning habits.

Edson RS, Beckman TJ, West CP, Aronowitz PB, Badgett RG, Feldstein DA, Henderson MC, Kolars JC, McDonald FS. A multi-institutional survey of internal medicine residents' learning habits. Med Teach. 2010;32(9):773-5.

95% reported UpToDate was the most effective resource for learning. 90% reported that UpToDate was their first choice for answering clinical questions.

Click here to read more

Resource utilization patterns of third-year medical students.

Cooper AL, Elnicki DM. Resource utilization patterns of third-year medical students. Clin Teach. 2011 Mar;8(1):43-7.

Most students used UpToDate to prepare for attending physician rounds and to admit patients (64% and 67%, respectively), but not for exam preparation.

Click here to read more

Relationship of Electronic Medical Knowledge Resource Use and Practice Characteristics with Internal Medicine Maintenance of Certification Examination Scores.

Darcy A. Reed, MD, MPH, Colin P. West, MD, PhD, Eric S. Holmboe, MD, Andrew J. Halvorsen, MS, Rebecca S. Lipner, PhD, Carola Jacobs, BA, and Furman S. McDonald, MD, MPH. Relationship of Electronic Medical Knowledge Resource Use and Practice Characteristics with Internal Medicine Maintenance of Certification Examination Scores. J Gen Intern Med. 2012 Feb.

Maintenance of certification (MOC) examination performance is associated with quality of care. A study at the Mayo Clinic aimed to examine relationships between electronic medical knowledge resource use, practice characteristics and examination scores among physicians recertifying in internal medicine.

  • Use of UpToDate was associated with improved scores on Internal Medicine Maintenance of Certification Examinations (IM-MOCE).
Click here to read the epub abstract

Impact of inpatient caseload, emergency department duties, and online learning resource on General Medicine In-Training Examination scores in Japan

Kinoshita K, Tsugawa Y, Shimizu T, Tanoue Y, Konishi R, Nishizaki Y, Shiojiri T, Tokuda Y. Impact of inpatient caseload, emergency department duties, and online learning resource on General Medicine In-Training Examination scores in Japan. International Journal of General Medicine 2015:8.

The authors conducted a survey with a clinical knowledge evaluation involving postgraduate year (PGY)-1 and -2 resident physicians at teaching hospitals offering 2-year postgraduate training programs required for residents in Japan, using the General Medicine In-Training Examination (GM-ITE).

  • Residents with four or five times per month of the emergency department (ED) duties exhibited the highest mean scores compared to those with greater or fewer ED duties.
  • Those with largest number of inpatients in charge exhibited the highest mean scores compared to the residents with fewer inpatients in charge.
  • Hospitals with the greater UpToDate topic viewing showed significantly greater mean score.

Appropriate ED workload and inpatient caseload, as well as use of evidence-based electronic resources, were associated with greater clinical knowledge of residents.

Click here to read more

The reading habits of medicine clerks at one medical school: frequency, usefulness, and difficulties.

Leff B, Harper GM, The reading habits of medicine clerks at one medical school: frequency, usefulness, and difficulties. Acad Med. 2006 May;81(5):489-94.

Among medical students at Johns Hopkins, UpToDate was the most commonly used reading source and was rated as the most useful.

Click here to read more

UpToDate use provides quantifiable ROI

The Total Economic Impact of UpToDate – A Case Study Based on The Experience of a Public and a Private Hospital in Brazil.

Forrester Consulting. A Forrester Total Economic Impact™ Study Prepared for Wolters Kluwer Health. December 2016.

To evaluate the potential financial impact of UpToDate on organizations, Forrester interviewed representatives from a public and a private (nonprofit) hospital in Brazil, both with multiple years of experience using UpToDate.

The analysis was based on a hospital in Brazil with 500 beds, approximately 4,000 employees (including 800 physicians), with 24,000 admissions, 570,000 outpatient visits per year and an annual budget of approximately R$800 million.

UpToDate Has Potential to Improve the Quality of Care While Reducing Costs

All interviewed physicians felt that the widespread use of UpToDate improved the quality of care. They noted several potential impact areas, including physician productivity gains, avoidance of unnecessary referrals, reduced length of stay, avoidance of unnecessary diagnostic tests, and increased physician satisfaction.

Conservatively estimating the impact of physician productivity gains and avoidance of unnecessary referrals for a composite hospital, Forrester’s analysis points to quantifiable benefits of R$1.1 million versus total costs of R$270,000, adding up to a net present value (NPV) of approximately R$816,000 over the three years of the analysis (Figure 1).

Figure 1

Click here to read the full study

The Total Economic Impact of UpToDate's Clinical Decision Support System for Healthcare Institutions: A Case Study of Salford Royal NHS Foundation Trust.

Forrester Consulting. A Forrester Total Economic Impact™ Study Prepared for UpToDate. November 2011.

In May 2011, Forrester Consulting examined the total economic impact and potential return on investment (ROI) UK hospitals may realize by deploying UpToDate's clinical decision support solution.

This study illustrates the financial impact of using UpToDate clinical decision support within the Salford Royal NHS Foundation Trust (SRFT), a large teaching hospital near Manchester, UK. SRFT employs more than 5,000 staff members treating some 400,000 patients annually and has subscribed to the UpToDate service for seven years.

An analysis of UpToDate use at SRFT and subsequent financial analysis found that SRFT's use of UpToDate resulted in significant improvements in the quality of care, staff productivity, and diagnostic test efficiencies. SRFT experienced quantifiable ROI, covering all the costs of the service within 90 days.

UpToDate: One-Year, Risk-Adjusted Benefits, Costs, and ROI Summary
ROI Payback period Total benefits Total costs
402% < 3 months £123,958 (£24,678)
Source: Forrester Research, Inc.
Click here to read the full study

UpToDate saves time

Can an electronic database help busy physicians answer clinical questions?

Blackman D, Cifu A, Levinson W. Can an electronic database help busy physicians answer clinical questions? J Gen Intern med 2002; 17Suppl1:220.

A pilot controlled trial of the impact of UpToDate was conducted at the University of Chicago among 10 physicians in four primary care clinics. Physicians were randomly assigned to use their usual information resources with or without UpToDate. Clinic sessions were monitored by an investigator and data was collected on 678 patient visits over 5 weeks.

  • Among physicians who had access to UpToDate, 89% of questions were answered; 78% of answers changed patient care. UpToDate users answered 79% more of their questions than did physicians who did not have access and 75% more of these answers led to a change in clinical decision making.
Blackman Abstract PDF

UpToDate is used more than other electronic clinical resources

Residents' patient-specific clinical questions: opportunities for evidence-based learning.

Schilling LM, Steiner JF, Lundahl K, Anderson RJ. Residents' patient-specific clinical questions: opportunities for evidence-based learning. Acad Med. 2005 Jan;80(1):51-6.

A study at the University of Colorado examined how answering patient-specific clinical questions affected residents' patient care decisions. Internal medicine residents were asked to formulate a clinical question and use evidence-based learning guidelines to answer them. Results from the study show that:

  • UpToDate was the most frequently used information source by residents.
Schilling Abstract PDF

Randomized trial for answers to clinical questions: Evaluating a pre-appraised versus a MEDLINE search tool.

Patel MR, Schardt CM, Sanders LL, Keitz SA. Randomized trial for answers to clinical questions: evaluating a pre-appraised versus a MEDLINE search protocol. J Med Libr Assoc. 2006 Oct;94(4):382-7.

A study at the Department of Medicine at Duke University Medical Center concluded that medical libraries need to provide both MEDLINE and pre-appraised resources such as UpToDate for answers to the largest proportion of clinical questions.

Click here to read more

Evidence-Based Library and Information Practice

Farrell A, Evidence Based Library and Information Practice 2008, 3:2

A survey study at the Memorial University of Newfoundland, Canada, found that UpToDate was able to provide information for the greatest number of clinical questions compared with other evidence-based information tools evaluated.

  • UpToDate was rated easy to use and informative.
Click here to read more

Answers to Questions Posed During Daily Patient Care Are More Likely to Be Answered by UpToDate Than PubMed.

Hoogendam A, Stalenhoef AFH, de Vries Robbé PF, Overbeke AJ, Answers to Questions Posed During Daily Patient Care Are More Likely to Be Answered by UpToDate Than PubMed. J Med Internet Res 2008;10(4):e29.

An observational study from the Netherlands evaluated 2986 patient-related questions asked by 70 physicians. Available information resources included PubMed, UpToDate, Harrison's Online and a drug database. The authors concluded that "Based on our data, there is no reason to start searching on a lower level of the evidence-based pyramid for any major medical topic but it is sensible to use UpToDate as the primary information resource."

  • UpToDate was used most often (78% of questions), and questions were significantly more likely to be answered by UpToDate than PubMed regardless of the topic of the search.
Click here to read more

Searching for medical information online: a survey of Canadian nephrologists.

Shariff SZ, Bejaimal SA, Sontrop JM, Iansavichus AV, Weir MA, Haynes RB, Speechley MR, Thind A, Garg AX. J Nephrol. 2011 Nov-Dec;24(6):723-32. doi: 10.5301/JN.2011.6373.

In this cross-sectional study (2008 to 2010), a random sample of Canadian nephrologists completed a survey of their online search practices. Respondents were queried on their searching preferences, practices, and use of 9 online information sources.

  • Nephrologists used a variety of online sources to retrieve information on patient treatment including UpToDate (92%), PubMed (89%), Google (76%) and Ovid MEDLINE (55%).
  • Community-based nephrologists were more likely to consult UpToDate first (91%), while academic nephrologists were divided between UpToDate (58%) and PubMed (41%).
  • When searching bibliographic resources such as PubMed, 80% of nephrologists scan a maximum of 40 citations (the equivalent of 2 search pages in PubMed). Searching practices did not differ by age, sex or years in practice.
Click here to read more

Psychiatry Residents’ Use of Educational Websites: A Pilot Survey Study

Torous J, Franzan J, O’Connor R, Mathew I, Keshavan M, Kitts R, Boland R. Psychiatry Residents’ Use of Educational Websites: A Pilot Survey Study. Acad Psychiatry 2015. DOI 10.1007/s40596-015-0335-8.

Results indicate that psychiatry residents at the study site rely more on online educational resources than traditional printed resources such as textbooks. Residents reported using online resources in nearly all aspects of training, ranging from reading at home to working in the emergency department, inpatient unit, outpatient clinic, and consult service. UpToDate, PubMed, and Wikipedia were the three most utilized resources by residents and reflect the varied nature of the resources available to residents today.

UpToDate, a secondary source, was the most utilized resource, and residents gave it the highest rating for trustworthiness of recommendations. PubMed, a primary source, received the highest rating as source of personal learning but a lower ranking as a source of clinical decision-making in comparison to UpToDate.

Click here to read more

Temporal trends in accessing online medical information

Lott JP, Roy B, Venkatesh AK (2014), Temporal trends in accessing online medical information. J. Hosp. Med., 9: 525–526. doi: 10.1002/jhm.2211.

Since January 2004, relative search interest associated with UpToDate has increased steadily, whereas web-based queries for other sources of online medical information have declined (Figure 1).

Relative search interest in UpToDate has, on average, exceeded that of JAMA, NEJM, and PDR since approximately July 2011 (Figure 1), whereas PubMed has been associated with the greatest, albeit diminishing, relative search interest.

Linear regression yielded the following significant (P < 0.001) coefficients of trend for UpToDate (coefficient = 0.010), JAMA (coefficient = −0.012), NEJM (coefficient = −0.030), PDR (coefficient = −0.020), and PubMed (coefficient = −0.011). Every coefficient differed significantly from each other (P < 0.001).

Figure 1:

Google Search trends for online medical information. Temporal trends in relative Google search query interest by online medical resource, 2004 to 2013. Abbreviations: JAMA, Journal of the American Medical Association; NEJM, New England Journal of Medicine; PDR, Physicians' Desk Reference.

Click here to read more

Using Clinician's Search Query Data to Monitor Influenza Epidemics

Marshall JG, Sollenberger J, Easterby-Gannett S, Morgan LK, Klem ML, Cavanaugh SK, Oliver KB, Thompson CA, Romanosky N, Hunter S. Using clinician's search query data to monitor influenza epidemics. Clin Infect Dis. (2014) 59 (10): 1446-1450 first published online August 12, 2014 doi:10.1093/cid/ciu647.

Search query information from a clinician's database, UpToDate, is shown to timely predict influenza epidemics in the United States.

Study results show that digital disease surveillance tools based on experts' databases may be able to provide an alternative, reliable, and stable signal for accurate predictions of flu outbreaks.

Click here to read more

Medical students' use of information resources: Is the digital age dawning?

Peterson MW, Rowat J, Kreiter C, Mandel J. Medical students' use of information resources: is the digital age dawning? Acad Med. 2004 79(1):89-95.

Investigators at the University of Iowa monitored the use of UpToDate by second year students as they transitioned to their clinical years. Students rapidly adopted UpToDate. By the end of their third year:

  • More than 85% of medical students identified electronic sources as their primary resource for medical information (UpToDate 53%, MDConsult 33%). UpToDate was preferred significantly more often than other resources.
  • They used electronic information resources daily and required less than 15 minutes to answer most of their clinical questions.
Peterson Abstract PDF

Multiprogram evaluation of reading habits of primary care internal medicine residents on ambulatory rotations.

Lai CJ, Aagaard E, Brandenburg S, Nadkarni M, Wei HG, Baron R., Multiprogram evaluation of reading habits of primary care internal medicine residents on ambulatory rotations. J Gen Intern Med. 2006 May;21(5):486-9.

Of online sources, 98% of residents reported using UpToDate regularly, 44% used literature search, and 35% used Google.com or other search engines.

Click here to read more

The reading habits of medicine clerks at one medical school: frequency, usefulness, and difficulties.

Leff B, Harper GM, The reading habits of medicine clerks at one medical school: frequency, usefulness, and difficulties. Acad Med. 2006 May;81(5):489-94.

Among medical students at Johns Hopkins, UpToDate was the most commonly used reading source and was rated as the most useful.

Click here to read more

An evaluation of five bedside information products using a user-centered, task-oriented approach.

Campbell R, Ash J. An evaluation of five bedside information products using a user-centered, task-oriented approach. J Med Libr Assoc 2006 Oct; 94(4):435-41, e206-7.
  • Participants successfully answered more clinical questions with UpToDate than with any of the other resource (P < 0.0001).
  • In response to the question, "Overall, did this database satisfy your needs?," UpToDate ranked significantly higher than all other resources (P = 0.006). UpToDate also ranked significantly higher on ease of use (P < 0.0001).
  • Most users (76%) ranked UpToDate to be the best product while none rated it the worst.
Campbell Abstract PDF

Answering physicians' clinical questions: obstacles and potential solutions.

Ely JW, Osheroff JA, Chambliss ML, Ebell MH, Rosenbaum ME. Answering physicians' clinical questions: obstacles and potential solutions. J Am Med Inform Assoc. 2005 Mar-Apr;12(2):217-24.

A study of generalist physicians in Iowa investigated the obstacles preventing physicians from answering their patient care questions. Physicians asked 5.5 questions per half-day but pursued answers to just 55% of them. Of the 55% of questions pursued, UpToDate was used the most (41%) out of over 10 information resources consulted during the study. Other resources on the list included Epocrates (25%), MICROMEDEX (15%), and the Sanford Guide to Antimicrobial Therapy (14%).

  • The typical primary care physician has approximately 11 questions a day. While they pursue answers to 55% of questions, only 40% get answered.
  • Answering all questions would change up to 5 management decisions per day.
  • The most common reason for not pursuing an answer was that they doubted that an answer existed.
Ely Abstract PDF