Skip to main navigation Skip to search Skip to main content

Leveraging electronic health records for data science: common pitfalls and how to avoid them

  • National Tsing Hua University
  • Microsoft Research Asia
  • Massachusetts Institute of Technology

Research output: Contribution to journalReview articleAcademicpeer-review

3 Downloads (Pure)

Abstract

Analysis of electronic health records (EHRs) is an increasingly common approach for studying real-world patient data. Use of routinely collected data offers several advantages compared with other study designs, including reduced administrative costs, the ability to update analysis as practice patterns evolve, and larger sample sizes. Methodologically, EHR analysis is subject to distinct challenges because data are not collected for research purposes. In this Viewpoint, we elaborate on the importance of in-depth knowledge of clinical workflows and describe six potential pitfalls to be avoided when working with EHR data, drawing on examples from the literature and our experience. We propose solutions for prevention or mitigation of factors associated with each of these six pitfalls-sample selection bias, imprecise variable definitions, limitations to deployment, variable measurement frequency, subjective treatment allocation, and model overfitting. Ultimately, we hope that this Viewpoint will guide researchers to further improve the methodological robustness of EHR analysis.

Original languageEnglish
Pages (from-to)e893-e898
JournalThe Lancet Digital Health
Volume4
Issue number12
Early online date22 Sept 2022
DOIs
Publication statusPublished - 1 Dec 2022

Fingerprint

Dive into the research topics of 'Leveraging electronic health records for data science: common pitfalls and how to avoid them'. Together they form a unique fingerprint.

Cite this