Methodological Research

Develop methods that earn trust

Observational data can reveal treatment use, outcomes, safety, effectiveness, and patient risk in routine care. But differences in populations, measurement, treatment selection, and study design can also produce misleading results. OHDSI researchers therefore develop methods—and empirically evaluate how those methods perform—rather than relying on methodological convention alone.

Three broad kinds of questions

  • Characterization: Characterization asks what is happening in a population by describing disease natural history, treatment patterns, outcomes, and incidence.
  • Population-level estimation: Population-level estimation compares outcomes between treatments or exposures under explicit causal assumptions.
The OHDSI open-source ecosystem turns shared standards and methodological practices into a coordinated workflow for study design, orchestration, execution, evaluation, and communication.
  • Patient-level prediction: Patient-level prediction estimates the probability that a future outcome will occur for an individual with particular characteristics.

Diagnostics before conclusions

OHDSI methods emphasize prespecified study designs and transparent diagnostics. Depending on the question, researchers may use active comparators, large-scale propensity scores, negative controls, empirical calibration, phenotype evaluation, covariate-balance checks, and replication across databases. These diagnostics expose assumptions and potential sources of error. If a result fails prespecified checks, it should not be treated as reliable evidence.

The goal is not to declare observational data trustworthy by default. It is to evaluate whether a particular study is fit for interpretation by making its assumptions, diagnostics, and uncertainty visible. Confidence increases when findings remain consistent across prespecified analyses, data sources, and populations—and decreases when those checks reveal important limitations.

Learn more

Useful Publications

  1. Schuemie MJ, Ryan PB, DuMouchel W, et al. Interpreting observational studies: why empirical calibration is needed to correct p-values. Statistics in Medicine. 2014;33:209–218. https://doi.org/10.1002/sim.5925 8
  2. Schuemie MJ, Hripcsak G, Ryan PB, et al. Empirical confidence interval calibration for population-level effect estimation studies in observational healthcare data. PNAS. 2018;115:2571–2577. https://doi.org/10.1073/pnas.1708282114 8
  3. Tian Y, Schuemie MJ, Suchard MA. Evaluating large-scale propensity score performance through real-world and synthetic data experiments. International Journal of Epidemiology. 2018;47:2005–2014. https://doi.org/10.1093/ije/dyy120 8
  4. Reps JM, Schuemie MJ, Suchard MA, et al. Design and implementation of a standardized framework to generate and evaluate patient-level prediction models using observational healthcare data. JAMIA. 2018;25:969–975. https://doi.org/10.1093/jamia/ocy032 14
  5. Schuemie MJ, Ryan PB, Pratt N, et al. Principles of Large-scale Evidence Generation and Evaluation across a Network of Databases (LEGEND). JAMIA. 2020;27:1331–1337. https://doi.org/10.1093/jamia/ocaa103 1
  6. Schuemie MJ, Cepeda MS, Suchard MA, et al. How confident are we about observational findings in health care: a benchmark study. Harvard Data Science Review. 2020;2. https://doi.org/10.1162/99608f92.147cc28e 8