Clinical documentation variations and NLP system portability: A case study in asthma birth cohorts across institutions

Sunghwan Sohn, Yanshan Wang, Chung Il Wi, Elizabeth A. Krusemark, Euijung Ryu, Mir H. Ali, Young J. Juhn, Hongfang Liu

Research output: Contribution to journalArticlepeer-review

17 Scopus citations


Objective: To assess clinical documentation variations across health care institutions using different electronic medical record systems and investigate how they affect natural language processing (NLP) system portability. Materials and Methods: Birth cohorts from Mayo Clinic and Sanford Children's Hospital (SCH) were used in this study (n=298 for each). Documentation variations regarding asthma between the 2 cohorts were examined in various aspects: (1) overall corpus at the word level (ie, lexical variation), (2) topics and asthma-related concepts (ie, semantic variation), and (3) clinical note types (ie, process variation). We compared those statistics and explored NLP system portability for asthma ascertainment in 2 stages: prototype and refinement. Results: There exist notable lexical variations (word-level similarity=0.669) and process variations (differences in major note types containing asthma-related concepts). However, semantic-level corpora were relatively homogeneous (topic similarity=0.944, asthma-related concept similarity=0.971). The NLP system for asthma ascertainment had an F-score of 0.937 at Mayo, and produced 0.813 (prototype) and 0.908 (refinement) when applied at SCH. Discussion: The criteria for asthma ascertainment are largely dependent on asthma-related concepts. Therefore, we believe that semantic similarity is important to estimate NLP system portability. As the Mayo Clinic and SCH corpora were relatively homogeneous at a semantic level, the NLP system, developed at Mayo Clinic, was imported to SCH successfully with proper adjustments to deal with the intrinsic corpus heterogeneity.

Original languageEnglish (US)
Pages (from-to)353-359
Number of pages7
JournalJournal of the American Medical Informatics Association
Issue number3
StatePublished - Mar 1 2018


  • Asthma
  • Documentation variation
  • Electronic medical records
  • Natural language processing
  • Portability

ASJC Scopus subject areas

  • Health Informatics


Dive into the research topics of 'Clinical documentation variations and NLP system portability: A case study in asthma birth cohorts across institutions'. Together they form a unique fingerprint.

Cite this