A review of auditing methods applied to the content of controlled biomedical terminologies

Xinxin Zhu, Jung Wei Fan, David M. Baorto, Chunhua Weng, James J. Cimino

Research output: Contribution to journalArticlepeer-review

71 Scopus citations


Although controlled biomedical terminologies have been with us for centuries, it is only in the last couple of decades that close attention has been paid to the quality of these terminologies. The result of this attention has been the development of auditing methods that apply formal methods to assessing whether terminologies are complete and accurate. We have performed an extensive literature review to identify published descriptions of these methods and have created a framework for characterizing them. The framework considers manual, systematic and heuristic methods that use knowledge (within or external to the terminology) to measure quality factors of different aspects of the terminology content (terms, semantic classification, and semantic relationships). The quality factors examined included concept orientation, consistency, non-redundancy, soundness and comprehensive coverage. We reviewed 130 studies that were retrieved based on keyword search on publications in PubMed, and present our assessment of how they fit into our framework. We also identify which terminologies have been audited with the methods and provide examples to illustrate each part of the framework.

Original languageEnglish (US)
Pages (from-to)413-425
Number of pages13
JournalJournal of Biomedical Informatics
Issue number3
StatePublished - Jun 2009


  • Auditing method
  • Automated
  • Heuristic
  • Knowledge source
  • Manual
  • Quality factor
  • Systematic
  • Terminology

ASJC Scopus subject areas

  • Computer Science Applications
  • Health Informatics


Dive into the research topics of 'A review of auditing methods applied to the content of controlled biomedical terminologies'. Together they form a unique fingerprint.

Cite this