An efficient strategy allowed English-speaking reviewers to identify foreign-language articles eligible for a systematic review

Jason W. Busse, Paul Bruno, Keshena Malik, Gaelan Connell, David Torrance, Trung Ngo, Karin Kirmayr, Daniel Avrahami, John J. Riva, Shanil Ebrahim, Peter A.A. Struijs, David Brunarski, Stephen J. Burnie, Frances Leblanc, Eric A. Coomes, Ivan A. Steenstra, Tesha Slack, Robert Rodine, Janey Jim, Victor M. MontoriGordon H. Guyatt

Research output: Contribution to journalArticlepeer-review

13 Scopus citations


Objective To assess English-speaking reviewers' accuracy in determining the eligibility of foreign-language articles for a systematic review. Study Design and Settings Systematic review of randomized controlled trials of therapy for fibromyalgia. Guided by 10 questions, English-speaking reviewers screened non-English-language articles for eligibility. Teams of two native-language speakers provided reference standard judgments of eligibility. Results Of 15,466 potentially eligible articles, we retrieved 763 in full text, of which 133 were published in 19 non-English languages; 53 trials published in 11 languages other than English proved eligible. Of the 53 eligible articles, English-language reviewers guided by the 10 questions mistakenly judged 6 as ineligible; of the 80 ineligible articles, 8 were incorrectly judged eligible by English-language reviewers (sensitivity = 0.89; specificity = 0.90). Use of a simple three-step rule (excluding languages with less than three articles, reviewing titles and abstracts for clear indications of eligibility, and noting the lack of a clearly reported statistical analysis unless the word "random" appears) led to accurate classification of 51 of 53 articles (sensitivity = 0.96; specificity = 0.70). Conclusion Our findings show promise for limiting the need for non-English-language review teams in systematic reviews with large numbers of potentially eligible non-English-language articles.

Original languageEnglish (US)
Pages (from-to)547-553
Number of pages7
JournalJournal of Clinical Epidemiology
Issue number5
StatePublished - May 2014


  • Bias
  • English-language bias
  • Fibromyalgia
  • Language of publication
  • Methodology
  • Systematic review

ASJC Scopus subject areas

  • Epidemiology


Dive into the research topics of 'An efficient strategy allowed English-speaking reviewers to identify foreign-language articles eligible for a systematic review'. Together they form a unique fingerprint.

Cite this