Development of a machine learning algorithm based on administrative claims data for identification of ED anaphylaxis patient visits

Background: Epidemiologic studies of anaphylaxis commonly rely on International Classification of Diseases (ICD) codes to identify anaphylaxis cases, which may lead to suboptimal epidemiologic classification. Objective: We sought to develop and assess the accuracy of a machine learning algorithm using ICD codes and other administrative data compared with ICD code–only algorithms to identify emergency department (ED) anaphylaxis visits. Methods: We conducted a retrospective review of ED visits from January 2013 to September 2017. Potential ED anaphylaxis visits were identified using 3 methods: anaphylaxis ICD diagnostic codes (method 1), ICD symptom-based codes with or without a code indicating an allergic trigger (method 2), and ICD codes indicating a potential allergic reaction only (method 3). A machine learning algorithm was developed from administrative data, and test characteristics were compared with ICD code–only algorithms. Results: A total of 699 of 2191 (31.9%) potential ED anaphylaxis visits were classified as anaphylaxis. The sensitivity and specificity of method 1 were 49.1% and 87.5%, respectively. Method 1 used in combination with method 2 resulted in a sensitivity of 53.9% and a specificity of 68.7%. Method 1 used in combination with method 3 resulted in a sensitivity of 98.4% and a specificity of 15.1%. The sensitivity and specificity of the machine learning algorithm were 87.3% and 79.1%, respectively. Conclusions: ICD coding alone demonstrated poor sensitivity in identifying cases of anaphylaxis, with venom-related anaphylaxis missing 96% of cases. The machine learning algorithm resulted in a better balance of sensitivity and specificity and improves upon previous strategies to identify ED anaphylaxis visits.