Evaluation of Healthprompt for Zero-shot Clinical Text Classification

This paper presents an evaluation of the Healthprompt, a prompt-based zero-shot clinical text classification framework. The lack of publicly available datasets and the expensive data annotation in the clinical domain make traditional NLP models difficult to train. To overcome this issue, Healthprompt utilizes Pre-trained Language Models (PLMs) and prompt-based Zero-Shot Learning (ZSL) to perform clinical NLP tasks without additional training data. However, the original Healthprompt paper missed the error analysis and ablation study of the framework. This paper conducts an error analysis and an ablation study on Electronic Health Records(EHR) notes to understand the capabilities and limitations of the Healthprompt framework for clinical text classification. The results provide insight into the potential and limitations of prompt-based zeroshot learning for clinical NLP tasks and offer suggestions for improvements to the Healthprompt framework, and for the future development of prompt-based ZSL.