TY - JOUR
T1 - Initial interactions with the FDA on developing a validation dataset as a medical device development tool
AU - Hart, Steven
AU - Garcia, Victor
AU - Dudgeon, Sarah N.
AU - Hanna, Matthew G.
AU - Li, Xiaoxian
AU - Blenman, Kim R.M.
AU - Elfer, Katherine
AU - Ly, Amy
AU - Salgado, Roberto
AU - Saltz, Joel
AU - Gupta, Rajarsi
AU - Hytopoulos, Evangelos
AU - Larsimont, Denis
AU - Lennerz, Jochen
AU - Gallas, Brandon D.
N1 - Publisher Copyright:
© 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland. This article has been contributed to by U.S. Government employees and their work is in the public domain in the USA.
PY - 2023/12
Y1 - 2023/12
N2 - Quantifying tumor-infiltrating lymphocytes (TILs) in breast cancer tumors is a challenging task for pathologists. With the advent of whole slide imaging that digitizes glass slides, it is possible to apply computational models to quantify TILs for pathologists. Development of computational models requires significant time, expertise, consensus, and investment. To reduce this burden, we are preparing a dataset for developers to validate their models and a proposal to the Medical Device Development Tool (MDDT) program in the Center for Devices and Radiological Health of the U.S. Food and Drug Administration (FDA). If the FDA qualifies the dataset for its submitted context of use, model developers can use it in a regulatory submission within the qualified context of use without additional documentation. Our dataset aims at reducing the regulatory burden placed on developers of models that estimate the density of TILs and will allow head-to-head comparison of multiple computational models on the same data. In this paper, we discuss the MDDT preparation and submission process, including the feedback we received from our initial interactions with the FDA and propose how a qualified MDDT validation dataset could be a mechanism for open, fair, and consistent measures of computational model performance. Our experiences will help the community understand what the FDA considers relevant and appropriate (from the perspective of the submitter), at the early stages of the MDDT submission process, for validating stromal TIL density estimation models and other potential computational models.
AB - Quantifying tumor-infiltrating lymphocytes (TILs) in breast cancer tumors is a challenging task for pathologists. With the advent of whole slide imaging that digitizes glass slides, it is possible to apply computational models to quantify TILs for pathologists. Development of computational models requires significant time, expertise, consensus, and investment. To reduce this burden, we are preparing a dataset for developers to validate their models and a proposal to the Medical Device Development Tool (MDDT) program in the Center for Devices and Radiological Health of the U.S. Food and Drug Administration (FDA). If the FDA qualifies the dataset for its submitted context of use, model developers can use it in a regulatory submission within the qualified context of use without additional documentation. Our dataset aims at reducing the regulatory burden placed on developers of models that estimate the density of TILs and will allow head-to-head comparison of multiple computational models on the same data. In this paper, we discuss the MDDT preparation and submission process, including the feedback we received from our initial interactions with the FDA and propose how a qualified MDDT validation dataset could be a mechanism for open, fair, and consistent measures of computational model performance. Our experiences will help the community understand what the FDA considers relevant and appropriate (from the perspective of the submitter), at the early stages of the MDDT submission process, for validating stromal TIL density estimation models and other potential computational models.
KW - artificial intelligence
KW - computational pathology
KW - machine learning
KW - medical device development tool
KW - model validation
KW - regulatory science
KW - triple-negative breast cancer
KW - tumor-infiltrating lymphocytes
UR - http://www.scopus.com/inward/record.url?scp=85173498095&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85173498095&partnerID=8YFLogxK
U2 - 10.1002/path.6208
DO - 10.1002/path.6208
M3 - Article
C2 - 37794720
AN - SCOPUS:85173498095
SN - 0022-3417
VL - 261
SP - 378
EP - 384
JO - Journal of Pathology
JF - Journal of Pathology
IS - 4
ER -