Explainability and causability for artificial intelligence-supported medical image analysis in the context of the European In Vitro Diagnostic Regulation
Heimo Müller, Andreas Holzinger, Markus Plass et al. · New Biotechnology · 2022
Artificial Intelligence (AI) for the biomedical domain is gaining significant interest and holds considerable potential for the future of healthcare, particularly also in the context of in vitro diagnostics. The European In Vitro Diagnostic Medical Device Regulation (IVDR) explicitly includes software in its requirements. This poses major challenges for In Vitro Diagnostic devices (IVDs) that involve Machine Learning (ML) algorithms for data analysis and decision support. This can increase the difficulty of applying some of the most successful ML and Deep Learning (DL) methods to the biomedical domain, just by missing the required explanatory components from the manufacturers. In this context, trustworthy AI has to empower biomedical professionals to take responsibility for their decision-making, which clearly raises the need for explainable AI methods. Explainable AI, such as layer-wise relevance propagation, can help in highlighting the relevant parts of inputs to, and representations in, a neural network that caused a result and visualize these relevant parts. In the same way that usability encompasses measurements for the quality of use, the concept of causability encompasses measurements for the quality of explanations produced by explainable AI methods. This paper describes both concepts and gives examples of how explainability and causability are essential in order to demonstrate scientific validity as well as analytical and clinical performance for future AI-based IVDs.
Cited by 87 works. doi:10.1016/j.nbt.2022.05.002 Open-access PDF
Authors
References
- On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
- A New Initiative on Precision Medicine
- Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices
- Causability and explainability of artificial intelligence in medicine
- The state of artificial intelligence-based FDA-approved medical devices and algorithms: an online database
- Usability engineering methods for software developers
- Development of a Companion Diagnostic PD-L1 Immunohistochemistry Assay for Pembrolizumab Therapy in Non–Small-cell Lung Cancer
- Artificial intelligence in digital pathology — new tools for diagnosis and precision oncology
- Unmasking Clever Hans predictors and assessing what machines really learn
- Measuring the Quality of Explanations: The System Causability Scale (SCS)