Abstract
The black box nature of current AI models has raised serious concerns about accountability, bias and trust in the models that might undermine their relevance and usefulness in the field of medicine where human lives are at risk. AI in medicine has the ability to derive meaningful inferences from real world data - an emerging school of thought namely Real World Evidence (RWE) studies - that can assist medical practitioners to improve evidence based quality of care. In the field of oncology, the accuracy and performance of inference models are as important as clinically relevant and sound explanations of the inference. In this paper, we present an Explainable AI (XAI) framework for our AI model that predicts the suitability of a chemotherapy treatment at the time of its prescription based on RWE. The framework provides explanations both for a specific patient and also for the model. It provides explanations like feature analysis, counterfactual, and top risk factors that contribute to a treatment failure. As a result, the framework adds an explainability layer between treatment failure predictive model and oncologists, thereby enabling evidence based assistance to oncologists in designing chemotherapy plans.
Key findings
- 1Black-box AI is a trust problem in medicine: concerns about accountability, bias and trust can undermine an AI model's usefulness where human lives are at risk.
- 2In oncology, the explanation counts as much as the accuracy: clinically relevant, sound explanations of an inference matter as much as the model's accuracy and performance.
- 3Built for a real treatment failure model: the framework explains CureMD's AI model that predicts, from real-world evidence, whether a chemotherapy treatment is suitable at the time it is prescribed.
- 4Explains the patient and the model: it gives explanations for a specific patient and for the model as a whole.
- 5Three kinds of explanation: feature analysis, counterfactuals, and the top risk factors that contribute to a treatment failure.
- 6A layer between model and oncologist: the result is an explainability layer that gives oncologists evidence-based assistance in designing chemotherapy plans.
Why this matters
An oncologist will not change a regimen because a model says so. A treatment failure prediction becomes usable when it shows its reasons: which factors raise this patient's risk, what would need to be different for the prediction to change, and which risk factors weigh most. Explanations at the model level let clinicians check what the model relies on across all patients against what they know clinically.
This chapter adds that layer to the treatment failure model CureMD Research published at AIME 2023 and in the Journal of Clinical Oncology the same year, and cites the AIME paper as reference 3. The framework assists the oncologist in designing the chemotherapy plan. The decision stays with the oncologist.
Keywords
- Explainability
- Machine Learning
- Model Agnostic
- Oncology
Editors and affiliations
Jose M. Juarez
University of Murcia, Murcia, Spain
Carlos Fernandez-Llatas
Universitat Politècnica de València, Valencia, Spain
Concha Bielza
Universidad Politécnica de Madrid, Boadilla del Monte, Madrid, Spain
Owen Johnson
University of Leeds, Leeds, UK
Primoz Kocbek
University of Maribor, Maribor, Slovenia
Pedro Larrañaga
Universidad Politécnica de Madrid, Boadilla del Monte, Spain
Niels Martin
Hasselt University, Diepenbeek, Belgium
Jorge Munoz-Gama
Pontificia Universidad Católica de Chile, Macul, Chile
Gregor Štiglic
University of Maribor, Maribor, Slovenia
Marcos Sepulveda
Pontificia Universidad Católica de Chile, Santiago, Chile
Alfredo Vellido
Universitat Politècnica de Catalunya, Barcelona, Spain
References
- 1.O’Brien, M.E.R., et al.: Mortality within 30 days of chemotherapy: a clinical governance benchmarking issue for oncology patients. Br. J. Cancer 95(12), 1632 to 1636 (2006). doi:10.1038/sj.bjc.6603498
- 2.Mohseni, S., Zarei, N., Ragan, E.D.: A multidisciplinary survey and framework for design and evaluation of explainable AI systems. ACM Trans. Interact. Intell. Syst. (TiiS) 11(3 to 4), 1 to 45 (2021).
- 3.Shahid, M.U., Farooq, M.: Boosted random forests for predicting treatment failure of chemotherapy regimens. In: Juarez, J.M., Marcos, M., Stiglic, G., Tucker, A. (eds.) AIME 2023. LNCS, vol. 13897, pp. 13 to 24. Springer, Cham (2023). doi:10.1007/978-3-031-34344-5_2
- 4.Lundberg, S.M., Lee, S.-I.: A unified approach to interpreting model predictions. In: Advances in Neural Information Processing Systems, vol. 30 (2017).
- 5.Mothilal, R.K., Sharma, A., Tan, C.: Explaining machine learning classifiers through diverse counterfactual explanations. In: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pp. 607 to 617 (2020).
- 6.Altmann, A., Toloşi, L., Sander, O., Lengauer, T.: Permutation importance: a corrected feature importance measure. Bioinformatics 26(10), 1340 to 1347 (2010). doi:10.1093/bioinformatics/btq134
Paper copyright 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG.
