Explainable Temporal Inference for Irregular Multivariate Time Series. A Case Study for Early Prediction of Multidrug Resistance

Abstract

Objective: Many healthcare problems involve complex patient trajectories represented as Multivariate Time Series (MTS), with predictions often coming as Time Series (TS) outputs. Despite recent advances, these “MTS-to-TS” inference tasks remain challenging due to data irregularity, temporal dependencies, and the need for clinical explainability. To address these demands, we propose novel eXplainable Artificial Intelligence (XAI) methods for “MTS-to-TS” architectures, enabling tracking of patient evolution and identification of key variable patterns associated with adverse outcomes. We evaluate our approach on private ICU data from the University Hospital of Fuenlabrada (UHF) for Multidrug Resistance (MDR) prediction and the public HiRID dataset (circulatory failure). Methods: We introduce three XAI techniques: i) Irregular Time SHapley Additive exPlanation (IT-SHAP), a post-hoc extension of TimeSHAP to TS outputs; ii) Hadamard Attention, an intrinsic mechanism for capturing temporal dependencies; and iii) Causal Conditional Mutual Information, a pre-hoc approach for feature selection. Results: MDR prediction achieved highest performance with a GRU using Hadamard Attention (ROC-AUC=0.783±0.023), while circulatory failure was best predicted with LSTM (ROC–AUC of 0.9970±1.6e−3). In terms of explainability, IT-SHAP uncovered clinically relevant risk factors—early antibiotic use and bacterial cultures—later validated by UHF clinicians. Conclusion: Our framework offers temporal explainability in “MTS-to-TS” architectures, allowing clinicians to trace disease trajectories and understand the contribution of each variable at each time step. Significance: Integrating explainable MDR risk predictions into EHR systems enables early interventions, improved antimicrobial stewardship, and infection control. The framework's scalability to other ICU challenges underscores its clinical impact.