Beyond Model Complexity: Explainable Machine Learning for Predictive-Maintenance-Oriented Alarm Quality Monitoring of ICU Bedside Monitors with Locked Alarm-Type Validation

Abstract

Predictive maintenance of medical equipment requires models that remain reliable when the operating phenotype changes. Public medical-equipment data rarely contain both device logs and maintenance work orders. We therefore evaluated a maintenance-oriented alarm-quality task using real ICU bedside-monitor waveforms and expert-reviewed alarm labels. We analysed 750 five-minute multiparameter recordings from the PhysioNet/Computing in Cardiology Challenge 2015. The endpoint was true versus false alarm. Waveform distribution, derivative, signal-quality, and band-power features were extracted without using alarm type as a predictor. We compared a compact logistic model using 150 features with a random forest, radial-basis-function support-vector machine, and multilayer perceptron using 399 features. Internal performance was estimated by stratified five-fold cross-validation. Locked transportability was assessed by holding out each of five alarm phenotypes in turn. Discrimination, average precision, Brier score, threshold-based operating characteristics, and model-based feature attribution were reported. The 750 recordings contained 294 true alarms and 456 false alarms. Random forest had the highest internal AUROC (0.811) and average precision (0.743), but its locked external AUROC was 0.620 and average precision was 0.493. The compact logistic model had lower internal AUROC (0.733) but the highest locked external AUROC (0.661) and average precision (0.496). The radial-basis-function support-vector machine and multilayer perceptron had external AUROCs of 0.547 and 0.532, respectively. At thresholds selected to achieve at least 0.90 internal sensitivity, the compact model retained 0.861 sensitivity and 0.366 specificity in the locked evaluation.Greater model complexity improved internal discrimination but did not improve transportability across alarm phenotypes. A compact signal-quality representation was more portable in this public ICU-monitor benchmark. The dataset supports alarm-quality and maintenance-oriented monitoring research, but it does not support claims of component failure prediction or maintenance-work-order forecasting.