Abstract
This paper proposes a repair score metric to trigger the time to repair deep learning-based cyberattack classifiers by monitoring performance degradation. The repair score integrates the F1-score, Euclidean distance ratio, Wasserstein distance, and the Kolmogorov-Smirnov test to signal when repair is recommended. A regression model is developed to predict the repair score under scenarios involving emerging threats, enabling early detection of degradation even without labeled data. The method is validated with a classifier pre-trained on a subset of classes from a network activity trace, with novel classes of activities introduced in testing across sliding windows. Experimental results show that spikes in the repair score consistently correspond to drops in performance, increased classification uncertainty, or distribution shifts, validating its use as a repair trigger. The regression model achieves high predictive accuracy \left( {r_{{\text{adj}}}^2 = 0.99} \right), demonstrating the potential of predictive modeling to guide timely interventions and enhance the resilience of network intrusion detection systems.