Abstract
Artificial Intelligence (AI) technologies are increasingly integrated into various critical areas of our daily life like autonomous vehicles, energy, healthcare, defense, manufacturing, and finance to improve efficiency and automation. Thus, it is paramount to model and evaluate the reliability of AI methods, ensuring that AI systems in those critical applications are safe, accountable and capable of functioning efficiently as expected. This paper puts forward a straightforward, comprehensive reliability index that integrates seven performance metrics spanning accuracy, efficiency, and robustness to evaluate and compare the performance of AI methods. The sequential least squares programming (SLSQP) algorithm is employed to optimize the weights of different metrics under application-specific constraints, facilitating fair comparisons across applications with different operational requirements. Based on the proposed index with the optimized weights, a set of 35 AI methods are studied and compared, including cutting-edge time series forecasting methods (e.g., random forest, long short-term memory (LSTM), TimeMixer), and hybrid methods (e.g., convolutional neural network-LSTM, empirical wavelet transform-LSTM, empirical mode decomposition-LSTM, variational mode decomposition LSTM, stationary wavelet transform-LSTM, and singular spectrum analysis-LSTM). Empirical studies of those AI methods using the UK-DALE household datasets are conducted, revealing that the stationary wavelet transform-LSTM is a top-performing model with strong generalizability and stability for example applications in short-term household energy consumption forecasting. The proposed index serves as a valuable tool for selecting reliable AI models to be deployed in critical applications.