Abstract
Despite the tremendous advances in artificial neural networks, a systematic and efficient method for hyperparameter tuning, including optimizing the number of hidden neurons, is still lacking. Today, the method for determining the appropriate number of hidden neurons involves trial and error, i.e., randomly selecting the number of hidden neurons, looking for possible overfitting or underfitting, and adjusting the hidden neurons accordingly. There are guidelines that can be used to help approach the optimal number of hidden neurons, such as using a number between the size of the input and output layers or using two-thirds the size of the input layer plus the output layer. However, they still need to be tested, trained, and the number of neurons must be adjusted by trial and error. The method proposed in this thesis uses an automated approach to optimizing the number of hidden neurons in a partially connected neural network (PCNN). The method starts with a fully connected neural network (FCNN) with a small number of hidden neurons, and then increases the number of hidden neurons until the accuracy starts to decrease due to overfitting. At this point, a certain percentage of weak links and weak neurons are identified and removed from the FCNN, resulting in a PCNN. We continue to fine-tune the number of hidden neurons in the PCNN until an optimal number of hidden neurons is reached, while accuracy decreases but remains above a threshold value as a performance requirement. To illustrate the feasibility and effectiveness of our method, we used three datasets to show that our PCNN-based approach can efficiently find the optimal number of hidden neurons without significant accuracy loss. The results also show that the use of PCNN results in a modest overhead and a shorter training time than the FCNN-only approach.