Abstract
Traditional Deep Neural Networks often struggle with new or unfamiliar data patterns since they operate on a closed-set assumption. This challenge arises due to inherent limitations in the model architecture, such as the softmax function commonly used for classification tasks, which tends to exhibit overconfidence and inaccuracies when faced with novel inputs. Prior studies have highlighted the need for open-set recognition (OSR) techniques to differentiate between known and unknown data points, but existing approaches often exhibit a bias toward flagging inputs as unknown. To address this issue, we introduce a novel OSR technique called VarMax, designed to maintain a balanced approach. VarMax leverages the variance in model predictions to discern between known and unknown inputs. We propose a method for classifying ambiguous samples based on prediction variance to detect out-of-distribution samples to enhance classification accuracy and reliability. Our experiments demonstrate that VarMax meets and exceeds the performance of existing methods in identifying unknown data points while also improving the model's confidence and robustness in distinguishing between known and unknown inputs.