Abstract
A large number of software reliability growth models have been proposed in the literature. Many of these models have also been the subject of optimization problems, including the optimal release problem in which a decision-maker seeks to minimize cost by balancing the cost of testing with field failures. However, the majority of these optimal release formulations are either unused or untested. In many cases, researchers derive expressions and apply them to the complete set of failure data in order to identify the time at which cost was minimized, but this is clearly unusable, since it is not possible to go back in time to make a release decision. The only other implicit strategy implied by these optimal release formulations is to refit a model every time a failure occurs and to assess if the optimal release time has past or if additional testing should be performed.
To assess these limitations, which prevent the application of software reliability growth models in practice, this paper presents an online approach to software optimal release planning. In this approach, the model is periodically refit and optimal release time updated. We objectively compare the quality of decisions by comparing the ratio of the cost achieved by a release decision and the unknown true optimum, which only becomes known after the complete data is observed. We also consider a more conservative approach, where the release recommendation must be satisfied for more than one observation period. Our results indicate that the optimal release strategy implied by past studies, which would release software after a single observation can be suboptimal with respect to cost attained and that a more conservative approach that defers release until release is recommended in multiple successive observation periods may be more appropriate. The approach can thus complement decision-maker judgement.