Brunel University Research Archive (BURA) >
Research Areas >
Software Engineering (B-SERC) >

Please use this identifier to cite or link to this item:

Title: Comparing software prediction techniques using simulation
Authors: Shepperd, M J
Kadoda, G
Keywords: Empirical software engineering
Prediction system
Effort prediction
Cost model
Publication Date: 2001
Publisher: IEEE Computer Society
Citation: IEEE Transactions on Software Engineering, 27(11): 1014 - 1022, Nov 2001
Abstract: The need for accurate software prediction systems increases as software becomes much larger and more complex. We believe that the underlying characteristics: size, number of features, type of distribution, etc., of the data set influence the choice of the prediction system to be used. For this reason, we would like to control the characteristics of such data sets in order to systematically explore the relationship between accuracy, choice of prediction system, and data set characteristic. It would also be useful to have a large validation data set. Our solution is to simulate data allowing both control and the possibility of large (1000) validation cases. The authors compare four prediction techniques: regression, rule induction, nearest neighbor (a form of case-based reasoning), and neural nets. The results suggest that there are significant differences depending upon the characteristics of the data set. Consequently, researchers should consider prediction context when evaluating competing prediction systems. We observed that the more "messy" the data and the more complex the relationship with the dependent variable, the more variability in the results. In the more complex cases, we observed significantly different results depending upon the particular training set that has been sampled from the underlying data set. However, our most important result is that it is more fruitful to ask which is the best prediction system in a particular context rather than which is the "best" prediction system.
ISSN: 0098-5589
Appears in Collections:Computer Science
Software Engineering (B-SERC)
Dept of Computer Science Research Papers

Files in This Item:

File Description SizeFormat
Comparing Software Prediction 2001.pdf294.81 kBAdobe PDFView/Open

Items in BURA are protected by copyright, with all rights reserved, unless otherwise indicated.