Brunel University Research Archive (BURA) >
Schools >
School of Information Systems, Computing and Mathematics >
School of Information Systems, Computing and Mathematics Research Papers >

Please use this identifier to cite or link to this item: http://bura.brunel.ac.uk/handle/2438/5006

Title: New ideas and emerging research: evaluating prediction system accuracy
Authors: Shepperd, M
Keywords: Software engineering
Software project management
Forecasting
Effort prediction
Accuracy
Empirical evaluation
Publication Date: 2011
Abstract: BACKGROUND: Prediction e.g. of project cost is an important concern in software engineering. PROBLEM: Although many empirical validations of software engineering prediction systems have been published, no one approach dominates and sense-making of conflicting empirical results is proving challenging. METHOD: We propose a new approach to evaluating competing prediction systems based upon an unbiased statistic (Standardised Accuracy), analysis of results relative to the baseline technique of guessing and calculation of effect sizes. RESULTS: Two empirical studies are revisited and the published results are shown to be misleading when re-analysed using our new approach. CONCLUSION: Biased statistics such as MMRE are deprecated. By contrast our approach leads to valid results. Such steps will greatly assist in performing future meta-analyses.
URI: http://bura.brunel.ac.uk/handle/2438/5006
Appears in Collections:B-SERC Research Papers
Information Systems and Computing
School of Information Systems, Computing and Mathematics Research Papers

Files in This Item:

File Description SizeFormat
Fulltext.pdf257.64 kBAdobe PDFView/Open

Items in BURA are protected by copyright, with all rights reserved, unless otherwise indicated.

 


Library (c) Brunel University.    Powered By: DSpace
Send us your
Feedback. Last Updated: September 14, 2010.
Managed by:
Hassan Bhuiyan