Please use this identifier to cite or link to this item:
http://bura.brunel.ac.uk/handle/2438/6473
Full metadata record
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Shepperd, M | - |
dc.contributor.author | MacDonell, S | - |
dc.date.accessioned | 2012-06-15T08:46:40Z | - |
dc.date.available | 2012-06-15T08:46:40Z | - |
dc.date.issued | 2012 | - |
dc.identifier.citation | Information and Software Technology, 54(8): 820 - 827, Aug 2012 | en_US |
dc.identifier.issn | 0950-5849 | - |
dc.identifier.uri | http://www.sciencedirect.com/science/article/pii/S095058491200002X | en |
dc.identifier.uri | http://bura.brunel.ac.uk/handle/2438/6473 | - |
dc.description | This is the Pre-print version of the Article - Copyright @ 2012 Elsevier | en_US |
dc.description.abstract | Context: Software engineering has a problem in that when we empirically evaluate competing prediction systems we obtain conflicting results. Objective: To reduce the inconsistency amongst validation study results and provide a more formal foundation to interpret results with a particular focus on continuous prediction systems. Method: A new framework is proposed for evaluating competing prediction systems based upon (1) an unbiased statistic, Standardised Accuracy, (2) testing the result likelihood relative to the baseline technique of random ‘predictions’, that is guessing, and (3) calculation of effect sizes. Results: Previously published empirical evaluations of prediction systems are re-examined and the original conclusions shown to be unsafe. Additionally, even the strongest results are shown to have no more than a medium effect size relative to random guessing. Conclusions: Biased accuracy statistics such as MMRE are deprecated. By contrast this new empirical validation framework leads to meaningful results. Such steps will assist in performing future meta-analyses and in providing more robust and usable recommendations to practitioners. | en_US |
dc.description.sponsorship | Martin Shepperd was supported by the UK Engineering and Physical Sciences Research Council (EPSRC) under Grant EP/H050329. | en_US |
dc.language.iso | en | en_US |
dc.publisher | Elsevier | en_US |
dc.subject | Software engineering | en_US |
dc.subject | Prediction system | en_US |
dc.subject | Empirical validation | en_US |
dc.subject | Randomisation techniques | en_US |
dc.title | Evaluating prediction systems in software project estimation | en_US |
dc.type | Article | en_US |
dc.identifier.doi | http://dx.doi.org/10.1016/j.infsof.2011.12.008 | - |
pubs.organisational-data | /Brunel | - |
pubs.organisational-data | /Brunel/Brunel Active Staff | - |
pubs.organisational-data | /Brunel/Brunel Active Staff/School of Info. Systems, Comp & Maths | - |
pubs.organisational-data | /Brunel/Brunel Active Staff/School of Info. Systems, Comp & Maths/IS and Computing | - |
pubs.organisational-data | /Brunel/University Research Centres and Groups | - |
pubs.organisational-data | /Brunel/University Research Centres and Groups/School of Information Systems, Computing and Mathematics - URCs and Groups | - |
pubs.organisational-data | /Brunel/University Research Centres and Groups/School of Information Systems, Computing and Mathematics - URCs and Groups/Centre for Information and Knowledge Management | - |
Appears in Collections: | Publications Computer Science Dept of Computer Science Research Papers |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
IST_Invited_2011_v7.pdf | 334.15 kB | Unknown | View/Open |
Items in BURA are protected by copyright, with all rights reserved, unless otherwise indicated.