http://bura.brunel.ac.uk/handle/2438/9053 Brunel Software Engineering ResearCh Group (B-SERC) 2026-04-05T16:49:04Z http://bura.brunel.ac.uk/handle/2438/5006 Title: New ideas and emerging research: evaluating prediction system accuracy Authors: Shepperd, M Abstract: BACKGROUND: Prediction e.g. of project cost is an important concern in software engineering. PROBLEM: Although many empirical validations of software engineering prediction systems have been published, no one approach dominates and sense-making of conflicting empirical results is proving challenging. METHOD: We propose a new approach to evaluating competing prediction systems based upon an unbiased statistic (Standardised Accuracy), analysis of results relative to the baseline technique of guessing and calculation of effect sizes. RESULTS: Two empirical studies are revisited and the published results are shown to be misleading when re-analysed using our new approach. CONCLUSION: Biased statistics such as MMRE are deprecated. By contrast our approach leads to valid results. Such steps will greatly assist in performing future meta-analyses. 2011-01-01T00:00:00Z http://bura.brunel.ac.uk/handle/2438/5005 Title: Group project work from the outset: an in-depth teaching experience report Authors: Shepperd, M Abstract: CONTEXT - we redesigned our undergraduate computing programmes to address problems of motivation and outdated content. METHOD - the primary vehicle for the new curriculum was the group project which formed a central spine for the entire degree right from the first year. RESULTS - so far this programme has been successfully run once. Failures, drop outs and students required to retake modules have been halved (from an average of 21.6% from the previous 4 years to 9.5%) and students obtaining the top two grades have increased from 25.2% to 38.9%. CONCLUSIONS - whilst we cannot be certain that all improvement is due to the group projects informally the change has been well received, however, we are looking for areas to improve including the possibility of more structured support for student metacognitive awareness. Description: This article is an extended version of a paper that was submitted to 24th IEEE Conference on Software Engineering Education and Training, Honolulu, May 2011 2011-01-01T00:00:00Z http://bura.brunel.ac.uk/handle/2438/16391 Title: Reaching and distinguishing states of distributed systems Authors: Hierons, RM Abstract: Some systems interact with their environment at physically distributed interfaces, called ports, and in testing such a system it is normal to place a tester at each port. Each tester observes only the events at its port and it is known that this limited observational power introduces additional controllability and observability problems into testing. Given a multiport finite state machine (FSM) $M$, we consider the problems of defining strategies for the testers either to reach a given state of $M$ or to distinguish two states of $M$. These are important problems since most techniques for testing from a single-port FSM use sequences that reach and distinguish states. Both problems can be solved in low-order polynomial time for single-port FSMs but we prove that the corresponding decision problems are undecidable for multiport FSMs. However, we also show that they can be solved in low-order polynomial times for deterministic FSMs if we restrict our attention to controllable tests. These results have important ramifications for testing from a multiport FSM since they suggest that methods for testing from a single-port FSM cannot be easily adapted. In addition, two FSMs can be distinguished if and only if their initial states can be distinguished and so the results suggest that, in contrast to single-port FSMs, we cannot expect to produce general complete test generation methods for multiport FSMs. 2010-01-01T00:00:00Z http://bura.brunel.ac.uk/handle/2438/4391 Title: Testing in the distributed test architecture: An extended abstract Authors: Hierons, RM Abstract: Some systems interact with their environment at a number of physically distributed interfaces/ports and when testing such a system it is normal to place a local tester at each port. If the local testers cannot interact with one another and there is no global clock then we are testing in the distributed test architecture and this can introduce additional controllability and observability problems. While there has been interest in test generation algorithms that overcome controllability and observability problems, such algorithms lack generality since controllability and observability problems cannot always be overcome. In addition, traditionally only deterministic systems and models have been considered despite distributed systems often being non-deterministic. This paper describes recent work that characterized the power of testing in the distributed test architecture in the context of testing from a deterministic finite state machine and also work that investigated testing from a non-deterministic finite state machine and testing from an input output transition system. This work has the potential to lead to more general test generation algorithms for the distributed test architecture. 2008-01-01T00:00:00Z