Please use this identifier to cite or link to this item:
Title: A Mapping Study on Testing Non-testable Systems
Authors: Patel, K
Hierons, RM
Keywords: Software Testing;Oracle Problem;Non-testable;Test Oracles;Mapping Study;Survey
Issue Date: 2017
Publisher: Springer
Citation: Software Quality Journal, (2017)
Abstract: The terms “Oracle Problem” and “Non-testable system” interchangeably refer to programs in which the application of test oracles is infeasible. Test oracles are an integral part of conventional testing techniques, thus such techniques are inoperable in these programs. The prevalence of the oracle problem has inspired the research community to develop several automated testing techniques that can detect functional software faults in such programs. These techniques include N-Version testing, Metamorphic Testing, Assertions, Machine Learning Oracles, and Statistical Hypothesis Testing. This paper presents a Mapping Study that covers these techniques. The Mapping Study presents a series of discussions about each technique, from different perspectives e.g. effectiveness, efficiency, and usability. It also presents a comparative analysis of these techniques in terms of these perspectives. Finally, potential research opportunities within the non-testable systems problem domain are highlighted within the Mapping Study. We believe that the aforementioned discussions and comparative analysis will be invaluable for new researchers that are attempting to familiarise themselves with the field, and be a useful resource for practitioners that are in the process of selecting an appropriate technique for their context, or deciding how to apply their selected technique. We also believe that our own insights, which are embedded throughout these discussions and the comparative analysis, will be useful for researchers that are already accustomed to the field. It is our hope that the potential research opportunities that have been highlighted by the Mapping Study will steer the direction of future research endeavours.
ISSN: 0963-9314
Appears in Collections:Dept of Clinical Sciences Embargoed Research Papers

Files in This Item:
File Description SizeFormat 
Fulltext.pdfAvailable from 24/11/2018598.62 kBAdobe PDFView/Open

Items in BURA are protected by copyright, with all rights reserved, unless otherwise indicated.