Please use this identifier to cite or link to this item:
http://bura.brunel.ac.uk/handle/2438/25622
Full metadata record
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Groen, D | - |
dc.contributor.author | Arabnejad, H | - |
dc.contributor.author | Suleimenova, D | - |
dc.contributor.author | Edeling, W | - |
dc.contributor.author | Raffin, E | - |
dc.contributor.author | Xue, Y | - |
dc.contributor.author | Bronik, K | - |
dc.contributor.author | Monnier, N | - |
dc.contributor.author | Coveney, P | - |
dc.date.accessioned | 2022-12-13T10:49:38Z | - |
dc.date.available | 2022-12-13T10:49:38Z | - |
dc.date.issued | 2022-11-24 | - |
dc.identifier | ORCID iDs: Derek Groen https://orcid.org/0000-0001-7463-3765; Hamid Arabnejad https://orcid.org/0000-0002-0789-1825; Diana Suleimenova https://orcid.org/0000-0003-4474-0943; Yani Xue https://orcid.org/0000-0002-7526-9085. | - |
dc.identifier | 108596 | - |
dc.identifier.citation | Groen, D. et al. (2023) ‘FabSim3: An automation toolkit for verified simulations using high performance computing’ in Computer Physics Communications, 283. 108596, pp.1 - 18. https://doi.org/10.1016/j.cpc.2022.108596. | en_US |
dc.identifier.issn | 0010-4655 | - |
dc.identifier.uri | https://bura.brunel.ac.uk/handle/2438/25622 | - |
dc.description | Data availability: Data will be made available on request. | - |
dc.description.abstract | Copyright © 2022 The Author(s). A common feature of computational modelling and simulation research is the need to perform many tasks in complex sequences to achieve a usable result. This will typically involve tasks such as preparing input data, pre-processing, running simulations on a local or remote machine, post-processing, and performing coupling communications, validations and/or optimisations. Tasks like these can involve manual steps which are time and effort intensive, especially when it involves the management of large ensemble runs. Additionally, human errors become more likely and numerous as the research work becomes more complex, increasing the risk of damaging the credibility of simulation results. Automation tools can help ensure the credibility of simulation results by reducing the manual time and effort required to perform these research tasks, by making more rigorous procedures tractable, and by reducing the probability of human error due to a reduced number of manual actions. In addition, efficiency gained through automation can help researchers to perform more research within the budget and effort constraints imposed by their projects. This paper presents the main software release of FabSim3, and explains how our automation toolkit can improve and simplify a range of tasks for researchers and application developers. FabSim3 helps to prepare, submit, execute, retrieve, and analyze simulation workflows. By providing a suitable level of abstraction, FabSim3 reduces the complexity of setting up and managing a large-scale simulation scenario, while still providing transparent access to the underlying layers for effective debugging. The tool also facilitates job submission and management (including staging and curation of files and environments) for a range of different supercomputing environments. Although FabSim3 itself is application-agnostic, it supports a provably extensible plugin system where users automate simulation and analysis workflows for their own application domains. To highlight this, we briefly describe a selection of these plugins and we demonstrate the efficiency of the toolkit in handling large ensemble workflows. Program summary: Program Title: FabSim3 CPC Library link to program files: https://doi.org/10.17632/6nfrwy7ptj.1 Licensing provisions: BSD 3-clause Programming language: Python 3 Nature of problem: Many aspects are crucial for obtaining reproducible and robust simulation results. For instance, we need to curate all the inputs and outputs for later scrutiny, scrutinize the model behaviour under slightly perturbed circumstances, quantify the propagation of key uncertainties from input data and known parameters and analyze the sensitivity for any parameters for which the exact specification eludes us. Solution method: FabSim3 uses a range of methods to provide automation. These primarily include: (i) SSH + Fabric2 to enable remote execution of SSH commands, (ii) an internal parameter state space using primarily Python dict objects that can be customized with machine- plugin- and user-specific modifications, (iii) Python templating to quickly enable the insertion of state space variables into supercomputing scripts, (iv) multiprocessing and/or QCG-PilotJob to enable efficient submission and execution of job arrays and (v) a system of flexibly installable and modifiable Python3 plugins which allows users to create and customize application-specific functionalities without modifying the core code base. In addition to the written code, FabSim3 also relies on a set of user conventions to maintain a separation of concerns (particularly between machine-, user- and application-specific settings). Additional comments including restrictions and unusual features: This paper serves as the definitive reference for FabSim3. | en_US |
dc.description.sponsorship | EPSRC under grant agreement EP/W007711/1, as well as by the VECMA and HiDALGO projects, which have received funding from the European Union Horizon 2020 research and innovation programme under grant agreement nos 800925 and 824115. In addition, FabFlee was supported by the ITFLOWS project and FabCovid19 by the STAMINA project, both of which have received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 882986 and No 883441 respectively | en_US |
dc.format.medium | Print-Electronic | - |
dc.publisher | Elsevier | en_US |
dc.rights | Copyright © 2022 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY license (https://creativecommons.org/licenses/by/4.0/). | - |
dc.rights.uri | https://creativecommons.org/licenses/by/4.0/ | - |
dc.subject | automation | en_US |
dc.subject | high-performance computing | en_US |
dc.subject | simulation | en_US |
dc.subject | materials | en_US |
dc.subject | migration | en_US |
dc.subject | climate | en_US |
dc.subject | biomedicine | en_US |
dc.subject | multiscale modelling | en_US |
dc.title | FabSim3: An automation toolkit for verified simulations using high performance computing | en_US |
dc.type | Article | en_US |
dc.identifier.doi | https://doi.org/10.1016/j.cpc.2022.108596 | - |
dc.relation.isPartOf | Computer Physics Communications | - |
pubs.publication-status | Published | - |
dc.identifier.eissn | 1879-2944 | - |
dc.rights.holder | The Author(s) | - |
Appears in Collections: | Dept of Computer Science Research Papers |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
FullText.pdf | Copyright © 2022 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY license (https://creativecommons.org/licenses/by/4.0/). | 1.41 MB | Adobe PDF | View/Open |
This item is licensed under a Creative Commons License