Please use this identifier to cite or link to this item: http://bura.brunel.ac.uk/handle/2438/31364
Title: A review of faithfulness metrics for hallucination assessment in Large Language Models
Authors: Malin, B
Kalganova, T
Boulgouris, N
Keywords: evaluation;fact extraction;faithfulness;hallucination;LLM;machine translation;question answering;RAG;summarization
Issue Date: 12-Jun-2025
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Citation: Malin, B., Kalganova, T. and Boulgouris, N. (2025) 'A review of faithfulness metrics for hallucination assessment in Large Language Models', IEEE Journal of Selected Topics in Signal Processing, 0 (Early Access, Special Issue), pp. 1 - 13. doi: 10.1109/JSTSP.2025.3579203.
Abstract: This review examines the means with which faithfulness has been evaluated across open-ended summarization, question answering and machine translation tasks. We find that the use of Large Language Models (LLMs) as a faithfulness evaluator is commonly the metric that is most highly correlated with human judgement. The means with which other studies have mitigated hallucinations is discussed, with both retrieval augmented generation (RAG) and prompting framework approaches having been linked with superior faithfulness, whilst other recommendations for mitigation are provided. Research into faithfulness is integral to the continued widespread use of LLMs, as unfaithful responses can pose major risks to many areas whereby LLMs would otherwise be suitable. Furthermore, evaluating open-ended generation provides a more comprehensive measure of LLM performance than commonly used multiplechoice benchmarking, which can help in advancing the trust that can be placed within LLMs.
Description: This article has been accepted for publication in IEEE Journal of Selected Topics in Signal Processing. This is the author's version which has not been fully edited and content may change prior to final publication. Citation information: DOI 10.1109/JSTSP.2025.3579203 .
URI: https://bura.brunel.ac.uk/handle/2438/31364
DOI: https://doi.org/10.1109/JSTSP.2025.3579203
ISSN: 1932-4553
Other Identifiers: ORCiD: Tatiana Kalganova https://orcid.org/0000-0003-4859-7152
ORCiD: Nikolaos Boulgouris https://orcid.org/0000-0002-5382-6856
Appears in Collections:Dept of Electronic and Electrical Engineering Research Papers

Files in This Item:
File Description SizeFormat 
FullText.pdfCopyright © 2025 Institute of Electrical and Electronics Engineers (IEEE). Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works ( https://journals.ieeeauthorcenter.ieee.org/become-an-ieee-journal-author/publishing-ethics/guidelines-and-policies/post-publication-policies/1.22 MBAdobe PDFView/Open


Items in BURA are protected by copyright, with all rights reserved, unless otherwise indicated.