Please use this identifier to cite or link to this item:
http://bura.brunel.ac.uk/handle/2438/31364
Title: | A review of faithfulness metrics for hallucination assessment in Large Language Models |
Authors: | Malin, B Kalganova, T Boulgouris, N |
Keywords: | evaluation;fact extraction;faithfulness;hallucination;LLM;machine translation;question answering;RAG;summarization |
Issue Date: | 12-Jun-2025 |
Publisher: | Institute of Electrical and Electronics Engineers (IEEE) |
Citation: | Malin, B., Kalganova, T. and Boulgouris, N. (2025) 'A review of faithfulness metrics for hallucination assessment in Large Language Models', IEEE Journal of Selected Topics in Signal Processing, 0 (Early Access, Special Issue), pp. 1 - 13. doi: 10.1109/JSTSP.2025.3579203. |
Abstract: | This review examines the means with which faithfulness has been evaluated across open-ended summarization, question answering and machine translation tasks. We find that the use of Large Language Models (LLMs) as a faithfulness evaluator is commonly the metric that is most highly correlated with human judgement. The means with which other studies have mitigated hallucinations is discussed, with both retrieval augmented generation (RAG) and prompting framework approaches having been linked with superior faithfulness, whilst other recommendations for mitigation are provided. Research into faithfulness is integral to the continued widespread use of LLMs, as unfaithful responses can pose major risks to many areas whereby LLMs would otherwise be suitable. Furthermore, evaluating open-ended generation provides a more comprehensive measure of LLM performance than commonly used multiplechoice benchmarking, which can help in advancing the trust that can be placed within LLMs. |
Description: | This article has been accepted for publication in IEEE Journal of Selected Topics in Signal Processing. This is the author's version which has not been fully edited and content may change prior to final publication. Citation information: DOI 10.1109/JSTSP.2025.3579203 . |
URI: | https://bura.brunel.ac.uk/handle/2438/31364 |
DOI: | https://doi.org/10.1109/JSTSP.2025.3579203 |
ISSN: | 1932-4553 |
Other Identifiers: | ORCiD: Tatiana Kalganova https://orcid.org/0000-0003-4859-7152 ORCiD: Nikolaos Boulgouris https://orcid.org/0000-0002-5382-6856 |
Appears in Collections: | Dept of Electronic and Electrical Engineering Research Papers |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
FullText.pdf | Copyright © 2025 Institute of Electrical and Electronics Engineers (IEEE). Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works ( https://journals.ieeeauthorcenter.ieee.org/become-an-ieee-journal-author/publishing-ethics/guidelines-and-policies/post-publication-policies/ | 1.22 MB | Adobe PDF | View/Open |
Items in BURA are protected by copyright, with all rights reserved, unless otherwise indicated.