Please use this identifier to cite or link to this item: http://bura.brunel.ac.uk/handle/2438/31390
Title: Piecewise convolutional neural network relation extraction with self-attention mechanism
Authors: Zhang, B
Xu, L
Liu, K-H
Yang, R
Li, M-Z
Guo, X-Y
Keywords: relation extraction;multi-head attention;PCNN;variational autoencoder
Issue Date: 18-Oct-2024
Publisher: Elsevier
Citation: Zhang, B. et al. (2024) 'Piecewise convolutional neural network relation extraction with self-attention mechanism', Pattern Recognition, 159, 111083, pp. 1 - 10. doi: 10.1016/j.patcog.2024.111083.
Abstract: The task of relation extraction in natural language processing is to identify the relation between two specified entities in a sentence. However, the existing model methods do not fully utilize the word feature information and pay little attention to the influence degree of the relative relation extraction results of each word. In order to address the aforementioned issues, we propose a relation extraction method based on self-attention mechanism (SPCNN-VAE) to solve the above problems. First, we use a multi-head self-attention mechanism to process word vectors and generate sentence feature vector representations, which can be used to extract semantic dependencies between words in sentences. Then, we introduce the word position to combine the sentence feature representation with the position feature representation of words to form the input representation of piecewise convolutional neural network (PCNN). Furthermore, to identify the word feature information that is most useful for relation extraction, an attention-based pooling operation is employed to capture key convolutional features and classify the feature vectors. Finally, regularization is performed by a variational autoencoder (VAE) to enhance the encoding ability of model word information features. The performance analysis is performed on SemEval 2010 task 8, and the experimental results show that the proposed relation extraction model is effective and outperforms some competitive baselines.
Description: Data availability: Data will be made available on request.
URI: https://bura.brunel.ac.uk/handle/2438/31390
DOI: https://doi.org/10.1016/j.patcog.2024.111083
ISSN: 0031-3203
Other Identifiers: ORCiD: Bo Zhang https://orcid.org/0000-0002-2289-2877
ORCiD: Ke-Hao Liu https://orcid.org/0000-0002-4364-5066
ORCiD: Ru Yang https://orcid.org/0000-0001-7879-681X
ORCiD: Mao-Zhen Li https://orcid.org/0000-0002-0820-5487
Article number: 111083
Appears in Collections:Dept of Electronic and Electrical Engineering Embargoed Research Papers

Files in This Item:
File Description SizeFormat 
FullText.pdfEmbargoed until 18 October 20251.37 MBAdobe PDFView/Open


This item is licensed under a Creative Commons License Creative Commons