TY - JOUR
T1 - Fusing external knowledge resources for natural language understanding techniques
T2 - A survey
AU - Wang, Yuqi
AU - Wang, Wei
AU - Chen, Qi
AU - Huang, Kaizhu
AU - Nguyen, Anh
AU - De, Suparna
AU - Hussain, Amir
N1 - Funding Information:
We would like to thank all the reviewers for their valuable and helpful comments to help us improve the quality and presentation of the paper. This research is funded by the Postgraduate Research Scholarship (PGRS) at Xi’an Jiaotong-Liverpool University (contract number PGRS2006013 ), and partially supported by Jiangsu Science and Technology Programme (contract number BK20221260 and BE2020006-4 ), National Natural Science Foundation of China (contract number 61876155 ) and Engineering and Physical Sciences Research Council (contract number EP/M026981/1 , EP/T021063/1 and EP/T024917/1 ).
Publisher Copyright:
© 2022 Elsevier B.V.
PY - 2023/4
Y1 - 2023/4
N2 - Knowledge resources, e.g. knowledge graphs, which formally represent essential semantics and information for logic inference and reasoning, can compensate for the unawareness nature of many natural language processing techniques based on deep neural networks. This paper provides a focused review of the emerging but intriguing topic that fuses quality external knowledge resources in improving the performance of natural language processing tasks. Existing methods and techniques are summarised in three main categories: (1) static word embeddings, (2) sentence-level deep learning models, and (3) contextualised language representation models, depending on when, how and where external knowledge is fused into the underlying learning models. We focus on the solutions to mitigate two issues: knowledge inclusion and inconsistency between language and knowledge. Details on the design of each representative method, as well as their strength and limitation, are discussed. We also point out some potential future directions in view of the latest trends in natural language processing research.
AB - Knowledge resources, e.g. knowledge graphs, which formally represent essential semantics and information for logic inference and reasoning, can compensate for the unawareness nature of many natural language processing techniques based on deep neural networks. This paper provides a focused review of the emerging but intriguing topic that fuses quality external knowledge resources in improving the performance of natural language processing tasks. Existing methods and techniques are summarised in three main categories: (1) static word embeddings, (2) sentence-level deep learning models, and (3) contextualised language representation models, depending on when, how and where external knowledge is fused into the underlying learning models. We focus on the solutions to mitigate two issues: knowledge inclusion and inconsistency between language and knowledge. Details on the design of each representative method, as well as their strength and limitation, are discussed. We also point out some potential future directions in view of the latest trends in natural language processing research.
KW - Deep learning
KW - Knowledge fusion
KW - Knowledge graph
KW - Natural language understanding
KW - Representation learning
UR - http://www.scopus.com/inward/record.url?scp=85144040289&partnerID=8YFLogxK
U2 - 10.1016/j.inffus.2022.11.025
DO - 10.1016/j.inffus.2022.11.025
M3 - Short survey
AN - SCOPUS:85144040289
SN - 1566-2535
VL - 92
SP - 190
EP - 204
JO - Information Fusion
JF - Information Fusion
ER -