Document Type
Article
Abstract
Knowledge Graphs are a great resource to capture semantic knowledge in terms of entities and relationships between the entities. However, current deep learning models takes as input distributed representations or vectors. Thus, the graph is compressed in a vectorized representation. We conduct a study to examine if the deep learning model can compress a graph and then output the same graph with most of the semantics intact. Our experiments show that Transformer models are not able to express the full semantics of the input knowledge graph. We find that this is due to the disparity between the directed, relationship and type based information contained in a Knowledge Graph and the fully connected token-token undirected graphical interpretation of the Transformer Attention matrix.
Digital Object Identifier (DOI)
Publication Info
Preprint version 2022.
Can Language Models Capture Graph Semantics? From Graphs to Language Model and Vice-Versa by Tarun Garg, Kaushik Roy, and Amit Sheth is licensed under a Creative Commons Attribution 4.0 International (CC BY 4.0) license.
APA Citation
Garg, T., Roy, K., & Sheth, A. (2022). Can Language Models Capture Graph Semantics? From Graphs to Language Model and Vice-Versa (arXiv:2206.09259). arXiv. https://doi.org/10.48550/arXiv.2206.09259