Knowledge-infused Learning for Entity Prediction in Driving Scenes
ORCID iD
Wickramarachchi: 0000−0001−5810−1849
Henson: 0000−0003−3875−3705
Sheth: 0000−0002−0021−5293
Document Type
Article
Abstract
Scene understanding is a key technical challenge within the autonomous driving domain. It requires a deep semantic understanding of the entities and relations found within complex physical and social environments that is both accurate and complete. In practice, this can be accomplished by representing entities in a scene and their relations as a knowledge graph (KG). This scene knowledge graph may then be utilized for the task of entity prediction, leading to improved scene understanding. In this paper, we will define and formalize this problem as Knowledge-based Entity Prediction (KEP). KEP aims to improve scene understanding by predicting potentially unrecognized entities by leveraging heterogeneous, high-level semantic knowledge of driving scenes. An innovative neuro-symbolic solution for KEP is presented, based on knowledge-infused learning, which 1) introduces a dataset agnostic ontology to describe driving scenes, 2) uses an expressive, holistic representation of scenes with knowledge graphs, and 3) proposes an effective, non-standard mapping of the KEP problem to the problem of link prediction (LP) using knowledge-graph embeddings (KGE). Using real, complex and high-quality data from urban driving scenes, we demonstrate its effectiveness by showing that the missing entities may be predicted with high precision (0.87 Hits@1) while significantly outperforming the non-semantic/rule-based baselines.
Digital Object Identifier (DOI)
Publication Info
Reprinted from Frontiers in Big Data, ed. Mayank Kejriwal, Volume 4, 2021.
Copyright © 2021 Wickramarachchi, Henson and Sheth . This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
APA Citation
Wickramarachchi, R., Henson, C., & Sheth, A. (2021). Knowledge-infused Learning for Entity Prediction in Driving Scenes. Frontiers in Big Data, 4. https://doi.org/10.3389/fdata.2021.759110