Knowledge Graph Empowered Machine Learning Pipelines for Improved Efficiency, Reusability, and Explainability

Author(s)

Revathy Venkataramanan, University of South Carolina - ColumbiaFollow
Aalap Tripathy, Hewlett-Packard EnterpriseFollow
Martin Foltin, Hewlett-Packard EnterpriseFollow
Hong Yung Yip, University of South Carolina - ColumbiaFollow
Annmary Justine, Hewlett-Packard EnterpriseFollow
Amit Sheth, University of South Carolina - ColumbiaFollow

Document Type

Article

Abstract

Artificial intelligence (AI) pipelines are complex, heavily parameterized, and expensive to execute in terms of time and computational resources. Consequently, it is onerous to run experiments with all possible parameter combinations to achieve an optimal solution. However, these AI experiments can be optimized by recommending relevant parameters to commence the experiments, reducing search space significantly, which can be fine tuned further. The relevant parameters can be identified by observing the metadata of pipelines executed in the past, and the relevant pipeline with relevant parameters can be recommended to the user. Currently, there are various metadata frameworks that automatically record the metadata of AI pipelines. Developing a recommendation system requires understanding pipeline metadata components and their interactions. There is a need to represent the metadata generated by these AI pipelines that capture the relationship among these pipeline entities. This article presents a knowledge-infused recommender that utilizes prior knowledge and metadata of already executed pipelines represented using the proposed metadata schema to recommend a relevant pipeline per user queries. Unlike black-box models, the use of knowledge graphs makes recommendations explainable, improving transparency and trustworthiness for the users.

Digital Object Identifier (DOI)

https://doi.ieeecomputersociety.org/10.1109/MIC.2022.3228087

Publication Info

Postprint version. Published in IEEE Internet Computing, Volume 27, Issue January/February, 2023, pages 81-88.

© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

APA Citation

Venkataramanan, R., Tripathy, A., Foltin, M., Yip, H. Y., Justine, A., & Sheth, A. (2023). Knowledge graph empowered machine learning pipelines for improved efficiency, reusability, and explainability. IEEE Internet Computing, 27(01), 81–88. https://doi.org/10.1109/MIC.2022.3228087