Date of Award

Spring 2026

Document Type

Open Access Dissertation

Department

Computer Science and Engineering

First Advisor

Biplav Srivastava

Abstract

As Artificial Intelligence (AI) systems become increasingly powerful and widely accessible, trustworthiness remains a major barrier to their adoption in critical domains such as finance and healthcare. With the advent of large language models (LLMs) that can perform multiple tasks with strong empirical performance, users increasingly rely on these black-box AI services to obtain recommendations, analyze text, and support everyday decisions. However, real-world incidents and empirical studies show that these systems can be overly sensitive to small changes in input, producing outputs that shift substantially under minor perturbations. In addition, models have been reported to exhibit stereotypical behavior and systematic bias with respect to sensitive or protected user attributes. Such instability and bias can erode user confidence and trust. Because these systems are often black boxes, their behavior is difficult to interpret, and existing explanation techniques are frequently one-of, instance-level, and not designed to support multiple stakeholders. Moreover, many deployed applications are not single models but composite systems in which multiple components are chained together to deliver one capability. When these models are chained together, it is not enough for each component to be robust on its own, instability and bias can compound as they interact.

Robustness, in this dissertation, encompasses both stability under small input changes and fairness with respect to protected attributes. Existing robustness assessment methods are often fragmented, addressing only a subset of these concerns, and there is limited support for a single framework or tool that can quantify and communicate model behavior across multiple dimensions. This dissertation introduces a causality-based rating method that quantifies and communicates black-box model behavior using a causally grounded experimental setup. The method not only measures the extent to which a model exhibits bias or instability, but also supports reasoning about why these behaviors occur under different input conditions. The resulting measurements are translated into comparable measures called ratings, allowing end- users to compare AI models and select the most suitable model for a given task using the data available to them. Further, these ratings can be combined with traditional eXplainable AI (XAI) techniques to provide more holistic support for different stake- holders. Finally, the ratings can serve as an objective for constructing composite AI models from a set of primitive models.

This dissertation applies the causality-based rating method to assess robustness across multiple domains and model families, including sentiment analysis systems (SASs), machine translation systems, and time-series forecasting models (TSFMs), spanning statistical approaches as well as multimodal and foundation model-based methods. The method is extended to support explanation by integrating it with traditional XAI techniques, yielding a Holistic eXplainable AI (H-XAI) framework. The dissertation further integrates the method with the probabilistic planning framework RDDL to construct robust and efficient composite AI systems, including composite chatbots that combine rule-based and LLM-based components. This setup allows developers make clear trade-offs between fairness, utility, and computational cost. The approach is shown to be scalable and efficient in practice. In addition, user studies demonstrate that ratings reduce the difficulty users face in understanding robustness properties and that they can aid users in constructing efficient composite AI models.

The rating method quantifies statistical and confounding bias in addition to sensitivity to perturbations, and hence multi-faceted. It is task-independent and extensible, and it is designed to be usable even by stakeholders who are not deeply versed in AI but who need to choose an AI service for a specific task, an increasingly common scenario as AI is offered through LLM-based assistants such as ChatGPT and Gemini. To make the method accessible to end-users, this dissertation presents the ARC (AI Rating through Causality) tool, which showcases the method on binary classification, sentiment analysis, time-series forecasting, and group recommendation tasks. ARC also allows users to explore trade-offs between robustness and predictive performance through Pareto analysis, and it allows users to assess their own models using the test data they have under the fixed causal setup introduced by the rating method. This work has also led to a forthcoming Springer Nature book titled Assessing, Explaining, and Rating AI Systems for Trust, With Applications in Finance.

Overall, the causality-based rating method supports trustworthy AI deployment by allowing stakeholders, especially AI developers, to select appropriate models based on the data available, and to build robust and efficient composite AI models from primitive components. It provides a principled, practical approach for assessing, explaining, and constructing trustworthy and robust AI systems in real-world settings.

Rights

© 2026, Kausik Lakkaraju

Download

Share

COinS