Document Type
Article
Abstract
Reinforcement learning systems are commonly adapted to new settings by retraining or fine-tuning policies. This default is costly, difficult to audit, and poorly aligned with structured requirement changes such as revised safety rules, new operational constraints, or updated user preferences. We argue for an alternative abstraction: adaptation via edits to an external, human-readable specification that the agent consults at execution time. We propose conditioning decision-making on an editable knowledge graph encoding (i) rules capturing action applicability and high-level effects, (ii) hard constraints defining feasibility, and (iii) soft preferences shaping tradeoffs among feasible behaviors. Requirement changes become graph edits, not policy rewrites, operationalized through constraint-based action shielding and preference-driven objective shaping. This enables immediate, auditable behavior updates with zero or minimal gradient-based adaptation. We outline an edit-based evaluation protocol and highlight open problems in specification design, edit inference, and guarantees.
Publication Info
Preprint version Association for the Advancement of Artificial Intelligence, 2026.
© 2026, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.
APA Citation
Khandelwal, V., Yip, H., & Sheth, A. (2026). Toward Neurosymbolic Reinforcement Learning via Editable Toward Neurosymbolic Reinforcement Learning via Editable Specifications Specifications Toward Neurosymbolic Reinforcement Learning via Editable Specifications.