CSE Colloquium: Neurosymbolic Reinforcement Learning

Zoom Information: Join from PC, Mac, Linux, iOS or Android: https://psu.zoom.us/j/93161513306?pwd=a2VFYk9MNEZrMTlDNXNJaXhMNEw5UT09 Password: 576476 

or iPhone one-tap (US Toll): +13126266799,93161513306# or +16468769923,93161513306# 

or Telephone: Dial: +1 312 626 6799 (US Toll) +1 646 876 9923 (US Toll) +1 301 715 8592 (US Toll) +1 346 248 7799 (US Toll) +1 669 900 6833 (US Toll) +1 253 215 8782 (US Toll) Meeting ID: 931 6151 3306 Password: 576476 International numbers available: https://psu.zoom.us/u/aHIR3Tu8M 

ABSTRACT: Recent advances in Artificial Intelligence (AI) have been driven by deep neural networks. However, neural networks have certain well-known flaws: they are difficult to interpret and verify, have high variability, and lack domain awareness. These issues create a deficiency of trust and are hence a significant impediment to the deployment of AI in safety-critical applications. In this talk, I will present work that addresses these drawbacks via neurosymbolic learning in the reinforcement learning paradigm. Neurosymbolic agents combine experience based neural learning with partial symbolic knowledge expressed via programs in a Domain Specific Language (DSL). Using a DSL provides a principled mechanism to leverage high-level abstractions for machine learning models. To overcome the challenges of policy search in non-differentiable program space we introduce a meta-algorithm that is based on mirror descent, program synthesis, and imitation learning. This approach interleaves the use of synthesized symbolic programs to regularize neural learning with the imitation of gradient-based learning to improve the quality of synthesized programs. This perspective allows us to prove robust expected regret bounds and finite-sample guarantees for this algorithm. The theoretical results guaranteeing more reliable learning are accompanied by promising empirical results on complex tasks such as learning autonomous driving agents and generating interpretable programs for behavior annotation. 

BIOGRAPHY: Abhinav Verma is a Ph D Candidate at the University of Texas at Austin, where he is advised by Swarat Chaudhuri. His research lies at the intersection of machine learning and formal methods, with a focus on building intelligent systems that are reliable, transparent, and secure. His work builds connections between the symbolic reasoning and inductive learning paradigms of artificial intelligence. He is currently supported by a JP Morgan AI Research Ph D Fellowship.