Scalable Multi-Modal Perception for Mobile Robots Across Environments

Abstract 

Reliable perception is essential for robots operating in complex real-world environments, yet single-modality sensing and standalone models often fail under occlusion, noise, and domain shift. This talk presents my work on AI-enabled 3D robot vision systems that couple learning-based models with multi-modal sensing and deployment-oriented pipeline integration. I will discuss model and pipeline designs for mapping, localization, scene understanding, and depth and structure estimation, and show how system-level integration improves robustness and real-world reliability under various complex environments. The talk spans applications in autonomous navigation, mobile sensing, and agriculture and life science settings, highlighting multi-modal 3D robot perception as a foundational layer for modern embodied AI.

Bio

Dr. Guoyu Lu is an associate professor in the School of Computing at SUNY Binghamton University. His research interests span robotics, 3D computer vision, AI and machine learning, and AI-enabled perception systems, with applications in autonomous systems, agriculture, life sciences, and large-scale sensing and infrastructure systems. His research has been supported by federal agencies, including NSF, USDA, the Army Research Laboratory, the Air Force Research Laboratory, the Georgia Department of Agriculture, and companies such as Ford, General Motors, Qualcomm, Tencent, and others. Dr. Lu has received multiple honors, including the NSF CAREER Award, USDA New Investigator Award, Aharon Katzir Young Investigator Award (INNS), Ford URP Award, Tencent Rhino-Bird Young Faculty Award, Scialog Fellowship, etc. He has served as a Visiting Faculty at the Air Force Research Laboratory and a Visiting Scholar at Auckland University of Technology. Dr. Lu currently serves as Co-Chair of the IEEE Robotics and Automation Society Technical Committee on Agricultural Robotics and Automation. He also serves as Associate Editor and Area Chair for leading journals and conferences in AI, robotics, and computer vision, such as KBS, IEEE RAL, JEI, CVPR, ICRA, and AAAI.