Perception and Navigation
At the Robotic Systems Lab, we have extraordinary robots that can locomote in challenging environments. However, to carry out autonomous missions, a high-level understanding of the environment and reliable pose and state estimation are necessary. These perception and navigation solutions are crucial for enabling the robots to operate effectively in complex and dynamic environments. In our group, the core aspects we focus on are:
Navigation
Outdoor
Legged robots possess remarkable agility, making them ideal for tackling challenging outdoor environments. These environments often feature intricate geometries, non-rigid obstacles such as bushes and grass, and demanding terrain, including muddy or snowy surfaces. Achieving full autonomy in these settings requires the development of navigation algorithms that consider terrain geometry, properties, and semantics. Our research is dedicated to addressing all aspects of navigation, encompassing both local and global path planning, as well as traversability analysis. Our planning methodologies employ sampling-based and end-to-end trained methods as well as reinforcement learning agents to ensure safe navigation through these complex terrains.
Contact:
external page Fan Yang ()
external page Pascal Roth ()
external page Julia Richter ()
external page Manthan Patel ()
Indoor
Indoor mapping and navigation are fundamental to building capable home robots. Unlike outdoor environments, indoor spaces are generally more structured and semantically rich - filled with objects, rooms, and contextual relationships that can guide intelligent behavior. Our research focuses on constructing semantics-rich representations of these environments, such as scene graphs and open-vocabulary maps, which capture not just geometry but also the meaning and function of spaces and objects. Building on this, we explore object-goal navigation: enabling robots to locate and navigate to specific objects in previously unseen environments. Ultimately, we aim to integrate language into this process, advancing toward vision-language navigation, where robots follow long-horizon natural language instructions from the user.
Contact:
external page Kaixian Qu ()
Changan Chen ()
Perception
SLAM and Spatial Mapping
We study multi-modal SLAM (vision, IMU, LiDAR) for legged robots in feature-sparse, mixed indoor–outdoor settings. Our focus is on detecting and mitigating observability loss to preserve map integrity and produce smooth, low-latency state estimates for planners (e.g., terrain traversability, foothold selection). We fuse all modalities with novel processing engines to handle lighting changes and faulty measurements. Alongside point-cloud maps, we build neural spatial representations such as Neural Radiance Fields and explicit scene representations such as 3D Gaussian Splatting for compact, view-consistent reconstructions that we use directly for downstream tasks and scene understanding (semantics-aware traversability, contact reasoning, uncertainty-aware planning).
Contact:
external page Turcan Tuna ()
State Estimation
To perform complex tasks in complex environments with limited resources and sensors, today's robots require efficient, accurate, and robust state estimation. We are leveraging the continuous-time paradigm and other modern advancements to create flexible, generalized estimators for odometry, localization, calibration, and SLAM. These representations, such as ‘exactly sparse’ Gaussian processes and splines, enable the seamless integration of high-frequency measurements from asynchronous sensors without expanding the state space or introducing simplifying assumptions, which renders existing solutions fragile.
Contact:
external page William Talbot ()
Extended Reality
To enable easier human-robot interaction, we utilize an extended reality toolkit capable of displaying various sensor feeds and transmitting control signals from multiple XR devices to various robotic platforms. Building upon the OpenXR standard enables us to use various AR and VR headsets to natively display sensor feeds, including RGB, depth, and stereo images, point clouds, reconstructed meshes, navigation paths, haptic information, and robot state information. Additionally, the use of OpenXR input devices enables us to send controls directly using hand gestures, controller and joystick inputs, pose interactions, and headset and hand positions. The open-source, modular system is designed for rapid deployment and easy teleoperation with any ROS platform and XR device or computer.
Contact:
external page Maximum Wilder-Smith ()
Datasets and Sensor Suite
Our robots and external sensor suits are based on a multimodal measurement principle: RGB, depth, LiDAR, and IMU, with strict calibration, time synchronization, and outlier modeling. Because noise/ bias are only partially captured in simulation, we stress-test on mixed indoor–outdoor runs with feature scarcity and lighting extremes. We extend the suite with novel sensors, including radar, FMCW LiDARs, and solid-state LiDARs, to enhance robustness in adverse conditions and achieve compact form factors. We also push the boundaries of active perception through the investigation of motorized solutions, such as a Pan–Tilt Unit (PTU), enabling contextually informed smart perception (closed-loop viewpoint control) to recover observability, reduce degeneracy, and disambiguate occlusions for downstream tasks and scene understanding. Representative datasets:
external page TartanGround (Patel et al.)
external page GrandTour (Frey & Tuna et al.)
Contact:
external page Turcan Tuna ()
external page Manthan Patel ()
