Research

I am the director of the SPARK (Sensing, Perception, Autonomy, and Robot Kinetics) group. The mission of my group is to develop theoretical understanding and practical algorithms for safe, robust, and efficient robot perception. For an autonomous vehicle (e.g., micro aerial vehicles, self-driving cars, planetary rovers) operating in an unknown environment, perception is the problem of creating an internal model of the surroundings using sensor data and prior knowledge. As such, perception includes a broad set of robotics and computer vision problems, including object detection and pose estimation, semantic understanding, robot localization and mapping, among others. My group is currently working on the key problems discussed below.

 

Sponsors

Certifiable Perception

Perception algorithms have been increasingly used in safety-critical applications, including intelligent transportation and military endeavors, where algorithmic failures may put human lives at risk. SPARK develops perception algorithms that work in extreme conditions (e.g., in the presence of extreme amounts of outliers due to false detections or sensor malfunction) and provide formal performance guarantees. In particular, we are currently developing a new generation of algorithms, certifiable perception algorithms, that are able to compute a robust world model and assess its correctness in real-time time. Contrary to the brittleness of existing perception methods, these algorithms are “hard to break” and have the potential to attain super-human performance, paving the way to safe and trustworthy autonomy.

High-level Perception and Spatial AI

Situational awareness in complex, unstructured, and dynamical environments requires autonomous agents to build and maintain a multifaceted model of the environment, including both a 3D geometric model (useful for navigation and coordination) and semantic labels (useful to characterize areas of interest and to provide more succinct information to human operators). SPARK is currently investigating metric-semantic perception, with the goal of developing practical algorithms for joint-metric semantic understanding. Joint metric-semantic understanding has the potential to increase robustness of perception by leveraging the redundancy between semantics and geometry, provide opportunity for data compression, and enhance high-level human-robot spatial interaction.

Efficient and Task-driven Perception

Pervasive applications of computational perception in virtual and augmented reality (AR/VR) and miniaturized aerial platforms require perception algorithms to run in real-time on computationally constrained platforms. This issue is exacerbated by the fact that perception has to process high-rate high-dimensional sensor data (e.g., camera images). Research in SPARK has focused on developing a framework for task-driven perception, that selects the most relevant subset of sensor data to complete a control task under given constraints on performance and computation/power budget. Moreover, past research includes algorithm and hardware co-design of the first chip for visual-inertial navigation, that enables a palm-size drone to see and understand the external world and autonomously navigate among obstacles.