My research pushes the boundaries of autonomous operation on agile micro aerial vehicles, through the design of robust and lightweight perception algorithms. My work is a combination of rigorous theory and practical implementation: I bring new theoretical tools to the robotics community (e.g., convex relaxations, spectral graph theory, distributed computing, compressive sensing) and demonstrate their revolutionary impact in real-world applications. My previous research provided fundamental insights and performance guarantees in robot localization and mapping. My current and future research will redesign the landscape of sensing and perception for resource-constrained robots, with a special focus on swarms of micro and nano aerial vehicles.

Autonomous robots are becoming a pervasive technology and will provide a critical asset to address some of the major societal challenges over the next decades. Unmanned aerial vehicles (UAV) for crop monitoring and spraying in precision agriculture will enable early disease detection and timely actions on crops, reducing yield loss and helping to cope with the increasing food demand. Fast and agile UAV will deliver medical supplies (e.g., vaccines) in rural areas and in underdeveloped countries, fighting the spread of diseases and providing new opportunities for the global welfare. Autonomous robots capable of dexterously moving among obstacles will be an invaluable support for search and rescue and disaster response. Inexpensive and lightweight platforms will collect a large amount of environmental data, providing new perspectives and actionable understanding of phenomena such as hurricanes and floods. A virtually endless list of applications includes infrastructure inspection, transportation, monitoring, construction, and entertainment, making robotics a multi-billion dollar market in rapid expansion.

A main challenge towards this vision is the design of robust and lightweight perception algorithms, which interpret sensor data into a coherent world representation, enabling on-board situational awareness and high-level decision-making. Perception constitutes a bottleneck in the deployment of robotics systems, and indeed most of the existing robotics applications either rely on low-level supervision by a human (e.g., robots used for bomb disposal or disaster response), or operate in structured environments (e.g., fenced areas in factory floors equipped with external markers/cameras for localization and guidance).

My research aims to bring autonomous robots into the real world. During my Ph.D. I addressed issues related to robust robot perception. Traditional algorithms for localization and mapping are fragile and rely on careful parameter tuning. Moreover, they are prone to failure in off-nominal conditions (e.g., large sensor noise, outliers). I demonstrated that using tools from nonlinear optimization (e.g, convex relaxation, Lagrangian duality), graph theory (e.g., cycle space, spectral graph theory), Riemannian geometry, and probabilistic inference, one can design faster and more robust algorithms, which are less sensitive to parameter tuning and adverse environmental conditions. These algorithms have been implemented in popular robotics software libraries and used by universities and companies. Robust perception algorithms relax the requirement of human supervision, making robot deployment cheaper, and enabling to scale to large teams of cooperative robots in real scenarios.

My current and future research will enable autonomous navigation of resource-constrained platforms, with a special focus on swarms of agile micro (MAV) and nano (NAV) aerial vehicles. Faster operational speed means more efficient task completion, which is crucial in time-critical applications (e.g., search and rescue). Agility, in particular, is a key requirement for indoor operation, pushing towards the adoption of smaller platforms. While the use of multiple small MAVs appears as a desirable alternative to more expensive monolithic solutions, the deployment of these platforms in the real world poses formidable challenges. The limited payload and power impose constraints on the onboard computation and sensing, preventing the use of information-rich sensors, such as lidars and depth cameras. Moreover, the adoption of a large number of vehicles largely limits the bandwidth available to each vehicle. Finally, the use of platforms with fast dynamics requires perception algorithms to operate in a very challenging regime (motion blur, sub-sampled data, high rate and low latency estimation for fast closed-loop control). These challenges require a paradigm shift and open a number of research endeavors, such as dealing with very sparse sensor data (sparse sensing), and designing algorithms that selectively process only sensor data that is relevant to complete a given task (perceptual attention), and that are aware of the on-board resources of the platform (algorithms-and-hardware co-design).

My research will have a broader impact on the robotic ecosystem beyond the micro aerial vehicles domain. The study of resource constrained perception systems will empower bio-inspired robots (e.g., robotic insects) with advanced navigation capabilities. Moreover, it will impact all domains in which sensing is limited (e.g., endoscopic surgery). Finally, it will promote the use of more affordable sensors in safety-critical applications (e.g., self-driving cars), by leveraging a tighter integration of sensing, perception, and control.

Resource-constrained Perception, Sensing and Control

  • [PDF] V. Tzoumas, L. Carlone, G. J. Pappas, and A. Jadbabaie. Sensing-Constrained LQG Control. Technical Report, 2017.
    [Bibtex]
    @techreport{Tzoumas17tr-sLQG,
    Author = {V. Tzoumas and L. Carlone and G.J. Pappas and A. Jadbabaie},
    FullAuthor = {Vasileios Tzoumas and Luca Carlone and George J. Pappas and Ali Jadbabaie},
    Title = {Sensing-Constrained {LQG} Control},
    nonote = {arxiv preprint: 1709.08826,
    \linkToPdf{https://arxiv.org/pdf/1709.08826.pdf}},
    pdf = "https://arxiv.org/pdf/1709.08826.pdf",
    Year = 2017}
  • [PDF] L. Carlone and S. Karaman. Attention and Anticipation in Fast Visual-Inertial Navigation. Technical Report, 2017.
    [Bibtex]
    @techreport{Carlone17tr-attentionVIN,
    Author = {L. Carlone and S. Karaman},
    Title = {Attention and Anticipation in Fast Visual-Inertial Navigation},
    nonote = {arxiv preprint: 1610.03344,
    \linkToPdf{https://www.dropbox.com/s/c19kyrikroypahw/2017j-visualAttention.pdf?dl=0}},
    pdf = "https://www.dropbox.com/s/c19kyrikroypahw/2017j-visualAttention.pdf?dl=0",
    Year = 2017}
  • [PDF] F. Ma, L. Carlone, U. Ayaz, and S. Karaman. Sparse Depth Sensing for Resource-Constrained Robot Perception. Technical Report, 2017.
    [Bibtex]
    @techreport{Ma17tr-sparseSensing,
    Author = {F. Ma and L. Carlone and U. Ayaz and S. Karaman},
    Title = {Sparse Depth Sensing for Resource-Constrained Robot Perception},
    nonote = {arxiv preprint: 1703.01398,
    \linkToPdf{https://arxiv.org/pdf/1703.01398.pdf}},
    pdf = "https://arxiv.org/pdf/1703.01398.pdf",
    Year = 2017}
  • [PDF] Z. Zhang, A. Suleiman, L. Carlone, V. Sze, and S. Karaman. Visual-Inertial Odometry on Chip: An Algorithm-and-Hardware Co-design Approach. In Robotics: Science and Systems (RSS), 2017.
    [Bibtex]
    @inproceedings{Zhang17rss-vioChip,
    Author = {Z. Zhang and A. Suleiman and L. Carlone and V. Sze and S. Karaman},
    title={Visual-Inertial Odometry on Chip: An Algorithm-and-Hardware Co-design Approach},
    booktitle= rss,
    nonote = {\linkToPdf{http://www.roboticsconference.org/static/papers/74.pdf}
    \linkToWeb{http://news.mit.edu/2017/miniaturizing-brain-smart-drones-0712}},
    pdf = "http://www.roboticsconference.org/static/papers/74.pdf",
    year={2017}}

Provably-safe Perception and Navigation

  • [PDF] D. M. Rosen, L. Carlone, A. S. Bandeira, and J. J. Leonard. SE-Sync: A Certifiably Correct Algorithm for Synchronization over the Special Euclidean Group. In Intl. Workshop on the Algorithmic Foundations of Robotics (WAFR), 2016.
    [Bibtex]
    @inproceedings{Rosen16wafr-sesync,
    Author = {D.M. Rosen and L. Carlone and A.S. Bandeira and J.J. Leonard},
    Booktitle = WAFR,
    Title = {{SE-Sync}: A Certifiably Correct Algorithm for Synchronization over the {Special Euclidean} Group},
    nonote = {
    extended arxiv preprint: 1611.00128,
    \linkToPdf{http://arxiv.org/abs/1611.00128}
    \linkToPdf{http://wafr2016.berkeley.edu/papers/WAFR_2016_paper_138.pdf}
    \linkToCode{https://github.com/david-m-rosen/SE-Sync},
    {\color{red} best paper award}},
    pdf = "http://arxiv.org/abs/1611.00128",
    Year = 2016}
  • [PDF] L. Carlone, G. Calafiore, C. Tommolillo, and F. Dellaert. Planar Pose Graph Optimization: Duality, Optimal Solutions, and Verification. IEEE Trans. Robotics, 32(3):545-565, 2016.
    [Bibtex]
    @article{Carlone16tro-duality2D,
    Author = {L. Carlone and G. Calafiore and C. Tommolillo and F. Dellaert},
    Journal = tro,
    Title = {Planar Pose Graph Optimization: Duality, Optimal Solutions, and Verification},
    Volume = 32,
    Number = 3,
    Pages = {545--565},
    nonote = {\linkToPdf{https://www.dropbox.com/s/peoktkct0cw42av/2015j-TRO-dualityPGO2D.pdf?dl=0}
    \linkToCode{https://www.bitbucket.org/lucacarlone/pgo2d-duality-opencode}},
    pdf = "https://www.dropbox.com/s/peoktkct0cw42av/2015j-TRO-dualityPGO2D.pdf?dl=0",
    Year = 2016}