What makes small aerial robots (UAVs) and their coordination beautiful?

The beauty lies in their ability to coordinate and form three-dimensional patterns in mid-air, allowing for dynamic shapes and enhanced creative possibilities. This distributed cooperation, where the whole is greater than the sum of its parts, is a key aspect of their elegance.

How do ants inspire robotic swarm behavior?

Ants are incredibly robust and resilient, both individually and as a population. Their instinct for colony preservation leads to amazing adaptations, like chaining together to cross gaps or recruiting others to move large food items, demonstrating consensus without direct communication.

What are the key challenges for robots to achieve true autonomy without external infrastructure?

True autonomy means navigating without GPS, external communications, or human pilots. Robots need to make decisions based solely on onboard sensors and their understanding of the immediate environment, which is a significant engineering challenge.

Why is flying through constrained spaces difficult for robots?

Flying in constrained spaces is difficult due to complex aerodynamic effects like blade flapping, ground effects, and wall effects, which are hard to model precisely. Robots often adapt to these conditions through iterative learning rather than relying solely on pre-programmed models.

What are the limitations of relying solely on computer vision for autonomous vehicles?

Computer vision struggles in challenging conditions like dark, dirty environments (e.g., mines). Furthermore, achieving extreme accuracy (99.9%+) for safety-critical applications requires vast amounts of data, and the process is very power-hungry.

Are flying cars feasible, and what are the biggest hurdles?

Flying cars are potentially viable, especially with technologies like small jet engines, but a major hurdle is the economic viability and the limitations of battery technology (energy and power density). Electric propulsion might be suitable for short distances, but fossil fuels could make them more practical.

What is the role of humans in future robotic systems?

Humans are ultimately the reason robots exist to solve human problems. Whether commanding, collaborating (like in search-and-rescue), or acting as bystanders, human-robot interaction is a crucial and complex area of study, requiring robots to model human behavior.

What are the biggest open problems in robotics today?

Robots are still largely designed for specific tasks in structured environments. A major challenge is enabling them to operate effectively in broader, unstructured, and changing environments, like the streets of Napoli or Mumbai. We lack a systematic understanding of how to combine environment, task, and robot capabilities.

Key Moments

Vijay Kumar: Flying Robots | Lex Fridman Podcast #37

Lex Fridman

Science & Technology4 min read57 min video

Sep 8, 2019|50,697 views|1,321|69

Save to Pod

Key Moments

On this page

TL;DR

Roboticist Vijay Kumar discusses multi-robot systems, aerial robots, biological inspiration, and the future of autonomous technology.

Key Insights

Multi-robot systems, especially aerial robots (drones), excel in coordinated 3D pattern formation and agile maneuvering in constrained spaces.

Biological systems like ants offer crucial insights into robustness, decentralized control, and emergent behaviors for robot swarms.

True autonomy for robots requires operating without GPS, external communication, or pre-existing maps, relying on local sensing and distributed decision-making.

The advancement of robotics is heavily influenced by commoditized sensors and computing power, seen in the development of quadcopters.

While machine learning excels in robot perception, control and planning still heavily rely on model-based approaches, with hybrid solutions being the future.

Autonomous flight offers advantages over driving due to simpler 3D trajectories, but faces challenges with aerodynamic complexities and 3D world modeling.

THE BEAUTY OF MULTI-ROBOT SYSTEMS AND AERIAL ROBOTS

Vijay Kumar highlights the elegance of multi-robot systems, particularly small unmanned aerial vehicles (UAVs), that can maneuver in constrained spaces and coordinate to form complex 3D patterns. This ability to create and dynamically alter shapes in mid-air represents a significant advancement, moving beyond earlier 2D formations of ground robots. The beauty in this domain, as an engineer, lies in the distributed cooperation and the emergent collective behaviors of these individual agile components.

BIOLOGICAL INSPIRATION FOR ROBUST AND RESILIENT ROBOTICS

Ants serve as a profound source of inspiration for roboticists due to their individual robustness and collective resilience. Individual ants can lose limbs and continue functioning, mirroring the desired robustness in robot components. Furthermore, ant colonies exhibit emergent phenomena like collective problem-solving, consensus-building without direct communication, and self-preservation instincts. These biological models inform the design of robot swarms that can adapt, reorganize, and function as a synergistic whole, far exceeding the sum of their individual parts.

ACHIEVING TRUE AUTONOMY BEYOND EXTERNAL INFRASTRUCTURE

Kumar defines true autonomy as the ability for robots to operate without reliance on external infrastructure like GPS, communication networks, or pre-existing maps. This contrasts with current systems that often depend on these aids, which can be unreliable in environments like urban canyons or through jamming. The challenge lies in enabling robots to navigate and make decisions based solely on local sensing and internal computations, mimicking the self-sufficiency observed in biological organisms and addressing critical limitations of current navigation technologies.

THE TECHNOLOGICAL EVOLUTION OF AGILITY AND UMV DEVELOPMENT

The development of agile autonomous aerial robots has been significantly enabled by advancements in sensor technology, particularly inertial measurement units (IMUs), and increased computing power. The commoditization of components, spurred by industries like automotive (for airbags), has driven down costs and improved performance. This progress, evident in the evolution of quadcopters, has made sophisticated aerial maneuvering, like stable hovering and precise directional control via motor speed regulation, feasible and accessible for research and development.

NAVIGATING COMPLEXITY: MODEL-BASED VS. LEARNING-BASED ROBOTICS

While machine learning, especially deep reinforcement learning, shows immense promise, particularly in robotic perception, Kumar notes that control and planning still heavily lean on model-based approaches. Real-world complexities, such as aerodynamic effects like blade flapping, ground effect, and wall effects, are difficult to model exhaustively. Consequently, a hybrid approach, combining the strengths of data-driven learning with traditional model-based design, is essential for achieving robust and adaptable robotic systems that can handle unforeseen environmental interactions.

AUTONOMOUS FLIGHT AND DRIVING: COMPARATIVE CHALLENGES AND FUTURES

Autonomous flight presents unique advantages over autonomous driving due to the exploitable 3D space, allowing for simpler, pre-programmable trajectories. However, it introduces complexities in modeling and navigating the aerodynamic peculiarities of flight and the 3D environment. Looking ahead, the potential for widespread applications like last-mile delivery is vast, but significant challenges remain, particularly in battery technology for sufficient power density and energy storage. Economic viability and finding solutions beyond lithium-ion batteries are critical hurdles for widespread adoption.

HUMAN-ROBOT COLLABORATION AND THE ETHICS OF AUTONOMY

Kumar emphasizes that robots are ultimately developed to solve problems for humans, making human-robot interaction a crucial aspect. Whether commanding, collaborating, or acting as bystanders, humans are integral. This raises ethical considerations, especially regarding the weaponization of robots and the need for 'good guys' to develop these technologies responsibly. Engineers must engage in discussions about societal impact and develop counter-technologies, akin to defending against swarm attacks, underscoring the necessity for technological literacy among policymakers and the public.

THE FUTURE OF ROBOTICS: BROADER APPLICATIONS AND FUNDAMENTAL PROBLEMS

The biggest open problems in robotics lie in enabling systems to operate reliably in a wider range of unstructured environments. While robots excel in controlled settings like parking lots, scaling to complex, dynamic environments like the streets of Naples or Mumbai remains a significant challenge. This necessitates a systematic understanding of how environmental complexity and task requirements intersect. Furthermore, the field requires deep mathematical foundations and robust representations, not just reliance on readily available software packages, to achieve true intelligence and explainability.

ADVICE FOR ASPIRING ENGINEERS IN RAPIDLY EVOLVING FIELDS

For aspiring engineers in robotics and AI, Kumar advises embracing continuous change, as the landscape will be drastically different upon graduation. He stresses the importance of both specialization and broad learning to adapt to unexpected technological shifts. Crucially, engineers must integrate their technical education with a liberal arts perspective, understanding the societal impact of their work. Strong mathematical foundations and an emphasis on representation are also highlighted as essential, going beyond superficial reliance on existing tools to foster deeper understanding and innovation.

Mentioned in This Episode

●Products

●Software & Apps

●Companies

●Organizations

●Books

●Concepts

●People Referenced

Common Questions

Vijay Kumar's first robot project was a very large hexapod robot, weighing close to 7,000 pounds, powered by hydraulics and controlled by multiple Intel processors. His thesis work focused on coordinating its 18 legs for efficient locomotion.

Topics

Swarm Intelligence AI & Machine Learning Technology & Innovation Future Of AI Human-robot Interaction Autonomous Systems Robotics Engineering Machine Learning In Robotics Aerial Robotics

Mentioned in this video

Concepts

drones

The common term for UAVs, which Vijay Kumar dislikes due to its connotation of being 'dumb' or pre-programmed, preferring the term 'aerial robot'.

UAVs

Unmanned Aerial Vehicles, also referred to as drones, which Vijay Kumar's students have developed for maneuvering in constrained spaces and coordinating in 3D patterns.

Machine Learning

A field of AI where systems learn from data. Vijay Kumar believes it has a role in robotics, particularly in perception, but also in control and planning through iterative learning.

Moore's Law

The observation that the number of transistors on integrated circuits roughly doubles every two years, driving advancements in computing power relevant to robotics.

aerial robot

The preferred term by Vijay Kumar for UAVs or drones, emphasizing their robotic capabilities.

Reinforcement Learning

A type of machine learning where an agent learns to make decisions by trial and error in an environment to maximize rewards.

Deep Reinforcement Learning

A subfield of machine learning that combines deep neural networks with reinforcement learning, applied in various AI domains.

Products

iPhone

Mentioned as a key product released in 2007, a year that marked a tipping point for advances in computing and other technologies relevant to robotics.

Global Hawk

A type of unmanned aerial vehicle, mentioned as an example of a large, winged drone that operates in less restrictive environments compared to agile robots.

Intel 8086

A coprocessor used in conjunction with the Intel 80-85 processor in a large hexapod robot project.

Lidar

A remote sensing method that uses light in the form of a pulsed laser to measure variable distances to the Earth. Used in dark and dusty environments where computer vision struggles.

Raspberry Pi

A small, affordable computer that a high school student can use to build basic robotic functionalities, highlighting the commoditization of low-level robotics.

GPS

Global Positioning System, a common infrastructure for airplanes, but considered a 'brutal' form of information due to its unreliability in certain environments like tall buildings.

Companies

X-N Technologies

A spin-off company Vijay Kumar is involved with that works with robots in underground mines, facing challenges like darkness and dust that hinder computer vision.

Boston Dynamics

A robotics company whose walking robots are often featured in sci-fi inspired fears about robotics.

Toyota

Automaker exploring 'shared autonomy' or 'collaborative autonomy' as a paradigm for human-robot interaction in vehicles.

Kitty Hawk

A company working on electric flying vehicles, mentioned in the context of the dream of flying cars.

Ghost Robotics

A robotics company founded by University of Pennsylvania alumni, mentioned in the context of modern, smaller robotics.

Tesla

Automaker known for its approach to autonomous vehicles, which involves significant human-machine collaboration and supervision.

Weave Robotics

A non-profit organization Vijay Kumar works with, developing drones for delivering supplies in remote areas like the Peruvian Amazon.

Jetoptera

A company that develops small jet engines capable of powering flying vehicles, mentioned as an alternative to electric propulsion for flying cars.

Organizations

University of Pennsylvania

Vijay Kumar's academic affiliation, where he is a professor and former director of the GRASP Lab.

DARPA

Defense Advanced Research Projects Agency, which provided federal funding that indirectly enabled advancements in small UAVs through research into inertial measurement units.

National Academies

An organization that produced a report on counter-UAS (unmanned aircraft systems) technologies, highlighting the threat of weaponized robot swarms.

GRASP Lab

The General Robotics Automation Sensing and Perception Laboratory at the University of Pennsylvania, formerly directed by Vijay Kumar.

Media

The Jetsons

An animated television series from the 1960s, used as a cultural reference for the futuristic dream of flying cars.

Locations

Napoli

A city in Italy, mentioned as an example of a complex urban environment where achieving level 5 autonomy for self-driving cars is currently not feasible.

Mumbai

A city in India, mentioned as an example of a complex environment where achieving level 5 autonomy for self-driving cars is not yet possible.

People

Elon Musk

Founder of Tesla and SpaceX, mentioned in the context of autonomous vehicles and their reliance on computer vision for perception.

Henry Ford

Founder of Ford Motor Company, referenced in the analogy of a structured factory setting to explain the ease of solving problems in highly structured environments like board games.

Vijay Kumar

A professor at the University of Pennsylvania, former director of GRASP Lab, and a leading roboticist known for his work in multi-robot systems, robot swarms, and micro aerial vehicles.

Sebastian Thrun

A researcher in AI and robotics, co-founder of Google X and Udacity, mentioned as someone working on ambitious projects like flying cars.

Software & Apps

IMU

Inertial Measurement Unit, a crucial sensor for quadcopters that measures acceleration and angular velocity, enabling agile maneuvering.

Books

Dostoyevsky

A Russian novelist, mentioned as an example of literature that aspiring engineers should read to broaden their perspective, aligning with a liberal arts approach to engineering education.