AI Powers Perseverance Rover’s Autonomous Journey

In December, NASA took another small, incremental step toward autonomous surface rovers. In a demonstration, the Perseverance team used AI to generate the rover’s waypoints. Perseverance used the AI ​​waypoints on two separate days, traveling a total of 456 meters without human control.

“This demonstration shows how far our capabilities have advanced and expands how we will explore other worlds,” said NASA Administrator Jared Isaacman. “Autonomous technologies like this can help missions operate more efficiently, respond to challenging terrain, and increase science outputs as the distance from Earth increases. This is a good example of teams applying new technologies carefully and responsibly in real-world operations.”

Mars is far away and there is a delay of about 25 minutes for a round-trip signal between Earth and Mars. This means that in one way or another, the rovers are alone for short periods of time.

Delay shapes the route planning process. Here on Earth, rover drivers look at images and altitude data and program a series of waypoints, which are usually no more than 100 meters from each other. The course plan is sent to NASA’s Deep Space Network (DSN), which transmits it to one of several orbiters, which then transmit it to Perseverance. (Perseverance can receive direct communications from the DSN as a backup, but the data rate is slower.)

AI improves navigation on Mars Rover

In this demonstration, the AI ​​model analyzed orbital images from the Mars Reconnaissance Orbiter’s HiRISE camera, as well as digital elevation models. The AI, based on Anthropic’s Claude AI, identified hazards such as sand traps, boulder fields, bedrock and rock outcrops. Then it generated a path defined by a series of waypoints that avoids hazards. From there, Perseverance’s automatic navigation system took over. It has more battery life than its predecessors and can process moving images and driving shots.

There was one more important step before these waypoints were transmitted to Perseverance. NASA’s Jet Propulsion Laboratory has a “twin” for Perseverance called the Vehicle System Test Bed (VSTB) in JPL’s Mars Yard. This is an engineering model that the team can work with here on Earth to solve problems or for situations like this. These engineering versions are common on Mars missions, and JPL also has one for Curiosity.

“The fundamental elements of generative AI hold great promise in streamlining the pillars of autonomous navigation for off-planet driving: perception (seeing rocks and ripples), localization (knowing where we are), and planning and control (deciding and executing the safest path),” said Vandi Verma, a space roboticist at JPL and a member of the Perseverance engineering team. “We are moving toward a day when generative AI and other intelligent tools will help our surface rovers manage kilometer-scale journeys while minimizing operator workload, and flag surface features of interest to our science team by combing through huge volumes of rover images.”

AI is quickly becoming ubiquitous in our lives, appearing in places that don’t necessarily have strong use cases. But it’s not NASA that’s jumping on the AI ​​train. They have been developing automatic navigation systems for some time, out of necessity. In fact, Perseverance’s primary means of driving is its autonomous navigation system.

One thing that prevents fully autonomous driving is how uncertainty increases as the rover operates without human assistance. The longer the rover travels, the more uncertain its position on the surface becomes. The solution is to relocate the rover on its map. Currently, humans do this. But this takes time, including a full cycle of communication between Earth and Mars. Overall, this limits how far Perseverance can go without a helping hand.

NASA/JPL is also working on how Perseverance can use AI to relocate. The main hurdle is matching the orbital images with the rover’s ground-level images. It seems very likely that AI will be trained to excel in this area.

It is clear that AI is set to play a much larger role in planetary exploration. The next Mars rover could be very different from current models, with more advanced autonomous navigation and other AI features. There are already concepts for a swarm of flying drones released by a rover to extend its exploratory reach on Mars. These swarms would be controlled by AI to work together and autonomously.

And it’s not just Mars exploration that will benefit from AI. NASA’s Dragonfly mission to Saturn’s moon Titan will make extensive use of AI. Not only for autonomous navigation while the rotorcraft is flying, but also for autonomous data retention.

“Imagine intelligent systems not only on the ground on Earth, but also in cutting-edge applications in our rovers, helicopters, drones and other surface elements formed with the collective wisdom of our NASA engineers, scientists and astronauts,” said Matt Wallace, director of JPL’s Office of Exploration Systems. “This is the revolutionary technology we need to establish the infrastructure and systems necessary for a permanent human presence on the Moon and take the United States to Mars and beyond.”