A Talking Robot Guide Dog Could Change How Visually Impaired People Navigate

Jake Juettner, a junior at the Thomas J. Watson College of Engineering and Applied Science, pursing a Masters in Science for Computer Science, demonstrates a robotic service dog he and other members of associate professor Shiqi Zhang team is programming, April 7, 2026. (Credit:
Binghamton University, State University of New York)

What if you could ask your guide dog where the nearest water fountain is and hear it answer back, complete with directions and an estimated walk time? Researchers at the State University of New York at Binghamton have built a robotic guide dog that can do something close to that, holding simple back-and-forth conversations about navigation with its handler, describing the surrounding environment, and talking through route options as it leads the way.

Real guide dogs are incredible companions, but they can only respond to a handful of short commands like “forward” or “left.” They can’t tell a person what’s around them or explain that reaching the kitchen means passing through two doors. And the supply problem is staggering: only about 2% of visually impaired people in the United States use guide dogs, partly because breeding and training takes years and fewer than half the dogs in training actually graduate. In China, the gap is even wider, with roughly 400 guide dogs serving more than 10 million visually impaired people.

Binghamton’s team set out to change that by giving a four-legged robot something no biological guide dog has: the ability to explain routes in words. Their work, presented at the 40th Annual AAAI Conference on Artificial Intelligence, pairs a large language model, a system that understands and generates language, with a navigation planner. Together, the two let the robot understand open-ended requests, suggest destinations, and adjust plans on the fly.

How a Robotic Guide Dog Learns to Talk

Two tools work in tandem to make this possible. A large language model handles conversation, interpreting what the handler says through a speech-to-text model, asking follow-up questions when a request is vague, and delivering responses aloud via text-to-speech. A route planner handles the logistics, calculating the step-by-step path the robot needs to take, including travel time and any doors along the way.

Say a handler wearing a headset says, “I’m thirsty.” Rather than picking a random destination, the robot identifies relevant options from places it already knows about and runs its planner in the background. According to the paper, the system then generates a response along these lines: “We can go to the kitchen or the water fountain. The kitchen requires opening one door and will take about three minutes. The water fountain has no doors and will take about one minute. Where would you like to go?” The handler makes a choice, and off they go.

Researchers call this “plan verbalization,” where the robot translates its internal route calculations into spoken language. Once the pair starts moving, a second feature kicks in: “scene verbalization.” As the robot crosses into new areas, approaching a door or entering a corridor, it announces what’s happening in real time, helping the handler build a mental map of a space they can’t see.

Testing the Talking Robot Guide Dog With Real Users

To evaluate the system, researchers recruited seven legally blind individuals ranging in age from 40 to 68, two of whom had prior experience with real guide dogs. Participants navigated an indoor office environment while the robot guided them. For safety, an expert operator controlled the robot’s physical movements remotely; the robot wasn’t yet navigating on its own. That setup let the team focus on how well the conversation features worked.

Each participant tried three setups: minimal verbal interaction during the walk, scene descriptions only, and the full system combining route information before departure with scene descriptions along the way. Full-system scores led across the board in this small study, hitting 4.83 out of 5 for usefulness and 4.50 for ease of communication. Participants using the full system were also most likely to say they’d prefer the robot over a real guide dog, though preference scores across all conditions stayed moderate.

One notable wrinkle: the full system scored slightly lower on perceived safety (3.83 vs. 4.00 for the other conditions). Participant feedback suggested this had nothing to do with the robot being dangerous. Walking alongside a robotic animal was simply new territory for most people.

Robotic Guide Dogs Hold Up Against Real-World Noise

Beyond the in-person trials, the team ran a simulation drawing on 77 navigation requests from 16 university students, ranging from direct (“I want to go to the bathroom”) to vague (“I want to sit down and rest”). Using GPT-4 to simulate a visually impaired user, researchers tested whether the system could determine where a person wanted to go from indirect language alone. This kind of simulation doesn’t perfectly reflect how real people speak, but with clarifying questions allowed, the system correctly identified the intended destination 94.8% of the time.

Researchers also stress-tested the system against garbled speech, simulating the errors that crop up in noisy real-world settings, with heavy simulated speech errors where nearly one in three characters could be distorted. Even under those harsh conditions, accuracy dropped by only about 5 percentage points. A simpler keyword-based system, by contrast, essentially collapsed under the same noise.

When the robot shared navigation details upfront, including distances and door counts, users consistently picked shorter, more efficient routes. Conversations ran a bit longer, but overall task time dropped because people made smarter choices.

Millions of visually impaired people worldwide will never have access to a trained guide dog. A robot that can hold a navigation conversation might be the next best option, and in some situations, a practical alternative.