AI Wheelchair Technology Moves Closer to Reality

Severely disabled wheelchair users can often navigate tight spaces better than most robotic systems. A wave of new research on smart wheelchairs, including results presented in Anaheim, California, earlier this month, is now testing whether AI-based systems can, or should, completely close that gap.

Christian Mandel—senior researcher at the German Research Center for Artificial Intelligence (DFKI) in Bremen, Germany—co-led a research team with his colleague Serge Autexier who developed a sensor-equipped electric wheelchair prototype designed to navigate a room full of potential obstacles. The researchers also tested a new safety system integrating data from wheelchair sensors and room sensors, including those of drone-base color and depth cameras.

Mandel says the team’s smart wheelchairs were both semi-autonomous and autonomous.

“Semi-autonomous is the shared control system in which the person sitting in the wheelchair uses the joystick to drive,” explains Mandel. “The full autonomy is controlled by natural language input. You say: ‘Please take me to the coffee machine’.”

Here’s a close-up of the wheelchair’s joystick and camera.DFKI

The researchers conducted experiments (part of a larger project called Reliable and Explainable Swarm Intelligence for People with Reduced Mobility, or REXASI-PRO) using two identical smart wheelchairs each containing two lidars, a 3D camera, odometers, user interfaces, and an embedded computer.

Unlike semi-autonomous mode, where the participant controls the wheelchair with a joystick, in autonomous mode, control involves the open source ROS2 Nav2 navigation system using natural language input. The wheelchairs also used simultaneous localization and mapping (SLAM) cards and local motion controllers to avoid obstacles.

One scenario Mandel and his team tested involved the user pressing a button on the wheelchair’s human-machine interface, speaking a command, and then confirming or rejecting the instruction through that same interface. Once the user confirmed the command, the mobility device guided them along a path to their destination, while sensors attempted to detect obstacles in the path and adjust the mobility device accordingly to avoid them.

When are smart wheelchairs a bad value?

According to Pooja Viswanathan, CEO and founder of Toronto-based Braze Mobility, research in the field of mobile assistive technologies should also prioritize making these devices easily accessible to everyday consumers.

“Cost remains a major barrier,” she says. “Often, financing systems are not designed to support advanced complementary intelligence unless there is very clear evidence of value and safety. Reliability is another barrier. A smart wheelchair must function not only in ideal conditions, but also in the messy and variable conditions of everyday life. And there is also the dimension of human factors. Users have different cognitive, motor, sensory and environmental needs, so one solution rarely fits all.”

For its part, Braze manufactures blind spot sensors for electric wheelchairs. The sensors detect obstacles in areas that are difficult for a user to see. The sensors can also be added to any wheelchair to turn it into a smart wheelchair by providing multi-modal alerts to the user. This approach attempts to support users rather than replace them.

According to Louise Devinge, a biomedical research engineer at IRISA (Institute for Research in Computer Science and Random Systems) in Rennes, France, the increasing complexity of smart wheelchairs requires more sensing. And this requires careful management of communication and synchronization within the wheelchair system. “The more sensing, computing and autonomy you add,” she says, “the more difficult it becomes to ensure robust performance across the full range of real-world environments that wheelchair users face.”

In other words, in the short term, the field’s biggest challenge is not replacing the wheelchair user with artificial intelligence, but rather designing better partnerships between the user and the technology.

This image shows the data representations used by the 3D Driving Assistant. These include immutable sensor perceptions such as laser scans and point clouds, as well as derived representations such as virtual laser scans and grid maps. Finally, the collection of robot shapes describes the physical limits of the wheelchair at different heights.DFKI

Where will smart wheelchairs go from here?

Mandel says he expects to see smart wheelchairs ready for the consumer market within 10 years.

Viswanathan says the REXASI-PRO system, while beyond the reach of current smart wheelchair technologies, is important in the long term. “This reflects the more ambitious end of the smart wheelchair spectrum,” she says. “Its strengths appear to lie in intelligent navigation, advanced sensing, and the broader effort to build a wheelchair that can interpret and respond to complex environments more autonomously. From a research perspective, this is exactly the kind of work that moves the field forward. It also appears to take seriously the importance of reliable, explainable AI, essential in any mobility technology where safety, reliability, and user trust are paramount.”

Mandel says he’s ultimately looking for the inspiration that led him to get into this field years ago. As a young researcher, he says, he helped develop a smart wheelchair system controllable with a head joystick.

However, Mandel says he realized after numerous trials that the smart wheelchair system he was working on still had a long way to go because, as he puts it, “at that moment I realized that even people with severe disabilities [traveling through] a narrow passage, they did very, very well.

“And then I realized, okay, there is this need for this technology, but never underestimate this [wheelchair users] we can do without it. »

DFKI researchers presented their work earlier this month at the CSUN Assistive Technology conference in Anaheim, California.

This article was supported by the IEEE Foundation and a Jon C. Taenzer Fellowship.