Apple Intelligence Brings Accessibility Updates Across iPhone, Mac and Vision Pro

Apple leverages AI to improve its accessibility features.

The company on Tuesday shared a handful of updates designed to make its products more useful to people with various disabilities. Apple Intelligence, the tech giant’s suite of AI capabilities, powers some of these new capabilities. The unveiling comes on Thursday to mark World Accessibility Awareness Day, which aims to boost digital access and inclusion.

Updates will arrive for features like VoiceOver, Magnifier, and video captions later this year. They come as more companies – from Google to Microsoft to Amazon – leverage AI to make their products and platforms accessible to a wider range of people. Many of Apple’s updates build on features announced last year. They also come ahead of the company’s Worldwide Developers Conference, scheduled for June 8 and expected to pave the way for more AI updates.

VoiceOver and Magnifier support more detailed descriptions

Apple’s VoiceOver and Magnifier features can describe on-screen information or real-world settings, and Apple Intelligence pushes each of them to be more useful.

VoiceOver is a screen reader that speaks out loud what’s on your screen and can help people who are blind or visually impaired navigate their device. The Picture Browser in VoiceOver will now give more detailed descriptions of the contents of objects such as photographs, scanned invoices, and personal files.

And as part of a Live Recognition update, you can use the Action button on your iPhone to quickly gather more information about your surroundings. Simply point the camera at what you want to learn more about and ask follow-up questions for more details.

Magnifier allows visually impaired people to zoom, view high-contrast text, and detect objects around them on an iPhone, iPad, or Mac. You can now ask questions about what your camera sees. For example, you can point your phone at a recipe and ask how many servings it makes or how long to cook it, and it will find and share that information using large, high-contrast text.

Voice control supports natural language

Apple Intelligence will also support natural language using voice control, which lets you navigate your iPhone or iPad with your voice. This means that if you want to click on something, you no longer need to pinpoint the location using a numbered grid on the screen. Instead, you can just say what you want to select, like a specific file name or folder color. You can also tell your device where you want to zoom in by indicating the word or image you want to zoom in on.

Voice Control with Apple Intelligence will be available in English in the United States, Canada, United Kingdom and Australia.

Accessibility Reader is more customizable

Apple released Accessibility Reader last year on iPhone, iPad, Mac and Vision Pro. It lets you customize the text and adjust the font, color, and spacing, making it easier to read for some people, like those with low vision or dyslexia.

Now Accessibility Reader can work on more complex documents such as scientific articles, which tend to format text in multiple columns among multiple images and tables. You can also remove headers and page numbers for a simpler interface and generate summaries for a quick overview before you get started. Additionally, you can translate the text into the language of your choice.

Subtitles are coming to more videos

Closed captioning may be almost ubiquitous in television shows and movies, but it’s not as common in some media like home videos. Now, generated captions automatically display transcriptions of spoken audio when captions or subtitles are not already available. This includes videos recorded from an iPhone, sent by friends and family, or streamed online.

On-device voice recognition generates captions privately. They will appear in uncaptioned videos on iPhone, iPad, Mac, Apple TV and Vision Pro. You can customize things like text font and background in the video playback menu or settings. The generated subtitles will be available in English in the United States and Canada.

Apple Vision Pro adds new wheelchair control feature

A new eye tracking feature on Vision Pro allows users to control their electric wheelchair – no joystick required. They can use their gaze to select controls and determine how and where their wheelchair moves. This can be particularly useful for people with limited mobility.

Eye tracking on Vision Pro does not require regular calibration and can work in a variety of lighting conditions, according to Apple. The feature will be compatible with Tolt Technologies and Luci alternative driving systems in the United States, with support for Bluetooth and wired accessory connections. Apple says it will work with developers to expand access to more wheelchairs.

“Over the past decade, we have seen eye-driven wheelchair systems evolve in incredible ways,” said Blair Casey, CEO of Team Gleason, in a statement. “Taking advantage of Apple Vision Pro’s eye-tracking capabilities in this way is a huge step forward. The Gleason team is proud to support this initiative and is excited to see the benefits it can bring to people who use electric wheelchairs.

Wider availability of Hikawa iPhone grips and mounts

Apple released the Hikawa Phone Grip and Stand late last year for the iPhone. It is an adaptive and ergonomic accessory that was designed using feedback from people with a variety of disabilities, including those affecting dexterity, grip and hand control. Now, as part of a collaboration between Hikawa and PopSockets, the accessory is on sale for the first time worldwide and in three new colors. It’s available today on Apple’s website.