News Google is rolling out Gemini’s real-time AI video features

[News]

Google has started rolling out new AI features to Gemini Live that let it “see” your screen or through your smartphone camera and answer questions about either in real-time, Google spokesperson Alex Joseph has confirmed in an email to The Verge. The features come nearly a year after Google first demonstrated the “Project Astra” work that powers them.

A Reddit user said the feature showed up on their Xiaomi phone, as spotted by 9to5Google. Today, the same user posted the video below demonstrating Gemini’s new screen-reading ability. It’s one of the two features Google said in early March would “start rolling out to Gemini Advanced Subscribers as part of the Google One AI Premium plan” later in the month.


The other Astra capability rolling out now is live video, which lets Gemini interpret the feed from your smartphone camera in real-time and answer questions about it. In the demonstration video below that Google published this month, a person uses the feature to ask Gemini for help deciding on a paint color to use for their freshly-glazed pottery.


Google’s rollout of these features is a fresh example of the company’s big AI assistant lead as Amazon is preparing the limited early access debut of its Alexa Plus upgrade and Apple has delayed its upgraded Siri. Both are supposed to have capabilities similar to the ones Astra is starting to enable now. Meanwhile, Samsung still has Bixby, but Gemini is nevertheless the default assistant on its phones.