News Gemini is about to get better at understanding what’s on your phone screen

[News]

The dream of contextual search is alive. | Photo by Amelia Holowaty Krales / The Verge

Google is updating Gemini on Android to let its AI better tap into what’s on your screen. The update should allow Gemini to lean into one of its best use cases, helping you make sense of a limited set of data as you go about your day.

If you set Gemini as the default assist on your Android phone, it can already summarize or answer questions about a webpage or a screenshot. Soon, it’ll also be able to tell if there’s a video on your screen and prompt you to ask questions about it. Gemini uses the video’s automatic captions to find answers — something you could already get it to do in a more roundabout way.

Gemini will also take a similar cue if you’re looking at a PDF, but there’s a catch: you’ll need access to Google’s paid version,...

Continue reading…