Google Lens now has video search functionality and the ability to ask questions by voice
Google Lens now has a video search function and a feature that lets you ask questions by voice about what you're shooting. This means that people who are registered with Google's experimental feature '
Google Lens Voice Input: How to search with your voice in Google Lens
https://blog.google/products/search/google-lens-voice-search/
Google updates: AI-Organized Search, Google Lens, and more
https://blog.google/products/search/google-search-lens-october-2024-updates/
Google Lens now lets you search with video - The Verge
https://www.theverge.com/2024/10/3/24259759/google-lens-search-video-ai-launch
Google announced the 'Video Understanding Feature' of Google Lens at the developer conference Google I/O in May 2024. A demo video of this feature starts at about 57 seconds in the video below.
Search in the Gemini era | Google I/O 2024 - YouTube
With the video understanding function, for example, you could hold your smartphone over a fish at an aquarium and record it swimming, while asking it a question: 'Why is that fish swimming with me?' and the Gemini AI model would then generate an answer.
If you've opted in to test AI features in Search Labs, you can hold down the shutter button in Google Lens to record a video and ask a question by voice, or tap the microphone icon in a search result to ask a different question.
Rajan Patel, vice president of engineering at Google, explained the feature: 'We capture video as a series of image frames and feed them into a custom Gemini model. The AI then makes sense of the frames in sequence and generates a web-based response.'
You can't yet ask questions about sounds in a video, such as recording a bird's call and asking what kind of bird it is, but Google says it's testing such a feature.
Google Lens' video understanding feature is rolling out to Android and iOS users globally, but voice questions are only supported in English so far.
Related Posts:
