Google has given one step further in the integration of artificial intelligence in everyday life by expanding the capabilities of Gemini Live, its conversational assistant with real-time features. This evolution now allows users to interact with AI not only through voice or text, but also through what their mobile phone's camera captures or by sharing the device's screen. The new system thus seeks to offer contextualized responses depending on the user's visual environment.
More natural interaction thanks to the camera and screen
La latest news from Gemini Live takes a step further in the idea of creating an assistant that truly interacts with the world around it. And with its newly announced features, Users can now focus the camera on different elements in their environment - such as a messy drawer, a defective object, or an item of clothing - and chat directly with Gemini to receive suggestions or resolve questions. It's also possible to share what appears on the mobile screen, which allows the AI to better understand the context of the interaction. By sharing the screen, users can view quick solutions across platforms, thus optimizing the experience.
This functionality is based on the advances of the Project Astra prototype, presented at the event Google I/O last year, which aimed to create a multimodal assistant capable of processing text, images, and videos simultaneously. By leveraging this technology, Gemini It becomes a useful tool for multiple tasks such as shopping, content creation, technical troubleshooting, or simply receiving organized guidance about the environment.
This feature is already available on select compatible Android devices: we are specifically talking about the Google Pixel 9 y Samsung Galaxy S25, both in its standard versions and in the Pro and Ultra variants, although it's expected to reach more phones later. Gemini Advanced subscribers have immediate access, although the feature is starting to roll out to some free users as well.
Practical uses of Gemini Live
The utility of the real-time function is expands and diverse. Notable uses include showing a messy closet or shelf and asking for advice on better organizing the space. Gemini can suggest how to classify objects, optimize order, or even indicate which items you could donate or discard.
On the creative side, users can also share images or designs with Gemini to receive ideas on how to improve a model, create a decoration, or write an original text. Whether showing inspiring photos or sharing a project on screen, AI can act as a source of inspiration for artists, students, or content creators, for example.
also can offer assistance in resolving technical issuesFor example, a user might focus on a squeaky chair or an unresponsive device. Gemini analyzes what it sees and provides quick suggestions that could help solve the problem without having to search for tutorials or help elsewhere.
Another use is to use Gemini as style advisor or for shoppingUsing the camera, you can show clothing items to ask for ideas on how to style them, or by sharing your screen while browsing online stores, the assistant can compare products and prices and offer advice based on the visual content.
Last but not least, Gemini can also act as a kind of visual guideBy pointing the camera at a building, plant, or animal, you can ask the AI what it is, which can be useful when traveling. This feature is reminiscent of tools like Google Lens, but taken to the next level with interaction.
The conversation with Gemini Live is also not limited to English.With support for over 45 languages, users can speak naturally in their preferred language while sharing images or information from their device, enabling more fluid and intimate communication, adapted to different cultural contexts and even forms of expression.
Technical operation and privacy
Google has implemented measures to protect user privacy while using these featuresFor example, the camera is only activated when you're in an active conversation with the AI, and it automatically turns off if the app is minimized or the screen is locked. The same goes for the screen sharing option, which requires manual activation every time you restart the session. You won't have to worry about that.
Furthermore, the mode of use is simpleSimply open the Gemini app, tap the audio icon to start the conversation, and then choose between screen sharing or activating your phone's camera. Once this is done, you can freely ask Gemini questions about what's happening in your surroundings, receiving real-time responses tailored to the situation.
The terms and conditions of use Users must be over 18 years of age and reside in countries where the feature is enabled. A stable internet connection is also required, as understanding live images and video requires continuous communication with Google's processing servers. Please keep this in mind.