Google announces 'Project Mariner' that can automatically operate Chrome with AI



On December 11, 2024 local time, Google announced Project Mariner, an AI that can understand and automatically operate information on a browser. With Project Mariner, you can automatically perform complex operations such as 'searching and compiling email addresses of companies based on the company names compiled in a spreadsheet.'

Project Mariner - Google DeepMind

https://deepmind.google/technologies/project-mariner/

Google introduces Gemini 2.0: A new AI model for the agentic era
https://blog.google/technology/google-deepmind/google-gemini-ai-update-december-2024/




Project Mariner is an AI assistant that can perform complex operations according to user instructions. Users can simply give instructions in natural language, such as 'Perform XX based on the XX information in this spread.' When the user gives instructions, the instructions and a screenshot of Chrome are sent to Gemini in the cloud, and cursor operations, searches, form input, etc. are automatically performed based on the results of Gemini's analysis.

The video below is a demo of Project Mariner. By simply inputting the tasks you want Project Mariner to complete, the AI will automatically understand and analyze the tasks and complete them in order.

Project Mariner | Solving complex tasks with an AI agent in the Chrome browser [full length] - YouTube


In the video, Project Mariner is shown a Google spreadsheet and given the following instructions: 'Memorize this list of companies. Then find their websites and find email addresses to contact them. Remember this for later.'



Then, Project Mariner started taking screenshots.



Automatically search the web for these companies' sites.



Once you find the website of the company you're looking for, look for an email address to contact them on the site.



Repeat this process for each company in your list.



Once you have completed this process, the contact email address for each company will be displayed.



Project Mariner will also be equipped with

Gemini 2.0 , which was announced on December 11, 2024, and will utilize Gemini 2.0's advanced natural language understanding and inference capabilities to interpret both typed and voice requests.

As an example, Google engineer Adi Osmani said, 'A user can simply ask, 'Find jobs near me,' and Project Mariner will understand that request, navigate to relevant job sites, and customize the search based on the user's location and preferences.'




According to Google, Project Mariner achieved a high score of 83.5% on WebVoyager, which tests the performance of AI agents on real-world web tasks. Google said about this result, 'While AI tasks are not always performed accurately or quickly, this shows that it is becoming technically possible for AI agents to perform tasks in the browser.'




Project Mariner places great importance on security, restricting users to operate only within active tabs to ensure that they understand what Project Mariner is doing, and prompting users for final confirmation before performing certain sensitive actions such as purchasing products. In addition, actions that may directly affect the user's rights or property, such as 'entering credit card numbers or billing information,' 'accepting website cookies,' and 'agreeing to terms of use,' are restricted.

In addition, even in the case of a prompt injection attack by a third party, Project Mariner has been trained to prioritize instructions from the user, making it difficult to follow malicious instructions from an external source. This makes it harder for users to fall victim to scams and phishing attempts, even if malicious instructions are hidden in emails, documents, or websites.

According to Google, at the time of writing, Project Mariner is being tested by trusted testers, and a waiting list for testers is also available.

Project Mariner Trusted Tester Waitlist
https://docs.google.com/forms/d/e/1FAIpQLSe2J4BvD48E-57giEiXIDz_yZeqGmX0Q3AvvR_LfzpRat2kGQ/viewform

Related Posts:

in Software,   Web Application,   Video, Posted by log1r_ut