Apr 17, 2023 21:00:00

``Web LLM'' that can run chat AI only with the WebGPU of the browser and can use Japanese, how to actually try it is like this

Interactive AIs such as ChatGPT and Bard that can converse with humans based on large-scale language models have appeared, but these interactive AIs require enormous computing resources to operate, and usually AIs can operate. It enables interaction by accessing the server. Interactive AI ' Web LLM ' based on

Vicuna-7B , which fine-tuned LLaMA, a large-scale language model developed by Meta, accesses the server by using WebGPU that enables GPU calculation on the browser. It is characterized by being able to access the interactive AI on the browser without any need. Since it is said that it also supports Japanese, I actually tried interacting with Web LLM in Japanese.

GitHub - mlc-ai/web-llm: Bringing large-language models and chat to web browsers. Everything runs inside the browser with no server support.
https://github.com/mlc-ai/web-llm

To experience Web LLM, you need to use a 'WebGPU compatible browser' on a 'PC with NVIDIA GPU'. However, WebGPU has just been reflected in Chrome, and at the time of writing the article, it must be accessed from Chrome version 113 beta or later, or Chrome Canary, which is a developer build.

This time it will be accessed from Chrome Canary. Chrome Canary is distributed on the following page.

Chrome Canary Developer Features - Google Chrome
https://www.google.com/intl/ja/chrome/canary/

Click 'Download Chrome Canary' on the distribution page to download the EXE format installer (file size 1.30MB).

When you start the downloaded installer, the installation will be performed automatically.

When the installation is finished, Chrome Canary will start.

A demo of Web LLM can be accessed here.

WebLLM | Home
https://mlc.ai/web-llm/

'Chat Demo' at the bottom of the above page is a demo of Web LLM. First, click 'Send' without entering anything.

Then the following message is displayed and the model download starts.

Ready when '[System Initialize] All initialization finished' is displayed. I entered 'Hello, can you speak Japanese?' in the input field, clicked 'Send', and suddenly spoke in Japanese.

'Yes, I can speak Japanese. Please listen to me.'

First of all, I asked what kind of site GIGAZINE is. They seem to recognize that it's a website, but I'm concerned about the meaningless phrase 'This is the primary online comment scene in Japan.'

Next, when I asked how to make curry, the Japanese of the answer was clearly broken, and words like German and Russian can be seen here and there.

Moreover, as a result of continuing to answer in broken Japanese, I stopped at 'curry'.

This time, when I asked him about

the answer to the ultimate question about life, the universe, and all things , he replied in questionable Japanese. Apparently, he refuses to answer, saying, 'I can't answer such questions because I'm an AI.'

This time, I entered

a competition programming problem and checked whether I could create code. However, it seems that I could not understand the conditions of the question text that I entered, and the confirmation content at the beginning of the answer was completely different from the question, and the code was also completely different.

The explanation of the contents of the code after that is also a sentence that does not make sense at all.

Finally, when I asked about Momotaro's content, not only was the content of the fairy tale completely different, but the Japanese grammar was also quite unstable.

Web LLM is certainly able to talk with AI on the browser, but the accuracy of the large-scale language model is still sweeter than ChatGPT's GPT-3.5, and in Japanese it returned a gibberish conversation. . Still, it worked even with WebGPU processing, so I would like to expect further evolution of the model.