Review of ``LM Studio'', an execution environment that allows you to test the performance of various language models without requiring commands even on non-GPU laptops
Research and development of text generation AI is progressing rapidly, and many large-scale language models (LLMs) that can be used for free have been released. However, in order to actually test the operation of LLM, high-performance hardware and complicated environment construction are often required, so many people probably give up from the beginning. It is said that if you use ' LM Studio ', which is available for free, you can easily run LLM on a PC with ordinary performance without the need for any commands, so let's check the steps to actually use LLM using LM Studio. Saw.
LM Studio - Discover, download, and run local LLMs
◆Installing LM Studio
LM Studio has Windows and macOS versions, as well as a beta version for Linux. The only requirement for using it on Windows is a CPU that supports AVX2. Most PCs sold in recent years are equipped with AVX2 compatible CPUs, so they can operate in a wide variety of environments. To install the Windows version of LM Studio, first access the link below and download the installer.
LM Studio - Discover, download, and run local LLMs
https://lmstudio.ai/
Click 'Download LM Studio for Windows' to download the installer (file size 401MB).
Double-click the installer to launch it.
Click Run when the following warning appears.
The installation process will proceed automatically without you having to click anything, and LM Studio will launch automatically after installation is complete. When you start LM Studio for the first time, a summary screen will be displayed, so click 'Close' to close it.
Now you are ready to use LM Studio.
◆Steps to operate LLM in LM Studio
LM Studio allows you to download, manage, and run LLM in one app. This time, I will download and run the Japanese-compatible LLM '
Then, the derived LLM of 'ELYZA-japanese-Llama-2-7b-instruct' available in LM Studio was displayed in the search results. You can download LLM by clicking 'Download' after selecting LLM. This time, I clicked on '
Wait for a while until the download is complete. Download progress will be displayed at the bottom of the screen.
Once the LLM download is complete, click the speech bubble button on the left side of the screen to switch to the chat screen.
When the chat screen is displayed, click 'Select a model to load' at the top of the screen to specify the LLM to be used in the chat, and click 'ELYZA japanese Llama 2 instruct' in the LLM list.
The status screen at the bottom of the screen is a nuisance, so click the ✕ button to hide it.
When the chat input field that was hidden on the status screen appears, enter any text and press the Enter key. This time, I entered 'Where is the capital of Japan?'
After waiting for a while, the answer came back: 'Tokyo is the capital of Japan.'
In addition, we were able to easily run tests such as ``Can you compose a haiku?'' and ``Can it suggest dishes?''
When we checked the system load when the AI was generating sentences, we found that the CPU was running at full capacity and the GPU was barely running.
Related Posts:
