I tried using the free chat AI app 'Jan' that can download LLMs and have conversations without requiring commands
Jan is an app that allows you to download a large-scale language model (LLM) and run it locally to actually chat, all in a GUI. It looked useful, so I installed Jan on my PC and checked the steps to chat with an LLM.
Jan | Rethink the Computer
Jan is an application that downloads LLM and runs it locally. To communicate smoothly with LLM, it is recommended to use a machine equipped with an NVIDIA GPU or a Mac equipped with Apple Silicon. This time, I will try to run it on a Windows machine equipped with a GeForce RTX 3090.
First, click the link below to access the Jan download page.
Download | Jan | Rethink the Computer
https://jan.ai/download/
Once you access the download page, click on 'Standard (64-bit)' in the area labeled Windows and download the Jan installer.
Once the installer has finished downloading, double-click it to run it.
The installation will be completed in a few seconds and Jan will start automatically. Jan can also talk to GPT-4 etc. by specifying the OpenAI API key, but this time we will download LLM locally and run it. To download LLM, click the button surrounded by a red frame on the left side of the screen.
A list of LLMs available for download will be displayed, so scroll down to find the LLM you want to download.
When you find the LLM you want to download, click 'Download' on the right. This time, I decided to download 'Llama 2 Chat 7B Q4' and chat.
The download will then begin, so please wait a while.
Once the download is complete, click 'Use'.
When the chat screen appears, enter text in the input field at the bottom of the screen and click 'Send' or press the Enter key.
This will start loading the LLM.
Once loading is complete, a response to the text you entered will be generated. On a machine equipped with a GeForce RTX 3090, text generation was fairly fast after the LLM finished loading.
When I asked it to respond in Japanese, it replied in Japanese. However, the answer was inaccurate. In Jan, you can easily download and run LLM on the GUI like this.
Jan is an open source project and its source code can be found at the following link:
GitHub - janhq/jan: Jan is an open source alternative to ChatGPT that runs 100% offline on your computer
https://github.com/janhq/jan
Related Posts:
