I tried running the AI chat tool `` Khoj '' that reads offline operation & data and responds based on the contents on a notebook PC purchased in 2019
Khoj is an AI chat tool that has the ability to read and answer documents on your PC and search the documents themselves. Since it is said that there is no need for an internet connection during inference and the operation is completed locally, I tried to see if it could work on a laptop equipped with an i7-8650U purchased in 2019.
khoj/docs at master khoj-ai/khoj
Python is required for Khoj to work. Go to the Python official website and click the button 'Python 3.XX.X' that appears when you place the cursor on 'Downloads'.
Download the installer and double click to run it.
Check 'Add python.exe to PATH' and click 'Install Now'.
Wait for a while as the installation starts. Click Close when done to exit the installer.
Also, 'Microsoft C++ Build Tools' is required when installing Khoj. Go to Microsoft's site and click 'Download Build Tools'.
Download the installer and double-click it to run it.
Click 'Continue'.
You will be prompted to select what to install, so check 'Desktop development with C++'. Since the options are displayed on the right side, select only four of 'MSVC v143 ~', 'Windows 11 SDK', 'C++ CMake Tools for Windows', and 'Test Core Functionality of Tools', and click 'Install'. increase.
Installation took over 30 minutes. After clicking 'OK' to exit the installer, restart the PC.
Preparations are now complete. Go to
Execute the downloaded file by double-clicking it.
A security warning will appear, click 'More Info'.
Click 'Run'.
Khoj started. Click 'Open Web UI' to launch your browser and open Khoj's 'Settings' screen.
To enable chat first, scroll down the Settings screen and click 'Enable' in the 'Offline Chat' item.
After waiting for about 10 minutes, a check mark entered next to 'Offline Chat'.
Go to the 'Chat' tab and ask 'What is the meaning of life?' as per the placeholder.
I got a rather lengthy answer. It took about 20 seconds to start answering, and more than a minute to finish answering.
I asked if they could speak Japanese, but the answer was English. However, the question seems to be well understood.
The CPU usage rate and memory usage during inference are like this. CPU uses i7-8650U. At the time of writing the article, it seems that there is no setting to use the GPU.
I found that there is no problem using it as a normal chat AI, so I will try the plug-in function next. You can check the list of plug-ins that can be used on the 'Settings' screen, and at the time of article creation, there are five plug-ins: 'Github', 'Notion', 'Markdown', 'Org', and 'PDF'. Click 'Markdown' this time.
Also, you need to prepare a file to be read by Khoj. This time, we will use Khoj's
Return to Khoj's screen, paste the path in the 'Folders' frame and click 'Save'.
Click 'settings page' to go back.
Scroll to the bottom of the page and click 'Configure'.
If the screen below is displayed, loading is complete.
If you ask 'How to install khoj' in chat, it will answer based on the contents read properly.
You can also search the document itself on the 'Search' screen. Unlike chat, it works quite fast, so you can search comfortably.
Although it will give up the characteristic 'offline operation', it is also possible to perform high-speed inference online using OpenAI's 'GPT-3.5-turbo'.
Related Posts:
in Review, Software, Web Application, Posted by log1d_ts