``Alpaca.cpp'' installation procedure summary that allows you to install chat AI ``Alpaca'' locally and talk to AI offline



In March 2023, Stanford University released a chat AI model '

Alpaca ' with performance equivalent to ChatGPT. `` Alpaca.cpp '' has appeared, which allows you to install this Alpaca on your PC and run the chat AI locally, so I tried to summarize the procedure to actually install it and talk to the chat AI.

GitHub - antimatter15/alpaca.cpp: Locally run an Instruction-Tuned Chat-Style LLM
https://github.com/antimatter15/alpaca.cpp

In order to install 'Alpaca.cpp' and build an environment where you can talk with chat AI on Windows, you first need to install various software such as C ++ compiler and 'CMake' 'Git'. This time, in order to prepare a C++ compiler, install Microsoft's IDE 'Visual Studio 2022', install various software, and then build a chat AI environment using 'Alpaca.cpp'.

·table of contents
◆1: Install Visual Studio 2022
◆2: Installing CMake
◆3: Git installation & repository cloning
◆4: Build Alpaca.cpp
◆5: Download Alpaca model data
◆ 6: Start chat AI

◆1: Install Visual Studio 2022
To install Visual Studio 2022, first visit the download page by clicking the link below.

Download Visual Studio Tools - Install Free for Windows, Mac, Linux
https://visualstudio.microsoft.com/en/downloads/

After accessing the download page, click 'Free download' in the part marked 'Community' to download the installer (file size: 3.53 MB).



After the download is complete, double-click the installer to run it.



When the following screen is displayed, click 'Execute'.



After a while, the following installation item selection screen will be displayed.



In the initial state, 'ASP.NET and web development', 'Azure development', and '.NET desktop development' are checked, but this time they are unnecessary, so uncheck them.



Next, check 'Desktop development with C ++' and click 'Install'.



Wait a few minutes for the installation to complete.



When the installation is completed, it is recommended to restart the PC, so click 'OK' and restart.



◆2: Installing CMake
Next, install the software 'CMake' required to build Alpaca.cpp. First, click the link below to access the download page.

Download | CMake

https://cmake.org/download/

After accessing the download page, scroll down.



Search for 'cmake-3.26.0-windows-x86_64.msi' and click it to download the installer (file size: 29.6MB).



After the download is complete, double-click the installer to launch it.



When the installer starts up, click 'Next'.



Read the license carefully, check the agreement and click 'Next'.



Check 'Add CMake to the system PATH for the current user' and click 'Next'.



When the installation destination selection screen is displayed, click 'Next' without changing anything.



Click 'install'.



The installation will complete in seconds.



When the installation is complete, click 'Finish' to exit the installer.



◆ 3: Git installation & repository cloning
Next, download the files needed to build Alpaca.cpp to your PC. This time, we will download using the version control tool 'Git', so if you have not installed Git, please install it by referring to the '

Installing Git ' section on the following page.

Image generation AI ``Stable Diffusion'' works even with 4 GB GPU & various functions such as learning your own pattern can be easily operated on Google Colabo or Windows Definitive edition ``Stable Diffusion web UI (AUTOMATIC 1111 version)'' installation method summary - GIGAZINE



After installing Git, open any folder on your PC, open the right-click menu, and click 'Open in Terminal'.



Then Windows Terminal will start and the following screen will be displayed.



When you see the above screen, type the command below and press Enter.
[code]git clone https://github.com/antimatter15/alpaca.cpp[/code]



It looks like this in the image. Just copy and paste the above command and press Enter.


If you enter the correct command, the characters will flow and the necessary files will be downloaded as follows.



When the file download is completed, the file 'alpaca.cpp' will be added to the opened folder, so double-click to check the contents.



If the following files are saved in the 'alpaca.cpp' folder, the download of the necessary files is complete.



◆ 4: Build Alpaca.cpp
Next, right-click the blank area in the 'alpaca.cpp' folder and click 'Open in Terminal'.



Then Windows Terminal will start and the following screen will be displayed.



Then run the following command: Be careful not to forget the '.' after the half-width space.
[code] cmake .[/code]



It looks like this in the image. Just copy and paste this command and press Enter.



Wait for a while as the letters flow smoothly. When '○○\alpaca.cpp' is displayed at the end, proceed to the next step.



Now run the following command:
[code] cmake --build . --config Release[/code]



It looks like this in the image.



I keep getting a warning that 'data may be lost', but it's okay. When '○○\alpaca.cpp' is displayed at the end, the build is complete.



◆ 5: Download Alpaca model data
Next, download the Alpaca model data (file size: 3.92GB). Model data is distributed directly or by torrent, and a client such as

BitTorrent is required to download with torrent.

direct download link
https://gateway.estuary.tech/gw/ipfs/QmQ1bf2BTnYxq73MFJWu1B7bQ2UD6qG7D7YDCxhTndVkPC
https://ipfs.io/ipfs/QmQ1bf2BTnYxq73MFJWu1B7bQ2UD6qG7D7YDCxhTndVkPC
https://cloudflare-ipfs.com/ipfs/QmQ1bf2BTnYxq73MFJWu1B7bQ2UD6qG7D7YDCxhTndVkPC

(Torrent compatible software required) Torrent file
https://btcache.me/torrent/5AACEAEC63B03E51A98F04FD5C42320B2A033010
https://torrage.info/torrent.php?h=5aaceaec63b03e51a98f04fd5c42320b2a033010
Magnet link

After the download is complete, select the file and press F2.



Change the name to 'ggml-alpaca-7b-q4.bin' when the name can be changed.



It is OK if the bin file is completed as follows.



Move the resulting bin file to the 'alpaca.cpp' folder.



When 'ggml-alpaca-7b-q4.bin' exists in the 'alpaca.cpp' folder, model data preparation is complete.



◆ 6: Start chat AI
The procedure to start the chat AI is as follows. First, open the 'alpaca.cpp' folder, right-click the blank area, and click 'Open in Terminal'.



Then Windows Terminal will start.



Then run the following command.
[code].\Release\chat.exe[/code]



It looks like this in the image.



When the characters flow smoothly and ']' is displayed at the bottom, the chat AI startup is complete.



After that, if you enter a sentence and press the Enter key, AI will reply. I tried typing 'Hello' and pressing the Enter key.



Then AI responded 'Hi there!'



If you enter 'Tell me how to make an apple pie.'



However, Alpaca does not seem to support Japanese, and when I typed 'Hello', I got an unintelligible response like garbled characters.



in Review,   Web Application, Posted by log1o_hf