Apr 10, 2023 06:00:00

Coding assistance AI 'tabby' that can be self-hosted on a local PC and can be used like Github Copilot, there is a Docker image, so I tried using it immediately Review

Github Copilot's coding assist performance is spectacular, but there are many people who have given up on using it for reasons such as not being able to take data out of the company. Under such circumstances, a coding assistance tool ' tabby ' that can be self-hosted on a local PC has appeared.

GitHub - TabbyML/tabby: Self-hosted AI coding assistant

https://github.com/TabbyML/tabby

Tabby can be easily started from Docker. How to install Docker is described in detail in the first half of the article below.

``Dream Booth Gui'' review that allows you to easily use ``Dream Booth'' that allows you to additionally learn patterns and styles from just a few illustrations to the image generation AI ``Stable Diffusion'' - GIGAZINE

With Docker running, copy the Docker code in 'Get started' on the github page...

Paste it into the shell.

When you press the enter key, the necessary files are automatically downloaded ......

Failed to start.

Apparently it seems that specifying a relative path in the mount specification was useless. Rewrite the relative path part to an absolute path and try again.

[code] docker run \
-it --rm \
-v /mnt/j/tabby/data:/data \
-v /mnt/j/tabby/data/hf_cache:/home/app/.cache/huggingface \
-p 5000:5000 \
-e MODEL_NAME=TabbyML/J-350M \
tabbyml/tabby[/code]

I was able to boot normally this time. Standby is OK if the following screen appears.

The server is running at localhost:5000, and when the code is sent to '/v1/completions', suggestions are returned. Also, since it is said that an interactive test editor is available in '_admin', try accessing 'localhost:5000/_admin'.

A page like the one below will open. Click Editor from the left panel.

The editor page looks like this. Buttons 'Fibonacci', 'Parse JSON' and 'Data ORM' are lined up above the editor.

When I clicked 'Fibonacci', a function with a name that seemed to calculate Fibonacci numbers was entered in the editor field.

I wasn't sure when the reasoning was working, but after waiting for about tens of seconds, the completion by tabby was safely displayed. well done! Until I thought about it, I looked at the code carefully and found that it was not the code to calculate Fibonacci numbers at all. I'm suddenly worried about the future.

In the case of Parse JSON it looks like this. If you press the Tab key with the completion displayed, the completion will be entered, but the previous result was wrong, so you will have to check the code many times before committing.

I will also check the CPU usage rate and memory usage rate during inference. With an i7-6800K, the CPU usage rate is 60% to 70%, the time required for inference is about 10 to 30 seconds, and it seems that about 30 GB of memory is required.

Tabby has an extension for Visual Studio Code. When you click the place written 'VSCode' on the 'localhost: 5000/_admin' page ......

The extension page of Visual Studio Code opens, so click 'Install'.

A confirmation dialog will appear, so click 'Open Visual Studio Code'.

The Tabby page opens in Visual Studio Code, so click 'Install'. You are now ready to go.

Just in case, I tried to infer the Fibonacci code that failed earlier, but the result did not change.

I thought it might be because I didn't know what the code was with 'fib' alone, so I added a comment.

Then the code to calculate the Fibonacci numbers was returned. The power of comments is great.