'GuppyLM' lets you create your own language model from scratch in just 5 minutes, and you can train it for free using Google Colab.
In recent years,
GitHub - arman-bd/guppylm: A ~9M parameter LLM that talks like a small fish. · GitHub
https://github.com/arman-bd/guppylm
arman-bd/guppylm-9M · Hugging Face
https://huggingface.co/arman-bd/guppylm-9M
arman-bd/guppylm-60k-generic · Datasets at Hugging Face
https://huggingface.co/datasets/arman-bd/guppylm-60k-generic
In modern times, many people use large-scale language models directly or indirectly, and many likely have a vague understanding that 'large-scale language models are language models composed of neural networks , trained on vast amounts of text data, and capable of processing and generating natural language like humans.' However, the reality is that large-scale language models are developed by major technology companies and cutting-edge startups that invest highly skilled human resources and enormous computing resources, making it difficult for individuals to grasp the true nature of large-scale language models.
That's when programmer Arman Hussein released 'GuppyLM,' a mini-sized language model that anyone can build from scratch in just five minutes. Hussein explained his motivation for developing GuppyLM: 'This project exists to show that training your own language model isn't magic.'
GuppyLM is a small-scale language model with 8.7 million parameters and 6 layers, utilizing a deep learning model called Transformer developed by Google researchers.
For comparison, Arcee AI, an American AI startup, announced ' Trinity-Large-Thinking ' in April 2026, which has 399 billion parameters. OpenAI's ' GPT-2 ,' developed in 2019, has 1.5 billion parameters, and ' GPT-3 ,' released in 2020, has 175 billion parameters. This illustrates just how small GuppyLM's language model is.
GuppyLM's models and datasets are publicly available on Hugging Face, allowing users to build GuppyLM themselves. Hussein states, 'You don't need a PhD. You don't need a large GPU cluster either,' explaining that you can build a language model from scratch in 5 minutes using Google Colab, Google's free development environment for machine learning education and research.
As its name suggests, GuppyLM is a language model that embodies the role of a guppy, a type of fish. It can respond to topics such as water, food, light, and aquarium creatures using short, lowercase sentences. However, it apparently cannot understand abstract human concepts like money or politics, and its maximum sequence size is relatively small at 128 tokens.
Looking at the dataset actually released by Hugging Face, we can see that it is trained on sentences that sound like they belong to a small fish. For example, the input 'I need a heater.' corresponds to the output 'The water feels cool today. I just hide and wait.', and the input 'You don't look well' corresponds to the output 'I feel ok. My side fins are working.'.
The following website also allows you to chat with GuppyLM in your browser via
GuppyLM — Chat with a Fish
https://arman-bd.github.io/guppylm/
This is what it looks like when you open the website mentioned above.
When I tried asking, 'Hello. How are you?', the reply I got was, 'Oh hi. I just found a nice spot at the top.'
When I asked, 'What do you usually eat?', he gave me a somewhat unexpected answer: 'I was just thinking about food. I am always ready for food.'
Related Posts:
