Jun 21, 2026 12:00:00

What else is included in the 'GGUF' file format used by llama.cpp for AI language models, besides weights?

GGUF is the file format used by llama.cpp for language models. One of the features of GGUF is that it is a single file, which is an advantage in terms of ease of use compared to other formats that involve multiple scattered files. However, if you don't know what information is contained within the file, you might wonder if all the information necessary for model execution is really contained in just the GGUF file. The development team of the on-device inference engine 'NobodyWho' explained what kind of information is contained in GGUF in their blog.

What's in a GGUF, besides the weights - and what's still missing? - NobodyWho
https://nobodywho.ooo/posts/whats-in-a-gguf/

◆Features included in GGUF
The following information, which is included in GGUF, has been featured on the blog.

Chat templates
Special tokens
- Sampler settings - Sampler chain sequencing

Chat templates define the response format for conversational language models and are written in the Jinja2 templating language to handle complex functionality. In GGUF, the default chat templates are stored under the key 'tokenizer.chat_template'. Because Jinja2 is a programming language with loops, conditional branching, assignment, lists, dictionaries, etc., conversational LLM applications need to include an interpreter every time a new message is added. Although there can be considerable performance differences depending on the interpreter implementation, chat template processing does not become a performance bottleneck for local LLM applications and is therefore not considered a problem.

Special tokens refer to tokens that have a broader meaning than their textual representation. A typical example of a special token is 'eos (end-of-sequence)'. Because language models continuously output the next token indefinitely for each input token sequence, using the eos token to stop this process is a common solution. Below is an example of a special token for Gemma 4.

Regarding sampler settings , it was common practice to manually copy and paste proven sampler settings from elsewhere to improve the 'sampling' results—which involve selecting the next token from the probability distribution output by the language model—but with

the improvement of the GGUF format and the addition of the 'general.sampling.sequence' field, it is now possible to directly specify the sampler chain within the model file. However, currently, many GGUF models omit the general.sampling.sequence field, so in many cases, they rely on the default settings in llama.cpp.

◆Features that GGUF lacks
NobodyWho's blog points out that the following features are currently missing from GGUF at the time of writing:

- Tool invocation format : Including the tool invocation format syntax in the GGUF standard would allow for the creation of more versatile parsers.
- Think token : Although the upstream Hugging Face repository introduced the think_token field, it is often omitted during conversion to GGUF, making it difficult to separate thought streams.
• Projection Model : Multimodal LLMs often require two GGUF files: one for the projection model and one for the main model, for processing images, audio, etc.
- Support Feature List : Allows model-independent inference libraries to provide more consistent error messages and warnings when they make tool calls that are not supported by the model.

◆Summary
The NobodyWho blog concludes that GGUF is an excellent format that combines the characteristics of 'ease of use due to being a single file,' 'covering everything necessary for model execution,' 'reducing model-specific code paths,' and 'openness and high extensibility,' and that because it has a strong community as its support base, further standard enhancements and improvements to the user experience can be expected in the future.