Aug 06, 2026 22:00:00

A report claims that an open model, priced at 1/100th of the cost, surpasses the search performance of GPT-5.6 Sol.

Neon, a provider of serverless Postgres databases, has collaborated with Castform, a company that provides a platform for further training open-source AI models for enterprise use, to post-train an open-source model with 4 billion parameters using reinforcement learning. They report that this achieved 'accuracy equivalent to OpenAI's

GPT-5.6 Sol in search tasks at 1/100th the cost.'

How Castform + Neon Beats Frontier Models on Price and Efficiency - Neon
https://neon.com/blog/how-castform-neon-beats-frontier-models-on-price-and-efficiency

Traditionally, when using AI to search internal company documents, 'embedded search,' which converts documents into numerical data to find similar content, has been widely used. However, with the proliferation of AI agents, there is a growing trend towards systems that break down large questions into multiple smaller problems and perform repeated searches until the necessary information is gathered.

In this type of 'agent-based search,' the AI model independently determines the search query, reviews the results, and then performs the next search. While this approach is better suited to handling more complex questions than a single search, it has the drawback of increasing processing time and costs because a high-performance model is called upon for each search. According to Neon, a typical search request using GPT-5.6 Sol takes more than 10 seconds and costs approximately $0.03 (about 4.5 yen) per request.

Castform then trained a relatively small open-source model to learn how to find the necessary information from specific data. Neon provides the document storage location and search functionality, while Castform is responsible for training the model to determine 'what to search for.'

The learning process involves the model tackling a task, scoring its results, and iteratively improving upon them. The model is evaluated on whether it found the correct documents, cited the appropriate passages, and whether its final answer was correct, and these results are incorporated into subsequent trials.

However, many companies do not have question and answer data that can be directly used for AI training. On the other hand, they have accumulated a large amount of information that can be used as training material, such as internal documents, product information, support articles, customer interactions, and business databases.

Castform automatically generates question-and-answer combinations based on this existing data. For example, if the travel regulations state, 'Use standard class for train travel and make reservations at least 14 days in advance,' it will create a question asking about that, and train the model with the answer derived from the regulations as the correct answer.

During training, you can see how the model responds and how its evaluation scores change. Because you can trace back to individual questions, it is said to be easier to find bugs in the search function or problems where users try to get high scores by simply circumventing the evaluation system.

This graph shows the change in the average evaluation score with additional learning. You can see that it rises sharply immediately after learning begins, and then gradually improves as trials continue.

In validation using Neon, the average evaluation score of the model further trained with Castform was 1.447, surpassing not only GPT-5.6 Sol's 1.369 but also GPT-5.4's 1.377, which was used as a comparison target, setting a new record. The graph also shows Qwen3.5-4B, which did not undergo further training, with an evaluation score of only 0.382. On the other hand, the inference cost of the further trained model was $0.000929 (approximately 0.14 yen) per inference, which is about 1/100th of GPT-5.6 Sol's $0.087338 (approximately 13 yen), placing it at the lowest level among the comparison targets, alongside Qwen3.5-4B.

During training, numerous models perform searches simultaneously and repeatedly, resulting in a concentrated load on the database in a short period. Neon explains that it increases or decreases computing resources as needed, and reduces them during periods of low access, thus avoiding the need to constantly run equipment optimized for maximum load.

The following shows the CPU resources used by the database during training. By automatically increasing or decreasing the required computing resources according to the load, it becomes unnecessary to always secure the maximum amount of resources.

This case study demonstrates the potential for improving the balance between performance and cost by training smaller models tailored to a company's data and intended use, rather than using the largest possible AI model for every application. However, the results shown are from a model optimized for the specific task of search, and do not necessarily mean that it is equivalent to GPT-5.6 Sol in general text creation or broader reasoning capabilities.