DeepSeek V4 Flash has been criticized for its high benchmark scores but struggles in real-world tasks, particularly with orchestration.



Chinese AI company DeepSeek offers two high-performance AI models: ' DeepSeek-V4 Flash ' and ' DeepSeek-V4 Pro .' Since its release, V4 Flash has been popular, consistently ranking high in usage rankings for its 'low cost and high performance.' However, DeepSeek has announced a price increase on August 6, 2026, and it has been pointed out that in actual agent environments, it may not perform as well as its benchmark scores suggest.

DeepSeek's top-ranked V4 Flash stumbles on real agent tasks as its prices surge | VentureBeat
https://venturebeat.com/orchestration/deepseeks-top-ranked-v4-flash-stumbles-on-real-agent-tasks-as-its-prices-surge



DeepSeek-V4 Flash was released on April 24, 2026, and an updated version, 'DeepSeek-V4-Flash-0731,' was released on July 31, 2026. DeepSeek-V4-Flash-0731 is a MoE model with a total of 284 billion parameters and 13 billion active parameters. In intelligence performance tests conducted by the third-party organization Artificial Analysis, it was evaluated as having performance equivalent to Google's Gemini 3.6 Flash, and among open models, it was evaluated as having performance following Kimi K3 and GLM-5.2. Furthermore, it surpassed GLM-5.2 in coding performance, significantly outperformed Gemini 3.6 Flash in agent performance, and beat OpenAI's GPT-5.6 Luna by 0.1 points.

'DeepSeek-V4-Flash-0731' appears, an open model offering comparable performance to GPT-5.6 Luna at a lower price - GIGAZINE



DeepSeek-V4 Flash was also notable for its high cost-effectiveness. OpenAI announced a significant price reduction for the GPT-5.6 series on July 31, 2026, but DeepSeek-V4 Flash, announced just a few hours later, was priced even lower than the GPT-5.6 series.

However, on August 6, 2026, DeepSeek notified users of a 'significant increase in API fees.' While no reason was given for the price increase, some say that 'DeepSeek is responding to an unprecedented surge in demand.' There have also been reports of the model's inference speed becoming extremely slow at times due to this surge in demand.

DeepSeek announces significant API fee hike, possibly due to surge in demand following low-price strategy - GIGAZINE



However, it has been pointed out that the performance of DeepSeek-V4 Flash may not be as good as the benchmarks officially announced.

Composio , an AI agent platform, has announced the results of running 30 agent tasks using four agent harnesses with DeepSeek-V4 Flash. The tasks were intentionally designed to be challenging, multi-step tasks that spanned real-world tools such as Gmail, GitHub, Slack, and Google Sheets.




As a result, out of a total of 240 runs, only 129 passed, and it was reported that only 6 out of 30 workflows completed successfully across all the harnesses tested.




The following shows the estimated cost for each successful task. Even the most expensive harness costs less than $0.20 (approximately 32 yen), and considering the low success rate, it can be said to be cost-effective. However, Composio points out that this could change if DeepSeek raises its prices.




While DeepSeek-V4 Flash achieved high scores in agent-based benchmarks, its success rate was lower on more challenging tasks. Taryn Prambu, a writer specializing in AI and cybersecurity, points out, 'This gap demonstrates why, in enterprise environments, the success or failure of a model depends not on the raw capabilities of the model itself, but on the 'orchestration' that brings multiple systems and tasks together. Even with the same model, the results can vary greatly depending on the execution environment, harness, tool configuration, cache behavior, retries, and provider stack.'

Technology analyst Karmi Levy stated that even after the price increase, DeepSeek remains 'far cheaper in every aspect' compared to competing models such as OpenAI, Anthropic, Google, and xAI, and emphasized the importance of appropriately allocating agent AI capabilities. For example, tasks like batch processing are often routine and repetitive, rather than requiring advanced technology, making it appropriate to use a cheaper and more efficient model. Levy said, 'Companies can consider high-performance inference engines as solutions for specific workloads, rather than completely replacing existing products.'

Sanchit Vil Gogia of Greyhound Research, a global technology research and consulting firm, said, 'What's important is whether the AI model's performance is sufficient to meet the actual workflows that companies operate, not whether it's the best at every benchmark. Companies need to determine which combination of model, harness, and provider will get the job done most securely and at the lowest cost.'

Related Posts:

in AI, Posted by log1e_dh