Alibaba's Qwen team releases 'QwQ-32B-Preview', an inference model comparable to OpenAI o1, demonstrating superior performance in mathematics and scientific inference

The research team behind Alibaba's large-scale language model 'Qwen' has announced the experimental research model ' QwQ-32B-Preview ', which focuses on improving inference capabilities. The research team claims that the inference capabilities of QwQ-32B-Preview are comparable to

https://qwenlm.github.io/blog/qwq-32b-preview/

Qwen/QwQ-32B-Preview · Hugging Face
https://huggingface.co/Qwen/QwQ-32B-Preview

Alibaba releases an 'open' challenger to OpenAI's o1 reasoning model | TechCrunch
https://techcrunch.com/2024/11/27/alibaba-releases-an-open-challenger-to-openais-o1-reasoning-model/

QwQ-32B-Preview is an experimental research model developed based on Qwen 2.5-32B, focusing on improving AI reasoning capabilities. According to the research team, this model adopts a philosophical approach that emphasizes deep introspection and exploration, and approaches all problems such as mathematics, coding, and world knowledge with essential questions and wonder.

The main features of QwQ-32B-Preview are its excellent performance in mathematics and scientific reasoning, achieving scores of 65.2% on GPQA , a graduate-level problem-solving ability evaluation benchmark, 50.0% on AIME , a junior high school level mathematics problem-solving benchmark, 90.6% on MATH-500 , a comprehensive dataset that tests the solving of mathematics problems, and 50.0% on LiveCodeBench , which measures programming code generation and problem-solving abilities. These scores are almost on par with OpenAI o1-preview and OpenAI o1-mini, and the research team argued that 'the benchmark results highlight the significant advances in QwQ-32B-Preview's analytical and problem-solving capabilities, especially in technical fields that require deep reasoning.'

However, the research team notes that the QwQ-32B-Preview is a model that is still in the experimental research stage, and that for practical applications, it is necessary to understand its limitations and find ways to use it appropriately.

For example, while QwQ-32B-Preview has shown excellent results in technical areas such as mathematics and coding, the research team says that 'challenges remain in everyday common sense reasoning and understanding the subtle nuances of language. In particular, it can be difficult to understand and respond naturally like a human when it comes to communication that includes context-dependent interpretation and emotional elements.

The researchers also found that the model would unexpectedly mix different languages or switch languages in the middle of a conversation. Furthermore, the researchers said that recursive reasoning loops could lead to the model repeating the same logical patterns and failing to reach a conclusion, leading to longer responses than necessary and distracting from the real problem-solving.

In addition, the research team noted that safety and ethical considerations require additional measures to ensure the reliability and safety of the model's output. As it stands, there is a risk that the model could generate potentially harmful or misleading information, so careful monitoring and control are required in actual use.

The research team commented, 'Understanding inference in large-scale language models is spreading into a wide range of research fields. There are various approaches, such as forming learning patterns through process reward models, deep analysis through large-scale language model criticism, multi-stage inference to build complex thoughts, and reinforcement learning that enables growth in the real world through system feedback. The destination is not clear, but we are moving forward with unwavering determination in the pursuit of truth and intelligence.'

The QwQ-32B-Preview model is available on Hugging Face, and you can see the demo on the following page.

QwQ-32B-Preview - a Hugging Face Space by Qwen
https://huggingface.co/spaces/Qwen/QwQ-32B-preview