ClovisLLM - GLM 5.2
ClovisLLM will evolve to GLM 5.2, the new generation of the GLM model on July 30.
This version does not merely slightly improve results on isolated questions. It targets a harder problem: enabling a model to remain effective during a long task involving many steps, multiple files, and repeated tool calls.
No changes to the Clovis API endpoints or response formats are necessary.
Key figures
The following scores compare GLM 5.2 to GLM 5.1 under the conditions published by Z.ai.
GLM 5.1
GLM 5.2
Terminal-Bench 2.1 — Executing real tasks in a terminal
+17.5 pts
SWE-bench Pro — Resolving issues in real software repositories
+3.7 pts
NL2Repo — Generating a repository from instructions
+6.2 pts
DeepSWE — Complex software engineering problems
+28.2 pts
ProgramBench — Development and code reasoning
+12.8 pts
FrontierSWE — Long and difficult engineering tasks
+43.9 pts
PostTrainBench — Model modification and training
+14.2 pts
SWE-Marathon — Long-running software projects
score ×13
MCP-Atlas — Using tools and MCP servers
+5 pts
Tool-Decathlon — Tool calls and orchestration
+7.5 pts
These results come from evaluations published by the model's publisher. They should therefore be considered as technical indicators, not as a guarantee of identical results for all use cases.
Why some gains are much larger than others
On SWE-bench Pro (Resolving issues in real software repositories), the score goes from 58.4 to 62.1. The gain is real, but relatively limited: +3.7 points.
The difference is much more pronounced on benchmarks that require the model to work for a long time:
- FrontierSWE: 30.5 → 74.4
- DeepSWE: 18.5 → 46.6
- SWE-Marathon: 1 → 13
- Terminal-Bench 2.1: 63.5 → 81.0
This contrast is significant. It indicates that GLM 5.2 is not just better at producing a function or fixing a one-off bug. It makes the most progress when it needs to:
analyze an existing environment
devise a plan
modify multiple elements
run commands or tests
read the results
identify what isn't working
adjust its strategy
keep going until achieving a usable result
This is precisely the expected behavior of a development agent or a business agent connected to multiple tools.
Comparison with other models
To put these results in perspective, the chart below compares GLM 5.2 to several reference models on software development, tool use, and reasoning tasks. Some high-end proprietary models, such as Claude Opus, maintain a lead on several benchmarks. Nevertheless, GLM 5.2 positions itself among their most serious competitors and achieves competitive performance across a wide diversity of tasks.
This comparison mainly shows that GLM 5.2 is no longer just an interesting alternative among open-weight models: it is capable of competing with some of the best proprietary models currently available.
Its performance level, significantly lower usage cost, and deployment in a French sovereign environment 🇫🇷 like Clovis make it a particularly competitive solution for organizations seeking both power, data control, and technological independence.
A concrete improvement in software development
Progress is not uniform depending on the nature of the task.
On SWE-bench Pro, the gain is 3.7 points. This benchmark measures the ability to resolve tickets in real software repositories. GLM 5.2 reaches 62.1, compared to 58.4 for GLM 5.1.
On NL2Repo, which requires producing a complete repository from a description, the model goes from 42.7 to 48.9.
On ProgramBench, it goes from 50.9 to 63.7.
Finally, on DeepSWE, the score progresses from 18 to 46.2. This 28.2-point increase suggests a clear improvement on problems that require more than a local fix and demand reasoning at the scale of an entire software system.
For a user, this can translate into a better ability to:
understand the organization of an existing project
apply a change across multiple files
respect architectural constraints
verify its work through tests
correct an insufficient initial implementation
maintain its objective after numerous tool calls
An interesting independent result in cybersecurity
The benchmarks published by the model's publisher do not constitute an independent validation. However, a test conducted by the Semgrep security team provides an external comparison point.
On an IDOR vulnerability detection task, GLM 5.2 achieved an F1 score of 39% with a minimalistic harness based on Pydantic AI.
In this same test:
- Claude Code scored between 28% and 37% depending on the version used;
- MiniMax M3 scored 23%;
- Kimi K2.7 Code scored 22%;
- GPT-5.5 used with Codex scored 20%;
- Semgrep's specialized solution reached between 53% and 61%.
GLM 5.2 therefore did not outperform Semgrep's specialized system, but it achieved the best result among the models tested with lightweight orchestration.
This is nevertheless a single benchmark, on a specific type of vulnerability and with a limited number of runs. This result does not allow concluding that GLM 5.2 is systematically better than other models in cybersecurity.
What GLM 5.2 does not solve
The model remains liable to produce incorrect information, call the wrong tool, or persist in an ineffective strategy.
GLM 5.2 must therefore always be used with:
- limited permissions;
- validation of sensitive commands;
- logging of tool calls;
- duration and consumption limits;
- human verification for critical operations.