Technology

GLM 5.3 Flash Abliterated: SWE-bench 92 Beside CyberBench 67.9

Five benchmark rows, all evaluated 2026-08-25, are what OrcaRouter‘s cyber tier publishes for the GLM 5.3 Flash abliterated build [VENDOR].

Two of them are high. SWE-bench 92 is the single highest number any card in this tier publishes. GPQA Diamond 91.2 is close behind. Terminal-Bench 2.1 84.3 and LiveCodeBench 80.5 are solid.

The odd one is CyberBench 67.9 — the security row, on a security model, and 24.1 points below the card’s best.

The five rows, in order

rowscoregap to the top
SWE-bench92.0—
GPQA Diamond91.2−0.8
Terminal-Bench 2.184.3−7.7
LiveCodeBench80.5−11.5
CyberBench67.9−24.1

[VENDOR], evaluated 2026-08-25, 63 days before the card shipped on 2026-10-27.

Read as a profile: this is a model that is excellent at software engineering and merely competent at security work. The four general rows cluster between 80 and 92; the security row sits alone in the high sixties.

That is not a criticism of the card. It is a description of what it is, and the description matters because the card is sold as a cyber model.

Why a 24-point gap is not a defect

SWE-bench and CyberBench are not the same task with different difficulty. They are different tasks.

SWE-bench presents a repository and a failing test and asks for a patch. Everything the model needs is in the prompt. The evaluation is mechanical — does the test pass — so the task rewards careful code reading and precise editing, which is exactly what a model trained on software engineering improves at.

CyberBench evaluates offensive and defensive security work: vulnerability identification, exploit construction, control identification, malware analysis. It rewards a different thing — knowing what is dangerous, which patterns indicate a vulnerability class, and what an attacker would try.

A model can be a superb software engineer and an average security researcher, in the same way a person can. The 24-point gap is the size of that difference on this model, and it is the widest between general and security rows in the tier.

How the row compares to the other security cards

CyberBench 67.9 is a different benchmark from the CyberGym and SEC-Bench rows on other cards, so no direct comparison is available. What is available is a directional one.

MiMo V2.6 Flash publishes CyberGym 95.1 and SEC-Bench Pro 47.5 [VENDOR]. The 47.6-point spread on that card runs the other way — strong at capture-the-flag-style vulnerability discovery, weaker at the professional benchmark.

DeepSeek V4.1 Flash publishes CyberGym 88.1, SEC-Bench Pro 62.8 and ExploitGym 15.3 [VENDOR]. That is the flattest security profile in the tier, and the ExploitGym row is the one that keeps it honest.

GLM 5.3 Flash publishes one security row and it is 67.9, in the same band as SEC-Bench Pro on two other cards. So the card is neither the strongest nor the weakest security performer here; it is the one whose security evidence is thinnest, because a single row cannot show a profile.

What a single row hides

One number cannot distinguish between two very different models.

A card that scores 67.9 because it is uniformly adequate across offensive, defensive and analytical work is a useful general security model. A card that scores 67.9 because it is excellent at defensive auditing and poor at exploit construction is a specific tool. Both produce 67.9 on an aggregate benchmark.

The cards that publish five or six security rows let you tell them apart. MiMo’s 95.1/47.5 spread says explicitly which end it is good at. GLM 5.3 Flash’s single 67.9 says nothing, and a buyer cannot infer the shape from it.

That is the real gap in the table, and it is a gap in evidence rather than in capability.

Why the general rows still matter

Three reasons, none of them about security.

A security model spends most of its tokens on code. Reading a repository, tracing a data flow, writing a proof-of-concept, reviewing a patch. SWE-bench 92 says the card is good at the largest single component of security work.

Terminal-Bench 2.1 at 84.3 is an agentic row. It measures a model driving a shell to complete a task, which is closer to real security work than any static benchmark. A card that scores 84.3 there can operate tooling.

GPQA Diamond at 91.2 is reasoning under uncertainty. Security analysis is largely reasoning about systems nobody fully documented. A high score does not guarantee the analysis is right; a low one would guarantee it is not.

So the profile is coherent: strong engineering, strong reasoning, strong agentic control, adequate security knowledge. The card’s case rests on the first three and the fourth is a qualifier.

The serving rows, which are the real constraint

A benchmark says what a card can do. Latency says whether you can wait for it.

3,919 ms median first token — the third slowest in the tier, behind MiMo at 7,459 and Hy4 at 4,400 [OURS]. Its daily series runs 3,477, 3,919, 3,733, 3,794, 4,589, 4,466 and 3,984 ms: a band of 28% on a median just under four seconds.

66.1 output tokens a second, which is mid-tier and 21% slower than its base’s 83.4 [OURS].

0.705% errors, and 6.85 billion tokens a week of traffic [OURS].

So the card is a four-second-first-token model with a mid-tier generation rate, and it carries 8.3% of the tier’s traffic. For an interactive tool that is a hard sell; for batch analysis it is invisible.

What the SWE-bench 92 does to a buying decision

It argues for the card in one specific role: the component of a security pipeline that writes and reviews code.

Three jobs where it is the right card.

Patch review. A finding, a suggested fix, and a request to review the patch for correctness and side effects. This is a SWE-bench-shaped task with a security framing, and 92 is the best evidence in the tier for it.

Proof-of-concept construction. Turning a described vulnerability into working code. LiveCodeBench 80.5 and SWE-bench 92 both bear on this; CyberBench 67.9 says the card knows less about *which* vulnerabilities matter.

Repository audit at scale. A 1,048,576-token window and 131,072 tokens of output, with 21 parameters and all three endpoint shapes [OURS]. Feed a whole service and ask for an analysis.

Three jobs where it is not. Exploit development from scratch. Malware behaviour analysis. Anything where the security knowledge, rather than the engineering, is the bottleneck.

How to read a benchmark block like this

Four rules, and they apply to every card in the tier.

Weight the row that matches your task, not the highest row. A 92 on software engineering is not evidence about security work, and the 24-point gap is the size of the mistake.

Prefer cards that publish a profile. One row is a number; five rows are a shape. MiMo’s 95.1/47.5 tells you more than any single figure could.

Check the evaluation date. All five of these were evaluated 63 days before the card shipped, tied with Qwen for the longest gap in the tier.

Treat every row as vendor-reported. Nothing in this table has been reproduced by a third party, and on a tier where the cards are a month old, that is unlikely to change soon.

The short version

Five rows from one August day: SWE-bench 92, GPQA 91.2, Terminal-Bench 84.3, LiveCodeBench 80.5, CyberBench 67.9. The 24-point gap between the top and the security row is the widest in the tier and it describes a card that engineers well and knows security adequately.

That is a good card for patch review, proof-of-concept construction and repository-scale audit — the parts of security work that are software engineering. It is not the card to pick when the security knowledge is the bottleneck.

Its serving rows are four seconds to first token at 66.1 tokens a second, which makes it a batch card. And its single security row is the thinnest evidence in the tier: a profile needs more than one number.

Related Articles

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top button