Vansa-3.4 (Vansa, hosted System One API)

Data: JevBench v1.6.1, published 2026-10-06

Capability 63.7 · API offering, ranked on the API leaderboard · Capability rank #9 of 14 within the Jev-class caps

Measured 2026-10-06 on A4 ∪ P, 600 items, equated (+2.57 I / +3.35 C); round score-a4-2.

proprietary hosted API

Base model: Qwen3.5-4B (self-reported) · Developer-reported to Benchmark Heaven (private correspondence, 1 Oct 2026): built on Qwen3.5-4B via an open decision fine-tune, with Vansa's own adapters; the intermediate fine-tune is not named. Not publicly documented or independently verified.

intelligence

40.9

calibration

86.5

speed

95.5

cost

61.9

Composite (secondary)

42.6 · Composite rank #11 of 27 on the API leaderboard

Cost and measurement conditions

$0.019 per 1,000 decisions (public tariff)

measured: token usage of the 6 Oct 2026 A4 u P run x tariff USD 0.034/M input, 0/M output (spec vansa-3.4.json), per 1,000 decisions on the v1.6.1 common cost basis (591 of 600 items; the 9 A4 u P items outside the Jev reference's accepted input range are excluded, cost-basis-a4.json)

p50 latency: 0.1 s

none (API)

Breakdowns (not part of the score)

Chance-corrected competence (0 = chance, 100 = perfect) on A4 ∪ P (600 items: 300 open + 300 sealed; fewer sealed items than the full set, so wider uncertainty). Raw values, not equated. Category cells use A4+P+L1+L2+L3; per-type/tier cells use the original basis above. Radars against Jev.

Request type	Open	Sealed
choice	62.9	60.2
noul	32.8	21.3
score	34.4	18.1

Capability by subject topic

Rules, policy & law	26.1	n=1133
Coding & software	40.9	n=202
Math & numbers	0.0	n=269
Finance & commerce	33.9	n=178
Support & operations	38.8	n=109
Everyday language	55.1	n=130
Safety & security	43.1	n=84

Use cases (TypeSafe categories)

Model routing	53.6	n=267
Legal & compliance	40.3	n=269
Customer support	30.9	n=156
E-commerce	32.2	n=88
Insurance claims	28.2	n=92
Risk assessment	26.6	n=88
Financial crime	7.2	n=76
Feature extraction	11.4	n=90
Lead generation	43.5	n=86
Recruiting	1.7	n=88
Knowledge graphs	3.3	n=82
LLM guardrails	39.9	n=82
Moderation	42.0	n=69
Code linting	7.2	n=81
Search & retrieval	46.8	n=84
Science	0.0	n=82
Advertising	21.8	n=85
Gaming	0.0	n=79
Demand forecasting	0.0	n=72

Cells need at least 15 answered items; † fewer than 30 (indicative only, listed below the radar in the compare view).

Previous measurement (v1.5.x, not comparable)

Capability 72.8 · composite 71.6 · cost $0.019 per 1,000 decisions (estimated)

Last measured 2026-10-03, measurement release v1.5.6. Older method and scale; the current figures above replace it.

Published source