JEV-27B-VL: A 27B Vision Model Leads the Jev Decision Index
JEV-27B-VL ranks first among 20 models on the Jev Decision Index Vision board, reaching a Full score of 69.82. Its result matters because the model combines that lead with approximately 15 times fewer total parameters than the stock Qwen3.5-397B-A17B reference, while producing calibrated decisions across text and images.
JEV-27B-VL ranks first among 20 models on the Jev Decision Index Vision board, reaching a Full score of 69.82. Its result matters because the model combines that lead with approximately 15 times fewer total parameters than the stock Qwen3.5-397B-A17B reference, while producing calibrated decisions across text and images.
A smaller model takes the top position on the vision board
The central launch result is straightforward: JEV-27B-VL ranks first among 20 models on the Jev Decision Index Vision board with a Full score of 69.82. That score is 6.4 points higher than the stock Qwen3.5-397B-A17B reference, despite JEV-27B-VL using approximately 15 times fewer total parameters. Its score is 72.78 on public benchmarks and 66.86 on private tests, giving the headline result a visible split between the two evaluation settings rather than a single undifferentiated number. The model’s design helps explain why its decision performance can be evaluated separately from the size of its underlying vision system.
Text-trained decision components sit on an unchanged vision encoder
JEV-27B-VL uses an unchanged Qwen3.8-27B vision encoder, with a JEV System 1 adapter and decision head added to the architecture. The System 1 adapter and decision head were trained on text only, while the system accepts both text and images. For each decision, it outputs a calibrated probability for every available option, supporting yes-or-no decisions, ratings from 0 to 5, and selections among 2 to 256 choices. It also supports prompts of up to 256K tokens, making the model’s intended interface broader than a fixed classification label. That combination of visual input and structured decision output becomes more meaningful when its calibration and cross-benchmark consistency are examined.
Calibration and breadth distinguish the benchmark result
The model records an expected calibration error of 0.019 across 13,695 rows. It ranks first on CharXiv and InfographicVQA, including InfographicVQA scores of 98.4 on public data and 98.7 on private data. Across 20 benchmark columns, it averages rank 2.25 and places in the top three on 17 columns. These results indicate broad placement rather than a lead confined to one reported task, although the public score remains higher than the private score on the overall board. The evaluations then extend beyond static vision benchmarks into decision-making and agent-oriented tests.
The same decision interface reaches demonstrations and reward evaluation
On a robot-arm demonstration, JEV-27B-VL completed 75% of 20 random scenes. In a computer-use demonstration, it completed 95% of 60 random multi-step tasks when element text was provided. It achieved 0.727 AUC on the covers-only MicroLens-100k recommendation evaluation, 73.2% macro-average accuracy on Plan-RewardBench, 78.3% overall accuracy on VL-RewardBench, and an AUROC of 0.91 on AgentRewardBench. These figures describe separate demonstrations and evaluations, so they should not be treated as one combined success rate. Readers who want to inspect the model directly can access the JEV-27B-VL page on Hugging Face, where it is published under the Apache-2.0 license. The practical implication is that assessment systems can compare not only whether a model chooses correctly, but also how its confidence behaves across visual, planning, recommendation, and agent-reward decisions.
Specs
- Architecture: Qwen3.8-27B vision encoder with a JEV System 1 adapter and decision head
- Decision output: Calibrated probability for every decision option
- Choice range: Two to 256 options
- Context length: Up to 256K tokens
- License: Apache-2.0
Sources
- JEV-27B-VL — Hugging Face