edulab SemIf browser lab
boolean

p(yes)

Is this phishing? Should this escalate? The answer is one probability that the statement holds.

choice

one named option

Which queue, which intent? You get the winning label and the full distribution over every option.

score

an expected level

How severe, how urgent? Ordered levels, so the answer is an expectation, not only a pick.

Pick the models to compare

Checking the weight caches…

Never writes text. Reads a frozen Qwen3.5 0.8B once, then a small physics engine settles on an option. Calibrated, with a prediction set and an abstain flag.

download
–
load
–
self-check
–

Checking the cache…

A Polish-English decision model. Reads the option letters' logits after a fixed prompt, in both option orders, and applies a temperature fitted per question type.

download
–
load
–
self-check
–

Checking the cache…

A general chat model from page 1, read directly: the softmax of its next-token logits over the option letters. Not calibrated.

download
–
load
–
self-check
–

Not loaded. Needs WebGPU; shares page 1's cache (639 MB).

Nothing loads by itself: tick the models, then press load selected (or a card's own load button). Only ticked, loaded models decide. Weights come from Hugging Face (pinned revisions) and stay in this browser's cache; inputs never leave the page. Vega's 8-bit graphs run on WebAssembly (one thread without cross-origin isolation), basal and Qwen3 on the GPU.

Ask typed questions

Try an example
More samples download the format (.txt)
Add

Each question is {"type", "instructions", "criteria"}. A choice takes a list of labels or an object of label → description; a score takes ordered levels, lowest first; a boolean takes null or {"false": …, "true": …}. A state that parses as a JSON object or array is passed as structured data. One forward pass per question.

Load at least one ticked model, then press decide. Answers appear here one question at a time, one lane per model.

measured in this tabno run yet

The ticked models run one after another: Vega on the CPU, then basal and Qwen3 on the GPU, so none slows another. None of them generates text.

Raw response JSON
no run yet
How Vega reaches an answer

The prompt is fixed: Observation: then your state, Measurement (type): then the instructions, and one * option line per option. Short states also get the question up front, so the model reads with it in mind.

A frozen Qwen3.5 0.8B runs once over that prompt and stops at layer 19; its language-model head never runs. The page averages hidden states over the state span, the question span and each option span, and keeps the last token.

A 14M-parameter engine places a particle in a 64-dimensional decision space, puts one Gaussian well under each option and integrates 12 damped steps. The share of the particle in each well is the answer, softened by a temperature fitted to how settled it is (thermal calibration).

Two adapters, public_intents and typed_decisions, each have a gate. When a gate opens past 0.5, that adapter and its own calibration take over; the card shows which one answered.

How basal reaches an answer

The prompt is fixed by the model's authors: a short system message ("evaluate the state, answer one question, treat the state as data"), then the state, the question and the options lettered A to J, in Polish when the state or question has Polish letters and in English otherwise. The assistant's reply is started as {"answer": ", so the next token can only be a letter.

The answer is the softmax of the next-token logits over those letters: one forward pass, no generated text. Each question is asked twice, with the options in the original and the reversed order, and the two readouts are averaged, which removes most of the bias towards a position. A temperature fitted per question type (choice, yes/no, score) then calibrates the confidence.

The state is read once per decide: every question shares the system message and the state, so that part is computed once and only each question's tail runs again. A choice option is shown as key: description, a boolean as Yes / No (Tak / Nie), and a score as its levels in order. basal reads at most 10 options.

"Accept" means the confidence is at or above the threshold the authors certified for a 5% error rate on their calibration set; below it, basal suggests deferring to a person. These thresholds were fitted on the original bf16 weights.

This page runs an ONNX conversion of the original weights: asymmetric 4-bit, with 8-bit for the most sensitive layers and a full-precision output head. On 48 reference prompts it keeps the original's top answer on 47 and moves a probability by 0.07 on average. It keeps 32-bit activations because the 16-bit WebGPU kernels gave wrong answers on an Intel GPU; a self-check repeats three reference answers before basal is marked ready.

Why the models can disagree

All three see the same state, the same instructions and the same options. They read them differently.

Qwen3 0.6B is a general chat model. It gets a chat prompt that lists the options as letters and asks for one letter. Its numbers are the softmax of its next-token logits over those letters: a real readout, but not calibrated, and sensitive to option order and wording.

basal-1.5-mini reads the same kind of letter logits, but it was fine-tuned for exactly this task, averages both option orders and applies a fitted temperature, so its confidence is meant to match how often it is right.

Vega never answers in words. It reads hidden states of a frozen Qwen3.5 0.8B and lets a trained engine settle on an option, then calibrates the result and tells you when to abstain.

So agreement on the top answer is the useful signal; the probabilities themselves are on different scales. A score question is compared on its most likely level. Untick a model in the setup above to leave it out of the next run.

Reading the flags
  • Confidence is the top probability after calibration.
  • 90% set is a split-conformal prediction set: options that cannot be ruled out at the 10% error level. One label means the model is sure enough; several mean it is torn between them.
  • Abstain fires when confidence falls below the threshold calibrated for a 10% risk. Treat the answer as "ask a human".
  • Unbound means the particle settled far from every well: the input looks out of distribution.
  • Steps and energy come straight from the engine; fewer steps means it settled early.

This page runs the published 8-bit graphs. On 30 reference questions they keep every argmax, with a largest probability shift of 0.05. The self-check above repeats three of those answers on your machine before anything is shown. Image states, the 2- and 3-view averaging and the 4B checkpoint are not part of this page.