All docs
Get startedWhat's newdashboard

What's new

The main capabilities of each release, each linking to the page that explains it.

The main capabilities each release added, newest first. Each links to the page that explains it. Smaller improvements and fixes are listed in each release's notes.

5.1.1 · November 2026 · Every score shown is the score that applies

What you getRead more
The whole run in the Sessions list. A Whole run column shows each session's tool correctness, path of steps and permitted tools (Passed, or what failed), and a Whole run failed view lists the sessions that took a wrong path, without opening each one.Sessions
Hand-offs you can read at a glance. Each hand-off field shows what the sending agent had and what it sent; the list names every finding ("sent no order_id (required) · had 45; sent 15").Sessions
Retrieval steps scored as retrieval. A step that answers with the document it found ("Retrieved document: Standard Refund Policy 2026") is supported by that document, and relevance, completeness and other answer measures no longer lower its score.Claims and figures
The final answer uses its sub-agents when it uses their documents. A final answer that draws on a retrieval step's documents counts as using that step, so a correct run no longer fails Final answer uses sub-agents.Sessions
Not applicable is never shown as 0. Attribution accuracy, context utilization and retrieval relevance show no value when there is nothing to measure; context utilization credits the documents behind a conclusion. Each claim keeps the documents that support it.Claims and figures

New answers are scored this way from this release; re-score past answers to apply it.

5.1 · November 2026 · Why a version got worse

What you getRead more
Compare two runs, step by step. Comparing runs opens with what changed in plain words, then every step of a multi-agent workflow: the first step to regress, the first step given different documents, and the tools each step called, before and after. The release gate names the step behind every newly failing check.Compare two runs
Expected answers compared by meaning. A different decision ("approved" against an expected "denied"), a different yes or no, or the right values attached to the wrong facts fails Matches the expected answer; values are still compared exactly in code.Expected answers
Facts the user stated count as evidence. An answer that uses a value from the question ("you bought it 45 days ago") is no longer failed for it, while the documents still decide everything they cover.Claims and figures
Your own keys for dataset items. Give each item a key (refund-001) and name it by that key when you log answers.Datasets
One context for every way you send. Set the environment, run and dataset item once with variably.context(); observe(), client.log(), the framework integrations and OpenTelemetry all carry them.Connect your app

Python SDK 2.20.0 or later (pip install -U variably-sdk).

5.0 · November 2026 · Evaluation-first

Every answer is scored the moment it arrives, by a versioned policy you can read, change and roll back.

What you getRead more
Evaluation from the first answer. Each project has an evaluation policy, created for you. Every answer records the version that scored it; publish a new version, compare, put an old one back, or re-score past answers and keep every earlier verdict.Policy, versions and re-scoring
Your business rules, checked on every answer. Limits, allowed values, date windows and required statements, decided in code on the facts the answer states, and confirmed by the reasoning model before an answer fails. Rules can cover a whole conversation (for example "no payment demand while a dispute is open"). Templates for insurance, HR, healthcare, lending, support, legal and collections.Business rules
Any agent stack, with nothing to wire by hand. Google ADK, LangGraph, Microsoft Agent Framework, Azure AI Foundry, MLflow and OpenTelemetry: the SQL, the rows it returned, documents and files are read from your agents' tool calls. One agent or several; the final answer of a multi-agent run is checked against everything the run gathered. An input that never arrives shows Couldn't check, with the fix.Every combination
Text-to-SQL and finance answers checked against the data. SQL against your declared schema and rules, and every figure in the answer against the rows the query returned. Journal entries against your posting rules and chart of accounts.SQL checks, Posting rules
A release loop, not just a score. Datasets from real answers, runs, and a release gate for CI; human review with agreement measured; alerts by webhook and email.Runs and the release gate, Human review
Built for teams. Project roles, single sign-on and an audit log of every change.Members & roles

Python SDK 2.19.0 or later (pip install -U variably-sdk).

4.3 · October 2026 · Exact figures

What you getRead more
Arithmetic in answers checked exactly. A total, difference or share the answer computes is recomputed from the sources; a figure proven wrong fails grounding.Claims and figures
A figure the sources don't mention is not called wrong. It is reported as unsupported, separately from a figure the sources contradict.Claims and figures
Tool results count as evidence. What an agent's tools returned supports its answer, as documents do.Every step
Usage you can see and plan. Usage in scoring units on a Plan & usage page; upgrade at once, downgrade from the next month, in the app.Pricing

4.0 · October 2026 · Failures first and document review

What you getRead more
Failures first. Every view leads with what failed, how often, and where to correct it: the retrieval, the documents, or the answer.Issues
Document review. Invoices, contracts, statements and reconciliations: each figure and claim in a review is checked against the documents it read.Document review
Google ADK, Vertex AI and OpenTelemetry. One plugin for ADK; any framework that already traces sends its traces as they are.Google ADK, OpenTelemetry
Multi-agent traces. Agent names, hand-offs and the agent that wrote the final answer, scored as one run.Multi-agent

3.0 · September 2026 · Max-Accuracy and self-hosted

What you getRead more
Max-Accuracy. A reasoning model double-checks the claims the standard checks can't settle, for the share of answers you choose.Max-Accuracy
Grounding decided by the weakest claim. One unsupported claim is enough to flag an answer; it is not averaged away by the claims around it.Reading your scores
Self-hosted and VPC. The whole platform in your own cloud, installed with Helm; your data stays inside it.Self-hosted

2.0 · September 2026 · Feature flags and documentation

What you getRead more
Feature flags with SDKs for Python, Node, Go, React and HTTP, which return the same value for the same user in every language.Feature flags
Documentation for setup, evaluation and reading your scores.Introduction

Next steps

Rolling this out across a team?

Thirty minutes with us and your own prompts — we'll get you scoring.

Last updated on 20 November 2026

Built byvariA/Bly