Run your first evaluation.
variA/Bly scores your AI system deterministically — 49 dimensions across 6 categories, the same score on every run. Start with the quickstart; it takes about five minutes.
Get started
Introduction
What variA/Bly does, how the scoring works, and where to start depending on why you are here.
Setup and installation
Create an account and a project, generate an API key, install the SDK, set two environment variables.
Self-hosted and VPC
Run the same platform inside your own infrastructure, so evaluation data never leaves your network. Available on Enterprise.
Evaluation
Introduction to evaluation
What you can evaluate, which guide fits the shape of your application, and how measuring turns into deciding.
Quickstart
Install the SDK, set two environment variables, and get your first score in about five minutes.
Score a RAG answer
Pass your retrieved context so grounding can be scored instead of reported as not-applicable.
Score every step, not just the last
Give your retriever and your generator their own scores, so a bad answer points at the component that caused it.
Score a conversation
Link turns into a session and get coherence, contradiction and quality trend scored across them.
Score a multi-agent trace
A complete setup for agentic systems: trajectory, tool-call correctness, cross-step coherence and aggregation grounding, scored across every step.
LangGraph
Instrument every node in a StateGraph with one call, and keep it working when you add a node.
Choose which dimensions score
Score the things your app is actually for, and weight the ones that decide whether you ship.
Define what "good" means
Build the number you actually ship on, from the dimensions you choose and the weights you give them.
Reading your scores
What the six categories mean, why determinism matters, and why N/A is not zero.
Move to BYOR
Observe tells you where you stand. BYOR tells you whether a change helped — same traffic, same moment, attributable.
Compare two prompt versions
Run both against the same traffic and get a statistical verdict rather than an impression.
Read the verdict
Tell a real difference from noise, a big difference from a trivial one, and a current result from a stale one.
Version and roll back prompts
Keep every prompt version so a score change can be traced to a diff.
Keep a set of examples
Collect real prompts and the answers you would want, so prompt generation is shaped by your domain.
Feature flags
Introduction to feature flags
What a gate is, the two credentials, and what a flag actually evaluates to.
Add flags to your app
Read flags on your server and in the browser, bootstrap the first paint, and update live.
Node.js
Read flags on a Node server, with streaming updates and start-up validation.
Environments
One flag, a different answer per environment — and per tenant, if that is how you are organised.
React
Flags in the browser, bootstrapped for the first paint and updating live.
Rules and overrides
Who gets what, the order it is decided in, and why a rule is not firing.
Python
Read flags from Python, including the config shape that is easy to get wrong.
Go
Read flags from Go, with typed accessors and an explicit not-found error.
Without an SDK
The endpoint every SDK wraps, both credential forms, and what each reason means.
Flag settings
Every field on a gate — key, type, default value, client-side visibility, rules and overrides.
Operate
Reference
Turn types
Mark what each LLM call is, so grounding is reported as not-applicable on routers and classifiers instead of scored as zero.
Metadata keys
Canonical names that let a failing dimension be linked to the setting behind it, grouped by the dimension each one informs.
Troubleshooting
Zero grounding scores, missing evaluations, key confusion, and the rest of the usual suspects.
Frequently asked questions
Short answers on integration, agent and RAG evaluation, determinism, N/A versus zero, and self-hosting.
Still stuck?
Ask in our Discord, email info@variably.dev, or book a slot and we'll walk through your setup together.