Run your first evaluation.
variA/Bly scores your AI system deterministically — 49 dimensions across 6 categories, the same score on every run. Start with the quickstart; it takes about five minutes.
Get started
Introduction
What variA/Bly does, how the scoring works, and where to start depending on why you are here.
Setup and installation
Create an account and a project, generate an API key, install the SDK, set two environment variables.
Self-hosted and VPC
Run the same platform inside your own infrastructure, so evaluation data never leaves your network. Available on Enterprise.
Evaluation
Introduction to evaluation
What you can evaluate, which guide fits the shape of your application, and how measuring turns into deciding.
Quickstart
Install the SDK, set two environment variables, and get your first score in about five minutes.
Score a RAG answer
Pass your retrieved context so grounding can be scored instead of reported as not-applicable.
Score every step, not just the last
Give your retriever and your generator their own scores, so a bad answer points at the component that caused it.
Score a conversation
Link turns into a session and get coherence, contradiction and quality trend scored across them.
Score a multi-agent trace
A complete setup for agentic systems: trajectory, tool-call correctness, cross-step coherence and aggregation grounding, scored across every step.
LangGraph
Instrument every node in a StateGraph with one call, and keep it working when you add a node.
Choose which dimensions score
Score the things your app is actually for, and weight the ones that decide whether you ship.
Define what "good" means
Build the number you actually ship on, from the dimensions you choose and the weights you give them.
Reading your scores
What the six categories mean, why determinism matters, and why N/A is not zero.
Move to BYOR
Observe tells you where you stand. BYOR tells you whether a change helped — same traffic, same moment, attributable.
Compare two prompt versions
Run both against the same traffic and get a statistical verdict rather than an impression.
Read the verdict
Tell a real difference from noise, a big difference from a trivial one, and a current result from a stale one.
Version and roll back prompts
Keep every prompt version so a score change can be traced to a diff.
Keep a set of examples
Collect real prompts and the answers you would want, so prompt generation is shaped by your domain.
Operate
Reference
Turn types
Mark what each LLM call is, so grounding is reported as not-applicable on routers and classifiers instead of scored as zero.
Metadata keys
Canonical names that let a failing dimension be linked to the setting behind it, grouped by the dimension each one informs.
Troubleshooting
Zero grounding scores, missing evaluations, key confusion, and the rest of the usual suspects.
Frequently asked questions
Short answers on integration, agent and RAG evaluation, determinism, N/A versus zero, and self-hosting.
Still stuck?
Ask in our Discord, email info@variably.dev, or book a slot and we'll walk through your setup together.