Introduction
What variA/Bly does, how the scoring works, and where to start depending on why you are here.
variA/Bly scores what your LLM application actually produces — deterministically, on your own traffic — so "is this any good?" becomes a number you can compare, gate on, and explain.
What it does
Every response is scored across six categories and 49 dimensions: grounding, quality, safety, semantic, coherence and depth. No judge model is involved, so the same input scores the same way every time, and a score that moved means your system changed.
Three things follow, and they are what the platform is for:
- Know where you stand. Real traffic scored per dimension, so you can see which answers fail and on what.
- Know whether a change helped. Two versions against the same traffic at the same time, with a statistical verdict rather than an impression.
- Know when it stops holding. Thresholds on the dimensions you care about, and a notification when they are crossed.
How it works
Your application calls its LLM exactly as it does now. Afterwards it hands us the response along with whatever context produced it — the retrieved chunks, the conversation id, the agent that generated it. Scoring runs on our infrastructure, so none of it sits in your request path.
| Your application | The SDK | variA/Bly |
|---|---|---|
| Answers as it already does | Sends the response and its context | Scores it deterministically |
| Nothing else changes | Returns immediately, never raises | Files it under your project |
Where to start
New here? Set up and install — an account, a project, an API key. About three minutes. Then Introduction to evaluation points you at the right guide for the shape of your application.
In a hurry? Straight to the Quickstart.
Evaluating us? Reading your scores is the honest version: what each dimension is measured with, and which are strong enough to gate a release on.
Data cannot leave your network? Self-hosted and VPC runs the same platform inside your own infrastructure.
Why the scores are different
Most evaluation platforms ask another language model whether an answer was good. That is an opinion — priced per call, varying between runs — so it cannot be compared across versions or used as a gate.
Ours is deterministic. Claims are checked against the context you retrieved, safety comes from a classifier, and the writing dimensions are arithmetic over the text. Reproducible, and therefore usable as a release gate. Frequently asked questions covers the rest.