All docs
ConceptsConceptsdashboard

Concepts

Every concept of variA/Bly evaluation in one place: what it is, why it matters, how it relates to the others, where you see it, with an example.

What you'll have

A clear picture of how a project, its evaluation policy and every score, check and loop fit together.

Everything in evaluation hangs off one object: the project's evaluation policy. A project is one AI application; creating it creates policy v1, so the first answer you send is scored with nothing to configure. Every answer records the policy version and the scoring rule that scored it, and a later change never rewrites an earlier verdict.

PageConcepts
Organisation, projects and accessOrganisation, project, project domain, environment, API keys and scopes, roles, sign-in
Answers, sessions and agentsAnswer (event), session and steps, agents and hand-offs
Policy, versions and re-scoringEvaluation policy, versions, publishing, comparing, policy as YAML, re-score, verdict history
How answers are judgedAnswer types, scoring rules, sample %, Max-Accuracy
Scores, checks and statusesDimensions, categories, weights, checks, per-dimension bars, statuses, Safety
Claims, grounding and where to correctClaim verdicts, explanations, grounding and documents, SQL rows, where to correct
Data definitionsCode lists, document scope, hand-off contracts, SQL checks, posting rules, expected answers, requirement templates
Improve, review and releaseIssues, human review, datasets, collection rules, runs, release gate, alerts, experiments
Integrations, data and usageIntegrations, deleting a project, data retention, usage and plans

How the pieces connect

  1. You create a project and choose its domain. Policy v1 is created for you.
  2. Your app sends answers with an API key; steps of one conversation share a session.
  3. Each answer is pinned to the policy version in force and the first scoring rule that matches it.
  4. It is scored on dimensions in six categories, its claims are checked against its sources, and checks compare exact values against your data definitions.
  5. Failing checks become issues, with where to correct. People can review answers; alerts watch the trend; datasets, runs and the release gate test a release; experiments compare variants.
  6. You change the policy as a draft, publish it with a note, and can re-score past answers.