This section is about getting your application scored, and then about deciding things with the scores. What the first part takes depends on one thing: the shape of your application.

What you can evaluate

	What it is	Guide
Single turn	One request in, one answer out — a support reply, a summary, a classification	Quickstart
RAG	Retrieval then generation, where answers must come from your sources	Score a RAG answer
Multi-step	A pipeline — retriever, reranker, generator	Score every step
Multi-turn	A conversation where context accumulates	Score a conversation
Agentic	Several agents, tool calls, a final synthesis	Score a multi-agent trace

Each is a complete setup you can follow on its own. They differ only in what you hand to observe(), because that decides which scores can be computed at all: retrieved chunks turn on grounding, a conversation id turns on cross-turn coherence, agent names turn on trajectory.

Choose your path

Measure first. Observe mode scores your live traffic without changing your prompts, your model or your control flow. It tells you where you stand and which answers are failing. Every guide above sets that up.

Then compare. Once you have a baseline worth beating, BYOR runs two versions against the same traffic at the same time, so a difference is attributable to your change rather than to the week. Scores move for all sorts of reasons — traffic shifts, the retriever changes, a provider ships a model update — and only a controlled comparison separates yours from those.

Most teams do both, in that order. Measuring first is what makes the comparison meaningful: you have to know what normal looks like before you can tell that something improved.

What else is in this section

Once traffic is arriving, the rest of it is about turning that into decisions.

Decide what to measure

• Choose which dimensions score — switch off what does not apply, weight what decides a release
• Define what "good" means — your own named score, built from the dimensions you chose
• Reading your scores — what each dimension is actually measured with

Compare versions

• Move to BYOR — run two versions on the same traffic
• Compare two prompt versions — set the experiment up
• Read the verdict — tell a real difference from noise
• Version and roll back prompts — so a score change traces to a diff
• Keep a set of examples — your own prompts and ideal answers

What you get either way

• 49 dimensions across six categories on every response, deterministic and reproducible
• A scoring profile you control — weights, thresholds, and the dimensions you care about
• Diagnosis, not just a number — which claim was unsupported, which step drifted, which tool was called that should not have been
• A statistical verdict when you compare versions, rather than two averages and an eyeball