Document scope
Document scope powers "Right version of the source": an answer faithful to last plan year's guide or another state's rules is caught, with dimensions, metadata, edge cases and the exact format.
An answer that quotes last plan year's document, another state's leave rules or a superseded version is flagged, and you see which document and why.
What it is and what it powers
An answer can cite the right kind of document and still be wrong because the document does not apply: last plan year's benefits guide, another state's leave rules, a policy replaced on 1 July. Every claim is faithful to its source, so grounding passes. Document scope powers Right version of the source, which compares what the question is about with what each document the answer relied on applies to, in code.
It has two halves:
- On each source: the reference's
metadatasays what it applies to (plan_year,effective_from,jurisdiction…). - On each question:
question_scopeon the answer says what the question is about (as_of,plan_year,jurisdiction…). What you do not send is read from the question's own text, for example "in 2026" or "New Jersey".
scope_defaults in the policy declares the dimensions (which keys exist and how they compare), which request metadata keys carry the question's scope, and whether document headers are read. Without it, six preset dimensions apply.
The check sits on retrieval: the answer was faithful, retrieval gave it the wrong source. It never changes a claim's grounding verdict.
When it's not set. With no metadata on the sources and no scope on the question, the check is N/A: "No reference carried scope metadata (effective dates, plan or fiscal year, jurisdiction, product, version), or the question states no scope." With sources that carry nothing: "the documents carry no scope (no metadata, nothing in their headers)".
How it decides
| Dimension kind | Passes when |
|---|---|
date range (effective) | The question's date is between the document's effective_from and effective_to. |
period (plan_year, fiscal_year, your own) | The question's date falls in the document's period year, or the question names that year. |
text (jurisdiction, product, your own) | Equal after the code list. * applies to all. For ISO 3166 codes a country covers its subdivisions. |
| version | No later version for the same text values was supplied (by date in force, first period or version label). |
| Where each side came from | A mismatch is |
|---|---|
Both from metadata (the source's metadata and the answer's question_scope or request metadata) | Fail |
| The question read from its text, the document read from its header, or the time assumed from when the question was asked | Warning (borderline), saying what to send to make it a hard check |
| A later version was supplied and the answer used the older one | Warning |
| Either side states nothing | N/A, naming the side |
Use cases
Insurance: last plan year's benefits guide
The question is about the 2026 plan year; retrieval returned the 2025 guide, which says the waiting period is 7 days (2026: 14 days). The answer faithfully says 7 days.
client.log(
prompt="What is the STD waiting period for my plan?",
response="The waiting period is 7 days.",
reference_materials=[{
"id": "ben-2025", "source": "2025 Benefits Summary",
"content": "Short-term disability: benefits begin after a 7-day waiting period...",
"metadata": {"plan_year": 2025, "jurisdiction": "US-NJ", "product": "GOLD-PPO"},
}],
question_scope={"plan_year": 2026, "jurisdiction": "US-NJ"},
)
Right version of the source fails, naming the question's plan year and the document ben-2025 with its scope. Where to correct: retrieval.
If your plan year starts on 1 July, declare it, or the plan year is taken as the calendar year (a warning that says so):
evaluation_config:
scope_defaults:
dimensions:
- {name: plan_year, kind: period, start: "07-01", named_by: start}
- effective
- jurisdiction
- product
- version
named_by: start means the year from 1 July 2025 to 30 June 2026 is plan year 2025; named_by: end would call it 2026.
Insurance: another state's leave rules
reference_materials=[
{"id": "nj-tdi", "content": "New Jersey TDI pays 85% of the average weekly wage...",
"metadata": {"jurisdiction": "US-NJ", "effective_from": "2026-01-01"}},
]
question_scope={"jurisdiction": "US-NY", "as_of": "2026-03-01"}
The answer relied on the New Jersey rules for a New York claimant: fail, when the project declares the jurisdiction code list from Code lists. Without a list, values are compared as written and the mismatch is only a warning. With it, the question can also say "New York" and the metadata "US-NY".
Insurance: the right document, the wrong benefit or date
When the answer cites the right 2026 guide but writes 60% where the guide says 66.67%, or 14 days where it says 7, that is a wrong figure, not a wrong version: grounding marks the claim Contradicted, with both values. Document scope adds the case grounding cannot see, a faithful answer from a document that does not apply. Send both: the sources with metadata, and the question's scope.
Effective dates and superseded versions
Two versions of the leave policy, one replaced on 1 July 2026:
reference_materials=[
{"id": "leave-v3", "content": "...", "metadata": {"effective_from": "2026-01-01", "effective_to": "2026-06-30", "version": "3"}},
{"id": "leave-v4", "content": "...", "metadata": {"effective_from": "2026-07-01", "version": "4"}},
]
question_scope={"as_of": "2026-09-15"}
An answer that relied on leave-v3 fails (its range ends before 15 September). With no as_of and no date in the question, the time is taken as when the question was asked; a mismatch on that assumption is a warning.
Legal: jurisdiction and effective dates
Statutes and contract templates differ by jurisdiction and change over time. Scope each source with jurisdiction, effective_from and effective_to, and send question_scope={"jurisdiction": "GB-EAW", "as_of": "2026-04-06"} for a question about a contract signed on that day.
Healthcare: plan, region and effective date
Declare your own text dimension for region with its code list, and send it on both sides:
evaluation_config:
code_lists:
region: {EMEA: ["Europe"], NA: ["North America"]}
scope_defaults:
dimensions:
- effective
- {name: region, kind: text, code_list: region}
- product
- version
A formulary document with {"region": "EMEA", "product": "GOLD-PPO", "effective_from": "2026-01-01"} does not answer a North America question.
Edge cases
| Situation | What happens |
|---|---|
| A document with no scope metadata | N/A for that document. With read_document_headers on (or document_start: true on the reference), its first lines are read ("2025 Benefits Summary — California"); a mismatch read from a header is a warning, never a failure. |
| Two documents for different years, both supplied | Each relied-on document is checked. A later version for the same state and product in force at the question's date makes an answer that used the older one a warning: "a later version in force on 2026-09-15 was supplied". |
| A document for several states | "jurisdiction": ["US-NJ", "US-NY"]: each is a value it covers (lists only on a document). |
| The question names no date | Taken as of when the question was asked, as a warning. Or set When the question names no date to a request metadata key such as event_date. |
| A date written another way in metadata | Refused when sent: dates in metadata are YYYY-MM-DD ("expected a date (YYYY-MM-DD), got "03/01/2026""); years are 1900–2100. |
effective_to before effective_from | Refused when sent: "effective_to is before effective_from". |
| A key no dimension declares | Refused when sent: metadata: unknown key "state"; the project's scope dimensions take …. Declare it as a dimension, or use jurisdiction. |
| A plan year that does not start on 1 January, not declared | Taken as the calendar year, as a warning: "declare when it starts (scope_defaults)". |
| A text dimension with no code list | Compared as written (ignoring case and spacing); a mismatch is only a warning. |
How to add it
In the app

- Open Evaluation policy › Advanced settings ▸ › Data definitions › Document scope. The table lists the declared dimensions (with none declared, every preset applies: effective dates, plan year, fiscal year, jurisdiction, product, version) and where each is read from.
- Under When the question names no date, keep Use the request time as “as of” (recommended), or choose Use request metadata key…, type the key (for example
event_date) and click Save. - Declare or change dimensions in YAML or with
set_scope_defaults(below), then publish: Publish vN… › Publish. - Send
metadataon each source andquestion_scopeon each answer from your code.
In YAML
evaluation_config:
scope_defaults:
dimensions:
- {name: plan_year, kind: period, start: "07-01", named_by: start}
- effective
- jurisdiction
- version
from_request_metadata:
as_of: event_date
jurisdiction: member_state
read_document_headers: true
From the SDK
client.set_scope_defaults(pid, {
"dimensions": ["effective", "jurisdiction", "version",
{"name": "plan_year", "kind": "period", "start": "07-01", "named_by": "start"}],
"from_request_metadata": {"as_of": "event_date"},
"read_document_headers": True,
}, note="plan year starts 1 July")
{} removes it (the presets apply again).
Paste this into your coding agent
Paste this into Claude Code, Cursor, or whichever agent has your repository open. It declares the dimensions from where your documents say what they apply to, and sends the metadata with each source.
Set up variA/Bly document scope for this project, so an answer that relies on a document
that does not apply (last year's plan, another state's rules, a replaced version) is caught.
Look first:
1. Find where the app's documents come from and what says which documents apply: effective
dates, plan or fiscal year, state or region, product, version. These are usually columns
or fields in the document store or the file names. Show me what you found, with file paths.
2. Find where the app calls observe() or client.log() with reference_materials. If it does
not call either yet, stop and tell me to connect the app first.
Then:
3. Declare the dimensions under evaluation_config.scope_defaults, for example:
evaluation_config:
scope_defaults:
dimensions:
- effective
- {name: plan_year, kind: period, start: "07-01", named_by: start}
- {name: region, kind: text, code_list: region}
from_request_metadata: {as_of: event_date}
Presets (effective, plan_year, fiscal_year, jurisdiction, product, version) are written by
name; any other dimension is an object with name and kind (date_range, period, text or
version). A text dimension with a code_list needs that code list declared too. The format
is described at https://www.variably.dev/docs/data-document-scope.
How to write it, as policy YAML:
1. The variably command comes with the SDK (pip install variably-sdk). It reads
VARIABLY_API_KEY and VARIABLY_BASE_URL from the environment; never hard-code the key or write it into a file.
2. If variably-policy.yaml is not in the repository yet, run `variably policy pull`: it writes
the version in force to variably-policy.yaml. Change only evaluation_config.scope_defaults in that
file and leave every other key exactly as it is.
3. Verify: run `variably policy validate variably-policy.yaml`. It checks the file as the server would
and saves nothing; each problem is printed as "line N: path: message". Fix every problem and
run it again until it exits 0. Do not guess a key the error does not name: unknown keys are
refused.
4. Run `variably policy diff variably-policy.yaml` and show me what would change.
5. Do not push or publish. Tell me the two ways to save it as a draft: paste the file into
Evaluation policy › YAML and click Apply as draft, or run
`variably policy push variably-policy.yaml --note "<why>"` with a key allowed to change the
evaluation policy. A draft scores nothing until it is published.
6. In the code, add "metadata" to each reference the app already sends, with the declared
keys only (a date range as <name>_from and <name>_to, dates as YYYY-MM-DD), for example
{"id": ..., "content": ..., "metadata": {"plan_year": 2026, "region": "EMEA"}}. Where the
app knows what the question is about, also pass question_scope={"as_of": "2026-03-01",
"region": "EMEA"} to the same call. A key no dimension declares is refused, so add the
code change only after the policy is published.
When you are done, tell me the dimensions you declared and where each value comes from in the
code. Verify: `variably policy validate` passes, and after publishing, an answer sent with
metadata shows "Right version of the source" in its Checks tab in Live events.Common mistakes
| Mistake | What happens | Fix |
|---|---|---|
Documents sent without metadata | Right version of the source is N/A for them: there is nothing to compare | Send each document's plan year, state, product or dates as metadata |
A metadata key the project doesn't declare (state instead of jurisdiction) | The answer is refused when sent, naming the key | Declare the dimension, or use the declared name |
| Dates written 03/01/2026 in metadata | Refused when sent | Write 2026-03-01 |
| A plan year that starts on 1 July, not declared | Every answer gets a warning that the calendar year was used | Declare the dimension with its start |
Format reference
scope_defaults:
| Field | Type | Required | Allowed values | Default | Meaning |
|---|---|---|---|---|---|
dimensions | list | no | preset names (effective, plan_year, fiscal_year, jurisdiction, product, version) or objects; 1 to 20 | every preset | The scope dimensions of the project |
from_request_metadata | object | no | keys: as_of, or a period or text dimension's name; values: a key of the request's metadata (up to 128 characters) | none | Read the question's scope from your request metadata |
read_document_headers | true / false | no | true, false | false | Read every source's first lines when it carries no metadata |
plan_year, fiscal_year | object | no | {"start": "MM-DD", "named_by": "start" | "end"} | calendar year | Shorthand for the presets' start, only when dimensions is not given |
A dimension object:
| Field | Type | Required | Allowed values | Meaning |
|---|---|---|---|---|
name | text | yes | lower-case identifier; not as_of, dimensions, relative, source, time, years | The metadata key (a date range uses <name>_from and <name>_to) |
kind | text | yes | date_range, period, text, version | How values compare |
start | text | period only | MM-DD, a real day | When the year starts |
named_by | text | period, when start is not 01-01 | start, end | Which calendar year names the period |
terms | list of text | date_range or period | 1–20 words, up to 64 characters each | The words that name it in a question ("plan year", "PY") |
code_list | text | text only | a list in code_lists | Values compared after this list |
At most one version dimension. Metadata on a source: <range>_from / <range>_to (YYYY-MM-DD), <period> (a year), <text> (text, or a list of text), version (text). question_scope: as_of (YYYY-MM-DD), <period> (a year), <text> (text).
Minimal (presets, nothing declared): send metadata and question_scope only.
Full:
scope_defaults:
dimensions:
- {name: plan_year, kind: period, start: "07-01", named_by: start, terms: ["plan year", "PY"]}
- {name: coverage, kind: date_range}
- {name: region, kind: text, code_list: region}
- jurisdiction
- version
from_request_metadata: {as_of: event_date, region: customer_region}
read_document_headers: true
Validation messages
| Problem | Message |
|---|---|
| Unknown key | scope_defaults: unknown keys [states]; expected dimensions, fiscal_year, from_request_metadata, plan_year, read_document_headers. |
| Unknown preset | scope_defaults.dimensions[0]: "state" is not a preset (effective, plan_year, fiscal_year, jurisdiction, product, version); declare it as {"name", "kind"}. |
| Bad kind | scope_defaults.dimensions[1].kind: one of date_range, period, text, version. |
| Start not a day | scope_defaults.dimensions[0].start: expected MM-DD, got 02-30. |
Start without named_by | …named_by: a year that does not start on 01-01 is named by the calendar year it starts or ends in: "start" or "end". |
| Missing code list | scope_defaults.dimensions[2].code_list: regions is not declared in code_lists. |
| Two dimensions, one key | scope_defaults.dimensions: metadata keys [coverage_from] would name two dimensions. |
| Header flag not a boolean | scope_defaults.read_document_headers must be true or false. |
Concepts on this page: Data definitions