Evaluation policyData definitionsData definitions: format referencedashboard
Data definitions: format reference
The generic format of every data definition: fields, types, allowed values, defaults and limits, a complete evaluation_config, and how to model your own use case.
What you'll have
You can write any data definition from scratch, for your own documents, agents, queries and ledger.
Where definitions live
All data definitions are keys of evaluation_config in the evaluation policy. A policy file has max_accuracy, weights, rules and evaluation_config (and dimensions when set). Keys are sorted when exported; comments are not kept.
| Key | Shape | Page |
|---|---|---|
code_lists | object: list → code → synonyms | Code lists |
scope_defaults | object: dimensions, from_request_metadata, read_document_headers | Document scope |
handoff_contracts | list of {to, fields, requirements?, requirements_ref?, description?} | Hand-off contracts |
sql_checks | object: dialect, schema, rules, column_units | SQL checks |
posting_rules | list of {name, document_match, action, debit_accounts, credit_accounts, forbidden_accounts, tax} | Posting rules |
closed_periods | {closed_through, periods} | Posting rules |
chart_of_accounts | list of {code, name, type} | Chart of accounts |
business_rules | object: facts ({name, type, labels, list, …}), rules ({name, check, when, unless, message}) | Business rules |
requirement_templates | object: name → {version, description, requirements} | Requirement templates |
date_order | MDY or DMY | Data definitions |
And on each answer (SDK log()): reference_materials[].metadata and rows, question_scope, handoff, sql, expected_answer, expected_answer_kind, required_checks, requirements_ref.
A complete example
evaluation_config:
date_order: MDY
code_lists:
jurisdiction:
US-NJ: ["New Jersey", "NJ"]
US-NY: ["New York", "NY"]
leave_type:
STD: ["short-term disability"]
LTD: ["long-term disability"]
FMLA: ["family and medical leave"]
scope_defaults:
dimensions:
- {name: plan_year, kind: period, start: "07-01", named_by: start}
- effective
- jurisdiction
- version
from_request_metadata: {as_of: event_date}
handoff_contracts:
- to: adjudication
requirements_ref: claim-handoff
fields:
- {name: claim_id, type: identifier, required: true, pattern: "^CLM-[0-9]{6}$"}
- {name: onset_date, type: date, required: true}
- {name: leave_type, type: code, code_list: leave_type, required: true}
- {name: claimed_amount, type: money, required: true, currency: USD, tolerance: {absolute: "0.01"}}
requirement_templates:
claim-handoff:
version: 1
requirements:
- {id: state, text: "Names the claimant's state", critical: true}
sql_checks:
dialect: postgres
schema:
tables:
claims: {columns: [claim_id, plan_year, status, amount], primary_key: claim_id}
rules:
- {id: plan-year, kind: required_filter, table: claims, column: plan_year}
column_units: {approval_rate: fraction}
posting_rules:
- {name: Supplier invoice, document_match: ["SUPPLIER INVOICE"], debit_accounts: [Inventory], credit_accounts: [Accounts Payable]}
chart_of_accounts:
- {code: "1200", name: Inventory, type: asset}
- {code: "2000", name: Accounts Payable, type: liability}
closed_periods: {closed_through: "2026-02-28"}
Model your own use case
| You want to check that… | Define | Pattern |
|---|---|---|
| a value comes from a fixed vocabulary, however it is spelled | a code list | one list per vocabulary; code → synonyms; reference it from a code field or a text scope dimension |
| the answer used the document that applies to the question | document scope | a dimension per thing that makes a document apply (dates in force → date_range; a year with a start day → period; a place, product or line of business → text + code list; editions → version); metadata on sources, question_scope on answers |
| agent B received what agent A had, unchanged | a hand-off contract | one contract per receiving agent; one typed field per fact; required for what must never be missing; tolerances for rounding; nested object fields for records |
| generated SQL stays within policy | SQL checks | the schema (names only); a required_filter per mandatory filter (tenant, plan year, snapshot date); forbidden_column per sensitive column; count_distinct per entity you count; column_units per ratio column |
| an accounting agent posts each document correctly | posting rules + chart + closed periods | document type match → required debit and credit accounts (at least one of each), forbidden accounts, the tax line at the rule's rate → do_not_post for reference-only documents; the chart lists every valid account (by name or code); closed periods; the entry's currency is its document's |
| an answer gives the values you expected | expected answers | the expected text with each value written once, in an unambiguous format (2026-03-31, 23.7%) |
| an answer covers everything it must | a requirement template | one requirement per item, critical for the ones that must never be missing; a value in the text when the exact value matters; action for a tool call that must happen |
Common rules
- Names are exact keys. Unknown keys are refused everywhere, with the key named.
- Nothing is half-saved. A refused definition stores nothing; the message names the item and what to fix.
- Definitions are checked against each other when you save any of them: lists before contracts and scope, templates before contracts.
- Vocabulary is data. No check has a built-in list of your codes, accounts or tables: it compares only against what you declare.
- Verify before you save: Compare in the YAML tab or
variably policy diff FILEvalidates a file and shows what it would change, storing nothing.
Concepts on this page: Data definitions