All docs
Connect your appReferenceWhat to send with each answercode

What to send with each answer

Each optional field of log() and observe(), and the check it turns on.

Every answer you send is scored on what it contains. Each optional field of log() (and observe()) turns on more checks, because it gives variA/Bly something exact to compare against. Send what your app already has; you never need all of them.

The fields and what they turn on

You sendIt turns onRead more
reference_materials=[{"id", "content", "source"}]Grounding: each claim checked against the documentsClaims and figures
environment="prod" (log() only)The Environment filter, and scoring rules on EnvironmentScoring rules
session_id, agent_name, is_aggregator=TrueSessions, the agent path, per-agent scoring rulesSessions
handoff={"to", "fields", "message"}Hand-offs carry the factsHand-off contracts
sql="SELECT ..."SQL follows the rulesSQL checks
The rows a query returned (rows on a reference or a tool call)Each figure in the answer checked against the rowsClaims and figures
question_scope={"as_of": ..., "plan_year": ...}, and metadata on each documentRight version of the sourceDocument scope
expected_answer="..."Matches the expected answerExpected answers
expected_answer_kind="amount"Answers what was askedExpected answers
required_checks=[...] or requirements_ref="benefit-answer@v1"Covers what it mustRequirement templates
metadata={"weeks_paid": 4} used by a business ruleFollows business rules with your own system's valuesBusiness rules
tools_called, expected_tools, allowed_tools, max_stepsTool correctness, Trajectory, Step efficiency, Within policyScore a multi-agent trace
tags, metadataFilters; scoring-rule conditions on tags and metadata keysScoring rules
dataset_item, run (log() only)Dataset runs and the release gateRuns and the release gate
provider, model, latency_ms, prompt_tokens, completion_tokens, costThe Latency and LLM cost columnsLive events

One answer with everything

An insurance assistant answers a short-term-disability question. It retrieved two documents, ran one SQL query, and you know what the right answer is because this question is in a test set:

client.log(
    "How many weeks of short-term disability can I still claim in 2026?",
    "You can claim 9 more weeks of short-term disability in 2026.",
    reference_materials=[{
        "id": "std-policy-2026", "source": "STD policy, plan year 2026",
        "content": "Short-term disability pays up to 13 weeks per plan year.",
        "metadata": {"plan_year": 2026, "jurisdiction": "US-NJ"},
    }],
    question_scope={"plan_year": 2026, "jurisdiction": "US-NJ"},
    sql="SELECT weeks_paid FROM claims WHERE member_id = 'M-20931' AND plan_year = 2026",
    expected_answer="9 weeks",
    expected_answer_kind="duration",
    requirements_ref="benefit-answer",
    metadata={"weeks_paid": 4},
    environment="prod",
    session_id="chat-4821",
    agent_name="benefits-agent",
    tags=["std"],
    model="your-model-name", latency_ms=1840, prompt_tokens=912, completion_tokens=38,
)

Rules worth knowing

  • A document's text goes under content. Not text, not body. A document with no content gives grounding nothing to check against.
  • question_scope keys are checked. A key the project doesn't declare is refused, so a typo can't silently skip the document-version check.
  • Document metadata keys are checked too. Every key must be a dimension of the project's document scope.
  • is_aggregator=True needs agent_name and session_id. Without them the SDK raises before sending, so you find out in development.
  • dataset_item and run go together. One without the other is refused before sending. Set once with variably.context(), they go only on the final answer (no agent_name, or the step marked is_aggregator=True).

Common mistakes

MistakeWhat happensFix
Sending the whole knowledge base as reference_materialsEvery claim looks supported by something; real mistakes hideSend only what the answer was given
Sending the question as expected_answerMatches the expected answer fails on every valueSend the right answer, with each value written once
A new session_id per messageEvery turn is its own session; conversation checks never see the whole conversationUse the conversation's id on every turn
Putting secrets or personal data in metadataThey are stored with the answer and shown to project membersSend only what a check or a filter needs