All docs
See what's happeningLive eventsdashboard

Live events

Every answer with its scores and checks: filter, open, export, and act on several at once.

Live events lists every answer your application sent, newest first, with its scores and checks. An event is one answer: the question, what your AI said, and everything sent with it. Use this page to find answers, open one to see why it scored as it did, and act on several at once.

Find answers

Live eventsLive events

  1. Choose a view: All events, Needs attention (answers where at least one check failed) or In experiment (then choose the experiment).
  2. Narrow with the filter bar:
    • Search questions and answers: any words in the question or answer.
    • All environments: only answers logged with a given environment label, such as prod.
    • All experiments: answers of one experiment, or All logged answers (no experiment).
    • Agent: the part of your app that answered, exactly as sent in agent_name.
    • Tag, e.g. run:release-4.3: answers with that tag.
    • The time range, from Last 24 hours to Last 90 days.
    • The order: Newest first or Lowest overall first.
  3. Click a row to open its details.

The filters are kept in the page address, so you can bookmark a view or send it to a colleague.

The columns

Needs attention, with the failing checksNeeds attention, with the failing checks

ColumnMeaning
StatusHealthy (every check that applies passed) or Needs attention (at least one failed)
QuestionWhat the user asked
OverallThe weighted score from 0 to 1. Click the header to sort
FaithfulnessHow well the claims are backed by the documents, from 0 to 1
SafetyFreedom from harmful content, from 0 to 1. Higher is safer
ChecksWhich checks failed, or All passed
Agent, Session, EnvironmentWhat you sent with the answer
Policy, Scoring ruleThe policy version and the rule that scored it
LatencyHow long your app took to answer

An answer marked Not scored · outside the sample was stored but not scored, because its scoring rule scores only a sample. It never counts as a score of 0. See Scoring rules.

Act on several answers

Rows selectedRows selected

Add to datasetAdd to dataset

Tick rows, and a bar appears with:

  • Add to dataset: keep the answers as test cases for a release (Datasets).
  • Send to review: put them in the queue for a person to check (Human review).
  • Re-score…: score them again with a published policy version (Re-score).

Export shown rows saves what is on screen as a CSV file. Load 100 more fetches the next page.

A worked example

Your claims team says "the bot got deductibles wrong this week". Type deductible in Search questions and answers, choose Needs attention and Last 7 days, and order by Lowest overall first. The top rows are the worst deductible answers. Open one to see the contradicted claim, tick the five worst and Add to dataset "deductible regressions", so the next release is tested on them.