All docs
ReferenceTroubleshootingdashboard

Troubleshooting

Zero grounding scores, missing evaluations, key confusion, and the rest of the usual suspects.

My grounding scores are all zero

They are probably not zero — they are probably N/A, shown as -1 in the API.

If the turn is a classifier, router, tool call, or transformer, it makes no factual claims and cannot be grounded. Tag it and it will be reported as not-applicable:

observe(prompt=q, response=r, kind="router")

If it is a factual answer and grounding is genuinely absent, you are most likely not passing reference_materials. See Score a RAG application.

Nothing showed up after I integrated

Work through these in order:

  1. Is VARIABLY_API_KEY set in the environment the app actually runs in? An API key in your shell is not an API key in your container.
  2. Is VARIABLY_BASE_URL set to https://api.variably.dev?
  3. Did the code path run? observe() after an early return never executes.
  4. Set VARIABLY_LOG_LEVEL=DEBUG and look for the outbound request. If you see it leave and get a 2xx back, the integration is fine and the evaluation is queued — give it a few seconds and refresh Evaluations.
  5. Is the API key the one for the project you are looking at? A key writes to its own project and nowhere else, so an evaluation sent with another project's key lands somewhere you are not looking.

Which API key do I use?

The project API key from API Keys in the dashboard. A key belongs to one project and writes only there, so give each application its own project and its own API key.

My async functions aren't producing events

Async node functions need SDK 2.12.0 or later. Upgrade:

pip install -U variably-sdk

Does observe() slow down my application?

Scoring runs asynchronously on our infrastructure, so it is not in your request path.

Can I run this in my own infrastructure?

Yes — see Self-hosted and VPC. Same scoring, same SDK, Enterprise plan. Talk to us.