Short answers to what people ask before and during an integration.

Do I need to change my prompts or my LLM code?

No. observe() runs after your LLM call. Your prompts, your model and your pipeline are unchanged.

Will this slow down my application?

No. The call is asynchronous and non-blocking. It returns immediately and scoring happens on our infrastructure, outside your request path.

What happens if variA/Bly is down?

Your application works normally. The SDK handles its own errors and never raises, so your LLM calls are never affected by ours.

What is the minimum I need to send?

A prompt and a response. That gives you the quality, safety, semantic and depth dimensions. Add reference_materials for grounding and session_id for anything cross-turn — see Quickstart.

Does it work with any LLM provider?

Yes — OpenAI, Anthropic, Google, Azure, Ollama or a custom one. Pass provider_response and the model, tokens and cost are read from it; or set model and provider yourself.

How do I evaluate a RAG application?

Pass reference_materials — your retrieved chunks — and faithfulness, hallucination rate and attribution accuracy are scored against them. See Score a RAG answer.

How do I evaluate multi-turn conversations?

Pass session_id to group turns into a session. Coherence is scored across turns automatically from what you have already sent. See Score a conversation.

How do I evaluate an AI agent or a multi-agent workflow?

Report every step rather than only the final answer: the same session_id on each, an agent_name per step, and is_aggregator=True on the step producing the answer. That fires deterministic cross-step scoring — coherence, trajectory and tool correctness, unauthorized actions, aggregation grounding and loop detection. On LangGraph, call instrument(graph) once instead of instrumenting each node. See Score a multi-agent trace.

How do I measure whether my agent called the right tools?

Pass tools_called on each step, and expected_tools where you know what should have happened. That gives you trajectory precision and recall plus tool-call correctness. Add allowed_tools and any call outside the list is flagged as an unauthorized action.

Why are some of my grounding scores 0.000?

Usually because the turn was a classifier, router or tool call with no factual claims to ground — a JSON output choosing the next step, scored as if it were prose. Pass kind="classifier" (or "router", "tool_use", "transformer") on those calls and grounding is marked not-applicable instead of zero. See Turn types.

What is the difference between N/A and a score of zero?

Zero means measured and bad. N/A means not measured, because the inputs for it were not there — most often grounding without reference_materials. They are deliberately different: a scorer that invented a number for a question you did not give it evidence to answer would be worse than one that declines.

Is the scoring deterministic?

Yes. The same input scores the same way every time, so a change in the number is a change in your system rather than a change in a judge's mood. That is what makes a regression gate possible.

How is this different from LLM-as-a-judge?

A judge is another model with an opinion, priced per call and varying between runs. Our scoring is deterministic and reproducible, so it can be compared across versions and used as a release gate.

Can I choose which dimensions are scored?

Yes — per experiment, with a weight and a threshold on each. See Choose which dimensions score.

How do I A/B test two prompts?

Run both against the same traffic at the same time so the difference is attributable to your change rather than to the week. See Move to BYOR.

Can I run this in my own infrastructure?

Yes — inside your own infrastructure or VPC, with the same scoring and the same SDK, on the Enterprise plan. See Self-hosted and VPC.

Nothing has appeared in the dashboard — what should I check?

Work through Troubleshooting; it covers the environment variable, the code path, the API key, and how to read the debug log.