Welcome to Benchouse
Benchouse records what your analytics agent does in production and measures how well it answers.
Updated 3 October 2026
Benchouse connects to your analytics agent and records every question, answer, SQL query and cost. From that it reports two numbers:
- Completeness: the share of questions the agent answered. Questions it declined count against it.
- Accuracy: the share of answers that were correct. Declined questions are left out.
We always show both, because either one can be raised at the expense of the other.
Supported agents
For each agent, Benchouse can do up to four things:
| Capability | What it means |
|---|---|
| evals | Ask the agent your questions and grade its answers. |
| logs | Record every question your users ask and every answer. |
| cost | Record what each question cost. |
| users | Record who asked. |
Databricks Genie- evals: supported
- logs: supported
- cost: supported
- users: supported
Snowflake Cortex- evals: supported
- logs: supported
- cost: supported
- users: supported
Lightdash- evals: supported
- logs: supported
- cost: supported
- users: supported
Sigma- evals: supported
- logs: supported
- cost: supported
- users: supported
Hex- evals: supported
- logs: supported
- cost: supported
- users: supported
ThoughtSpot- evals: supported
- logs: supported
- cost: supported
- users: supported
Power BI- evals: supported
- logs: supported
- cost: supported
- users: supported
nao- evals: supported
- logs: supported
- cost: supported
- users: supported
Cube- evals: supported
- logs: not available
- cost: supported
- users: supported
Omni- evals: supported
- logs: not available
- cost: supported
- users: supported
Looker- evals: supported
- logs: not available
- cost: supported
- users: supported
GoodData- evals: supported
- logs: not available
- cost: not available
- users: not available
AnthropicBuilt your own agent on these? We integrate with your setup.
OpenAIBuilt your own agent on these? We integrate with your setup.
If your agent isn’t listed but stores its conversations in a database, connect it as a custom agent.