The Accuracy Sprint
Four weeks to an
analytics agent
you can trust.
I find what makes your agent give wrong answers and exactly what to change. Then I prove the fixes work.
Book a 30-minute agent review $10,000 founding price →
Fixed price. Apply the changes, and if it isn't worth the fee, you get it back.
From André Baaij, who runs the analytics agent benchmark.
Two ways in
About to roll out? Already live?
Rolling out soon?
Four questions your CXO will ask first.
- Which of its answers are wrong today?
- Why are they wrong?
- What would you change first?
- How would you prove the change worked?
Already live?
Any of these sound familiar?
- Someone caught a wrong number in a meeting.
- People quietly stopped using it.
- You have a list of suggested fixes and no idea which matter.
- A model upgrade changed its answers.
Either way, I find what's going wrong and what to change first.
What the sprint gives you
You get the fixes, and proof they worked.
Every wrong answer gets a card like this one: what the agent said, the right answer, why it went wrong, and the change that fixes it. The cards are ranked by how many answers each change fixes, so you start with the one that matters most.
Tools already suggest fixes. I test every change on questions held back from the work, three runs each, to prove which fixes work. Testing on the questions used to make the fixes can inflate the score.
Example card with invented figures. This is the format you'll get.
- Question
- "Net revenue by channel last quarter, excluding refunds?"
- Agent said
- €1.42M for paid search
- Correct
- €1.18M
- Root cause
- Refunds are in
payments. No join fromordersis declared, so the agent summed gross order value. - Change
relationships: - orders.order_id → payments.order_id metrics: - net_revenue: gross − refunds- Also fixes
- 7 other failed questions
- Who fixes it
- Your semantic layer
Week 1
Baseline
We connect your systems, and over about 3 hours we design the 60–80 questions your people actually ask and confirm their answers. I run your agent on all of them three times, then split them: half to fix against, half held back. Already live? I start from your logs: the questions people really asked, and the answers that went wrong.
Week 2
Diagnose
I trace every wrong, missing or overconfident answer to the join, column, definition or instruction that caused it.
Week 3
Change
You get the change list, ranked by how many answers each change fixes. Your team spends 3–5 days applying it. I work alongside them or review the changes. Your team makes every design decision.
Week 4
Prove
I re-run the held-back questions three times each and walk you through the result: what improved, what did not, and what to watch from here.
Who's behind this
Hi, I'm André Baaij. I ran data at Babylist, and now I run Benchouse: the public benchmark where analytics agents answer 300 questions against a warehouse with withheld ground truth, and the results get published whether vendors like them or not.
The benchmark taught me that you can't tell a great agent from a dangerous one in the chat window. Both answer fluently. You only see the difference when you record what they did and check it. Almost no team has time to do that on its own traffic.
So I check your agent the way I check the vendors.
— André
The benchmark
Every vendor, the same questions, in public.
Every season, every ranked agent answers the same questions, graded against ground truth it never saw. I publish the results, and none of them can buy a position.
I use the same method to test your agent on your questions.
| rank | agent | accuracy | completeness | restraint | $/q |
|---|---|---|---|---|---|
| 1 | Snowflake Cortex Agents  claude-opus-4-8 · snowflake semantic view | 83.7% | 99.7% | 100.0% | $0.59 |
| 2 |  Hex  gpt-5.6 (Hex) · hex semantic model (dbt) | 74.4% | 100.0% | 100.0% | $0.35 |
| 3 |  nao claude-sonnet-5 · nao table docs | 70.0% | 100.0% | 100.0% | $0.95 |
Season 2026-S2, all tiers. Ranked by accuracy.
Accuracy Sprint
$10,000 founding price
1 of 3 founding places open. After that, $15,000.
- Accuracy Sprint, four weeks
- $15,000
- Three months of Benchouse Growth ($999/mo)
- $2,997
- Total value
- $17,997
You keep the 60–80 checked questions, ranked change list, before-and-after results and one-page brief for whoever signs off the rollout.
- Apply the changes, and if the before-and-after isn't worth the fee, I refund it in full. Invoiced at the start. Say so at the readout. No forms, no argument.
- Your part: connecting your systems and about 3 hours on the questions in week 1, then 3–5 days applying the changes in week 3. You pay your LLM and SQL costs.
- In exchange for the founding price, you agree to a case study. You approve it before anything is published.
- One sprint at a time, because I do the work myself.
After the sprint
Your analytics agent is an employee. Trust it like one.
A sprint fixes your agent once. Then the model gets upgraded or someone adds a table, and the answers drift. Benchouse records every answer, re-runs your questions on a schedule, and tells you when accuracy slips. Every sprint includes three months of it.
If your agent is already live, Benchouse can start watching every answer now.
Questions
What people ask first.
Does it work with our agent?
I can work with any vendor's agent, provided it exposes its logs and has an API or MCP server.
Where does our data go?
Your data stays in Benchouse, which is fully GDPR compliant. It's deleted when you delete your account. Privacy notice →
Our semantic layer is a mess, or we barely have one.
That's more common than not. I start with what you have and build the change list from there.
Do you want an analytics agent you can trust?
Work with the person who tests analytics agents in public: 7 configurations, 300 questions, graded against answers they never saw.
Thirty minutes with me, on the agent you run. If a sprint won't help, I'll say so.