The Accuracy Sprint · four weeks · fixed price

Fix your agent's wrong answers in four weeks,
without guessing which fix matters.

I find what caused every wrong answer. You get a change list ranked by how many answers each change fixes. I measure the result on questions held back from that work. Apply the changes, and if the result isn't worth the fee, you get it back.

Book a 30-minute agent review Ready to start? Email me for a start date →

$10,000 founding price →

Your part: connecting your systems and about 3 hours on the questions in week 1, then 3–5 days applying the changes in week 3.

For any analytics agent with logs and an API or MCP server: Cortex, Genie, Hex, nao or your own.

Rolling out soon?

Know which answers are wrong, why, and what to change first, before the business sees it.

Already live?

I start with your logs and rank the fixes by how many of your people's answers they affect.

What you walk away with

You get the fixes, and proof they worked.

60–80 of your real business questions, each with a checked answer. Yours to keep.
What went wrong with every wrong or missing answer, and why.
Edits to your semantic model, dbt models, agent instructions and column descriptions, ranked by impact.
Before-and-after accuracy and completeness, measured on questions held back from the work. Each gets its own score.
Which answers each change fixed and which it broke, question by question.
A one-page brief for whoever signs off the rollout.
Three months of Benchouse Growth ($999/mo list, included), so the next model upgrade can't quietly undo the work.

What it takes from you

You'll need time and access, too.

Week 1: connections: Access to your agent so I can ask it questions, to your database, and to your dbt project if you have one.
Week 1: about 3 hours: Working with me to write the 60–80 questions and confirm their answers.
Week 3: 3–5 days: Applying the changes to your semantic layer, dbt models and agent instructions. This is the work the refund depends on.
Your LLM and SQL costs: I run every question three times, before and after, using your agent and warehouse. Your vendors bill you for that usage.
After three months: Growth continues at $999/mo if you want Benchouse to keep watching. No obligation.

Not for you if

You can't give access to your agent, its logs and your database.
Nobody on your team can change the semantic layer.
You only want a report and don't plan to make changes.
You can't free 3–5 days in week 3 to apply the fixes.

No data team, or no semantic layer yet? I also build agents from scratch. Ask on the call.

Questions

What people ask before they book.

Does it work with our agent?

I can work with any vendor's agent, provided it exposes its logs and has an API or MCP server.

Where does our data go?

Your data stays in Benchouse, which is fully GDPR compliant. It's deleted when you delete your account. Privacy notice →

Our semantic layer is a mess, or we barely have one.

That's more common than not. I start with what you have and build the change list from there.

What access do you need?

Access to your agent so I can ask it questions, to your database, and to your dbt project if you have one.

We built our own agent.

I can work with that. Let's talk through your setup on the call.

Will you share our results with our vendor, or publish them?

Never with your vendor, and nothing is published unless you approve it.

How does the refund work?

Apply the changes in week 3. If the before-and-after isn't worth the fee, say so at the readout and I refund it in full. No forms, no argument.

When can we start?

I run one sprint at a time. Book a meeting to plan your sprint →

Price and terms

André Baaij. I ran data at Babylist and I run the public analytics agent benchmark. I do every sprint myself.

Accuracy Sprint

$10,000 founding price

1 of 3 founding places open. After that, $15,000.

Accuracy Sprint, four weeks: $15,000
Three months of Benchouse Growth ($999/mo): $2,997
Total value: $17,997

Apply the changes, and if the before-and-after isn't worth the fee, I refund it in full. Invoiced at the start. Say so at the readout. No forms, no argument.
Your part: connecting your systems and about 3 hours on the questions in week 1, then 3–5 days applying the changes in week 3. You pay your LLM and SQL costs.
In exchange for the founding price, you agree to a case study. You approve it before anything is published.
One sprint at a time, because I do the work myself.

Book a 30-minute agent review Ready to start? Email me for a start date →

One card per wrong answer.

Every failure gets a card like this one: the question, what the agent said, the right answer, why it went wrong, and the change that fixes it. The cards are ranked by how many answers each change fixes, so you start with the one that matters most.

Example card with invented figures. This is the format you'll get.

Question

"Net revenue by channel last quarter, excluding refunds?"

Agent said

€1.42M for paid search

Correct

€1.18M

Root cause

Refunds are in payments. No join from orders is declared, so the agent summed gross order value.

Change

relationships:
  - orders.order_id → payments.order_id
metrics:
  - net_revenue: gross − refunds

Also fixes

7 other failed questions

Who fixes it

Your semantic layer

How it runs

How the four weeks run.

Week 1

Baseline

We connect your systems, and over about 3 hours we design the 60–80 questions your people actually ask and confirm their answers. I run your agent on all of them three times, then split them: half to fix against, half held back. Already live? I start from your logs: the questions people really asked, and the answers that went wrong.
Week 2

Diagnose

I trace every wrong, missing or overconfident answer to the join, column, definition or instruction that caused it.
Week 3

Change

You get the change list, ranked by how many answers each change fixes. Your team spends 3–5 days applying it. I work alongside them or review the changes. Your team makes every design decision.
Week 4

Prove

I re-run the held-back questions three times each and walk you through the result: what improved, what did not, and what to watch from here.

Why the held-back questions matter

Anyone can raise a score they also grade.

If you tune the context using the same questions you test on, the score can rise without the agent getting better. I split your questions before I change anything. I use one half to guide the fixes and the other half to measure the result. Every vendor on the public benchmark is held to the same rule.

And every question runs three times. A single run of the same agent, with nothing changed, flips about one answer in six on its own, so one before-and-after proves nothing. Tools already suggest fixes; the sprint proves which ones work.

What I look for

Three kinds of problem, and who fixes each.

Facts

Something the agent needs is missing or wrong.

Missing joins, wrong join keys, wrong cardinality
Wrong grain: a join that double-counts
Missing columns and undefined metrics
Values nobody wrote down: status codes, "NL" vs "Netherlands"
Missing synonyms for the words your people use
No history, so "as of last quarter" cannot be answered
Empty or stale columns, broken references
A source that was never loaded

Design

How the layer is built gets in the agent’s way.

Context bloat: so much context the agent drowns in it
Symptom patches: a rule that fixes one question and breaks others
Tables the agent needs but cannot see
Model shape: wide vs narrow, the wrong grain for the questions asked
Untagged models, so nothing says which domain a table serves

Consistency

The layer contradicts itself.

One name, two meanings across models
Two definitions of the same metric
Docs that point at columns or models that no longer exist

Every finding tells you where the fix belongs: your semantic layer, your pipeline or the source system. You'll know which needs changing before anyone starts editing.

Bring the agent you run.

Genie, Cortex, Hex or your own. Thirty minutes with me, and I'll tell you whether a sprint will help. If it won't, I'll say so and point you somewhere that will.

Book a 30-minute agent review

Already had the call? Email me for a start date.