The Independent Analytics Agent Benchmark
Every configuration answers the same 300 questions (100 descriptive, 100 diagnostic, 100 prescriptive) against a deliberately messy simulated warehouse with withheld ground truth. No vendor can buy a position, or removal. How it is measured → Compare any two →
| 1 | Snowflake Cortex Agents | 83.7% | 99.7% | 100.0% | $0.59 | 1 |
| 2 | 74.4% | 100.0% | 100.0% | $0.35 | 2 | |
| 3 | 70.0% | 100.0% | 100.0% | $0.95 | 3 | |
| 4 | LLM not disclosed · supersimple semantic model | 69.7% | 98.3% | 100.0% | – | 4 |
| 5 | Databricks Genie LLM not disclosed · genie space instructions | 65.7% | 68.0% | 80.0% | – | 5 |
| 6 | 54.8% | 81.0% | 100.0% | $0.13 | 6 | |
| 7 | 53.6% | 95.4% | 100.0% | $0.31 | 7 |
Season 2026-S2 · question bank seed-v1 · universe seed 42 · judge claude-opus-4-8-r4.
Your agent seems fine. Benchouse finds out.
Benchouse traces wrong and refused answers in your agent’s real conversations back to their causes. Get proposed fixes for your semantic layer, dbt models, and agent context, ranked by how many questions each defect broke.
Show me what mine gets wrong30 minutes on your agent. I'll name the defects I can see.