ServicesProductBenchmarkBlogDocs Log in
Docs menuCustom agent

Custom agent

Connect any agent that stores its conversations in a database.

Updated 3 October 2026

Use a custom agent when your agent stores its conversations in a database. You give Benchouse a connection string and one SQL query. We run the query on a schedule as a read-only user, and read nothing else.

The columns the query returns are listed in the turns/v1 contract. Each database page has an example query.

Connection string

DatabaseConnection string
PostgreSQL, Supabase, Neon, Aurorapostgres://benchouse_read:PASSWORD@HOST:5432/DB?sslmode=verify-full
MySQL, MariaDB, Aurora MySQL, PlanetScalemysql://benchouse_read:PASSWORD@HOST:3306/DB?tls=true
SQL Server, Azure SQLsqlserver://benchouse_read:PASSWORD@HOST:1433?database=DB&encrypt=true
ClickHouseclickhouse://benchouse_read:PASSWORD@HOST:9440/DB?secure=true
Snowflakesnowflake://benchouse_read:TOKEN@ACCOUNT?warehouse=WH&role=BENCHOUSE_READ

The connection must use TLS, and the host must be reachable from the internet. We store the connection string encrypted and never display it again.

The query

  • Write one SELECT. A WITH clause is fine; a second statement is rejected.
  • Return one row per turn, with the columns in the turns/v1 contract. Use AS to rename your own columns.
  • Leave out ORDER BY and LIMIT at the top level. We add our own to page through the results.
  • On SQL Server, also leave out WITH. Put that logic in a view and select from the view.
  • Selecting from a view works on every database: SELECT * FROM benchouse.turns.

Connect as a user that can only read the tables the query uses. As a second safeguard, we run the query in a read-only transaction on PostgreSQL and MySQL, in a transaction that is always rolled back on SQL Server, and with readonly set on ClickHouse.

Testing the connection

Test connection runs the query without reading any rows. If a required column is missing or has the wrong type, the test fails and names the column.

Errors

ProblemWhat happens
A required column is missing or has the wrong typeThe run fails and the agent’s page names the column. Nothing is stored.
A row has NULL in a required column, or a value of the wrong typeThe row is skipped and the rest are stored. We log the turn ID.
A row is larger than 4 MiBThe row is skipped. The result sample is usually the cause.
The query returns many rowsEach run reads up to 100,000 rows, or for up to 10 minutes, and the next run continues where it stopped.
A single query runs for more than 10 minutesThe run fails and is retried on the next run.

Index turn_id and the column we read changes by (updated_at or asked_at). Every run filters and sorts on them.