How does this even work?

How do we route?

Yes, I was confused myself since we kept adding rules to the scaling and query routing based upon logging, failures, cost optimizations, performance tuning, and recently started looking into query transpiling to access the different engines (not quite GA). I could not keep track of the current scaling and decision tree in my head anymore since it's constantly changing with each piece of new information and intuition from our technical staff.

For this reason, I put together a visual representation of the router decision tree with some mocked Iceberg datasets that push our actual unit tests into the public realm for better or worse so that we can see the stages of decisions and maintain a public channel over time.

Moving towards minimum viable compute

When using resources, there remains a surplus of compute (if it’s not too little) which represents the multidimensional zone of compute, memory, and storage needed to successfully complete an operation. Serverless compute has a huge advantage here, especially under conditions where cold start times can be swallowed easily, like analytics workloads or agent sessions. If you're looking for something that’s always available we recommend connecting an always available compute instance for your Iceberg catalog like we did with ClickHouse Cloud for some of our blog posts.

It’s hard to know ahead of time how complicated a query is without running an EXPLAIN PLAN type operation, so we keep tuning some statistics on our side, parsing the AST and running the numbers of the Iceberg metadata we have available. In the end, we still index on the side of headroom since it would be insane to try and grind against the constraint lines and fail data operations trying to size our on-demand serverless at such a low level of clearance that we start failing queries when sizing resources.

How does this work with an agent?

The router works the same way for an agent as if it's a real user; clients are just clients. We're currently working on a set of granular permissions to restrict agent access at the DB level. All of the compute we provision is scoped to only one client (or agent), which does create a natural trade off between isolation and concurrency. If users wished to create concurrent stateful served compute with their Iceberg catalogs, we recommend connecting the catalog to ClickHouse (or another provider) to have a more server type experience. One of the interesting tidbits of an agent is that it often does single actions at a time, waits for the response and takes another action, and often it's another query... and the sequence is continued until a non-deterministic answer is arrived at, sometimes the result is even right, amazing, what a time to be alive. And if it's not, we just append another instance of "think harder" to the end of our skills and try again. (This is satire)

This has led to our agent simulator, a tale of how agents & oleander make decisions together to arrive at a goal.