Architecture Invariant
Zero DB Connections in Flight: The 2-Transaction LLM Tool Proxy
RH
By Rakib Hasan
•
Published September 28, 2026
•
6 min read
When an AI agent invokes an external API, such as searching deep databases or enriching company information, the upstream call can take anywhere from 400ms to 45 seconds. Holding an open database transaction across that network round-trip is how database connection pools exhaust and platforms crash under high concurrency.
Non-Negotiable Invariant #3
Zero database connections are held while an upstream request is in flight. This is why reserve and settle are strictly two separate, decoupled database transactions.
The Anatomy of a High-Throughput Proxy Call
Instead of wrapping the entire HTTP request in an ambient database session, olywork breaks every proxied execution into three isolated execution stages:
hold_id = await money.reserve(db, team_id, max_micro_usd)
await db.commit()
response = await relay.dispatch_upstream(target_url, headers, payload)
if response.is_success:
await money.settle(db, hold_id, actual_micro_usd)
else:
await money.release(db, hold_id)
await db.commit()
Why 1-Transaction Proxies Crash
In traditional web architectures, developers open a database session in middleware, perform the upstream request, record the audit row, and commit on response return.
- With a pool of 50 connections and average upstream latency of 3 seconds, a single team running 20 concurrent agent threads will monopolize 40% of the entire database pool.
- Under a burst of 100 simultaneous agent tasks, connection acquisition timeouts cascade into 500 errors across unrelated endpoints.
- By releasing the connection during Phase 2, olywork scales to 10,000+ simultaneous in-flight agent calls on a modest 20-connection PostgreSQL pool with sub-2ms connection acquisition latency.
RH
Rakib Hasan
Software Engineer at Liberate Labs | AI Researcher |