postgrok

infinity

Properties3
tagsrust, agent-runtime, concurrency, aws, tool
typeentity
statusseedling

3 min read

Infinity

A Rust framework for building massively concurrent agentic systems, released 2026-10-23 under Apache 2.0 as part of the Hydro project, a collaboration between AWS and the Sky Computing Lab at UC Berkeley. Announced on the AWS Open Source Blog by Shadaj Laddad and positioned as an early research project rather than a managed service https://aws.amazon.com/blogs/opensource/introducing-infinity-a-runtime-for-extreme-agent-concurrency/. Repository https://github.com/hydro-project/infinity, homepage https://infinity.hydro.run/.

The design in one line

An agent is a stackless coroutine. When a message arrives it runs a bounded slice, persists its state and dispatches at most one tool call, and then yields. Between turns an agent is only its stored conversation history, and the runtime doesn’t keep a task, a stack or an open connection for it https://infinity.hydro.run/docs/infinity-runtime/overview. A waiting agent doesn’t cost compute, which is the property a process-per-agent harness cannot offer.

The pieces

  • Slices: a message arrives and history loads from a pluggable durable store. Then one model completion, state persisted, tool dispatched, and the slice ends. Results arrive later as new messages and start the next slice.
  • RAP, the Reactive Agent Protocol: tools are served over the network by fire-and-forget HTTP POST with a callback URL, and results and subscription events are delivered whenever ready. That includes agents that currently have no process. Existing MCP servers run unchanged through a compatibility layer https://infinity.hydro.run/docs/rap/what-is-rap.
  • Synthetic tool calls: asynchronous events are injected into conversations as tool calls suffixed _injected, because the three message kinds a model already understands are user, system, and tool call, and misattributing events as user messages would break that contract.
  • Serverless: the same agent code runs embedded in a resident process or step-per-message on AWS Lambda, where scale-to-zero is the default. The Lambda path uses SQS FIFO queues and Aurora DSQL for history, with DynamoDB for deduplication and bundled CDK constructs.
  • Infinity Code: a coding agent built on the runtime, with background builds and tests whose output streams in. It runs parallel threads with stacked sandboxes and returns results as diffs to review.

The density claim, with its conditions

AWS copy says Infinity fits 75k agents in the memory of a Raspberry Pi. The stated measurement is 75,000 idle concurrent agents in the 8 GB of a Raspberry Pi 5, tested against a simulated model on a single server thread with in-memory stores https://techbytes.app/posts/infinity-a-runtime-for-extreme-agent-concurrency. An independent rerun matched the figure at 20 turns of history per agent and measured about 5 KB per additional tool-calling turn, roughly 10,100 agents at 160 turns https://agentopsguide.com/aws-infinity-agent-runtime. The measurement covers idle agents with short histories, not long conversations or live inference, and the vault’s skepticism about headline numbers applies exactly in that gap. See wet-not-dry for why the conditions are restated here rather than left to a link.

Relation to the vault’s argument

Infinity is Rust as agent substrate, the building-on side of rust-for-agents rather than the writing-in side. Its slice-and-durable-storage architecture is the economic-motive sibling of durable-agents. Pi checkpoints steps so a crash doesn’t lose work, Infinity suspends steps so idleness doesn’t cost compute, and both need history in durable storage and results as messages. Its message-queue substrate is also a standing answer to the proactive-agents trigger problem: the wake-up event is the design’s center, not an add-on.

Sources

Graph View

Backlinks