postgrok

2026-11-02-youtube-lets-get-rusty-amazon-infinity-runtime

Properties3
tagsrust, coding-agents, agent-runtime, concurrency, youtube
typesource
statusseedling

6 min read

YouTube: Amazon built a new AI agent runtime in Rust

Source: YouTube video os5eNeAVRpc, “Amazon built a new AI agent runtime in Rust” https://www.youtube.com/watch?v=os5eNeAVRpc, uploaded 2026-11-02 on the Let’s Get Rusty channel https://www.youtube.com/@letsgetrusty, 7:06. oEmbed record read 2026-11-04: title and author as above.

Ingested: 2026-11-04, from the auto-generated English captions of the full video, filed at Inbox/2026-11-02-amazon-built-a-new-ai-agent-runtime-in-rust-os5eNeAVRpc-transcript.txt. The architectural claims were checked against the AWS announcement and the project docs the day of filing; the checks are inline below.

Summary

lets-get-rusty walks through the runtime AWS announced on 2026-10-23 on its Open Source Blog, in Shadaj Laddad’s post “Introducing Infinity: a runtime for extreme agent concurrency” https://aws.amazon.com/blogs/opensource/introducing-infinity-a-runtime-for-extreme-agent-concurrency/. Infinity is a Rust framework from the Hydro project, a collaboration between AWS and UC Berkeley’s Sky Computing Lab, released under Apache 2.0 as a research prototype rather than a managed service. The presenter’s framing: agents spend most of their lives waiting (builds take twenty minutes, webhooks arrive hours later, humans reply the next morning), and a process-per-agent harness pays for all that waiting. Infinity hibernates the waiting agent and wakes it when a message arrives (0:26-0:54). See infinity.

The video is a faithful relay. Its one substantive gap is the headline number: “light enough to fit 75,000 agents in the memory of a Raspberry Pi, which is about 8 GB” (0:06-0:17) repeats AWS’s own copy, and the conditions the figure depends on appear in the benchmark but not in the video.

The architecture as relayed, checked against the blog

  1. Slice execution (2:08-3:08). A slice begins when one or more messages arrive on a thread’s input queue: load message history from durable storage, preprocess inputs, call the model provider (the blog names Anthropic and OpenAI backends), persist the slice state, dispatch any tool call, end. Confirmed against the blog and the runtime docs https://infinity.hydro.run/docs/infinity-runtime/overview.
  2. The async analogy (0:56-1:12, 3:10-4:02). The presenter maps the design to Rust async/await. A future is a compiler-generated state machine, an await is a pause point, and each tool call is an await point for the agent. The blog has a section titled “Agents as stackless coroutines” making exactly this case, and it credits the memory difference to stackless coroutines keeping only the values live at each pause. Confirmed.
  3. RAP (2:48-3:08). Tool calls are posted to a Reactive Agent Protocol tool server, which executes them while the runtime does other work and sends the result back to the input queue minutes, hours, or days later. Confirmed: RAP is the project’s open protocol, an HTTP POST carrying a callback URL, results delivered asynchronously, and existing MCP servers run unchanged through a compatibility layer https://infinity.hydro.run/docs/rap/what-is-rap.
  4. Synthetic tool calls (4:16-5:36). Because a model conversation only carries user messages, system messages, and tool calls, Infinity delivers events as tool calls that the runtime injects, distinguished by a _injected suffix on the name. The blog uses the term and the suffix; both confirmed in the announcement page. The presenter’s reasoning for the design, that a new user message would misattribute the event to the human, is his gloss and matches the docs’ stated rationale for threaded child handling.
  5. Shutdown between slices (5:12-5:54). No in-process state machine tracks outstanding work, so the runtime can end completely between slices; the pending result arrives as a message and the next slice rebuilds history from storage. Scale to zero on AWS Lambda follows, and the docs carry a CDK path for it. Confirmed; this is the property the whole design buys.

The conditions the video leaves out

The 75,000-agent figure is an idle-density measurement with stated conditions. AWS tested 80,000 agents against a simulated model on a single thread of an AMD EPYC server CPU with in-memory stores, and the Raspberry Pi is the extrapolation https://techbytes.app/posts/infinity-a-runtime-for-extreme-agent-concurrency. An independent rerun matched 75,000 agents at 20 turns of history per agent and found each additional tool-calling turn costs about 5 KB per agent, putting roughly 10,100 agents with 160 turns of history inside the same 8 GB https://agentopsguide.com/aws-infinity-agent-runtime. The presenter’s own use case, hundreds of thousands of concurrent customer-service agents each mostly waiting (6:09-6:19), is exactly the regime the benchmark measures, and exactly the regime where long-lived conversations push the count past the headline. This is a gap in the relay, not an error in it: the video repeats the primary’s marketing line, and the primary itself states the test conditions.

The captions garble three names: the course plug at 1:55 comes out “Voss Lab accelerator”, which is the Rust Live Accelerator, named correctly at 6:32; “RIP tool server” at 4:02 and 4:10 is the RAP tool server; “Affinity” at 6:17 and 6:21 is Infinity. Two ad segments run 1:49-2:06 and 6:24-6:53, about 11% of runtime, well below the share in the channel’s October video.

My reaction

The interesting object here is not the density number, it is the ordering of motives the design encodes. Pi’s durability work checkpoints steps so a crashed process doesn’t lose work, which is a correctness motive wearing an availability story. Infinity suspends agents between steps so an idle thread doesn’t cost compute, which is an economic motive wearing the same story. Both land on the same primitives, bounded execution slices, durable history, results delivered as messages, which is what you would expect once waiting rather than computing became the dominant cost of an agent’s life. See durable-agents, where this belongs next to the Pi machinery, and rust-for-agents, where the channel’s own October argument, Rust as a substrate for building agents on, is the category this runtime falls into.

Touched in the vault

Named without pages: Shadaj Laddad, AWS, UC Berkeley Sky Computing Lab, Hydro project.

Sources

Graph View

Backlinks