AI on AWS Is Mostly the AWS You Already Know

Most of an AI system on AWS is not AI. The model call is one box on the diagram. Everything that decides whether the system survives a bad Tuesday is the same plumbing AWS shops have run for a decade: queues, workflows, identity, storage, and logs. Teams that get this wrong start from the model catalog and work outward. Teams that get it right start from the failure modes and treat the model as an unreliable, expensive, slow dependency.

That last description is the useful one. A model call takes seconds to minutes. It fails transiently. It costs real money per request. Its output has to be checked before anything acts on it. You already know how to architect around a dependency like that. You did it for payment processors and fax gateways.

The Model Is an Endpoint

Bedrock gives you a managed API in front of several model families, billed per token, behind IAM. That is its main architectural virtue. Model access becomes an IAM permission, not a vendor key pasted into a secret, and usage lands in the same bill and the same CloudTrail as everything else.

Treat it as an endpoint and the design questions get familiar. Which callers may invoke which models. What the timeout is. What happens on a throttle. Where the request and response get recorded. None of those are prompt questions, and all of them come up before the first customer sees an answer.

I’ve argued before that inference cost is an architecture decision, not a procurement one. On AWS that shows up concretely: the cost of a feature is set by how many times your workflow calls the endpoint.

Long Work Needs a Durable Workflow

An agent that does real work runs longer than a request. It calls tools, waits on them, retries, and sometimes waits on a human. The tempting design is one long-lived process holding all of that in memory. It works in the demo. Then a deploy, a timeout, or a crash loses an hour of half-finished work, and nobody can say which side effects already happened.

AWS has had the answer for years in Step Functions, and it now has a second one inside Lambda. Lambda durable functions, announced in December 2025, checkpoint each step and can suspend on a wait or callback for up to a year without paying for compute while suspended.¹ Each invocation is still capped at fifteen minutes. The durability comes from the checkpoints.

Pick either. The rule is the same: every model call and every tool call is a step with a recorded result, so a retry resumes instead of repeating. That is the difference between an agent that worked once and one you can operate.

Managed Agent Runtimes Solve Part of It

Bedrock AgentCore is AWS’s managed layer for agents: a runtime, a gateway for tools, identity, memory, and observability, usable separately. The runtime gives each session its own microVM, ends it after fifteen minutes idle, and caps its lifetime at eight hours.²

That is a reasonable home for the interactive loop: a user talking to an agent that thinks, calls a few tools, and answers. It is not a substitute for a workflow engine. Session state is ephemeral by design, and AWS points you to its separate Memory service for anything that must survive.² Work that has to resume after a failure still wants the durable step history above.

The pattern I’ve had success with: the managed runtime hosts the conversational agent, and anything with side effects that matter gets handed off to a durable workflow the agent starts and checks on.

Identity Is the Hard Part

The question that takes longest on every AI project I’ve seen is whose permissions the agent uses. A Lambda execution role that can read every bucket will, eventually, read the wrong one on behalf of the wrong user.

The answer on AWS is the same one I gave for user code: the agent acts as the user, with credentials scoped to that user for that request. Tools enforce authorization on their side, from the caller’s identity, not from instructions in the prompt. A rule in a prompt is not a control. An IAM policy and a tool that checks the caller are.

AgentCore Identity and its gateway help with the OAuth plumbing for third-party tools. They do not decide your authorization model. Somebody still has to write down which users may do what through which tool.

Storage and Logs Are the Audit Trail

Store every model request, response, tool call, and decision with the IDs that tie them to a user and a workflow run. S3 for the payloads, DynamoDB or your existing database for the index, CloudWatch for the operational signal. Tracing is necessary but observability stops at the trace: the trace tells you what happened, and the stored record lets you decide whether it should have.

This is also where evaluation comes from. The recorded production traffic is your test set. Teams that skip this step end up evaluating against examples they wrote themselves, which measure the examples.

The Shape

A request arrives through API Gateway or a queue. A durable workflow owns the job. Each step calls Bedrock or a tool with the user’s scoped credentials, records the result, and moves on. Anything slow or approval-bound waits on a callback. Outputs pass a check before they touch a system of record. Everything lands in storage you can query.

None of that diagram is new. The new parts are an endpoint that is slower, pricier, and less predictable than the ones you’re used to, and a managed runtime that is useful for the interactive loop. Design for the dependency you actually have and the AWS pieces mostly pick themselves.

AWS, “AWS Lambda announces durable functions for multi-step applications and AI workflows,” December 2, 2025; and the Lambda durable functions guide. ↩
AWS, “How Amazon Bedrock AgentCore Runtime works” and “Configure lifecycle settings.” ↩ ↩²