SERVERLESS SYSTEMS

Field guide · Serverless on AWS

Serverless
system design,
from first principles.

Seven patterns cover most of what gets built on AWS without servers — from a static site to a compliance-grade audit trail. This guide walks through each one: the diagram, how a request actually moves through it, what tends to go wrong, and how to keep it secure. Written for builders early in their careers who want the shape of good systems, not just the service names.

LEVEL — EARLY CAREER · PM · BUILDER READING TIME — ~25 MIN PATTERNS — 07
00

The primer: what “serverless” buys you

MANAGED SERVICES · PAY-PER-USE · NOTHING TO PATCH

Every system does some mix of five jobs: it accepts requests, runs logic, stores state, moves messages, and keeps records. Traditionally you did all five on machines you owned — which meant patching, scaling, capacity planning, and 2 a.m. pages. Serverless design means picking a managed AWS service for each job, so the undifferentiated work belongs to AWS and your effort goes into the part that's actually yours: how the pieces fit together.

That's why system design matters more, not less, in a serverless world. The services are commodities; the architecture is the product. Three properties are worth internalizing early:

Pay-per-use. Lambda bills per invocation, DynamoDB per request, S3 per gigabyte. An idle system costs approximately nothing — ideal for internal tools with bursty, unpredictable traffic. Scale-to-zero and scale-to-lots. The same design serves ten requests a day or ten thousand a minute without re-architecture. Everything is an API call. Every action against every service is an authenticated, loggable API call — which is precisely what makes serverless systems so amenable to audit and access control, a theme we'll return to in chapter 07.

The building blocks

You can assemble nearly everything in this guide from about a dozen services. Meet the cast:

ServiceWhat it doesThink of it as…
CloudFrontGlobal CDN — caches and serves content close to users, terminates TLS.the front door and doorman
API GatewayManaged HTTP front-end — routing, throttling, auth hooks, request validation.the receptionist who checks IDs
LambdaRuns your code in short bursts, triggered by events. No servers to manage.a contractor paid by the minute
S3Object storage — files, documents, exports, evidence, backups. Eleven nines of durability.an infinite filing cabinet
DynamoDBKey-value database with single-digit-millisecond reads at any scale.a ledger with instant lookup
SQSMessage queue — buffers work between a producer and a consumer.an inbox tray between two desks
EventBridgeEvent bus — publishes facts (“a thing happened”) and routes them by rule to many targets.the office announcements channel
SNSPub/sub notifications — fan a message out to email, SMS, or other services.a megaphone with a mailing list
Step FunctionsWorkflow orchestrator — multi-step processes with branching, retries, and human approvals.a project manager who never forgets a step
CognitoUser sign-in and identity — issues the tokens other services verify.the badge office
IAM & KMSWho may do what (IAM), and encryption keys with their own access controls (KMS).the locks, and who holds which key
AthenaSQL queries directly over files in S3 — no database to run.a librarian who reads the whole archive on demand
CloudTrail / CloudWatchWho called which API (Trail); logs, metrics, and alarms (Watch).the security camera and the smoke detector

Reading the diagrams

Every diagram in this guide uses the same visual language, borrowed from AWS's own architecture-icon conventions: nodes are coloured by category, solid arrows are synchronous calls (the caller waits), dashed arrows are asynchronous events (fire and move on), and numbered badges match the step-by-step walkthrough beneath each diagram.

01

Static site behind a CDN

S3 + CLOUDFRONT + CI/CD — THE PATTERN SERVING THIS PAGE

The simplest serverless system, and the one you're using right now: HTML, CSS, and JavaScript sit in an S3 bucket, CloudFront caches them at edge locations around the world, and a CI/CD pipeline redeploys the whole site on every push to main. There is no server, no runtime, no database — and therefore almost nothing to attack, patch, or pay for.

Don't let the simplicity fool you into thinking it's only for brochures. Internal documentation portals, dashboards that read from an API (pattern 02), report viewers, training material — a surprising share of internal tooling is "a static front-end plus a small API."

content changes on deploy, not per user global, fast, cheap delivery minimal attack surface
PATTERN 01 — STATIC DELIVERYSYNC ─── · ASYNC ┄┄┄
Requests never touch a server you run; deploys are a sync-and-invalidate, not a release window.
  1. A browser requests the page. DNS (Route 53) points at CloudFront, which serves the file from the nearest edge cache — often without touching the origin at all.
  2. On a cache miss, CloudFront fetches the file from the private S3 bucket (the “origin”) and caches it per its Cache-Control headers.
  3. To ship a change, a developer pushes to GitHub; a GitHub Actions workflow syncs the files to S3.
  4. The workflow then tells CloudFront to invalidate its cache, so edges refetch the new files.

Watch out for

  • Cache-control strategy. Long-cache your assets (versioned filenames or query strings), short-cache your HTML — or users will see stale pages for days.
  • “Static” means static. The moment you need per-user data, secrets, or writes, you've grown into pattern 02. Don't smuggle API keys into client-side JavaScript.

Security notes

  • Keep the bucket private; grant CloudFront read access via Origin Access Control so nobody can bypass the CDN (and its TLS, logging, and WAF) to hit S3 directly.
  • The deploy pipeline is your real attack surface here — protect the main branch and scope the CI role to this bucket only.
02

The serverless API

API GATEWAY → LAMBDA → DYNAMODB — THE WORKHORSE THREE-TIER

This is the pattern to know cold, because it's the serverless answer to the classic three-tier web app: a front door, business logic, and a database. A client calls API Gateway, which authenticates the request and hands it to a Lambda function, which reads or writes DynamoDB and returns a response — all synchronous, all within a couple hundred milliseconds, all fully managed.

Nearly every internal tool starts here: a CRUD API over a well-modeled table. The discipline that pays off is designing the DynamoDB table around your access patterns — the exact queries you'll ask — rather than around entities the way you would in SQL. Write those queries down first; the table design follows.

request/response in real time CRUD over well-known queries per-user auth on every call
PATTERN 02 — SYNCHRONOUS APISYNC ─── · ASYNC ┄┄┄
The caller waits end-to-end — so every hop on this path must be fast, and every failure returns to the user.
  1. The client (your static site from pattern 01, say) sends an HTTPS request with a token obtained by signing in through Cognito.
  2. API Gateway validates the token, applies throttling, and rejects malformed requests before any of your code runs.
  3. The matched route invokes a Lambda function with the request as an event. The function holds your business logic — and an IAM role scoped to exactly the tables it needs.
  4. Lambda reads or writes DynamoDB and returns a response back up the same path.
  5. Lambda's logs and metrics land in CloudWatch automatically. API Gateway's access logs and X-Ray tracing are opt-in stage settings — turn them on when you build the stage; you want the full picture before the first incident, not after it.

Watch out for

  • Cold starts. A function that hasn't run recently takes extra time (~100ms–1s) on its first invocation. Usually fine; matters for latency-critical paths.
  • Doing slow work synchronously. If a request triggers anything that takes more than a second or two — PDF generation, third-party calls, LLM inference — accept the request, return 202, and finish it asynchronously (pattern 03).
  • Modeling DynamoDB like SQL. There are no joins and ad-hoc queries are painful. Access patterns first, table second.

Security notes

  • Authenticate at the edge: a Cognito or Lambda authorizer on API Gateway means unauthenticated requests never reach your code.
  • One function, one role: each Lambda gets an IAM role allowing only its own tables and actions — dynamodb:GetItem on one table, not dynamodb:* on everything.
  • Validate input at the gateway with request schemas; it's free defense-in-depth against injection and malformed payloads.
03

Queues & asynchronous work

SQS BETWEEN PRODUCER AND WORKER — ABSORB BURSTS, SURVIVE FAILURES

The single most important idea in distributed systems is decoupling: letting the part that accepts work run at a different speed — and fail at different times — than the part that does the work. A queue is how. The producer drops a message in SQS and immediately responds to the user; a worker Lambda picks messages up at its own pace and processes them.

This buys you three things at once. Burst absorption: a spike of 10,000 requests becomes a backlog that drains smoothly instead of a wall of timeouts. Failure isolation: if the worker (or a downstream dependency) is broken, messages wait safely in the queue instead of being lost. Retries for free: a message that fails processing reappears on the queue and is tried again — and after too many failures, moves to a dead-letter queue (DLQ) for a human to inspect.

work takes seconds, not milliseconds bursty or spiky load must not lose requests
PATTERN 03 — QUEUE-BASED ASYNCSYNC ─── · ASYNC ┄┄┄
The user's request ends at the queue; everything to the right happens on the system's schedule, not the user's.
  1. The client submits a job through API Gateway and a thin producer Lambda, exactly as in pattern 02.
  2. The producer validates the request, writes a message to SQS, records the job as PENDING, and returns 202 Accepted with a job ID — total elapsed time, tens of milliseconds.
  3. A worker Lambda polls the queue (AWS manages the polling) and processes messages in small batches, updating the job record as it goes.
  4. If processing throws, the message becomes visible again and is retried; after N failures, SQS moves it to the DLQ, which alarms.
  5. The client checks status via GET /jobs/{id} — or gets notified through pattern 04.

Watch out for

  • Idempotency. SQS guarantees at-least-once delivery, so your worker will occasionally see the same message twice. Design handlers so processing twice equals processing once (e.g., conditional writes keyed on the job ID).
  • A silent DLQ. A dead-letter queue nobody monitors is a place where work goes to die. Alarm on DLQ depth > 0.
  • Ordering. Standard queues don't guarantee order. If order matters, use a FIFO queue and accept lower throughput.

Security notes

  • Queues carry your data at rest — enable KMS encryption on the queue and restrict sqs:SendMessage to the producer role only.
  • Treat message contents as untrusted input in the worker; validate again there. The queue is a boundary, and boundaries re-validate.
04

Event-driven fan-out

EVENTBRIDGE — PUBLISH FACTS, LET SUBSCRIBERS REACT

A queue connects one producer to one consumer. But mature systems have facts that many parts care about: a document was submitted, an assessment was approved, a policy changed. An event bus inverts the relationship — the producer publishes the fact once, to EventBridge, without knowing or caring who's listening. Rules on the bus match events by pattern and route copies to every interested target.

The architectural payoff is extensibility. When a new requirement arrives — “we also need to notify the review team,” “we also need to update the search index” — you add a rule and a subscriber. The producing code doesn't change, isn't redeployed, and can't be broken by the addition. Compare that to a Lambda that calls four downstream services in a row: every new consumer is a code change, and any consumer's failure is the producer's problem.

one fact, many reactions teams/features evolving independently “don't make the producer care”
PATTERN 04 — EVENT FAN-OUTSYNC ─── · ASYNC ┄┄┄
Producers publish once; rules decide who hears about it. New consumers are additions, not modifications.
  1. Something happens — a Lambda from pattern 02 or 03 finishes its work and publishes an event like document.submitted to the EventBridge bus, with the details in the payload.
  2. Rules match events by pattern (source, type, fields) and route a copy to each configured target.
  3. One rule triggers a notification Lambda (email the reviewer via SNS/SES); another feeds an SQS queue in front of an indexing worker (a queue per consumer keeps a slow consumer from dropping events); a third starts a Step Functions review workflow (pattern 05).
  4. The bus archive retains events so you can replay history — into a new consumer, or after fixing a buggy one.

Watch out for

  • Event schemas are contracts. Consumers you don't know about depend on your field names. Version events, add fields rather than renaming, and document them like an API.
  • Debugging is different. There's no single call stack; a “request” is now a trail of events. Put a correlation ID in every event and log it everywhere, or you'll be grepping in the dark.
  • Eventual consistency. Subscribers react in milliseconds-to-seconds, not instantly. Interfaces should communicate “submitted, processing” honestly.

Security notes

  • Scope events:PutEvents to specific sources per role, so a compromised function can't forge someone else's events.
  • Events sprawl by design — make sure payloads carry references, not sensitive blobs (an ID and an S3 pointer, not the personal data itself).
05

Orchestrated workflows

STEP FUNCTIONS — MULTI-STEP PROCESSES WITH BRANCHES, RETRIES & HUMANS

Some processes are too important to leave implicit. An intake review, an access request, an approval chain — these have states, branches, deadlines, and often a human in the loop. You could wire them together with queues and events, but the logic ends up smeared across a dozen functions where nobody can see it. Step Functions makes the workflow itself a first-class, visible artifact: a state machine you can read as a diagram, where every execution records exactly which path it took, with what data, and why.

That last property is quietly the killer feature for governance-heavy work: the execution history of a state machine is documentation that the defined process was followed — every time, with timestamps. One honest caveat: Step Functions only keeps that history for about 90 days after an execution completes (and Express workflows only record history via CloudWatch Logs at all). So treat it as the working record, not the archive — the durable proof lives wherever you export it, which is exactly what pattern 07 is for.

multi-step process with branching human approval in the loop must prove the process was followed
PATTERN 05 — ORCHESTRATED WORKFLOWSYNC ─── · ASYNC ┄┄┄
The dashed boundary is one state machine: the process is defined once, visible to everyone, and every run is recorded.
  1. A submission (from pattern 02's API, or an event from pattern 04) starts an execution of the state machine.
  2. A validate step (Lambda) checks completeness and enriches the request; transient failures retry automatically with backoff — no code for that, it's declared on the state.
  3. The workflow pauses for human review using a callback token: it sends the reviewer a task (via the notification patterns you already know) and waits — for hours or days — costing nothing while it waits.
  4. A choice state branches on the decision: approved requests are recorded to DynamoDB; rejections notify the requester with the reason via SNS.
  5. Either way, the execution history — every state, input, output, and timestamp — is queryable for ~90 days, and a completion event carrying the decision, actor, and timestamps is published for pattern 07 to record durably. The event is the evidence; the history is the debugger.

Watch out for

  • Don't orchestrate everything. Step Functions shines when the process is the point. For simple “A then B,” a queue is cheaper and simpler. Orchestrate workflows; choreograph plumbing.
  • State payload limits. Executions pass data between states with a size cap (~256 KB) — pass S3 references for anything bulky.
  • Standard vs. Express. Standard workflows run up to a year (human approvals); Express workflows are for high-volume, short-lived runs. Pick deliberately.

Security notes

  • The state machine's role should be able to invoke its steps and nothing else — it's the conductor, not a superuser.
  • Approval callbacks must verify the approver: the “approve” endpoint authenticates the human (Cognito, pattern 02) and checks they're authorized for this request. The task token proves which execution; it doesn't prove who.
06

Ingestion & analytics pipelines

S3-TRIGGERED PROCESSING INTO A QUERYABLE ARCHIVE

Systems that deal in documents and evidence — reports, exports, uploads, records from other systems — converge on a pipeline shape: files land in an S3 bucket, landing triggers processing, processing produces both structured metadata (for the application to query instantly) and curated files (for analysis at leisure). The elegant trick is that S3 itself is the database for the analytical half: Athena runs SQL directly over the files, so there is no warehouse to run, size, or patch.

This “raw bucket → processed bucket → query in place” shape is a miniature data lake, and it scales from a hundred files to a hundred million without changing the picture. The raw bucket also quietly serves another purpose: it preserves the original, untouched input — which matters the moment anyone asks you to prove what was received.

files arrive, insight must follow keep originals, query derivatives reporting without a warehouse
PATTERN 06 — INGEST & ANALYZESYNC ─── · ASYNC ┄┄┄
Two products from one ingest: hot metadata for the app, a cold queryable archive for analysis — originals preserved untouched.
  1. A file arrives in the raw bucket — uploaded through the API (via a pre-signed URL, so bytes go straight to S3 without transiting your Lambda), or delivered by another system.
  2. The ObjectCreated event triggers a processing Lambda (through a queue, per pattern 03, if volumes are spiky).
  3. The processor validates and parses the file, writes extracted metadata to DynamoDB for the application's real-time queries…
  4. …and writes a normalized, columnar copy (e.g., Parquet, partitioned by date) to the curated bucket. The original in raw is never modified.
  5. Athena queries the curated data with plain SQL — feeding dashboards, scheduled reports, or ad-hoc questions — paying only per query.

Watch out for

  • Partition or pay. Athena bills by data scanned. Partitioning by date (year=2026/month=07/) and using columnar formats cuts both cost and latency by orders of magnitude.
  • Poison files. One malformed upload shouldn't stall the pipeline — route failures to a DLQ with the object key, and process files independently.
  • Small-file sprawl. Millions of tiny files make Athena slow; compact them periodically.

Security notes

  • Treat every uploaded file as hostile: constrain pre-signed uploads (short expiry, fixed content type — and note a plain pre-signed PUT can't cap file size; use an S3 POST policy with content-length-range for that), scan or sandbox parsing, and never execute or render uploads server-side without hardening.
  • Separate roles per stage: the uploader can only PutObject to raw; the processor reads raw and writes curated; analysts read curated only. Nobody needs write access to raw after landing.
  • Encrypt both buckets with KMS and block all public access at the account level.
07

The compliance-grade audit trail

APPEND-ONLY EVENTS · STREAMS · WORM ARCHIVE — RECORDS YOU CAN DEFEND

Every pattern so far does things. This one remembers them — in a way that holds up when someone skeptical asks. For governance, risk, and compliance systems, the audit trail isn't a logging afterthought; it's a load-bearing feature with three hard requirements: it must be complete (every consequential action recorded), immutable (nobody — including administrators — can quietly edit history), and queryable (an auditor's question is answerable in minutes, not weeks).

The design principle: record events, not state. Don't just store “assessment #42 is approved” — store the append-only sequence of facts that got it there: submitted by whom, at when; reviewed by whom; approved on what basis. Current state is derivable from events; events are not derivable from state. And crucially, the trail has two layers: your application's events (what users did) and CloudTrail (what every IAM identity did to the infrastructure itself). Auditors will ask about both.

GRC / compliance systems “who did what, when, and prove it” regulator- or auditor-facing records
PATTERN 07 — TAMPER-EVIDENT AUDIT TRAILSYNC ─── · ASYNC ┄┄┄
Two layers of record — application events and infrastructure API calls — both draining into write-once storage.
  1. Application actions (an approval from pattern 05, an ingestion from pattern 06) append an event to a DynamoDB events table. Append-only takes two locks, not one: IAM grants roles PutItem — no UpdateItem, no DeleteItem — and every write carries an attribute_not_exists condition expression, because a plain PutItem on an existing key would silently overwrite it. Unique event IDs plus that condition close the loophole.
  2. DynamoDB Streams emits every write to an archiver Lambda — the application can't skip this step; it happens at the database layer. (Streams orders changes per item, not across the whole table — so each event carries its own timestamp and sequence number, and those are the chronology, not stream order.)
  3. The archiver batches events into the archive bucket with S3 Object Lock in compliance mode: write-once-read-many (WORM) storage that even the root account cannot alter or delete until the retention period ends.
  4. In parallel, CloudTrail records AWS API activity — including any attempt to tamper with this very pipeline — into its own locked bucket. By default that means management events (creating, configuring, deleting things); reads and writes of the data itself — S3 object gets, DynamoDB item calls — are data events you must enable explicitly for the buckets and tables that matter.
  5. When the auditor asks, Athena answers with SQL across the archive: every action on record #42, everything a given user touched in March, every permission change this quarter.

Watch out for

  • Completeness is a design review, not a hope. Enumerate consequential actions and verify each emits an event — an audit trail with gaps is worse than none, because it implies coverage it doesn't have.
  • The stream is a 24-hour buffer, not a guarantee. If the archiver is broken or throttled longer than the Streams retention window, records expire unarchived — a silent gap. Alarm on the stream's iterator age, give the archiver an on-failure destination, and treat that alarm as a sev-2.
  • Retention cuts both ways. Regulations set floors (keep at least N years) and privacy laws set ceilings (don't keep personal data longer than needed). Decide retention per record class, deliberately, up front.
  • Compliance-mode Object Lock is genuinely irreversible. Test retention settings on a scratch bucket first; a misconfigured 10-year lock is a 10-year mistake.

Security notes

  • Separate the watcher from the watched: ship CloudTrail and the archive to a dedicated logging account that operators of the workload can read but never write. A trail an admin can edit is testimony, not evidence.
  • Alarm on the meta-signals: CloudTrail configuration changes, Object Lock policy changes, and denied write attempts against the archive.
  • Include integrity aids — event hashes or CloudTrail's built-in digest files — so you can demonstrate, not just assert, that history is intact.
08

The security layer

CROSS-CUTTING CONCERNS — THE SAME SIX MOVES, EVERY PATTERN

Notice what the security notes across seven patterns kept repeating. That's not laziness — serverless security really is a small number of moves applied consistently. AWS handles the layers below you (hypervisors, OS patching, physical security); under the shared-responsibility model, what remains yours is configuration: who can do what, to which data, and how you'd know. The good news for the security-minded: configuration is reviewable, diffable, and auditable in a way that patch levels never were.

IAM

Identity is the perimeter

In serverless there's no network wall to hide behind — every request to every service is decided by IAM. One role per function, scoped to exactly its own resources and actions. When a function is compromised, its role is the blast radius.

KMS

Encrypt by default

S3, DynamoDB, SQS, and logs all encrypt at rest with a checkbox — check it, with a customer-managed KMS key for sensitive data. The key's own policy becomes a second, independent lock: access requires both the resource permission and the key.

Boundaries

Validate at every edge

Authenticate and validate at the outermost layer (API Gateway authorizers, request schemas, WAF), then re-validate at internal boundaries — queue consumers, event handlers, file parsers. Trust is never transitive across an async hop.

Secrets

Secrets are infrastructure

API keys and credentials live in Secrets Manager or SSM Parameter Store, fetched by role at runtime — never in code, config files, or client-side JavaScript. Rotation becomes an operation, not a migration.

Detection

Watch the watchers

CloudTrail on, in every region, delivered to a bucket the workload can't write. Alarm on the control plane: IAM policy changes, trail configuration changes, failed authorization spikes. Most incidents are visible in the API record before they're visible anywhere else.

Data

Hold less, deliberately

The cheapest data to secure is data you didn't keep. Classify what each store holds, minimize what events and logs carry (IDs and pointers, not payloads), and set retention per record class — privacy regulation makes over-retention a liability, not a safety net.

If you internalize one thing: in serverless systems, the IAM policy review is the architecture review. The diagram tells you what the system is supposed to do; the roles tell you everything it can do. Keep the gap between those two as close to zero as you can.

09

Choosing well

A DECISION TABLE, FIVE HONEST GOTCHAS, AND A CLOSING ARGUMENT

Real systems compose these patterns rather than choosing one: a static front-end (01) calling an API (02) that queues heavy work (03), publishes events (04) that start workflows (05), over an ingestion pipeline (06), with everything writing to the audit trail (07). The skill is knowing which shape each requirement maps to:

When the requirement sounds like…Reach for
“Everyone sees the same content, fast, everywhere.”01 · S3 + CloudFront
“A user asks a question and waits for the answer.”02 · API GW → Lambda → DynamoDB
“The work takes a while, and we can't lose requests.”03 · SQS + worker
“When X happens, several things should follow — and the list will grow.”04 · EventBridge
“There's a defined process, with branches and an approver.”05 · Step Functions
“Files come in; questions get asked about them later.”06 · S3 → Lambda → Athena
“We need to prove who did what, when.”07 · Append-only + WORM archive

Five honest gotchas

THE COST MODEL INVERTS

Serverless is nearly free when idle and linear with use — which means a runaway loop or a chatty poller costs real money. Set billing alarms on day one, and know that CloudWatch log ingestion is a classic surprise line item.

EVERYTHING HAS A LIMIT

Lambda runs 15 minutes max; payloads, batch sizes, and account-level quotas all have numbers. Skim the quotas page for each service you adopt before the design review, not during the incident.

IDEMPOTENCY IS NOT OPTIONAL

Queues redeliver, events duplicate, retries retry. Any handler that isn't safe to run twice is a latent bug. Conditional writes and idempotency keys are the standing fix.

OBSERVABILITY IS A DESIGN INPUT

In a distributed system, “what happened to request 42?” spans five services. Correlation IDs propagated through every hop, structured logs, and traces are decided in the design, not bolted on after.

START BORING

Patterns 02 and 03 cover most internal tools. Reach for events and orchestration when the requirements demand them — every hop you add is latency, cost, and a new place for messages to get lost. The best architecture is the smallest one that meets the requirement.

A last thought. Diagrams like these are the easy part — any of them can be assembled in an afternoon. The durable work is in the requirements the boxes encode: what must never be lost, who must approve what, what you'd need to prove a year from now. Get those right, and the architecture mostly draws itself.