BuiltForProd blog

The true cost of building your own landing zone

BuiltForProd — Thu, 24 Sep 2026 00:00:00 GMT

The true cost of building your own landing zone is not the quarter you put in the plan. It is the sum of eleven work streams, only three of which anyone estimates, plus an owner for as long as the company exists. Most teams can build a cloud foundation. The question worth arguing about is what else does not get built while they do.

This post is the honest version of that arithmetic: the work items, the engineer-months, the recurring maintenance, the opportunity cost, and the cases where building it yourself is clearly the right call. No prices, because the figure that decides this is your engineering capacity, not anyone’s invoice.

Why the estimate is always wrong

Landing zone estimates are made from the parts you can picture. Someone sketches the account structure on a whiteboard, someone else draws the VPCs, and the two of them agree it is about a quarter of work. They are right about those two things.

The estimate is wrong because a landing zone is not an architecture. It is a set of properties that have to hold across every account, forever, and properties are finished only when nobody can accidentally break them. That is a different kind of work from drawing boxes, and it is the kind that does not compress.

Three specific errors recur:

Confusing “deployed” with “done.” Terraform applies cleanly, therefore the network is finished. It is not finished until a second engineer can add a spoke without asking the author what the address plan means.
Pricing the happy path. The estimate covers creating accounts. It does not cover deleting one, moving one between organizational units, or recovering the one where somebody deleted the role the pipeline assumes.
Treating documentation as a phase. Documentation is not a phase; it is the difference between an asset and a liability with one maintainer.

What actually has to be built

Here is the full list, with what “done” honestly means for each. The engineer-month ranges are planning estimates for a competent team building from a blank repository — they are not measurements, and your numbers will differ. They are here to make the shape of the work visible, not to be quoted.

Work stream	Usually scoped	What “done” means	Engineer-months (estimate)
Account structure and organizational units	Yes	The layout survives the first team you did not plan for	0.5–1
Account factory	Rarely	Adding account fifteen is a pull request, not a project	1–2
Network topology and address plan	Yes	Routing, isolation domains and egress, with addresses assigned by a system rather than a spreadsheet	1.5–3
Identity model and permission sets	Partly	Every human role defined, production restricted, joiners and leavers handled in one place	1.5–3
Preventive guardrails	Rarely	Policies engineers cannot route around and do not want to	1–2
Centralized logging and the audit archive	Partly	Logs land in an account the logged systems cannot reach, and nobody can turn them off	1–2
Security services and their findings pipeline	Rarely	Findings reach a human who is expected to act, with severity that means something	1–2
CI trust and deploy identity	Rarely	The pipeline holds no cloud keys, and its role is scoped to a repository and a branch	0.5–1.5
Compliance control mapping and evidence	Rarely	An auditor’s question is a query, not a screenshot hunt	1–2
Documentation, runbooks and onboarding	Rarely	A new engineer ships an infrastructure change in week one	1–2
Day-2 procedures: drift, upgrades, incidents	Almost never	Drift is detected and triaged; provider and module upgrades have a reviewed path	1–2

Add the ranges and the honest answer is somewhere between eleven and twenty-two engineer-months for a foundation you would defend in a security review. Teams that finish nearer the bottom of that range have usually done it before. Teams that finish above it have usually discovered identity and compliance halfway through.

Figure 1

The three work streams almost every estimate includes, and the eight that decide whether the foundation is defensible. The second group is where schedules are lost, because each item is finished only when someone other than its author can operate it.

The line item everybody forgets: it never finishes

A landing zone is not delivered; it is adopted. From the day the first workload lands on it, someone owns:

Provider and module upgrades. A cloud provider ships breaking changes on its own schedule. Somebody reads the changelog, plans the upgrade, and explains to a reviewer why the diff is safe.
Drift triage. Something in production does not match the code. Reverting it blindly is how a discrepancy becomes an outage — a point we argue at length in drift is a fact, not a failure.
Growth. New team, new account. New market, new region. If the address plan was a spreadsheet, the second region is a redesign.
Evidence. Every enterprise deal now arrives with a security questionnaire. Answering it from the foundation’s own records is cheap; answering it from memory is a week.
The bus factor. The foundation’s author leaves eventually. What they leave behind is either a documented system or a folklore system.

Call that one to two engineer-months a year in steady state, more in a year with a major version bump or a new region. It is not a large number. It is a permanent number, and permanent numbers belong in the business case.

Opportunity cost is the number that decides it

Engineering capacity is the scarcest thing a scale-up has, and a landing zone consumes it at exactly the wrong moment: after the first enterprise prospect asks for a security review, before the product has won the segment.

Two senior engineers on a foundation for two quarters is roughly a year of senior product engineering that did not happen. Nothing about that year appears in a roadmap, which is why it never gets argued about properly. We made the general version of this argument in production readiness is a velocity multiplier: the tax is real whether or not you choose to see it, and the only choice is whether you pay it as a project or as recurring rework.

There is also a sequencing cost. A foundation built while under deadline pressure gets its expensive properties — identity, guardrails, evidence — deferred, and deferred properties are the ones that are hardest to retrofit. Adding an account boundary later means new identities, new network paths, new logging destinations and a migration window.

Build, buy or hire: the comparison that matters

A worked example, not a customer story: a 300-person B2B SaaS company running in two AWS accounts, with a first enterprise contract contingent on a security review in four months and no platform team. Here is how the three options actually compare for them.

	Build in-house	Buy a foundation	Hire a platform team
Time to first production workload on it	Two to four quarters	Weeks of engagement time, engagement-specific	One to two quarters before any code exists, then the build
Who owns the code afterward	You	You, if the delivery model hands over the repositories	You
Engineering capacity consumed	11–22 engineer-months, from your product team	Review and decision time, plus the handover	Recruiting, onboarding, then the same build
Risk concentration	High: usually one or two authors	Lower: the handover is the test of whether your team can run it	High at first: a single hire from a blank repository
Fit to strange requirements	Perfect, by construction	Good if the foundation is customized and source-available; poor if it is a black box	Perfect, eventually
Ongoing cost	Yours, permanently	Yours, permanently — optionally shared with a service	Yours, permanently, plus salary
What it is bad at	Deadlines, and documenting itself	Being the right shape when your constraints are genuinely unusual	Hitting a date that is one quarter away
Fails when	The author leaves before the documentation exists	Nobody on your team is made to operate it before handover	The role is open for five months

The columns are not mutually exclusive, and the strongest pattern for that worked example is usually the third combined with the second: a foundation in place, and the platform hire starting on top of it rather than under it. Their first quarter then goes to developer experience and product-facing platform work, which is what you actually hired them for.

When building it yourself is the right answer

This is a genuine list, not a rhetorical one. Build it yourself when:

The foundation is the product’s advantage. If you sell infrastructure, run at a scale where standard topologies stop applying, or your margin depends on a bespoke network design, nobody else’s layout will fit and you should not want it to.
You already have the team and the library. An existing platform team with a maintained internal module library, a working pipeline and a documented review culture is most of the way there. The marginal cost of a landing zone for them is far below the ranges above.
Your constraints are unusual. Data residency in an uncommon combination of regions, an air-gapped environment, a regulator who dictates topology, a contract that dictates key custody. A standard layout is a starting point that you will spend longer arguing with than replacing.
You are genuinely still early. A single account with good hygiene is the correct architecture for a five-person company. The mistake is not starting there; it is staying there past the date you knew you should have moved.
You want the learning, and you have the time to buy it. Building it teaches the team things reading never will. That is a legitimate reason, as long as it is a decision rather than an accident.

And the honest inverse: if your foundation looks like almost every other company’s foundation, building it from scratch is paying full price for a commodity. Your competitive advantage is your application, not your VPC design.

What “buying” should mean, and what it should not

The failure mode of buying is renting. If the foundation is a managed black box, you have swapped an engineering problem for a dependency, and you will discover the terms of that trade during an incident.

The version worth considering has three properties, and they are the ones to test any vendor against:

The accounts and the repositories are yours. The code lives in your version control, in a language your team can read, and you can change it without asking.
Nobody keeps standing access. After delivery, no external identity, credential or role should remain. Ask the question directly and ask what removes it.
The handover is a test, not a presentation. Your engineers should be required to perform real tasks unaided — make a change, review it, apply it, promote it, respond to drift — before anyone calls it delivered.

That is the shape of the BuiltForProd Baseline: a multi-account foundation deployed into your own accounts, customized, with the repositories and documentation handed over. The AWS Enterprise Baseline edition deploys 14 accounts across 2 organizational units and closes with an 84-item checklist, 17 verification checks mapped to SOC 2 controls run twice, and a hands-on handover in which your team performs 11 tasks unaided. The code is source-available under the PolyForm Internal Use License, so your team can read, modify, fork and extend it inside your organization.

The next step, whichever way you go

If you are about to commission a build, do this first: take the eleven work streams in the table above into a room with the two engineers who would do the work, and have them write “done means…” for each row in their own words. The rows they argue about longest are the rows that will overrun.

If you would rather get an outside read on what you already have before committing a quarter to it, that is what the BuiltForProd Assessment is for — a measurement against the BuiltForProd Standard, which is the same eight properties we describe in what production-ready actually means. Pricing is quoted per engagement and starts from the pricing page. And if we are not the right fit for you, we will say so.

SOC 2: what your infrastructure has to produce

BuiltForProd — Tue, 22 Sep 2026 00:00:00 GMT

A SOC 2 audit tests whether the controls you say you have actually operated over a period of time. It is, before it is anything else, an evidence problem. The auditor is not grading your architecture. They are asking a narrow question — show me that this control was working on these dates — several hundred times, and writing down whether you could.

That framing changes what infrastructure is for in an audit. The useful question is not “is our cloud secure enough to pass”. It is “what does our cloud produce, by itself, that answers an auditor’s question without anyone taking a screenshot?”

Readiness is not certification

This distinction is worth being pedantic about, because the market blurs it constantly.

Readiness means the controls exist, operate continuously, and generate records you can produce on demand. It is engineering work. It can be delivered, inherited or bought.
Certification — strictly, an attestation report — is a licensed auditor forming an opinion after testing those controls. It covers your organization, not your infrastructure: policies, onboarding, vendor management, background checks, incident response and the humans who run all of it.

No product makes you certified. A product can make the evidence trivial to produce, which removes most of the cost and nearly all of the panic. That is the honest claim, and it is the only one we make.

The two-week screenshot scramble, and why it happens

The familiar failure looks like this. Four weeks before fieldwork, someone opens a spreadsheet of control requests. Engineers start logging into consoles, screenshotting settings, exporting user lists, and pasting them into a shared drive. A week is lost. The screenshots prove the setting was true on the day it was taken, which is the one thing a Type II audit is not asking.

The cause is structural, not a discipline problem. If evidence is something you collect, collecting it will always be a project. If evidence is something the system emits, the audit becomes a query. The difference is whether the recording was switched on before the period started.

The evidence sources worth building first

Six sources answer most of what an auditor asks. Each is a capability, not a product, and every major cloud has an equivalent.

Evidence source	The questions it answers	Why an auditor accepts it
Immutable API audit log	Who did what, where, when, from which identity	Complete, timestamped, and outside the control of the person being tested
Configuration recorder and history	Was this resource configured correctly throughout the period	Shows state over time, not on the day of the screenshot
Continuous control checks	Which controls are passing right now, and which drifted	Turns a control into a measurement with a history
Identity and access records	Who had access, when it was granted, when it was revoked	Directly answers the largest control family in the framework
Threat detection findings	Were anomalies detected, triaged and resolved	Evidence of monitoring and of response, which is usually the weaker half
Version control and review history	What changed in the environment, who approved it, when it shipped	Change management evidence, produced as a by-product of doing the work

The last row is the one teams underestimate. If every infrastructure change is a reviewed pull request applied by a pipeline, your change management evidence already exists, in order, with approvals attached, for the entire period. No separate change log to maintain and no reconciliation between what the log says and what actually happened.

Make the log archive untouchable

One control deserves special attention because auditors probe it and incidents depend on it: the integrity of the audit log itself.

Logs that can be deleted by the same identity that operates the system are weak evidence, and worse, they are weak forensics. Three properties make the difference:

The archive lives in a separate account from the systems it records, with access limited to a small number of principals.
Stopping or deleting the logging configuration is denied at the organization level, above every account, so it is not a matter of nobody choosing to do it.
Integrity validation is enabled, so tampering is detectable rather than merely discouraged.

In the AWS Enterprise Baseline, that is how the audit account is set up: an organization-wide multi-region CloudTrail with log file validation and customer-managed encryption keys, delivered into a dedicated log archive account, with a service control policy that denies stopping or deleting CloudTrail and Config anywhere in the organization. Those specifics describe the AWS edition; the property is part of the BuiltForProd Standard in every edition.

Give auditors access — to the configuration, not the data

Auditors need to verify that controls are in place. That is a configuration question. It is not a reason for them to be able to read your customers’ records.

The pattern that works is a dedicated read-only role with an explicit denial on data-plane reads: they can see that a storage bucket is encrypted, that a database requires TLS, that access reviews happened. They cannot open the objects or query the rows. This is easier to grant, easier to defend to your own customers, and it makes the access review of your auditors uneventful.

What “ready” looks like in practice

Some numbers from the AWS edition, offered as a worked example rather than a target. The AWS Enterprise Baseline ships a SOC 2 baseline conformance pack enabled by default — thirteen automated configuration rules mapped to specific Trust Services criteria — with a further twenty-eight conformance pack templates behind a single flag each, covering frameworks including HIPAA, PCI DSS 4.0, NIST CSF, CMMC 2.0 and CIS Controls. Eleven frameworks have documented control mappings. The deployment itself finishes with seventeen verification checks mapped to SOC 2 controls, run twice, and the handover includes a quarterly evidence-snapshot procedure and a monthly compliance checklist.

Two honest caveats. Most of those packs ship off, because they cost money to run and the right set depends on which frameworks you are actually in scope for; each switch states its price next to it so the decision is informed rather than surprising. And a mapped control is not a passing control — the mapping tells you where to look, and the finding tells you whether you are there.

A sequence that works

If you are starting from a single cloud account and a deadline, this order produces evidence fastest:

Turn on the recorders. Audit logging, configuration history, threat detection. Do this first, today if possible: the clock on your observation window starts when they do, and nothing else you do this quarter matters as much.
Fix identity. Single sign-on, no long-lived credentials, defined roles, and a revocation path that actually revokes. This is the largest control family and the one where evidence is hardest to fake. We wrote about the credential half of it in zero long-lived cloud credentials.
Route every change through review. Infrastructure as code with pull requests and an approval gate on production. Your change management evidence then generates itself.
Separate the log archive and deny anyone the ability to stop it.
Map controls to the framework and let the continuous checks tell you what is failing, rather than discovering it during fieldwork.
Rehearse. Pull a quarter’s evidence as though the auditor asked. Whatever takes more than an hour is a gap.

Steps one through three cover a surprising share of what gets tested, and all three are things you want anyway.

The part infrastructure cannot do

Your report will also cover policies, employee onboarding and offboarding, security awareness training, vendor risk, business continuity and incident response. None of that lives in a cloud account. A compliance automation platform helps track it, and a good auditor will tell you where the gaps are early if you ask them early.

What infrastructure can do is make its own half boring — so that the week before fieldwork is spent on the human controls, which is the half that usually needs the attention.

If you want the infrastructure half handled as part of building the foundation rather than as a project afterward, that is what the BuiltForProd Baseline does, and SOC 2 readiness is the situation it is most often bought for.

Zero long-lived cloud credentials: OIDC in CI/CD

BuiltForProd — Tue, 15 Sep 2026 00:00:00 GMT

OIDC federation is a way for a CI/CD pipeline to obtain cloud credentials without storing any. Instead of holding a secret key, the pipeline receives a short-lived, signed token from its CI provider describing the job that is running — which repository, which workflow, which branch — and exchanges that token for a temporary cloud credential. The credential is issued for the run and expires with it. Nothing durable is kept anywhere.

The reason this matters is not that key storage is hard. It is that a long-lived cloud access key is a password with no expiry, living in a system whose entire purpose is to make things available to automation. It is copied into a local environment file by a developer who is debugging a pipeline. It ends up in a screenshot in a support ticket. It survives the contractor who created it. And unlike a password, nobody is ever prompted to change it.

What a stored key actually costs

The clean way to think about a static credential is to ask what an attacker gets from a single copy of it. The answer is: everything that credential can do, from anywhere on the internet, until somebody notices.

Property	Stored access key	Federated short-lived credential
Lifetime	Until revoked, which usually means until an incident	Minutes to an hour
Where it can be used from	Anywhere	Only from a job that can prove the right claims
What it is scoped to	Whatever the identity can do	The role the trust policy allows for that specific job
Rotation	A manual process someone owns and postpones	Not applicable; there is nothing to rotate
Blast radius of a leak	Full, silent, indefinite	Bounded by the remaining minutes of a session
Evidence trail	A key was used	A named job in a named repository assumed a named role

That last row is underrated. With federation, every action in the cloud traces back to a workflow run, which traces back to a commit and a reviewer. Attribution stops being a forensic exercise.

How the exchange works

There are four parties and one handshake.

Figure 1

The pipeline never holds a secret. It presents a signed claim about itself; the cloud verifies the signature against the provider's public keys, checks the claim against the role's trust policy, and issues a credential that expires with the job.

The job asks its CI provider for an identity token. The provider mints a JSON Web Token describing the run: the issuer, the audience, and a subject claim naming the repository and the branch, tag or environment.
The job presents that token to the cloud. No secret is involved; the token is the evidence.
The cloud verifies it. The signature is checked against the provider’s published keys, and the claims are checked against the trust policy attached to the role being requested.
A temporary credential comes back, scoped to that role and expiring shortly.

The design is unremarkable. The failure modes are where the interesting work is.

The trust policy is the whole control

Everything that makes federation safe lives in one document, and it is easy to write a version that is worse than the key it replaced.

Pin the subject claim to the smallest thing that is true. A condition that matches repo:your-org/*:* trusts every repository in the organization, including one a contractor created this morning, including one an attacker can open a pull request against. Pin it to the repository, and then to the branch, tag or deployment environment that is allowed to deploy.

Prefer environments to branches for production. A branch condition is satisfied by anything that can push to that branch. An environment condition can additionally require a human approval before the token is issued at all — which is the difference between a pipeline that can reach production and one that reaches production only when someone says so.

Check the audience. It is one line and it prevents a token minted for one relying party from being replayed at another.

Use separate roles per environment. One role that can deploy anywhere is a convenience that removes the benefit you just bought. A role per environment, each with only the permissions that environment needs, keeps the blast radius aligned with the account boundaries described in the multi-account landing zone.

Deleting the keys is the point

The step teams skip is the only one that changes the risk. Running federation alongside the old access keys means you have added a mechanism and removed nothing. The keys are still there, still valid, still copied into whatever they were copied into two years ago.

A migration that finishes looks like this:

Stand up federation and move one non-production pipeline to it.
Move the rest, environment by environment, production last.
Inventory every long-lived credential in the organization. Note who created each one and what it is used for. The list will contain surprises.
Disable them, watch for a defined window, then delete them.
Prevent new ones structurally.

Step five is what makes it stick. A policy at the organization level that denies the creation of long-lived credentials outright turns “we don’t use access keys” from a convention someone has to remember into a property of the environment. Conventions decay at the first deadline; structure does not.

What this looks like when it is finished

In the AWS Enterprise Baseline — the AWS edition of the BuiltForProd Baseline — there are no IAM users anywhere and no long-lived AWS access keys for anyone, human or machine. People sign in through IAM Identity Center against one of eleven permission sets, with multi-factor authentication required and sessions that expire — the human half of the same model, covered in zero trust access for engineers. Pipelines federate through GitHub OIDC into per-environment deployer roles in a dedicated automation account. A service control policy denies the creation of IAM users and access keys across the organization, which means the property is enforced above every account rather than agreed inside each one. Production is read-only for everyone except the platform and DevOps leads, and every apply to production passes a repository environment gate first.

Those specifics describe the AWS edition. The property they deliver — no standing credential exists for routine work — is part of the BuiltForProd Standard and applies to every edition.

A short checklist

No long-lived cloud access key exists for a human or for a pipeline.
Every trust policy pins issuer, audience and a subject claim narrowed to a repository and a branch, tag or environment.
Production deployment requires an approval that gates token issuance, not only the merge.
Roles are per environment and least privilege, not one role that can reach everything.
Creating a long-lived credential is denied by policy, not discouraged by documentation.
A break-glass path exists, is time-boxed, is alerted on, and has been exercised this year.

If more than two of those are open, the work is worth a sprint. If all six are open, it is worth a conversation — with us or with anyone, but soon.

The multi-account landing zone, explained

BuiltForProd — Tue, 08 Sep 2026 00:00:00 GMT

A landing zone is the account structure, identity model, network topology, logging pipeline and guardrails that everything else in a cloud organization is built on top of. It is the layer no workload owns and every workload inherits. Get it right and most security and compliance questions have boring answers. Get it wrong and every subsequent decision is made inside a constraint nobody chose.

The reason it matters more than it looks is simple: the account boundary is the strongest isolation a cloud provider gives you. A security group is a rule inside an environment. An IAM policy is a rule inside an account. An account boundary is enforced above both, by the provider, and it holds even when the thing inside it is compromised. It is also the boundary that billing, quotas, audit scope and blast radius all happen to follow.

Why account structure is a security control

Most teams arrive at the multi-account question from the wrong direction. They start with one account because one account is simple, add a second when production and development collide, and end up with a structure that records their history rather than their requirements.

Three properties make the account boundary different from anything inside it:

It is enforced by the provider. Cross-account access has to be granted explicitly. There is no default path, no shared network, no accidental permission inherited from a policy someone wrote in a hurry.
It is the natural audit scope. “Which systems handle regulated data” becomes a list of accounts instead of a survey of tags.
It fails safe. A misconfiguration inside an account is contained by the account. A misconfiguration in a single shared account is contained by nothing.

The trade-off is real: more accounts mean more automation is required, because manual management does not scale past about three. That is the actual decision. Not “how many accounts” but “are we willing to automate the boundary we need”.

A worked example: 14 accounts, 2 organizational units

Abstract advice about landing zones is easy to agree with and hard to use. So here is a concrete one. The AWS Enterprise Baseline — the AWS edition of the BuiltForProd Baseline — deploys one AWS Organization with 14 accounts across 2 organizational units. The numbers below describe that edition specifically.

Figure 1

The AWS Enterprise Baseline: an empty management account outside the organizational units, nine core accounts that provide shared capabilities, and four workload accounts generated from one template.

The management account is empty

core-root sits outside both organizational units and runs nothing. It exists to own the organization: the account tree, the service control policies, the tag policy. That is all.

This is the decision in the layout that pays back the most for the least effort. The management account is the one account that can change the rules everything else lives under. If a workload runs there, then compromising that workload means compromising the guardrails, and every other control in the organization is downstream of a thing an attacker now owns. Keeping it empty is not fastidiousness. It removes an entire class of incident.

Nine core accounts, each with one job

Account	What lives there	Why it is separate
`security`	Delegated administration for the security services, and the observability sink	The account that watches everything should not be an account that runs anything
`audit`	The log archive and the configuration aggregator	Evidence must survive a compromise of the thing it is evidence about
`identity`	The single sign-on directory and permission sets	One place to grant and, more importantly, revoke
`network`	The hub network, address management, private DNS	Network topology is shared infrastructure, not a workload concern
`dns`	Public DNS zones	Public name resolution has a different risk profile from private
`artifacts`	Container registries	Build outputs are shared across environments and outlive any one of them
`auto`	CI/CD federation, deployer roles, runners	The automation identity is a high-value target; it gets its own boundary
`corp`	Internal tooling	Employee-facing systems are not customer-facing systems
`public`	Assets that are deliberately public	Making “public” an explicit place stops it becoming an accident elsewhere

The pattern is the same in every row: a capability that belongs to the organization rather than to a workload gets its own boundary, so that granting access to it is a deliberate act.

Four workload accounts from one template

sandbox, dev, staging and prod are generated from a single definition. They differ in sizing, retention and who may act in them — not in structure. That is what makes “it works in staging” a meaningful sentence: staging and production are the same code with different inputs, so a difference between them is a bug rather than a fact of life.

Prod is also where the network stops being permissive. In this edition, the transit gateway carries deliberate blackhole routes between the production and non-production isolation domains: they cannot reach each other, by construction rather than by firewall rule.

The test: what does adding an account cost?

A landing zone is healthy when growth is cheap. The question to ask about any structure — yours or a vendor’s — is not how many accounts it has today. It is what happens when you need the fifteenth.

In the AWS Enterprise Baseline, a new account is an entry in a map and a folder in a repository: two pull requests, reviewed and applied by the same pipeline as everything else. The same property holds for regions. A region is a folder; the address plan, the region-restriction policy and the tag policy discover it automatically. The address plan itself is sized for 16 regions with 8 workload stages each, so expansion is a decision rather than a redesign.

When adding an account is a project, three things follow, and they follow every time: teams share accounts they should not share, environments diverge because nobody wants to touch the structure, and the eventual restructure lands during the quarter when something urgent is already happening.

What a landing zone does not solve

Being honest about the boundary of the idea:

It does not make your application secure. It makes the failure of your application survivable.
It does not produce compliance. It produces the evidence sources that make compliance work possible — a different and much smaller claim. We wrote about that distinction in SOC 2 on cloud infrastructure.
It does not remove the need for people who understand it. It reduces the number of things they have to hold in their heads, and it writes the rest down.

Where to start

If you are on a single account today, the useful first move is not a migration. It is a map: list every system, the data it touches, who may change it, and what would happen if it were compromised. The account boundaries usually draw themselves from that list, and the ones that do not are the ones worth arguing about.

If you would rather inherit a structure than design one, that is what the BuiltForProd Baseline is: a foundation deployed into your own accounts, with the repositories and documentation handed over, so the thing you run is yours rather than rented. If you want an outside read first, the BuiltForProd Assessment exists for that, and we will tell you if we are not the right fit.

Backups you haven't restored are assumptions

BuiltForProd — Tue, 01 Sep 2026 00:00:00 GMT

A restore test is an exercise in which a team recovers a real system from its backups, against a clock, and verifies that the recovered system works. Until that exercise has been run, a backup is a hypothesis: there is a job, the job reports success, and everything else about it is assumption. The backup industry has trained us to check the wrong thing. A green backup job proves a job ran. It does not prove the data is complete, that the restore path exists, that anyone has permission to run it, or that the restored system comes back in an order that works.

This post is about closing that gap: what recovery objectives actually promise, why untested restores fail, and how to run a drill that produces a number you can put in front of a board or an auditor.

What RTO and RPO actually promise

Two numbers describe recovery, and they point in opposite directions from the moment of failure.

Figure 1

RPO is measured backward from the failure and is paid in lost data. RTO is measured forward and is paid in downtime. Both are business decisions; only the delivery is an engineering problem.

RPO, the recovery point objective, is how much data you have agreed to lose. If your database is snapshotted every six hours, your RPO cannot be better than six hours no matter how fast the restore runs. RTO, the recovery time objective, is how long you have agreed to be unavailable, measured from the failure to verified working service — not to “the restore command finished”.

Both numbers are set by the business and delivered by engineering, and the most common failure in the conversation is that only one side ever says a number out loud. An RPO of fifteen minutes and an RTO of an hour are perfectly reasonable requests, and they imply a specific, costed set of mechanisms. Write both down for each system that matters, then check what you actually have.

Mechanism	Typical RPO	Typical RTO	What it does not protect against
Synchronous replica	Near zero	Minutes	Deletion, bad migration, ransomware — the replica copies them
Daily snapshot	Up to 24 hours	Hours	Anything that happened since the snapshot
Point-in-time recovery	Minutes	Hours	Loss of the storage account or the key
Object versioning	Near zero per object	Hours to days at scale	A bulk delete that also removes versions
Rebuild from code	Not applicable — infrastructure carries no data	Hours, if rehearsed	Application state, which still needs its own copy

The last row is the one teams forget. Recovering data and recovering the environment the data lives in are different problems with different mechanisms, and a real incident often needs both.

The four ways an untested restore fails

Across postmortems, failed restores cluster into four causes. None of them are exotic.

The backup was incomplete. The database was captured; the object store holding user uploads was not. The application comes back with a consistent database and broken references. This is the most common finding in a first drill, and the only reliable way to find it is to restore and then use the system.

The restore path was undocumented. Someone knew how to do it in the old architecture. The current architecture has different encryption, a different network path and a different bootstrap order. The knowledge did not survive the migration.

A dependency was missing. The restore needs a KMS key that lived in the account that was deleted, a DNS record that no longer exists, or a secret that was rotated after the backup was taken. Encrypted backups are only as recoverable as the key material, and key material has its own lifecycle.

Nobody had permission. In a well-governed environment, production is read-only for almost everyone. That is correct, until the one person with the restore role is on a plane. A break-glass path that has never been exercised is not a path.

Running the drill

A restore drill is not a tabletop exercise. Somebody types the commands.

Pick a real target and a real clock. Choose the system whose loss would stop the business. Announce a window, then start a timer that does not stop until someone other than the operator confirms the system works.
Restore into a separate, isolated environment. A sandbox account or subscription, not production and not staging-that-someone-depends-on. The point is to prove the path, not to create a second incident.
Restore everything the system needs, in order. Data store, object storage, secrets, configuration, DNS, and whatever else the application reads at startup. Write down the order you discovered, because next time that order is the runbook.
Verify with a business transaction, not a health check. A health endpoint returning 200 proves the process started. Place an order, run a report, log in as a real user. That is the definition of “restored”.
Record the number you actually achieved. Not the target. The achieved figure is the only honest input to the next planning conversation.
Fix one thing. Every drill surfaces more than one problem. Pick the one that most reduces the achieved RTO and fix it before the next drill, rather than filing all of them and doing none.

The first drill will be slow and slightly humiliating. That is the point: it is the cheapest possible version of the bad day. Teams routinely find that a promised four-hour RTO is a nine-hour RTO the first time, and a three-hour RTO the third time.

What the drill produces

The output of a drill is not a feeling of relief. It is three artifacts.

A runbook written during the exercise, by the person doing the work, in the order the work actually happened. Runbooks written in advance describe an imagined system; runbooks written during a drill describe the real one.
An achieved RTO and RPO for that system, dated. Two consecutive drills give you a trend, which is far more useful than a single number.
Evidence. Most control frameworks ask whether recovery has been tested and when. A dated record naming the system, the operator, the duration and the outcome answers that question directly. Compliance evidence produced as a by-product of engineering work is the only kind that does not require a scramble later — the same principle applies to SOC 2 evidence from infrastructure.

Rebuilding the environment is also recovery

Data restore gets the attention, but a large share of real incidents are not “we lost the data”. They are “we lost the environment”: an account was misconfigured beyond repair, a region became unusable, a change cascaded. In that case the question is not what your backup software can do. It is whether the environment exists anywhere other than in the cloud console.

Infrastructure defined entirely in code changes the shape of that answer. When every network, role, policy and cluster is a reviewed file in a repository, rebuilding is an apply, not an archaeology project. It is not free — you still need the data, the DNS, the certificates and the secrets — but it converts the most improvisational part of a recovery into something a second engineer can rehearse.

That is the property the BuiltForProd Standard calls recoverable: recovery is engineered and tested, not assumed. In the AWS Enterprise Baseline, the foundation itself is 100% infrastructure as code, the audit log archive lives in a separate account that the organization’s guardrails prevent anyone from stopping, and drift detection runs on a schedule so the code and the cloud do not quietly diverge between drills. None of that replaces a data backup strategy, and we are explicit with customers that it does not. It removes one large source of improvisation from the worst hour of the year.

A calendar you can actually keep

Ambitious recovery programs die in month two. A modest one survives:

Cadence	Exercise	Time cost
Quarterly	Full restore of the one system that would stop the business	Half a day
Quarterly	Break-glass access exercised by someone who does not normally use it	30 minutes
On change	Re-test after any change to backup configuration, storage location or keys	1 hour
Annually	Environment rebuild from code into a clean account	One day

Four entries. Put them in the calendar with names attached, not team names. An unowned drill does not happen.

Where to start this week

Pick one system. Ask three questions and insist on dated answers: When did we last restore this? How long did it take? Who did it? If any answer is “I don’t know” or “we haven’t”, you have found your first drill, and you have learned something important before it cost you anything.

If you want an outside read on where the gaps are, that is exactly what a BuiltForProd Assessment is for — and if your systems are already in better shape than you feared, we will tell you that too.