Skip to content
On this page 9

Tamper-resistant audit logging, honestly

A separate account, an organization-wide trail with file validation, and a policy that forbids stopping it — plus what tamper-resistant does not mean.

A tamper-resistant log archive is one where the systems that generate the records have no ability to change them afterwards. The property an auditor is buying is not that your logs are complete — everyone’s logs look complete — but that they could not quietly have been made to look complete. That is an architecture question about account boundaries and key policies, and it is answered before any log is written.

It is also a claim people overstate. “Immutable audit logs” appears on a lot of slides in front of a lot of designs where an administrator can delete the bucket. This post is about building the real thing, and about being precise regarding where the real thing stops.

The three properties, and what each one is worth

#
PropertyThe mechanismWhat it stopsWhat it does not stop
SeparationThe archive lives in its own account; source accounts can deliver, not read or deleteEvery path from a compromised workload account to the evidence about itA highly privileged principal inside the archive account
DetectionHourly digest files hashing each log and chaining to the previous digestA quiet edit or deletion; the chain breaks and verification failsThe deletion itself, which is detected rather than prevented
PreventionAn organization policy denying the calls that stop or delete the trailA local administrator switching logging off before actingSomeone with control of the organization management account

Read the right-hand column carefully, because that is the honest part. Each mechanism has a residual risk, and the residual risks are different, which is exactly why you want all three. What none of them does is make deletion physically impossible. That requires write-once storage, and it is a deliberate, separate decision with consequences of its own.

Separation is the control that does the most work

#

Put the archive in an account that exists only for that purpose. In a multi-account landing zone it is usually called audit, and it holds the log archive and the configuration aggregation and nothing else. No workloads, no build agents, no shared tooling.

The delivery path is deliberately one-way. The trail is configured by the organization, not by the accounts it covers, and CloudTrail writes into the archive bucket as a service principal. Members of the source accounts have no read permission, no delete permission and no route to the objects. The bucket policy denies anything the delivery does not require, and the encryption key is a customer-managed KMS key whose policy is at least as restrictive as the bucket policy — because a bucket policy alone is only half the door when the objects are encrypted.

This is the part that makes an incident survivable. If an attacker owns your production account entirely, they can do a great deal of damage, and the one thing they cannot do is edit the record of it. That property comes from the account boundary being enforced by the provider above everything in either account — the same reason the account boundary is the first real security control.

Figure 1
A one-way log delivery path into a separate archive account

Workload and core accounts emit trail events. A service control policy sits above every account and denies the API calls that stop, update or delete the trail. The organization trail delivers into a bucket in a dedicated audit account encrypted with a customer-managed key; the source accounts have no read or delete path back. Alongside the logs, hourly digest files each hash the logs of that hour and reference the previous digest, forming a chain that can be verified.

source accountscorestagingprodno read path · no delete pathservice control policy: deny stop · update · delete traildeliveraudit accountlog archive bucketcustomer-managed KMS keydigest, hour nhashes of this hour’s logsreferencesdigest, hour n−1lifecycle: standard → infrequent → archive
Delivery is one-way. Every account writes through the trail into an archive account they cannot read or delete from; a service control policy applied above all of them denies the calls that would stop the trail; digest files chain each hour to the previous one.

What log file validation actually proves

#

Enable log file validation on the trail and CloudTrail starts writing digest files, one per hour, into their own prefix. Each digest contains the SHA-256 hash of every log file delivered in that hour, and a reference to the previous hour’s digest. The digest is signed with a private key whose public counterpart AWS publishes, so anyone holding the files can verify the signature without trusting you.

That structure produces two useful guarantees. If a log file is altered, its hash no longer matches the digest that recorded it. If a log file or an entire hour is removed, the chain of references has a hole in it. Both are verifiable with the standard tooling — aws cloudtrail validate-logs over a date range — which means the verification can be performed by an auditor, by an incident responder, or by you on a schedule.

Two caveats worth stating before someone else states them. Validation is detection, not prevention: it tells you the record was altered, it does not stop the alteration. And it only proves anything if someone runs it. A validation that has never been executed is a feature, not a control. Run it quarterly at minimum, and run it during any incident before you start drawing conclusions from the logs.

The policy that keeps the trail running

#

The weakest point in most logging designs is not the storage. It is that logging is a setting in an account, and account administrators can change settings in their account. An attacker with administrative access does not need to edit the logs if they can stop them being written.

A service control policy closes that. It is evaluated above the account, so it constrains every principal in that account — including administrators, including the root user — and denies the calls outright: stopping logging, updating the trail, deleting the trail, stopping the configuration recorder, deleting the delivery channel. The calls fail with an explicit denial that no local permission can override. Only the organization management account can change the policy, and the management account should be empty, with access to it exceptional and alarmed.

This is the step that converts a claim into a property. “Logging is enabled in all accounts” is a statement about the current state, and current states drift. “Logging cannot be disabled in any account” is a statement about the rules, and an auditor can test it in ten seconds by trying it.

Retention, lifecycle and the cost conversation

#

Retention is where compliance and finance meet, and it goes better when the two are discussed in that order rather than the reverse.

Start with the obligation. A Type II report tests controls over an observation window, so the logs must cover the whole window plus the audit itself. Contracts may require more. Incident response usually wants at least a year, because the gap between a breach and its discovery is routinely measured in months. Pick the period from those constraints, and only then make it cheap.

Making it cheap is a lifecycle policy. Audit logs have a distinctive access pattern: heavily queried for days, occasionally queried for months, and then almost never read but required to exist. Storage classes map onto that almost exactly — standard storage for the recent window, infrequent access for the middle, archival storage for the tail. The archived tail costs a fraction of standard storage and takes minutes to hours to retrieve, which is acceptable for a request from an auditor and not acceptable for an incident, so the boundary between the tiers should sit outside your realistic incident window.

The AWS Enterprise Baseline’s default is a 365-day retention with lifecycle transitions to infrequent access and then archival storage. That is a starting point rather than a rule: an organization with a longer contractual obligation should set the longer period deliberately and price it, rather than discovering the limit during an audit.

What we deliberately do not claim

#

This is the section that usually gets left out, so it gets its own heading.

There is no S3 Object Lock in what ships, and no write-once retention. That means the honest description of the archive is: the accounts that produce the records cannot touch them, disabling the trail is denied organization-wide, and any alteration or gap is detectable through a signed chain — but a sufficiently privileged principal inside the archive account could still delete objects, and you would find out rather than be prevented.

If your obligations require true write-once storage, add Object Lock, and go in with open eyes. In compliance mode, objects cannot be deleted or overwritten by anyone, including the root user, until the retention period expires. That is precisely the point, and it is also the risk: a misconfigured retention period is not correctable, and you are committing to pay for that storage for the full term. It is the right choice for some organizations and an expensive mistake for others, which is why it should be a decision rather than a default.

Two other things worth being equally direct about, because they appear on competitor feature lists and are not part of this design: there is no cross-region replication of the archive, and no separate backup product is layered on top of the bucket’s own durability. If your threat model includes losing an entire region’s copy of the evidence, that is a gap to close explicitly, and closing it is a small piece of work — far smaller than discovering the gap during an audit.

What an auditor will actually ask

#

Four questions, and they are the same four every time:

  1. Is logging enabled everywhere, including accounts created during the period? Answer with the organization trail, and with the fact that new accounts are covered automatically.
  2. Could someone have turned it off? Answer with the service control policy, and let them test it.
  3. Could someone have edited what was collected? Answer with the separate account, the key policy and the validated digest chain — and show them a validation run.
  4. Is it still there? Answer with the retention period and the lifecycle configuration.

Notice that all four are configuration questions, not screenshot questions. That is the difference between an audit that is a query and an audit that is a two-week screenshot scramble.

Where this sits in the foundation

#

In the AWS Enterprise Baseline — the AWS edition of the BuiltForProd Baseline — the audit account holds the log archive and the AWS Config aggregator and nothing else. The organization trail is multi-region, encrypted with a customer-managed key that rotates annually, and has log file validation on. One of the four service control policies exists specifically to protect the audit path: no account can stop or delete the trail or the configuration recorder. Retention is 365 days with lifecycle transitions, and the detection services that consume the same telemetry are administered separately, from the security account — which is the subject of nine security services, one account.

Those specifics describe the AWS edition. The property they deliver — evidence stored where the systems it describes cannot reach it — belongs to the BuiltForProd Standard, and readiness is the claim we make about it. Certification remains your auditor’s decision.

Where to start this week

#
  • Find out which account your logs are in. If the answer is “the same account as the workloads”, that is the first thing to change.
  • Check whether log file validation is enabled on your trail. If it is not, enabling it is a single setting and it starts working immediately — for future logs only, which is the argument for doing it today.
  • Run a validation over last quarter. If the tooling reports gaps, you have learned something important about your archive before an auditor does.
  • Try to stop the trail from a member account with an administrator role. If it succeeds, your logging is a setting rather than a control.

If those four take longer than a day because nobody is certain where the logs go, that uncertainty is the real finding. The BuiltForProd Assessment measures an existing estate against the Standard, and the compliance page sets out which evidence sources a foundation should generate on its own.

Questions

What makes an audit log tamper-resistant?

Three things working together: separation, so the accounts producing the records cannot modify the archive; detection, so any alteration or gap is provable through a signed hash chain; and prevention, so no member account administrator can stop the logging in the first place. Each is weak alone and strong in combination.

How does CloudTrail log file validation work?

CloudTrail writes a digest file every hour containing the SHA-256 hash of each log file delivered in that hour and a reference to the previous digest file, signed with a private key whose public half AWS publishes. Validating the chain proves that the logs you hold are the logs that were written, and that no hour is missing.

Can a service control policy really stop an administrator disabling logging?

Yes, within the organization. A service control policy is evaluated above the account, so it constrains every principal in that account including its administrators and its root user. A policy denying the calls that stop, update or delete the trail means those calls fail regardless of what the local IAM policy allows. Only the organization management account can change it.

Is a log archive without Object Lock good enough?

It depends on what you must demonstrate. For most SOC 2 style evidence, separation plus a validated digest chain plus a policy that prevents disabling the trail is a defensible position, provided you say plainly that deletion by a highly privileged principal in the archive account is detectable rather than impossible. If a regulator or contract requires write-once storage, add S3 Object Lock and accept that objects then cannot be removed by anyone, including you, until the retention period expires.

How long should cloud audit logs be retained?

Long enough to cover your audit observation window and any contractual commitment, which for a Type II report means the whole period plus the time it takes to complete the audit. A year is a common baseline. Decide the period from the obligation, then use storage lifecycle transitions to make it cheap rather than shortening the period to make it cheap.

This is what we build.

BuiltForProd deploys a production foundation into your own cloud accounts, hands over the repositories and the documentation, and, if you want it, keeps running it with your team. Tell us what you are running and what is coming next. The first call is 30 minutes, with an engineer who scopes engagements, not a salesperson.

The calendar invite and the video link go to this address. Prefer to write? Send a message instead.

Nine security services, one account

What GuardDuty, Security Hub, Config, CloudTrail and their siblings each actually catch, what they cost, and the order to switch them on.