Data engineering
Part 6 of 6 · Object StorageSecurity — IAM, Encryption, Public Buckets & Threat Model
The famous object-storage incidents are public buckets, over-broad IAM, leaked long-lived keys, and confused-deputy access. This lesson is the threat model: roles instead of static keys, block public access, encryption that does not excuse a public policy, and short-lived sharing.
- 1Gist
- 2Maps
- 3Q&A
- 4Sandbox
Voice readout needs Web Speech Synthesis in this browser.
Who holds the credential
Prefer
A role on the workload
The platform issues short-lived credentials to the job. The bucket policy allows that role, on one prefix, for the verbs you named.
- Nothing durable sits in a repo or a laptop shell history.
- You can scope the resource to a prefix.
- Rotation is the platform's session, not a ticket to replace a key.
Alternative
A static access key in config
It works until the key is committed, copied into a log, or left behind when someone leaves. The blast radius is every action the key allowed, for as long as you take to notice.
- The key does not expire unless you rotate it.
- s3-star on star-resources is how one leak becomes every bucket.
- A presign for a browser is a narrower tool than handing the key to the browser.
Overview
The incidents people remember are not broken ciphers. They are a bucket policy that allows * to GetObject, an IAM statement that allows every action on every bucket, a key in a .env that reached git, or a third party allowed to act on your bucket without a tight condition. This page is that threat model.
Presigned URLs and origin access control were the mechanism on the lifecycle page. Here they are part of the blast radius: a long TTL, a broad signature, or a public ACL undoes the rest of the design.
Top risks
| Risk | What it looks like | What you do |
|---|---|---|
| Public ACL or policy | GetObject or ListBucket for everyone | Block Public Access; private by default |
| Long-lived keys | A key in a shell config or a repo | Roles, SSO, or workload identity |
| Over-broad role | Every S3 action on every resource | Least privilege and a prefix condition |
| Presign leak | The URL in a ticket or an access log | Short TTL, one key, do not log the query |
| Wipe or ransomware | Mass delete | Versioning, Object Lock, MFA Delete where required |
| Metadata credential theft | A workload tricked into revealing its role | Lock down the metadata service and egress |
| Customer key loss | A CMK scheduled for deletion | Key policy, dual control, and a restore story |
Confused deputy, in one line: you allow another service or account to call your bucket, and a third party gets that service to call it on their behalf. The fix is a condition on the caller (source account, source ARN, or organization), not a wider allow. This page does not walk exploit steps.
Flow
- 1
1. A scanner finds the name
- next2. LIST and GET
- 2
2. LIST and GET
- next3. Objects are copied out
- 3
3. Objects are copied out
- next4. Block public access
- 4
4. Block public access
Lesson map
Security — IAM, Encryption, Public Buckets & Threat Model
The famous object-storage incidents are public buckets, over-broad IAM, leaked long-lived keys, and confused-deputy access. This lesson is the threat model: roles instead of static keys, block public access, encryption that does not excuse a public policy, and short-lived sharing.
Architecture. Architecture
Select a node to see why it exists, or an edge to see the protocol, direction, effect, and consequence.
Mermaid export
flowchart TB scan["1. A scanner finds the name"] read["2. LIST and GET"] leave["3. Objects are copied out"] fix["4. Block public access"] scan -->|1. A scanner finds the name| read read -->|2. LIST and GET to 3. Objects are copied out| leave leave -->|3. Objects are copied out| fix
IAM surfaces
| Idea | Mechanism | Phrase to use |
|---|---|---|
| Identity | Role, workload identity, managed identity | Prefer roles over static keys |
| Resource policy | Bucket policy and conditions | Deny by default, allow a prefix |
| Temporary share | Presigned URL or SAS | Time-boxed and method-scoped |
| Network | Private endpoint or service endpoint | Shrink the public path |
| Org guardrail | Organization policy | Block public ACLs above the account |
Avoid legacy ACLs. Use IAM and the bucket policy. Turn on Block Public Access at the account and on the bucket: block public ACLs, ignore public ACLs, block public bucket policies, and restrict public buckets. Uniform bucket-level access on Cloud Storage is the same instinct: one IAM model, not per-object ACLs that drift.
A teaching policy shape, not a policy to paste into production:
{
"Effect": "Allow",
"Action": ["s3:GetObject", "s3:PutObject"],
"Resource": ["arn:aws:s3:::lake-prod/lake/acme/*"],
"Condition": {
"StringEquals": { "s3:x-amz-server-side-encryption": "aws:kms" }
}
}The resource is one prefix. The actions are the two verbs. The condition refuses an unencrypted PUT. There is no s3:* and no *. ListBucket, if you need it, is on the bucket ARN, not on the object ARN, and it should carry a prefix condition too. Say that split out loud.
Close the bucket before you tune ciphers
Identity and public access decide who receives bytes. Encryption decides which key wraps those bytes.
- 1
Block public access
Account and bucket. Then confirm a public ACL cannot stick. - 2
Role, prefix, verbs
The job gets PutObject and GetObject on one prefix. Humans use SSO, not keys. - 3
Short share
Presign or SAS for a browser. Minutes, one key, one method. Do not log it. - 4
Encrypt and keep the key
Default SSE or a CMK you can still use. A deleted CMK is lost data.
Encryption
Decisions
- 1
Object bytes
- nextKey ownership
- ?
Key ownership
- serviceSSE with a service key
- yoursWhere the key lives
- 3
SSE with a service key
- ?
Where the key lives
- KMSCMK plus an audit of decrypt
- clientEncrypt before PUT
- 5
CMK plus an audit of decrypt
- 6
Encrypt before PUT
Decisions
- 1
Any of those modes
- nextTLS on the wire
- 2
TLS on the wire
- nextIs GetObject public
- ?
Is GetObject public
- yesCaller receives bytes
- noCaller is refused
- 4
Caller receives bytes
- 5
Caller is refused
SSE with a service key (SSE-S3) is the default you can defend for a lake: the platform owns the key, and the API decrypts for a caller who is allowed to GetObject. SSE-KMS does the same with your CMK. SSE-C is different: the caller must send the key again on GET.
SSE with a customer-managed key adds a key policy and an audit of who decrypted. Deleting that key, or scheduling it for deletion without a backup plan, makes the ciphertext unreadable. That is an availability incident, not a security win.
Client-side encryption before PUT means the provider stores ciphertext it cannot read. You own rotation, loss, and the fact that a bug in your client leaks plaintext anyway. Use it when the threat model says the provider must not see plaintext. It is not the default.
TLS covers the network. It does not cover a public policy.
The caveat: encryption does not fix a public bucket. For SSE-S3 and SSE-KMS, a caller who is allowed to GetObject receives plaintext bytes, because the service decrypts for that call. If the world is allowed, the world receives the object. SSE-C and client-side encryption still require the key on the read. Do not blur those in the answer.
Public bucket checklist
- Block Public Access at the account and on every bucket.
- Prefer IAM and bucket policies. Disable ACLs (object ownership: bucket owner enforced).
- Alert on public-access findings. A scanner should not be the first to tell you.
- Static sites go through a CDN with origin access control, not through a public bucket. The lifecycle lesson has the fetch path.
Object Lock and MFA Delete are the compliance names from the data-model page. Here they are the control against a stolen role that tries to delete every version. A versioned bucket without Object Lock can still be wiped by a principal allowed to delete version ids. Say who is allowed to delete, not only who is allowed to put.
Decisions you can run
Press Run. Snippets must be self-contained — no network, files, or native modules.
Press Run. Snippets must be self-contained — no network, files, or native modules.
The Python guard is naive and still the right instinct: a signed query string is a credential, so it does not belong in logs. The TypeScript choice never returns a public read. Marketing traffic goes to a CDN with origin access control. A browser upload gets a presign. A server job uses its role.
Interview Q&A
The bucket is encrypted and the policy allows the world to read. Is it safe?
Answer
No, for SSE-S3 and SSE-KMS. Those modes decrypt for any caller allowed to GetObject, and a public allow means that caller is everyone. SSE-C and client-side encryption still need the key on the read. Say which mode you mean before you call the bucket safe.
Static access keys versus a workload role?
Answer
A role gets short-lived credentials from the platform identity. A static key works until it leaks, and then it works for the attacker until you revoke it. Prefer the role. Humans use SSO the same way.
How do you share one private object for 10 minutes?
Answer
Authenticate the person in your app. Mint a presigned GET for that exact key with a 10-minute expiry. Do not change the bucket policy. Do not log the URL.
What does Block Public Access actually block?
Answer
Four switches, at account and bucket scope: new public ACLs, existing public ACLs, new public bucket policies, and public or cross-account access that a public policy would have allowed. Turn them on, then keep the bucket policy private anyway.
Why is s3-star on star a wrong allow?
Answer
The leaked credential, or the confused job, can read and write every bucket in the account, including ones this workload should never see. Scope actions and a prefix. Add a condition when a third party is involved.
What is the confused deputy here?
Answer
Another account or service is allowed to use your bucket, and someone outside convinces that service to act for them. Constrain the allow with the source account or source ARN. Do not answer by granting more.
Does TLS replace encryption at rest?
Answer
No. TLS protects the hop. At rest protects the stored object if the disk or a backup is handled by someone who is not allowed to call GetObject. You want both. Neither replaces IAM.
What happens if the customer-managed key is deleted?
Answer
SSE-KMS objects encrypted with that key become unreadable. Treat key deletion as data loss. Separate the people who can schedule deletion from the people who write data, and have a story before you rotate.
Versioning versus Object Lock?
Answer
Versioning keeps generations so an ordinary delete becomes a marker. A principal who can delete version ids can still destroy them. Object Lock in compliance mode refuses that delete until the retain-until date. Name both if the threat is ransomware.
Where should a static website live?
Answer
Behind a CDN with origin access control and a private bucket. A public bucket is not a website host you should default to. If a product truly needs anonymous GetObject, it is an explicit exception with no list permission, and it is still the security review, not a shortcut.
Pitfalls
Write six lines: public policy, static key, broad role, leaked presign, mass delete, lost CMK. For each, one mitigation you can say in a sentence. Circle the two that still cause breaches when the objects are encrypted with a service key.