Security
Part 6 of 7 · OAuth & OIDCService-to-Service Auth — mTLS, Client Credentials & Workload Identity
Service-to-service auth uses Client Credentials, mTLS/SPIFFE, or cloud Workload Identity federation instead of user OAuth dances. Narrow audiences, short-lived creds, and no long-lived JSON keys. Interviews probe blast radius, metadata SSRF, and token exchange.
- 1Gist
- 2Maps
- 3Q&A
- 4Sandbox
Voice readout needs Web Speech Synthesis in this browser.
Short-lived service identity vs a shared env-var key
Prefer
Client credentials, mTLS/SPIFFE, or federated workload identity
The caller proves who it is with a short-lived secret the platform can rotate. Audiences stay narrow. Stolen creds expire.
- Per-service client_id or SPIFFE ID — not one god key.
- private_key_jwt or mTLS client auth beats a shared password.
- Cloud federation: no long-lived JSON keys on disk.
- Same iss/aud/exp checks as user JWTs; subject is the service.
Alternative
Static API keys in env / config
Easy until the crash dump, the staging/prod copy-paste, or the intern's blog post. Rotation is a ticket. Blast radius is the estate.
- Years-long bearer knowledge with no audience.
- Logs and support bundles become credential stores.
- Mesh mTLS on + app authz off still lets any meshed pod call you.
- IMDS SSRF steals whatever the node identity can mint.
Client credentials happy path
Sequence diagram below. Workload identity is the same idea with a platform JWT instead of a password.
- 1
Service authenticates to /token
grant_type=client_credentials, plus secret, private_key_jwt, or mTLS. Ask for a narrow resource/audience. - 2
AS mints a short access token
Often a JWT with sub/client_id and limited aud. - 3
Call the resource server
Authorization: Bearer. RS verifies sig, iss, exp, aud, scopes. - 4
Wrong audience dies
A payments token must not pass ledger. That is confused-deputy control, not style.
Overview
User-less APIs need service identity, not end-user OAuth dances. Three production pillars:
- OAuth Client Credentials — the AS issues access tokens to confidential clients.
- mTLS — mutual TLS binds both sides of the hop (often SPIFFE X.509 SVIDs).
- Workload Identity — cloud platforms mint short-lived credentials for pods/VMs (IRSA, GKE Workload Identity, Azure Federated Identity).
Senior interviews compare blast radius, key distribution, and how these compose with JWT aud and mesh sidecars.
Comparative
| Mechanism | Proves | Token/key lifetime | Key distribution | Best fit |
|---|---|---|---|---|
| Client Credentials | Client id+secret/assertion | Short AT | Secret/KMS | Simple S2S via AS |
| mTLS (service certs) | Both TCP endpoints | Cert days–hours | PKI / SPIFFE | Mesh / zero-trust links |
| Workload Identity | Cloud SA ↔ IdP federation | Minutes | Platform OIDC | Cloud-native, no long secrets |
| Static API keys | Bearer knowledge | Often years | Config/env | Legacy only |
What fails if you choose wrong
- Long-lived shared secrets in env vars → leak via logs/crashdumps; no rotation.
- Client Credentials with broad
aud/scope→ one stolen secret owns many APIs. - mTLS without SPIFFE/trust domains → brittle cert sprawl.
- Trusting ambient cloud metadata without audience binding → SSRF steals tokens.
Client Credentials grant
- Confidential client authenticates to
/token(grant_type=client_credentials,scope/resource). - AS returns an access token (often JWT) with
sub/client_idand a limitedaud. - Client calls the RS with Bearer; RS validates (JWKS) — JWT study.
- Prefer private_key_jwt or mTLS client auth over shared passwords (RFC 7523 / 8705).
Sequence
- 1
Service A
Step1 client auth
- 2
Service A → Auth Server
POST /token client_credentials
- 3
Auth Server → Service A
access_token
- 4
Service A
Step2 call RS
- 5
Service A → Service B
API + Bearer
- 6
Service B
Step3 validate JWT aud+sig
- 7
Service B → Service A
200
Lesson map
Service-to-Service Auth — mTLS, Client Credentials & Workload Identity
Service-to-service auth uses Client Credentials, mTLS/SPIFFE, or cloud Workload Identity federation instead of user OAuth dances. Narrow audiences, short-lived creds, and no long-lived JSON keys. Interviews probe blast radius, metadata SSRF, and token exchange.
Architecture. Service A Ready. Auth Server Ready. Service B Ready
Select a node to see why it exists, or an edge to see the protocol, direction, effect, and consequence.
Mermaid export
flowchart TB s["Service A Ready"] as["Auth Server Ready"] r["Service B Ready"] s -->|POST /token client_credentials| as as -->|access_token| s s -->|API + Bearer| r r -->|200| s
Validation is a note on R, not a labeled R->>R self-loop sitting on the 200 arrow.
mTLS and SPIFFE
- Each workload gets an identity document (X.509 SVID) from a SPIRE / mesh CA.
- L7 policy: allow
spiffe://prod/payments→spiffe://prod/ledger. - You can bind access tokens to the client cert (
cnfthumbprint) so a stolen bearer alone fails (RFC 8705).
Mesh mTLS encrypts the hop. It does not replace application authorization. “Any pod in the mesh may call /admin” is a common production miss.
SPIFFE and SPIRE
A SPIFFE ID names the workload, not a human: spiffe://prod.example/ns/payments/sa/charge-api. The SPIRE agent attests the workload (Kubernetes service account, cloud instance identity, and similar) and the SPIRE server issues a short-lived SVID.
- X.509-SVID — for mTLS. The peer checks the cert chain and the SPIFFE URI in the SAN.
- JWT-SVID — when the peer expects a bearer and cannot terminate mutual TLS the same way.
Meshes often speak SPIFFE-compatible IDs. SPIFFE is the portable standard; a single mesh CA is an implementation. Naked client credentials prove knowledge of a secret. An SVID is platform-attested, short-lived, and rotated for you. Hours is a typical TTL; minutes are fine when the control plane can take the load. A client secret in a Kubernetes Secret is readable by controllers and backups — prefer a projected identity or a short-lived cert. Service-account JSON keys in git are a critical finding: rotate, switch to federation, and scan history.
Flow
- 1
Workload
- nextSPIRE Agent attests
- 2
SPIRE Agent attests
- nextSPIRE Server issues SVID
- 3
SPIRE Server issues SVID
- nextAgent delivers SVID
- 4
Agent delivers SVID
- nextmTLS to peer with X.509-SVID
- 5
mTLS to peer with X.509-SVID
Token exchange (RFC 8693)
Trade a token you already have for a new one with a tighter audience, fewer scopes, or a preserved actor.
- On-behalf-of: a user access token becomes a downstream token that still carries user context. Do not forward the user's refresh token.
- Audience restriction: a broad gateway token becomes a token whose
audis only payments. - Delegation versus impersonation: the
actclaim records who is acting.actor_tokenis the acting party on the exchange.
Client Credentials does not replace user authentication. A god-mode M2M token that impersonates users without an exchange policy is the confused deputy. Stolen bearers replay until exp; mTLS and certificate-bound tokens (cnf) make the bearer alone useless.
Sequence
- 1
API Gateway → Auth Server
token exchange subject_token audience payments
- 2
Auth Server → API Gateway
access_token aud payments
- 3
API Gateway → Payments API
Bearer exchanged token
Kubernetes and cloud workload identity
| Platform | Typical S2S identity |
|---|---|
| GKE | Workload Identity to a GCP service account, then an OIDC token |
| EKS | IRSA: pod service account to AWS STS |
| Azure AKS | Workload identity federation |
| Istio or Linkerd | mTLS identities, often SPIFFE |
Prefer the platform identity and a short-lived token over a mounted long-lived JSON key.
| Situation | Prefer |
|---|---|
| Two services in a mesh | mTLS or a SPIFFE X.509-SVID |
| Call a SaaS API as your service | Client Credentials with strong client auth |
| User request through a gateway to many services | Token exchange with shrinking audiences |
| Multi-cloud workload | SPIFFE or cloud federation OIDC |
| Legacy static API keys | Migrate. Treat them as debt. |
Keep OAuth M2M when there is no shared CA: cross-org APIs, SaaS partners, internet-facing OAuth. Inside the cluster, mTLS or SPIFFE is the stronger default.
Workload Identity (cloud pattern)
- Pod/VM has a platform identity (K8s SA / AWS role / GCP SA).
- Platform issues an OIDC JWT to a cloud STS / IdP.
- Exchange for cloud credentials or an OAuth AT (token exchange RFC 8693).
- No long-lived JSON keys on disk.
Flow
- 1
Step1 Workload SA
- nextStep2 platform OIDC JWT
- 2
Step2 platform OIDC JWT
- nextStep3 STS or AS exchange
- 3
Step3 STS or AS exchange
- nextStep4 short-lived creds
- 4
Step4 short-lived creds
- nextStep5 call cloud API
- 5
Step5 call cloud API
No one-node subgraphs — each step is a real node.
Sandbox: client credentials + audience (Python)
A payments client can call api://orders. The same token must fail api://ledger.
Press Run. Snippets must be self-contained — no network, files, or native modules.
Sandbox: workload token-exchange shape (TypeScript)
Illustrative RFC 8693 form. The sandbox does not fetch — it shows the grant shape you would POST.
Press Run. Snippets must be self-contained — no network, files, or native modules.
Pitfalls
Payments and ledger share a client secret today. Draw the stolen-secret path. Then split clients, bind aud, add SPIFFE allowlists, and show what SSRF against the pod IMDS can still mint.
Interview Q&A
Client Credentials vs Auth Code?
Answer
No user. The client is the subject. Use it for S2S. Auth Code + PKCE is how a human delegates access.
Why mTLS in a mesh?
Answer
Cryptographic peer identity plus encryption without every app implementing TLS plumbing. Pair with L7 allowlists on SPIFFE IDs.
What is workload identity federation?
Answer
A platform-issued OIDC proof exchanged for cloud/API credentials — so you are not shipping static JSON keys. IRSA, GKE Workload Identity, Azure federated credentials are the named products.
How do you limit blast radius?
Answer
Per-service clients, narrow scopes/audiences, short TTL, rotation, separate prod/staging issuers, sender-constrained tokens.
JWT access token for S2S — validate what?
Answer
Sig, iss, exp, aud, client/sub, scopes — same as user tokens, but the subject is a service. See JWT vs opaque.
SPIFFE ID vs OAuth client_id?
Answer
SPIFFE is transport / workload identity. OAuth client_id is an AS registration. Production systems often map between them (cnf / URI SAN).
Token exchange use case?
Answer
Turn a cloud identity into an AS access token for first-party APIs without embedding secrets. RFC 8693.
Why do static API keys fail interviews?
Answer
No audience, years of lifetime, painful rotation, and they leak through the same channels as config. Say “legacy only” and name the replacement.
JWT-SVID or X.509-SVID?
Answer
X.509-SVID when both sides speak mTLS. JWT-SVID when the peer wants a bearer and will not terminate mutual TLS. Either way the TTL is short and rotation is automatic.
What is the act claim on token exchange?
Answer
It records the acting party in a delegation or impersonation. actor_token on the RFC 8693 request is that party. Shrinking aud on the result is how you stop a gateway token from being replayed at every downstream API.
Why is a client secret in a Kubernetes Secret a weak S2S story?
Answer
Many controllers and backups can read it. Prefer projected workload identity, SPIRE, or a short-lived cert. A bearer copied from logs replays until expiry; mTLS does not.