End-to-End Notification Analytics — Send, Delivery, Receipt & Engagement

Overview

Push without a funnel is flying blind. Seniors name clock skew, attribution, privacy, and tracing — not just "we use FCM analytics."

You should be able to:

1. Draw queued → engaged as a single-column join on nid.
2. Say what you cannot see (silent OS drops).
3. Point at SLIs for how you turn rates into error budgets — without copying that lesson.

What you can vs cannot see

OS policy, power, revoked permission — infer via cohort gaps.

Architecture (TB funnel, not an LR row)

Propagate notification_id / traceparent across queue → provider client → app open. Trace concepts: OpenTelemetry traces and the SLI cluster — do not recap sampling here.

Diagrams - step by step

Three small diagrams for notification analytics. Step numbers in the labels give the animation order. The lesson map under Diagram 1 plays those steps.

Diagram 1 - Happy path: the funnel joined on notification_id

Each stage is a separate event keyed by the same nid. Accepted means the provider took it, not that a person saw it. Delivered and displayed are often incomplete, so do not pretend full visibility.

flowchart TB
  A["Step 1 queued - intent persisted with nid Ready"]
  B["Step 2 accepted - provider HTTP success, store provider message id Ready"]
  C["Step 3 delivered - device ack when available Ready"]
  D["Step 4 displayed - SDK or OS hook Ready"]
  E["Step 5 opened - the tap carries nid Ready"]
  F["Step 6 engaged - in-app action within the attribution window Ready"]
  W["Step 7 Join on nid in the warehouse using server time Ready"]
  X["Failure path - inflated reach, silent drops hidden Ready"]
  A -->|continues| B
  B -->|continues| C
  C -->|continues| D
  D -->|continues| E
  E -->|continues| F
  F -->|continues| W
  B -->|count accepted as seen| X

Diagram 2 - Failure path: client clock skew drops opens

Client clocks drift and can be set by hand. Use the server receive time to order and join funnel events, and keep the client timestamp only as a diagnostic field.

Diagram 3 - Decision: which signal answers which question

Pick the metric that matches the question. Provider accept is a strong delivery-pipeline signal but says nothing about human attention; opens and engagement need first-party instrumentation.

Durable funnel events ride a bus whose delivery semantics live in Kafka.

Clock skew and attribution

• Use server receive time for funnel joins; carry client_event_ts separately.
• Skew-tolerant windows (e.g. open within 1h of send, configurable).
• Multi-device: attribute open to device_id but conversion to user_id carefully.
• Collapse/replace: later send supersedes; do not double-count opens across collapsed nids without rules.
• Tracing: same nid on outbound workers and the open beacon.

Privacy

• Payloads: opaque nid, type codes — never email/phone/message body secrets.
• Analytics events: hash user keys per policy; respect opt-out / deletion.
• Rich media URLs: short-lived; do not embed account tokens in query strings logged by CDNs.

Sandbox: funnel reducer (Python)

Rates are unique nids per stage over queued. Educational — not a warehouse.

Press Run. Snippets must be self-contained — no network, files, or native modules.

Same idea (TypeScript): open beacon

Playgrounds cannot fetch. An in-memory store stands in for POST /v1/push/events. Server stamps tsServer.

Press Run. Snippets must be self-contained — no network, files, or native modules.

Failure modes

• Analytics lag — warehouse delay misleads "delivery drop" pages; show pipeline lag SLO.
• Missing nid on legacy payloads — opens unattributed; enforce schema at compose.
• Counting provider accept as delivered — overstates success to execs.
• PII in data payload — compliance incident; redact in a compose linter.
• Double open events — debounce on nid+device; idempotent event upsert.

Pitfalls

Interview Q&A

Go Deeper

• FCM message lifecycle
• APNs response codes
• Understand FCM delivery
• OpenTelemetry traces
• Next: Reliability, Dedup & Idempotency