Generators, yield & yield* — Composing Streams and Decision Workflows
Two composition styles collide in AI backends: sequential token streams via generators/yield, and one-shot parallel decision round-trips (System One). Interviews expect you to implement async generators for LLM chunks and to compose typed decision steps without confusing them with token streaming.
- 1Gist
- 2Maps
- 3Q&A
- 4Sandbox
Voice readout needs Web Speech Synthesis in this browser.
Compose an explain-policy path after an intent Choice
Prefer
Await the decision, then stream only if needed
One System One round-trip (or a short effect chain when dependency exists). If intent is explain and confidence is high, yield tokens. Otherwise yield a routed record and stop.
- Generators still help: progress events around the decision, or token yields after the gate.
- Backpressure: consumers pull; do not unbounded-buffer model output.
- ReadableStream is the HTTP boundary; async generators are app-code ergonomics.
Alternative
Stream the classifier, or yield N System One calls
Token-drip on a closed decision adds complexity without UX. Sequential System One yields throw away parallel questions.
- No autoregressive tokens — the parallel sampler returns the bundle together.
- Effect.gen is optional sequential sugar, not a second model class.
- Fan-out belongs in one POST — a questions map, not a for-loop of calls.
Overview
Two composition styles collide in AI backends:
- Sequential token streams via generators /
yield. - One-shot parallel decision round-trips (System One).
Interviews expect you to implement async generators for LLM chunks and to compose typed decision steps without confusing them with token streaming. Teach Effect-style Effect.gen + yield* for decision workflows as an idea — do not require @compootor/effective-jev.
You should be able to:
- Write
async function*/async defthat yields chunks. - Put gates in code between a decision await and a token stream.
- Name where cancellation lives.
Generators for streaming
- Python:
def gen(): yield x— lazy iterators;async def agen(): yield xfor async. LLM SDKs often expose async iterators of tokens / events. - TypeScript:
async function* agen() { yield chunk }— pairs withReadableStreamfor HTTP SSE / chunked responses. - Backpressure: consumers pull; don’t unbounded-buffer model output.
- Cancellation: AbortSignal / aiohttp cancel /
breakout ofasync for.
System One is not a token stream
One POST returns all answers together. No TTFT token drip. Composing System One calls is workflow orchestration (sequential only when dependency requires), not streaming.
Generators still help: pipeline stages that yield intermediate decision records, or async generators that yield progress events around decision calls.
Effect.gen + yield* (pattern)
Effect (TypeScript) uses generator-looking syntax: Effect.gen(function* () { const x = yield* someEffect }). yield* sequences typed effects; errors and context propagate.
Analogous pattern for System One: each yield* is a client.systemOne(...) effect; compose confidence gates in the same generator. Teach the idea with a thin mock — package optional.
Architecture (stream vs decision compose)
Single-column fork: tokens vs typed round-trip. Hybrid dashed link is the next two lessons.
Decisions
- ?
1 Prose stream or decision?
- tokens2 LLM async generator
- typed answers2 Build state
- 2
2 LLM async generator
- next3 Progressive UI
- 3
3 Progressive UI
- 4
2 Build state
- next3 System One round-trip
- 5
3 System One round-trip
- next4 Gates in code
- 6
4 Gates in code
- dependent5 Second System One
- done5 Side effects
- 7
5 Second System One
- 8
5 Side effects
Lesson map
Generators, yield & yield* — Composing Streams and Decision Workflows
Two composition styles collide in AI backends: sequential token streams via generators/yield, and one-shot parallel decision round-trips (System One). Interviews expect you to implement async generators for LLM chunks and to compose typed decision steps without confusing them with token streaming.
Architecture. Architecture
Select a node to see why it exists, or an edge to see the protocol, direction, effect, and consequence.
Mermaid export
flowchart TB job["1 Prose stream or decision?"] l["2 LLM async generator"] s["2 Build state"] ui["3 Progressive UI"] job -->|tokens| l l -->|2 LLM async generator| ui job -->|typed answers| s
Sandbox: token async generator + decision stage (Python)
Step A is not token yields. Step B streams only after the gate.
Press Run. Snippets must be self-contained — no network, files, or native modules.
Async generator + Effect-like compose (TypeScript)
yield produces a value; yield* (Effect) runs a sub-effect. This mock uses yield of { run } so the sandbox stays package-free.
Press Run. Snippets must be self-contained — no network, files, or native modules.
Comparative
| Style | Pros | Cons |
|---|---|---|
| Generator stream | TTFT UX, cancellable, composable transforms | Complexity, backpressure bugs; wrong tool for closed decisions |
| System One round-trip | All answers at once; simple await | No progressive tokens; dependent questions need an explicit second call |
| Effect.gen / yield* | Readable sequential async with typed errors | Learning curve; optional dependency |
Pitfalls
Intent Choice: explain / refund / human. If explain and confidence ≥ 0.7, yield policy tokens. If refund, yield a typed action record. If low confidence, yield escalated. Where does AbortSignal sit on each path?
Interview Q&A
yield vs yield*?
Answer
yield produces a value to the consumer. yield* delegates to another iterable / effect (Effect: run the sub-effect). In async generators, yield chunk; composition of pipelines often uses for await or yield*.
Why not stream System One?
Answer
No autoregressive tokens — the parallel sampler returns structured answers together. There is nothing useful to paint after the first “token.”
Where do you put AbortSignal?
Answer
On fetch / SDK for LLM streams; on the System One HTTP client timeout / abort for the single round-trip. Depth: streaming vs parallel.
Generator for fan-out?
Answer
Prefer one multi-question request over yielding N sequential System One calls unless dependency forces sequencing. Speculative fan-out is a questions-map, not a for-loop of POSTs.
ReadableStream vs async generator?
Answer
Generators are ergonomic in app code; ReadableStream is the Web Streams boundary for HTTP responses (SSE / chunked). Convert at the edge of the BFF, not in every helper.
Can generators wrap decision workflows anyway?
Answer
Yes — yield progress records or gated side-effect stages. That is orchestration, not TTFT. The Effect.gen mock on this page is that pattern.
Python sync yield vs async yield?
Answer
Sync yield is a lazy iterator (the token-shape helper). async def + yield is an async generator you async for. LLM SDKs are usually the latter.
Backpressure in one sentence?
Answer
The consumer pulls. If you push faster than the UI can paint, bound the buffer and cancel — dropping tokens corrupts prose.
Is Effect required to use Jev?
Answer
No. Vendor how-to-build is HTTP + questions. Effect.gen is typed sequential sugar. Do not block an interview on @compootor/effective-jev.
What is the next lesson for?
Answer
TTFT, SSE vs the messaging cluster, when not to stream, and AbortSignal on the token path. Stay here until yield vs round-trip is crisp.