Language Internals
Part 1 of 6 · Python Language ProficiencyPython — Intermediate to Advanced Proficiency
Hub: intermediate–advanced Python proficiency — data model, asyncio vs threads vs processes, typing, packaging and profiling.
- 1Gist
- 2Maps
- 3Q&A
- 4Sandbox
Voice readout needs Web Speech Synthesis in this browser.
Question ladder
L1
What is a dunder, and when does Python call it?
Answer
A dunder is a special method such as __len__ or __enter__. The interpreter calls it for an operation. You rarely call it yourself.
L2
Are property and the descriptor protocol the same machinery?
Answer
Yes. property is a data descriptor. __get__, __set__, and __delete__ on a class attribute are the protocol. function and classmethod are descriptors too.
L3
What is the cancellation rule for an asyncio Task?
Answer
cancel injects CancelledError at the next await. Clean up, then re-raise. A bare except or except BaseException that swallows it leaves the task half-dead. except Exception does not catch it.
L4
When do threads beat asyncio, and when do processes win?
Answer
Threads fit sync libraries whose I/O releases the GIL. asyncio fits many sockets. Processes win for CPU-bound pure Python because each process has its own interpreter.
L5
Protocol, TypedDict, or ABC at an API boundary?
Answer
Protocol is structural. TypedDict describes dict keys. ABC is nominal and wants inheritance or registration. Prefer Protocol when you do not own the implementer.
L6
Why does from x import y freeze a binding?
Answer
The import binds the object that y referred to at that moment. A later rebinding of x.y does not update the local name.
L7
How do you prove a hot path before a C or Rust rewrite?
Answer
cProfile or py-spy first. Confirm the function, then the cause (loop, allocation, or blocking call). Rewrite only the measured hotspot.
Failure modes
Blocked event loop
time.sleep, sync HTTP, or a CPU loop inside async def stalls every other task.
Shared mutable state
Threads mutate a list or dict without a lock or a queue, and the GIL does not make that atomic.
Swallowed cancellation
A bare except or except BaseException hides CancelledError, so the task continues after the caller gave up. except Exception does not catch it.
Import-time traps
Circular imports, sys.path hacks, and heavy side effects at import dominate cold start.
Types that only silence the checker
Any and type-ignore comments ship a lie. Untrusted JSON still needs a runtime validator.
Misconceptions
The GIL means Python cannot do concurrency.
I/O concurrency is real. Threads progress when the GIL is released. CPU-bound pure Python is the case that needs processes or native code.
asyncio makes CPU work faster.
It overlaps waits on one thread. A tight Python loop still holds the GIL.
async def runs the body in parallel.
Calling an async function returns a coroutine. Nothing runs until the loop schedules it.
Type hints are enforced at runtime by default.
Annotations are data for checkers. Parsing untrusted input needs pydantic, msgspec, or an equivalent.
Interviewer traps
Answering a GIL question with a distributed lock diagram.
Stay on one process. Name who holds the interpreter lock, then which tool you pick.
Saying await blocks the OS thread.
await yields the coroutine. The thread keeps running other ready tasks.
Treating slots as a default for every class.
Slots pay off for many tiny objects. They complicate inheritance and dynamic attributes.
Design scenario
Same prompt for every reader.
Requirements
One event loop for the sockets, a process pool for the pure-Python CPU work, typed DTOs, a lockfile, and cancellation that propagates.
Traffic / scale
A few hundred concurrent connections, each awaiting several I/O calls and one CPU job.
Latency
Socket waits overlap. The CPU job must not sit on the loop thread.
Consistency
A cancelled request does not publish a partial result.
Availability
The process stays up. Orphaned tasks and a wedged loop are the outage.
Failure assumptions
- A sync HTTP client is called directly from a coroutine.
- The CPU transform is pure Python and holds the GIL.
- CancelledError is caught by a bare except and ignored.
Constraints
- Stay on the language runtime. Do not redesign a queue cluster.
- Name the loop, the process pool, and the cancellation rule.
Prompt
An HTTP service fans out about 200 downstream calls, runs a CPU-bound feature transform, and must drop in-flight work when the client disconnects.
What this cluster trains
Prefer
CPython language proficiency
Object model, the GIL, cooperative scheduling, gradual types, and the install path. You can explain a stall and a wheel without a framework diagram.
- await yields. It does not spawn a thread.
- The GIL is per process, not an application lock.
- Annotations help the checker. Edges still validate.
Alternative
Syntax and framework surface
List comprehensions and a web framework API. Useful later. They sit on the rules in this cluster.
- A blocked loop is still a blocked loop inside a route.
- A race on a dict is still a race under the GIL.
- A green type-check can still parse bad JSON.
Read the cluster in this order
Each page is one mechanism. The hub is only the map.
- 1
Data model
Dunder methods, descriptors, and slots. - 2
asyncio
The loop, Tasks, and cancellation. - 3
GIL
Threads, processes, and when each wins. - 4
Typing
Protocols, generics, TypedDict, and runtime checks. - 5
Packaging
Imports, lockfiles, wheels, and profiles.
Overview
Senior Python interviews rarely ask for syntax trivia. They ask whether you can reason about the data model, pick a concurrency tool, keep types honest at the boundary, and ship an environment you can reproduce.
The sibling pages go deep. This page is the menu and the decision.
The JS/TS language hub and the JavaScript event loop are the parallel intuition for a single-threaded scheduler. They are not a substitute for the GIL, descriptors, or packaging.
Python and the JS scheduler
| Concern | Python (CPython) | JS/TS |
|---|---|---|
| Unit of async | async def coroutine plus one loop | Promise plus microtask and macrotask queues |
| CPU parallelism | Processes, or native code that releases the GIL | Worker threads or separate processes |
| Object layout | __dict__, __slots__, type object | Hidden classes and shapes |
| Static types | Optional typing plus mypy or pyright | TypeScript erased at emit |
| Packaging | Wheels, pyproject.toml, venv or uv | Lockfiles and node_modules |
Use those two JS pages for queue intuition. Keep the Python pages for who holds the interpreter lock.
Which concurrency tool
Decisions
- ?
Workload shape?
- many waitsasyncio
- sync librariesthreading
- 2
asyncio
- nextNeed cancel?
- 3
threading
- nextPure Python CPU?
- ?
Need cancel?
- yesTasks plus timeouts
- nogather or TaskGroup
- 5
Tasks plus timeouts
- 6
gather or TaskGroup
- ?
Pure Python CPU?
- yesProcess pool
- noThreads if C releases GIL
- 8
Process pool
- 9
Threads if C releases GIL
Lesson map
Python — Intermediate to Advanced Proficiency
Hub: intermediate–advanced Python proficiency — data model, asyncio vs threads vs processes, typing, packaging and profiling.
Architecture. Architecture
Select a node to see why it exists, or an edge to see the protocol, direction, effect, and consequence.
Mermaid export
flowchart TB shape["Workload shape?"] async["asyncio"] threads["threading"] cancel["Need cancel?"] shape -->|many waits| async shape -->|sync libraries| threads async -->|asyncio to Need cancel?| cancel
Press Run. Snippets must be self-contained — no network, files, or native modules.
What the rest of the cluster adds
- Data model covers dunder families, descriptors, and slots.
- asyncio covers Tasks, timeouts, and cancellation.
- GIL covers threads versus processes versus the loop.
- Typing covers Protocols, generics, and runtime validation.
- Packaging covers imports, wheels, and profiles.
Interview Q&A
Does the GIL mean Python is single-threaded?
Answer
One thread executes Python bytecode at a time in a process. I/O and many C extensions release the GIL, so threads still help waiting. CPU-bound pure Python needs another interpreter or native code.
Why prefer TaskGroup over a bare gather?
Answer
On 3.11 and later, a TaskGroup is a nursery. If one child fails, siblings are cancelled. Lifetimes stay in one block, so you leak fewer orphaned tasks.
Protocol or ABC?
Answer
A Protocol is structural. The checker accepts any object with the right shape. An ABC is nominal. The class must inherit or register. Prefer a Protocol at a boundary you do not own.
What does async def actually start?
Answer
Nothing. The call returns a coroutine object. The loop runs it when you await it or schedule a Task.
Why can a type-clean service still accept bad JSON?
Answer
Hints are erased. The checker never saw the payload. Validate at the edge with a parser that rejects the shape.
When is multiprocessing slower?
Answer
Short tasks. Pickle and process startup dominate. Batch the work and measure before you split a 50 microsecond function.
What is the first sentence of the answer?
Answer
Name the tool from the workload, who holds the GIL, and what you measured. Then stop.
Where do the JS pages fit?
Answer
Read the JS event-loop page for microtasks versus timers. Come back here for the GIL and for cooperative awaits that you write yourself.
Pitfalls
- Blocking the loop with
time.sleepor a sync HTTP client. - Sharing a dict across threads and calling the GIL a lock.
- Catching
CancelledErrorand continuing. - Installing into the system interpreter in CI.
- Adding
__slots__to every class because a blog said so. - Silencing the type checker instead of modeling the value.
Take a bug that says the API is slow under fan-out. Decide whether the stall is the loop, the GIL, or import time. Name the sibling page you would open, and the one measurement you would take before a rewrite.