Language Internals
Part 2 of 6 · Python Language ProficiencyData Model — dunder methods, protocols, descriptors & slots
Dunder methods, protocols, descriptors, and slots in the CPython object model.
- 1Gist
- 2Maps
- 3Q&A
- 4Sandbox
Voice readout needs Web Speech Synthesis in this browser.
Question ladder
L1
How does len(x) find its answer?
Answer
It calls the type's __len__ slot with x. The return value must be a non-negative integer.
L2
Why can a property block assignment?
Answer
property is a data descriptor. If fset is missing, __set__ raises. The instance dictionary never stores that name.
L3
What is the difference between __getattr__ and __getattribute__?
Answer
__getattribute__ runs on every lookup. __getattr__ runs only after the normal path raises AttributeError. Overriding __getattribute__ recurses easily.
L4
Data descriptor or non-data descriptor?
Answer
A data descriptor defines __set__ or __delete__ and overrides the instance dictionary. A non-data descriptor is __get__ only, so an instance attribute hides it. Functions are non-data, which is why you can assign over a method on one instance.
L5
When does __init__ still run on a cache hit in __new__?
Answer
Whenever __new__ returns an instance of the class. Guard __init__ if a cached object must not reset its fields.
L6
What does __slots__ change?
Answer
Instances skip the per-object dictionary and refuse unexpected attributes, unless you also list __dict__ or __weakref__.
L7
Why set __hash__ to None after a custom __eq__ on a mutable object?
Answer
Equality and hashing must agree. A mutable object that compares by value should not be a dict key. Setting __hash__ to None makes that explicit.
Failure modes
Mutable default argument
def f(items=[]) stores one list on the function. Every caller shares it.
Hashable mutation
__eq__ by value with the default hash lets a mutated object vanish from a set.
Descriptor without __set_name__
A reusable descriptor guesses its storage key and collides across classes.
Slots and multiple inheritance
Bases with incompatible slots fail at class creation, often far from the bug you meant to fix.
Misconceptions
Dunder methods are just a naming convention you call yourself.
The interpreter looks them up on the type for operators, construction, and iteration. Calling them directly skips some checks.
property is a special keyword, not a descriptor.
property is a class that implements the descriptor protocol. ORM columns use the same idea.
Slots are always faster, so every class should have them.
They save memory on many small instances. They fight dynamic attributes and some inheritance patterns.
Interviewer traps
Explaining attribute lookup as a simple instance dictionary read.
Mention data descriptors first. That is why a property wins over a same-named instance key.
Using __del__ as a destructor you can rely on.
Finalization timing is not a resource API. Use a context manager.
Implementing __eq__ and leaving the object hashable.
Say whether the value is immutable. If it is mutable, set __hash__ to None.
Design scenario
Same prompt for every reader.
Requirements
Fixed fields, a validated hp range, and a repr that is useful in logs.
Traffic / scale
Millions of short-lived objects per tick, almost no I/O.
Latency
Attribute access stays a slot load, not a dictionary probe, on the hot path.
Consistency
hp outside 0 to 100 raises before it is stored.
Availability
Class creation fails fast if a subclass layout is incompatible.
Failure assumptions
- A mutable default list caches particles across ticks.
- Someone stores extra attributes for debugging and defeats the layout.
Constraints
- Use the data model, not a database.
- Name descriptor versus slots and why both can coexist.
Prompt
A simulation creates millions of particles with x, y, and hp. Authors keep adding ad-hoc attributes, and the process RAM climbs.
Overview
CPython objects behave through the data model. Operators, len, iteration, and with are lookups of special methods on the type. Mastering that lookup is the difference between using classes and owning a library.
Attribute lookup
Decisions
- 1
obj.attr
- nextData descriptor on type?
- ?
Data descriptor on type?
- yesdescriptor get or set
- noInstance dict hit?
- 3
descriptor get or set
- ?
Instance dict hit?
- yesreturn instance value
- noNon-data on type?
- 5
return instance value
- ?
Non-data on type?
- yestype lookup or get
- nogetattr if defined
- 7
type lookup or get
- 8
getattr if defined
Lesson map
Data Model — dunder methods, protocols, descriptors & slots
Dunder methods, protocols, descriptors, and slots in the CPython object model.
Architecture. Architecture
Select a node to see why it exists, or an edge to see the protocol, direction, effect, and consequence.
Mermaid export
flowchart TB start["obj.attr"] data["Data descriptor on type?"] desc["descriptor get or set"] inst["Instance dict hit?"] start -->|obj.attr to Data descriptor on type?| data data -->|yes| desc data -->|no| inst
__getattribute__ is this walk. __getattr__ runs only when the walk raises AttributeError.
Dunder families
| Family | Examples | Interview use |
|---|---|---|
| Construction | __new__, __init__, __del__ | Interning and immutables live in __new__ |
| Representation | __repr__, __str__, __format__ | Debug text versus user text |
| Comparison | __eq__, __hash__, __lt__ | Value objects that can be dict keys |
| Containers | __len__, __getitem__, __iter__, __contains__ | Sequence and mapping protocols |
| Callables | __call__ | Functors and some decorators |
| Context | __enter__, __exit__, async pair | Resource safety |
| Numeric | __add__, __radd__, __iadd__ | Operators, including the reflected form |
If it implements __iter__ and __next__, it is an iterator. A typing Protocol is how a checker states that without a base class. The typing page owns that half.
Descriptors
A descriptor is an object on the class with __get__, __set__, or __delete__. property, classmethod, staticmethod, and typical ORM columns are descriptors.
Press Run. Snippets must be self-contained — no network, files, or native modules.
Data versus non-data. If the descriptor defines __set__ or __delete__, it wins over the instance dictionary. A __get__-only descriptor loses to an instance attribute. That is why a function on the class can be shadowed by assigning obj.method = ..., and a property cannot.
__set_name__ (PEP 487) tells the descriptor the attribute name when the class body finishes. Reusable descriptors should use it instead of a hard-coded storage key.
Slots and dictionaries
__dict__ (default) | __slots__ | |
|---|---|---|
| Memory | One dict per instance | Fixed layout, smaller |
| Dynamic attributes | Yes | No, unless __dict__ is listed |
| Multiple inheritance | Ordinary | Every base needs a compatible layout |
| Weak references | Yes | Add __weakref__ to the slots |
Use slots for many tiny objects (graph nodes, particles). Do not put them on every class out of habit.
__new__ versus __init__
__new__ allocates or returns an existing instance. __init__ initializes, and it still runs when __new__ returns an instance of the class. Cached or interned objects need a guard so a second call does not reset fields.
Most application code only needs __init__. Reach for __new__ for immutables and interning.
Interview Q&A
How does len(x) work?
Answer
The interpreter calls the type slot for __len__. You get a non-negative int back, or a TypeError if the slot is missing.
Why can a property block assignment?
Answer
It is a data descriptor. A property without a setter implements __set__ by raising. The name never lands in the instance dictionary.
__getattr__ or __getattribute__?
Answer
__getattribute__ is the normal lookup. __getattr__ is the fallback after AttributeError. Override the fallback unless you are ready to call the base implementation on every access.
Why do instance attributes hide methods but not properties?
Answer
Functions are non-data descriptors. Properties are data descriptors. Data descriptors are checked before the instance dictionary.
What breaks if __eq__ changes and __hash__ stays inherited?
Answer
Equal objects can have different hashes, so sets and dicts misplace them. Mutable values should set __hash__ to None.
When do you include __dict__ in slots?
Answer
When a small fixed core still needs the occasional extra attribute. You pay for a dictionary on those instances that use it, and you keep the declared fields.
Does __del__ close files?
Answer
Not reliably. Cycles and interpreter shutdown delay it. A context manager is the resource protocol.
What do you say in the first minute?
Answer
Lookup checks data descriptors, then the instance, then the rest of the type. property is a data descriptor. Slots drop the per-instance dictionary. __new__ can return an existing object, and __init__ may still run.
Pitfalls
- A mutable default argument is one shared object.
__eq__without thinking about__hash__.- Forgetting
__set_name__on a descriptor used by more than one class. - A slots layout that cannot combine with a base class.
- Recursing inside
__getattribute__by readingself.__dict__the wrong way. Use the base implementation orobject.__getattribute__.
Create a class with a property and also assign a same-named key in __init__ through object.__setattr__ or the instance dictionary. Predict which value obj.name returns, then explain the lookup step that won.