spec: Rewrite TypedDict spec #2072
Conversation
This is an edit of the TypedDict spec for clarity and flow. My goal was to unify the pieces of the spec that derive from the various PEPs into a coherent whole. I removed excessive examples and motivations: the spec should specify, not justify. The length of the spec chapter is reduced by more than half. This change is on top of python#2068 (adding PEP 728). The general approach I took is to first define the kinds of TypedDicts that can exist, then explain the syntax for defining TypedDicts, then discuss other aspects of TypedDict types. I introduce some new terminology around PEP 728 to make it easier to talk about the different kinds of TypedDict. TypedDicts are defined to have a property called openness, which can have three states: - Open: all TypedDicts prior to PEP 728 - Closed: no extra keys are allowed (closed=True) - With extra items: extra_items=... from PEP 728 I retained existing text where it made sense but also wrote some from scratch.
|
Posted about this on Discuss: https://discuss.python.org/t/revision-of-the-typeddict-spec/102675 |
Co-authored-by: Joren Hammudoglu <[email protected]>
|
Thanks for the feedback! I pushed some changes. |
There was a problem hiding this comment.
Thanks for doing this @JelleZijlstra!
I left some comments for you to review regarding equivalency vs consistency.
|
|
||
| Movie = TypedDict('Movie', {'name': str, 'year': int}) | ||
| - If it is mutable in ``A``, it must also be mutable in ``B``, and the item type in ``B`` must be | ||
| :term:`equivalent` to the item type in ``A``. (It follows that for assignability, the two item types |
There was a problem hiding this comment.
I don't think equivalence is correct here. I think it should say "consistent with".
class Foo(TypedDict):
x: list[int]
class Bar(Foo):
x: list[Any] # OKWe should be very suspect if the word "equivalent" shows up anywhere in sublcassing or assignability rules in the spec.
There was a problem hiding this comment.
This rule list is specifically about subtyping, not assignability. There is a note that this implies that for assignability, you can read "consistency" wherever "equivalence" shows up. Perhaps it would be clearer to describe the assignability procedure, but subtyping is more fundamental.
There was a problem hiding this comment.
Yeah, I think the current wording is confusing and not as useful to users of the type system or for type checker authors. I see your point about subtyping being more fundamental, but assignability rules are more important for understanding typing and for properly implementing a type checker. I doubt if any of the major type checkers have code that tests for subtyping. Pyright does not. It's all about assignability.
The note helps, but it doesn't cover all aspects of the description. For example, the note doesn't apply to the statement "If it is read-only in A, the item type in B must be a subtype of the item type in A". Without additional notes, the assignability rules are arguably ambiguous.
There was a problem hiding this comment.
That makes sense and I'm happy to recast this in terms of assignability (and consistency), but I'd like to hear what others think (maybe @carljm?).
There was a problem hiding this comment.
assignability rules are more important for understanding typing
I'm not sure I agree with this. For a surface-level understanding, maybe. It's very hard to understand the logic behind the actual assignability rules without understanding their relationship to subtyping via materialization.
and for properly implementing a type checker
That really depends on how the type checker chooses to implement the assignability rules.
I doubt if any of the major type checkers have code that tests for subtyping.
Ty certainly does, and according to a conversation I had with Jukka at PyCon, mypy does as well. Subtyping is the relevant relation for simplifying a union (or in the case of ty, intersection) type. Since ty uses intersections for narrowing, their correct simplification is important.
I think it is important that the spec be clear about the rules for both subtyping and assignability. In general I think the best way to do that is to present the rules for subtyping, and derive the rules for assignability from the definition of materialization. I don't think an accurate definition of subtyping can be clearly derived from a definition of assignability.
the note doesn't apply to the statement "If it is read-only in A, the item type in B must be a subtype of the item type in A".
I think the note already does correctly apply here. If you use the subtyping relation as given for two TypedDicts, and apply materialization, you will get a correct assignability relation.
I think it might be helpful to expand the note about materialization to say explicitly here what is already said in the core concepts section: that this means that where equivalence is mentioned for subtyping, it should be consistency instead for assignability, and where subtyping is mentioned, it should be assignability instead.
|
|
||
| .. _typeddict-assignability: | ||
| - If ``B`` has an item with the same key, it must also be mutable, and its item type must be | ||
| :term:`equivalent` to the item type in ``A``. (As before, it follows that for assignability, the two item types |
There was a problem hiding this comment.
I think this should say "consistent with" rather than "equivalent to".
| First, any TypedDict type is :term:`assignable` to ``Mapping[str, object]``. | ||
| - If ``B`` is closed, the check fails. | ||
| - If ``B`` has extra items, the extra items type must not be read-only and must | ||
| be :term:`equivalent` to the item type in ``A``. |
| must be a subtype of the extra items type in ``A``. Additionally, for any items in ``B`` that are not present in ``A``, | ||
| the item type must be a subtype of the extra items type in ``A``. | ||
| - If ``A`` has mutable extra items, ``B`` must also have mutable extra items, and the extra items type in ``B`` | ||
| must be :term:`equivalent` to the extra items type in ``A``. Additionally, for any items in ``B`` that are not present in ``A``, |
| the item type must be a subtype of the extra items type in ``A``. | ||
| - If ``A`` has mutable extra items, ``B`` must also have mutable extra items, and the extra items type in ``B`` | ||
| must be :term:`equivalent` to the extra items type in ``A``. Additionally, for any items in ``B`` that are not present in ``A``, | ||
| the item type must be :term:`equivalent` to the extra items type in ``A``. |
|
|
||
| def g(b: B) -> None: | ||
| f(b) # Type check error: 'B' not assignable to 'A' | ||
| - The TypedDict type has mutable :term:`extra items` of a type that is :term:`equivalent` to ``VT``. |
| c: C = {'x': 0, 'y': 'foo'} | ||
| g(c) | ||
| c['y'] + 'bar' # Runtime error: int + str | ||
| - The value type of the item is :term:`equivalent` to ``VT``. |
This is an edit of the TypedDict spec for clarity and flow. My goal was to
unify the pieces of the spec that derive from the various PEPs into a coherent
whole. I removed excessive examples and motivations: the spec should specify,
not justify. The length of the spec chapter is reduced by more than half.
The general approach I took is to first define the kinds of TypedDicts that
can exist, then explain the syntax for defining TypedDicts, then discuss
other aspects of TypedDict types.
I introduce some new terminology around PEP 728 to make it easier to talk
about the different kinds of TypedDict. TypedDicts are defined to have a
property called openness, which can have three states:
I retained existing text where it made sense but also wrote some from
scratch.