From b88f0281f632e6eedd8911e0c9059036f4e598ec Mon Sep 17 00:00:00 2001 From: Zixuan James Li Date: Fri, 18 Apr 2025 00:37:50 -0400 Subject: [PATCH 1/3] PEP 728: Incorporate discussions since last revision - Mention indexed accesses and assignments with arbitrary keys, per Carl's feedback. - Prepare for submission by clearing the "open issues" sections, with edits when needed. - Include a reference to a prior discussion on `@final` addressing Victorien feedback (https://discuss.python.org/t/pep-728-typeddict-with-typed-extra-items/45443/138). - Update reference implementations. Signed-off-by: Zixuan James Li --- peps/pep-0728.rst | 79 +++++++++++++++++++++++++++++++++++++---------- 1 file changed, 62 insertions(+), 17 deletions(-) diff --git a/peps/pep-0728.rst b/peps/pep-0728.rst index e81d868a27b..d064befda86 100644 --- a/peps/pep-0728.rst +++ b/peps/pep-0728.rst @@ -585,8 +585,34 @@ arguments of this type when constructed by calling the class object:: year=2007, ) # Not OK. Extra items not allowed -Interaction with Mapping[KT, VT] --------------------------------- +Supported and Unsupported Operations +------------------------------------ + +This statement from the `typing spec +`__ +still holds true. + + Operations with arbitrary str keys (instead of string literals or other + expressions with known string values) should generally be rejected. + +This means that indexed accesses and assignments with arbitrary keys can still +be rejected even when ``extra_items`` is specified. + +Operations that already apply to ``NotRequired`` items should generally also +apply to extra items, following the same rationale from the `typing spec +`__ +: + + The exact type checking rules are up to each type checker to decide. In some + cases potentially unsafe operations may be accepted if the alternative is to + generate false positive errors for idiomatic code. + +Some operations are allowed due to the TypedDict being +:term:`typing:assignable` to ``Mapping[str, VT]`` or ``dict[str, VT]``. +The two following sections will expand on that. + +Interaction with Mapping[str, VT] +--------------------------------- A TypedDict type is :term:`typing:assignable` to a type of the form ``Mapping[str, VT]`` when all value types of the items in the TypedDict @@ -618,12 +644,12 @@ and ``items()`` on such TypedDict types:: reveal_type(movie.items()) # Revealed type is 'dict_items[str, str]' reveal_type(movie.values()) # Revealed type is 'dict_values[str, str]' -Interaction with dict[KT, VT] ------------------------------ +Interaction with dict[str, VT] +------------------------------ Because the presence of ``extra_items`` on a closed TypedDict type prohibits additional required keys in its :term:`typing:structural` -:term:`typing:subtypes `, we can determine if the TypedDict type and +:term:`subtypes `, we can determine if the TypedDict type and its structural subtypes will ever have any required key during static analysis. The TypedDict type is :term:`typing:assignable` to ``dict[str, VT]`` if all @@ -708,8 +734,25 @@ been removed in Python 3.13. Because this is a type-checking feature, it can be made available to older versions as long as the type checker supports it. -Open Issues -=========== +Rejected Ideas +============== + +Use ``@final`` instead of ``closed`` Class Parameter +----------------------------------------------------- + +This was discussed `here `__. + +Quoting a relevant `comment +`__ +from Eric Traut: + + The @final class decorator indicates that a class cannot be subclassed. This + makes sense for classes that define nominal types. However, TypedDict is a + structural type, similar to a Protocol. That means two TypedDict classes + with different names but the same field definitions are equivalent types. + Their names and hierarchies don't matter for determining type consistency. + For that reason, @final has no impact on a TypedDict type consistency rules, + nor should it change the behavior of items or values. Use a Special ``__extra_items__`` Key with the ``closed`` Class Parameter ------------------------------------------------------------------------- @@ -767,9 +810,6 @@ types altogether, but there are some disadvantages. `For example - The types don't appear in an annotation context, so their evaluation will not be deferred. -Rejected Ideas -============== - Allowing Extra Items without Specifying the Type ------------------------------------------------ @@ -827,19 +867,24 @@ For example: [index: string]: number | string } -This is a known limitation discussed in `TypeScript's issue tracker -`__, -where it is suggested that there should be a way to exclude the defined keys -from the index signature so that it is possible to define a type like -``MovieWithExtraNumber``. +While this restriction allows for sound indexed accesses with arbitrary keys, +it comes with usability limitations discussed in `TypeScript's issue tracker +`__. +A suggestion was to allow excluding the defined keys from the index signature, +to define a type like ``MovieWithExtraNumber``. This probably involves +subtraction types, which is beyond the scope of this PEP. Reference Implementation ======================== -An earlier revision of proposal is supported in `pyright 1.1.352 -`_, and `pyanalyze +This is supported in `pyright 1.1.386 +`_, and an earlier +revision is supported in `pyanalyze 0.12.0 `_. +This is also supported in `typing-extensions 4.13.0rc1 +`_. + Acknowledgments =============== From 6add4269e0a1a1589cafc39d5d11b032c4d2a0a8 Mon Sep 17 00:00:00 2001 From: Zixuan James Li Date: Fri, 18 Apr 2025 23:10:41 -0400 Subject: [PATCH 2/3] Address follow-ups Signed-off-by: Zixuan James Li --- peps/pep-0728.rst | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/peps/pep-0728.rst b/peps/pep-0728.rst index d064befda86..44c617de941 100644 --- a/peps/pep-0728.rst +++ b/peps/pep-0728.rst @@ -290,10 +290,13 @@ the same rules that apply to ``extra_items=Never`` also apply to This will be further discussed in :ref:`a later section `. -When neither ``extra_items`` nor ``closed=True`` is specified, the TypedDict -is assumed to allow non-required extra items of value type ``ReadOnly[object]`` -during inheritance or assignability checks. This preserves the existing behavior -of TypedDict. +When neither ``extra_items`` nor ``closed=True`` is specified, ``closed=False`` +is assumed. The TypedDict should allow non-required extra items of value type +``ReadOnly[object]`` during inheritance or assignability checks, to +preserve the default TypedDict behavior. Extra keys included in TypedDict +object construction should still be caught, as mentioned in TypedDict's +`typing spec +`__. ``closed`` is also supported with the functional syntax:: @@ -600,8 +603,7 @@ be rejected even when ``extra_items`` is specified. Operations that already apply to ``NotRequired`` items should generally also apply to extra items, following the same rationale from the `typing spec -`__ -: +`__: The exact type checking rules are up to each type checker to decide. In some cases potentially unsafe operations may be accepted if the alternative is to @@ -882,8 +884,8 @@ This is supported in `pyright 1.1.386 revision is supported in `pyanalyze 0.12.0 `_. -This is also supported in `typing-extensions 4.13.0rc1 -`_. +This is also supported in `typing-extensions 4.13.0 +`_. Acknowledgments =============== From 8196d07a7d9887bd77857336a30422cf7fc9cf48 Mon Sep 17 00:00:00 2001 From: Zixuan James Li Date: Sun, 20 Apr 2025 11:34:43 -0400 Subject: [PATCH 3/3] Address feedback from Andrej Also comments from discussion thread: https://discuss.python.org/t/pep-728-typeddict-with-typed-extra-items/45443/148 --- peps/pep-0728.rst | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/peps/pep-0728.rst b/peps/pep-0728.rst index 44c617de941..790d57510b3 100644 --- a/peps/pep-0728.rst +++ b/peps/pep-0728.rst @@ -187,9 +187,10 @@ determining `supported and unsupported operations The ``extra_items`` Class Parameter ----------------------------------- -For a TypedDict type that specifies ``extra_items``, during construction, the -value type of each unknown item is expected to be non-required and assignable -to the ``extra_items`` argument. For example:: +By default ``extra_items`` is unset. For a TypedDict type that specifies +``extra_items``, during construction, the value type of each unknown item +is expected to be non-required and assignable to the ``extra_items`` argument. +For example:: class Movie(TypedDict, extra_items=bool): name: str @@ -233,15 +234,14 @@ Here, ``'year'`` in ``a`` is an extra key defined on ``Movie`` whose value type is ``int``. ``'other_extra_key'`` in ``b`` is another extra key whose value type must be assignable to the value of ``extra_items`` defined on ``MovieBase``. -``extra_items`` is also supported with the functional syntax:: - - Movie = TypedDict("Movie", {"name": str}, extra_items=int | None) - .. _typed-dict-closed: The ``closed`` Class Parameter ------------------------------ +When neither ``extra_items`` nor ``closed=True`` is specified, ``closed=False`` +is assumed. + When ``closed=True`` is set, no extra items are allowed. This is equivalent to ``extra_items=Never``, because there can't be a value type that is assignable to :class:`~typing.Never`. It is a runtime error to use the ``closed`` and @@ -275,8 +275,11 @@ child class is also closed:: As a consequence of ``closed=True`` being equivalent to ``extra_items=Never``, the same rules that apply to ``extra_items=Never`` also apply to -``closed=True``. It is possible to use ``closed=True`` when subclassing if the -``extra_items`` argument is a read-only type:: +``closed=True``. While they both have the same effect, ``closed=True`` is +preferred over ``extra_items=Never``. + +It is possible to use ``closed=True`` when subclassing if the ``extra_items`` +argument is a read-only type:: class Movie(TypedDict, extra_items=ReadOnly[str]): pass @@ -290,8 +293,7 @@ the same rules that apply to ``extra_items=Never`` also apply to This will be further discussed in :ref:`a later section `. -When neither ``extra_items`` nor ``closed=True`` is specified, ``closed=False`` -is assumed. The TypedDict should allow non-required extra items of value type +The TypedDict should allow non-required extra items of value type ``ReadOnly[object]`` during inheritance or assignability checks, to preserve the default TypedDict behavior. Extra keys included in TypedDict object construction should still be caught, as mentioned in TypedDict's @@ -770,7 +772,7 @@ where ``closed=True`` is required for ``__extra_items__`` to be treated specially, to avoid key collision. Some members of the community concern about the elegance of the syntax. -Practiaclly, the key collision with a regular key can be mitigated with +Practically, the key collision with a regular key can be mitigated with workarounds, but since using a reserved key is central to this proposal, there are limited ways forward to address the concerns.