From dd86fb4ba5a9db1c1e7293917af68d0cf0ddeaaf Mon Sep 17 00:00:00 2001 From: Adam Turner <9087854+AA-Turner@users.noreply.github.com> Date: Wed, 3 Sep 2025 07:20:16 +0100 Subject: [PATCH 01/10] GH-101100: Resolve reference warnings in whatsnew/3.7.rst (#138410) Resolve reference warnings in whatsnew/3.7.rst --- Doc/howto/instrumentation.rst | 2 + Doc/library/hmac.rst | 4 +- Doc/reference/datamodel.rst | 5 ++ Doc/tools/.nitignore | 1 - Doc/whatsnew/3.7.rst | 95 +++++++++++++++++++---------------- 5 files changed, 61 insertions(+), 46 deletions(-) diff --git a/Doc/howto/instrumentation.rst b/Doc/howto/instrumentation.rst index 6e03ef20a21fa3..b3db1189e5dcbc 100644 --- a/Doc/howto/instrumentation.rst +++ b/Doc/howto/instrumentation.rst @@ -269,6 +269,8 @@ should instead read: (assuming a :ref:`debug build ` of CPython 3.6) +.. _static-markers: + Available static markers ------------------------ diff --git a/Doc/library/hmac.rst b/Doc/library/hmac.rst index 57076c38086c79..d5608bd7543eb1 100644 --- a/Doc/library/hmac.rst +++ b/Doc/library/hmac.rst @@ -50,7 +50,9 @@ cannot be used with HMAC. .. versionadded:: 3.7 -An HMAC object has the following methods: +.. class:: HMAC + + An HMAC object has the following methods: .. method:: HMAC.update(msg) diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index 7af3457070b84a..da04cfde3bd587 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -2351,6 +2351,9 @@ Customizing module attribute access single: __dir__ (module attribute) single: __class__ (module attribute) +.. method:: module.__getattr__ + module.__dir__ + Special names ``__getattr__`` and ``__dir__`` can be also used to customize access to module attributes. The ``__getattr__`` function at the module level should accept one argument which is the name of an attribute and return the @@ -2364,6 +2367,8 @@ The ``__dir__`` function should accept no arguments, and return an iterable of strings that represents the names accessible on module. If present, this function overrides the standard :func:`dir` search on a module. +.. attribute:: module.__class__ + For a more fine grained customization of the module behavior (setting attributes, properties, etc.), one can set the ``__class__`` attribute of a module object to a subclass of :class:`types.ModuleType`. For example:: diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index 7d0512fd0fa72d..07671fe17c1d89 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -66,6 +66,5 @@ Doc/whatsnew/3.3.rst Doc/whatsnew/3.4.rst Doc/whatsnew/3.5.rst Doc/whatsnew/3.6.rst -Doc/whatsnew/3.7.rst Doc/whatsnew/3.8.rst Doc/whatsnew/3.10.rst diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index f420fa5c04479b..6e6934befccb3b 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -320,9 +320,9 @@ effort will be made to add such support. PEP 562: Customization of Access to Module Attributes ----------------------------------------------------- -Python 3.7 allows defining :meth:`__getattr__` on modules and will call +Python 3.7 allows defining :meth:`~module.__getattr__` on modules and will call it whenever a module attribute is otherwise not found. Defining -:meth:`__dir__` on modules is now also allowed. +:meth:`~module.__dir__` on modules is now also allowed. A typical example of where this may be useful is module attribute deprecation and lazy loading. @@ -409,8 +409,8 @@ PEP 560: Core Support for ``typing`` module and Generic Types Initially :pep:`484` was designed in such way that it would not introduce *any* changes to the core CPython interpreter. Now type hints and the :mod:`typing` module are extensively used by the community, so this restriction is removed. -The PEP introduces two special methods :meth:`__class_getitem__` and -``__mro_entries__``, these methods are now used by most classes and special +The PEP introduces two special methods :meth:`~object.__class_getitem__` and +:meth:`~object.__mro_entries__`, these methods are now used by most classes and special constructs in :mod:`typing`. As a result, the speed of various operations with types increased up to 7 times, the generic types can be used without metaclass conflicts, and several long standing bugs in :mod:`typing` module are @@ -603,7 +603,7 @@ The new :mod:`importlib.resources` module provides several new APIs and one new ABC for access to, opening, and reading *resources* inside packages. Resources are roughly similar to files inside packages, but they needn't be actual files on the physical file system. Module loaders can provide a -:meth:`get_resource_reader` function which returns +:meth:`!get_resource_reader` function which returns a :class:`importlib.abc.ResourceReader` instance to support this new API. Built-in file path loaders and zip file loaders both support this. @@ -910,9 +910,9 @@ which allows listing the names of properties which should not become enum members. (Contributed by Ethan Furman in :issue:`31801`.) -In Python 3.8, attempting to check for non-Enum objects in :class:`Enum` +In Python 3.8, attempting to check for non-Enum objects in :class:`~enum.Enum` classes will raise a :exc:`TypeError` (e.g. ``1 in Color``); similarly, -attempting to check for non-Flag objects in a :class:`Flag` member will +attempting to check for non-Flag objects in a :class:`~enum.Flag` member will raise :exc:`TypeError` (e.g. ``1 in Perm.RW``); currently, both operations return :const:`False` instead and are deprecated. (Contributed by Ethan Furman in :issue:`33217`.) @@ -969,7 +969,7 @@ uses the current working directory. (Contributed by Stéphane Wirtel and Julien Palard in :issue:`28707`.) The new :class:`ThreadingHTTPServer ` class -uses threads to handle requests using :class:`~socketserver.ThreadingMixin`. +uses threads to handle requests using :class:`~socketserver.ThreadingMixIn`. It is used when ``http.server`` is run with ``-m``. (Contributed by Julien Palard in :issue:`31639`.) @@ -1052,12 +1052,12 @@ support the loading of resources from packages. See also lacks a spec. (Contributed by Garvit Khatri in :issue:`29851`.) -:func:`importlib.find_spec` now raises :exc:`ModuleNotFoundError` instead of +:func:`importlib.util.find_spec` now raises :exc:`ModuleNotFoundError` instead of :exc:`AttributeError` if the specified parent module is not a package (i.e. lacks a ``__path__`` attribute). (Contributed by Milan Oberkirch in :issue:`30436`.) -The new :func:`importlib.source_hash` can be used to compute the hash of +The new :func:`importlib.util.source_hash` can be used to compute the hash of the passed source. A :ref:`hash-based .pyc file ` embeds the value returned by this function. @@ -1148,7 +1148,7 @@ running. (Contributed by Antoine Pitrou in :issue:`30596`.) The new :meth:`Process.kill() ` method can -be used to terminate the process using the :data:`SIGKILL` signal on Unix. +be used to terminate the process using the :data:`~signal.SIGKILL` signal on Unix. (Contributed by Vitor Pereira in :issue:`30794`.) Non-daemonic threads created by :class:`~multiprocessing.Process` are now @@ -1280,9 +1280,10 @@ This function should be used instead of :func:`os.close` for better compatibility across platforms. (Contributed by Christian Heimes in :issue:`32454`.) -The :mod:`socket` module now exposes the :const:`socket.TCP_CONGESTION` -(Linux 2.6.13), :const:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37), and -:const:`socket.TCP_NOTSENT_LOWAT` (Linux 3.12) constants. +The :mod:`socket` module now exposes the :ref:`socket.TCP_CONGESTION +` (Linux 2.6.13), :ref:`socket.TCP_USER_TIMEOUT +` (Linux 2.6.37), and :ref:`socket.TCP_NOTSENT_LOWAT +` (Linux 3.12) constants. (Contributed by Omar Sandoval in :issue:`26273` and Nathaniel J. Smith in :issue:`29728`.) @@ -1298,11 +1299,14 @@ by default. socketserver ------------ -:meth:`socketserver.ThreadingMixIn.server_close` now waits until all non-daemon -threads complete. :meth:`socketserver.ForkingMixIn.server_close` now waits +:meth:`socketserver.ThreadingMixIn.server_close +` now waits until all non-daemon +threads complete. :meth:`socketserver.ForkingMixIn.server_close +` now waits until all child processes complete. -Add a new :attr:`socketserver.ForkingMixIn.block_on_close` class attribute to +Add a new :attr:`socketserver.ForkingMixIn.block_on_close +` class attribute to :class:`socketserver.ForkingMixIn` and :class:`socketserver.ThreadingMixIn` classes. Set the class attribute to ``False`` to get the pre-3.7 behaviour. @@ -1323,7 +1327,7 @@ ssl --- The :mod:`ssl` module now uses OpenSSL's builtin API instead of -:func:`~ssl.match_hostname` to check a host name or an IP address. Values +:func:`!match_hostname` to check a host name or an IP address. Values are validated during TLS handshake. Any certificate validation error including failing the host name check now raises :exc:`~ssl.SSLCertVerificationError` and aborts the handshake with a proper @@ -1341,7 +1345,7 @@ Host name validation can be customized with The ``ssl`` module no longer sends IP addresses in SNI TLS extension. (Contributed by Christian Heimes in :issue:`32185`.) -:func:`~ssl.match_hostname` no longer supports partial wildcards like +:func:`!match_hostname` no longer supports partial wildcards like ``www*.example.org``. (Contributed by Mandeep Singh in :issue:`23033` and Christian Heimes in :issue:`31399`.) @@ -1438,7 +1442,7 @@ The new :func:`sys.get_coroutine_origin_tracking_depth` function returns the current coroutine origin tracking depth, as set by the new :func:`sys.set_coroutine_origin_tracking_depth`. :mod:`asyncio` has been converted to use this new API instead of -the deprecated :func:`sys.set_coroutine_wrapper`. +the deprecated :func:`!sys.set_coroutine_wrapper`. (Contributed by Nathaniel J. Smith in :issue:`32591`.) @@ -1615,7 +1619,7 @@ external entities by default. xml.etree --------- -:ref:`ElementPath ` predicates in the :meth:`find` +:ref:`ElementPath ` predicates in the :meth:`!find` methods can now compare text of the current node with ``[. = "text"]``, not only text in children. Predicates also allow adding spaces for better readability. (Contributed by Stefan Behnel in :issue:`31648`.) @@ -1624,7 +1628,7 @@ better readability. (Contributed by Stefan Behnel in :issue:`31648`.) xmlrpc.server ------------- -:meth:`SimpleXMLRPCDispatcher.register_function ` +:meth:`!SimpleXMLRPCDispatcher.register_function` can now be used as a decorator. (Contributed by Xiang Zhang in :issue:`7769`.) @@ -1682,15 +1686,15 @@ The :mod:`tracemalloc` now exposes a C API through the new functions. (Contributed by Victor Stinner in :issue:`30054`.) -The new :c:func:`import__find__load__start` and -:c:func:`import__find__load__done` static markers can be used to trace -module imports. +The new :ref:`import__find__load__start ` and +:ref:`import__find__load__done ` static markers can be used +to trace module imports. (Contributed by Christian Heimes in :issue:`31574`.) The fields :c:member:`!name` and :c:member:`!doc` of structures :c:type:`PyMemberDef`, :c:type:`PyGetSetDef`, :c:type:`PyStructSequence_Field`, :c:type:`PyStructSequence_Desc`, -and :c:struct:`wrapperbase` are now of type ``const char *`` rather of +and :c:struct:`!wrapperbase` are now of type ``const char *`` rather of ``char *``. (Contributed by Serhiy Storchaka in :issue:`28761`.) The result of :c:func:`PyUnicode_AsUTF8AndSize` and :c:func:`PyUnicode_AsUTF8` @@ -1719,8 +1723,8 @@ Added C API support for timezones with timezone constructors and access to the UTC singleton with :c:data:`PyDateTime_TimeZone_UTC`. Contributed by Paul Ganssle in :issue:`10381`. -The type of results of :c:func:`PyThread_start_new_thread` and -:c:func:`PyThread_get_thread_ident`, and the *id* parameter of +The type of results of :c:func:`!PyThread_start_new_thread` and +:c:func:`!PyThread_get_thread_ident`, and the *id* parameter of :c:func:`PyThreadState_SetAsyncExc` changed from :c:expr:`long` to :c:expr:`unsigned long`. (Contributed by Serhiy Storchaka in :issue:`6532`.) @@ -1847,8 +1851,8 @@ make the creation of named tuples 4 to 6 times faster. (Contributed by Jelle Zijlstra with further improvements by INADA Naoki, Serhiy Storchaka, and Raymond Hettinger in :issue:`28638`.) -:meth:`date.fromordinal` and :meth:`date.fromtimestamp` are now up to -30% faster in the common case. +:meth:`datetime.date.fromordinal` and :meth:`datetime.date.fromtimestamp` +are now up to 30% faster in the common case. (Contributed by Paul Ganssle in :issue:`32403`.) The :func:`os.fwalk` function is now up to 2 times faster thanks to @@ -1997,9 +2001,9 @@ modes (this will be an error in future Python releases). enum ---- -In Python 3.8, attempting to check for non-Enum objects in :class:`Enum` +In Python 3.8, attempting to check for non-Enum objects in :class:`~enum.Enum` classes will raise a :exc:`TypeError` (e.g. ``1 in Color``); similarly, -attempting to check for non-Flag objects in a :class:`Flag` member will +attempting to check for non-Flag objects in a :class:`~enum.Flag` member will raise :exc:`TypeError` (e.g. ``1 in Perm.RW``); currently, both operations return :const:`False` instead. (Contributed by Ethan Furman in :issue:`33217`.) @@ -2034,14 +2038,14 @@ favour of :class:`importlib.abc.ResourceReader`. locale ------ -:func:`locale.format` has been deprecated, use :meth:`locale.format_string` +:func:`!locale.format` has been deprecated, use :meth:`locale.format_string` instead. (Contributed by Garvit in :issue:`10379`.) macpath ------- -The :mod:`macpath` is now deprecated and will be removed in Python 3.8. +The :mod:`!macpath` is now deprecated and will be removed in Python 3.8. (Contributed by Chi Hsuan Yen in :issue:`9850`.) @@ -2066,7 +2070,7 @@ if the passed argument is larger than 16 bits, an exception will be raised. ssl --- -:func:`ssl.wrap_socket` is deprecated. Use +:func:`!ssl.wrap_socket` is deprecated. Use :meth:`ssl.SSLContext.wrap_socket` instead. (Contributed by Christian Heimes in :issue:`28124`.) @@ -2082,8 +2086,8 @@ Use :func:`!sunau.open` instead. sys --- -Deprecated :func:`sys.set_coroutine_wrapper` and -:func:`sys.get_coroutine_wrapper`. +Deprecated :func:`!sys.set_coroutine_wrapper` and +:func:`!sys.get_coroutine_wrapper`. The undocumented ``sys.callstats()`` function has been deprecated and will be removed in a future Python version. @@ -2093,7 +2097,7 @@ will be removed in a future Python version. wave ---- -:func:`wave.openfp` has been deprecated and will be removed in Python 3.9. +:func:`!wave.openfp` has been deprecated and will be removed in Python 3.9. Use :func:`wave.open` instead. (Contributed by Brian Curtin in :issue:`31985`.) @@ -2173,8 +2177,8 @@ The following features and APIs have been removed from Python 3.7: * Removed previously deprecated in Python 2.4 classes ``Plist``, ``Dict`` and ``_InternalDict`` in the :mod:`plistlib` module. Dict values in the result - of functions :func:`~plistlib.readPlist` and - :func:`~plistlib.readPlistFromBytes` are now normal dicts. You no longer + of functions :func:`!readPlist` and + :func:`!readPlistFromBytes` are now normal dicts. You no longer can use attribute access to access items of these dictionaries. * The ``asyncio.windows_utils.socketpair()`` function has been @@ -2191,7 +2195,7 @@ The following features and APIs have been removed from Python 3.7: * Direct instantiation of :class:`ssl.SSLSocket` and :class:`ssl.SSLObject` objects is now prohibited. The constructors were never documented, tested, or designed as public constructors. Users were supposed to use - :func:`ssl.wrap_socket` or :class:`ssl.SSLContext`. + :func:`!ssl.wrap_socket` or :class:`ssl.SSLContext`. (Contributed by Christian Heimes in :issue:`32951`.) * The unused ``distutils`` ``install_misc`` command has been removed. @@ -2275,15 +2279,18 @@ Changes in Python Behavior Changes in the Python API ------------------------- -* :meth:`socketserver.ThreadingMixIn.server_close` now waits until all +* :meth:`socketserver.ThreadingMixIn.server_close + ` now waits until all non-daemon threads complete. Set the new :attr:`socketserver.ThreadingMixIn.block_on_close` class attribute to ``False`` to get the pre-3.7 behaviour. (Contributed by Victor Stinner in :issue:`31233` and :issue:`33540`.) -* :meth:`socketserver.ForkingMixIn.server_close` now waits until all +* :meth:`socketserver.ForkingMixIn.server_close + ` now waits until all child processes complete. Set the new - :attr:`socketserver.ForkingMixIn.block_on_close` class attribute to ``False`` + :attr:`socketserver.ForkingMixIn.block_on_close + ` class attribute to ``False`` to get the pre-3.7 behaviour. (Contributed by Victor Stinner in :issue:`31151` and :issue:`33540`.) From 51244ba16a7312945a60dff18ed155a581295fbf Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A9n=C3=A9dikt=20Tran?= <10796600+picnixz@users.noreply.github.com> Date: Wed, 3 Sep 2025 09:55:53 +0200 Subject: [PATCH 02/10] gh-116946: add `Py_TPFLAGS_IMMUTABLETYPE` to `_random.Random` (#138341) --- .../2025-09-02-10-23-09.gh-issue-116946.U6RpwK.rst | 2 ++ Modules/_randommodule.c | 13 ++++++++----- 2 files changed, 10 insertions(+), 5 deletions(-) create mode 100644 Misc/NEWS.d/next/Library/2025-09-02-10-23-09.gh-issue-116946.U6RpwK.rst diff --git a/Misc/NEWS.d/next/Library/2025-09-02-10-23-09.gh-issue-116946.U6RpwK.rst b/Misc/NEWS.d/next/Library/2025-09-02-10-23-09.gh-issue-116946.U6RpwK.rst new file mode 100644 index 00000000000000..015cf24c8869f8 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2025-09-02-10-23-09.gh-issue-116946.U6RpwK.rst @@ -0,0 +1,2 @@ +The :class:`!_random.Random` C type is now immutable. Patch by Bénédikt +Tran. diff --git a/Modules/_randommodule.c b/Modules/_randommodule.c index 2f4f388ce1161a..aa2fd28c232f28 100644 --- a/Modules/_randommodule.c +++ b/Modules/_randommodule.c @@ -595,11 +595,14 @@ static PyType_Slot Random_Type_slots[] = { }; static PyType_Spec Random_Type_spec = { - "_random.Random", - sizeof(RandomObject), - 0, - Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, - Random_Type_slots + .name = "_random.Random", + .basicsize = sizeof(RandomObject), + .flags = ( + Py_TPFLAGS_DEFAULT + | Py_TPFLAGS_BASETYPE + | Py_TPFLAGS_IMMUTABLETYPE + ), + .slots = Random_Type_slots }; PyDoc_STRVAR(module_doc, From 572df47840d910b9fc9cd951074232ae89442be1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A9n=C3=A9dikt=20Tran?= <10796600+picnixz@users.noreply.github.com> Date: Wed, 3 Sep 2025 10:17:17 +0200 Subject: [PATCH 03/10] gh-116946: fully implement GC protocol for `_curses_panel.panel` (#138333) This commit fixes possible reference loops via `panel.set_userptr` by implementing `tp_clear` and `tp_traverse` for panel objects. --- Modules/_curses_panel.c | 58 +++++++++++++++++++++++++++++++---------- 1 file changed, 44 insertions(+), 14 deletions(-) diff --git a/Modules/_curses_panel.c b/Modules/_curses_panel.c index 3408a505575580..66a8c40953da8c 100644 --- a/Modules/_curses_panel.c +++ b/Modules/_curses_panel.c @@ -410,8 +410,11 @@ static PyObject * PyCursesPanel_New(_curses_panel_state *state, PANEL *pan, PyCursesWindowObject *wo) { - PyCursesPanelObject *po = PyObject_New(PyCursesPanelObject, - state->PyCursesPanel_Type); + assert(state != NULL); + PyTypeObject *type = state->PyCursesPanel_Type; + assert(type != NULL); + assert(type->tp_alloc != NULL); + PyCursesPanelObject *po = (PyCursesPanelObject *)type->tp_alloc(type, 0); if (po == NULL) { return NULL; } @@ -426,20 +429,31 @@ PyCursesPanel_New(_curses_panel_state *state, PANEL *pan, return (PyObject *)po; } +static int +PyCursesPanel_Clear(PyObject *op) +{ + PyCursesPanelObject *self = _PyCursesPanelObject_CAST(op); + PyObject *extra = (PyObject *)panel_userptr(self->pan); + if (extra != NULL) { + Py_DECREF(extra); + if (set_panel_userptr(self->pan, NULL) == ERR) { + curses_panel_panel_set_error(self, "set_panel_userptr", NULL); + return -1; + } + } + // self->wo should not be cleared because an associated WINDOW may exist + return 0; +} + static void PyCursesPanel_Dealloc(PyObject *self) { - PyObject *tp, *obj; - PyCursesPanelObject *po = _PyCursesPanelObject_CAST(self); + PyTypeObject *tp = Py_TYPE(self); + PyObject_GC_UnTrack(self); - tp = (PyObject *) Py_TYPE(po); - obj = (PyObject *) panel_userptr(po->pan); - if (obj) { - Py_DECREF(obj); - if (set_panel_userptr(po->pan, NULL) == ERR) { - curses_panel_panel_set_error(po, "set_panel_userptr", "__del__"); - PyErr_FormatUnraisable("Exception ignored in PyCursesPanel_Dealloc()"); - } + PyCursesPanelObject *po = _PyCursesPanelObject_CAST(self); + if (PyCursesPanel_Clear(self) < 0) { + PyErr_FormatUnraisable("Exception ignored in PyCursesPanel_Dealloc()"); } if (del_panel(po->pan) == ERR && !PyErr_Occurred()) { curses_panel_panel_set_error(po, "del_panel", "__del__"); @@ -452,10 +466,20 @@ PyCursesPanel_Dealloc(PyObject *self) PyErr_FormatUnraisable("Exception ignored in PyCursesPanel_Dealloc()"); } } - PyObject_Free(po); + tp->tp_free(po); Py_DECREF(tp); } +static int +PyCursesPanel_Traverse(PyObject *op, visitproc visit, void *arg) +{ + PyCursesPanelObject *self = _PyCursesPanelObject_CAST(op); + Py_VISIT(Py_TYPE(op)); + Py_VISIT(panel_userptr(self->pan)); + Py_VISIT(self->wo); + return 0; +} + /* panel_above(NULL) returns the bottom panel in the stack. To get this behaviour we use curses.panel.bottom_panel(). */ /*[clinic input] @@ -647,7 +671,9 @@ static PyMethodDef PyCursesPanel_Methods[] = { /* -------------------------------------------------------*/ static PyType_Slot PyCursesPanel_Type_slots[] = { + {Py_tp_clear, PyCursesPanel_Clear}, {Py_tp_dealloc, PyCursesPanel_Dealloc}, + {Py_tp_traverse, PyCursesPanel_Traverse}, {Py_tp_methods, PyCursesPanel_Methods}, {0, 0}, }; @@ -655,7 +681,11 @@ static PyType_Slot PyCursesPanel_Type_slots[] = { static PyType_Spec PyCursesPanel_Type_spec = { .name = "_curses_panel.panel", .basicsize = sizeof(PyCursesPanelObject), - .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_DISALLOW_INSTANTIATION, + .flags = ( + Py_TPFLAGS_DEFAULT + | Py_TPFLAGS_DISALLOW_INSTANTIATION + | Py_TPFLAGS_HAVE_GC + ), .slots = PyCursesPanel_Type_slots }; From 4a33077fdb546db71b16d50cdd552529a9a1c910 Mon Sep 17 00:00:00 2001 From: Serhiy Storchaka Date: Wed, 3 Sep 2025 11:26:56 +0300 Subject: [PATCH 04/10] gh-138264: Fix gcc 14 compiler warnings (GH-138265) --- Include/internal/pycore_stats.h | 2 +- Lib/test/test_generated_cases.py | 2 +- Modules/_sre/sre.c | 2 +- Python/executor_cases.c.h | 8 +++--- Python/generated_cases.c.h | 30 +++++++++++----------- Tools/cases_generator/generators_common.py | 2 +- 6 files changed, 23 insertions(+), 23 deletions(-) diff --git a/Include/internal/pycore_stats.h b/Include/internal/pycore_stats.h index ab649574f33dbf..24f239a2135b93 100644 --- a/Include/internal/pycore_stats.h +++ b/Include/internal/pycore_stats.h @@ -77,7 +77,7 @@ PyAPI_FUNC(PyObject*) _Py_GetSpecializationStats(void); #define RARE_EVENT_INTERP_INC(interp, name) \ do { \ /* saturating add */ \ - int val = FT_ATOMIC_LOAD_UINT8_RELAXED(interp->rare_events.name); \ + uint8_t val = FT_ATOMIC_LOAD_UINT8_RELAXED(interp->rare_events.name); \ if (val < UINT8_MAX) { \ FT_ATOMIC_STORE_UINT8(interp->rare_events.name, val + 1); \ } \ diff --git a/Lib/test/test_generated_cases.py b/Lib/test/test_generated_cases.py index ec44a0f9ce3fb3..6e0e2eafd3281c 100644 --- a/Lib/test/test_generated_cases.py +++ b/Lib/test/test_generated_cases.py @@ -1572,7 +1572,7 @@ def test_instruction_size_macro(self): frame->instr_ptr = next_instr; next_instr += 1; INSTRUCTION_STATS(OP); - frame->return_offset = 1 ; + frame->return_offset = 1u ; DISPATCH(); } """ diff --git a/Modules/_sre/sre.c b/Modules/_sre/sre.c index 49eb52b635bfbb..fdf00e6499cb6b 100644 --- a/Modules/_sre/sre.c +++ b/Modules/_sre/sre.c @@ -2359,7 +2359,7 @@ match_getindex(MatchObject* self, PyObject* index) } // Check that i*2 cannot overflow to make static analyzers happy - assert(i <= SRE_MAXGROUPS); + assert((size_t)i <= SRE_MAXGROUPS); return i; } diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index 3dcb2decc43737..635ca659394c32 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -1688,7 +1688,7 @@ _PyInterpreterFrame* pushed_frame = _PyFrame_PushUnchecked(tstate, getitem, 2, frame); pushed_frame->localsplus[0] = container; pushed_frame->localsplus[1] = sub; - frame->return_offset = 6 ; + frame->return_offset = 6u ; new_frame = PyStackRef_Wrap(pushed_frame); stack_pointer[-3] = new_frame; stack_pointer += -2; @@ -2057,8 +2057,8 @@ gen->gi_frame_state = FRAME_EXECUTING; gen->gi_exc_state.previous_item = tstate->exc_info; tstate->exc_info = &gen->gi_exc_state; - assert( 2 + oparg <= UINT16_MAX); - frame->return_offset = (uint16_t)( 2 + oparg); + assert( 2u + oparg <= UINT16_MAX); + frame->return_offset = (uint16_t)( 2u + oparg); pushed_frame->previous = frame; gen_frame = PyStackRef_Wrap(pushed_frame); stack_pointer[-1] = gen_frame; @@ -4610,7 +4610,7 @@ gen->gi_exc_state.previous_item = tstate->exc_info; tstate->exc_info = &gen->gi_exc_state; pushed_frame->previous = frame; - frame->return_offset = (uint16_t)( 2 + oparg); + frame->return_offset = (uint16_t)( 2u + oparg); gen_frame = PyStackRef_Wrap(pushed_frame); stack_pointer[0] = gen_frame; stack_pointer += 1; diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index 7547eaad125370..c3e37f080fa6e4 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -653,7 +653,7 @@ _PyInterpreterFrame* pushed_frame = _PyFrame_PushUnchecked(tstate, getitem, 2, frame); pushed_frame->localsplus[0] = container; pushed_frame->localsplus[1] = sub; - frame->return_offset = 6 ; + frame->return_offset = 6u ; new_frame = PyStackRef_Wrap(pushed_frame); } // _PUSH_FRAME @@ -1587,7 +1587,7 @@ if (new_frame == NULL) { JUMP_TO_LABEL(error); } - frame->return_offset = 4 ; + frame->return_offset = 4u ; DISPATCH_INLINED(new_frame); } STACKREFS_TO_PYOBJECTS(arguments, total_args, args_o); @@ -2603,7 +2603,7 @@ if (new_frame == NULL) { JUMP_TO_LABEL(error); } - assert( 1 == 1); + assert( 1u == 1); frame->return_offset = 1; DISPATCH_INLINED(new_frame); } @@ -2883,8 +2883,8 @@ if (new_frame == NULL) { JUMP_TO_LABEL(error); } - assert( 4 == 1 + INLINE_CACHE_ENTRIES_CALL_KW); - frame->return_offset = 4 ; + assert( 4u == 1 + INLINE_CACHE_ENTRIES_CALL_KW); + frame->return_offset = 4u ; DISPATCH_INLINED(new_frame); } STACKREFS_TO_PYOBJECTS(arguments, total_args, args_o); @@ -5716,7 +5716,7 @@ gen->gi_exc_state.previous_item = tstate->exc_info; tstate->exc_info = &gen->gi_exc_state; pushed_frame->previous = frame; - frame->return_offset = (uint16_t)( 2 + oparg); + frame->return_offset = (uint16_t)( 2u + oparg); gen_frame = PyStackRef_Wrap(pushed_frame); } // _PUSH_FRAME @@ -6334,7 +6334,7 @@ if (new_frame == NULL) { JUMP_TO_LABEL(error); } - frame->return_offset = 4 ; + frame->return_offset = 4u ; DISPATCH_INLINED(new_frame); } STACKREFS_TO_PYOBJECTS(arguments, total_args, args_o); @@ -6547,7 +6547,7 @@ if (new_frame == NULL) { JUMP_TO_LABEL(error); } - assert( 1 == 1); + assert( 1u == 1); frame->return_offset = 1; DISPATCH_INLINED(new_frame); } @@ -6690,8 +6690,8 @@ if (new_frame == NULL) { JUMP_TO_LABEL(error); } - assert( 4 == 1 + INLINE_CACHE_ENTRIES_CALL_KW); - frame->return_offset = 4 ; + assert( 4u == 1 + INLINE_CACHE_ENTRIES_CALL_KW); + frame->return_offset = 4u ; DISPATCH_INLINED(new_frame); } STACKREFS_TO_PYOBJECTS(arguments, total_args, args_o); @@ -8060,7 +8060,7 @@ stack_pointer += -1; assert(WITHIN_STACK_BOUNDS()); new_frame->localsplus[1] = PyStackRef_FromPyObjectNew(name); - frame->return_offset = 10 ; + frame->return_offset = 10u ; DISPATCH_INLINED(new_frame); } @@ -10459,8 +10459,8 @@ gen->gi_frame_state = FRAME_EXECUTING; gen->gi_exc_state.previous_item = tstate->exc_info; tstate->exc_info = &gen->gi_exc_state; - assert( 2 + oparg <= UINT16_MAX); - frame->return_offset = (uint16_t)( 2 + oparg); + assert( 2u + oparg <= UINT16_MAX); + frame->return_offset = (uint16_t)( 2u + oparg); assert(gen_frame->previous == NULL); gen_frame->previous = frame; DISPATCH_INLINED(gen_frame); @@ -10560,8 +10560,8 @@ gen->gi_frame_state = FRAME_EXECUTING; gen->gi_exc_state.previous_item = tstate->exc_info; tstate->exc_info = &gen->gi_exc_state; - assert( 2 + oparg <= UINT16_MAX); - frame->return_offset = (uint16_t)( 2 + oparg); + assert( 2u + oparg <= UINT16_MAX); + frame->return_offset = (uint16_t)( 2u + oparg); pushed_frame->previous = frame; gen_frame = PyStackRef_Wrap(pushed_frame); } diff --git a/Tools/cases_generator/generators_common.py b/Tools/cases_generator/generators_common.py index cc0edcdf6006fd..61e855eb003706 100644 --- a/Tools/cases_generator/generators_common.py +++ b/Tools/cases_generator/generators_common.py @@ -470,7 +470,7 @@ def instruction_size(self, """Replace the INSTRUCTION_SIZE macro with the size of the current instruction.""" if uop.instruction_size is None: raise analysis_error("The INSTRUCTION_SIZE macro requires uop.instruction_size to be set", tkn) - self.out.emit(f" {uop.instruction_size} ") + self.out.emit(f" {uop.instruction_size}u ") return True def _print_storage(self, reason:str, storage: Storage) -> None: From 7274d076077212ecda4e83932dcc4ae69c62af58 Mon Sep 17 00:00:00 2001 From: Serhiy Storchaka Date: Wed, 3 Sep 2025 11:31:34 +0300 Subject: [PATCH 05/10] gh-78502: Add a trackfd parameter to mmap.mmap() on Windows (GH-138238) If trackfd is False, the file handle corresponding to fileno will not be duplicated. --- Doc/library/mmap.rst | 16 +++- Doc/whatsnew/3.15.rst | 9 +++ Lib/test/test_mmap.py | 77 +++++++++---------- ...5-08-29-12-05-33.gh-issue-78502.VpIMxg.rst | 2 + Modules/mmapmodule.c | 59 +++++++------- 5 files changed, 92 insertions(+), 71 deletions(-) create mode 100644 Misc/NEWS.d/next/Library/2025-08-29-12-05-33.gh-issue-78502.VpIMxg.rst diff --git a/Doc/library/mmap.rst b/Doc/library/mmap.rst index 5d81477443ca31..d9d401a2789c0e 100644 --- a/Doc/library/mmap.rst +++ b/Doc/library/mmap.rst @@ -48,10 +48,11 @@ update the underlying file. To map anonymous memory, -1 should be passed as the fileno along with the length. -.. class:: mmap(fileno, length, tagname=None, access=ACCESS_DEFAULT, offset=0) +.. class:: mmap(fileno, length, tagname=None, \ + access=ACCESS_DEFAULT, offset=0, *, trackfd=True) **(Windows version)** Maps *length* bytes from the file specified by the - file handle *fileno*, and creates a mmap object. If *length* is larger + file descriptor *fileno*, and creates a mmap object. If *length* is larger than the current size of the file, the file is extended to contain *length* bytes. If *length* is ``0``, the maximum length of the map is the current size of the file, except that if the file is empty Windows raises an @@ -69,6 +70,17 @@ To map anonymous memory, -1 should be passed as the fileno along with the length will be relative to the offset from the beginning of the file. *offset* defaults to 0. *offset* must be a multiple of the :const:`ALLOCATIONGRANULARITY`. + If *trackfd* is ``False``, the file handle corresponding to *fileno* will + not be duplicated, and the resulting :class:`!mmap` object will not + be associated with the map's underlying file. + This means that the :meth:`~mmap.mmap.size` and :meth:`~mmap.mmap.resize` + methods will fail. + This mode is useful to limit the number of open file handles. + The original file can be renamed (but not deleted) after closing *fileno*. + + .. versionchanged:: next + The *trackfd* parameter was added. + .. audit-event:: mmap.__new__ fileno,length,access,offset mmap.mmap .. class:: mmap(fileno, length, flags=MAP_SHARED, prot=PROT_WRITE|PROT_READ, \ diff --git a/Doc/whatsnew/3.15.rst b/Doc/whatsnew/3.15.rst index b5e138aa674697..932bb100cbee23 100644 --- a/Doc/whatsnew/3.15.rst +++ b/Doc/whatsnew/3.15.rst @@ -358,6 +358,15 @@ math (Contributed by Bénédikt Tran in :gh:`135853`.) +mmap +---- + +* :class:`mmap.mmap` now has a *trackfd* parameter on Windows; + if it is ``False``, the file handle corresponding to *fileno* will + not be duplicated. + (Contributed by Serhiy Storchaka in :gh:`78502`.) + + os.path ------- diff --git a/Lib/test/test_mmap.py b/Lib/test/test_mmap.py index 32881d36dcae10..da69770915092a 100644 --- a/Lib/test/test_mmap.py +++ b/Lib/test/test_mmap.py @@ -1,3 +1,4 @@ +from test import support from test.support import ( requires, _2G, _4G, gc_collect, cpython_only, is_emscripten, is_apple, in_systemd_nspawn_sync_suppressed, @@ -269,41 +270,44 @@ def test_access_parameter(self): self.assertRaises(TypeError, m.write_byte, 0) m.close() - @unittest.skipIf(os.name == 'nt', 'trackfd not present on Windows') - def test_trackfd_parameter(self): + @support.subTests('close_original_fd', (True, False)) + def test_trackfd_parameter(self, close_original_fd): size = 64 with open(TESTFN, "wb") as f: f.write(b"a"*size) - for close_original_fd in True, False: - with self.subTest(close_original_fd=close_original_fd): - with open(TESTFN, "r+b") as f: - with mmap.mmap(f.fileno(), size, trackfd=False) as m: - if close_original_fd: - f.close() - self.assertEqual(len(m), size) - with self.assertRaises(ValueError): - m.size() - with self.assertRaises(ValueError): - m.resize(size * 2) - with self.assertRaises(ValueError): - m.resize(size // 2) - self.assertEqual(m.closed, False) - - # Smoke-test other API - m.write_byte(ord('X')) - m[2] = ord('Y') - m.flush() - with open(TESTFN, "rb") as f: - self.assertEqual(f.read(4), b'XaYa') - self.assertEqual(m.tell(), 1) - m.seek(0) - self.assertEqual(m.tell(), 0) - self.assertEqual(m.read_byte(), ord('X')) - - self.assertEqual(m.closed, True) - self.assertEqual(os.stat(TESTFN).st_size, size) - - @unittest.skipIf(os.name == 'nt', 'trackfd not present on Windows') + with open(TESTFN, "r+b") as f: + with mmap.mmap(f.fileno(), size, trackfd=False) as m: + if close_original_fd: + f.close() + self.assertEqual(len(m), size) + with self.assertRaises(ValueError): + m.size() + with self.assertRaises(ValueError): + m.resize(size * 2) + with self.assertRaises(ValueError): + m.resize(size // 2) + self.assertIs(m.closed, False) + + # Smoke-test other API + m.write_byte(ord('X')) + m[2] = ord('Y') + m.flush() + with open(TESTFN, "rb") as f: + self.assertEqual(f.read(4), b'XaYa') + self.assertEqual(m.tell(), 1) + m.seek(0) + self.assertEqual(m.tell(), 0) + self.assertEqual(m.read_byte(), ord('X')) + + if os.name == 'nt' and not close_original_fd: + self.assertRaises(PermissionError, os.rename, TESTFN, TESTFN+'1') + else: + os.rename(TESTFN, TESTFN+'1') + os.rename(TESTFN+'1', TESTFN) + + self.assertIs(m.closed, True) + self.assertEqual(os.stat(TESTFN).st_size, size) + def test_trackfd_neg1(self): size = 64 with mmap.mmap(-1, size, trackfd=False) as m: @@ -315,15 +319,6 @@ def test_trackfd_neg1(self): m[0] = ord('a') assert m[0] == ord('a') - @unittest.skipIf(os.name != 'nt', 'trackfd only fails on Windows') - def test_no_trackfd_parameter_on_windows(self): - # 'trackffd' is an invalid keyword argument for this function - size = 64 - with self.assertRaises(TypeError): - mmap.mmap(-1, size, trackfd=True) - with self.assertRaises(TypeError): - mmap.mmap(-1, size, trackfd=False) - def test_bad_file_desc(self): # Try opening a bad file descriptor... self.assertRaises(OSError, mmap.mmap, -2, 4096) diff --git a/Misc/NEWS.d/next/Library/2025-08-29-12-05-33.gh-issue-78502.VpIMxg.rst b/Misc/NEWS.d/next/Library/2025-08-29-12-05-33.gh-issue-78502.VpIMxg.rst new file mode 100644 index 00000000000000..1043ee3310558f --- /dev/null +++ b/Misc/NEWS.d/next/Library/2025-08-29-12-05-33.gh-issue-78502.VpIMxg.rst @@ -0,0 +1,2 @@ +:class:`mmap.mmap` now has a *trackfd* parameter on Windows; if it is +``False``, the file handle corresponding to *fileno* will not be duplicated. diff --git a/Modules/mmapmodule.c b/Modules/mmapmodule.c index 1c300546c33fe8..dcaadb818e0bf7 100644 --- a/Modules/mmapmodule.c +++ b/Modules/mmapmodule.c @@ -119,12 +119,12 @@ typedef struct { #ifdef UNIX int fd; - _Bool trackfd; int flags; #endif PyObject *weakreflist; access_mode access; + _Bool trackfd; } mmap_object; #define mmap_object_CAST(op) ((mmap_object *)(op)) @@ -636,13 +636,11 @@ is_resizeable(mmap_object *self) "mmap can't resize with extant buffers exported."); return 0; } -#ifdef UNIX if (!self->trackfd) { PyErr_SetString(PyExc_ValueError, "mmap can't resize with trackfd=False."); return 0; } -#endif if ((self->access == ACCESS_WRITE) || (self->access == ACCESS_DEFAULT)) return 1; PyErr_Format(PyExc_TypeError, @@ -734,8 +732,6 @@ mmap_size_method(PyObject *op, PyObject *Py_UNUSED(ignored)) return PyLong_FromLong((long)low); size = (((long long)high)<<32) + low; return PyLong_FromLongLong(size); - } else { - return PyLong_FromSsize_t(self->size); } #endif /* MS_WINDOWS */ @@ -750,6 +746,7 @@ mmap_size_method(PyObject *op, PyObject *Py_UNUSED(ignored)) return PyLong_FromLong(status.st_size); #endif } +#endif /* UNIX */ else if (self->trackfd) { return PyLong_FromSsize_t(self->size); } @@ -758,7 +755,6 @@ mmap_size_method(PyObject *op, PyObject *Py_UNUSED(ignored)) "can't get size with trackfd=False"); return NULL; } -#endif /* UNIX */ } /* This assumes that you want the entire file mapped, @@ -1476,7 +1472,7 @@ static PyObject * new_mmap_object(PyTypeObject *type, PyObject *args, PyObject *kwdict); PyDoc_STRVAR(mmap_doc, -"Windows: mmap(fileno, length[, tagname[, access[, offset]]])\n\ +"Windows: mmap(fileno, length[, tagname[, access[, offset[, trackfd]]]])\n\ \n\ Maps length bytes from the file specified by the file handle fileno,\n\ and returns a mmap object. If length is larger than the current size\n\ @@ -1737,16 +1733,17 @@ new_mmap_object(PyTypeObject *type, PyObject *args, PyObject *kwdict) PyObject *tagname = Py_None; DWORD dwErr = 0; int fileno; - HANDLE fh = 0; + HANDLE fh = INVALID_HANDLE_VALUE; int access = (access_mode)ACCESS_DEFAULT; + int trackfd = 1; DWORD flProtect, dwDesiredAccess; static char *keywords[] = { "fileno", "length", "tagname", - "access", "offset", NULL }; + "access", "offset", "trackfd", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kwdict, "in|OiL", keywords, + if (!PyArg_ParseTupleAndKeywords(args, kwdict, "in|OiL$p", keywords, &fileno, &map_size, - &tagname, &access, &offset)) { + &tagname, &access, &offset, &trackfd)) { return NULL; } @@ -1813,22 +1810,27 @@ new_mmap_object(PyTypeObject *type, PyObject *args, PyObject *kwdict) m_obj->map_handle = NULL; m_obj->tagname = NULL; m_obj->offset = offset; + m_obj->trackfd = trackfd; - if (fh) { - /* It is necessary to duplicate the handle, so the - Python code can close it on us */ - if (!DuplicateHandle( - GetCurrentProcess(), /* source process handle */ - fh, /* handle to be duplicated */ - GetCurrentProcess(), /* target proc handle */ - (LPHANDLE)&m_obj->file_handle, /* result */ - 0, /* access - ignored due to options value */ - FALSE, /* inherited by child processes? */ - DUPLICATE_SAME_ACCESS)) { /* options */ - dwErr = GetLastError(); - Py_DECREF(m_obj); - PyErr_SetFromWindowsErr(dwErr); - return NULL; + if (fh != INVALID_HANDLE_VALUE) { + if (trackfd) { + /* It is necessary to duplicate the handle, so the + Python code can close it on us */ + if (!DuplicateHandle( + GetCurrentProcess(), /* source process handle */ + fh, /* handle to be duplicated */ + GetCurrentProcess(), /* target proc handle */ + &fh, /* result */ + 0, /* access - ignored due to options value */ + FALSE, /* inherited by child processes? */ + DUPLICATE_SAME_ACCESS)) /* options */ + { + dwErr = GetLastError(); + Py_DECREF(m_obj); + PyErr_SetFromWindowsErr(dwErr); + return NULL; + } + m_obj->file_handle = fh; } if (!map_size) { DWORD low,high; @@ -1836,7 +1838,8 @@ new_mmap_object(PyTypeObject *type, PyObject *args, PyObject *kwdict) /* low might just happen to have the value INVALID_FILE_SIZE; so we need to check the last error also. */ if (low == INVALID_FILE_SIZE && - (dwErr = GetLastError()) != NO_ERROR) { + (dwErr = GetLastError()) != NO_ERROR) + { Py_DECREF(m_obj); return PyErr_SetFromWindowsErr(dwErr); } @@ -1898,7 +1901,7 @@ new_mmap_object(PyTypeObject *type, PyObject *args, PyObject *kwdict) off_lo = (DWORD)(offset & 0xFFFFFFFF); /* For files, it would be sufficient to pass 0 as size. For anonymous maps, we have to pass the size explicitly. */ - m_obj->map_handle = CreateFileMappingW(m_obj->file_handle, + m_obj->map_handle = CreateFileMappingW(fh, NULL, flProtect, size_hi, From 974532e75888a82adbaa11c832fe67c132832b65 Mon Sep 17 00:00:00 2001 From: Cody Maloney Date: Wed, 3 Sep 2025 02:37:06 -0700 Subject: [PATCH 06/10] gh-138013: Move I/O tests to test_io (#138365) Centralize `io` tests into the `test_io` module so they are easier to find and work on. This will make it easier to split `test_general` which takes 30+ seconds in a debug build on my machine. This renames `test_bufio` to be `test_bufferedio` so that it matches the implementation file name (`bufferedio.c`). Validation performed: Tests are run in parallel after change: ```bash ./python.exe -m test test_io -uall,largefile,extralargefile -M12G -j8 ``` Docstring reformat in `test_io/__init__.py` looks reasonable: ```python >>> import test.test_io >>> help(test.test_io) ``` Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> --- Lib/test/libregrtest/findtests.py | 1 + Lib/test/test_io/__init__.py | 23 +++++++++++++++++ .../test_bufferedio.py} | 0 Lib/test/{ => test_io}/test_file.py | 0 Lib/test/{ => test_io}/test_fileio.py | 0 Lib/test/test_io/test_general.py | 25 +++---------------- Lib/test/{ => test_io}/test_largefile.py | 0 Lib/test/{ => test_io}/test_memoryio.py | 0 Lib/test/{ => test_io}/test_univnewlines.py | 0 9 files changed, 28 insertions(+), 21 deletions(-) rename Lib/test/{test_bufio.py => test_io/test_bufferedio.py} (100%) rename Lib/test/{ => test_io}/test_file.py (100%) rename Lib/test/{ => test_io}/test_fileio.py (100%) rename Lib/test/{ => test_io}/test_largefile.py (100%) rename Lib/test/{ => test_io}/test_memoryio.py (100%) rename Lib/test/{ => test_io}/test_univnewlines.py (100%) diff --git a/Lib/test/libregrtest/findtests.py b/Lib/test/libregrtest/findtests.py index f01c1240774707..79afaf9083ae59 100644 --- a/Lib/test/libregrtest/findtests.py +++ b/Lib/test/libregrtest/findtests.py @@ -24,6 +24,7 @@ "test_future_stmt", "test_gdb", "test_inspect", + "test_io", "test_pydoc", "test_multiprocessing_fork", "test_multiprocessing_forkserver", diff --git a/Lib/test/test_io/__init__.py b/Lib/test/test_io/__init__.py index 4b16ecc31156a5..c94fad3e779381 100644 --- a/Lib/test/test_io/__init__.py +++ b/Lib/test/test_io/__init__.py @@ -1,3 +1,26 @@ +"""Tests for the io module and its implementations (_io and _pyio) + +Tests are split across multiple files to increase +parallelism and focus on specific implementation pieces. + +* test_io + * test_bufferedio - tests file buffering + * test_memoryio - tests BytesIO and StringIO + * test_fileio - tests FileIO + * test_file - tests the file interface + * test_general - tests everything else in the io module + * test_univnewlines - tests universal newline support + * test_largefile - tests operations on a file greater than 2**32 bytes + (only enabled with -ulargefile) +* test_free_threading/test_io - tests thread safety of io objects + +.. attention:: + When writing tests for io, it's important to test both the C and Python + implementations. This is usually done by writing a base test that refers to + the type it is testing as an attribute. Then it provides custom subclasses to + test both implementations. This directory contains lots of examples. +""" + import os from test.support import load_package_tests diff --git a/Lib/test/test_bufio.py b/Lib/test/test_io/test_bufferedio.py similarity index 100% rename from Lib/test/test_bufio.py rename to Lib/test/test_io/test_bufferedio.py diff --git a/Lib/test/test_file.py b/Lib/test/test_io/test_file.py similarity index 100% rename from Lib/test/test_file.py rename to Lib/test/test_io/test_file.py diff --git a/Lib/test/test_fileio.py b/Lib/test/test_io/test_fileio.py similarity index 100% rename from Lib/test/test_fileio.py rename to Lib/test/test_io/test_fileio.py diff --git a/Lib/test/test_io/test_general.py b/Lib/test/test_io/test_general.py index e3d7d26a7e0f9c..30fe1e2f866091 100644 --- a/Lib/test/test_io/test_general.py +++ b/Lib/test/test_io/test_general.py @@ -1,24 +1,7 @@ -"""Unit tests for the io module.""" - -# Tests of io are scattered over the test suite: -# * test_bufio - tests file buffering -# * test_memoryio - tests BytesIO and StringIO -# * test_fileio - tests FileIO -# * test_file - tests the file interface -# * test_io.test_general - tests everything else in the io module -# * test_univnewlines - tests universal newline support -# * test_largefile - tests operations on a file greater than 2**32 bytes -# (only enabled with -ulargefile) -# * test_free_threading/test_io - tests thread safety of io objects - -################################################################################ -# ATTENTION TEST WRITERS!!! -################################################################################ -# When writing tests for io, it's important to test both the C and Python -# implementations. This is usually done by writing a base test that refers to -# the type it is testing as an attribute. Then it provides custom subclasses to -# test both implementations. This file has lots of examples. -################################################################################ +"""General tests for the io module. + +New tests should go in more specific modules; see test_io/__init__.py +""" import abc import array diff --git a/Lib/test/test_largefile.py b/Lib/test/test_io/test_largefile.py similarity index 100% rename from Lib/test/test_largefile.py rename to Lib/test/test_io/test_largefile.py diff --git a/Lib/test/test_memoryio.py b/Lib/test/test_io/test_memoryio.py similarity index 100% rename from Lib/test/test_memoryio.py rename to Lib/test/test_io/test_memoryio.py diff --git a/Lib/test/test_univnewlines.py b/Lib/test/test_io/test_univnewlines.py similarity index 100% rename from Lib/test/test_univnewlines.py rename to Lib/test/test_io/test_univnewlines.py From 6dd21e9f56cfc9eb5a5c21382e3241e700d86aa6 Mon Sep 17 00:00:00 2001 From: Cody Maloney Date: Wed, 3 Sep 2025 02:49:59 -0700 Subject: [PATCH 07/10] gh-138013: Remove `test_io` load_tests namespace manipulation (#138366) Reduce what happens in `load_tests` so that the next change, moving the `Buffered*` tests to `test_bufferdio` is purely mechanical movement and updating imports. This adds two classes, one per I/O implementation, to act as dispatch to the implementation-specific mocks as well as module members. Previously the mappings CTestCase and PyTestCase provide were injected directly during `load_tests`. CTestCase and PyTestCase inherit from `unittest.TestCase` so when the split happens default test discovery will work for the classes in `test_bufferedio`. `test_general` keeps a manual test list for this refactoring; some of the tests (ex. `ProtocolsTest`) aren't currently run and fixing that + helpers to not be picked up is out of my current scope. CTestCase and PyTestCase have an `io` class member which points to the implementation meaning that can be removed from individual test cases which now inherit from them. This code is picking up `MockRawIO` which is defined globally in the module but these should use the mock specific to the I/O implementation being tested. Co-authored-by: Victor Stinner --- Lib/test/test_io/test_general.py | 119 +++++++++++++++++-------------- 1 file changed, 67 insertions(+), 52 deletions(-) diff --git a/Lib/test/test_io/test_general.py b/Lib/test/test_io/test_general.py index 30fe1e2f866091..604b56cea21fac 100644 --- a/Lib/test/test_io/test_general.py +++ b/Lib/test/test_io/test_general.py @@ -317,6 +317,45 @@ class PyMockNonBlockWriterIO(MockNonBlockWriterIO, pyio.RawIOBase): BlockingIOError = pyio.BlockingIOError +# Build classes which point to all the right mocks per io implementation +class CTestCase(unittest.TestCase): + io = io + is_C = True + + MockRawIO = CMockRawIO + MisbehavedRawIO = CMisbehavedRawIO + MockFileIO = CMockFileIO + CloseFailureIO = CCloseFailureIO + MockNonBlockWriterIO = CMockNonBlockWriterIO + MockUnseekableIO = CMockUnseekableIO + MockRawIOWithoutRead = CMockRawIOWithoutRead + SlowFlushRawIO = CSlowFlushRawIO + MockCharPseudoDevFileIO = CMockCharPseudoDevFileIO + + # Use the class as a proxy to the io module members. + def __getattr__(self, name): + return getattr(io, name) + + +class PyTestCase(unittest.TestCase): + io = pyio + is_C = False + + MockRawIO = PyMockRawIO + MisbehavedRawIO = PyMisbehavedRawIO + MockFileIO = PyMockFileIO + CloseFailureIO = PyCloseFailureIO + MockNonBlockWriterIO = PyMockNonBlockWriterIO + MockUnseekableIO = PyMockUnseekableIO + MockRawIOWithoutRead = PyMockRawIOWithoutRead + SlowFlushRawIO = PySlowFlushRawIO + MockCharPseudoDevFileIO = PyMockCharPseudoDevFileIO + + # Use the class as a proxy to the _pyio module members. + def __getattr__(self, name): + return getattr(pyio, name) + + class IOTest(unittest.TestCase): def setUp(self): @@ -1083,7 +1122,7 @@ def reader(file, barrier): write_count * thread_count) -class CIOTest(IOTest): +class CIOTest(IOTest, CTestCase): def test_IOBase_finalize(self): # Issue #12149: segmentation fault on _PyIOBase_finalize when both a @@ -1207,7 +1246,7 @@ def test_stringio_setstate(self): obj.__setstate__(('', '', 0, {})) self.assertEqual(obj.getvalue(), '') -class PyIOTest(IOTest): +class PyIOTest(IOTest, PyTestCase): pass @@ -1438,7 +1477,7 @@ def test_buffer_freeing(self) : bufio.close() self.assertEqual(sys.getsizeof(bufio), size) -class BufferedReaderTest(unittest.TestCase, CommonBufferedTests): +class BufferedReaderTest(CommonBufferedTests): read_mode = "rb" def test_constructor(self): @@ -1773,7 +1812,7 @@ def test_seek_character_device_file(self): self.assertEqual(buf.seek(0, io.SEEK_CUR), 0) -class CBufferedReaderTest(BufferedReaderTest, SizeofTest): +class CBufferedReaderTest(BufferedReaderTest, SizeofTest, CTestCase): tp = io.BufferedReader def test_initialization(self): @@ -1832,11 +1871,11 @@ def test_bad_readinto_type(self): self.assertIsInstance(cm.exception.__cause__, TypeError) -class PyBufferedReaderTest(BufferedReaderTest): +class PyBufferedReaderTest(BufferedReaderTest, PyTestCase): tp = pyio.BufferedReader -class BufferedWriterTest(unittest.TestCase, CommonBufferedTests): +class BufferedWriterTest(CommonBufferedTests): write_mode = "wb" def test_constructor(self): @@ -2131,8 +2170,7 @@ def test_slow_close_from_thread(self): t.join() - -class CBufferedWriterTest(BufferedWriterTest, SizeofTest): +class CBufferedWriterTest(BufferedWriterTest, SizeofTest, CTestCase): tp = io.BufferedWriter def test_initialization(self): @@ -2172,10 +2210,10 @@ def test_args_error(self): self.tp(self.BytesIO(), 1024, 1024, 1024) -class PyBufferedWriterTest(BufferedWriterTest): +class PyBufferedWriterTest(BufferedWriterTest, PyTestCase): tp = pyio.BufferedWriter -class BufferedRWPairTest(unittest.TestCase): +class BufferedRWPairTest: def test_constructor(self): pair = self.tp(self.MockRawIO(), self.MockRawIO()) @@ -2204,14 +2242,14 @@ def test_constructor_max_buffer_size_removal(self): self.tp(self.MockRawIO(), self.MockRawIO(), 8, 12) def test_constructor_with_not_readable(self): - class NotReadable(MockRawIO): + class NotReadable(self.MockRawIO): def readable(self): return False self.assertRaises(OSError, self.tp, NotReadable(), self.MockRawIO()) def test_constructor_with_not_writeable(self): - class NotWriteable(MockRawIO): + class NotWriteable(self.MockRawIO): def writable(self): return False @@ -2357,9 +2395,9 @@ def writer_close(): writer.close = lambda: None def test_isatty(self): - class SelectableIsAtty(MockRawIO): + class SelectableIsAtty(self.MockRawIO): def __init__(self, isatty): - MockRawIO.__init__(self) + super().__init__() self._isatty = isatty def isatty(self): @@ -2383,10 +2421,10 @@ def test_weakref_clearing(self): brw = None ref = None # Shouldn't segfault. -class CBufferedRWPairTest(BufferedRWPairTest): +class CBufferedRWPairTest(BufferedRWPairTest, CTestCase): tp = io.BufferedRWPair -class PyBufferedRWPairTest(BufferedRWPairTest): +class PyBufferedRWPairTest(BufferedRWPairTest, PyTestCase): tp = pyio.BufferedRWPair @@ -2645,7 +2683,7 @@ def test_interleaved_readline_write(self): test_truncate_on_read_only = None -class CBufferedRandomTest(BufferedRandomTest, SizeofTest): +class CBufferedRandomTest(BufferedRandomTest, SizeofTest, CTestCase): tp = io.BufferedRandom def test_garbage_collection(self): @@ -2658,7 +2696,7 @@ def test_args_error(self): self.tp(self.BytesIO(), 1024, 1024, 1024) -class PyBufferedRandomTest(BufferedRandomTest): +class PyBufferedRandomTest(BufferedRandomTest, PyTestCase): tp = pyio.BufferedRandom @@ -4058,8 +4096,7 @@ def _to_memoryview(buf): return memoryview(arr) -class CTextIOWrapperTest(TextIOWrapperTest): - io = io +class CTextIOWrapperTest(TextIOWrapperTest, CTestCase): shutdown_error = "LookupError: unknown encoding: ascii" def test_initialization(self): @@ -4159,8 +4196,7 @@ def write(self, data): buf._write_stack) -class PyTextIOWrapperTest(TextIOWrapperTest): - io = pyio +class PyTextIOWrapperTest(TextIOWrapperTest, PyTestCase): shutdown_error = "LookupError: unknown encoding: ascii" @@ -4282,6 +4318,8 @@ def test_translate(self): self.assertEqual(decoder.decode(b"\r\r\n"), "\r\r\n") class CIncrementalNewlineDecoderTest(IncrementalNewlineDecoderTest): + IncrementalNewlineDecoder = io.IncrementalNewlineDecoder + @support.cpython_only def test_uninitialized(self): uninitialized = self.IncrementalNewlineDecoder.__new__( @@ -4293,7 +4331,7 @@ def test_uninitialized(self): class PyIncrementalNewlineDecoderTest(IncrementalNewlineDecoderTest): - pass + IncrementalNewlineDecoder = pyio.IncrementalNewlineDecoder # XXX Tests for open() @@ -4662,8 +4700,7 @@ def test_text_encoding(self): self.assertEqual(b"utf-8", proc.out.strip()) -class CMiscIOTest(MiscIOTest): - io = io +class CMiscIOTest(MiscIOTest, CTestCase): name_of_module = "io", "_io" extra_exported = "BlockingIOError", @@ -4728,8 +4765,7 @@ def test_daemon_threads_shutdown_stderr_deadlock(self): self.check_daemon_threads_shutdown_deadlock('stderr') -class PyMiscIOTest(MiscIOTest): - io = pyio +class PyMiscIOTest(MiscIOTest, PyTestCase): name_of_module = "_pyio", "io" extra_exported = "BlockingIOError", "open_code", not_exported = "valid_seek_flags", @@ -4990,11 +5026,11 @@ def test_interrupted_write_retry_text(self): self.check_interrupted_write_retry("x", mode="w", encoding="latin1") -class CSignalsTest(SignalsTest): - io = io +class CSignalsTest(SignalsTest, CTestCase): + pass -class PySignalsTest(SignalsTest): - io = pyio +class PySignalsTest(SignalsTest, PyTestCase): + pass # Handling reentrancy issues would slow down _pyio even more, so the # tests are disabled. @@ -5034,27 +5070,6 @@ def load_tests(loader, tests, pattern): ProtocolsTest, ) - # Put the namespaces of the IO module we are testing and some useful mock - # classes in the __dict__ of each test. - mocks = (MockRawIO, MisbehavedRawIO, MockFileIO, CloseFailureIO, - MockNonBlockWriterIO, MockUnseekableIO, MockRawIOWithoutRead, - SlowFlushRawIO, MockCharPseudoDevFileIO) - all_members = io.__all__ - c_io_ns = {name : getattr(io, name) for name in all_members} - py_io_ns = {name : getattr(pyio, name) for name in all_members} - globs = globals() - c_io_ns.update((x.__name__, globs["C" + x.__name__]) for x in mocks) - py_io_ns.update((x.__name__, globs["Py" + x.__name__]) for x in mocks) - for test in tests: - if test.__name__.startswith("C"): - for name, obj in c_io_ns.items(): - setattr(test, name, obj) - test.is_C = True - elif test.__name__.startswith("Py"): - for name, obj in py_io_ns.items(): - setattr(test, name, obj) - test.is_C = False - suite = loader.suiteClass() for test in tests: suite.addTest(loader.loadTestsFromTestCase(test)) From 04c236bf7a273ff7b5cb4dd8c9e90f0f7145e0e6 Mon Sep 17 00:00:00 2001 From: Adam Turner <9087854+AA-Turner@users.noreply.github.com> Date: Wed, 3 Sep 2025 12:10:12 +0100 Subject: [PATCH 08/10] GH-123299: Copyedit 3.14 What's New: Move and consolidate Porting (#138400) --- Doc/whatsnew/3.14.rst | 186 +++++++++++++++++++++--------------------- 1 file changed, 94 insertions(+), 92 deletions(-) diff --git a/Doc/whatsnew/3.14.rst b/Doc/whatsnew/3.14.rst index aaa3487cb5ca8a..853882ebec58af 100644 --- a/Doc/whatsnew/3.14.rst +++ b/Doc/whatsnew/3.14.rst @@ -2819,41 +2819,6 @@ CPython bytecode changes * Replaced the opcode ``BINARY_SUBSCR`` by :opcode:`BINARY_OP` with oparg ``NB_SUBSCR``. (Contributed by Irit Katriel in :gh:`100239`.) -Porting to Python 3.14 -====================== - -This section lists previously described changes and other bugfixes -that may require changes to your code. - -Changes in the Python API -------------------------- - -* :class:`functools.partial` is now a method descriptor. - Wrap it in :func:`staticmethod` if you want to preserve the old behavior. - (Contributed by Serhiy Storchaka and Dominykas Grigonis in :gh:`121027`.) - -* The :ref:`garbage collector is now incremental `, - which means that the behavior of :func:`gc.collect` changes slightly: - - * ``gc.collect(1)``: Performs an increment of garbage collection, - rather than collecting generation 1. - * Other calls to :func:`!gc.collect` are unchanged. - -* The :func:`locale.nl_langinfo` function now temporarily sets the ``LC_CTYPE`` - locale in some cases. - This temporary change affects other threads. - (Contributed by Serhiy Storchaka in :gh:`69998`.) - -* :class:`types.UnionType` is now an alias for :class:`typing.Union`, - causing changes in some behaviors. - See :ref:`above ` for more details. - (Contributed by Jelle Zijlstra in :gh:`105499`.) - -* The runtime behavior of annotations has changed in various ways; see - :ref:`above ` for details. While most code that interacts - with annotations should continue to work, some undocumented details may behave - differently. - Build changes ============= @@ -3048,63 +3013,6 @@ Limited C API changes (Contributed by Victor Stinner in :gh:`91417`.) -Porting to Python 3.14 ----------------------- - -* :c:func:`Py_Finalize` now deletes all interned strings. This - is backwards incompatible to any C-Extension that holds onto an interned - string after a call to :c:func:`Py_Finalize` and is then reused after a - call to :c:func:`Py_Initialize`. Any issues arising from this behavior will - normally result in crashes during the execution of the subsequent call to - :c:func:`Py_Initialize` from accessing uninitialized memory. To fix, use - an address sanitizer to identify any use-after-free coming from - an interned string and deallocate it during module shutdown. - (Contributed by Eddie Elizondo in :gh:`113601`.) - -* The :ref:`Unicode Exception Objects ` C API - now raises a :exc:`TypeError` if its exception argument is not - a :exc:`UnicodeError` object. - (Contributed by Bénédikt Tran in :gh:`127691`.) - -.. _whatsnew314-refcount: - -* The interpreter internally avoids some reference count modifications when - loading objects onto the operands stack by :term:`borrowing ` - references when possible. This can lead to smaller reference count values - compared to previous Python versions. C API extensions that checked - :c:func:`Py_REFCNT` of ``1`` to determine if an function argument is not - referenced by any other code should instead use - :c:func:`PyUnstable_Object_IsUniqueReferencedTemporary` as a safer replacement. - - -* Private functions promoted to public C APIs: - - * ``_PyBytes_Join()``: :c:func:`PyBytes_Join` - * ``_PyLong_IsNegative()``: :c:func:`PyLong_IsNegative` - * ``_PyLong_IsPositive()``: :c:func:`PyLong_IsPositive` - * ``_PyLong_IsZero()``: :c:func:`PyLong_IsZero` - * ``_PyLong_Sign()``: :c:func:`PyLong_GetSign` - * ``_PyUnicodeWriter_Dealloc()``: :c:func:`PyUnicodeWriter_Discard` - * ``_PyUnicodeWriter_Finish()``: :c:func:`PyUnicodeWriter_Finish` - * ``_PyUnicodeWriter_Init()``: use :c:func:`PyUnicodeWriter_Create` - * ``_PyUnicodeWriter_Prepare()``: (no replacement) - * ``_PyUnicodeWriter_PrepareKind()``: (no replacement) - * ``_PyUnicodeWriter_WriteChar()``: :c:func:`PyUnicodeWriter_WriteChar` - * ``_PyUnicodeWriter_WriteStr()``: :c:func:`PyUnicodeWriter_WriteStr` - * ``_PyUnicodeWriter_WriteSubstring()``: :c:func:`PyUnicodeWriter_WriteSubstring` - * ``_PyUnicode_EQ()``: :c:func:`PyUnicode_Equal` - * ``_PyUnicode_Equal()``: :c:func:`PyUnicode_Equal` - * ``_Py_GetConfig()``: :c:func:`PyConfig_Get` and :c:func:`PyConfig_GetInt` - * ``_Py_HashBytes()``: :c:func:`Py_HashBuffer` - * ``_Py_fopen_obj()``: :c:func:`Py_fopen` - * ``PyMutex_IsLocked()`` : :c:func:`PyMutex_IsLocked` - - The `pythoncapi-compat project`_ can be used to get most of these new - functions on Python 3.13 and older. - -.. _pythoncapi-compat project: https://github.com/python/pythoncapi-compat/ - - .. _whatsnew314-c-api-deprecated: Deprecated @@ -3210,3 +3118,97 @@ Removed Please use :c:func:`Py_EnterRecursiveCall` to guard against runaway recursion in C code. (Removed in :gh:`133079`, see also :gh:`130396`.) + + +Porting to Python 3.14 +====================== + +This section lists previously described changes and other bugfixes +that may require changes to your code. + + +Changes in the Python API +------------------------- + +* :class:`functools.partial` is now a method descriptor. + Wrap it in :func:`staticmethod` if you want to preserve the old behavior. + (Contributed by Serhiy Storchaka and Dominykas Grigonis in :gh:`121027`.) + +* The :ref:`garbage collector is now incremental `, + which means that the behavior of :func:`gc.collect` changes slightly: + + * ``gc.collect(1)``: Performs an increment of garbage collection, + rather than collecting generation 1. + * Other calls to :func:`!gc.collect` are unchanged. + +* The :func:`locale.nl_langinfo` function now temporarily sets the ``LC_CTYPE`` + locale in some cases. + This temporary change affects other threads. + (Contributed by Serhiy Storchaka in :gh:`69998`.) + +* :class:`types.UnionType` is now an alias for :class:`typing.Union`, + causing changes in some behaviors. + See :ref:`above ` for more details. + (Contributed by Jelle Zijlstra in :gh:`105499`.) + +* The runtime behavior of annotations has changed in various ways; see + :ref:`above ` for details. While most code that interacts + with annotations should continue to work, some undocumented details may behave + differently. + + +Changes in the C API +-------------------- + +* :c:func:`Py_Finalize` now deletes all interned strings. This + is backwards incompatible to any C extension that holds onto an interned + string after a call to :c:func:`Py_Finalize` and is then reused after a + call to :c:func:`Py_Initialize`. Any issues arising from this behavior will + normally result in crashes during the execution of the subsequent call to + :c:func:`Py_Initialize` from accessing uninitialized memory. To fix, use + an address sanitizer to identify any use-after-free coming from + an interned string and deallocate it during module shutdown. + (Contributed by Eddie Elizondo in :gh:`113601`.) + +* The :ref:`Unicode Exception Objects ` C API + now raises a :exc:`TypeError` if its exception argument is not + a :exc:`UnicodeError` object. + (Contributed by Bénédikt Tran in :gh:`127691`.) + +.. _whatsnew314-refcount: + +* The interpreter internally avoids some reference count modifications when + loading objects onto the operands stack by :term:`borrowing ` + references when possible. This can lead to smaller reference count values + compared to previous Python versions. C API extensions that checked + :c:func:`Py_REFCNT` of ``1`` to determine if an function argument is not + referenced by any other code should instead use + :c:func:`PyUnstable_Object_IsUniqueReferencedTemporary` as a safer replacement. + + +* Private functions promoted to public C APIs: + + * ``_PyBytes_Join()``: :c:func:`PyBytes_Join` + * ``_PyLong_IsNegative()``: :c:func:`PyLong_IsNegative` + * ``_PyLong_IsPositive()``: :c:func:`PyLong_IsPositive` + * ``_PyLong_IsZero()``: :c:func:`PyLong_IsZero` + * ``_PyLong_Sign()``: :c:func:`PyLong_GetSign` + * ``_PyUnicodeWriter_Dealloc()``: :c:func:`PyUnicodeWriter_Discard` + * ``_PyUnicodeWriter_Finish()``: :c:func:`PyUnicodeWriter_Finish` + * ``_PyUnicodeWriter_Init()``: use :c:func:`PyUnicodeWriter_Create` + * ``_PyUnicodeWriter_Prepare()``: (no replacement) + * ``_PyUnicodeWriter_PrepareKind()``: (no replacement) + * ``_PyUnicodeWriter_WriteChar()``: :c:func:`PyUnicodeWriter_WriteChar` + * ``_PyUnicodeWriter_WriteStr()``: :c:func:`PyUnicodeWriter_WriteStr` + * ``_PyUnicodeWriter_WriteSubstring()``: :c:func:`PyUnicodeWriter_WriteSubstring` + * ``_PyUnicode_EQ()``: :c:func:`PyUnicode_Equal` + * ``_PyUnicode_Equal()``: :c:func:`PyUnicode_Equal` + * ``_Py_GetConfig()``: :c:func:`PyConfig_Get` and :c:func:`PyConfig_GetInt` + * ``_Py_HashBytes()``: :c:func:`Py_HashBuffer` + * ``_Py_fopen_obj()``: :c:func:`Py_fopen` + * ``PyMutex_IsLocked()`` : :c:func:`PyMutex_IsLocked` + + The `pythoncapi-compat project`_ can be used to get most of these new + functions on Python 3.13 and older. + +.. _pythoncapi-compat project: https://github.com/python/pythoncapi-compat/ From 14e25177c685b83d71f5650769b243c98578ba78 Mon Sep 17 00:00:00 2001 From: Adam Turner <9087854+AA-Turner@users.noreply.github.com> Date: Wed, 3 Sep 2025 12:20:43 +0100 Subject: [PATCH 09/10] GH-101100: Resolve reference warnings in whatsnew/3.3.rst (#138414) --- Doc/library/curses.rst | 6 ++-- Doc/tools/.nitignore | 1 - Doc/whatsnew/3.3.rst | 77 +++++++++++++++++++++--------------------- 3 files changed, 42 insertions(+), 42 deletions(-) diff --git a/Doc/library/curses.rst b/Doc/library/curses.rst index fab54ca87efee9..fb84cf32246879 100644 --- a/Doc/library/curses.rst +++ b/Doc/library/curses.rst @@ -716,8 +716,10 @@ The module :mod:`curses` defines the following functions: Window Objects -------------- -Window objects, as returned by :func:`initscr` and :func:`newwin` above, have -the following methods and attributes: +.. class:: window + + Window objects, as returned by :func:`initscr` and :func:`newwin` above, have + the following methods and attributes: .. method:: window.addch(ch[, attr]) diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index 07671fe17c1d89..bcf756a481858c 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -62,7 +62,6 @@ Doc/whatsnew/2.4.rst Doc/whatsnew/2.5.rst Doc/whatsnew/2.6.rst Doc/whatsnew/2.7.rst -Doc/whatsnew/3.3.rst Doc/whatsnew/3.4.rst Doc/whatsnew/3.5.rst Doc/whatsnew/3.6.rst diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst index 89fd68686454e2..206ab2c90a12c4 100644 --- a/Doc/whatsnew/3.3.rst +++ b/Doc/whatsnew/3.3.rst @@ -331,7 +331,7 @@ simplified and finer-grained. You don't have to worry anymore about choosing the appropriate exception type between :exc:`OSError`, :exc:`IOError`, :exc:`EnvironmentError`, -:exc:`WindowsError`, :exc:`mmap.error`, :exc:`socket.error` or +:exc:`WindowsError`, :exc:`!mmap.error`, :exc:`socket.error` or :exc:`select.error`. All these exception types are now only one: :exc:`OSError`. The other names are kept as aliases for compatibility reasons. @@ -805,7 +805,7 @@ Some smaller changes made to the core Python language are: * New methods have been added to :class:`list` and :class:`bytearray`: ``copy()`` and ``clear()`` (:issue:`10516`). Consequently, :class:`~collections.abc.MutableSequence` now also defines a - :meth:`~collections.abc.MutableSequence.clear` method (:issue:`11388`). + :meth:`!clear` method (:issue:`11388`). * Raw bytes literals can now be written ``rb"..."`` as well as ``br"..."``. @@ -869,10 +869,10 @@ faulthandler This new debug module :mod:`faulthandler` contains functions to dump Python tracebacks explicitly, on a fault (a crash like a segmentation fault), after a timeout, or on a user signal. Call :func:`faulthandler.enable` to install fault handlers for the -:const:`SIGSEGV`, :const:`SIGFPE`, :const:`SIGABRT`, :const:`SIGBUS`, and -:const:`SIGILL` signals. You can also enable them at startup by setting the -:envvar:`PYTHONFAULTHANDLER` environment variable or by using :option:`-X` -``faulthandler`` command line option. +:const:`~signal.SIGSEGV`, :const:`~signal.SIGFPE`, :const:`~signal.SIGABRT`, +:const:`~signal.SIGBUS`, and :const:`~signal.SIGILL` signals. +You can also enable them at startup by setting the :envvar:`PYTHONFAULTHANDLER` +environment variable or by using :option:`-X` ``faulthandler`` command line option. Example of a segmentation fault on Linux: @@ -916,7 +916,7 @@ abc Improved support for abstract base classes containing descriptors composed with abstract methods. The recommended approach to declaring abstract descriptors is -now to provide :attr:`__isabstractmethod__` as a dynamically updated +now to provide :attr:`!__isabstractmethod__` as a dynamically updated property. The built-in descriptors have been updated accordingly. * :class:`abc.abstractproperty` has been deprecated, use :class:`property` @@ -979,7 +979,7 @@ new features have been added: (Contributed by Nir Aides in :issue:`1625`.) * :class:`bz2.BZ2File` now implements all of the :class:`io.BufferedIOBase` API, - except for the :meth:`detach` and :meth:`truncate` methods. + except for the :meth:`!detach` and :meth:`!truncate` methods. codecs @@ -1064,7 +1064,7 @@ curses * If the :mod:`curses` module is linked to the ncursesw library, use Unicode functions when Unicode strings or characters are passed (e.g. - :c:func:`waddwstr`), and bytes functions otherwise (e.g. :c:func:`waddstr`). + :c:func:`!waddwstr`), and bytes functions otherwise (e.g. :c:func:`!waddstr`). * Use the locale encoding instead of ``utf-8`` to encode Unicode strings. * :class:`curses.window` has a new :attr:`curses.window.encoding` attribute. * The :class:`curses.window` class has a new :meth:`~curses.window.get_wch` @@ -1137,15 +1137,15 @@ API changes * The C module has the following context limits, depending on the machine architecture: - +-------------------+----------------+-------------------------+ - | | 32-bit | 64-bit | - +===================+================+=========================+ - | :const:`MAX_PREC` | ``425000000`` | ``999999999999999999`` | - +-------------------+----------------+-------------------------+ - | :const:`MAX_EMAX` | ``425000000`` | ``999999999999999999`` | - +-------------------+----------------+-------------------------+ - | :const:`MIN_EMIN` | ``-425000000`` | ``-999999999999999999`` | - +-------------------+----------------+-------------------------+ + +----------------------------+----------------+-------------------------+ + | | 32-bit | 64-bit | + +============================+================+=========================+ + | :const:`~decimal.MAX_PREC` | ``425000000`` | ``999999999999999999`` | + +----------------------------+----------------+-------------------------+ + | :const:`~decimal.MAX_EMAX` | ``425000000`` | ``999999999999999999`` | + +----------------------------+----------------+-------------------------+ + | :const:`~decimal.MIN_EMIN` | ``-425000000`` | ``-999999999999999999`` | + +----------------------------+----------------+-------------------------+ * In the context templates (:const:`~decimal.DefaultContext`, :const:`~decimal.BasicContext` and :const:`~decimal.ExtendedContext`) @@ -1434,7 +1434,7 @@ html :class:`html.parser.HTMLParser` is now able to parse broken markup without raising errors, therefore the *strict* argument of the constructor and the -:exc:`~html.parser.HTMLParseError` exception are now deprecated. +:exc:`!HTMLParseError` exception are now deprecated. The ability to parse broken markup is the result of a number of bug fixes that are also available on the latest bug fix releases of Python 2.7/3.2. (Contributed by Ezio Melotti in :issue:`15114`, and :issue:`14538`, @@ -1486,7 +1486,7 @@ already exists. It is based on the C11 'x' mode to fopen(). The constructor of the :class:`~io.TextIOWrapper` class has a new *write_through* optional argument. If *write_through* is ``True``, calls to -:meth:`~io.TextIOWrapper.write` are guaranteed not to be buffered: any data +:meth:`!write` are guaranteed not to be buffered: any data written on the :class:`~io.TextIOWrapper` object is immediately handled to its underlying binary buffer. @@ -1504,7 +1504,7 @@ logging The :func:`~logging.basicConfig` function now supports an optional ``handlers`` argument taking an iterable of handlers to be added to the root logger. -A class level attribute :attr:`~logging.handlers.SysLogHandler.append_nul` has +A class level attribute :attr:`!append_nul` has been added to :class:`~logging.handlers.SysLogHandler` to allow control of the appending of the ``NUL`` (``\000``) byte to syslog records, since for some daemons it is required while for others it is passed through to the log. @@ -1536,8 +1536,8 @@ The new :func:`multiprocessing.connection.wait` function allows polling multiple objects (such as connections, sockets and pipes) with a timeout. (Contributed by Richard Oudkerk in :issue:`12328`.) -:class:`multiprocessing.Connection` objects can now be transferred over -multiprocessing connections. +:class:`multiprocessing.connection.Connection` objects can now be transferred +over multiprocessing connections. (Contributed by Richard Oudkerk in :issue:`4892`.) :class:`multiprocessing.Process` now accepts a ``daemon`` keyword argument @@ -1611,7 +1611,7 @@ os :func:`~os.rename`, :func:`~os.replace`, :func:`~os.rmdir`, :func:`~os.stat`, :func:`~os.symlink`, :func:`~os.unlink`, :func:`~os.utime`. Platform support for using these parameters can be checked via the sets - :data:`os.supports_dir_fd` and :data:`os.supports_follows_symlinks`. + :data:`os.supports_dir_fd` and :data:`os.supports_follow_symlinks`. - The following functions now support a file descriptor for their path argument: :func:`~os.chdir`, :func:`~os.chmod`, :func:`~os.chown`, @@ -1698,7 +1698,7 @@ os :const:`~os.RTLD_NOLOAD`, and :const:`~os.RTLD_DEEPBIND` are available on platforms that support them. These are for use with the :func:`sys.setdlopenflags` function, and supersede the similar constants - defined in :mod:`ctypes` and :mod:`DLFCN`. (Contributed by Victor Stinner + defined in :mod:`ctypes` and :mod:`!DLFCN`. (Contributed by Victor Stinner in :issue:`13226`.) * :func:`os.symlink` now accepts (and ignores) the ``target_is_directory`` @@ -1728,8 +1728,8 @@ reduction functions to be set. pydoc ----- -The Tk GUI and the :func:`~pydoc.serve` function have been removed from the -:mod:`pydoc` module: ``pydoc -g`` and :func:`~pydoc.serve` have been deprecated +The Tk GUI and the :func:`!serve` function have been removed from the +:mod:`pydoc` module: ``pydoc -g`` and :func:`!serve` have been deprecated in Python 3.2. @@ -1931,7 +1931,7 @@ ssl * :func:`~ssl.RAND_bytes`: generate cryptographically strong pseudo-random bytes. - * :func:`~ssl.RAND_pseudo_bytes`: generate pseudo-random bytes. + * :func:`!RAND_pseudo_bytes`: generate pseudo-random bytes. (Contributed by Victor Stinner in :issue:`12049`.) @@ -2020,8 +2020,7 @@ tarfile tempfile -------- -:class:`tempfile.SpooledTemporaryFile`\'s -:meth:`~tempfile.SpooledTemporaryFile.truncate` method now accepts +:class:`tempfile.SpooledTemporaryFile`\'s :meth:`!truncate` method now accepts a ``size`` parameter. (Contributed by Ryan Kelly in :issue:`9957`.) @@ -2129,7 +2128,7 @@ xml.etree.ElementTree The :mod:`xml.etree.ElementTree` module now imports its C accelerator by default; there is no longer a need to explicitly import -:mod:`xml.etree.cElementTree` (this module stays for backwards compatibility, +:mod:`!xml.etree.cElementTree` (this module stays for backwards compatibility, but is now deprecated). In addition, the ``iter`` family of methods of :class:`~xml.etree.ElementTree.Element` has been optimized (rewritten in C). The module's documentation has also been greatly improved with added examples @@ -2197,7 +2196,7 @@ Changes to Python's build process and to the C API include: * :c:func:`PyUnicode_AsUCS4`, :c:func:`PyUnicode_AsUCS4Copy` * :c:macro:`PyUnicode_DATA`, :c:macro:`PyUnicode_1BYTE_DATA`, :c:macro:`PyUnicode_2BYTE_DATA`, :c:macro:`PyUnicode_4BYTE_DATA` - * :c:macro:`PyUnicode_KIND` with :c:enum:`PyUnicode_Kind` enum: + * :c:macro:`PyUnicode_KIND` with :c:enum:`!PyUnicode_Kind` enum: :c:data:`!PyUnicode_WCHAR_KIND`, :c:data:`PyUnicode_1BYTE_KIND`, :c:data:`PyUnicode_2BYTE_KIND`, :c:data:`PyUnicode_4BYTE_KIND` * :c:macro:`PyUnicode_READ`, :c:macro:`PyUnicode_READ_CHAR`, :c:macro:`PyUnicode_WRITE` @@ -2232,17 +2231,17 @@ Deprecated Python modules, functions and methods (``utf-32-le`` or ``utf-32-be``) * :meth:`ftplib.FTP.nlst` and :meth:`ftplib.FTP.dir`: use :meth:`ftplib.FTP.mlsd` -* :func:`platform.popen`: use the :mod:`subprocess` module. Check especially +* :func:`!platform.popen`: use the :mod:`subprocess` module. Check especially the :ref:`subprocess-replacements` section (:issue:`11377`). * :issue:`13374`: The Windows bytes API has been deprecated in the :mod:`os` module. Use Unicode filenames, instead of bytes filenames, to not depend on the ANSI code page anymore and to support any filename. -* :issue:`13988`: The :mod:`xml.etree.cElementTree` module is deprecated. The +* :issue:`13988`: The :mod:`!xml.etree.cElementTree` module is deprecated. The accelerator is used automatically whenever available. -* The behaviour of :func:`time.clock` depends on the platform: use the new +* The behaviour of :func:`!time.clock` depends on the platform: use the new :func:`time.perf_counter` or :func:`time.process_time` function instead, depending on your requirements, to have a well defined behaviour. -* The :func:`os.stat_float_times` function is deprecated. +* The :func:`!os.stat_float_times` function is deprecated. * :mod:`abc` module: * :class:`abc.abstractproperty` has been deprecated, use :class:`property` @@ -2381,8 +2380,8 @@ Porting Python code for top-level modules. E.g. ``__import__('sys', level=1)`` is now an error. * Because :data:`sys.meta_path` and :data:`sys.path_hooks` now have finders on - them by default, you will most likely want to use :meth:`list.insert` instead - of :meth:`list.append` to add to those lists. + them by default, you will most likely want to use :meth:`!list.insert` instead + of :meth:`!list.append` to add to those lists. * Because ``None`` is now inserted into :data:`sys.path_importer_cache`, if you are clearing out entries in the dictionary of paths that do not have a @@ -2466,7 +2465,7 @@ Porting C code * In the course of changes to the buffer API the undocumented :c:member:`!smalltable` member of the :c:type:`Py_buffer` structure has been removed and the - layout of the :c:type:`PyMemoryViewObject` has changed. + layout of the :c:type:`!PyMemoryViewObject` has changed. All extensions relying on the relevant parts in ``memoryobject.h`` or ``object.h`` must be rebuilt. From 8c1ae6de111a55b131552d126eb70c8d6a4d1739 Mon Sep 17 00:00:00 2001 From: Adam Turner <9087854+AA-Turner@users.noreply.github.com> Date: Wed, 3 Sep 2025 12:24:42 +0100 Subject: [PATCH 10/10] GH-101100: Resolve reference warnings in whatsnew/3.8.rst (#138409) Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> --- Doc/library/unittest.rst | 4 ++ Doc/tools/.nitignore | 1 - Doc/whatsnew/3.8.rst | 93 +++++++++++++++++++++------------------- 3 files changed, 52 insertions(+), 46 deletions(-) diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst index ec96e8416120fa..b5331e0676d52a 100644 --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -2532,6 +2532,10 @@ instead of as an error. setUpModule and tearDownModule ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. function:: setUpModule + tearDownModule + :no-typesetting: + These should be implemented as functions:: def setUpModule(): diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index bcf756a481858c..5443bc99c9d300 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -65,5 +65,4 @@ Doc/whatsnew/2.7.rst Doc/whatsnew/3.4.rst Doc/whatsnew/3.5.rst Doc/whatsnew/3.6.rst -Doc/whatsnew/3.8.rst Doc/whatsnew/3.10.rst diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index bc2eb1d0e263f0..545a17aecab8ee 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -536,7 +536,7 @@ Other Language Changes element is a callable with a ``(obj, state)`` signature. This allows the direct control over the state-updating behavior of a specific object. If not *None*, this callable will have priority over the object's - :meth:`~__setstate__` method. + :meth:`~object.__setstate__` method. (Contributed by Pierre Glaser and Olivier Grisel in :issue:`35900`.) New Modules @@ -1035,9 +1035,9 @@ symlinks and directory junctions) has been delegated to the operating system. Specifically, :func:`os.stat` will now traverse anything supported by the operating system, while :func:`os.lstat` will only open reparse points that identify as "name surrogates" while others are opened as for :func:`os.stat`. -In all cases, :attr:`stat_result.st_mode` will only have ``S_IFLNK`` set for +In all cases, :attr:`os.stat_result.st_mode` will only have ``S_IFLNK`` set for symbolic links and not other kinds of reparse points. To identify other kinds -of reparse point, check the new :attr:`stat_result.st_reparse_tag` attribute. +of reparse point, check the new :attr:`os.stat_result.st_reparse_tag` attribute. On Windows, :func:`os.readlink` is now able to read directory junctions. Note that :func:`~os.path.islink` will return ``False`` for directory junctions, @@ -1285,20 +1285,20 @@ now matches what the C tokenizer does internally. tkinter ------- -Added methods :meth:`~tkinter.Spinbox.selection_from`, -:meth:`~tkinter.Spinbox.selection_present`, -:meth:`~tkinter.Spinbox.selection_range` and -:meth:`~tkinter.Spinbox.selection_to` -in the :class:`tkinter.Spinbox` class. +Added methods :meth:`!selection_from`, +:meth:`!selection_present`, +:meth:`!selection_range` and +:meth:`!selection_to` +in the :class:`!tkinter.Spinbox` class. (Contributed by Juliette Monsel in :issue:`34829`.) -Added method :meth:`~tkinter.Canvas.moveto` -in the :class:`tkinter.Canvas` class. +Added method :meth:`!moveto` +in the :class:`!tkinter.Canvas` class. (Contributed by Juliette Monsel in :issue:`23831`.) -The :class:`tkinter.PhotoImage` class now has -:meth:`~tkinter.PhotoImage.transparency_get` and -:meth:`~tkinter.PhotoImage.transparency_set` methods. (Contributed by +The :class:`!tkinter.PhotoImage` class now has +:meth:`!transparency_get` and +:meth:`!transparency_set` methods. (Contributed by Zackery Spytz in :issue:`25451`.) @@ -1432,7 +1432,7 @@ and ``{namespace}*`` which returns all tags in the given namespace. (Contributed by Stefan Behnel in :issue:`28238`.) The :mod:`xml.etree.ElementTree` module provides a new function -:func:`–xml.etree.ElementTree.canonicalize` that implements C14N 2.0. +:func:`~xml.etree.ElementTree.canonicalize` that implements C14N 2.0. (Contributed by Stefan Behnel in :issue:`13611`.) The target object of :class:`xml.etree.ElementTree.XMLParser` can @@ -1573,7 +1573,7 @@ Build and C API Changes * :c:func:`Py_INCREF`, :c:func:`Py_DECREF` * :c:func:`Py_XINCREF`, :c:func:`Py_XDECREF` - * :c:func:`PyObject_INIT`, :c:func:`PyObject_INIT_VAR` + * :c:macro:`!PyObject_INIT`, :c:macro:`!PyObject_INIT_VAR` * Private functions: :c:func:`!_PyObject_GC_TRACK`, :c:func:`!_PyObject_GC_UNTRACK`, :c:func:`!_Py_Dealloc` @@ -1677,7 +1677,7 @@ Deprecated constant nodes. (Contributed by Serhiy Storchaka in :issue:`36917`.) -* The :func:`asyncio.coroutine` :term:`decorator` is deprecated and will be +* The :deco:`!asyncio.coroutine` :term:`decorator` is deprecated and will be removed in version 3.10. Instead of ``@asyncio.coroutine``, use :keyword:`async def` instead. (Contributed by Andrew Svetlov in :issue:`36921`.) @@ -1697,22 +1697,22 @@ Deprecated (Contributed by Yury Selivanov in :issue:`34790`.) * The following functions and methods are deprecated in the :mod:`gettext` - module: :func:`~gettext.lgettext`, :func:`~gettext.ldgettext`, - :func:`~gettext.lngettext` and :func:`~gettext.ldngettext`. + module: :func:`!lgettext`, :func:`!ldgettext`, + :func:`!lngettext` and :func:`!ldngettext`. They return encoded bytes, and it's possible that you will get unexpected Unicode-related exceptions if there are encoding problems with the translated strings. It's much better to use alternatives which return Unicode strings in Python 3. These functions have been broken for a long time. - Function :func:`~gettext.bind_textdomain_codeset`, methods - :meth:`~gettext.NullTranslations.output_charset` and - :meth:`~gettext.NullTranslations.set_output_charset`, and the *codeset* + Function :func:`!bind_textdomain_codeset`, methods + :meth:`!NullTranslations.output_charset` and + :meth:`!NullTranslations.set_output_charset`, and the *codeset* parameter of functions :func:`~gettext.translation` and :func:`~gettext.install` are also deprecated, since they are only used for the ``l*gettext()`` functions. (Contributed by Serhiy Storchaka in :issue:`33710`.) -* The :meth:`~threading.Thread.isAlive` method of :class:`threading.Thread` +* The :meth:`!isAlive` method of :class:`threading.Thread` has been deprecated. (Contributed by Donghee Na in :issue:`35283`.) @@ -1727,7 +1727,7 @@ Deprecated * Deprecated passing the following arguments as keyword arguments: - *func* in :func:`functools.partialmethod`, :func:`weakref.finalize`, - :meth:`profile.Profile.runcall`, :meth:`cProfile.Profile.runcall`, + :meth:`profile.Profile.runcall`, :meth:`!cProfile.Profile.runcall`, :meth:`bdb.Bdb.runcall`, :meth:`trace.Trace.runfunc` and :func:`curses.wrapper`. - *function* in :meth:`unittest.TestCase.addCleanup`. @@ -1735,11 +1735,11 @@ Deprecated :class:`concurrent.futures.ThreadPoolExecutor` and :class:`concurrent.futures.ProcessPoolExecutor`. - *callback* in :meth:`contextlib.ExitStack.callback`, - :meth:`contextlib.AsyncExitStack.callback` and + :meth:`!contextlib.AsyncExitStack.callback` and :meth:`contextlib.AsyncExitStack.push_async_callback`. - - *c* and *typeid* in the :meth:`~multiprocessing.managers.Server.create` - method of :class:`multiprocessing.managers.Server` and - :class:`multiprocessing.managers.SharedMemoryServer`. + - *c* and *typeid* in the :meth:`!create` + method of :class:`!multiprocessing.managers.Server` and + :class:`!multiprocessing.managers.SharedMemoryServer`. - *obj* in :func:`weakref.finalize`. In future releases of Python, they will be :ref:`positional-only @@ -1757,14 +1757,14 @@ The following features and APIs have been removed from Python 3.8: able to import from collections was marked for removal in 3.8, but has been delayed to 3.9. (See :gh:`81134`.) -* The :mod:`macpath` module, deprecated in Python 3.7, has been removed. +* The :mod:`!macpath` module, deprecated in Python 3.7, has been removed. (Contributed by Victor Stinner in :issue:`35471`.) -* The function :func:`platform.popen` has been removed, after having been +* The function :func:`!platform.popen` has been removed, after having been deprecated since Python 3.3: use :func:`os.popen` instead. (Contributed by Victor Stinner in :issue:`35345`.) -* The function :func:`time.clock` has been removed, after having been +* The function :func:`!time.clock` has been removed, after having been deprecated since Python 3.3: use :func:`time.perf_counter` or :func:`time.process_time` instead, depending on your requirements, to have well-defined behavior. @@ -1800,8 +1800,8 @@ The following features and APIs have been removed from Python 3.8: :func:`fileinput.FileInput` which was ignored and deprecated since Python 3.6 has been removed. :issue:`36952` (Contributed by Matthias Bussonnier.) -* The functions :func:`sys.set_coroutine_wrapper` and - :func:`sys.get_coroutine_wrapper` deprecated in Python 3.7 have been removed; +* The functions :func:`!sys.set_coroutine_wrapper` and + :func:`!sys.get_coroutine_wrapper` deprecated in Python 3.7 have been removed; :issue:`36933` (Contributed by Matthias Bussonnier.) @@ -1864,9 +1864,10 @@ Changes in the Python API * :class:`subprocess.Popen` can now use :func:`os.posix_spawn` in some cases for better performance. On Windows Subsystem for Linux and QEMU User - Emulation, the :class:`Popen` constructor using :func:`os.posix_spawn` no longer raises an - exception on errors like "missing program". Instead the child process fails with a - non-zero :attr:`~Popen.returncode`. + Emulation, the :class:`~subprocess.Popen` constructor using + :func:`os.posix_spawn` no longer raises an exception on errors like + "missing program". Instead the child process fails with a + non-zero :attr:`~subprocess.Popen.returncode`. (Contributed by Joannah Nanjekye and Victor Stinner in :issue:`35537`.) * The *preexec_fn* argument of * :class:`subprocess.Popen` is no longer @@ -1875,11 +1876,11 @@ Changes in the Python API (Contributed by Eric Snow in :issue:`34651`, modified by Christian Heimes in :issue:`37951`.) -* The :meth:`imap.IMAP4.logout` method no longer silently ignores arbitrary +* The :meth:`imaplib.IMAP4.logout` method no longer silently ignores arbitrary exceptions. (Contributed by Victor Stinner in :issue:`36348`.) -* The function :func:`platform.popen` has been removed, after having been deprecated since +* The function :func:`!platform.popen` has been removed, after having been deprecated since Python 3.3: use :func:`os.popen` instead. (Contributed by Victor Stinner in :issue:`35345`.) @@ -1894,9 +1895,11 @@ Changes in the Python API specialized methods like :meth:`~tkinter.ttk.Treeview.selection_set` for changing the selection. (Contributed by Serhiy Storchaka in :issue:`31508`.) -* The :meth:`writexml`, :meth:`toxml` and :meth:`toprettyxml` methods of - :mod:`xml.dom.minidom`, and the :meth:`write` method of :mod:`xml.etree`, - now preserve the attribute order specified by the user. +* The :meth:`~xml.dom.minidom.Node.writexml`, :meth:`~xml.dom.minidom.Node.toxml` + and :meth:`~xml.dom.minidom.Node.toprettyxml` methods of + :mod:`xml.dom.minidom` and the :meth:`~xml.etree.ElementTree.ElementTree.write` + method of :mod:`xml.etree.ElementTree` now preserve the attribute order + specified by the user. (Contributed by Diego Rojas and Raymond Hettinger in :issue:`34160`.) * A :mod:`dbm.dumb` database opened with flags ``'r'`` is now read-only. @@ -1916,8 +1919,8 @@ Changes in the Python API ``type.__new__``. A :exc:`DeprecationWarning` was emitted in Python 3.6--3.7. (Contributed by Serhiy Storchaka in :issue:`23722`.) -* The :class:`cProfile.Profile` class can now be used as a context - manager. (Contributed by Scott Sanderson in :issue:`29235`.) +* The :class:`cProfile.Profile ` class can now be used as + a context manager. (Contributed by Scott Sanderson in :issue:`29235`.) * :func:`shutil.copyfile`, :func:`shutil.copy`, :func:`shutil.copy2`, :func:`shutil.copytree` and :func:`shutil.move` use platform-specific @@ -1952,7 +1955,7 @@ Changes in the Python API (Contributed by Christian Heimes in :issue:`17239`.) * Deleting a key from a read-only :mod:`dbm` database (:mod:`dbm.dumb`, - :mod:`dbm.gnu` or :mod:`dbm.ndbm`) raises :attr:`error` (:exc:`dbm.dumb.error`, + :mod:`dbm.gnu` or :mod:`dbm.ndbm`) raises :attr:`!error` (:exc:`dbm.dumb.error`, :exc:`dbm.gnu.error` or :exc:`dbm.ndbm.error`) instead of :exc:`KeyError`. (Contributed by Xiang Zhang in :issue:`33106`.) @@ -2044,7 +2047,7 @@ Changes in the C API :c:func:`PyType_FromSpec`) hold a reference to their type object. Increasing the reference count of these type objects has been moved from :c:func:`PyType_GenericAlloc` to the more low-level functions, - :c:func:`PyObject_Init` and :c:func:`PyObject_INIT`. + :c:func:`PyObject_Init` and :c:macro:`!PyObject_INIT`. This makes types created through :c:func:`PyType_FromSpec` behave like other classes in managed code. @@ -2064,7 +2067,7 @@ Changes in the C API This may happen after calling :c:macro:`PyObject_New`, :c:macro:`PyObject_NewVar`, :c:func:`PyObject_GC_New`, :c:func:`PyObject_GC_NewVar`, or any other custom allocator that uses - :c:func:`PyObject_Init` or :c:func:`PyObject_INIT`. + :c:func:`PyObject_Init` or :c:macro:`!PyObject_INIT`. Example: