diff --git a/Doc/library/profile.rst b/Doc/library/profile.rst index 0d96232658bddf..faf8079db3ddd8 100644 --- a/Doc/library/profile.rst +++ b/Doc/library/profile.rst @@ -28,7 +28,7 @@ The Python standard library provides three different profiling implementations: **Statistical Profiler:** -1. :mod:`profile.sample` provides statistical profiling of running Python processes +1. :mod:`!profiling.sampling` provides statistical profiling of running Python processes using periodic stack sampling. It can attach to any running Python process without requiring code modification or restart, making it ideal for production debugging. @@ -55,26 +55,26 @@ The Python standard library provides three different profiling implementations: **Profiler Comparison:** -+-------------------+----------------------+----------------------+----------------------+ -| Feature | Statistical | Deterministic | Deterministic | -| | (``profile.sample``) | (``cProfile``) | (``profile``) | -+===================+======================+======================+======================+ -| **Target** | Running process | Code you run | Code you run | -+-------------------+----------------------+----------------------+----------------------+ -| **Overhead** | Virtually none | Moderate | High | -+-------------------+----------------------+----------------------+----------------------+ -| **Accuracy** | Statistical approx. | Exact call counts | Exact call counts | -+-------------------+----------------------+----------------------+----------------------+ -| **Setup** | Attach to any PID | Instrument code | Instrument code | -+-------------------+----------------------+----------------------+----------------------+ -| **Use Case** | Production debugging | Development/testing | Profiler extension | -+-------------------+----------------------+----------------------+----------------------+ -| **Implementation**| C extension | C extension | Pure Python | -+-------------------+----------------------+----------------------+----------------------+ ++-------------------+--------------------------+----------------------+----------------------+ +| Feature | Statistical | Deterministic | Deterministic | +| | (``profiling.sampling``) | (``cProfile``) | (``profile``) | ++===================+==========================+======================+======================+ +| **Target** | Running process | Code you run | Code you run | ++-------------------+--------------------------+----------------------+----------------------+ +| **Overhead** | Virtually none | Moderate | High | ++-------------------+--------------------------+----------------------+----------------------+ +| **Accuracy** | Statistical approx. | Exact call counts | Exact call counts | ++-------------------+--------------------------+----------------------+----------------------+ +| **Setup** | Attach to any PID | Instrument code | Instrument code | ++-------------------+--------------------------+----------------------+----------------------+ +| **Use Case** | Production debugging | Development/testing | Profiler extension | ++-------------------+--------------------------+----------------------+----------------------+ +| **Implementation**| C extension | C extension | Pure Python | ++-------------------+--------------------------+----------------------+----------------------+ .. note:: - The statistical profiler (:mod:`profile.sample`) is recommended for most production + The statistical profiler (:mod:`!profiling.sampling`) is recommended for most production use cases due to its extremely low overhead and ability to profile running processes without modification. It can attach to any Python process and collect performance data with minimal impact on execution speed, making it ideal for debugging @@ -138,11 +138,11 @@ on an existing application. To profile an existing running process:: - python -m profile.sample 1234 + python -m profiling.sampling 1234 To profile with custom settings:: - python -m profile.sample -i 50 -d 30 1234 + python -m profiling.sampling -i 50 -d 30 1234 **Deterministic Profiling (Development/Testing):** @@ -218,34 +218,34 @@ them in various ways. Statistical Profiler Command Line Interface =========================================== -.. program:: profile.sample +.. program:: profiling.sampling -The :mod:`profile.sample` module can be invoked as a script to profile running processes:: +The :mod:`!profiling.sampling` module can be invoked as a script to profile running processes:: - python -m profile.sample [options] PID + python -m profiling.sampling [options] PID **Basic Usage Examples:** Profile process 1234 for 10 seconds with default settings:: - python -m profile.sample 1234 + python -m profiling.sampling 1234 Profile with custom interval and duration, save to file:: - python -m profile.sample -i 50 -d 30 -o profile.stats 1234 + python -m profiling.sampling -i 50 -d 30 -o profile.stats 1234 Generate collapsed stacks to use with tools like `flamegraph.pl `_:: - python -m profile.sample --collapsed 1234 + python -m profiling.sampling --collapsed 1234 Profile all threads, sort by total time:: - python -m profile.sample -a --sort-tottime 1234 + python -m profiling.sampling -a --sort-tottime 1234 Profile with real-time sampling statistics:: - python -m profile.sample --realtime-stats 1234 + python -m profiling.sampling --realtime-stats 1234 **Command Line Options:** @@ -339,13 +339,13 @@ The statistical profiler produces output similar to deterministic profilers but .. _profile-cli: -:mod:`profile.sample` Module Reference +:mod:`!profiling.sampling` Module Reference ======================================================= -.. module:: profile.sample +.. module:: profiling.sampling :synopsis: Python statistical profiler. -This section documents the programmatic interface for the :mod:`profile.sample` module. +This section documents the programmatic interface for the :mod:`!profiling.sampling` module. For command-line usage, see :ref:`sampling-profiler-cli`. For conceptual information about statistical profiling, see :ref:`statistical-profiling` @@ -373,14 +373,14 @@ about statistical profiling, see :ref:`statistical-profiling` Examples:: # Basic usage - profile process 1234 for 10 seconds - import profile.sample - profile.sample.sample(1234) + import profiling.sampling + profiling.sampling.sample(1234) # Profile with custom settings - profile.sample.sample(1234, duration_sec=30, sample_interval_usec=50, all_threads=True) + profiling.sampling.sample(1234, duration_sec=30, sample_interval_usec=50, all_threads=True) # Generate collapsed stack traces for flamegraph.pl - profile.sample.sample(1234, output_format='collapsed', filename='profile.collapsed') + profiling.sampling.sample(1234, output_format='collapsed', filename='profile.collapsed') .. class:: SampleProfiler(pid, sample_interval_usec, all_threads) @@ -856,7 +856,7 @@ What Is Deterministic Profiling? call*, *function return*, and *exception* events are monitored, and precise timings are made for the intervals between these events (during which time the user's code is executing). In contrast, :dfn:`statistical profiling` (which is -provided by the :mod:`profile.sample` module) periodically samples the effective instruction pointer, and +provided by the :mod:`!profiling.sampling` module) periodically samples the effective instruction pointer, and deduces where time is being spent. The latter technique traditionally involves less overhead (as the code does not need to be instrumented), but provides only relative indications of where time is being spent. diff --git a/Doc/library/winreg.rst b/Doc/library/winreg.rst index b3a824fb69a49f..61056d41cf7a69 100644 --- a/Doc/library/winreg.rst +++ b/Doc/library/winreg.rst @@ -173,6 +173,24 @@ This module offers the following functions: See :ref:`above `. +.. function:: DeleteTree(key, sub_key=None) + + Deletes the specified key and all its subkeys and values recursively. + + *key* is an already open key, or one of the predefined + :ref:`HKEY_* constants `. + + *sub_key* is a string that names the subkey to delete. If ``None``, + deletes all subkeys and values of the specified key. + + This function deletes a key and all its descendants. If *sub_key* is + ``None``, all subkeys and values of the specified key are deleted. + + .. audit-event:: winreg.DeleteTree key,sub_key winreg.DeleteTree + + .. versionadded:: next + + .. function:: DeleteValue(key, value) Removes a named value from a registry key. diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst index 4e49a49c08167a..05f05bc41a217f 100644 --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -416,12 +416,14 @@ clauses. -------------------------- If :keyword:`!finally` is present, it specifies a 'cleanup' handler. The -:keyword:`try` clause is executed, including any :keyword:`except` and -:keyword:`else` clauses. If an exception occurs in any of the clauses and is -not handled, the exception is temporarily saved. The :keyword:`!finally` clause -is executed. If there is a saved exception it is re-raised at the end of the -:keyword:`!finally` clause. If the :keyword:`!finally` clause raises another -exception, the saved exception is set as the context of the new exception. +:keyword:`try` clause is executed, including any :keyword:`except` +and :keyword:`else ` clauses. +If an exception occurs in any of the clauses and is not handled, +the exception is temporarily saved. +The :keyword:`!finally` clause is executed. If there is a saved exception +it is re-raised at the end of the :keyword:`!finally` clause. +If the :keyword:`!finally` clause raises another exception, the saved exception +is set as the context of the new exception. If the :keyword:`!finally` clause executes a :keyword:`return`, :keyword:`break` or :keyword:`continue` statement, the saved exception is discarded. For example, this function returns 42. diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst index e320eedfa67a27..6338c181813bbe 100644 --- a/Doc/reference/lexical_analysis.rst +++ b/Doc/reference/lexical_analysis.rst @@ -628,10 +628,10 @@ to indicate that an ending quote ends the literal. STRING: [`stringprefix`] (`stringcontent`) stringprefix: <("r" | "u" | "b" | "br" | "rb"), case-insensitive> stringcontent: - | "'" ( !"'" `stringitem`)* "'" - | '"' ( !'"' `stringitem`)* '"' | "'''" ( !"'''" `longstringitem`)* "'''" | '"""' ( !'"""' `longstringitem`)* '"""' + | "'" ( !"'" `stringitem`)* "'" + | '"' ( !'"' `stringitem`)* '"' stringitem: `stringchar` | `stringescapeseq` stringchar: longstringitem: `stringitem` | newline diff --git a/Doc/whatsnew/3.15.rst b/Doc/whatsnew/3.15.rst index 24c19200e035fc..b5e138aa674697 100644 --- a/Doc/whatsnew/3.15.rst +++ b/Doc/whatsnew/3.15.rst @@ -75,8 +75,8 @@ New features High frequency statistical sampling profiler -------------------------------------------- -A new statistical sampling profiler has been added to the :mod:`profile` module as -:mod:`profile.sample`. This profiler enables low-overhead performance analysis of +A new statistical sampling profiler has been added to the new :mod:`!profiling` module as +:mod:`!profiling.sampling`. This profiler enables low-overhead performance analysis of running Python processes without requiring code modification or process restart. Unlike deterministic profilers (:mod:`cProfile` and :mod:`profile`) that instrument @@ -97,19 +97,19 @@ Key features include: Profile process 1234 for 10 seconds with default settings:: - python -m profile.sample 1234 + python -m profiling.sampling 1234 Profile with custom interval and duration, save to file:: - python -m profile.sample -i 50 -d 30 -o profile.stats 1234 + python -m profiling.sampling -i 50 -d 30 -o profile.stats 1234 Generate collapsed stacks for flamegraph:: - python -m profile.sample --collapsed 1234 + python -m profiling.sampling --collapsed 1234 Profile all threads and sort by total time:: - python -m profile.sample -a --sort-tottime 1234 + python -m profiling.sampling -a --sort-tottime 1234 The profiler generates statistical estimates of where time is spent:: diff --git a/Lib/test/test_grammar.py b/Lib/test/test_grammar.py index 7f5d48b9c63ab7..cfb24a5c457820 100644 --- a/Lib/test/test_grammar.py +++ b/Lib/test/test_grammar.py @@ -1554,6 +1554,8 @@ def check(test): check('[None [i, j]]') check('[True [i, j]]') check('[... [i, j]]') + check('[t"{x}" [i, j]]') + check('[t"x={x}" [i, j]]') msg=r'indices must be integers or slices, not tuple; perhaps you missed a comma\?' check('[(1, 2) [i, j]]') @@ -1564,8 +1566,6 @@ def check(test): check('[f"x={x}" [i, j]]') check('["abc" [i, j]]') check('[b"abc" [i, j]]') - check('[t"{x}" [i, j]]') - check('[t"x={x}" [i, j]]') msg=r'indices must be integers or slices, not tuple;' check('[[1, 2] [3, 4]]') @@ -1586,6 +1586,7 @@ def check(test): check('[[1, 2] [f"{x}"]]') check('[[1, 2] [f"x={x}"]]') check('[[1, 2] ["abc"]]') + msg=r'indices must be integers or slices, not string.templatelib.Template;' check('[[1, 2] [t"{x}"]]') check('[[1, 2] [t"x={x}"]]') msg=r'indices must be integers or slices, not' diff --git a/Lib/test/test_io/test_general.py b/Lib/test/test_io/test_general.py index a56a2c0157f764..e3d7d26a7e0f9c 100644 --- a/Lib/test/test_io/test_general.py +++ b/Lib/test/test_io/test_general.py @@ -5029,12 +5029,12 @@ def write(self, b: bytes): pass def test_reader_subclass(self): - self.assertIsSubclass(MyReader, io.Reader[bytes]) - self.assertNotIsSubclass(str, io.Reader[bytes]) + self.assertIsSubclass(self.MyReader, io.Reader) + self.assertNotIsSubclass(str, io.Reader) def test_writer_subclass(self): - self.assertIsSubclass(MyWriter, io.Writer[bytes]) - self.assertNotIsSubclass(str, io.Writer[bytes]) + self.assertIsSubclass(self.MyWriter, io.Writer) + self.assertNotIsSubclass(str, io.Writer) def load_tests(loader, tests, pattern): @@ -5048,6 +5048,7 @@ def load_tests(loader, tests, pattern): CTextIOWrapperTest, PyTextIOWrapperTest, CMiscIOTest, PyMiscIOTest, CSignalsTest, PySignalsTest, TestIOCTypes, + ProtocolsTest, ) # Put the namespaces of the IO module we are testing and some useful mock diff --git a/Lib/test/test_long.py b/Lib/test/test_long.py index f336d49fa4f008..d63bc19ed9c9a2 100644 --- a/Lib/test/test_long.py +++ b/Lib/test/test_long.py @@ -1693,5 +1693,21 @@ class MyInt(int): # GH-117195 -- This shouldn't crash object.__sizeof__(1) + def test_hash(self): + # gh-136599 + self.assertEqual(hash(-1), -2) + self.assertEqual(hash(0), 0) + self.assertEqual(hash(10), 10) + + self.assertEqual(hash(sys.hash_info.modulus - 2), sys.hash_info.modulus - 2) + self.assertEqual(hash(sys.hash_info.modulus - 1), sys.hash_info.modulus - 1) + self.assertEqual(hash(sys.hash_info.modulus), 0) + self.assertEqual(hash(sys.hash_info.modulus + 1), 1) + + self.assertEqual(hash(-sys.hash_info.modulus - 2), -2) + self.assertEqual(hash(-sys.hash_info.modulus - 1), -2) + self.assertEqual(hash(-sys.hash_info.modulus), 0) + self.assertEqual(hash(-sys.hash_info.modulus + 1), -sys.hash_info.modulus + 1) + if __name__ == "__main__": unittest.main() diff --git a/Lib/test/test_mmap.py b/Lib/test/test_mmap.py index b2a299ed172967..38d1a496c30e21 100644 --- a/Lib/test/test_mmap.py +++ b/Lib/test/test_mmap.py @@ -901,35 +901,69 @@ def test_madvise(self): self.assertEqual(m.madvise(mmap.MADV_NORMAL, 0, 2), None) self.assertEqual(m.madvise(mmap.MADV_NORMAL, 0, size), None) - @unittest.skipUnless(os.name == 'nt', 'requires Windows') - def test_resize_up_when_mapped_to_pagefile(self): + def test_resize_up_anonymous_mapping(self): """If the mmap is backed by the pagefile ensure a resize up can happen and that the original data is still in place """ start_size = PAGESIZE new_size = 2 * start_size - data = bytes(random.getrandbits(8) for _ in range(start_size)) + data = random.randbytes(start_size) - m = mmap.mmap(-1, start_size) - m[:] = data - m.resize(new_size) - self.assertEqual(len(m), new_size) - self.assertEqual(m[:start_size], data[:start_size]) + with mmap.mmap(-1, start_size) as m: + m[:] = data + if sys.platform.startswith(('linux', 'android')): + # Can't expand a shared anonymous mapping on Linux. + # See https://bugzilla.kernel.org/show_bug.cgi?id=8691 + with self.assertRaises(ValueError): + m.resize(new_size) + else: + try: + m.resize(new_size) + except SystemError: + pass + else: + self.assertEqual(len(m), new_size) + self.assertEqual(m[:start_size], data) + self.assertEqual(m[start_size:], b'\0' * (new_size - start_size)) - @unittest.skipUnless(os.name == 'nt', 'requires Windows') - def test_resize_down_when_mapped_to_pagefile(self): + @unittest.skipUnless(os.name == 'posix', 'requires Posix') + def test_resize_up_private_anonymous_mapping(self): + start_size = PAGESIZE + new_size = 2 * start_size + data = random.randbytes(start_size) + + with mmap.mmap(-1, start_size, flags=mmap.MAP_PRIVATE) as m: + m[:] = data + try: + m.resize(new_size) + except SystemError: + pass + else: + self.assertEqual(len(m), new_size) + self.assertEqual(m[:start_size], data) + self.assertEqual(m[start_size:], b'\0' * (new_size - start_size)) + + def test_resize_down_anonymous_mapping(self): """If the mmap is backed by the pagefile ensure a resize down up can happen and that a truncated form of the original data is still in place """ - start_size = PAGESIZE + start_size = 2 * PAGESIZE new_size = start_size // 2 - data = bytes(random.getrandbits(8) for _ in range(start_size)) + data = random.randbytes(start_size) - m = mmap.mmap(-1, start_size) - m[:] = data - m.resize(new_size) - self.assertEqual(len(m), new_size) - self.assertEqual(m[:new_size], data[:new_size]) + with mmap.mmap(-1, start_size) as m: + m[:] = data + try: + m.resize(new_size) + except SystemError: + pass + else: + self.assertEqual(len(m), new_size) + self.assertEqual(m[:], data[:new_size]) + if sys.platform.startswith(('linux', 'android')): + # Can't expand to its original size. + with self.assertRaises(ValueError): + m.resize(start_size) @unittest.skipUnless(os.name == 'nt', 'requires Windows') def test_resize_fails_if_mapping_held_elsewhere(self): diff --git a/Lib/test/test_threading.py b/Lib/test/test_threading.py index 0ba78b9a1807d2..6e612b06281ca2 100644 --- a/Lib/test/test_threading.py +++ b/Lib/test/test_threading.py @@ -2281,6 +2281,9 @@ def test__all__(self): @unittest.skipUnless(hasattr(_thread, 'set_name'), "missing _thread.set_name") @unittest.skipUnless(hasattr(_thread, '_get_name'), "missing _thread._get_name") def test_set_name(self): + # Ensure main thread name is restored after test + self.addCleanup(_thread.set_name, _thread._get_name()) + # set_name() limit in bytes truncate = getattr(_thread, "_NAME_MAXLEN", None) limit = truncate or 100 @@ -2320,7 +2323,8 @@ def test_set_name(self): tests.append(os_helper.TESTFN_UNENCODABLE) if sys.platform.startswith("sunos"): - encoding = "utf-8" + # Use ASCII encoding on Solaris/Illumos/OpenIndiana + encoding = "ascii" else: encoding = sys.getfilesystemencoding() @@ -2336,7 +2340,7 @@ def work(): if truncate is not None: encoded = encoded[:truncate] if sys.platform.startswith("sunos"): - expected = encoded.decode("utf-8", "surrogateescape") + expected = encoded.decode("ascii", "surrogateescape") else: expected = os.fsdecode(encoded) else: @@ -2355,7 +2359,11 @@ def work(): if '\0' in expected: expected = expected.split('\0', 1)[0] - with self.subTest(name=name, expected=expected): + with self.subTest(name=name, expected=expected, thread="main"): + _thread.set_name(name) + self.assertEqual(_thread._get_name(), expected) + + with self.subTest(name=name, expected=expected, thread="worker"): work_name = None thread = threading.Thread(target=work, name=name) thread.start() diff --git a/Lib/test/test_winreg.py b/Lib/test/test_winreg.py index 924a962781a75b..6f2a6ac900be82 100644 --- a/Lib/test/test_winreg.py +++ b/Lib/test/test_winreg.py @@ -517,6 +517,21 @@ def test_exception_numbers(self): with self.assertRaises(FileNotFoundError) as ctx: QueryValue(HKEY_CLASSES_ROOT, 'some_value_that_does_not_exist') + def test_delete_tree(self): + with CreateKey(HKEY_CURRENT_USER, test_key_name) as main_key: + with CreateKey(main_key, "subkey1") as subkey1: + SetValueEx(subkey1, "value1", 0, REG_SZ, "test_value1") + with CreateKey(subkey1, "subsubkey1") as subsubkey1: + SetValueEx(subsubkey1, "value2", 0, REG_DWORD, 42) + + with CreateKey(main_key, "subkey2") as subkey2: + SetValueEx(subkey2, "value3", 0, REG_SZ, "test_value3") + + DeleteTree(HKEY_CURRENT_USER, test_key_name) + + with self.assertRaises(OSError): + OpenKey(HKEY_CURRENT_USER, test_key_name) + if __name__ == "__main__": if not REMOTE_NAME: diff --git a/Misc/ACKS b/Misc/ACKS index dc28ccf8f57eda..37b7988606fa99 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -483,6 +483,7 @@ Weilin Du John DuBois Paul Dubois Jacques Ducasse +Jadon Duff Andrei Dorian Duma Graham Dumpleton Quinn Dunkan diff --git a/Misc/NEWS.d/next/Core_and_Builtins/2025-08-21-06-31-42.gh-issue-138004.FH2Hre.rst b/Misc/NEWS.d/next/Core_and_Builtins/2025-08-21-06-31-42.gh-issue-138004.FH2Hre.rst new file mode 100644 index 00000000000000..e73be998f4be7b --- /dev/null +++ b/Misc/NEWS.d/next/Core_and_Builtins/2025-08-21-06-31-42.gh-issue-138004.FH2Hre.rst @@ -0,0 +1 @@ +On Solaris/Illumos platforms, thread names are now encoded as ASCII to avoid errors on systems (e.g. OpenIndiana) that don't support non-ASCII names. diff --git a/Misc/NEWS.d/next/Core_and_Builtins/2025-09-02-09-10-06.gh-issue-138372.h1Xk4-.rst b/Misc/NEWS.d/next/Core_and_Builtins/2025-09-02-09-10-06.gh-issue-138372.h1Xk4-.rst new file mode 100644 index 00000000000000..475e01cecf5a5f --- /dev/null +++ b/Misc/NEWS.d/next/Core_and_Builtins/2025-09-02-09-10-06.gh-issue-138372.h1Xk4-.rst @@ -0,0 +1,2 @@ +Fix :exc:`SyntaxWarning` emitted for erroneous subscript expressions involving +:ref:`template string literals `. Patch by Brian Schubert. diff --git a/Misc/NEWS.d/next/Library/2025-08-28-13-20-09.gh-issue-138204.8oLOud.rst b/Misc/NEWS.d/next/Library/2025-08-28-13-20-09.gh-issue-138204.8oLOud.rst new file mode 100644 index 00000000000000..8eb5497f5da545 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2025-08-28-13-20-09.gh-issue-138204.8oLOud.rst @@ -0,0 +1,2 @@ +Forbid expansion of shared anonymous :mod:`memory maps ` on Linux, +which caused a bus error. diff --git a/Misc/NEWS.d/next/Windows/2025-09-03-01-07-44.gh-issue-138314.IeWQ2i.rst b/Misc/NEWS.d/next/Windows/2025-09-03-01-07-44.gh-issue-138314.IeWQ2i.rst new file mode 100644 index 00000000000000..1f9eadcccf223f --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2025-09-03-01-07-44.gh-issue-138314.IeWQ2i.rst @@ -0,0 +1 @@ +Add :func:`winreg.DeleteTree`. diff --git a/Modules/_threadmodule.c b/Modules/_threadmodule.c index 1a64289ea01fdb..a436a553db9802 100644 --- a/Modules/_threadmodule.c +++ b/Modules/_threadmodule.c @@ -2523,7 +2523,9 @@ _thread__get_name_impl(PyObject *module) } #ifdef __sun - return PyUnicode_DecodeUTF8(name, strlen(name), "surrogateescape"); + // gh-138004: Decode Solaris/Illumos (e.g. OpenIndiana) thread names + // from ASCII, since OpenIndiana only supports ASCII names. + return PyUnicode_DecodeASCII(name, strlen(name), "surrogateescape"); #else return PyUnicode_DecodeFSDefault(name); #endif @@ -2561,8 +2563,9 @@ _thread_set_name_impl(PyObject *module, PyObject *name_obj) { #ifndef MS_WINDOWS #ifdef __sun - // Solaris always uses UTF-8 - const char *encoding = "utf-8"; + // gh-138004: Encode Solaris/Illumos thread names to ASCII, + // since OpenIndiana does not support non-ASCII names. + const char *encoding = "ascii"; #else // Encode the thread name to the filesystem encoding using the "replace" // error handler diff --git a/Modules/mmapmodule.c b/Modules/mmapmodule.c index 8413ebe668dffe..5d5b53717c829c 100644 --- a/Modules/mmapmodule.c +++ b/Modules/mmapmodule.c @@ -120,6 +120,7 @@ typedef struct { #ifdef UNIX int fd; _Bool trackfd; + int flags; #endif PyObject *weakreflist; @@ -882,6 +883,13 @@ mmap_resize_method(PyObject *op, PyObject *args) #else void *newmap; +#ifdef __linux__ + if (self->fd == -1 && !(self->flags & MAP_PRIVATE) && new_size > self->size) { + PyErr_Format(PyExc_ValueError, + "mmap: can't expand a shared anonymous mapping on Linux"); + return NULL; + } +#endif if (self->fd != -1 && ftruncate(self->fd, self->offset + new_size) == -1) { PyErr_SetFromErrno(PyExc_OSError); return NULL; @@ -1678,6 +1686,7 @@ new_mmap_object(PyTypeObject *type, PyObject *args, PyObject *kwdict) else { m_obj->fd = -1; } + m_obj->flags = flags; Py_BEGIN_ALLOW_THREADS m_obj->data = mmap(NULL, map_size, prot, flags, fd, offset); diff --git a/PC/clinic/winreg.c.h b/PC/clinic/winreg.c.h index 45d54e3c90289a..d76a8d8aef8cb8 100644 --- a/PC/clinic/winreg.c.h +++ b/PC/clinic/winreg.c.h @@ -1633,6 +1633,70 @@ winreg_EnableReflectionKey(PyObject *module, PyObject *arg) #if (defined(MS_WINDOWS_DESKTOP) || defined(MS_WINDOWS_SYSTEM) || defined(MS_WINDOWS_GAMES)) && (defined(MS_WINDOWS_DESKTOP) || defined(MS_WINDOWS_SYSTEM)) +PyDoc_STRVAR(winreg_DeleteTree__doc__, +"DeleteTree($module, key, sub_key=None, /)\n" +"--\n" +"\n" +"Deletes the specified key and all its subkeys and values recursively.\n" +"\n" +" key\n" +" An already open key, or any one of the predefined HKEY_* constants.\n" +" sub_key\n" +" A string that names the subkey to delete. If None, deletes all subkeys\n" +" and values of the specified key.\n" +"\n" +"This function deletes a key and all its descendants. If sub_key is None,\n" +"all subkeys and values of the specified key are deleted."); + +#define WINREG_DELETETREE_METHODDEF \ + {"DeleteTree", _PyCFunction_CAST(winreg_DeleteTree), METH_FASTCALL, winreg_DeleteTree__doc__}, + +static PyObject * +winreg_DeleteTree_impl(PyObject *module, HKEY key, const wchar_t *sub_key); + +static PyObject * +winreg_DeleteTree(PyObject *module, PyObject *const *args, Py_ssize_t nargs) +{ + PyObject *return_value = NULL; + HKEY key; + const wchar_t *sub_key = NULL; + + if (!_PyArg_CheckPositional("DeleteTree", nargs, 1, 2)) { + goto exit; + } + if (!clinic_HKEY_converter(_PyModule_GetState(module), args[0], &key)) { + goto exit; + } + if (nargs < 2) { + goto skip_optional; + } + if (args[1] == Py_None) { + sub_key = NULL; + } + else if (PyUnicode_Check(args[1])) { + sub_key = PyUnicode_AsWideCharString(args[1], NULL); + if (sub_key == NULL) { + goto exit; + } + } + else { + _PyArg_BadArgument("DeleteTree", "argument 2", "str or None", args[1]); + goto exit; + } +skip_optional: + return_value = winreg_DeleteTree_impl(module, key, sub_key); + +exit: + /* Cleanup for sub_key */ + PyMem_Free((void *)sub_key); + + return return_value; +} + +#endif /* (defined(MS_WINDOWS_DESKTOP) || defined(MS_WINDOWS_SYSTEM) || defined(MS_WINDOWS_GAMES)) && (defined(MS_WINDOWS_DESKTOP) || defined(MS_WINDOWS_SYSTEM)) */ + +#if (defined(MS_WINDOWS_DESKTOP) || defined(MS_WINDOWS_SYSTEM) || defined(MS_WINDOWS_GAMES)) && (defined(MS_WINDOWS_DESKTOP) || defined(MS_WINDOWS_SYSTEM)) + PyDoc_STRVAR(winreg_QueryReflectionKey__doc__, "QueryReflectionKey($module, key, /)\n" "--\n" @@ -1771,7 +1835,11 @@ winreg_QueryReflectionKey(PyObject *module, PyObject *arg) #define WINREG_ENABLEREFLECTIONKEY_METHODDEF #endif /* !defined(WINREG_ENABLEREFLECTIONKEY_METHODDEF) */ +#ifndef WINREG_DELETETREE_METHODDEF + #define WINREG_DELETETREE_METHODDEF +#endif /* !defined(WINREG_DELETETREE_METHODDEF) */ + #ifndef WINREG_QUERYREFLECTIONKEY_METHODDEF #define WINREG_QUERYREFLECTIONKEY_METHODDEF #endif /* !defined(WINREG_QUERYREFLECTIONKEY_METHODDEF) */ -/*[clinic end generated code: output=be4b6857b95558b5 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=ce7e8e38884851fb input=a9049054013a1b77]*/ diff --git a/PC/winreg.c b/PC/winreg.c index 9bbacb0f50bd63..8633f29670e263 100644 --- a/PC/winreg.c +++ b/PC/winreg.c @@ -2019,6 +2019,45 @@ winreg_EnableReflectionKey_impl(PyObject *module, HKEY key) Py_RETURN_NONE; } +/*[clinic input] +winreg.DeleteTree + + key: HKEY + An already open key, or any one of the predefined HKEY_* constants. + sub_key: Py_UNICODE(accept={str, NoneType}) = None + A string that names the subkey to delete. If None, deletes all subkeys + and values of the specified key. + / + +Deletes the specified key and all its subkeys and values recursively. + +This function deletes a key and all its descendants. If sub_key is None, +all subkeys and values of the specified key are deleted. +[clinic start generated code]*/ + +static PyObject * +winreg_DeleteTree_impl(PyObject *module, HKEY key, const wchar_t *sub_key) +/*[clinic end generated code: output=c34395ee59290501 input=419ef9bb8b06e4bf]*/ +{ + LONG rc; + + if (PySys_Audit("winreg.DeleteTree", "nu", + (Py_ssize_t)key, sub_key) < 0) { + return NULL; + } + + Py_BEGIN_ALLOW_THREADS + rc = RegDeleteTreeW(key, sub_key); + Py_END_ALLOW_THREADS + + if (rc != ERROR_SUCCESS) { + PyErr_SetFromWindowsErrWithFunction(rc, "RegDeleteTreeW"); + return NULL; + } + + Py_RETURN_NONE; +} + /*[clinic input] winreg.QueryReflectionKey @@ -2077,6 +2116,7 @@ static struct PyMethodDef winreg_methods[] = { WINREG_DELETEKEY_METHODDEF WINREG_DELETEKEYEX_METHODDEF WINREG_DELETEVALUE_METHODDEF + WINREG_DELETETREE_METHODDEF WINREG_DISABLEREFLECTIONKEY_METHODDEF WINREG_ENABLEREFLECTIONKEY_METHODDEF WINREG_ENUMKEY_METHODDEF diff --git a/Python/codegen.c b/Python/codegen.c index 53388592330a5d..b0778518b2eeb2 100644 --- a/Python/codegen.c +++ b/Python/codegen.c @@ -29,6 +29,7 @@ #include "pycore_symtable.h" // PySTEntryObject #include "pycore_unicodeobject.h" // _PyUnicode_EqualToASCIIString #include "pycore_ceval.h" // SPECIAL___ENTER__ +#include "pycore_template.h" // _PyTemplate_Type #define NEED_OPCODE_METADATA #include "pycore_opcode_metadata.h" // _PyOpcode_opcode_metadata, _PyOpcode_num_popped/pushed @@ -3617,10 +3618,11 @@ infer_type(expr_ty e) return &PyGen_Type; case Lambda_kind: return &PyFunction_Type; - case JoinedStr_kind: case TemplateStr_kind: - case FormattedValue_kind: case Interpolation_kind: + return &_PyTemplate_Type; + case JoinedStr_kind: + case FormattedValue_kind: return &PyUnicode_Type; case Constant_kind: return Py_TYPE(e->v.Constant.value); @@ -3674,6 +3676,8 @@ check_subscripter(compiler *c, expr_ty e) case Set_kind: case SetComp_kind: case GeneratorExp_kind: + case TemplateStr_kind: + case Interpolation_kind: case Lambda_kind: { location loc = LOC(e); return _PyCompile_Warn(c, loc, "'%.200s' object is not subscriptable; " @@ -3708,9 +3712,7 @@ check_index(compiler *c, expr_ty e, expr_ty s) case List_kind: case ListComp_kind: case JoinedStr_kind: - case TemplateStr_kind: - case FormattedValue_kind: - case Interpolation_kind: { + case FormattedValue_kind: { location loc = LOC(e); return _PyCompile_Warn(c, loc, "%.200s indices must be integers " "or slices, not %.200s; " diff --git a/Tools/c-analyzer/c_parser/parser/__init__.py b/Tools/c-analyzer/c_parser/parser/__init__.py index ff4f303c4a2bec..f3f09107aefce6 100644 --- a/Tools/c-analyzer/c_parser/parser/__init__.py +++ b/Tools/c-analyzer/c_parser/parser/__init__.py @@ -116,6 +116,8 @@ * alt impl using a state machine (& tokenizer or split on delimiters) """ +import textwrap + from ..info import ParsedItem from ._info import SourceInfo @@ -208,7 +210,27 @@ def _iter_source(lines, *, maxtext=11_000, maxlines=200, showtext=False): return # At this point either the file ended prematurely # or there's "too much" text. - filename, lno, text = srcinfo.filename, srcinfo._start, srcinfo.text + filename, lno_from, lno_to = srcinfo.filename, srcinfo.start, srcinfo.end + text = srcinfo.text if len(text) > 500: text = text[:500] + '...' - raise Exception(f'unmatched text ({filename} starting at line {lno}):\n{text}') + + if srcinfo.too_much_text(maxtext): + msg = f''' + too much text, try to increase MAX_SIZES[MAXTEXT] in cpython/_parser.py + {filename} starting at line {lno_from} to {lno_to} + has code with length {len(text)} greater than {maxtext}: + {text} + ''' + raise RuntimeError(textwrap.dedent(msg)) + + if srcinfo.too_many_lines(maxlines): + msg = f''' + too many lines, try to increase MAX_SIZES[MAXLINES] in cpython/_parser.py + {filename} starting at line {lno_from} to {lno_to} + has code with number of lines {lno_to - lno_from} greater than {maxlines}: + {text} + ''' + raise RuntimeError(textwrap.dedent(msg)) + + raise RuntimeError(f'unmatched text ({filename} starting at line {lno_from}):\n{text}') diff --git a/Tools/c-analyzer/c_parser/parser/_info.py b/Tools/c-analyzer/c_parser/parser/_info.py index 340223db933c90..13b192e2ce982f 100644 --- a/Tools/c-analyzer/c_parser/parser/_info.py +++ b/Tools/c-analyzer/c_parser/parser/_info.py @@ -123,10 +123,16 @@ def resolve(self, kind, data, name, parent=None): def done(self): self._set_ready() + def too_much_text(self, maxtext): + return maxtext and len(self.text) > maxtext + + def too_many_lines(self, maxlines): + return maxlines and self.end - self.start > maxlines + def too_much(self, maxtext, maxlines): - if maxtext and len(self.text) > maxtext: + if self.too_much_text(maxtext): pass - elif maxlines and self.end - self.start > maxlines: + elif self.too_many_lines(maxlines): pass else: return False diff --git a/Tools/c-analyzer/cpython/_parser.py b/Tools/c-analyzer/cpython/_parser.py index 1e754040eaf1cc..95a474a49229a1 100644 --- a/Tools/c-analyzer/cpython/_parser.py +++ b/Tools/c-analyzer/cpython/_parser.py @@ -326,7 +326,7 @@ def clean_lines(text): _abs('Modules/_testcapimodule.c'): (20_000, 400), _abs('Modules/expat/expat.h'): (10_000, 400), _abs('Objects/stringlib/unicode_format.h'): (10_000, 400), - _abs('Objects/typeobject.c'): (35_000, 200), + _abs('Objects/typeobject.c'): (380_000, 13_000), _abs('Python/compile.c'): (20_000, 500), _abs('Python/optimizer.c'): (100_000, 5_000), _abs('Python/parking_lot.c'): (40_000, 1000),