Merge branch 'master' into pep757-final

.pre-commit-config.yaml

@@ -93,6 +93,7 @@ repos:
     hooks:
       - id: codespell
         name: "Check for common misspellings in text files"
+        require_serial: true
         stages: [manual]
 
   # Local checks for PEP headers and more

peps/pep-0376.rst

@@ -4,7 +4,6 @@ Author: Tarek Ziadé <[email protected]>
 Status: Final
 Type: Standards Track
 Topic: Packaging
-Content-Type: text/x-rst
 Created: 22-Feb-2009
 Python-Version: 2.7, 3.2
 Post-History: `22-Jun-2009 <https://mail.python.org/archives/list/[email protected]/thread/ILLTIOZAULMDY5CAS6GOITEYJ4HNFATQ/>`__
@@ -607,32 +606,26 @@ formats in addition to the new format, in order to ease the transition.
 References
 ==========
 
-.. [#distutils]
-   http://docs.python.org/distutils
-
-.. [#distutils2]
-   http://hg.python.org/distutils2
-
 .. [#setuptools]
-   http://peak.telecommunity.com/DevCenter/setuptools
+   https://peak.telecommunity.com/DevCenter/setuptools
 
 .. [#easyinstall]
-   http://peak.telecommunity.com/DevCenter/EasyInstall
+   https://peak.telecommunity.com/DevCenter/EasyInstall
 
 .. [#pip]
-   http://pypi.python.org/pypi/pip
+   https://pypi.org/project/pip/
 
 .. [#eggformats]
-   http://peak.telecommunity.com/DevCenter/EggFormats
+   https://peak.telecommunity.com/DevCenter/EggFormats
 
 .. [#fedora]
-   http://fedoraproject.org/wiki/Packaging/Python/Eggs#Providing_Eggs_using_Setuptools
+   https://fedoraproject.org/wiki/Packaging/Python/Eggs#Providing_Eggs_using_Setuptools
 
 .. [#debian]
-   http://wiki.debian.org/DebianPython/NewPolicy
+   https://wiki.debian.org/DebianPython/NewPolicy
 
 .. [#prototype]
-   http://bitbucket.org/tarek/pep376/
+   https://web.archive.org/web/20090726092550/http://bitbucket.org/tarek/pep376/
 
 Acknowledgements
 ================
@@ -644,14 +637,3 @@ Copyright
 =========
 
 This document has been placed in the public domain.
-
-
-
-..
-   Local Variables:
-   mode: indented-text
-   indent-tabs-mode: nil
-   sentence-end-double-space: t
-   fill-column: 70
-   coding: utf-8
-   End:

peps/pep-0386.rst

@@ -1,12 +1,9 @@
 PEP: 386
 Title: Changing the version comparison module in Distutils
-Version: $Revision$
-Last-Modified: $Date$
 Author: Tarek Ziadé <[email protected]>
 Status: Superseded
 Type: Standards Track
 Topic: Packaging
-Content-Type: text/x-rst
 Created: 04-Jun-2009
 Superseded-By: 440
 
@@ -475,34 +472,31 @@ References
 ==========
 
 .. [#distutils]
-   http://docs.python.org/distutils
+   https://docs.python.org/3.11/distutils/
 
 .. [#setuptools]
-   http://peak.telecommunity.com/DevCenter/setuptools
+   https://peak.telecommunity.com/DevCenter/setuptools
 
 .. [#setuptools-version]
-   http://peak.telecommunity.com/DevCenter/setuptools#specifying-your-project-s-version
+   https://peak.telecommunity.com/DevCenter/setuptools#specifying-your-project-s-version
 
 .. [#pypi]
-   http://pypi.python.org/pypi
+   https://pypi.org/
 
 .. [#pip]
-   http://pypi.python.org/pypi/pip
+   https://pypi.org/project/pip/
 
 .. [#ezinstall]
-   http://peak.telecommunity.com/DevCenter/EasyInstall
+   https://peak.telecommunity.com/DevCenter/EasyInstall
 
 .. [#zc.buildout]
-   http://pypi.python.org/pypi/zc.buildout
+   https://pypi.org/project/zc.buildout/
 
 .. [#twisted]
-   http://twistedmatrix.com/trac/
-
-.. [#requires]
-   http://peak.telecommunity.com/DevCenter/setuptools
+   https://twisted.org/
 
 .. [#prototype]
-   http://bitbucket.org/tarek/distutilsversion/
+   https://web.archive.org/web/20090726093825/http://bitbucket.org/tarek/distutilsversion/
 
 Acknowledgments
 ===============
@@ -514,14 +508,3 @@ Copyright
 =========
 
 This document has been placed in the public domain.
-
-
-
-..
-   Local Variables:
-   mode: indented-text
-   indent-tabs-mode: nil
-   sentence-end-double-space: t
-   fill-column: 70
-   coding: utf-8
-   End:

peps/pep-0768.rst

@@ -1,10 +1,12 @@
 PEP: 768
 Title: Safe external debugger interface for CPython
 Author: Pablo Galindo Salgado <[email protected]>, Matt Wozniski <[email protected]>, Ivona Stojanovic <[email protected]>
+Discussions-To: https://discuss.python.org/t/pep-768-safe-external-debugger-interface-for-cpython/73969
 Status: Draft
 Type: Standards Track
 Created: 25-Nov-2024
 Python-Version: 3.14
+Post-History: `11-Dec-2024 <https://discuss.python.org/t/pep-768-safe-external-debugger-interface-for-cpython/73969>`__
 
 Abstract
 ========
@@ -145,7 +147,7 @@ provides Python code to be executed when the interpreter reaches a safe point.
 
 The value for ``MAX_SCRIPT_SIZE`` will be a trade-off between binary size and
 how big debugging scripts can be. As most of the logic should be in libraries
-and arbitrary code can be executed with very short ammount of Python we are
+and arbitrary code can be executed with very short amount of Python we are
 proposing to start with 4kb initially. This value can be extended in the future
 if we ever need to.
 
@@ -169,7 +171,7 @@ debugger support:
         uint64_t eval_breaker;            // Location of the eval breaker flag
         uint64_t remote_debugger_support; // Offset to our support structure
         uint64_t debugger_pending_call;   // Where to write the pending flag
-        uint64_t debugger_script;         // Where to write the script
+        uint64_t debugger_script;         // Where to write the script path
     } debugger_support;
 
 These offsets allow debuggers to locate critical debugging control structures in
@@ -197,8 +199,8 @@ When a debugger wants to attach to a Python process, it follows these steps:
 
 5. Write control information:
 
-   - Write a string of Python code to be executed into the ``debugger_script``
-     field in ``_PyRemoteDebuggerSupport``.
+   - Write a filename containing Python code to be executed into the
+     ``debugger_script`` field in ``_PyRemoteDebuggerSupport``.
    - Set ``debugger_pending_call`` flag in ``_PyRemoteDebuggerSupport``
    - Set ``_PY_EVAL_PLEASE_STOP_BIT`` in the ``eval_breaker`` field
 
@@ -218,22 +220,35 @@ normal execution, allowing modern CPUs to effectively speculate past it.
 
 
 When a debugger has set both the ``eval_breaker`` flag and ``debugger_pending_call``,
-the interpreter will execute the provided debugging code at the next safe point
-and executes the provided code. This all happens in a completely safe context, since
-the interpreter is guaranteed to be in a consistent state whenever the eval breaker
-is checked.
+the interpreter will execute the provided debugging code at the next safe point.
+This all happens in a completely safe context, since the interpreter is
+guaranteed to be in a consistent state whenever the eval breaker is checked.
+
+An audit event will be raised before the code is executed, allowing this mechanism
+to be audited or disabled if desired by a system's administrator.
 
 .. code-block:: c
 
     // In ceval.c
     if (tstate->eval_breaker) {
         if (tstate->remote_debugger_support.debugger_pending_call) {
             tstate->remote_debugger_support.debugger_pending_call = 0;
-            if (tstate->remote_debugger_support.debugger_script[0]) {
-               if (PyRun_SimpleString(tstate->remote_debugger_support.debugger_script)<0) {
-                   PyErr_Clear();
-               };
-               // ...
+            const char *path = tstate->remote_debugger_support.debugger_script;
+            if (*path) {
+                if (0 != PySys_Audit("debugger_script", "%s", path)) {
+                    PyErr_Clear();
+                } else {
+                    FILE* f = fopen(path, "r");
+                    if (!f) {
+                        PyErr_SetFromErrno(OSError);
+                    } else {
+                        PyRun_AnyFile(f, path);
+                        fclose(f);
+                    }
+                    if (PyErr_Occurred()) {
+                        PyErr_WriteUnraisable(...);
+                    }
+                }
             }
         }
     }
@@ -290,11 +305,16 @@ mechanism piggybacks on existing interpreter safe points.
 Security Implications
 =====================
 
-This interface does not introduce new security concerns as it relies entirely on
-existing operating system security mechanisms for process memory access. Although
-the PEP doesn't specify how memory should be written to the target process, in practice
-this will be done using standard system calls that are already being used by other
-debuggers and tools. Some examples are:
+This interface does not introduce new security concerns as it is only usable by
+processes that can already write to arbitrary memory within your process and
+execute arbitrary code on the machine (in order to create the file containing
+the Python code to be executed).
+
+Existing operating system security mechanisms are effective for guarding
+against attackers gaining arbitrary memory write access. Although the PEP
+doesn't specify how memory should be written to the target process, in practice
+this will be done using standard system calls that are already being used by
+other debuggers and tools. Some examples are:
 
 * On Linux, the `process_vm_readv() <https://man7.org/linux/man-pages/man2/process_vm_readv.2.html>`__
   and `process_vm_writev() <https://man7.org/linux/man-pages/man2/process_vm_writev.2.html>`__ system calls
@@ -325,14 +345,17 @@ All mechanisms ensure that:
 1. Only authorized processes can read/write memory
 2. The same security model that governs traditional debugger attachment applies
 3. No additional attack surface is exposed beyond what the OS already provides for debugging
+4. Even if an attacker can write arbitrary memory, they cannot escalate this
+   to arbitrary code execution unless they already have filesystem access
 
 The memory operations themselves are well-established and have been used safely
 for decades in tools like GDB, LLDB, and various system profilers.
 
 It's important to note that any attempt to attach to a Python process via this
-mechanism would be detectable by system-level monitoring tools. This
-transparency provides an additional layer of accountability, allowing
-administrators to audit debugging operations in sensitive environments.
+mechanism would be detectable by system-level monitoring tools as well as by
+Python audit hooks. This transparency provides an additional layer of
+accountability, allowing administrators to audit debugging operations in
+sensitive environments.
 
 Further, the strict reliance on OS-level security controls ensures that existing
 system policies remain effective. For enterprise environments, this means
@@ -343,7 +366,7 @@ or macOS's ``taskgated`` to restrict debugger access will equally govern the
 proposed interface.
 
 By maintaining compatibility with existing security frameworks, this design
-ensures that adopting the new interface requires no changes to established
+ensures that adopting the new interface requires no changes to established.
 
 How to Teach This
 =================
@@ -367,17 +390,22 @@ can be found `here
 Rejected Ideas
 ==============
 
-Using a path as the debugger input
-----------------------------------
-
-We have selected that the mechanism for executing remote code is that tools
-write the code directly in the remote process to eliminate a possible security
-vulnerability in which the file to be executed can be altered by parties other
-than the debugger process if permissions are not set correctly or filesystem
-configurations allow for this to happen. It is also trivial to write code that
-executes the contents of a file so the current mechanism doesn't disallow tools
-that want to just execute files to just do so if they are ok with the security
-profile of such operation.
+Writing Python code into the buffer
+-----------------------------------
+
+We have chosen to have debuggers write the code to be executed into a file
+whose path is written into a buffer in the remote process. This has been deemed
+more secure than writing the Python code to be executed itself into a buffer in
+the remote process, because it means that an attacker who has gained arbitrary
+writes in a process but not arbitrary code execution or file system
+manipulation can't escalate to arbitrary code execution through this interface.
+
+This does require the attaching debugger to pay close attention to filesystem
+permissions when creating the file containing the code to be executed, however.
+If an attacker has the ability to overwrite the file, or to replace a symlink
+in the file path to point to somewhere attacker controlled, this would allow
+them to force their malicious code to be executed rather than the code the
+debugger intends to run.
 
 Thanks
 ======