From 94ac54d695c2e20c42da58371a5550aa59adea9a Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Wed, 27 Nov 2024 13:25:59 +0000 Subject: [PATCH 01/17] Add PEP 0778.rst --- peps/pep-0778.rst | 340 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 340 insertions(+) create mode 100644 peps/pep-0778.rst diff --git a/peps/pep-0778.rst b/peps/pep-0778.rst new file mode 100644 index 00000000000..1676f51a9cc --- /dev/null +++ b/peps/pep-0778.rst @@ -0,0 +1,340 @@ +PEP: 778 +Title: Safe external debugger interface for CPython +Author: Pablo Galindo Salgado , Matt Wozniski , Ivona Stojanovic +Sponsor: Pablo Galindo Salgado +Discussions-To: xxxxx +Status: Draft +Type: Standards Track +Topic: Interpreter Core +Created: 25-Nov-2024 +Python-Version: 3.14 + +Abstract +======== + +This PEP proposes adding a zero-overhead debugging interface to CPython that +allows debuggers and profilers to safely attach to running Python processes. The +interface provides safe execution points for attaching debugger code without +modifying the interpreter's normal execution path or adding runtime overhead. + +A key application of this interface will be enabling pdb to attach to live +processes by process ID, similar to ``gdb -p``, allowing developers to inspect and +debug Python applications interactively in real-time without stopping or +restarting them. + +Motivation +========== + + +Debugging Python processes in production and live environments presents unique +challenges. Developers often need to analyze application behavior without +stopping or restarting services, which is especially crucial for +high-availability systems. Common scenarios include diagnosing deadlocks, +inspecting memory usage, or investigating unexpected behavior in real-time. + +Very few Python tools can attach to running processes, primarily because doing +so requires deep expertise in both operating system debugging interfaces and +CPython internals. While C/C++ debuggers like gdb and lldb can attach to +processes using well-understood techniques, Python tools must implement all of +these low-level mechanisms plus handle additional complexity. For example, when +gdb needs to execute code in a target process, it: + +1. Uses ptrace to allocate a small chunk of executable memory (easier said than done) +2. Writes a small sequence of machine code - typically a function prologue, the + desired instructions, and code to restore registers +3. Saves all the target thread's registers +4. Changes the instruction pointer to the injected code +5. Lets the process run until it hits a breakpoint at the end of the injected code +6. Restores the original registers and continues execution + +Python tools face this same challenge of code injection, but with an additional +layer of complexity. Not only do they need to implement the above mechanism, +they must also understand and safely interact with CPython's runtime state, +including the interpreter loop, garbage collector, thread state, and reference +counting system. This combination of low-level system manipulation and +high-level interpreter knowledge makes implementing Python debugging tools +exceptionally difficult. + +The few tools that do attempt this must resort to suboptimal and unsafe methods, +using system debuggers like gdb and lldb to forcefully inject code. This +approach is fundamentally unsafe because the injected code can execute at any +point during the interpreter's execution cycle - even during critical operations +like memory allocation, garbage collection, or thread state management. When +this happens, the results are catastrophic: attempting to allocate memory while +already inside ``malloc()`` causes crashes, modifying objects during garbage +collection corrupts the interpreter's state, and touching thread state at the +wrong time leads to deadlocks. + +Various tools attempt to minimize these risks through complex workarounds, such +as spawning separate threads for injected code or carefully timing their +operations or trying to select some good points to stop the process. However, +these mitigations cannot fully solve the underlying problem: without cooperation +from the interpreter, there's no way to know if it's safe to execute code at any +given moment. Even carefully implemented tools can crash the interpreter because +they're fundamentally working against it rather than with it. + + +Rationale +========= + + +Rather than forcing tools to work around interpreter limitations with unsafe +code injection, we can extend CPython with a proper debugging interface that +guarantees safe execution. By adding minimal thread state fields and integrating +with the interpreter's existing evaluation loop, we can ensure debugging +operations only occur at well-defined safe points. This eliminates the +possibility of crashes and corruption while maintaining zero overhead during +normal execution. + + +The key insight is that we don't need to inject code at arbitrary points - we +just need to signal to the interpreter that we want code executed at the next +safe opportunity. This approach works with the interpreter's natural execution +flow rather than fighting against it. + +After describing this idea to the PyPy development team, this proposal has +already `been implemented in PyPy `__, +proving both its feasibility and effectiveness. Their implementation +demonstrates that we can provide safe debugging capabilities with zero runtime +overhead during normal execution. The proposed mechanism not only reduces risks +associated with current debugging practices but also lays the foundation for +future enhancements. For instance, this framework could enable integration with +popular observability tools, providing real-time insights into interpreter +performance or memory usage. One compelling use case for this interface is +enabling pdb to attach to running Python processes, similar to how gdb allows +users to attach to a program by process ID (``gdb -p ``). With this +feature, developers could inspect the state of a running application, evaluate +expressions, and step through code dynamically. This approach would align +Python's debugging capabilities with those of other major programming languages +and debugging tools that support this mode. + +Specification +============= + + +This proposal introduces a safe debugging mechanism that allows external +processes to trigger code execution in a Python interpreter at well-defined safe +points. The key insight is that rather than injecting code directly via system +debuggers, we can leverage the interpreter's existing evaluation loop and thread +state to coordinate debugging operations. + +The mechanism works by having debuggers write to specific memory locations in +the target process that the interpreter then checks during its normal execution +cycle. When the interpreter detects a debugger wants to attach, it executes the +requested operations only when it's safe to do so - that is, when no internal +locks are held and all data structures are in a consistent state. + + +Runtime State Extensions +------------------------ + +A new structure is added to PyThreadState to support remote debugging: + +.. code-block:: C + + typedef struct _remote_debugger_support { + int debugger_pending_call; + char debugger_script[MAX_SCRIPT_SIZE]; + } _PyRemoteDebuggerSupport; + + +This structure is appended to ``PyThreadState``, adding only a few fields that +are **never accessed during normal execution**. The ``debugger_pending_call`` field +indicates when a debugger has requested execution, while ``debugger_script`` +provides Python code to be executed when the interpreter reaches a safe point. + + +Debug Offsets Table +------------------- + + +Python 3.12 introduced a debug offsets table placed at the start of the +PyRuntime structure. This section contains the ``_Py_DebugOffsets`` structure that +allows external tools to reliably find critical runtime structures regardless of +`ASLR `__ or +how Python was compiled. + +This proposal extends the existing debug offsets table with new fields for +debugger support: + +.. code-block:: C + + struct _debugger_support { + uint64_t eval_breaker; /* Location of the eval breaker flag */ + uint64_t remote_debugger_support; /* Offset to our support structure */ + uint64_t debugger_pending_call; /* Where to write the pending flag */ + uint64_t debugger_script; /* Where to write the script */ + } debugger_support; + +These offsets allow debuggers to locate critical debugging control structures in +the target process's memory space. The offsets are relative to the relevant +structure address, making them valid regardless of where structures are actually +loaded in memory. + +Attachment Protocol +------------------- +When a debugger wants to attach to a Python process, it follows these steps: + +1. Locate ``PyRuntime`` structure in the process: + - Find Python binary (executable or libpython) in process memory (OS dependent process) + - Extract ``.PyRuntime`` section offset from binary's format (ELF/Mach-O/PE) + - Calculate the actual ``PyRuntime`` address in the running process by relocating the offset to the binary's load address + +2. Access debug offset information by read ``_Py_DebugOffsets`` table from located ``PyRuntime`` structure. + +3. Use the offsets to locate the debugger interface structure withing the desired thread state. + +4. Write control information: + - Write python code to be executed. + - Set ``debugger_pending_call`` flag in ``_PyRemoteDebuggerSupport`` + - Set ``_PY_EVAL_PLEASE_STOP_BIT`` in the ``eval_breaker`` field + - Wait for the interpreter to reach next safe point and execute debugger code + +Interpreter Integration +----------------------- + +The interpreter's regular evaluation loop already includes a check of the +eval_breaker flag for handling signals, periodic tasks, and other interrupts. We +leverage this existing mechanism by checking for debugger pending calls only +when the ``eval_breaker`` is set, ensuring zero overhead during normal execution. +This check has no overhead. Indeed, profiling with Linux perf shows this branch +is highly predictable - the ``debugger_pending_call`` check is never taken during +normal execution, allowing modern CPUs to effectively speculate past it. + + +When a debugger has set both the ``eval_breaker`` flag and ``debugger_pending_call``, +the interpreter will execute the provided debugging code at the next safe point +and executes the provided code. This all happens in a completely safe context as +any of the operations that happen in the eval breaker as the interpreter is in a +consistent state: + +.. code-block:: c + + /* In ceval.c */ + if (tstate->eval_breaker) { + if (tstate->remote_debugger_support.debugger_pending_call) { + tstate->remote_debugger_support.debugger_pending_call = 0; + if (tstate->remote_debugger_support.debugger_script[0]) { + if (PyRun_SimpleString(tstate->remote_debugger_support.debugger_script)<0) { + PyErr_Clear(); + }; + // ... + } + } + } + + +Python API +---------- + +To support safe execution of Python code in a remote process without having to +re-implement all these steps in every tool, this proposal extends the sys module +with a new function. This function allows debuggers or external tools to execute +arbitrary Python code within the context of a specified Python process: + +.. code-block:: python + + def remote_exec(pid: int, code: str) -> None: + Executes a block of Python code in a remote Python process, identified by its process ID. + + Args: + pid (int): The process ID of the target Python interpreter. + code (str): A string containing the Python code to be executed. + +An example usage of the API would look like: + +.. code-block:: python + + import sys + # Execute a print statement in a remote Python process with PID 12345 + try: + sys.remote_execute(12345, "print('Hello from remote execution!')") + except Exception as e: + print(f"Failed to execute code: {e}") + + +Backwards Compatibility +======================= + +This change has no impact on existing Python code or interpreter performance. +The added fields are only accessed during debugger attachment, and the checking +mechanism piggybacks on existing interpreter safe points. + + +Security Implications +===================== + +This interface does not introduce new security concerns as it relies entirely on +existing operating system security mechanisms for process memory access. Although +the PEP doesn't specify how memory should be written to the target process, in practice +this will be done using standard system calls that are already being used by other +debuggers and tools. Some examples are: + +* On Linux, the ``process_vm_readv()`` and ``process_vm_writev()`` system calls + are used to read and write memory from another process. These operations are + controlled by ptrace access mode checks - the same ones that govern debugger + attachment. A process can only read from or write to another process's memory + if it has the appropriate permissions (typically requiring the same user ID as + the target process or ``CAP_SYS_PTRACE`` capability). + +* On macOS, the interface would leverage ``mach_vm_read_overwrite()`` and + ``mach_vm_write()`` through the Mach task system. These operations require + ``task_for_pid()`` access, which is strictly controlled by the operating + system. By default, access is limited to processes running as root or those + with specific entitlements granted by Apple's security framework. + +* On Windows, the ``ReadProcessMemory()`` and ``WriteProcessMemory()`` functions + provide similar functionality. Access is controlled through the Windows + security model - a process needs ``PROCESS_VM_READ`` and ``PROCESS_VM_WRITE`` + permissions, which typically require the same user context or appropriate + privileges. These are the same permissions required by debuggers, ensuring + consistent security semantics across platforms. + +All mechanisms ensure that: + +1. Only authorized processes can read/write memory +2. The same security model that governs traditional debugger attachment applies +3. No additional attack surface is exposed beyond what the OS already provides for debugging + +The memory operations themselves are well-established and have been used safely +for decades in tools like gdb, lldb, and various system profilers. + +It’s important to note that any attempt to attach to a Python process via this +mechanism would be detectable by system-level monitoring tools. This +transparency provides an additional layer of accountability, allowing +administrators to audit debugging operations in sensitive environments. + +Further, the strict reliance on OS-level security controls ensures that existing +system policies remain effective. For enterprise environments, this means +administrators can continue to enforce debugging restrictions using standard +tools and policies without requiring additional configuration. For instance, +leveraging Linux’s ``ptrace_scope`` or macOS’s ``taskgated`` to restrict +debugger access will equally govern the proposed interface. + +By maintaining compatibility with existing security frameworks, this design +ensures that adopting the new interface requires no changes to established +security practices, thereby minimizing barriers to adoption. + +How to Teach This +================= + +For tool authors, this interface becomes the standard way to implement debugger +attachment, replacing unsafe system debugger approaches.A section in the Python +Developer Guide could describe the internal workings of the mechanism, including +the ``debugger_support`` offsets and how to interact with them using system +APIs. + +End users need not be aware of the interface, benefiting only from improved +debugging tool stability and reliability. + +Reference Implementation +======================== + +https://github.com/pablogsal/cpython/commits/remote_pdb/ + + +Copyright +========= + +This document is placed in the public domain or under the CC0-1.0-Universal +license, whichever is more permissive. \ No newline at end of file From be294a56263554494025a60a7c6df660c4b9a22d Mon Sep 17 00:00:00 2001 From: Matt Wozniski Date: Thu, 5 Dec 2024 20:22:22 -0500 Subject: [PATCH 02/17] Suggested changes to PEP 778 Mostly minor nits, with a few sections reworded for clarity. The switch to C99 compatible `//` comments instead of `/*` comments is just because neovim's reST syntax highlighter is getting tripped up by the code blocks and gets stuck thinking that the rest of the document is emphasized. Signed-off-by: Matt Wozniski --- peps/pep-0778.rst | 83 ++++++++++++++++++++++++++--------------------- 1 file changed, 46 insertions(+), 37 deletions(-) diff --git a/peps/pep-0778.rst b/peps/pep-0778.rst index 1676f51a9cc..9391681d01e 100644 --- a/peps/pep-0778.rst +++ b/peps/pep-0778.rst @@ -30,7 +30,7 @@ Debugging Python processes in production and live environments presents unique challenges. Developers often need to analyze application behavior without stopping or restarting services, which is especially crucial for high-availability systems. Common scenarios include diagnosing deadlocks, -inspecting memory usage, or investigating unexpected behavior in real-time. +inspecting memory usage, or investigating unexpected behavior in real-time. Very few Python tools can attach to running processes, primarily because doing so requires deep expertise in both operating system debugging interfaces and @@ -52,10 +52,10 @@ layer of complexity. Not only do they need to implement the above mechanism, they must also understand and safely interact with CPython's runtime state, including the interpreter loop, garbage collector, thread state, and reference counting system. This combination of low-level system manipulation and -high-level interpreter knowledge makes implementing Python debugging tools +deep domain specific interpreter knowledge makes implementing Python debugging tools exceptionally difficult. -The few tools that do attempt this must resort to suboptimal and unsafe methods, +The few tools that do attempt this resort to suboptimal and unsafe methods, using system debuggers like gdb and lldb to forcefully inject code. This approach is fundamentally unsafe because the injected code can execute at any point during the interpreter's execution cycle - even during critical operations @@ -80,7 +80,7 @@ Rationale Rather than forcing tools to work around interpreter limitations with unsafe code injection, we can extend CPython with a proper debugging interface that -guarantees safe execution. By adding minimal thread state fields and integrating +guarantees safe execution. By adding a few thread state fields and integrating with the interpreter's existing evaluation loop, we can ensure debugging operations only occur at well-defined safe points. This eliminates the possibility of crashes and corruption while maintaining zero overhead during @@ -97,7 +97,7 @@ already `been implemented in PyPy `__, proving both its feasibility and effectiveness. Their implementation demonstrates that we can provide safe debugging capabilities with zero runtime overhead during normal execution. The proposed mechanism not only reduces risks -associated with current debugging practices but also lays the foundation for +associated with current debugging approaches but also lays the foundation for future enhancements. For instance, this framework could enable integration with popular observability tools, providing real-time insights into interpreter performance or memory usage. One compelling use case for this interface is @@ -120,7 +120,7 @@ state to coordinate debugging operations. The mechanism works by having debuggers write to specific memory locations in the target process that the interpreter then checks during its normal execution -cycle. When the interpreter detects a debugger wants to attach, it executes the +cycle. When the interpreter detects that a debugger wants to attach, it executes the requested operations only when it's safe to do so - that is, when no internal locks are held and all data structures are in a consistent state. @@ -160,16 +160,18 @@ debugger support: .. code-block:: C struct _debugger_support { - uint64_t eval_breaker; /* Location of the eval breaker flag */ - uint64_t remote_debugger_support; /* Offset to our support structure */ - uint64_t debugger_pending_call; /* Where to write the pending flag */ - uint64_t debugger_script; /* Where to write the script */ + uint64_t eval_breaker; // Location of the eval breaker flag + uint64_t remote_debugger_support; // Offset to our support structure + uint64_t debugger_pending_call; // Where to write the pending flag + uint64_t debugger_script; // Where to write the script } debugger_support; These offsets allow debuggers to locate critical debugging control structures in -the target process's memory space. The offsets are relative to the relevant -structure address, making them valid regardless of where structures are actually -loaded in memory. +the target process's memory space. The ``eval_breaker`` and ``remote_debugger_support`` +offsets are relative to each ``PyThreadState``, while the ``debugger_pending_call`` +and ``debugger_script`` offsets are relative to each ``_PyRemoteDebuggerSupport`` +structure, allowing the new structure and its fields to be found regardless of +where they are in memory. Attachment Protocol ------------------- @@ -178,39 +180,43 @@ When a debugger wants to attach to a Python process, it follows these steps: 1. Locate ``PyRuntime`` structure in the process: - Find Python binary (executable or libpython) in process memory (OS dependent process) - Extract ``.PyRuntime`` section offset from binary's format (ELF/Mach-O/PE) - - Calculate the actual ``PyRuntime`` address in the running process by relocating the offset to the binary's load address + - Calculate the actual ``PyRuntime`` address in the running process by relocating the offset to the binary's load address -2. Access debug offset information by read ``_Py_DebugOffsets`` table from located ``PyRuntime`` structure. - -3. Use the offsets to locate the debugger interface structure withing the desired thread state. +2. Access debug offset information by reading the ``_Py_DebugOffsets`` at the start of the ``PyRuntime`` structure. -4. Write control information: - - Write python code to be executed. +3. Use the offsets to locate the desired thread state + +4. Use the offsets to locate the debugger interface fields within that thread state + +5. Write control information: + - Write python code to be executed into the ``debugger_script`` field in ``_PyRemoteDebuggerSupport`` - Set ``debugger_pending_call`` flag in ``_PyRemoteDebuggerSupport`` - Set ``_PY_EVAL_PLEASE_STOP_BIT`` in the ``eval_breaker`` field - - Wait for the interpreter to reach next safe point and execute debugger code + +Once the interpreter reaches the next safe point, it will execute the script +provided by the debugger. Interpreter Integration ----------------------- The interpreter's regular evaluation loop already includes a check of the -eval_breaker flag for handling signals, periodic tasks, and other interrupts. We +``eval_breaker`` flag for handling signals, periodic tasks, and other interrupts. We leverage this existing mechanism by checking for debugger pending calls only when the ``eval_breaker`` is set, ensuring zero overhead during normal execution. -This check has no overhead. Indeed, profiling with Linux perf shows this branch +This check has no overhead. Indeed, profiling with Linux ``perf`` shows this branch is highly predictable - the ``debugger_pending_call`` check is never taken during normal execution, allowing modern CPUs to effectively speculate past it. When a debugger has set both the ``eval_breaker`` flag and ``debugger_pending_call``, the interpreter will execute the provided debugging code at the next safe point -and executes the provided code. This all happens in a completely safe context as -any of the operations that happen in the eval breaker as the interpreter is in a -consistent state: +and executes the provided code. This all happens in a completely safe context, since +the interpreter is guaranteed to be in a consistent state whenever the eval breaker +is checked. .. code-block:: c - /* In ceval.c */ + // In ceval.c if (tstate->eval_breaker) { if (tstate->remote_debugger_support.debugger_pending_call) { tstate->remote_debugger_support.debugger_pending_call = 0; @@ -228,27 +234,29 @@ Python API ---------- To support safe execution of Python code in a remote process without having to -re-implement all these steps in every tool, this proposal extends the sys module +re-implement all these steps in every tool, this proposal extends the ``sys`` module with a new function. This function allows debuggers or external tools to execute arbitrary Python code within the context of a specified Python process: .. code-block:: python def remote_exec(pid: int, code: str) -> None: - Executes a block of Python code in a remote Python process, identified by its process ID. + """ + Executes a block of Python code in a given remote Python process. - Args: - pid (int): The process ID of the target Python interpreter. - code (str): A string containing the Python code to be executed. + Args: + pid (int): The process ID of the target Python process. + code (str): A string containing the Python code to be executed. + """ An example usage of the API would look like: .. code-block:: python - import sys + import sys # Execute a print statement in a remote Python process with PID 12345 try: - sys.remote_execute(12345, "print('Hello from remote execution!')") + sys.remote_exec(12345, "print('Hello from remote execution!')") except Exception as e: print(f"Failed to execute code: {e}") @@ -274,8 +282,9 @@ debuggers and tools. Some examples are: are used to read and write memory from another process. These operations are controlled by ptrace access mode checks - the same ones that govern debugger attachment. A process can only read from or write to another process's memory - if it has the appropriate permissions (typically requiring the same user ID as - the target process or ``CAP_SYS_PTRACE`` capability). + if it has the appropriate permissions (typically requiring either root or the + ``CAP_SYS_PTRACE`` capability, though less security minded distributions may + allow any process running as the same uid to attach). * On macOS, the interface would leverage ``mach_vm_read_overwrite()`` and ``mach_vm_write()`` through the Mach task system. These operations require @@ -319,7 +328,7 @@ How to Teach This ================= For tool authors, this interface becomes the standard way to implement debugger -attachment, replacing unsafe system debugger approaches.A section in the Python +attachment, replacing unsafe system debugger approaches. A section in the Python Developer Guide could describe the internal workings of the mechanism, including the ``debugger_support`` offsets and how to interact with them using system APIs. @@ -337,4 +346,4 @@ Copyright ========= This document is placed in the public domain or under the CC0-1.0-Universal -license, whichever is more permissive. \ No newline at end of file +license, whichever is more permissive. From 16f0d5bc1c448cf4a435218645dfaac3a018dc7e Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Fri, 6 Dec 2024 16:58:36 +0000 Subject: [PATCH 03/17] Add links --- peps/pep-0778.rst | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/peps/pep-0778.rst b/peps/pep-0778.rst index 9391681d01e..8d1bb0c5406 100644 --- a/peps/pep-0778.rst +++ b/peps/pep-0778.rst @@ -55,7 +55,8 @@ counting system. This combination of low-level system manipulation and deep domain specific interpreter knowledge makes implementing Python debugging tools exceptionally difficult. -The few tools that do attempt this resort to suboptimal and unsafe methods, +The few tools ( see for example [1]_ and [2]_ )that do attempt this resort to +suboptimal and unsafe methods, using system debuggers like gdb and lldb to forcefully inject code. This approach is fundamentally unsafe because the injected code can execute at any point during the interpreter's execution cycle - even during critical operations @@ -342,6 +343,15 @@ Reference Implementation https://github.com/pablogsal/cpython/commits/remote_pdb/ +References +========== + +.. [1] DebugPy + https://github.com/microsoft/debugpy/blob/43f41029eabce338becbd1fa1a09727b3cfb1140/src/debugpy/_vendored/pydevd/pydevd_attach_to_process/linux_and_mac/attach.cpp#L4 + +.. [2] Memray + https://github.com/bloomberg/memray/blob/main/src/memray/_memray/inject.cpp + Copyright ========= From a85ea657d2b8153fbe1a69dc5942d57f4ed3a396 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Salgado Date: Fri, 6 Dec 2024 18:30:15 +0000 Subject: [PATCH 04/17] Update pep-0778.rst Co-authored-by: Jelle Zijlstra --- peps/pep-0778.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0778.rst b/peps/pep-0778.rst index 8d1bb0c5406..fd69360e3b8 100644 --- a/peps/pep-0778.rst +++ b/peps/pep-0778.rst @@ -55,7 +55,7 @@ counting system. This combination of low-level system manipulation and deep domain specific interpreter knowledge makes implementing Python debugging tools exceptionally difficult. -The few tools ( see for example [1]_ and [2]_ )that do attempt this resort to +The few tools (see for example [1]_ and [2]_) that do attempt this resort to suboptimal and unsafe methods, using system debuggers like gdb and lldb to forcefully inject code. This approach is fundamentally unsafe because the injected code can execute at any From 1a68c39a9cfc9260da810c3adde526f43ab1dfdf Mon Sep 17 00:00:00 2001 From: Pablo Galindo Salgado Date: Fri, 6 Dec 2024 18:30:22 +0000 Subject: [PATCH 05/17] Update pep-0778.rst Co-authored-by: Jelle Zijlstra --- peps/pep-0778.rst | 1 - 1 file changed, 1 deletion(-) diff --git a/peps/pep-0778.rst b/peps/pep-0778.rst index fd69360e3b8..f538bae1e8c 100644 --- a/peps/pep-0778.rst +++ b/peps/pep-0778.rst @@ -1,7 +1,6 @@ PEP: 778 Title: Safe external debugger interface for CPython Author: Pablo Galindo Salgado , Matt Wozniski , Ivona Stojanovic -Sponsor: Pablo Galindo Salgado Discussions-To: xxxxx Status: Draft Type: Standards Track From 728d583df4013b25c9437376dc3ebec5ea442c14 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Salgado Date: Fri, 6 Dec 2024 18:30:27 +0000 Subject: [PATCH 06/17] Update pep-0778.rst Co-authored-by: Jelle Zijlstra --- peps/pep-0778.rst | 1 - 1 file changed, 1 deletion(-) diff --git a/peps/pep-0778.rst b/peps/pep-0778.rst index f538bae1e8c..fa12bc556a2 100644 --- a/peps/pep-0778.rst +++ b/peps/pep-0778.rst @@ -86,7 +86,6 @@ operations only occur at well-defined safe points. This eliminates the possibility of crashes and corruption while maintaining zero overhead during normal execution. - The key insight is that we don't need to inject code at arbitrary points - we just need to signal to the interpreter that we want code executed at the next safe opportunity. This approach works with the interpreter's natural execution From eefe6cc8ae5f7f9de28515a229eb6b6992bbd8f8 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Salgado Date: Sat, 7 Dec 2024 00:24:47 +0000 Subject: [PATCH 07/17] Update pep-0778.rst Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> --- peps/pep-0778.rst | 1 - 1 file changed, 1 deletion(-) diff --git a/peps/pep-0778.rst b/peps/pep-0778.rst index fa12bc556a2..e601d260018 100644 --- a/peps/pep-0778.rst +++ b/peps/pep-0778.rst @@ -4,7 +4,6 @@ Author: Pablo Galindo Salgado , Matt Wozniski Date: Sat, 7 Dec 2024 00:24:59 +0000 Subject: [PATCH 08/17] Update pep-0778.rst Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> --- peps/pep-0778.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0778.rst b/peps/pep-0778.rst index e601d260018..a736b7e7c5f 100644 --- a/peps/pep-0778.rst +++ b/peps/pep-0778.rst @@ -32,7 +32,7 @@ inspecting memory usage, or investigating unexpected behavior in real-time. Very few Python tools can attach to running processes, primarily because doing so requires deep expertise in both operating system debugging interfaces and -CPython internals. While C/C++ debuggers like gdb and lldb can attach to +CPython internals. While C/C++ debuggers like GDB and LLDB can attach to processes using well-understood techniques, Python tools must implement all of these low-level mechanisms plus handle additional complexity. For example, when gdb needs to execute code in a target process, it: From e8b1f097fd792c49cf6d528ef6f9903de022ab07 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Salgado Date: Sat, 7 Dec 2024 00:25:06 +0000 Subject: [PATCH 09/17] Update pep-0778.rst Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> --- peps/pep-0778.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0778.rst b/peps/pep-0778.rst index a736b7e7c5f..a52b6d34dfe 100644 --- a/peps/pep-0778.rst +++ b/peps/pep-0778.rst @@ -35,7 +35,7 @@ so requires deep expertise in both operating system debugging interfaces and CPython internals. While C/C++ debuggers like GDB and LLDB can attach to processes using well-understood techniques, Python tools must implement all of these low-level mechanisms plus handle additional complexity. For example, when -gdb needs to execute code in a target process, it: +GDB needs to execute code in a target process, it: 1. Uses ptrace to allocate a small chunk of executable memory (easier said than done) 2. Writes a small sequence of machine code - typically a function prologue, the From b03dbded87867e9f0ecfbddd0c7c2389e109f2d5 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Salgado Date: Sat, 7 Dec 2024 00:25:15 +0000 Subject: [PATCH 10/17] Update pep-0778.rst Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> --- peps/pep-0778.rst | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/peps/pep-0778.rst b/peps/pep-0778.rst index a52b6d34dfe..65736e7ab1d 100644 --- a/peps/pep-0778.rst +++ b/peps/pep-0778.rst @@ -53,8 +53,11 @@ counting system. This combination of low-level system manipulation and deep domain specific interpreter knowledge makes implementing Python debugging tools exceptionally difficult. -The few tools (see for example [1]_ and [2]_) that do attempt this resort to -suboptimal and unsafe methods, +The few tools (see for example `DebugPy +`__ +and `Memray +`__) +that do attempt this resort to suboptimal and unsafe methods, using system debuggers like gdb and lldb to forcefully inject code. This approach is fundamentally unsafe because the injected code can execute at any point during the interpreter's execution cycle - even during critical operations From 8cfa2aea659fab3b0743325155b6185d02d9a3f1 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Salgado Date: Sat, 7 Dec 2024 00:25:21 +0000 Subject: [PATCH 11/17] Update pep-0778.rst Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> --- peps/pep-0778.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0778.rst b/peps/pep-0778.rst index 65736e7ab1d..3e8902031b0 100644 --- a/peps/pep-0778.rst +++ b/peps/pep-0778.rst @@ -58,7 +58,7 @@ The few tools (see for example `DebugPy and `Memray `__) that do attempt this resort to suboptimal and unsafe methods, -using system debuggers like gdb and lldb to forcefully inject code. This +using system debuggers like GDB and LLDB to forcefully inject code. This approach is fundamentally unsafe because the injected code can execute at any point during the interpreter's execution cycle - even during critical operations like memory allocation, garbage collection, or thread state management. When From b1b72a9f8f68c2f0f8ca735231e2e1afcbdb2e3d Mon Sep 17 00:00:00 2001 From: Pablo Galindo Salgado Date: Sat, 7 Dec 2024 00:25:37 +0000 Subject: [PATCH 12/17] Update pep-0778.rst Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> --- peps/pep-0778.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/peps/pep-0778.rst b/peps/pep-0778.rst index 3e8902031b0..bfd7c335b1a 100644 --- a/peps/pep-0778.rst +++ b/peps/pep-0778.rst @@ -179,6 +179,7 @@ Attachment Protocol When a debugger wants to attach to a Python process, it follows these steps: 1. Locate ``PyRuntime`` structure in the process: + - Find Python binary (executable or libpython) in process memory (OS dependent process) - Extract ``.PyRuntime`` section offset from binary's format (ELF/Mach-O/PE) - Calculate the actual ``PyRuntime`` address in the running process by relocating the offset to the binary's load address From e200b41c011ed1c021033be446746bd754ab5324 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Salgado Date: Sat, 7 Dec 2024 00:25:44 +0000 Subject: [PATCH 13/17] Update pep-0778.rst Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> --- peps/pep-0778.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/peps/pep-0778.rst b/peps/pep-0778.rst index bfd7c335b1a..745cd7c7164 100644 --- a/peps/pep-0778.rst +++ b/peps/pep-0778.rst @@ -191,6 +191,7 @@ When a debugger wants to attach to a Python process, it follows these steps: 4. Use the offsets to locate the debugger interface fields within that thread state 5. Write control information: + - Write python code to be executed into the ``debugger_script`` field in ``_PyRemoteDebuggerSupport`` - Set ``debugger_pending_call`` flag in ``_PyRemoteDebuggerSupport`` - Set ``_PY_EVAL_PLEASE_STOP_BIT`` in the ``eval_breaker`` field From 7b4db66ed9e9b4ba2a6a1f5101b200e10af1993d Mon Sep 17 00:00:00 2001 From: Pablo Galindo Salgado Date: Sat, 7 Dec 2024 00:25:59 +0000 Subject: [PATCH 14/17] Update pep-0778.rst Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> --- peps/pep-0778.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0778.rst b/peps/pep-0778.rst index 745cd7c7164..c3a49d4491a 100644 --- a/peps/pep-0778.rst +++ b/peps/pep-0778.rst @@ -309,7 +309,7 @@ All mechanisms ensure that: 3. No additional attack surface is exposed beyond what the OS already provides for debugging The memory operations themselves are well-established and have been used safely -for decades in tools like gdb, lldb, and various system profilers. +for decades in tools like GDB, LLDB, and various system profilers. It’s important to note that any attempt to attach to a Python process via this mechanism would be detectable by system-level monitoring tools. This From 9ccf65e2c7218d269ea5ccfbe1cf0182e987c35b Mon Sep 17 00:00:00 2001 From: Pablo Galindo Salgado Date: Sat, 7 Dec 2024 00:26:12 +0000 Subject: [PATCH 15/17] Update pep-0778.rst Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> --- peps/pep-0778.rst | 9 --------- 1 file changed, 9 deletions(-) diff --git a/peps/pep-0778.rst b/peps/pep-0778.rst index c3a49d4491a..f4224d9af11 100644 --- a/peps/pep-0778.rst +++ b/peps/pep-0778.rst @@ -345,15 +345,6 @@ Reference Implementation https://github.com/pablogsal/cpython/commits/remote_pdb/ -References -========== - -.. [1] DebugPy - https://github.com/microsoft/debugpy/blob/43f41029eabce338becbd1fa1a09727b3cfb1140/src/debugpy/_vendored/pydevd/pydevd_attach_to_process/linux_and_mac/attach.cpp#L4 - -.. [2] Memray - https://github.com/bloomberg/memray/blob/main/src/memray/_memray/inject.cpp - Copyright ========= From 982284eea59d25062f86a02ab7f991c250ecd19a Mon Sep 17 00:00:00 2001 From: Pablo Galindo Salgado Date: Sat, 7 Dec 2024 14:41:34 +0000 Subject: [PATCH 16/17] Update pep-0778.rst Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> --- peps/pep-0778.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0778.rst b/peps/pep-0778.rst index f4224d9af11..4017c535118 100644 --- a/peps/pep-0778.rst +++ b/peps/pep-0778.rst @@ -1,4 +1,4 @@ -PEP: 778 +PEP: 768 Title: Safe external debugger interface for CPython Author: Pablo Galindo Salgado , Matt Wozniski , Ivona Stojanovic Discussions-To: xxxxx From 4c81a923c438a3857767f24f6eba97d11ee31d0f Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Mon, 9 Dec 2024 00:16:58 +0000 Subject: [PATCH 17/17] Update pep number --- peps/{pep-0778.rst => pep-0768.rst} | 1 - 1 file changed, 1 deletion(-) rename peps/{pep-0778.rst => pep-0768.rst} (99%) diff --git a/peps/pep-0778.rst b/peps/pep-0768.rst similarity index 99% rename from peps/pep-0778.rst rename to peps/pep-0768.rst index 4017c535118..6d347ba3f25 100644 --- a/peps/pep-0778.rst +++ b/peps/pep-0768.rst @@ -1,7 +1,6 @@ PEP: 768 Title: Safe external debugger interface for CPython Author: Pablo Galindo Salgado , Matt Wozniski , Ivona Stojanovic -Discussions-To: xxxxx Status: Draft Type: Standards Track Created: 25-Nov-2024