PEP 796: relative virtual environments, initial draft

.github/CODEOWNERS

@@ -672,6 +672,7 @@ peps/pep-0791.rst  @vstinner
 peps/pep-0792.rst  @dstufft
 peps/pep-0793.rst  @encukou
 peps/pep-0794.rst  @brettcannon
+peps/pep-0796.rst  @ncoghlan
 # ...
 peps/pep-0801.rst  @warsaw
 # ...

peps/pep-0796.rst

@@ -0,0 +1,263 @@
+PEP: 796
+Title: Relative Virtual Environment Home
+Author: Richard Levasseur <[email protected]>
+Sponsor: Alyssa Coghlan <[email protected]>
+Discussions-To: Pending
+Status: Draft
+Type: Standards Track
+Created: 26-Jun-2025
+Python-Version: 3.15
+
+.. highlight:: rst
+
+
+Abstract
+========
+
+This PEP describes how a relative path for ``home`` in a Python virtual
+environment's ``pyvenv.cfg`` is understood by the Python startup process.
+Specifically, how it is canonicalized into an absolute path later used
+by the runtime. This small detail is a fundamental building block for
+virtual environments to be more portable.
+
+Motivation
+==========
+
+There are two main motivations for allowing relative paths in ``pyvenv.cfg``.
+
+First, it is currently prescribed that the ``home`` value in ``pyvenv.cfg`` be
+an absolute path. The behavior of relative paths is unspecified. While
+techniques exist to work around this for every other sub-part of a virtual
+environment, the one remaining part without a tenable solution is how the
+Python runtime itself finds ``PYTHONHOME``. This is because, currently, the
+startup process requires absolute paths be used for the ``home`` key in
+``pyvenv.cfg``. If a relative path is used, behavior is unspecified (the
+current implementation ends up making it relative to the process's current
+working directory, making it untenable to use).
+
+This requirement is overly proscriptive and restrictive because, given a known
+anchor point, it's easy to transform a relative path to an absolute path and
+still retain predictable and reliable behavior. Thus, the absolute path
+requirement should be relaxed and relative path behavior allowed and defined.
+
+Second, such relative paths are a building block to enable portable virtual
+environments, i.e. copying a virtual environment as-is between hosts of
+compatible platforms. Portable venvs are appealing because virtual environments
+are a popular mechanism for running Python applications. This provides several
+benefits:
+
+* The closer the development environment is to the non-development environment,
+  the more reliable software can be achieved, and the easier it is to reproduce
+  issues
+* The simpler the process of re-creating the environment, the more reliable
+  software can be achieved, and the faster the process can be.
+
+Making it simpler to copy a virtual environment from one host to another
+mitigates these categories of problems. Additionally, the development tools to
+create a virtual environment and install its dependencies aren't needed on the
+host that intends to run the program.
+
+When the virtual environment doesn't require modifications to be usable, it
+also allows more advanced deployment mechanisms, e.g. remote mounting and
+caching of artifacts. While this PEP on its own isn't sufficient to enable
+that, it allows tools like ``bazel`` or ``venvstacks`` to more easily prepare
+constrained environments that allow for such use cases.
+
+Rationale
+=========
+
+The reason support for relative virtual environments needs to be
+in the interpreter itself is because locating ``PYTHONHOME`` happens
+very early in the interpreter startup process, which limits the options for
+customizing how it's computed. Without the ability to specify where the
+supporting Python runtime files are, the interpreter can't finish startup,
+so other hook points (e.g. ``site`` initialization) never trigger.
+
+Tools that currently look to enable virtual environment portability across
+machines do so either by relying on undocumented interpreter behaviour
+(``bazel``, omitting the ``home`` key entirely to trigger an implementation
+dependent fallback to resolving via a symlinked interpreter binary on
+non-Windows systems) or by requiring a post-installation script to be executed
+after the environment is placed in its target location (``venvstacks``).
+
+Specification
+=============
+
+The ``home`` value in ``pyvenv.cfg`` is permitted to use a relative path value.
+These may contain up-references outside of the virtual environment root
+directory.  Examples:
+
+* ``subdir/whatever/bin`` (a directory within the virtual environment).
+* ``./subdir/whatever/bin`` (same as above)
+* ``../../../../../elsewhere/runtime/bin`` (a directory outside the virtual
+  environment).
+
+Relative paths are relative to the directory containing ``pyvenv.cfg``. During
+interpreter startup (i.e. ``getpath.py``), the relative path is joined to the
+directory to form an absolute path. Up-references (``../``) and current
+directory references (``./``) are resolved syntactically (i.e. not resolving
+symlinks). Symlinks are *not* resolved prior to construction of the absolute
+path to ensure semantics between a relative path and absolute path remain the
+same.
+
+For example, given
+``/home/user/venv/bin/pyvenv.cfg`` with
+``home = ../../runtime/./bin``, the result is ``home = /home/user/runtime/bin``,
+i.e. it's equivalent to using that value verbatim in ``pyvenv.cfg``.
+
+
+CPython Runtime Changes
+=======================
+
+The CPython runtime itself *almost* already supports relative paths. The
+primitives are there, so the only change needed is to define how it resolves
+relative paths for ``home`` in ``pyvenv.cfg``.
+
+Currently, relative paths resolve relative to the process's current working
+directory. Because CWD isn't knowable in advance, it makes relative paths today
+effectively impossible.
+
+Instead, the paths should be relative to the location of the ``pyvenv.cfg``
+file. This file is chosen as the anchor point because the tool that creates the
+file also has to know where the Python runtime is, so can easily calculate the
+correct relative path. For tools that read the ``pyvenv.cfg``, it is also easy
+to simply join the directory name of where ``pyvenv.cfg`` was found with the
+path in the config file. When a person reads the config file, they can do
+something similar, which results in a lower cognitive burden and helps avoid
+the question of "relative to what?"
+
+This change is only a couple of lines in the startup code. Specifically, when
+parsing the ``pyvenv.cfg`` file and finding the ``home`` value, it just needs
+to be checked if it's already absolute. If not, then join it to the directory
+name of the ``pyvenv.cfg`` file. The code already knows the directory and has
+helpers already present for checking if a path is absolute and joining two
+paths.
+
+A proof-of-concept of this is implemented in
+`rickeylev/feat.relative.pyvenv.home <https://github.com/python/cpython/compare/main...rickeylev:cpython:feat.relative.pyvenv.home>`__
+
+Backwards Compatibility
+=======================
+
+
+Tools that work around the absolute ``home`` key limitation the way ``bazel``
+and ``venvstacks`` currently do (omitting the ``home`` key, or editing it after
+moving the environment) will be unaffected.
+
+While the PEP author and sponsor aren't aware of any projects that work around
+the limitation by carefully controlling the current working directory used to
+launch the deployed Python environments on target systems, any such projects
+would be unaffected if they already ensured the working directory was set to
+the folder containing ``pyvenv.cfg`` (which seems like a plausible choice,
+since that is typically the root directory of the virtual environment). In the
+even more unlikely case where that assumption doesn't hold, tools generating
+relative virtual environment paths will typically be aware of the underlying
+base runtime Python version, and hence able to update the emitted relative path
+accordingly.
+
+
+How to Teach This
+=================
+
+Teaching this should be simple: if you use a relative path in ``pyvenv.cfg``,
+then it's relative to the directory containing the ``pyvenv.cfg`` file. This
+is simple to explain and understand.
+
+
+Reference Implementation
+========================
+
+A reference implementation is available by using the combination of:
+
+* Python runtime from `rickeylev/feat.relative.pyvenv.home <https://github.com/python/cpython/compare/main...rickeylev:cpython:feat.relative.pyvenv.home>`__
+* Relative venv from `rickeylev/relvenv <https://github.com/rickeylev/relvenv>`__
+
+And following the
+`relvenv README <https://github.com/rickeylev/relvenv/blob/main/README.md>`__.
+
+Open Issues
+===========
+
+This PEP does not specify how to create a ``pyvenv.cfg`` with a relative path,
+nor how downstream tools (e.g. installers) should identify them or process
+them. These questions are best addressed separately by tool owners.
+
+References
+==========
+
+* `rules_python <https://github.com/bazel-contrib/rules_python>`__: implements
+  host-relocatable virtual environments.
+* `rules_py <https://github.com/aspect-build/rules_py>`__: implements
+  host-relocatable virtual environments.
+* `python-build-standalone <https://github.com/astral-sh/python-build-standalone>`__:
+* `venvstacks <https://pypi.org/project/venvstacks/>`__: a tool for creating reproducible distribution artifacts from virtual environments
+  A relocatable Python runtime.
+* `PoC for relative home in Python startup <https://github.com/python/cpython/compare/main...rickeylev:cpython:feat.relative.pyvenv.home>`__
+* `Python Ideas "Making venvs relocatable friendly" discussion <https://discuss.python.org/t/making-venvs-relocatable-friendly/96177>`__
+* `GH-136051: relative pyvenv.cfg home <https://github.com/python/cpython/issues/136051>`__
+
+Rejected Ideas
+=====================
+
+Relative to virtual env root
+----------------------------
+
+Having the ``home`` value in ``pyvenv.cfg`` relative to the virtual
+environment's root directory would work just as well, but this idea is rejected
+because it requires additional effort to compute the virtual env root.
+
+Unspecified home means to dynamically compute home
+----------------------------------------------------
+
+Today, if a ``pyvenv.cfg`` file doesn't set ``home``, the runtime will try to
+dynamically compute it by checking if the current executable (which is
+typically the venv's ``bin/python3`` symlink) is a symlink and, if so, use
+where that points as ``PYTHONHOME``.
+
+While currently used as a workaround by some tools, *standardising* this
+behavior is undesirable for a couple reasons:
+
+1. It presents platform-specific issues, namely with Windows. Windows does
+   support symlinks, but not by default, and it can require special
+   permissions to do so.
+2. It *requires* that a symlink be used, which precludes using otherwise
+   equivalent mechanisms for creating an executable (e.g. a wrapper script,
+   hard links, etc).
+
+In general, symlinks work best when they aren't special cased by consumers.
+
+Using the term "relocatable"
+----------------------------
+
+Discussions pointed out that the term "relocatable" is somewhat ambiguous and
+misleading for a couple reasons.
+
+First, absolute paths make a venv arbitrarily relocatable *within* a host, but
+not between hosts, so "relocatable" requires *some* qualification for
+clarity.
+
+Second, when using relative paths that point outside the venv, the venv is only
+relocatable insofar as those external artifacts are also relocated. This is an
+additional nuance that requires qualification of the term.
+
+To better avoid this confusion, "relative" is chosen, which more naturally
+invites the question *"Relative to what?"*.
+
+
+Using PYTHONHOME at runtime to specify home
+-------------------------------------------
+
+Using the ``PYTHONHOME`` environment variable (or any environment variable) is
+problematic because it's difficult to know and control when an environment
+variable should or shouldn't be inherited by subprocesses. In some cases, it's
+not feasible because of how layers of programs calling programs interact.
+
+Code generally assumes that any virtual environment will be
+automatically detected and activated by the presence of ``pyvenv.cfg``, so
+things work better when alterations to the environment aren't a concern.
+
+Copyright
+=========
+
+This document is placed in the public domain or under the
+CC0-1.0-Universal license, whichever is more permissive.