From ceeaa3ba3256923ca2bec94d0d3265fbdb928eab Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 27 Jun 2025 13:43:14 -0700 Subject: [PATCH 01/29] initial file --- peps/pep-0796.rst | 806 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 806 insertions(+) create mode 100644 peps/pep-0796.rst diff --git a/peps/pep-0796.rst b/peps/pep-0796.rst new file mode 100644 index 00000000000..6691584b4f6 --- /dev/null +++ b/peps/pep-0796.rst @@ -0,0 +1,806 @@ +PEP: 796 +Title: Host-relocatable virtual environments +Author: Richard Levasseur +Discussions-To: TODO +Status: Draft +Type: Standards Track +Topic: Packaging +Created: 06-26-2025 +Python-Version: 3.15 +Post-History: `DD-MM-YYYY `__ + +.. highlight:: rst + + +Abstract +======== + +This PEP describes how a Python virtual environment can be configured +to allow copying it as-is to different hosts and still have a functioning +virtual environment. Making this work consists of two main pieces: + +* Using a relative path for `home` in `pyvenv.cfg`, which Python canonicalizes + at runtime. +* Having package installers recognize a host-relocatable venv and generate + appropriate e.g. script wrappers. + + +Rationale +========= + +There are two primary rationales for host-relocatable virtual environments. + +First, it is currently proscribed that the `home` value in `pyvenv.cfg` be an +absolute path. The behavior of relative paths is unspecified. This requirement +is overly proscriptive and restrictive because, given a known anchor point, +it's easy to tranform a relative path to absolute path and still retain +expected behavior. Thus, this requirement should be lifted. + +Second, virtual environments are the standard way for running Python +applications, which means how they're created is important. This means +recreating a virtual environment for each instance ... + +When they need to be recreated ... + +Currently, virtual environments are effectively tied to the particular host +that the virtual environment is originally created on. This is because symlinks +to absolute paths are commonly used in various places (e.g. `bin/python3`), +which are frequently specific to a host (e.g. by containing a user's home +directory). + + + +How to Use This Template +======================== + +To use this template you must first decide whether your PEP is going +to be an Informational or Standards Track PEP. Most PEPs are +Standards Track because they propose a new feature for the Python +language or standard library. When in doubt, read :pep:`1` for details, +or open a tracker issue on the PEPs repo to ask for assistance. + +Once you've decided which type of PEP yours is going to be, follow the +directions below. + +- Make a copy of this file (the ``.rst`` file, **not** the HTML!) and + perform the following edits. Name the new file :file:`pep-{NNNN}.rst`, using + the next available number (not used by a published or in-PR PEP). + +- Replace the "PEP: 12" header with "PEP: NNNN", + matching the file name. Note that the file name should be padded with + zeros (eg ``pep-0012.rst``), but the header should not (``PEP: 12``). + +- Change the Title header to the title of your PEP. + +- Change the Author header to include your name, and optionally your + email address. Be sure to follow the format carefully: your name + must appear first, and it must not be contained in parentheses. + Your email address may appear second (or it can be omitted) and if + it appears, it must appear in angle brackets. It is okay to + obfuscate your email address. + +- If none of the authors are Python core developers, include a Sponsor + header with the name of the core developer sponsoring your PEP. + +- Add the direct URL of the PEP's canonical discussion thread + (on e.g. Python-Dev, Discourse, etc) under the Discussions-To header. + If the thread will be created after the PEP is submitted as an official + draft, it is okay to just put "Pending" initially, but remember to + update the PEP with the URL as soon as the PEP is successfully merged + to the PEPs repository and you create the corresponding discussion thread. + See :pep:`PEP 1 <1#discussing-a-pep>` for more details. + +- Change the Status header to "Draft". + +- For Standards Track PEPs, change the Type header to "Standards + Track". + +- For Informational PEPs, change the Type header to "Informational". + +- For Standards Track PEPs, if your feature depends on the acceptance + of some other currently in-development PEP, add a Requires header + right after the Type header. The value should be the PEP number of + the PEP yours depends on. Don't add this header if your dependent + feature is described in a Final PEP. + +- Change the Created header to today's date. Be sure to follow the + format carefully: it must be in ``dd-mmm-yyyy`` format, where the + ``mmm`` is the 3 English letter month abbreviation, i.e. one of Jan, + Feb, Mar, Apr, May, Jun, Jul, Aug, Sep, Oct, Nov, Dec. + +- For Standards Track PEPs, after the Created header, add a + Python-Version header and set the value to the next planned version + of Python, i.e. the one your new feature will hopefully make its + first appearance in. Do not use an alpha or beta release + designation here. Thus, if the last version of Python was 2.2 alpha + 1 and you're hoping to get your new feature into Python 2.2, set the + header to: + + .. code-block:: email + + Python-Version: 2.2 + +- Add a Topic header if the PEP belongs under one shown at the :ref:`topic-index`. + Most PEPs don't. + +- Leave Post-History alone for now; you'll add dates and corresponding links + to this header each time you post your PEP to the designated discussion forum + (and update the Discussions-To header with said link, as above). + For each thread, use the date (in the ``dd-mmm-yyy`` format) as the + linked text, and insert the URLs inline as anonymous reST `hyperlinks`_, + with commas in between each posting. + + If you posted threads for your PEP on August 14, 2001 and September 3, 2001, + the Post-History header would look like, e.g.: + + .. code-block:: email + + Post-History: `14-Aug-2001 `__, + `03-Sept-2001 `__ + + You should add the new dates/links here as soon as you post a + new discussion thread. + +- Add a Replaces header if your PEP obsoletes an earlier PEP. The + value of this header is the number of the PEP that your new PEP is + replacing. Only add this header if the older PEP is in "final" + form, i.e. is either Accepted, Final, or Rejected. You aren't + replacing an older open PEP if you're submitting a competing idea. + +- Now write your Abstract, Rationale, and other content for your PEP, + replacing all this gobbledygook with your own text. Be sure to + adhere to the format guidelines below, specifically on the + prohibition of tab characters and the indentation requirements. + See "Suggested Sections" below for a template of sections to include. + +- Update your Footnotes section, listing any footnotes and + non-inline link targets referenced by the text. + +- Run ``./build.py`` to ensure the PEP is rendered without errors, + and check that the output in :file:`build/pep-{NNNN}.html` looks as you intend. + +- Create a pull request against the `PEPs repository`_. + +For reference, here are all of the possible header fields (everything +in brackets should either be replaced or have the field removed if +it has a leading ``*`` marking it as optional and it does not apply to +your PEP): + +.. code-block:: email + + PEP: [NNN] + Title: [...] + Author: [Full Name ] + Sponsor: *[Full Name ] + PEP-Delegate: + Discussions-To: [URL] + Status: Draft + Type: [Standards Track | Informational | Process] + Topic: *[Governance | Packaging | Release | Typing] + Requires: *[NNN] + Created: [DD-MMM-YYYY] + Python-Version: *[M.N] + Post-History: [`DD-MMM-YYYY `__] + Replaces: *[NNN] + Superseded-By: *[NNN] + Resolution: + + +ReStructuredText PEP Formatting Requirements +============================================ + +The following is a PEP-specific summary of reStructuredText syntax. +For the sake of simplicity and brevity, much detail is omitted. For +more detail, see `Resources`_ below. `Literal blocks`_ (in which no +markup processing is done) are used for examples throughout, to +illustrate the plaintext markup. + + +General +------- + +Lines should usually not extend past column 79, +excepting URLs and similar circumstances. +Tab characters must never appear in the document at all. + + +Section Headings +---------------- + +PEP headings must begin in column zero and the initial letter of each +word must be capitalized as in book titles. Acronyms should be in all +capitals. Section titles must be adorned with an underline, a single +repeated punctuation character, which begins in column zero and must +extend at least as far as the right edge of the title text (4 +characters minimum). First-level section titles are underlined with +"=" (equals signs), second-level section titles with "-" (hyphens), +and third-level section titles with "'" (single quotes or +apostrophes). For example:: + + First-Level Title + ================= + + Second-Level Title + ------------------ + + Third-Level Title + ''''''''''''''''' + +If there are more than three levels of sections in your PEP, you may +insert overline/underline-adorned titles for the first and second +levels as follows:: + + ============================ + First-Level Title (optional) + ============================ + + ----------------------------- + Second-Level Title (optional) + ----------------------------- + + Third-Level Title + ================= + + Fourth-Level Title + ------------------ + + Fifth-Level Title + ''''''''''''''''' + +You shouldn't have more than five levels of sections in your PEP. If +you do, you should consider rewriting it. + +You must use two blank lines between the last line of a section's body +and the next section heading. If a subsection heading immediately +follows a section heading, a single blank line in-between is +sufficient. + +The body of each section is not normally indented, although some +constructs do use indentation, as described below. Blank lines are +used to separate constructs. + + +Paragraphs +---------- + +Paragraphs are left-aligned text blocks separated by blank lines. +Paragraphs are not indented unless they are part of an indented +construct (such as a block quote or a list item). + + +Inline Markup +------------- + +Portions of text within paragraphs and other text blocks may be +styled. For example:: + + Text may be marked as *emphasized* (single asterisk markup, + typically shown in italics) or **strongly emphasized** (double + asterisks, typically boldface). ``Inline literals`` (using double + backquotes) are typically rendered in a monospaced typeface. No + further markup recognition is done within the double backquotes, + so they're safe for any kind of code snippets. + + +Block Quotes +------------ + +Block quotes consist of indented body elements. For example:: + + This is a paragraph. + + This is a block quote. + + A block quote may contain many paragraphs. + +Block quotes are used to quote extended passages from other sources. +Block quotes may be nested inside other body elements. Use 4 spaces +per indent level. + + +Literal Blocks +-------------- + +.. + In the text below, double backquotes are used to denote inline + literals. "``::``" is written so that the colons will appear in a + monospaced font; the backquotes (``) are markup, not part of the + text. See "Inline Markup" above. + + By the way, this is a comment, described in "Comments" below. + +Literal blocks are used for code samples and other preformatted text. +To indicate a literal block, preface the indented text block with +"``::``" (two colons), or use the ``.. code-block::`` directive. +Indent the text block by 4 spaces; the literal block continues until the end +of the indentation. For example:: + + This is a typical paragraph. A literal block follows. + + :: + + for a in [5, 4, 3, 2, 1]: # this is program code, shown as-is + print(a) + print("it's...") + +"``::``" is also recognized at the end of any paragraph; if not immediately +preceded by whitespace, one colon will remain visible in the final output:: + + This is an example:: + + Literal block + +By default, literal blocks will be syntax-highlighted as Python code. +For specific blocks that contain code or data in other languages/formats, +use the ``.. code-block:: language`` directive, substituting the "short name" +of the appropriate `Pygments lexer `_ +(or ``text`` to disable highlighting) for ``language``. For example:: + + .. code-block:: rst + + An example of the ``rst`` lexer (i.e. *reStructuredText*). + +For PEPs that predominantly contain literal blocks of a specific language, +use the ``.. highlight:: language`` directive with the appropriate ``language`` +at the top of the PEP body (below the headers and above the Abstract). +All literal blocks will then be treated as that language, +unless specified otherwise in the specific ``.. code-block``. For example:: + + .. highlight:: c + + Abstract + ======== + + Here's some C code:: + + printf("Hello, World!\n"); + + +Lists +----- + +Bullet list items begin with one of "-", "*", or "+" (hyphen, +asterisk, or plus sign), followed by whitespace and the list item +body. List item bodies must be left-aligned and indented relative to +the bullet; the text immediately after the bullet determines the +indentation. For example:: + + This paragraph is followed by a list. + + * This is the first bullet list item. The blank line above the + first list item is required; blank lines between list items + (such as below this paragraph) are optional. + + * This is the first paragraph in the second item in the list. + + This is the second paragraph in the second item in the list. + The blank line above this paragraph is required. The left edge + of this paragraph lines up with the paragraph above, both + indented relative to the bullet. + + - This is a sublist. The bullet lines up with the left edge of + the text blocks above. A sublist is a new list so requires a + blank line above and below. + + * This is the third item of the main list. + + This paragraph is not part of the list. + +Enumerated (numbered) list items are similar, but use an enumerator +instead of a bullet. Enumerators are numbers (1, 2, 3, ...), letters +(A, B, C, ...; uppercase or lowercase), or Roman numerals (i, ii, iii, +iv, ...; uppercase or lowercase), formatted with a period suffix +("1.", "2."), parentheses ("(1)", "(2)"), or a right-parenthesis +suffix ("1)", "2)"). For example:: + + 1. As with bullet list items, the left edge of paragraphs must + align. + + 2. Each list item may contain multiple paragraphs, sublists, etc. + + This is the second paragraph of the second list item. + + a) Enumerated lists may be nested. + b) Blank lines may be omitted between list items. + +Definition lists are written like this:: + + what + Definition lists associate a term with a definition. + + how + The term is a one-line phrase, and the definition is one + or more paragraphs or body elements, indented relative to + the term. + + +Tables +------ + +Simple tables are easy and compact:: + + ===== ===== ======= + A B A and B + ===== ===== ======= + False False False + True False False + False True False + True True True + ===== ===== ======= + +There must be at least two columns in a table (to differentiate from +section titles). Column spans use underlines of hyphens ("Inputs" +spans the first two columns):: + + ===== ===== ====== + Inputs Output + ------------ ------ + A B A or B + ===== ===== ====== + False False False + True False True + False True True + True True True + ===== ===== ====== + +Text in a first-column cell starts a new row. No text in the first +column indicates a continuation line; the rest of the cells may +consist of multiple lines. For example:: + + ===== ========================= + col 1 col 2 + ===== ========================= + 1 Second column of row 1. + 2 Second column of row 2. + Second line of paragraph. + 3 - Second column of row 3. + + - Second item in bullet + list (row 3, column 2). + ===== ========================= + + +Hyperlinks +---------- + +When referencing an external web page in the body of a PEP, you should +include the title of the page or a suitable description in the text, with +either an inline hyperlink or a separate explicit target with the URL. +Do not include bare URLs in the body text of the PEP, and use HTTPS +links wherever available. + +Hyperlink references use backquotes and a trailing underscore to mark +up the reference text; backquotes are optional if the reference text +is a single word. For example, to reference a hyperlink target named +``Python website``, you would write: + +.. code-block:: rst + + In this paragraph, we refer to the `Python website`_. + +If you intend to only reference a link once, and want to define it inline +with the text, insert the link into angle brackets (``<>``) after the text +you want to link, but before the closing backtick, with a space between the +text and the opening backtick. You should also use a double-underscore after +the closing backtick instead of a single one, which makes it an anonymous +reference to avoid conflicting with other target names. For example: + +.. code-block:: rst + + Visit the `website `__ for more. + +If you want to use one link multiple places with different linked text, +or want to ensure you don't have to update your link target names when +changing the linked text, include the target name within angle brackets +following the text to link, *with an underscore after the target name +but before the closing angle bracket* (or the link **will not work**). +For example: + +.. code-block:: rst + + For further examples, see the `documentation `_. + +An explicit target provides the URL. Put targets in the Footnotes section +at the end of the PEP, or immediately after the paragraph with the reference. +Hyperlink targets begin with two periods and a space (the "explicit +markup start"), followed by a leading underscore, the reference text, +a colon, and the URL. + +.. code-block:: rst + + .. _Python web site: https://www.python.org/ + .. _pydocs: https://docs.python.org/ + +The reference text and the target text must match (although the match +is case-insensitive and ignores differences in whitespace). Note that +the underscore trails the reference text but precedes the target text. +If you think of the underscore as a right-pointing arrow, it points +*away* from the reference and *toward* the target. + + +Internal and PEP/RFC Links +-------------------------- + +The same mechanism as hyperlinks can be used for internal references. +Every unique section title implicitly defines an internal hyperlink target. +We can make a link to the Abstract section like this: + +.. code-block:: rst + + Here is a hyperlink reference to the `Abstract`_ section. The + backquotes are optional since the reference text is a single word; + we can also just write: Abstract_. + +To refer to PEPs or RFCs, always use the ``:pep:`` and ``:rfc:`` roles, +never hardcoded URLs. +For example: + +.. code-block:: rst + + See :pep:`1` for more information on how to write a PEP, + and :pep:`the Hyperlink section of PEP 12 <12#hyperlinks>` for how to link. + +This renders as: + + See :pep:`1` for more information on how to write a PEP, + and :pep:`the Hyperlink section of PEP 12 <12#hyperlinks>` for how to link. + +PEP numbers in the text are never padded, and there is a space (not a dash) +between "PEP" or "RFC" and the number; the above roles will take care of +that for you. + + +Footnotes +--------- + +Footnote references consist of a left square bracket, a label, a +right square bracket, and a trailing underscore. +Instead of a number, use a label of the +form "#word", where "word" is a mnemonic consisting of alphanumerics +plus internal hyphens, underscores, and periods (no whitespace or +other characters are allowed). +For example: + +.. code-block:: rst + + Refer to The TeXbook [#TeXbook]_ for more information. + +which renders as + + Refer to The TeXbook [#TeXbook]_ for more information. + +Whitespace must precede the footnote reference. Leave a space between +the footnote reference and the preceding word. + +Use footnotes for additional notes, explanations and caveats, as well as +for references to books and other sources not readily available online. +Native reST hyperlink targets or inline hyperlinks in the text should be +used in preference to footnotes for including URLs to online resources. + +Footnotes begin with ".. " (the explicit +markup start), followed by the footnote marker (no underscores), +followed by the footnote body. For example: + +.. code-block:: rst + + .. [#TeXbook] Donald Knuth's *The TeXbook*, pages 195 and 196. + +which renders as + + .. [#TeXbook] Donald Knuth's *The TeXbook*, pages 195 and 196. + +Footnotes and footnote references will be numbered automatically, and +the numbers will always match. + + +Images +------ + +If your PEP contains a diagram or other graphic, you may include it in the +processed output using the ``image`` directive: + +.. code-block:: rst + + .. image:: diagram.png + +Any browser-friendly graphics format is possible; PNG should be +preferred for graphics, JPEG for photos and GIF for animations. +Currently, SVG must be avoided due to compatibility issues with the +PEP build system. + +For accessibility and readers of the source text, you should include +a description of the image and any key information contained within +using the ``:alt:`` option to the ``image`` directive: + +.. code-block:: rst + + .. image:: dataflow.png + :alt: Data flows from the input module, through the "black box" + module, and finally into (and through) the output module. + + +Comments +-------- + +A comment is an indented block of arbitrary text immediately +following an explicit markup start: two periods and whitespace. Leave +the ".." on a line by itself to ensure that the comment is not +misinterpreted as another explicit markup construct. Comments are not +visible in the processed document. For example: + +.. code-block:: rst + + .. + This section should be updated in the final PEP. + Ensure the date is accurate. + + +Escaping Mechanism +------------------ + +reStructuredText uses backslashes ("``\``") to override the special +meaning given to markup characters and get the literal characters +themselves. To get a literal backslash, use an escaped backslash +("``\\``"). There are two contexts in which backslashes have no +special meaning: `literal blocks`_ and inline literals (see `Inline +Markup`_ above). In these contexts, no markup recognition is done, +and a single backslash represents a literal backslash, without having +to double up. + +If you find that you need to use a backslash in your text, consider +using inline literals or a literal block instead. + + +Intersphinx +----------- + +You can use +`Intersphinx references +`_ +to other Sphinx sites, +such as the `Python documentation `_ +`packaging.python.org `_, +and `typing.python.org `_, +to easily cross-reference pages, sections and Python/C objects. + +For example, +to create a link pointing to a section of the typing docs, +you would write the following:: + + :ref:`type expression ` + + +Canonical Documentation +----------------------- + +As :pep:`PEP 1 describes <1#pep-maintenance>`, +PEPs are considered historical documents once marked Final, +and their canonical documentation/specification should be moved elsewhere. +To indicate this, use the ``canonical-doc`` directive +or an appropriate subclass: + +* ``canonical-pypa-spec`` for packaging standards +* ``canonical-typing-spec`` for typing standards + +Add the directive between the headers and the first section of the PEP +(typically the Abstract) +and pass as an argument an Intersphinx reference of the canonical doc/spec +(or if the target is not on a Sphinx site, a `reST hyperlink `__). + +For example, +to create a banner pointing to the :mod:`python:sqlite3` docs, +you would write the following:: + + .. canonical-doc:: :mod:`python:sqlite3` + +which would generate the banner: + + .. canonical-doc:: :mod:`python:sqlite3` + +Or for a PyPA spec, +such as the :ref:`packaging:core-metadata`, +you would use:: + + .. canonical-pypa-spec:: :ref:`packaging:core-metadata` + +which renders as: + + .. canonical-pypa-spec:: :ref:`packaging:core-metadata` + +For a typing PEP that introduces no new runtime objects, +you might use something like the first one of these; +for a typing PEP that introduces a new object to the typing module at runtime, +you might use the second:: + + .. canonical-typing-spec:: :ref:`typing:packaging-typed-libraries` + .. canonical-typing-spec:: :ref:`typing:literal-types` and + :py:data:`typing.Literal` + +The two render as: + + .. canonical-typing-spec:: :ref:`typing:packaging-typed-libraries` + .. canonical-typing-spec:: :ref:`typing:literal-types` and + :py:data:`typing.Literal` + +The argument accepts arbitrary reST, +so you can include multiple linked docs/specs and name them whatever you like, +and you can also include directive content that will be inserted into the text. +The following advanced example:: + + .. canonical-doc:: the :ref:`python:sqlite3-connection-objects` and :exc:`python:~sqlite3.DataError` docs + + Also, see the :ref:`Data Persistence docs ` for other examples. + +would render as: + + .. canonical-doc:: the :ref:`python:sqlite3-connection-objects` and :exc:`python:sqlite3.DataError` docs + + Also, see the :ref:`Data Persistence docs ` for other examples. + + +Habits to Avoid +=============== + +Many programmers who are familiar with TeX often write quotation marks +like this: + +.. code-block:: text + + `single-quoted' or ``double-quoted'' + +Backquotes are significant in reStructuredText, so this practice +should be avoided. For ordinary text, use ordinary 'single-quotes' or +"double-quotes". For inline literal text (see `Inline Markup`_ +above), use double-backquotes:: + + ``literal text: in here, anything goes!`` + + +Suggested Sections +================== + +Various sections are found to be common across PEPs and are outlined in +:pep:`1`. Those sections are provided here for convenience. + +.. _template: + +.. include:: pep-0012/pep-NNNN.rst + :code: rst + + +Resources +========= + +Many other constructs and variations are possible, +both those supported by basic `Docutils `_ +and the extensions added by `Sphinx `_. + +A number of resources are available to learn more about them: + +* `Sphinx ReStructuredText Primer + `_, + a gentle but fairly detailed introduction. + +* `reStructuredText Markup Specification + `_, + the authoritative, comprehensive documentation of the basic reST syntax, + directives, roles and more. + +* `Sphinx Roles + `_ + and `Sphinx Directives + `_, + the extended constructs added by the Sphinx documentation system used to + render the PEPs to HTML. + +If you have questions or require assistance with writing a PEP that the above +resources don't address, ping ``@python/pep-editors`` on GitHub, open an +`issue on the PEPs repository `_ +or reach out to a PEP editor directly. + + +Copyright +========= + +This document is placed in the public domain or under the +CC0-1.0-Universal license, whichever is more permissive. From d0a92890b987bc451796912001432dc28362075a Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 28 Jun 2025 13:13:25 -0700 Subject: [PATCH 02/29] add some core content --- peps/pep-0796.rst | 812 +++++----------------------------------------- 1 file changed, 90 insertions(+), 722 deletions(-) diff --git a/peps/pep-0796.rst b/peps/pep-0796.rst index 6691584b4f6..81c54dd78a2 100644 --- a/peps/pep-0796.rst +++ b/peps/pep-0796.rst @@ -7,6 +7,7 @@ Type: Standards Track Topic: Packaging Created: 06-26-2025 Python-Version: 3.15 +Sponsor: TODO Post-History: `DD-MM-YYYY `__ .. highlight:: rst @@ -22,7 +23,7 @@ virtual environment. Making this work consists of two main pieces: * Using a relative path for `home` in `pyvenv.cfg`, which Python canonicalizes at runtime. * Having package installers recognize a host-relocatable venv and generate - appropriate e.g. script wrappers. + appropriate outputs, e.g. script wrappers. Rationale @@ -34,769 +35,136 @@ First, it is currently proscribed that the `home` value in `pyvenv.cfg` be an absolute path. The behavior of relative paths is unspecified. This requirement is overly proscriptive and restrictive because, given a known anchor point, it's easy to tranform a relative path to absolute path and still retain -expected behavior. Thus, this requirement should be lifted. +predictable and reliable behavior. Thus, the absolute path requirement should +be relaxed and relative path behavior allowed and defined. Second, virtual environments are the standard way for running Python -applications, which means how they're created is important. This means -recreating a virtual environment for each instance ... +applications. This means ... -When they need to be recreated ... +* The closer the dev environment is to the non-dev environment, the more reliable + software one can get. +* The closer the dev and non-dev envs are, the easier it is to reproduce issues. +* The simpler the process is to re-create the environment, the more reliable + software one can get. +* The simpler the process is to re-create the environment, the faster the process + can be. -Currently, virtual environments are effectively tied to the particular host -that the virtual environment is originally created on. This is because symlinks -to absolute paths are commonly used in various places (e.g. `bin/python3`), -which are frequently specific to a host (e.g. by containing a user's home -directory). +By making it trivial to copy a virtual environment from one host to another, +it avoids these categories of problems. +Another use case it enables it for multiple virtual environments to share +a Python interpreter. This allows a group of independent applications to have +different third party dependency closures, while benefiting from a shared +runtime. +Python Runtime Changes +====================== -How to Use This Template -======================== - -To use this template you must first decide whether your PEP is going -to be an Informational or Standards Track PEP. Most PEPs are -Standards Track because they propose a new feature for the Python -language or standard library. When in doubt, read :pep:`1` for details, -or open a tracker issue on the PEPs repo to ask for assistance. - -Once you've decided which type of PEP yours is going to be, follow the -directions below. - -- Make a copy of this file (the ``.rst`` file, **not** the HTML!) and - perform the following edits. Name the new file :file:`pep-{NNNN}.rst`, using - the next available number (not used by a published or in-PR PEP). - -- Replace the "PEP: 12" header with "PEP: NNNN", - matching the file name. Note that the file name should be padded with - zeros (eg ``pep-0012.rst``), but the header should not (``PEP: 12``). - -- Change the Title header to the title of your PEP. - -- Change the Author header to include your name, and optionally your - email address. Be sure to follow the format carefully: your name - must appear first, and it must not be contained in parentheses. - Your email address may appear second (or it can be omitted) and if - it appears, it must appear in angle brackets. It is okay to - obfuscate your email address. - -- If none of the authors are Python core developers, include a Sponsor - header with the name of the core developer sponsoring your PEP. - -- Add the direct URL of the PEP's canonical discussion thread - (on e.g. Python-Dev, Discourse, etc) under the Discussions-To header. - If the thread will be created after the PEP is submitted as an official - draft, it is okay to just put "Pending" initially, but remember to - update the PEP with the URL as soon as the PEP is successfully merged - to the PEPs repository and you create the corresponding discussion thread. - See :pep:`PEP 1 <1#discussing-a-pep>` for more details. - -- Change the Status header to "Draft". - -- For Standards Track PEPs, change the Type header to "Standards - Track". - -- For Informational PEPs, change the Type header to "Informational". - -- For Standards Track PEPs, if your feature depends on the acceptance - of some other currently in-development PEP, add a Requires header - right after the Type header. The value should be the PEP number of - the PEP yours depends on. Don't add this header if your dependent - feature is described in a Final PEP. - -- Change the Created header to today's date. Be sure to follow the - format carefully: it must be in ``dd-mmm-yyyy`` format, where the - ``mmm`` is the 3 English letter month abbreviation, i.e. one of Jan, - Feb, Mar, Apr, May, Jun, Jul, Aug, Sep, Oct, Nov, Dec. - -- For Standards Track PEPs, after the Created header, add a - Python-Version header and set the value to the next planned version - of Python, i.e. the one your new feature will hopefully make its - first appearance in. Do not use an alpha or beta release - designation here. Thus, if the last version of Python was 2.2 alpha - 1 and you're hoping to get your new feature into Python 2.2, set the - header to: - - .. code-block:: email - - Python-Version: 2.2 - -- Add a Topic header if the PEP belongs under one shown at the :ref:`topic-index`. - Most PEPs don't. - -- Leave Post-History alone for now; you'll add dates and corresponding links - to this header each time you post your PEP to the designated discussion forum - (and update the Discussions-To header with said link, as above). - For each thread, use the date (in the ``dd-mmm-yyy`` format) as the - linked text, and insert the URLs inline as anonymous reST `hyperlinks`_, - with commas in between each posting. - - If you posted threads for your PEP on August 14, 2001 and September 3, 2001, - the Post-History header would look like, e.g.: - - .. code-block:: email - - Post-History: `14-Aug-2001 `__, - `03-Sept-2001 `__ - - You should add the new dates/links here as soon as you post a - new discussion thread. - -- Add a Replaces header if your PEP obsoletes an earlier PEP. The - value of this header is the number of the PEP that your new PEP is - replacing. Only add this header if the older PEP is in "final" - form, i.e. is either Accepted, Final, or Rejected. You aren't - replacing an older open PEP if you're submitting a competing idea. - -- Now write your Abstract, Rationale, and other content for your PEP, - replacing all this gobbledygook with your own text. Be sure to - adhere to the format guidelines below, specifically on the - prohibition of tab characters and the indentation requirements. - See "Suggested Sections" below for a template of sections to include. - -- Update your Footnotes section, listing any footnotes and - non-inline link targets referenced by the text. - -- Run ``./build.py`` to ensure the PEP is rendered without errors, - and check that the output in :file:`build/pep-{NNNN}.html` looks as you intend. - -- Create a pull request against the `PEPs repository`_. - -For reference, here are all of the possible header fields (everything -in brackets should either be replaced or have the field removed if -it has a leading ``*`` marking it as optional and it does not apply to -your PEP): - -.. code-block:: email - - PEP: [NNN] - Title: [...] - Author: [Full Name ] - Sponsor: *[Full Name ] - PEP-Delegate: - Discussions-To: [URL] - Status: Draft - Type: [Standards Track | Informational | Process] - Topic: *[Governance | Packaging | Release | Typing] - Requires: *[NNN] - Created: [DD-MMM-YYYY] - Python-Version: *[M.N] - Post-History: [`DD-MMM-YYYY `__] - Replaces: *[NNN] - Superseded-By: *[NNN] - Resolution: - - -ReStructuredText PEP Formatting Requirements -============================================ - -The following is a PEP-specific summary of reStructuredText syntax. -For the sake of simplicity and brevity, much detail is omitted. For -more detail, see `Resources`_ below. `Literal blocks`_ (in which no -markup processing is done) are used for examples throughout, to -illustrate the plaintext markup. - - -General -------- - -Lines should usually not extend past column 79, -excepting URLs and similar circumstances. -Tab characters must never appear in the document at all. - - -Section Headings ----------------- - -PEP headings must begin in column zero and the initial letter of each -word must be capitalized as in book titles. Acronyms should be in all -capitals. Section titles must be adorned with an underline, a single -repeated punctuation character, which begins in column zero and must -extend at least as far as the right edge of the title text (4 -characters minimum). First-level section titles are underlined with -"=" (equals signs), second-level section titles with "-" (hyphens), -and third-level section titles with "'" (single quotes or -apostrophes). For example:: - - First-Level Title - ================= - - Second-Level Title - ------------------ - - Third-Level Title - ''''''''''''''''' - -If there are more than three levels of sections in your PEP, you may -insert overline/underline-adorned titles for the first and second -levels as follows:: - - ============================ - First-Level Title (optional) - ============================ - - ----------------------------- - Second-Level Title (optional) - ----------------------------- - - Third-Level Title - ================= - - Fourth-Level Title - ------------------ - - Fifth-Level Title - ''''''''''''''''' - -You shouldn't have more than five levels of sections in your PEP. If -you do, you should consider rewriting it. - -You must use two blank lines between the last line of a section's body -and the next section heading. If a subsection heading immediately -follows a section heading, a single blank line in-between is -sufficient. - -The body of each section is not normally indented, although some -constructs do use indentation, as described below. Blank lines are -used to separate constructs. - - -Paragraphs ----------- - -Paragraphs are left-aligned text blocks separated by blank lines. -Paragraphs are not indented unless they are part of an indented -construct (such as a block quote or a list item). - - -Inline Markup -------------- - -Portions of text within paragraphs and other text blocks may be -styled. For example:: - - Text may be marked as *emphasized* (single asterisk markup, - typically shown in italics) or **strongly emphasized** (double - asterisks, typically boldface). ``Inline literals`` (using double - backquotes) are typically rendered in a monospaced typeface. No - further markup recognition is done within the double backquotes, - so they're safe for any kind of code snippets. - - -Block Quotes ------------- - -Block quotes consist of indented body elements. For example:: - - This is a paragraph. - - This is a block quote. - - A block quote may contain many paragraphs. - -Block quotes are used to quote extended passages from other sources. -Block quotes may be nested inside other body elements. Use 4 spaces -per indent level. - - -Literal Blocks --------------- - -.. - In the text below, double backquotes are used to denote inline - literals. "``::``" is written so that the colons will appear in a - monospaced font; the backquotes (``) are markup, not part of the - text. See "Inline Markup" above. - - By the way, this is a comment, described in "Comments" below. - -Literal blocks are used for code samples and other preformatted text. -To indicate a literal block, preface the indented text block with -"``::``" (two colons), or use the ``.. code-block::`` directive. -Indent the text block by 4 spaces; the literal block continues until the end -of the indentation. For example:: - - This is a typical paragraph. A literal block follows. - - :: - - for a in [5, 4, 3, 2, 1]: # this is program code, shown as-is - print(a) - print("it's...") - -"``::``" is also recognized at the end of any paragraph; if not immediately -preceded by whitespace, one colon will remain visible in the final output:: - - This is an example:: - - Literal block - -By default, literal blocks will be syntax-highlighted as Python code. -For specific blocks that contain code or data in other languages/formats, -use the ``.. code-block:: language`` directive, substituting the "short name" -of the appropriate `Pygments lexer `_ -(or ``text`` to disable highlighting) for ``language``. For example:: - - .. code-block:: rst - - An example of the ``rst`` lexer (i.e. *reStructuredText*). - -For PEPs that predominantly contain literal blocks of a specific language, -use the ``.. highlight:: language`` directive with the appropriate ``language`` -at the top of the PEP body (below the headers and above the Abstract). -All literal blocks will then be treated as that language, -unless specified otherwise in the specific ``.. code-block``. For example:: - - .. highlight:: c - - Abstract - ======== - - Here's some C code:: - - printf("Hello, World!\n"); - - -Lists ------ - -Bullet list items begin with one of "-", "*", or "+" (hyphen, -asterisk, or plus sign), followed by whitespace and the list item -body. List item bodies must be left-aligned and indented relative to -the bullet; the text immediately after the bullet determines the -indentation. For example:: - - This paragraph is followed by a list. - - * This is the first bullet list item. The blank line above the - first list item is required; blank lines between list items - (such as below this paragraph) are optional. - - * This is the first paragraph in the second item in the list. - - This is the second paragraph in the second item in the list. - The blank line above this paragraph is required. The left edge - of this paragraph lines up with the paragraph above, both - indented relative to the bullet. - - - This is a sublist. The bullet lines up with the left edge of - the text blocks above. A sublist is a new list so requires a - blank line above and below. - - * This is the third item of the main list. - - This paragraph is not part of the list. - -Enumerated (numbered) list items are similar, but use an enumerator -instead of a bullet. Enumerators are numbers (1, 2, 3, ...), letters -(A, B, C, ...; uppercase or lowercase), or Roman numerals (i, ii, iii, -iv, ...; uppercase or lowercase), formatted with a period suffix -("1.", "2."), parentheses ("(1)", "(2)"), or a right-parenthesis -suffix ("1)", "2)"). For example:: - - 1. As with bullet list items, the left edge of paragraphs must - align. - - 2. Each list item may contain multiple paragraphs, sublists, etc. - - This is the second paragraph of the second list item. - - a) Enumerated lists may be nested. - b) Blank lines may be omitted between list items. +The Python runtime itself *almost* already supports host-relocating virtual +environment and the only change needed is to define how it resolves relative +paths for `home` in `pyvenv.cfg`. -Definition lists are written like this:: +Today, relative paths resolve relative to the process's current working +directory. Because CWD isn't knowable in advance, it makes relative paths today +effective impossible. - what - Definition lists associate a term with a definition. +Instead, the paths should be relative to the location of the `pyvenv.cfg` file. +This file is chosen as the anchor point because the tool that creates the file +also has to know where the Python runtime is, so can easily calculate the +correct relative path. For tools that read the `pyvenv.cfg`, it is also easy +to simple join the directory name of where `pyvenv.cfg` was found with the +path in the config file. When a person reads the config file, they can do +similar, which is a lower cognitive burden and helps avoids the question of +"relative to what?" - how - The term is a one-line phrase, and the definition is one - or more paragraphs or body elements, indented relative to - the term. +This change is only a couple of lines in the start up code. Specifically, when +parsing the `pyvenv.cfg` file and finding the `home` value, it just needs to +be checked if it's already absolute. If not, then join to the directory name +of the `pyvenv.cfg` file. The code alredy knows the directory and has helpers +already present for checking if a path is absolute and joining two paths. +The core change in the Python runtime is to define how it finds it's +For the most part, the Python runtime itself is already -Tables ------- -Simple tables are easy and compact:: +Virtual Environment Tool Changes +================================ - ===== ===== ======= - A B A and B - ===== ===== ======= - False False False - True False False - False True False - True True True - ===== ===== ======= +Venv tools should accept a `--relative` flag. When set, they generate +`pyvenv.cfg` file with a `home` key that is a relative path to the relevant +Python installation. -There must be at least two columns in a table (to differentiate from -section titles). Column spans use underlines of hyphens ("Inputs" -spans the first two columns):: +TODO: or `--relocatable`? This is the term uv uses. However, discussion points +out that "relocatable" is a big ambiguous insofar as absolute paths allow +arbitrary relocation *on a host*, while relative paths allow relocation +_between_ hosts. - ===== ===== ====== - Inputs Output - ------------ ------ - A B A or B - ===== ===== ====== - False False False - True False True - False True True - True True True - ===== ===== ====== -Text in a first-column cell starts a new row. No text in the first -column indicates a continuation line; the rest of the cells may -consist of multiple lines. For example:: - ===== ========================= - col 1 col 2 - ===== ========================= - 1 Second column of row 1. - 2 Second column of row 2. - Second line of paragraph. - 3 - Second column of row 3. - - Second item in bullet - list (row 3, column 2). - ===== ========================= +Package Installer Changes +========================= +Backwards Compatibility +======================= -Hyperlinks ----------- +Because relative paths for `home` aren't usable in practice, and their +behavior undocumented and unspecified, there shouldn't be any backward +compatibility concerns. -When referencing an external web page in the body of a PEP, you should -include the title of the page or a suitable description in the text, with -either an inline hyperlink or a separate explicit target with the URL. -Do not include bare URLs in the body text of the PEP, and use HTTPS -links wherever available. -Hyperlink references use backquotes and a trailing underscore to mark -up the reference text; backquotes are optional if the reference text -is a single word. For example, to reference a hyperlink target named -``Python website``, you would write: +How to Teach This +================= -.. code-block:: rst +Teaching this should be simple: if you use a relative path in `pyvenv.cfg`, +then it's relative to the directory containing the `pyvenv.cfg` file. This +is simple to explain and understand. - In this paragraph, we refer to the `Python website`_. -If you intend to only reference a link once, and want to define it inline -with the text, insert the link into angle brackets (``<>``) after the text -you want to link, but before the closing backtick, with a space between the -text and the opening backtick. You should also use a double-underscore after -the closing backtick instead of a single one, which makes it an anonymous -reference to avoid conflicting with other target names. For example: - -.. code-block:: rst - - Visit the `website `__ for more. - -If you want to use one link multiple places with different linked text, -or want to ensure you don't have to update your link target names when -changing the linked text, include the target name within angle brackets -following the text to link, *with an underscore after the target name -but before the closing angle bracket* (or the link **will not work**). -For example: - -.. code-block:: rst - - For further examples, see the `documentation `_. - -An explicit target provides the URL. Put targets in the Footnotes section -at the end of the PEP, or immediately after the paragraph with the reference. -Hyperlink targets begin with two periods and a space (the "explicit -markup start"), followed by a leading underscore, the reference text, -a colon, and the URL. - -.. code-block:: rst - - .. _Python web site: https://www.python.org/ - .. _pydocs: https://docs.python.org/ - -The reference text and the target text must match (although the match -is case-insensitive and ignores differences in whitespace). Note that -the underscore trails the reference text but precedes the target text. -If you think of the underscore as a right-pointing arrow, it points -*away* from the reference and *toward* the target. - - -Internal and PEP/RFC Links --------------------------- - -The same mechanism as hyperlinks can be used for internal references. -Every unique section title implicitly defines an internal hyperlink target. -We can make a link to the Abstract section like this: - -.. code-block:: rst - - Here is a hyperlink reference to the `Abstract`_ section. The - backquotes are optional since the reference text is a single word; - we can also just write: Abstract_. - -To refer to PEPs or RFCs, always use the ``:pep:`` and ``:rfc:`` roles, -never hardcoded URLs. -For example: - -.. code-block:: rst +Reference Implementation +======================== - See :pep:`1` for more information on how to write a PEP, - and :pep:`the Hyperlink section of PEP 12 <12#hyperlinks>` for how to link. +TODO link to rickeylev/cpython branch for runtime changes -This renders as: +TODO link to rules_python and rules_py implementation of relative venvs - See :pep:`1` for more information on how to write a PEP, - and :pep:`the Hyperlink section of PEP 12 <12#hyperlinks>` for how to link. -PEP numbers in the text are never padded, and there is a space (not a dash) -between "PEP" or "RFC" and the number; the above roles will take care of -that for you. +Open Issues +=========== Footnotes ---------- - -Footnote references consist of a left square bracket, a label, a -right square bracket, and a trailing underscore. -Instead of a number, use a label of the -form "#word", where "word" is a mnemonic consisting of alphanumerics -plus internal hyphens, underscores, and periods (no whitespace or -other characters are allowed). -For example: - -.. code-block:: rst - - Refer to The TeXbook [#TeXbook]_ for more information. - -which renders as - - Refer to The TeXbook [#TeXbook]_ for more information. - -Whitespace must precede the footnote reference. Leave a space between -the footnote reference and the preceding word. - -Use footnotes for additional notes, explanations and caveats, as well as -for references to books and other sources not readily available online. -Native reST hyperlink targets or inline hyperlinks in the text should be -used in preference to footnotes for including URLs to online resources. - -Footnotes begin with ".. " (the explicit -markup start), followed by the footnote marker (no underscores), -followed by the footnote body. For example: - -.. code-block:: rst - - .. [#TeXbook] Donald Knuth's *The TeXbook*, pages 195 and 196. - -which renders as - - .. [#TeXbook] Donald Knuth's *The TeXbook*, pages 195 and 196. - -Footnotes and footnote references will be numbered automatically, and -the numbers will always match. - - -Images ------- - -If your PEP contains a diagram or other graphic, you may include it in the -processed output using the ``image`` directive: - -.. code-block:: rst - - .. image:: diagram.png - -Any browser-friendly graphics format is possible; PNG should be -preferred for graphics, JPEG for photos and GIF for animations. -Currently, SVG must be avoided due to compatibility issues with the -PEP build system. - -For accessibility and readers of the source text, you should include -a description of the image and any key information contained within -using the ``:alt:`` option to the ``image`` directive: - -.. code-block:: rst - - .. image:: dataflow.png - :alt: Data flows from the input module, through the "black box" - module, and finally into (and through) the output module. - - -Comments --------- - -A comment is an indented block of arbitrary text immediately -following an explicit markup start: two periods and whitespace. Leave -the ".." on a line by itself to ensure that the comment is not -misinterpreted as another explicit markup construct. Comments are not -visible in the processed document. For example: - -.. code-block:: rst - - .. - This section should be updated in the final PEP. - Ensure the date is accurate. - - -Escaping Mechanism ------------------- - -reStructuredText uses backslashes ("``\``") to override the special -meaning given to markup characters and get the literal characters -themselves. To get a literal backslash, use an escaped backslash -("``\\``"). There are two contexts in which backslashes have no -special meaning: `literal blocks`_ and inline literals (see `Inline -Markup`_ above). In these contexts, no markup recognition is done, -and a single backslash represents a literal backslash, without having -to double up. - -If you find that you need to use a backslash in your text, consider -using inline literals or a literal block instead. - - -Intersphinx ------------ - -You can use -`Intersphinx references -`_ -to other Sphinx sites, -such as the `Python documentation `_ -`packaging.python.org `_, -and `typing.python.org `_, -to easily cross-reference pages, sections and Python/C objects. - -For example, -to create a link pointing to a section of the typing docs, -you would write the following:: - - :ref:`type expression ` - - -Canonical Documentation ------------------------ - -As :pep:`PEP 1 describes <1#pep-maintenance>`, -PEPs are considered historical documents once marked Final, -and their canonical documentation/specification should be moved elsewhere. -To indicate this, use the ``canonical-doc`` directive -or an appropriate subclass: - -* ``canonical-pypa-spec`` for packaging standards -* ``canonical-typing-spec`` for typing standards - -Add the directive between the headers and the first section of the PEP -(typically the Abstract) -and pass as an argument an Intersphinx reference of the canonical doc/spec -(or if the target is not on a Sphinx site, a `reST hyperlink `__). - -For example, -to create a banner pointing to the :mod:`python:sqlite3` docs, -you would write the following:: - - .. canonical-doc:: :mod:`python:sqlite3` - -which would generate the banner: - - .. canonical-doc:: :mod:`python:sqlite3` - -Or for a PyPA spec, -such as the :ref:`packaging:core-metadata`, -you would use:: - - .. canonical-pypa-spec:: :ref:`packaging:core-metadata` - -which renders as: - - .. canonical-pypa-spec:: :ref:`packaging:core-metadata` - -For a typing PEP that introduces no new runtime objects, -you might use something like the first one of these; -for a typing PEP that introduces a new object to the typing module at runtime, -you might use the second:: - - .. canonical-typing-spec:: :ref:`typing:packaging-typed-libraries` - .. canonical-typing-spec:: :ref:`typing:literal-types` and - :py:data:`typing.Literal` - -The two render as: - - .. canonical-typing-spec:: :ref:`typing:packaging-typed-libraries` - .. canonical-typing-spec:: :ref:`typing:literal-types` and - :py:data:`typing.Literal` - -The argument accepts arbitrary reST, -so you can include multiple linked docs/specs and name them whatever you like, -and you can also include directive content that will be inserted into the text. -The following advanced example:: - - .. canonical-doc:: the :ref:`python:sqlite3-connection-objects` and :exc:`python:~sqlite3.DataError` docs - - Also, see the :ref:`Data Persistence docs ` for other examples. - -would render as: - - .. canonical-doc:: the :ref:`python:sqlite3-connection-objects` and :exc:`python:sqlite3.DataError` docs - - Also, see the :ref:`Data Persistence docs ` for other examples. - - -Habits to Avoid -=============== - -Many programmers who are familiar with TeX often write quotation marks -like this: - -.. code-block:: text - - `single-quoted' or ``double-quoted'' - -Backquotes are significant in reStructuredText, so this practice -should be avoided. For ordinary text, use ordinary 'single-quotes' or -"double-quotes". For inline literal text (see `Inline Markup`_ -above), use double-backquotes:: - - ``literal text: in here, anything goes!`` - - -Suggested Sections -================== - -Various sections are found to be common across PEPs and are outlined in -:pep:`1`. Those sections are provided here for convenience. - -.. _template: +========= -.. include:: pep-0012/pep-NNNN.rst - :code: rst +Rejected Ideas +===================== -Resources -========= +Relative to virtual env root +---------------------------- -Many other constructs and variations are possible, -both those supported by basic `Docutils `_ -and the extensions added by `Sphinx `_. +Having the `home` value in `pyvenv.cfg` relative to the virtual environments +root directory would work just as well, but this idea is rejected because it +requires additional effort to compute the virtual env root. -A number of resources are available to learn more about them: +Unspecified `home` means to dynamically compute home +---------------------------------------------------- -* `Sphinx ReStructuredText Primer - `_, - a gentle but fairly detailed introduction. +Today, if a `pyvenv.cfg` file doesn't set `home`, the runtime will try to +dynamically compute it by checking if the current executable (which is typicall +the venv's `bin/python3` symlink) is a symlink and, if so, use where that +points as `PYTHONHOME`. -* `reStructuredText Markup Specification - `_, - the authoritative, comprehensive documentation of the basic reST syntax, - directives, roles and more. +This behavior is undesirable for a couple reasons: -* `Sphinx Roles - `_ - and `Sphinx Directives - `_, - the extended constructs added by the Sphinx documentation system used to - render the PEPs to HTML. +1. It presents platform-specific issues, namely with Windows. Windows does + support symlinks, but not by default, and it can require special + permissions to do so. +2. It _requires_ that a symlink be used, which precludes using otherwise + equivalent mechanisms for creating an executable (e.g. a wrapper script, + hard links, etc). -If you have questions or require assistance with writing a PEP that the above -resources don't address, ping ``@python/pep-editors`` on GitHub, open an -`issue on the PEPs repository `_ -or reach out to a PEP editor directly. +In general, symlinks work best when they aren't special cased by consumers. Copyright From 41d2ea87efe7d58151d10baf9eabe315771e4820 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 28 Jun 2025 14:01:09 -0700 Subject: [PATCH 03/29] rename to motivation --- peps/pep-0796.rst | 71 ++++++++++++++++++++++++++++++++++------------- 1 file changed, 52 insertions(+), 19 deletions(-) diff --git a/peps/pep-0796.rst b/peps/pep-0796.rst index 81c54dd78a2..3d6459e5c2e 100644 --- a/peps/pep-0796.rst +++ b/peps/pep-0796.rst @@ -26,17 +26,26 @@ virtual environment. Making this work consists of two main pieces: appropriate outputs, e.g. script wrappers. -Rationale -========= +Motivation +========== -There are two primary rationales for host-relocatable virtual environments. +There are two main motivations for allowing relative paths in `pyvenv.cfg` +and thus enabling host-relocatable virtual environments. First, it is currently proscribed that the `home` value in `pyvenv.cfg` be an -absolute path. The behavior of relative paths is unspecified. This requirement -is overly proscriptive and restrictive because, given a known anchor point, -it's easy to tranform a relative path to absolute path and still retain -predictable and reliable behavior. Thus, the absolute path requirement should -be relaxed and relative path behavior allowed and defined. +absolute path. The behavior of relative paths is unspecified. +While techniques exist to work around this for every other +sub-part of a virtual environment, the one remaining part without a teneable +solution is how the Python runtime itself finds `PYTHONHOME`. This is because, +currently, the startup process requires absolute paths be used for the `home` +key in `pyvenv.cfg`. If a relative path is used, behavior is unspecified +(currently, it is relative to the process's current working directory, making +it untenable to use). + +This requirement is overly proscriptive and restrictive because, given a known +anchor point, it's easy to tranform a relative path to an absolute path and +still retain predictable and reliable behavior. Thus, the absolute path +requirement should be relaxed and relative path behavior allowed and defined. Second, virtual environments are the standard way for running Python applications. This means ... @@ -50,12 +59,13 @@ applications. This means ... can be. By making it trivial to copy a virtual environment from one host to another, -it avoids these categories of problems. +it avoids these categories of problems. Additionally, the development tools +to create a virtual environment and install its dependencies aren't needed +on the host that intends to run the program. + +Rationale +========= -Another use case it enables it for multiple virtual environments to share -a Python interpreter. This allows a group of independent applications to have -different third party dependency closures, while benefiting from a shared -runtime. Python Runtime Changes ====================== @@ -83,14 +93,12 @@ be checked if it's already absolute. If not, then join to the directory name of the `pyvenv.cfg` file. The code alredy knows the directory and has helpers already present for checking if a path is absolute and joining two paths. -The core change in the Python runtime is to define how it finds it's -For the most part, the Python runtime itself is already - +A proof of concept for this is at TODO-link to code Virtual Environment Tool Changes ================================ -Venv tools should accept a `--relative` flag. When set, they generate +Venv tools should accept a `--relative` flag. When set, they generate a `pyvenv.cfg` file with a `home` key that is a relative path to the relevant Python installation. @@ -100,11 +108,31 @@ arbitrary relocation *on a host*, while relative paths allow relocation _between_ hosts. - - Package Installer Changes ========================= +Package installers must be aware of when a relative virtual environment has +been defined so they can avoid using absolute paths. The two main places +absolute paths occur today are: + +1. `bin/` scripts, where shebangs are rewritten. +2. `site-packages` symlinks. + +Instead, they should create relative symlinks pointing to the actual +installation directory for packages. + + +Relocating a Venv to Another Host +================================= + +Copying the venv to another host is simple: copy the venv directory itself, the +relative location `home` points to, and preserve the relative directory +structures. + +In practice, this typically means copying a parent directory of the virtual +environment, which contains the runtime, installations of dependencies, and +the group of virtual environments to relocate. + Backwards Compatibility ======================= @@ -136,6 +164,11 @@ Open Issues Footnotes ========= +* PBS +* rules_python +* rules_py +* uv venv relocatable + Rejected Ideas ===================== From 65a415c6878a333b5c0d9bf405559963063dd215 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 28 Jun 2025 14:11:57 -0700 Subject: [PATCH 04/29] note it enables more advanced optimizations --- peps/pep-0796.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/peps/pep-0796.rst b/peps/pep-0796.rst index 3d6459e5c2e..3b439db23b5 100644 --- a/peps/pep-0796.rst +++ b/peps/pep-0796.rst @@ -63,6 +63,11 @@ it avoids these categories of problems. Additionally, the development tools to create a virtual environment and install its dependencies aren't needed on the host that intends to run the program. +Simple copying allows additional optimizations in more advanced deployments, +such as remote mounting +Because only simply copying is need, it allows more advanced d +This simple copying enables more advanced deployments to employ + Rationale ========= From 4175b206dbce493cc05eb5e0f237d4cda9d4aaff Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 29 Jun 2025 15:53:06 -0700 Subject: [PATCH 05/29] add foot notes, relocatable term section, links --- peps/pep-0796.rst | 52 +++++++++++++++++++++++++++++++++-------------- 1 file changed, 37 insertions(+), 15 deletions(-) diff --git a/peps/pep-0796.rst b/peps/pep-0796.rst index 3b439db23b5..078d3fbe971 100644 --- a/peps/pep-0796.rst +++ b/peps/pep-0796.rst @@ -98,20 +98,18 @@ be checked if it's already absolute. If not, then join to the directory name of the `pyvenv.cfg` file. The code alredy knows the directory and has helpers already present for checking if a path is absolute and joining two paths. -A proof of concept for this is at TODO-link to code +A proof-of-concept of this is implemented in +[rickeylev/feat.relative.pyvenv.home](https://github.com/python/cpython/compare/main...rickeylev:cpython:feat.relative.pyvenv.home) Virtual Environment Tool Changes ================================ -Venv tools should accept a `--relative` flag. When set, they generate a -`pyvenv.cfg` file with a `home` key that is a relative path to the relevant +Virtual environment management tools should support a mechanism to generate +a `pyvenv.cfg` file with a `home` key that is a relative path to the relevant Python installation. -TODO: or `--relocatable`? This is the term uv uses. However, discussion points -out that "relocatable" is a big ambiguous insofar as absolute paths allow -arbitrary relocation *on a host*, while relative paths allow relocation -_between_ hosts. - +For the standard library's `venv` module, the flag `--relative` will be added +to trigger this behavior. Package Installer Changes ========================= @@ -157,22 +155,29 @@ is simple to explain and understand. Reference Implementation ======================== -TODO link to rickeylev/cpython branch for runtime changes - -TODO link to rules_python and rules_py implementation of relative venvs +A reference implementation is available by using the combination of: +* Python runtime from [rickeylev/feat.relative.pyvenv.home](https://github.com/python/cpython/compare/main...rickeylev:cpython:feat.relative.pyvenv.home) +* rules_python with (todo: alterations to use cpython build as in-build + runtime) Open Issues =========== +todo: list any that are brought up and unresolved Footnotes ========= -* PBS -* rules_python -* rules_py -* uv venv relocatable +* [rules_python](https://github.com/bazel-contrib/rules_python): implements + host-relocatable virtual environments. +* [rules_py](https://github.com/aspect-build/rules_py): implements + host-relocatable virtual environments. +* [`uv venv + relocatable`](https://docs.astral.sh/uv/reference/cli/#uv-venv--relocatable): + implements same-host relocatable virtual environments. +* [python-build-standalone](https://github.com/astral-sh/python-build-standalone): + A relocatable Python runtime. Rejected Ideas @@ -204,6 +209,23 @@ This behavior is undesirable for a couple reasons: In general, symlinks work best when they aren't special cased by consumers. +Using the term "relocatable" +---------------------------- + +Discussions pointed out the the term "relocatable" is somewhat ambigious and +misleading for a couple reasons. + +First, absolute paths makes a venv arbitrarily relocatable _within_ a host, but +not between hosts, so "relocatable" requires _some_ qualification for +clarity. + +Second, when using relative paths that point outside the venv, the venv is only +relocatable insofar as those external artifacts are also relocated. This is an +additional nuance that requires qualification of the term. + +To better avoid this confusiong, "relative" is chosen, which more naturally +invites the question _"Relative to what?"_. + Copyright ========= From bbc85b7a12aa0f0eda65219499bb81b67c784729 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 29 Jun 2025 16:01:08 -0700 Subject: [PATCH 06/29] update discussion, sponsor headers to pending --- peps/pep-0796.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/peps/pep-0796.rst b/peps/pep-0796.rst index 078d3fbe971..a86b5b41466 100644 --- a/peps/pep-0796.rst +++ b/peps/pep-0796.rst @@ -1,13 +1,13 @@ PEP: 796 Title: Host-relocatable virtual environments Author: Richard Levasseur -Discussions-To: TODO +Discussions-To: Pending Status: Draft Type: Standards Track Topic: Packaging Created: 06-26-2025 Python-Version: 3.15 -Sponsor: TODO +Sponsor: Pending Post-History: `DD-MM-YYYY `__ .. highlight:: rst From a0c2529c371faa91154279fc37f7593cb702938f Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 29 Jun 2025 16:22:28 -0700 Subject: [PATCH 07/29] rst fixups --- peps/pep-0796.rst | 65 ++++++++++++++++++++++++----------------------- 1 file changed, 33 insertions(+), 32 deletions(-) diff --git a/peps/pep-0796.rst b/peps/pep-0796.rst index a86b5b41466..6b3f9900c18 100644 --- a/peps/pep-0796.rst +++ b/peps/pep-0796.rst @@ -20,7 +20,7 @@ This PEP describes how a Python virtual environment can be configured to allow copying it as-is to different hosts and still have a functioning virtual environment. Making this work consists of two main pieces: -* Using a relative path for `home` in `pyvenv.cfg`, which Python canonicalizes +* Using a relative path for ``home`` in ``pyvenv.cfg``, which Python canonicalizes at runtime. * Having package installers recognize a host-relocatable venv and generate appropriate outputs, e.g. script wrappers. @@ -29,16 +29,16 @@ virtual environment. Making this work consists of two main pieces: Motivation ========== -There are two main motivations for allowing relative paths in `pyvenv.cfg` +There are two main motivations for allowing relative paths in ``pyvenv.cfg`` and thus enabling host-relocatable virtual environments. -First, it is currently proscribed that the `home` value in `pyvenv.cfg` be an +First, it is currently proscribed that the ``home`` value in ``pyvenv.cfg`` be an absolute path. The behavior of relative paths is unspecified. While techniques exist to work around this for every other sub-part of a virtual environment, the one remaining part without a teneable -solution is how the Python runtime itself finds `PYTHONHOME`. This is because, -currently, the startup process requires absolute paths be used for the `home` -key in `pyvenv.cfg`. If a relative path is used, behavior is unspecified +solution is how the Python runtime itself finds ``PYTHONHOME``. This is because, +currently, the startup process requires absolute paths be used for the ``home`` +key in ``pyvenv.cfg``. If a relative path is used, behavior is unspecified (currently, it is relative to the process's current working directory, making it untenable to use). @@ -77,38 +77,38 @@ Python Runtime Changes The Python runtime itself *almost* already supports host-relocating virtual environment and the only change needed is to define how it resolves relative -paths for `home` in `pyvenv.cfg`. +paths for ``home`` in ``pyvenv.cfg``. Today, relative paths resolve relative to the process's current working directory. Because CWD isn't knowable in advance, it makes relative paths today effective impossible. -Instead, the paths should be relative to the location of the `pyvenv.cfg` file. +Instead, the paths should be relative to the location of the ``pyvenv.cfg`` file. This file is chosen as the anchor point because the tool that creates the file also has to know where the Python runtime is, so can easily calculate the -correct relative path. For tools that read the `pyvenv.cfg`, it is also easy -to simple join the directory name of where `pyvenv.cfg` was found with the +correct relative path. For tools that read the ``pyvenv.cfg``, it is also easy +to simple join the directory name of where ``pyvenv.cfg`` was found with the path in the config file. When a person reads the config file, they can do similar, which is a lower cognitive burden and helps avoids the question of "relative to what?" This change is only a couple of lines in the start up code. Specifically, when -parsing the `pyvenv.cfg` file and finding the `home` value, it just needs to +parsing the ``pyvenv.cfg`` file and finding the ``home`` value, it just needs to be checked if it's already absolute. If not, then join to the directory name -of the `pyvenv.cfg` file. The code alredy knows the directory and has helpers +of the ``pyvenv.cfg`` file. The code alredy knows the directory and has helpers already present for checking if a path is absolute and joining two paths. A proof-of-concept of this is implemented in -[rickeylev/feat.relative.pyvenv.home](https://github.com/python/cpython/compare/main...rickeylev:cpython:feat.relative.pyvenv.home) +`rickeylev/feat.relative.pyvenv.home `__ Virtual Environment Tool Changes ================================ Virtual environment management tools should support a mechanism to generate -a `pyvenv.cfg` file with a `home` key that is a relative path to the relevant +a ``pyvenv.cfg`` file with a ``home`` key that is a relative path to the relevant Python installation. -For the standard library's `venv` module, the flag `--relative` will be added +For the standard library's ``venv`` module, the flag ``--relative`` will be added to trigger this behavior. Package Installer Changes @@ -118,8 +118,8 @@ Package installers must be aware of when a relative virtual environment has been defined so they can avoid using absolute paths. The two main places absolute paths occur today are: -1. `bin/` scripts, where shebangs are rewritten. -2. `site-packages` symlinks. +1. ``bin/`` scripts, where shebangs are rewritten. +2. ``site-packages`` symlinks. Instead, they should create relative symlinks pointing to the actual installation directory for packages. @@ -129,7 +129,7 @@ Relocating a Venv to Another Host ================================= Copying the venv to another host is simple: copy the venv directory itself, the -relative location `home` points to, and preserve the relative directory +relative location ``home`` points to, and preserve the relative directory structures. In practice, this typically means copying a parent directory of the virtual @@ -139,7 +139,7 @@ the group of virtual environments to relocate. Backwards Compatibility ======================= -Because relative paths for `home` aren't usable in practice, and their +Because relative paths for ``home`` aren't usable in practice, and their behavior undocumented and unspecified, there shouldn't be any backward compatibility concerns. @@ -147,8 +147,8 @@ compatibility concerns. How to Teach This ================= -Teaching this should be simple: if you use a relative path in `pyvenv.cfg`, -then it's relative to the directory containing the `pyvenv.cfg` file. This +Teaching this should be simple: if you use a relative path in ``pyvenv.cfg``, +then it's relative to the directory containing the ``pyvenv.cfg`` file. This is simple to explain and understand. @@ -157,7 +157,7 @@ Reference Implementation A reference implementation is available by using the combination of: -* Python runtime from [rickeylev/feat.relative.pyvenv.home](https://github.com/python/cpython/compare/main...rickeylev:cpython:feat.relative.pyvenv.home) +* Python runtime from `rickeylev/feat.relative.pyvenv.home `__ * rules_python with (todo: alterations to use cpython build as in-build runtime) @@ -169,14 +169,15 @@ todo: list any that are brought up and unresolved Footnotes ========= -* [rules_python](https://github.com/bazel-contrib/rules_python): implements +* `rules_python `__: implements host-relocatable virtual environments. -* [rules_py](https://github.com/aspect-build/rules_py): implements +* `rules_py `__: implements host-relocatable virtual environments. -* [`uv venv - relocatable`](https://docs.astral.sh/uv/reference/cli/#uv-venv--relocatable): +* `uv venv + relocatable + `__: implements same-host relocatable virtual environments. -* [python-build-standalone](https://github.com/astral-sh/python-build-standalone): +* `python-build-standalone `__: A relocatable Python runtime. @@ -186,17 +187,17 @@ Rejected Ideas Relative to virtual env root ---------------------------- -Having the `home` value in `pyvenv.cfg` relative to the virtual environments +Having the ``home`` value in ``pyvenv.cfg`` relative to the virtual environments root directory would work just as well, but this idea is rejected because it requires additional effort to compute the virtual env root. -Unspecified `home` means to dynamically compute home +Unspecified home means to dynamically compute home ---------------------------------------------------- -Today, if a `pyvenv.cfg` file doesn't set `home`, the runtime will try to +Today, if a ``pyvenv.cfg`` file doesn't set ``home``, the runtime will try to dynamically compute it by checking if the current executable (which is typicall -the venv's `bin/python3` symlink) is a symlink and, if so, use where that -points as `PYTHONHOME`. +the venv's ``bin/python3`` symlink) is a symlink and, if so, use where that +points as ``PYTHONHOME``. This behavior is undesirable for a couple reasons: From 8e6e21e1596ef558f38eab51d3b4ce9bd8569d7a Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 29 Jun 2025 16:23:23 -0700 Subject: [PATCH 08/29] fixup italics --- peps/pep-0796.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/peps/pep-0796.rst b/peps/pep-0796.rst index 6b3f9900c18..fbf719d28d0 100644 --- a/peps/pep-0796.rst +++ b/peps/pep-0796.rst @@ -204,7 +204,7 @@ This behavior is undesirable for a couple reasons: 1. It presents platform-specific issues, namely with Windows. Windows does support symlinks, but not by default, and it can require special permissions to do so. -2. It _requires_ that a symlink be used, which precludes using otherwise +2. It *requires* that a symlink be used, which precludes using otherwise equivalent mechanisms for creating an executable (e.g. a wrapper script, hard links, etc). @@ -216,8 +216,8 @@ Using the term "relocatable" Discussions pointed out the the term "relocatable" is somewhat ambigious and misleading for a couple reasons. -First, absolute paths makes a venv arbitrarily relocatable _within_ a host, but -not between hosts, so "relocatable" requires _some_ qualification for +First, absolute paths makes a venv arbitrarily relocatable *within* a host, but +not between hosts, so "relocatable" requires *some* qualification for clarity. Second, when using relative paths that point outside the venv, the venv is only @@ -225,7 +225,7 @@ relocatable insofar as those external artifacts are also relocated. This is an additional nuance that requires qualification of the term. To better avoid this confusiong, "relative" is chosen, which more naturally -invites the question _"Relative to what?"_. +invites the question *"Relative to what?"*. Copyright From 37202ca4ef4d186d280f34d20f6b68d7798cdb26 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 29 Jun 2025 16:47:07 -0700 Subject: [PATCH 09/29] rename to relative-venv --- peps/pep-0796.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/peps/pep-0796.rst b/peps/pep-0796.rst index fbf719d28d0..ff19812a649 100644 --- a/peps/pep-0796.rst +++ b/peps/pep-0796.rst @@ -1,5 +1,5 @@ PEP: 796 -Title: Host-relocatable virtual environments +Title: Relative Virtual Environments Author: Richard Levasseur Discussions-To: Pending Status: Draft @@ -22,7 +22,7 @@ virtual environment. Making this work consists of two main pieces: * Using a relative path for ``home`` in ``pyvenv.cfg``, which Python canonicalizes at runtime. -* Having package installers recognize a host-relocatable venv and generate +* Having package installers recognize a relative venv and generate appropriate outputs, e.g. script wrappers. @@ -125,7 +125,7 @@ Instead, they should create relative symlinks pointing to the actual installation directory for packages. -Relocating a Venv to Another Host +Copying a Venv to Another Host ================================= Copying the venv to another host is simple: copy the venv directory itself, the From b81c50f001899297f6c8a9413062abe0bb64d3ea Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 29 Jun 2025 17:03:19 -0700 Subject: [PATCH 10/29] clarify advanced deployment options, reflow some text --- peps/pep-0796.rst | 33 ++++++++++++++++----------------- 1 file changed, 16 insertions(+), 17 deletions(-) diff --git a/peps/pep-0796.rst b/peps/pep-0796.rst index ff19812a649..621b63b4c03 100644 --- a/peps/pep-0796.rst +++ b/peps/pep-0796.rst @@ -32,15 +32,15 @@ Motivation There are two main motivations for allowing relative paths in ``pyvenv.cfg`` and thus enabling host-relocatable virtual environments. -First, it is currently proscribed that the ``home`` value in ``pyvenv.cfg`` be an -absolute path. The behavior of relative paths is unspecified. -While techniques exist to work around this for every other -sub-part of a virtual environment, the one remaining part without a teneable -solution is how the Python runtime itself finds ``PYTHONHOME``. This is because, -currently, the startup process requires absolute paths be used for the ``home`` -key in ``pyvenv.cfg``. If a relative path is used, behavior is unspecified -(currently, it is relative to the process's current working directory, making -it untenable to use). +First, it is currently proscribed that the ``home`` value in ``pyvenv.cfg`` be +an absolute path. The behavior of relative paths is unspecified. While +techniques exist to work around this for every other sub-part of a virtual +environment, the one remaining part without a teneable solution is how the +Python runtime itself finds ``PYTHONHOME``. This is because, currently, the +startup process requires absolute paths be used for the ``home`` key in +``pyvenv.cfg``. If a relative path is used, behavior is unspecified (currently, +it is relative to the process's current working directory, making it untenable +to use). This requirement is overly proscriptive and restrictive because, given a known anchor point, it's easy to tranform a relative path to an absolute path and @@ -58,15 +58,14 @@ applications. This means ... * The simpler the process is to re-create the environment, the faster the process can be. -By making it trivial to copy a virtual environment from one host to another, -it avoids these categories of problems. Additionally, the development tools -to create a virtual environment and install its dependencies aren't needed -on the host that intends to run the program. +By making it trivial to copy a virtual environment from one host to another, it +avoids these categories of problems. Additionally, the development tools to +create a virtual environment and install its dependencies aren't needed on the +host that intends to run the program. -Simple copying allows additional optimizations in more advanced deployments, -such as remote mounting -Because only simply copying is need, it allows more advanced d -This simple copying enables more advanced deployments to employ +Because the virtual environment doesn't require modifications to be usable, it +also allows more advanced deployment mechansisms, e.g. remote mounting and +caching of artifacts. Rationale ========= From 55a5e2857487d435a82772717f6a6d5083f94cd8 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 30 Jun 2025 09:56:39 -0700 Subject: [PATCH 11/29] add specification, rationale section; doc why env vars arent an option --- peps/pep-0796.rst | 99 +++++++++++++++++++++++++++++++++++------------ 1 file changed, 75 insertions(+), 24 deletions(-) diff --git a/peps/pep-0796.rst b/peps/pep-0796.rst index 621b63b4c03..cb1b6287b04 100644 --- a/peps/pep-0796.rst +++ b/peps/pep-0796.rst @@ -50,13 +50,14 @@ requirement should be relaxed and relative path behavior allowed and defined. Second, virtual environments are the standard way for running Python applications. This means ... -* The closer the dev environment is to the non-dev environment, the more reliable - software one can get. -* The closer the dev and non-dev envs are, the easier it is to reproduce issues. +* The closer the dev environment is to the non-dev environment, the more + reliable software one can get. +* The closer the dev and non-dev envs are, the easier it is to reproduce + issues. * The simpler the process is to re-create the environment, the more reliable software one can get. -* The simpler the process is to re-create the environment, the faster the process - can be. +* The simpler the process is to re-create the environment, the faster the + process can be. By making it trivial to copy a virtual environment from one host to another, it avoids these categories of problems. Additionally, the development tools to @@ -70,6 +71,36 @@ caching of artifacts. Rationale ========= +The reason support for relative virtual environments needs to be +in the interpreter itself is because locating ``PYTHONHOME`` happens +very early in the interpreter startup process, which limits the options for +customizing how it's computed. Without the ability to specify where the +supporting Python runtime files are, the interpreter can't finish startup, +so other hook points (e.g. ``site`` initialization) never trigger. + +Specification +============= + +The ``home`` value in ``pyvenv.cfg`` is permitted to use a relative path value. +These may contain up-references outside of the virtual environnment root +directory. Examples: + +* ``subdir/whatever/bin`` (a directory within the virtual environment). +* ``../../../../../elsewhere/runtime/bin`` (a directory outside the virtual + environment). + +Relative paths are relative to the directory containing ``pyvenv.cfg``. During +interpreter startup (i.e. ``getpath.py``), the relative path is joined to the +directory to form an absolute path. Up-references are resolved syntactically +(i.e. not resolving symlinks). Symlinks are *not* resolved prior to +construction of the absolute path to ensure semantics between a relative path +and absolute path remain the same. + +For example, given +``/home/user/venv/bin/pyvenv.cfg`` with +``home = ../../runtime/bin``, the result is ``home = /home/user/runtime/bin``, +i.e. it's equivalent to using that value verbatim in ``pyvenv.cfg``. + Python Runtime Changes ====================== @@ -82,9 +113,9 @@ Today, relative paths resolve relative to the process's current working directory. Because CWD isn't knowable in advance, it makes relative paths today effective impossible. -Instead, the paths should be relative to the location of the ``pyvenv.cfg`` file. -This file is chosen as the anchor point because the tool that creates the file -also has to know where the Python runtime is, so can easily calculate the +Instead, the paths should be relative to the location of the ``pyvenv.cfg`` +file. This file is chosen as the anchor point because the tool that creates the +file also has to know where the Python runtime is, so can easily calculate the correct relative path. For tools that read the ``pyvenv.cfg``, it is also easy to simple join the directory name of where ``pyvenv.cfg`` was found with the path in the config file. When a person reads the config file, they can do @@ -92,8 +123,8 @@ similar, which is a lower cognitive burden and helps avoids the question of "relative to what?" This change is only a couple of lines in the start up code. Specifically, when -parsing the ``pyvenv.cfg`` file and finding the ``home`` value, it just needs to -be checked if it's already absolute. If not, then join to the directory name +parsing the ``pyvenv.cfg`` file and finding the ``home`` value, it just needs +to be checked if it's already absolute. If not, then join to the directory name of the ``pyvenv.cfg`` file. The code alredy knows the directory and has helpers already present for checking if a path is absolute and joining two paths. @@ -103,12 +134,12 @@ A proof-of-concept of this is implemented in Virtual Environment Tool Changes ================================ -Virtual environment management tools should support a mechanism to generate -a ``pyvenv.cfg`` file with a ``home`` key that is a relative path to the relevant +Virtual environment management tools should support a mechanism to generate a +``pyvenv.cfg`` file with a ``home`` key that is a relative path to the relevant Python installation. -For the standard library's ``venv`` module, the flag ``--relative`` will be added -to trigger this behavior. +For the standard library's ``venv`` module, the flag ``--relative`` will be +added to trigger this behavior. Package Installer Changes ========================= @@ -120,20 +151,29 @@ absolute paths occur today are: 1. ``bin/`` scripts, where shebangs are rewritten. 2. ``site-packages`` symlinks. -Instead, they should create relative symlinks pointing to the actual +For symlinks, they should create relative symlinks pointing to the actual installation directory for packages. +For ``bin/`` scripts, installers should generate executables that are able to +locate the venv's interpreter at runtime. How to do this is implementation +defined. Typically it means having shell code use ``$0`` to locate the +appropriate ``bin/python3`` executable, but installers are free to use whatever +mechanism they want (e.g. a native polyglot executable). + +todo: copy/paste example shell? Copying a Venv to Another Host ================================= -Copying the venv to another host is simple: copy the venv directory itself, the -relative location ``home`` points to, and preserve the relative directory -structures. +Copying the venv to another host is simple: copy the venv directory itself, and +any directories outside the venv it needs, while preserving the relative +directory structures. + +How to do this, exactly, is implementation-defined. In practice, this typically means copying a parent directory of the virtual -environment, which contains the runtime, installations of dependencies, and -the group of virtual environments to relocate. +environment, which contains the runtime, one or more venvs, installations +of dependencies, and any other supporting files. Backwards Compatibility ======================= @@ -157,8 +197,7 @@ Reference Implementation A reference implementation is available by using the combination of: * Python runtime from `rickeylev/feat.relative.pyvenv.home `__ -* rules_python with (todo: alterations to use cpython build as in-build - runtime) +* rules_python with (todo: link to branch that uses above) Open Issues =========== @@ -178,7 +217,7 @@ Footnotes implements same-host relocatable virtual environments. * `python-build-standalone `__: A relocatable Python runtime. - +* `PoC for relative home in Python startup `__ Rejected Ideas ===================== @@ -223,10 +262,22 @@ Second, when using relative paths that point outside the venv, the venv is only relocatable insofar as those external artifacts are also relocated. This is an additional nuance that requires qualification of the term. -To better avoid this confusiong, "relative" is chosen, which more naturally +To better avoid this confusion, "relative" is chosen, which more naturally invites the question *"Relative to what?"*. +Using PYTHONHOME at runtime to specify home +------------------------------------------- + +Using the ``PYTHONHOME`` environment variable (or any environment variable) is +problematic because it's difficult to know and control when an environment +variable should or shouldn't be inherited by subprocesses. In some cases, it's +not feasible because of how layers of programs calling programs interact. + +Code generally assumes that any virtual environment will be +automatically detected and activated by the presence of ``pyvenv.cfg``, so +things work better when alterations to the environemtn aren't a concern. + Copyright ========= From e09b00d85fdb0fedc75127bda8be0a6c83d77a6a Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 30 Jun 2025 10:41:50 -0700 Subject: [PATCH 12/29] also mention curdir syntax handling --- peps/pep-0796.rst | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/peps/pep-0796.rst b/peps/pep-0796.rst index cb1b6287b04..ebca5195b34 100644 --- a/peps/pep-0796.rst +++ b/peps/pep-0796.rst @@ -86,19 +86,21 @@ These may contain up-references outside of the virtual environnment root directory. Examples: * ``subdir/whatever/bin`` (a directory within the virtual environment). +* ``./subdir/whatever/bin`` (same as above) * ``../../../../../elsewhere/runtime/bin`` (a directory outside the virtual environment). Relative paths are relative to the directory containing ``pyvenv.cfg``. During interpreter startup (i.e. ``getpath.py``), the relative path is joined to the -directory to form an absolute path. Up-references are resolved syntactically -(i.e. not resolving symlinks). Symlinks are *not* resolved prior to -construction of the absolute path to ensure semantics between a relative path -and absolute path remain the same. +directory to form an absolute path. Up-references (``../``) and current +directory references (``./``) are resolved syntactically (i.e. not resolving +symlinks). Symlinks are *not* resolved prior to construction of the absolute +path to ensure semantics between a relative path and absolute path remain the +same. For example, given ``/home/user/venv/bin/pyvenv.cfg`` with -``home = ../../runtime/bin``, the result is ``home = /home/user/runtime/bin``, +``home = ../../runtime/./bin``, the result is ``home = /home/user/runtime/bin``, i.e. it's equivalent to using that value verbatim in ``pyvenv.cfg``. From c7a6a9b0868cccd04ab367a4689217fa3695603d Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 30 Jun 2025 10:47:53 -0700 Subject: [PATCH 13/29] update some todo text --- peps/pep-0796.rst | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/peps/pep-0796.rst b/peps/pep-0796.rst index ebca5195b34..d60a37a931c 100644 --- a/peps/pep-0796.rst +++ b/peps/pep-0796.rst @@ -162,8 +162,6 @@ defined. Typically it means having shell code use ``$0`` to locate the appropriate ``bin/python3`` executable, but installers are free to use whatever mechanism they want (e.g. a native polyglot executable). -todo: copy/paste example shell? - Copying a Venv to Another Host ================================= @@ -204,7 +202,7 @@ A reference implementation is available by using the combination of: Open Issues =========== -todo: list any that are brought up and unresolved +(todo: placeholder section to be updated after discussions) Footnotes ========= From bc1ffbd6f7ffb8591e854d76ed1652b961b104e9 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 30 Jun 2025 10:52:06 -0700 Subject: [PATCH 14/29] update codeowners --- .github/CODEOWNERS | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index e000d3934b1..fd2e1204674 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -672,6 +672,7 @@ peps/pep-0791.rst @vstinner peps/pep-0792.rst @dstufft peps/pep-0793.rst @encukou peps/pep-0794.rst @brettcannon +peps/pep-0796.rst @rickeylev # ... peps/pep-0801.rst @warsaw # ... From 2301d2ae1755e0016490389fcb3eb66f9d488589 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 30 Jun 2025 10:54:52 -0700 Subject: [PATCH 15/29] add initial discussion footnote link --- peps/pep-0796.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/peps/pep-0796.rst b/peps/pep-0796.rst index d60a37a931c..fb3beaf6696 100644 --- a/peps/pep-0796.rst +++ b/peps/pep-0796.rst @@ -218,6 +218,7 @@ Footnotes * `python-build-standalone `__: A relocatable Python runtime. * `PoC for relative home in Python startup `__ +* `Python Ideas "Making venvs relocatable friendly" discussion `__ Rejected Ideas ===================== From e14b01f8afa5f76142fa08a7126b12b90d1459ba Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 30 Jun 2025 10:59:16 -0700 Subject: [PATCH 16/29] add fr footnote --- peps/pep-0796.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/peps/pep-0796.rst b/peps/pep-0796.rst index fb3beaf6696..869550d2881 100644 --- a/peps/pep-0796.rst +++ b/peps/pep-0796.rst @@ -219,6 +219,7 @@ Footnotes A relocatable Python runtime. * `PoC for relative home in Python startup `__ * `Python Ideas "Making venvs relocatable friendly" discussion `__ +* `GH-136051: relative pyvenv.cfg home `__ Rejected Ideas ===================== From 175f3780739c8c0127bfcc810aa470eb91ba1af0 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 30 Jun 2025 13:29:25 -0700 Subject: [PATCH 17/29] revert codeowners change --- .github/CODEOWNERS | 1 - 1 file changed, 1 deletion(-) diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index fd2e1204674..e000d3934b1 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -672,7 +672,6 @@ peps/pep-0791.rst @vstinner peps/pep-0792.rst @dstufft peps/pep-0793.rst @encukou peps/pep-0794.rst @brettcannon -peps/pep-0796.rst @rickeylev # ... peps/pep-0801.rst @warsaw # ... From 31cb75617bc0d838d97cc27bd18f5c7559f2c76c Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 1 Jul 2025 18:17:18 -0700 Subject: [PATCH 18/29] remove packaging-related specs --- peps/pep-0796.rst | 87 ++++++++++++++--------------------------------- 1 file changed, 25 insertions(+), 62 deletions(-) diff --git a/peps/pep-0796.rst b/peps/pep-0796.rst index 869550d2881..5edf8c4a2ad 100644 --- a/peps/pep-0796.rst +++ b/peps/pep-0796.rst @@ -16,21 +16,16 @@ Post-History: `DD-MM-YYYY `__ Abstract ======== -This PEP describes how a Python virtual environment can be configured -to allow copying it as-is to different hosts and still have a functioning -virtual environment. Making this work consists of two main pieces: - -* Using a relative path for ``home`` in ``pyvenv.cfg``, which Python canonicalizes - at runtime. -* Having package installers recognize a relative venv and generate - appropriate outputs, e.g. script wrappers. - +This PEP describes how a relative path for ``home`` in a Python virtual +environment's ``pyvenv.cfg`` is understood by the Python startup process. +Specifically, how is it canonicalized into an absolute path later used +by the runtime. This small detail is a fundamental building block for +virtual environments to be more portable. Motivation ========== -There are two main motivations for allowing relative paths in ``pyvenv.cfg`` -and thus enabling host-relocatable virtual environments. +There are two main motivations for allowing relative paths in ``pyvenv.cfg``. First, it is currently proscribed that the ``home`` value in ``pyvenv.cfg`` be an absolute path. The behavior of relative paths is unspecified. While @@ -38,26 +33,24 @@ techniques exist to work around this for every other sub-part of a virtual environment, the one remaining part without a teneable solution is how the Python runtime itself finds ``PYTHONHOME``. This is because, currently, the startup process requires absolute paths be used for the ``home`` key in -``pyvenv.cfg``. If a relative path is used, behavior is unspecified (currently, -it is relative to the process's current working directory, making it untenable -to use). +``pyvenv.cfg``. If a relative path is used, behavior is unspecified (the +current implementation ends up making it relative to the process's current +working directory, making it untenable to use). This requirement is overly proscriptive and restrictive because, given a known anchor point, it's easy to tranform a relative path to an absolute path and still retain predictable and reliable behavior. Thus, the absolute path requirement should be relaxed and relative path behavior allowed and defined. -Second, virtual environments are the standard way for running Python -applications. This means ... +Second, such relative paths are a building block to enable portable virtual +environments, i.e. copying a virtual environment as-is between hosts of +compatible platforms. Portable venvs are appealing because virtual environments +are the standard way for running Python applications. This means... * The closer the dev environment is to the non-dev environment, the more - reliable software one can get. -* The closer the dev and non-dev envs are, the easier it is to reproduce - issues. + reliable software one can get, and the easier it is to reproduce issues. * The simpler the process is to re-create the environment, the more reliable - software one can get. -* The simpler the process is to re-create the environment, the faster the - process can be. + software one can get, and the faster the process can be. By making it trivial to copy a virtual environment from one host to another, it avoids these categories of problems. Additionally, the development tools to @@ -127,43 +120,15 @@ similar, which is a lower cognitive burden and helps avoids the question of This change is only a couple of lines in the start up code. Specifically, when parsing the ``pyvenv.cfg`` file and finding the ``home`` value, it just needs to be checked if it's already absolute. If not, then join to the directory name -of the ``pyvenv.cfg`` file. The code alredy knows the directory and has helpers -already present for checking if a path is absolute and joining two paths. +of the ``pyvenv.cfg`` file. The code already knows the directory and has +helpers already present for checking if a path is absolute and joining two +paths. A proof-of-concept of this is implemented in `rickeylev/feat.relative.pyvenv.home `__ -Virtual Environment Tool Changes -================================ - -Virtual environment management tools should support a mechanism to generate a -``pyvenv.cfg`` file with a ``home`` key that is a relative path to the relevant -Python installation. - -For the standard library's ``venv`` module, the flag ``--relative`` will be -added to trigger this behavior. - -Package Installer Changes -========================= - -Package installers must be aware of when a relative virtual environment has -been defined so they can avoid using absolute paths. The two main places -absolute paths occur today are: - -1. ``bin/`` scripts, where shebangs are rewritten. -2. ``site-packages`` symlinks. - -For symlinks, they should create relative symlinks pointing to the actual -installation directory for packages. - -For ``bin/`` scripts, installers should generate executables that are able to -locate the venv's interpreter at runtime. How to do this is implementation -defined. Typically it means having shell code use ``$0`` to locate the -appropriate ``bin/python3`` executable, but installers are free to use whatever -mechanism they want (e.g. a native polyglot executable). - Copying a Venv to Another Host -================================= +============================== Copying the venv to another host is simple: copy the venv directory itself, and any directories outside the venv it needs, while preserving the relative @@ -202,7 +167,9 @@ A reference implementation is available by using the combination of: Open Issues =========== -(todo: placeholder section to be updated after discussions) +This PEP does not specify how to create a ``pyvenv.cfg`` with a relative path, +nor how downstream tools (e.g. installers) should identify them or process +them. These questions are best addressed separately by tool owners. Footnotes ========= @@ -211,10 +178,6 @@ Footnotes host-relocatable virtual environments. * `rules_py `__: implements host-relocatable virtual environments. -* `uv venv - relocatable - `__: - implements same-host relocatable virtual environments. * `python-build-standalone `__: A relocatable Python runtime. * `PoC for relative home in Python startup `__ @@ -227,9 +190,9 @@ Rejected Ideas Relative to virtual env root ---------------------------- -Having the ``home`` value in ``pyvenv.cfg`` relative to the virtual environments -root directory would work just as well, but this idea is rejected because it -requires additional effort to compute the virtual env root. +Having the ``home`` value in ``pyvenv.cfg`` relative to the virtual +environments root directory would work just as well, but this idea is rejected +because it requires additional effort to compute the virtual env root. Unspecified home means to dynamically compute home ---------------------------------------------------- From cd46f4d4e2a56bacb30f316e3bf5c147b4d50a48 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 1 Jul 2025 18:29:40 -0700 Subject: [PATCH 19/29] fix lint errors with headers --- peps/pep-0796.rst | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/peps/pep-0796.rst b/peps/pep-0796.rst index 5edf8c4a2ad..2c55e71cadb 100644 --- a/peps/pep-0796.rst +++ b/peps/pep-0796.rst @@ -1,14 +1,12 @@ PEP: 796 Title: Relative Virtual Environments Author: Richard Levasseur +Sponsor: Pending Discussions-To: Pending Status: Draft Type: Standards Track -Topic: Packaging -Created: 06-26-2025 +Created: 26-Jun-2025 Python-Version: 3.15 -Sponsor: Pending -Post-History: `DD-MM-YYYY `__ .. highlight:: rst From b532f5cd696c210b07b1738d4f2393cdc7fbd5bb Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 1 Jul 2025 18:45:44 -0700 Subject: [PATCH 20/29] set sponsor --- peps/pep-0796.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0796.rst b/peps/pep-0796.rst index 2c55e71cadb..2bf81240a0f 100644 --- a/peps/pep-0796.rst +++ b/peps/pep-0796.rst @@ -1,7 +1,7 @@ PEP: 796 Title: Relative Virtual Environments Author: Richard Levasseur -Sponsor: Pending +Sponsor: Alyssa Coghlan Discussions-To: Pending Status: Draft Type: Standards Track From 99434fc0ddc3af22407345b5ca096e8a4916705b Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 1 Jul 2025 18:51:48 -0700 Subject: [PATCH 21/29] set sponsor as codeowner --- .github/CODEOWNERS | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index e000d3934b1..db44eec3c69 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -672,6 +672,7 @@ peps/pep-0791.rst @vstinner peps/pep-0792.rst @dstufft peps/pep-0793.rst @encukou peps/pep-0794.rst @brettcannon +peps/pep-0796.rst @ncoghlan # ... peps/pep-0801.rst @warsaw # ... From 862142a6686938d89800f6f39a2f2baecf242bf6 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 1 Jul 2025 19:06:37 -0700 Subject: [PATCH 22/29] clarify title to focus on core pyvenv.cfg home behavior --- peps/pep-0796.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0796.rst b/peps/pep-0796.rst index 2bf81240a0f..99ec67fb64a 100644 --- a/peps/pep-0796.rst +++ b/peps/pep-0796.rst @@ -1,5 +1,5 @@ PEP: 796 -Title: Relative Virtual Environments +Title: Relative Virtual Environment Home Author: Richard Levasseur Sponsor: Alyssa Coghlan Discussions-To: Pending From 1c8fb2d8f7af11342a43948cb67af130866c8c60 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 1 Jul 2025 19:20:48 -0700 Subject: [PATCH 23/29] fix typo; remove host-relocatable mention; remove non-germane copying-venv section --- peps/pep-0796.rst | 21 ++++----------------- 1 file changed, 4 insertions(+), 17 deletions(-) diff --git a/peps/pep-0796.rst b/peps/pep-0796.rst index 99ec67fb64a..dada7bd2e2d 100644 --- a/peps/pep-0796.rst +++ b/peps/pep-0796.rst @@ -56,7 +56,7 @@ create a virtual environment and install its dependencies aren't needed on the host that intends to run the program. Because the virtual environment doesn't require modifications to be usable, it -also allows more advanced deployment mechansisms, e.g. remote mounting and +also allows more advanced deployment mechanisms, e.g. remote mounting and caching of artifacts. Rationale @@ -98,9 +98,9 @@ i.e. it's equivalent to using that value verbatim in ``pyvenv.cfg``. Python Runtime Changes ====================== -The Python runtime itself *almost* already supports host-relocating virtual -environment and the only change needed is to define how it resolves relative -paths for ``home`` in ``pyvenv.cfg``. +The Python runtime itself *almost* already supports relative paths. The +primitives are there, so the only change needed is to define how it resolves +relative paths for ``home`` in ``pyvenv.cfg``. Today, relative paths resolve relative to the process's current working directory. Because CWD isn't knowable in advance, it makes relative paths today @@ -125,19 +125,6 @@ paths. A proof-of-concept of this is implemented in `rickeylev/feat.relative.pyvenv.home `__ -Copying a Venv to Another Host -============================== - -Copying the venv to another host is simple: copy the venv directory itself, and -any directories outside the venv it needs, while preserving the relative -directory structures. - -How to do this, exactly, is implementation-defined. - -In practice, this typically means copying a parent directory of the virtual -environment, which contains the runtime, one or more venvs, installations -of dependencies, and any other supporting files. - Backwards Compatibility ======================= From c5428902a5cc08941659d0869cf66043b9119c61 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 1 Jul 2025 19:56:29 -0700 Subject: [PATCH 24/29] fix typos, grammar --- peps/pep-0796.rst | 62 ++++++++++++++++++++++++----------------------- 1 file changed, 32 insertions(+), 30 deletions(-) diff --git a/peps/pep-0796.rst b/peps/pep-0796.rst index dada7bd2e2d..d0ee13cc404 100644 --- a/peps/pep-0796.rst +++ b/peps/pep-0796.rst @@ -16,7 +16,7 @@ Abstract This PEP describes how a relative path for ``home`` in a Python virtual environment's ``pyvenv.cfg`` is understood by the Python startup process. -Specifically, how is it canonicalized into an absolute path later used +Specifically, how it is canonicalized into an absolute path later used by the runtime. This small detail is a fundamental building block for virtual environments to be more portable. @@ -25,10 +25,10 @@ Motivation There are two main motivations for allowing relative paths in ``pyvenv.cfg``. -First, it is currently proscribed that the ``home`` value in ``pyvenv.cfg`` be +First, it is currently prescribed that the ``home`` value in ``pyvenv.cfg`` be an absolute path. The behavior of relative paths is unspecified. While techniques exist to work around this for every other sub-part of a virtual -environment, the one remaining part without a teneable solution is how the +environment, the one remaining part without a tenable solution is how the Python runtime itself finds ``PYTHONHOME``. This is because, currently, the startup process requires absolute paths be used for the ``home`` key in ``pyvenv.cfg``. If a relative path is used, behavior is unspecified (the @@ -36,24 +36,26 @@ current implementation ends up making it relative to the process's current working directory, making it untenable to use). This requirement is overly proscriptive and restrictive because, given a known -anchor point, it's easy to tranform a relative path to an absolute path and +anchor point, it's easy to transform a relative path to an absolute path and still retain predictable and reliable behavior. Thus, the absolute path requirement should be relaxed and relative path behavior allowed and defined. Second, such relative paths are a building block to enable portable virtual environments, i.e. copying a virtual environment as-is between hosts of compatible platforms. Portable venvs are appealing because virtual environments -are the standard way for running Python applications. This means... +are the standard way for running Python applications. This provides several +benefits: -* The closer the dev environment is to the non-dev environment, the more - reliable software one can get, and the easier it is to reproduce issues. -* The simpler the process is to re-create the environment, the more reliable - software one can get, and the faster the process can be. +* The closer the development environment is to the non-development environment, + the more reliable software can be achieved, and the easier it is to reproduce + issues +* The simpler the process of re-creating the environment, the more reliable + software can be achieved, and the faster the process can be. -By making it trivial to copy a virtual environment from one host to another, it -avoids these categories of problems. Additionally, the development tools to -create a virtual environment and install its dependencies aren't needed on the -host that intends to run the program. +Making it trivial to copy a virtual environment from one host to another avoids +these categories of problems. Additionally, the development tools to create a +virtual environment and install its dependencies aren't needed on the host that +intends to run the program. Because the virtual environment doesn't require modifications to be usable, it also allows more advanced deployment mechanisms, e.g. remote mounting and @@ -73,7 +75,7 @@ Specification ============= The ``home`` value in ``pyvenv.cfg`` is permitted to use a relative path value. -These may contain up-references outside of the virtual environnment root +These may contain up-references outside of the virtual environment root directory. Examples: * ``subdir/whatever/bin`` (a directory within the virtual environment). @@ -102,23 +104,23 @@ The Python runtime itself *almost* already supports relative paths. The primitives are there, so the only change needed is to define how it resolves relative paths for ``home`` in ``pyvenv.cfg``. -Today, relative paths resolve relative to the process's current working +Currently, relative paths resolve relative to the process's current working directory. Because CWD isn't knowable in advance, it makes relative paths today -effective impossible. +effectively impossible. Instead, the paths should be relative to the location of the ``pyvenv.cfg`` file. This file is chosen as the anchor point because the tool that creates the file also has to know where the Python runtime is, so can easily calculate the correct relative path. For tools that read the ``pyvenv.cfg``, it is also easy -to simple join the directory name of where ``pyvenv.cfg`` was found with the +to simply join the directory name of where ``pyvenv.cfg`` was found with the path in the config file. When a person reads the config file, they can do -similar, which is a lower cognitive burden and helps avoids the question of -"relative to what?" +something similar, which results in a lower cognitive burden and helps avoid +the question of "relative to what?" -This change is only a couple of lines in the start up code. Specifically, when +This change is only a couple of lines in the startup code. Specifically, when parsing the ``pyvenv.cfg`` file and finding the ``home`` value, it just needs -to be checked if it's already absolute. If not, then join to the directory name -of the ``pyvenv.cfg`` file. The code already knows the directory and has +to be checked if it's already absolute. If not, then join it to the directory +name of the ``pyvenv.cfg`` file. The code already knows the directory and has helpers already present for checking if a path is absolute and joining two paths. @@ -129,7 +131,7 @@ Backwards Compatibility ======================= Because relative paths for ``home`` aren't usable in practice, and their -behavior undocumented and unspecified, there shouldn't be any backward +behavior is undocumented and unspecified, there shouldn't be any backward compatibility concerns. @@ -176,16 +178,16 @@ Relative to virtual env root ---------------------------- Having the ``home`` value in ``pyvenv.cfg`` relative to the virtual -environments root directory would work just as well, but this idea is rejected +environment's root directory would work just as well, but this idea is rejected because it requires additional effort to compute the virtual env root. Unspecified home means to dynamically compute home ---------------------------------------------------- Today, if a ``pyvenv.cfg`` file doesn't set ``home``, the runtime will try to -dynamically compute it by checking if the current executable (which is typicall -the venv's ``bin/python3`` symlink) is a symlink and, if so, use where that -points as ``PYTHONHOME``. +dynamically compute it by checking if the current executable (which is +typically the venv's ``bin/python3`` symlink) is a symlink and, if so, use +where that points as ``PYTHONHOME``. This behavior is undesirable for a couple reasons: @@ -201,10 +203,10 @@ In general, symlinks work best when they aren't special cased by consumers. Using the term "relocatable" ---------------------------- -Discussions pointed out the the term "relocatable" is somewhat ambigious and +Discussions pointed out that the term "relocatable" is somewhat ambiguous and misleading for a couple reasons. -First, absolute paths makes a venv arbitrarily relocatable *within* a host, but +First, absolute paths make a venv arbitrarily relocatable *within* a host, but not between hosts, so "relocatable" requires *some* qualification for clarity. @@ -226,7 +228,7 @@ not feasible because of how layers of programs calling programs interact. Code generally assumes that any virtual environment will be automatically detected and activated by the presence of ``pyvenv.cfg``, so -things work better when alterations to the environemtn aren't a concern. +things work better when alterations to the environment aren't a concern. Copyright ========= From 3240227d71940e5abd90414bb6a9f2e60dd5864e Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 2 Jul 2025 10:37:48 -0700 Subject: [PATCH 25/29] Apply suggestions from code review Co-authored-by: Alyssa Coghlan --- peps/pep-0796.rst | 41 ++++++++++++++++++++++++++++++----------- 1 file changed, 30 insertions(+), 11 deletions(-) diff --git a/peps/pep-0796.rst b/peps/pep-0796.rst index d0ee13cc404..7107d9ab6ae 100644 --- a/peps/pep-0796.rst +++ b/peps/pep-0796.rst @@ -43,7 +43,7 @@ requirement should be relaxed and relative path behavior allowed and defined. Second, such relative paths are a building block to enable portable virtual environments, i.e. copying a virtual environment as-is between hosts of compatible platforms. Portable venvs are appealing because virtual environments -are the standard way for running Python applications. This provides several +are a popular mechanism for running Python applications. This provides several benefits: * The closer the development environment is to the non-development environment, @@ -52,14 +52,16 @@ benefits: * The simpler the process of re-creating the environment, the more reliable software can be achieved, and the faster the process can be. -Making it trivial to copy a virtual environment from one host to another avoids +Making it simpler to copy a virtual environment from one host to another mitigates these categories of problems. Additionally, the development tools to create a virtual environment and install its dependencies aren't needed on the host that intends to run the program. -Because the virtual environment doesn't require modifications to be usable, it +When the virtual environment doesn't require modifications to be usable, it also allows more advanced deployment mechanisms, e.g. remote mounting and -caching of artifacts. +caching of artifacts. While this PEP on its own isn't sufficient to enable that, it +allows tools like `bazel` or `venvstacks` to more easily prepare constrained +environments that allow for such use cases. Rationale ========= @@ -71,6 +73,11 @@ customizing how it's computed. Without the ability to specify where the supporting Python runtime files are, the interpreter can't finish startup, so other hook points (e.g. ``site`` initialization) never trigger. +Tools that currently look to enable virtual environment portability across machines do +so either by relying on undocumented interpreter behaviour (`bazel`, omitting the `home` +key entirely to trigger an implementation dependent fallback to resolving via a symlinked +interpreter binary on non-Windows systems) or by requiring a post-installation script to +be executed after the environment is placed in its target location (`venvstacks`). Specification ============= @@ -97,10 +104,10 @@ For example, given i.e. it's equivalent to using that value verbatim in ``pyvenv.cfg``. -Python Runtime Changes +CPython Runtime Changes ====================== -The Python runtime itself *almost* already supports relative paths. The +The CPython runtime itself *almost* already supports relative paths. The primitives are there, so the only change needed is to define how it resolves relative paths for ``home`` in ``pyvenv.cfg``. @@ -130,9 +137,19 @@ A proof-of-concept of this is implemented in Backwards Compatibility ======================= -Because relative paths for ``home`` aren't usable in practice, and their -behavior is undocumented and unspecified, there shouldn't be any backward -compatibility concerns. + +Tools that work around the absolute `home` key limitation the way `bazel` and `venvstacks` +currently do (omitting the `home` key, or editing it after moving the environment) will be +unaffected. + +While the PEP author and sponsor aren't aware of any projects that work around the limitation +by carefully controlling the current working directory used to launch the deployed Python +environments on target systems, any such projects would be unaffected if they already ensured +the working directory was set to the folder containing `pyvenv.cfg` (which seems like a plausible +choice, since that is typically the root directory of the virtual environment). In the even more +unlikely case where that assumption doesn't hold, tools generating relative virtual environment +paths will typically be aware of the underlying base runtime Python version, and hence able to +update the emitted relative path accordingly. How to Teach This @@ -158,7 +175,7 @@ This PEP does not specify how to create a ``pyvenv.cfg`` with a relative path, nor how downstream tools (e.g. installers) should identify them or process them. These questions are best addressed separately by tool owners. -Footnotes +References ========= * `rules_python `__: implements @@ -166,6 +183,7 @@ Footnotes * `rules_py `__: implements host-relocatable virtual environments. * `python-build-standalone `__: +* `venvstacks `__: a tool for creating reproducible distribution artifacts from virtual environments A relocatable Python runtime. * `PoC for relative home in Python startup `__ * `Python Ideas "Making venvs relocatable friendly" discussion `__ @@ -189,7 +207,8 @@ dynamically compute it by checking if the current executable (which is typically the venv's ``bin/python3`` symlink) is a symlink and, if so, use where that points as ``PYTHONHOME``. -This behavior is undesirable for a couple reasons: +While currently used as a workaround by some tools, *standardising* this behavior +is undesirable for a couple reasons: 1. It presents platform-specific issues, namely with Windows. Windows does support symlinks, but not by default, and it can require special From 557cb71623ea75fb523959a62fe18fc249926a47 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 2 Jul 2025 10:41:00 -0700 Subject: [PATCH 26/29] fix up markdown and text width --- peps/pep-0796.rst | 54 +++++++++++++++++++++++++---------------------- 1 file changed, 29 insertions(+), 25 deletions(-) diff --git a/peps/pep-0796.rst b/peps/pep-0796.rst index 7107d9ab6ae..98bf192a8a4 100644 --- a/peps/pep-0796.rst +++ b/peps/pep-0796.rst @@ -52,16 +52,16 @@ benefits: * The simpler the process of re-creating the environment, the more reliable software can be achieved, and the faster the process can be. -Making it simpler to copy a virtual environment from one host to another mitigates -these categories of problems. Additionally, the development tools to create a -virtual environment and install its dependencies aren't needed on the host that -intends to run the program. +Making it simpler to copy a virtual environment from one host to another +mitigates these categories of problems. Additionally, the development tools to +create a virtual environment and install its dependencies aren't needed on the +host that intends to run the program. When the virtual environment doesn't require modifications to be usable, it also allows more advanced deployment mechanisms, e.g. remote mounting and -caching of artifacts. While this PEP on its own isn't sufficient to enable that, it -allows tools like `bazel` or `venvstacks` to more easily prepare constrained -environments that allow for such use cases. +caching of artifacts. While this PEP on its own isn't sufficient to enable +that, it allows tools like ``bazel`` or ``venvstacks`` to more easily prepare +constrained environments that allow for such use cases. Rationale ========= @@ -73,11 +73,13 @@ customizing how it's computed. Without the ability to specify where the supporting Python runtime files are, the interpreter can't finish startup, so other hook points (e.g. ``site`` initialization) never trigger. -Tools that currently look to enable virtual environment portability across machines do -so either by relying on undocumented interpreter behaviour (`bazel`, omitting the `home` -key entirely to trigger an implementation dependent fallback to resolving via a symlinked -interpreter binary on non-Windows systems) or by requiring a post-installation script to -be executed after the environment is placed in its target location (`venvstacks`). +Tools that currently look to enable virtual environment portability across +machines do so either by relying on undocumented interpreter behaviour +(``bazel``, omitting the ``home`` key entirely to trigger an implementation +dependent fallback to resolving via a symlinked interpreter binary on +non-Windows systems) or by requiring a post-installation script to be executed +after the environment is placed in its target location (``venvstacks``). + Specification ============= @@ -138,18 +140,20 @@ Backwards Compatibility ======================= -Tools that work around the absolute `home` key limitation the way `bazel` and `venvstacks` -currently do (omitting the `home` key, or editing it after moving the environment) will be -unaffected. +Tools that work around the absolute ``home`` key limitation the way ``bazel`` +and ``venvstacks`` currently do (omitting the ``home`` key, or editing it after +moving the environment) will be unaffected. -While the PEP author and sponsor aren't aware of any projects that work around the limitation -by carefully controlling the current working directory used to launch the deployed Python -environments on target systems, any such projects would be unaffected if they already ensured -the working directory was set to the folder containing `pyvenv.cfg` (which seems like a plausible -choice, since that is typically the root directory of the virtual environment). In the even more -unlikely case where that assumption doesn't hold, tools generating relative virtual environment -paths will typically be aware of the underlying base runtime Python version, and hence able to -update the emitted relative path accordingly. +While the PEP author and sponsor aren't aware of any projects that work around +the limitation by carefully controlling the current working directory used to +launch the deployed Python environments on target systems, any such projects +would be unaffected if they already ensured the working directory was set to +the folder containing ``pyvenv.cfg`` (which seems like a plausible choice, +since that is typically the root directory of the virtual environment). In the +even more unlikely case where that assumption doesn't hold, tools generating +relative virtual environment paths will typically be aware of the underlying +base runtime Python version, and hence able to update the emitted relative path +accordingly. How to Teach This @@ -207,8 +211,8 @@ dynamically compute it by checking if the current executable (which is typically the venv's ``bin/python3`` symlink) is a symlink and, if so, use where that points as ``PYTHONHOME``. -While currently used as a workaround by some tools, *standardising* this behavior -is undesirable for a couple reasons: +While currently used as a workaround by some tools, *standardising* this +behavior is undesirable for a couple reasons: 1. It presents platform-specific issues, namely with Windows. Windows does support symlinks, but not by default, and it can require special From 057962374c5ff024fc88be8a53c0bc6a22be0229 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 2 Jul 2025 10:44:20 -0700 Subject: [PATCH 27/29] fix title underlines --- peps/pep-0796.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/peps/pep-0796.rst b/peps/pep-0796.rst index 98bf192a8a4..8e64dbe534d 100644 --- a/peps/pep-0796.rst +++ b/peps/pep-0796.rst @@ -107,7 +107,7 @@ i.e. it's equivalent to using that value verbatim in ``pyvenv.cfg``. CPython Runtime Changes -====================== +======================= The CPython runtime itself *almost* already supports relative paths. The primitives are there, so the only change needed is to define how it resolves @@ -180,7 +180,7 @@ nor how downstream tools (e.g. installers) should identify them or process them. These questions are best addressed separately by tool owners. References -========= +========== * `rules_python `__: implements host-relocatable virtual environments. From c9388b3a93d6340cc921723a26ee0243319e5cc8 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 2 Jul 2025 13:08:46 -0700 Subject: [PATCH 28/29] link to relative venv reference implementation --- peps/pep-0796.rst | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/peps/pep-0796.rst b/peps/pep-0796.rst index 8e64dbe534d..3cc10fe838e 100644 --- a/peps/pep-0796.rst +++ b/peps/pep-0796.rst @@ -170,7 +170,10 @@ Reference Implementation A reference implementation is available by using the combination of: * Python runtime from `rickeylev/feat.relative.pyvenv.home `__ -* rules_python with (todo: link to branch that uses above) +* Relative venv from `rickeylev/relvenv `__ + +And following the +`relvenv README `__. Open Issues =========== From b959f2435209ffef03dcacb0b7f4fefe4cd09cf9 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 6 Jul 2025 15:59:52 -0700 Subject: [PATCH 29/29] Apply suggestions from code review Co-authored-by: Alyssa Coghlan --- peps/pep-0796.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/peps/pep-0796.rst b/peps/pep-0796.rst index 3cc10fe838e..e35ba5d66ce 100644 --- a/peps/pep-0796.rst +++ b/peps/pep-0796.rst @@ -159,9 +159,10 @@ accordingly. How to Teach This ================= -Teaching this should be simple: if you use a relative path in ``pyvenv.cfg``, +Teaching this should be straightforward: if you use a relative path in ``pyvenv.cfg``, then it's relative to the directory containing the ``pyvenv.cfg`` file. This -is simple to explain and understand. +is simple to explain and easy to understand for anyone that is already familiar +with handling relative filesystem paths. Reference Implementation