Write up to the specification

python · brettcannon · Jul 24, 2024 · Mar 20, 2024 · Mar 21, 2024 · Mar 21, 2024
commit 5e9e27461685738b237bd47538974765f9c7bbba
diff --git a/peps/pep-9999.rst b/peps/pep-9999.rst
@@ -0,0 +1,158 @@
+PEP: <REQUIRED: pep number>
+Title: A file format to list Python dependencies for installation reproducibility
+Author: Brett Cannon <brett@python.org>
+PEP-Delegate: <PEP delegate's real name>
+Discussions-To: <REQUIRED: URL of current canonical discussion thread>
+Status: Draft
+Type: Standards Track
+Topic: Packaging
+Created: <date created on, in dd-mmm-yyyy format>
+Post-History: <REQUIRED: dates, in dd-mmm-yyyy format, and corresponding links to PEP discussion threads>
+Replaces: 665
+
+Abstract
+========
+
+[A short (~200 word) description of the technical issue being addressed.]
+
+This PEP proposes a new file format for specifying what dependencies to
+(possibly) install into a Python environment for consitent installation
+repoducibility. The format is designed to be human-readable but
+machine-generated. Installers consuming the file should also not need to resolve
+which listed dependencies to install, but instead evaluate each dependency in
+question in isolation.
+
+
+Motivation
+==========
+
+Currently, there is no standard way to specify what top-level dependencies one
+would like to see installed into a Python environment and then subsequently
+record what war eventually installed in a file (called a *lock file*).
+Considering there are four well-known solutions to this problem in the
+community (``pip freeze``, pip-tools_, Poetry_, and PDM_), there seems to be an
+appetite for lock files in general.
+
+Those tools also vary in what locking scenarios they approach. For instance,
+``pip freeze`` and pip-tools only generate lock files for the current
+environment while PDM and Poetry try to lock for *any* environment to some
+degree. And none of them directly support locking to specific files to install
+which can be important for some workflows. There's also concerns around secure
+defaults in the face of supply chain attacks. Finally, not all the formats are
+easy to audit to determine what what would be installed into an environment
+ahead of time.
+
+Ghe lack of a standard also has some drawbacks. For instance, any tooling that
+wants to work with lock files must choose which format to support, potentially
+leaving users unsupported (e.g. if Dependabot_ chose not to support PDM.
+support by cloud providers who can do dependency installations on your behalf,
+etc.).
+
+
+Rationale
+=========
+
+[Describe why particular design decisions were made.]
+
+The file format is designed to be human-readable. This is
+so that the contents of the file can be audited by a human to make sure no
+undesired dependencies end up being included in the lock file. It is also to
+facilitate understanding what would be installed if the lock file were to be
+used without necessitating running a tool, once again to help with auditing.
+
+The file format is also designed to not require a resolver at install time. Being
+able to analyze dependencies in isolation from one another when listed in a lock
+file provides a few benefits. First, it supports auditing by making it easy to
+figure out if a certain dependency would be installed for a certain environment.
+It should also lead to faster installs which are much more frequent than
+creating a lock file. Finally, the four tools mentioned in the Motivation
+section either already implement this approach of evaluating dependencies in
+isolation or have suggested they could in
+`Poetry's case <https://discuss.python.org/t/lock-files-again-but-this-time-w-sdists/46593/83>`__.
+
+The lock file format is designed to support two locking scenarios. The first is
+locking to specific files for an environment. This is to support workflows where
+the environment(s) are known ahead of time and specifying what exactly to
+install is important. This is similar to what ``pip freeze`` and pip-tools_
+support, but with more strictness of the exact files as well as incorporating
+into the file format the ability to specify the locked files for multiple
+environments.
+
+The second locking scenario is to lock to package versions. This is the approach
+taken by PDM_ (and indirectly by Poetry_). This is to support workflows where
+the environment(s) are not known ahead of time. This can happen in open source
+projects where contributors can have a myriad of different environments but
+there's a desire to lock the packages used for e.g. building the project's
+documentation. Since trying to lock the files for all possible environments
+could be tedious at best and very expensive at worst, locking to the package
+acts as a pragmatic compromise.
+
+
+Specification
+=============
+
+[Describe the syntax and semantics of any new language feature.]
+
+XXX core metadata 2.4
+
+XXX file name
+
+XXX file format
+
+XXX expectations for lockers
+
+XXX expectations for installers
+
+Backwards Compatibility
+=======================
+
+[Describe potential impact and severity on pre-existing code.]
+
+
+Security Implications
+=====================
+
+[How could a malicious user take advantage of this new feature?]
+
+
+How to Teach This
+=================
+
+[How to teach users, new and experienced, how to apply the PEP to their work.]
+
+
+Reference Implementation
+========================
+
+[Link to any existing implementation and details about its state, e.g. proof-of-concept.]
+
+
+Rejected Ideas
+==============
+
+[Why certain ideas that were brought while discussing this PEP were not ultimately pursued.]
+
+
+Open Issues
+===========
+
+[Any points that are still being decided/discussed.]
+
+
+Footnotes
+=========
+
+[A collection of footnotes cited in the PEP, and a place to list non-inline hyperlink targets.]
+
+
+Copyright
+=========
+
+This document is placed in the public domain or under the
+CC0-1.0-Universal license, whichever is more permissive.
+
+
+_Dependabot: https://docs.github.com/en/code-security/dependabot
+_PDM: https://pypi.org/project/pdm/
+_pip-tools: https://pypi.org/project/pip-tools/
+_Poetry: https://python-poetry.org/