diff --git a/.gitignore b/.gitignore index c74e78d0fa..bc095e99dd 100644 --- a/.gitignore +++ b/.gitignore @@ -81,12 +81,20 @@ celerybeat-schedule .env # virtualenv -venv/ +venv*/ ENV/ -venv/ # Spyder project settings .spyderproject # Rope project settings .ropeproject + +# Eclipse project settings +.project +.settings/ + +# PyCharm project settings +.idea/ + + diff --git a/.readthedocs.yaml b/.readthedocs.yaml new file mode 100644 index 0000000000..c072c1c320 --- /dev/null +++ b/.readthedocs.yaml @@ -0,0 +1,31 @@ +# .readthedocs.yaml + +# Read the Docs configuration file for Sphinx projects +# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details + +# Required +version: 2 + +# Set the OS, Python version and other tools you might need +build: + os: ubuntu-22.04 + tools: + python: "3.12" + # You can also specify other tool versions + +# Build documentation in the root directory with Sphinx +sphinx: + configuration: ./conf.py + +# Optionally build additional formats such as PDF and ePub +formats: + - htmlzip +# - pdf + - epub + +# Optional but recommended, declare the Python requirements required +# to build your documentation +# See https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html +python: + install: + - requirements: ./requirements.txt diff --git a/.travis.yml b/.travis.yml deleted file mode 100644 index a29774d73e..0000000000 --- a/.travis.yml +++ /dev/null @@ -1,9 +0,0 @@ -language: python -python: 3.6 -cache: pip - -install: python3 -m pip install sphinx - -script: - # TODO: add -b linkcheck - - sphinx-build -n -W -q -b html -d _build/doctrees . _build/html diff --git a/README.rst b/README.rst index 47cb09f09f..1b26afad11 100644 --- a/README.rst +++ b/README.rst @@ -1,15 +1,21 @@ -The CPython Developer's Guide -============================= +The Jython Developer's Guide +============================ -.. image:: https://readthedocs.org/projects/cpython-devguide/badge/ +.. This will be wrong now, but we need something similar: + image:: https://readthedocs.org/projects/python-devguide/badge/ :target: https://devguide.python.org :alt: Documentation Status +This guide is a straight copy of the CPython Developer's Guide, with +the idea of adapting it to describe Jython development processes +using GitHub. +The difference is made by adding files for chapters that need to be +different from the CPython edition, +keeping the disused CPython files, +and changing other files as little as possible. +That way may continue to pull changes from CPython's devguide. -This guide covers how to contribute to CPython. It is known by the -nickname of "the devguide" by the Python core developers. - -The official home of this guide is https://devguide.python.org. +The official home of this guide is still to be chosen. Compilation ----------- diff --git a/buildbots_jy.rst b/buildbots_jy.rst new file mode 100644 index 0000000000..00de7eeef2 --- /dev/null +++ b/buildbots_jy.rst @@ -0,0 +1,173 @@ +.. Jython companion to buildbots.rst + +.. _buildbots-jy: + +Continuous Integration +====================== + +.. warning:: At present, this is not much more than a copy of the CPython original + with the obviously inapplicable crudely hacked out. + +.. note:: This description, while aimed at the future process, is applicable to + PRs raised today, as far as test goes, but we merge the PR by a round-about + route using Mercurial. It is also fair to point out that we have difficulty + keeping these buildbots green. + +To assert that there are no regressions in the :doc:`development and maintenance +branches `, Jython uses continuous integration services, integrated +into GitHub. +When a new commit appears in a PR on, all corresponding builders +will build and run the regression tests. + +The build steps run by the buildbots are the following: + +* Checkout of the source tree for the changeset which triggered the build +* Compiling Jython for a particular JVM +* Running the test suite +* Cleaning up + +It is your responsibility, as a core developer, to check the automatic +build results as part of assessing a PR. It is therefore +important that you get acquainted with the way these results are presented, +and how various kinds of failures can be explained and diagnosed. + +Checking results of automatic builds +------------------------------------ + +The way to view recent build results is to go to the list of commits on GitHub +and click on the little red cross (or whatever the symbol is when a build +passes). In a PR-based process, the PR page itself contains a panel which shows +what tests would pass or fail if the PR were merged. + +When several changes are committed in a quick succession in the same +branch, it often happens that only the latest is tested. + + +Stability +--------- + +At the time of writing, it is common for one or more build bots +to show failures that are due to configuration errors or transient failures. + +The rule is that all stable builders must be free of +persistent failures when the release is cut. It is absolutely **vital** +that core developers fix any issue they introduce on the buildbots, +as soon as possible. + + +Flags-dependent failures +------------------------ + +Sometimes, while you have run the :doc:`whole test suite ` before +committing, you may witness unexpected failures on the buildbots. One source +of such discrepancies is if different flags have been passed to the test runner +or to Python itself. To reproduce, make sure you use the same flags as the +buildbots: they can be found out simply by clicking the **stdio** link for +the failing build's tests. For example:: + + ant regrtest + +.. note:: Mention subtly different ant targets. Is this a problem? + +.. note:: + Running ``Lib/test/regrtest.py`` is nearly equivalent to running + ``-m test``. + +Ordering-dependent failures +--------------------------- + +.. warning:: CPython content + +Sometimes the failure is even subtler, as it relies on the order in which +the tests are run. The buildbots *randomize* test order (by using the ``-r`` +option to the test runner) to maximize the probability that potential +interferences between library modules are exercised; the downside is that it +can make for seemingly sporadic failures. + +The ``--randseed`` option makes it easy to reproduce the exact randomization +used in a given build. Again, open the ``stdio`` link for the failing test +run, and check the beginning of the test output proper. + +Let's assume, for the sake of example, that the output starts with:: + + ./python -Wd -E -bb Lib/test/regrtest.py -uall -rwW + == CPython 3.3a0 (default:22ae2b002865, Mar 30 2011, 13:58:40) [GCC 4.4.5] + == Linux-2.6.36-gentoo-r5-x86_64-AMD_Athlon-tm-_64_X2_Dual_Core_Processor_4400+-with-gentoo-1.12.14 little-endian + == /home/buildbot/buildarea/3.x.ochtman-gentoo-amd64/build/build/test_python_29628 + Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=1, verbose=0, bytes_warning=2, quiet=0) + Using random seed 2613169 + [ 1/353] test_augassign + [ 2/353] test_functools + +You can reproduce the exact same order using:: + + ./python -Wd -E -bb -m test -uall -rwW --randseed 2613169 + +It will run the following sequence (trimmed for brevity):: + + [ 1/353] test_augassign + [ 2/353] test_functools + [ 3/353] test_bool + [ 4/353] test_contains + [ 5/353] test_compileall + [ 6/353] test_unicode + +If this is enough to reproduce the failure on your setup, you can then +bisect the test sequence to look for the specific interference causing the +failure. Copy and paste the test sequence in a text file, then use the +``--fromfile`` (or ``-f``) option of the test runner to run the exact +sequence recorded in that text file:: + + ./python -Wd -E -bb -m test -uall -rwW --fromfile mytestsequence.txt + +In the example sequence above, if ``test_unicode`` had failed, you would +first test the following sequence:: + + [ 1/353] test_augassign + [ 2/353] test_functools + [ 3/353] test_bool + [ 6/353] test_unicode + +And, if it succeeds, the following one instead (which, hopefully, shall +fail):: + + [ 4/353] test_contains + [ 5/353] test_compileall + [ 6/353] test_unicode + +Then, recursively, narrow down the search until you get a single pair of +tests which triggers the failure. It is very rare that such an interference +involves more than **two** tests. If this is the case, we can only wish you +good luck! + +.. note:: + You cannot use the ``-j`` option (for parallel testing) when diagnosing + ordering-dependent failures. Using ``-j`` isolates each test in a + pristine subprocess and, therefore, prevents you from reproducing any + interference between tests. + + +Transient failures +------------------ + +While we try to make the test suite as reliable as possible, some tests do +not reach a perfect level of reproducibility. Some of them will sometimes +display spurious failures, depending on various conditions. Here are common +offenders: + +* Network-related tests, such as ``test_poplib``, ``test_urllibnet``, etc. + Their failures can stem from adverse network conditions, or imperfect + thread synchronization in the test code, which often has to run a + server in a separate thread. + +* Tests dealing with delicate issues such as inter-thread or inter-process + synchronization, or Unix signals: ``test_multiprocessing``, + ``test_threading``, ``test_subprocess``, ``test_threadsignals``. + +When you think a failure might be transient, it is recommended you confirm by +waiting for the next build. Still, even if the failure does turn out sporadic +and unpredictable, the issue should be reported on the bug tracker; even +better if it can be diagnosed and suppressed by fixing the test's implementation, +or by making its parameters - such as a timeout - more robust. + + diff --git a/committing.rst b/committing.rst index 2e64c3fbe4..c797ec8bef 100644 --- a/committing.rst +++ b/committing.rst @@ -1,8 +1,14 @@ +.. This file is derived from a file of the same name in the CPython devguide + and will receive updates from the CPython guide by merging. + .. _committing: Accepting Pull Requests ======================= +.. warning:: At present, this is not much modified from the CPython base. + + This page is aimed to core developers, and covers the steps required to accept, merge, and possibly backport a pull request on the main repository. @@ -30,8 +36,9 @@ merging (details of various steps can be found later in this document): .. note:: If you want to share your work-in-progress code on a feature or bugfix, either open a ``WIP``-prefixed PR, publish patches on the - `issue tracker`_ or create a public fork of the repository. + `Jython issue tracker`_ or create a public fork of the repository. +.. _Jython issue tracker: https://bugs.jython.org .. _issue tracker: https://bugs.python.org @@ -47,7 +54,7 @@ passes before merging any code changes. or library modules. Running the entire test suite doesn't guarantee that the changes - will pass the :ref:`continuous integration ` tests, as those + will pass the :ref:`continuous integration ` tests, as those will exercise more possibilities still (such as different platforms or build options). But it will at least catch non-build specific, non-platform specific errors, therefore minimizing the chance for diff --git a/committing_hg_jy.rst b/committing_hg_jy.rst new file mode 100644 index 0000000000..a2978194d9 --- /dev/null +++ b/committing_hg_jy.rst @@ -0,0 +1,584 @@ +.. Jython and Mercurial-specific companion to committing.rst, + needed while we use Mercurial. + +.. _committing-hg-jy: + +Committing and Pushing Changes (Mercurial) +========================================== + +.. note:: + This has been adapted for Jython from an old version of the CPython devguide, + to describe the Jython process based on Mercurial. + +Is the change ready for committing? +----------------------------------- + +Before a change is committed, you must make sure it is ready to enter the +public source tree. Draft commits are prohibited. Therefore, you must +ensure your changes fulfill several mandatory criteria. + +.. note:: + If you want to share your work-in-progress code on a feature or bugfix, + either publish patches or create a public fork of the repository. + + +Does the test suite still pass? +''''''''''''''''''''''''''''''' + +You must :ref:`run the whole test suite ` to ensure that it +passes before pushing any code changes. + +.. note:: + You really need to run the **entire** test suite. Running a single test + is not enough as your changes may have unforeseen effects on other tests + or library modules. + + Running the entire test suite doesn't guarantee that your changes + will pass the :ref:`continuous integration ` tests, as those + will exercise more possibilities still (such as different platforms or + build options). But it will at least catch non-build specific, + non-platform specific errors, therefore minimizing the chance for + breakage. + +Patch checklist +''''''''''''''' + +Apart from running the tests, there's a simple patch checklist that +``make patchcheck`` (or ``./python.exe Tools/scripts/patchcheck.py`` on +Windows) will run through: + +* Are there any whitespace problems in Python files? + (using ``Tools/scripts/reindent.py``) +* Are there any whitespace problems in C files? +* Are there any whitespace problems in the documentation? + (using ``Tools/scripts/reindent-rst.py``) +* Has the documentation been updated? +* Has the test suite been updated? +* Has ``Misc/NEWS`` been updated? +* Has ``Misc/ACKS`` been updated? +* Has ``configure`` been regenerated, if necessary? +* Has ``pyconfig.h.in`` been regenerated, if necessary? + +Note that the automated patch check can't actually *answer* all of these +questions, and even if it could, it still wouldn't know whether or not +those answers were appropriate. Aside from the whitespace checks, it is just +a memory aid to help with remembering the various elements that can go into +making a complete patch. + + +Commit Style +------------ + +Once a change patch is ready and tested, it can be committed to the repository. +We usually prefer to put a whole feature or bugfix into a single commit, but no +more. In particular: + +* Do **not** fix more than one issue in the same commit (except, of course, if + one code change fixes all of them). +* Do **not** do cosmetic changes to unrelated code in the same commit as some + feature/bugfix. + +It is of course okay to pile up several commits to one branch and merge them +into another in one merge commit. + + +Handling Others' Code +--------------------- + +As a core developer you will occasionally want to commit a patch created by +someone else. When doing so you will want to make sure of some things. + +First, make sure the patch is in a good state. Both :ref:`patch-hg-jy` and +:ref:`helptriage` +explain what is to be expected of a patch. Typically patches that get cleared by +triagers are good to go except maybe lacking ``Misc/ACKS`` and ``Misc/NEWS`` +entries. + +Second, make sure the patch does not break backwards-compatibility without a +good reason. This means :ref:`running the entire test suite ` to +make sure everything still passes. It also means that if semantics do change +there must be a good reason for the breakage of code the change will cause +(and it **will** break someone's code). If you are unsure if the breakage +is worth it, ask on python-dev. + +Third, ensure the patch is attributed correctly by adding the contributor's +name to ``Misc/ACKS`` if they aren't already there (and didn't add themselves +in their patch) and by mentioning "Patch by " in the ``Misc/NEWS`` entry +and the checkin message. If the patch has been heavily modified then "Initial +patch by " is an appropriate alternate wording. + +If you omit correct attribution in the initial checkin, then update ``ACKS`` +and ``NEWS`` in a subsequent checkin (don't worry about trying to fix the +original checkin message in that case). + +Finally, especially for larger patches, check if the submitter of the +patch has a CLA in place (indicated by an asterisk following their username +in the issue tracker). If the asterisk is missing and the patch is +non-trivial, direct them to the electronic `Contributor Licensing Agreement`_ +to ensure the PSF has the appropriate authorizations in place to relicense +and redistribute their code. + + +Contributor Licensing Agreements +-------------------------------- + +It's unlikely bug fixes will require a `Contributor Licensing Agreement`_ +unless they touch a *lot* of code. For new features, it is preferable to +ask that the contributor submit a signed CLA to the PSF as the associated +comments, docstrings and documentation are far more likely to reach a +copyrightable standard. + +These days, the CLA can be signed electronically through the form linked +above, and this process is strongly preferred to the old mechanism that +involved sending a scanned copy of the signed paper form. + +As discussed on the PSF Contribution_ page, it is the CLA itself that gives +the PSF the necessary relicensing rights to redistribute contributions under +the Python license stack. This is an additional permission granted above and +beyond the normal permissions provided by the chosen open source license. + +Some developers may object to the relicensing permissions granted to the PSF +by the CLA. They're entirely within their rights to refuse to sign the CLA +on that basis, but that refusal *does* mean we can't accept their patches +for inclusion. + +.. _Contribution: http://www.python.org/psf/contrib/ +.. _Contributor Licensing Agreement: + http://www.python.org/psf/contrib/contrib-form/ + + +NEWS Entries +------------ + +Almost all changes made to the code base deserve an entry in ``Misc/NEWS``. +If the change is particularly interesting for end users (e.g. new features, +significant improvements, or backwards-incompatible changes), an entry in +the ``What's New in Python`` document (in ``Doc/whatsnew/``) should be added +as well. There are two notable exceptions to this general principle, and they +both relate to changes that *already* have a NEWS entry, and have not yet +been included in any formal release (including alpha and beta releases). +These exceptions are: + +* If a change is reverted prior to release, then the corresponding entry + is simply removed. Otherwise, a new entry must be added noting that the + change has been reverted (e.g. when a feature is released in an alpha and + then cut prior to the first beta). + +* If a change is a fix (or other adjustment) to an earlier unreleased change + and the original NEWS entry remains valid, then no additional entry is + needed. + +New NEWS entries are customarily added at or near the top of their +respective sections, so that entries within a section appear in approximate +order from newest to oldest. However, this is customary and not a +requirement. + +The NEWS file is now read by Sphinx to produce the "Changelog" page; accordingly +it should be valid reStructuredText. The "default role" (single backticks) can +be used to refer to objects in the documentation. Example NEWS entry:: + + - Issue #15304: Fix warning message when `os.chdir()` fails inside + `test.support.temp_cwd()`. Patch by Chris Jerdonek. + +(In all other ``.rst`` files, the single backticks should not be used. They are +allowed here because NEWS is meant to be as readable as possible unprocessed.) + +A nice trick to make Mercurial’s automatic file merge work more smoothly is to +put a new entry after the first or first two entries rather than at the very +top. This way if you commit, pull new changesets and merge, the merge will +succeed automatically. + + +Commit Messages +--------------- + +Every commit has a commit message to document why a change was made and to +communicate that reason to other core developers. Python core developers have +developed a standard way of formatting commit messages that everyone is +expected to follow. + +Our usual convention mimics that used in the ``Misc/NEWS`` file. Actually, +it is common to simply paste the NEWS entry into the commit message. Here +is an example:: + + Issue #42: the spam module is now more spammy. + The spam module sporadically came up short on spam. This change + raises the amount of spam in the module by making it more spammy. + Thanks to Monty Python for the patch. + +The first line or sentence is meant to be a dense, to-the-point explanation +of what the purpose of the commit is. If this is not enough detail for a commit, +a new paragraph(s) can be added to explain in proper depth what has happened +(detail should be good enough that a core developer reading the commit message +understands the justification for the change). Also, if a non-core developer +contributed to the resolution, it is good practice to credit them. + +Mercurial hooks +''''''''''''''' + +Special hooks have been added to the Mercurial repository to enable notifying +the issue tracker of a commit related to an issue. + +A commit message can mention one or several issues in one of the following +ways:: + + #12345 + issue12345 + issue 12345 + bug12345 + bug 12345 + +where 12345 is the number of the issue. The commit details (including its +changeset, branch and commit message) will then be posted as a message to the +issue's page in the tracker, for each mentioned issue. + +If "closes" (or "closed", or "closing") is prepended, the issue is +automatically closed as "fixed". + +Working with Mercurial_ +======================= + +As a core developer, the ability to push changes to the official Python +repositories means you have to be more careful with your workflow: + +* You should not push new named branches to the main repository. You can + still use them in clones that you use for development of patches; you can + also push these branches to a **separate** public repository that will be + dedicated to maintenance of the work before the work gets integrated in the + main repository. + +* You should collapse changesets of a single feature or bugfix before pushing + the result to the main repository. The reason is that we don't want the + history to be full of intermediate commits recording the private history + of the person working on a patch. If you are using the rebase_ extension, + consider adding the ``--collapse`` option to ``hg rebase``. The collapse_ + extension is another choice. + +Because of these constraints, it can be practical to use other approaches +such as mq_ (Mercurial Queues), in order to maintain patches in a single +local repository and to push them seamlessly when they are ready. + +It can also be useful to keep a pristine clone of the main repository around, +as it allows simple reversion of all local changes (even "committed" ones) if +your local clone gets into a state you aren't happy with. + + +.. _Mercurial: http://www.hg-scm.org/ +.. _mq: http://mercurial.selenic.com/wiki/MqExtension +.. _rebase: http://mercurial.selenic.com/wiki/RebaseExtension +.. _collapse: http://mercurial.selenic.com/wiki/CollapseExtension + + +Minimal Configuration +--------------------- + +To use Mercurial as a committer (both of your and others' patches), you should +set up some basic options in your `configuration file`_. Under Windows, +TortoiseHg has a graphical settings dialog for most options, meaning you +don't need to edit the file directly (it is still available in +``%USERPROFILE%\Mercurial.ini``). Under other platforms, you must edit +``~/.hgrc``. + +Here are the minimal options you need to activate: + +* your *username*: this setting defines the name that will be used when you + :ref:`commit ` changes. The usual convention is to also include + an e-mail contact address in there:: + + [ui] + username = Your Name + +* *extended diffing*: this setting enables an `extended diff format`_ + which is more useful than the standard unified diff format as it includes + metadata about file copies, permission bits, and is able to represent + binary files:: + + [diff] + git = on + +Under Windows, you should also enable the `eol extension`_, which will +fix any Windows-specific line endings your text editor might insert when you +create or modify versioned files. The public repository has a hook which +will reject all changesets having the wrong line endings, so enabling this +extension on your local computer is in your best interest. + + +.. _configuration file: http://www.selenic.com/mercurial/hgrc.5.html#files +.. _extended diff format: http://www.selenic.com/mercurial/hg.1.html#diffs +.. _eol extension: http://mercurial.selenic.com/wiki/EolExtension + + +Clones Setup +------------ + +There are several possible ways to set up your Mercurial clone(s). If you are +a core developer, you often need to work on the different branches, so the best +approach is to have a separate clone/directory for each active branch. If you +are a contributor, having a single clone might be enough. + +Single Clone Approach +''''''''''''''''''''' + +This approach has the advantage of being simpler because it requires a single +clone/directory, but, on the other hand, it requires you to recompile Python +every time you need to switch branch. For this reason, this approach is not +suggested to core developers, but it's usually suitable for contributors. + +See :ref:`checkout-jy` to find information about cloning and switching branches. + +.. _multiple-clones: + +Multiple Clones Approach +'''''''''''''''''''''''' + +This approach requires you to keep a separate clone/directory for each active +branch, but, on the other hand, it doesn't require you to switch branches and +recompile Python, so it saves times while merging and testing a patch on the +different branches. For this reason, this approach is suggested to core +developers. + +The easiest way to do this is by using the `share extension`_, that can be +enabled by adding the following lines to your ``~/.hgrc``:: + + [extensions] + share = + +Once you have :ref:`cloned the hg.python.org/jython repository ` +you can create the other shared clones using:: + + $ hg share jython 2.7 # create a new shared clone + $ cd 2.7 # enter the directory + $ hg up 2.7 # switch to the 2.7 branch + +You can then repeat the same operation for the other active branches. +This will create different clones/directories that share the same history. +This means that once you commit or pull new changesets in one of the clones, +they will be immediately available in all the other clones (note however that +while you only need to use ``hg pull`` once, you still need to use ``hg up`` +in each clone to update its working copy). + +If you don't want to specify ssh://hg@hg.python.org/jython every time you pull +or push, you should add to the ``.hg/hgrc`` files of the clones:: + + [paths] + default = ssh://hg@hg.python.org/jython + +In order to apply a patch, commit, and merge it on all the branches, you can do +as follow:: + + $ cd 2.7 + $ hg pull + $ hg up + $ hg import --no-c http://bugs.python.org/url/to/the/patch.diff + $ # review, run tests, run `make patchcheck` + $ hg ci -m '#12345: fix some issue.' + $ # switch to 3.4 and port the changeset using `hg graft` + $ cd ../3.4 + $ hg up + $ hg graft 2.7 + $ # switch to 3.x, merge, commit, and push everything + $ cd ../3.x + $ hg up + $ hg merge 3.4 + $ hg ci -m '#12345: merge with 3.4.' + $ hg push + +Unless noted otherwise, the rest of the page will assume you are using the +multiple clone approach, and explain in more detail these basic steps. + +For more advanced explanations about :ref:`null merges `, +:ref:`heads merges `, :ref:`merge conflicts +`, etc., see the :ref:`FAQs for core developers +`. + +.. _share extension: http://mercurial.selenic.com/wiki/ShareExtension + + +Active branches +--------------- + +If you do ``hg branches`` you will see a :ref:`list of branches `. +``default`` is the in-development branch, and is the only branch that receives +new features. The other branches only receive bug fixes or security fixes. +Depending on what you are committing (feature, bug fix, or security fix), you +should commit to the oldest branch applicable, and then forward-port until the +in-development branch. + + +Merging order +------------- + +There are two separate lines of development: one for Python 2 (the ``2.x`` +branches) and one for Python 3 (the ``3.x`` branches and ``default``). +You should *never* merge between the two major versions (2.x and 3.x) --- +only between minor versions (e.g. 3.x->3.y). The merge always happens from +the oldest applicable branch to the newest branch within the same major +Python version. + + +.. _branch-merge-hg-jy: + +Merging between different branches (within the same major version) +------------------------------------------------------------------ + +Assume that Python 3.5 is the current in-development version of Python and that +you have a patch that should also be applied to Python 3.4. To properly port +the patch to both versions of Python, you should first apply the patch to +Python 3.4:: + + cd 3.4 + hg import --no-commit patch.diff + # Compile; run the test suite + hg ci -m '#12345: fix some issue.' + +Then you can switch to the ``3.5`` clone, merge, run the tests and commit:: + + cd ../3.5 + hg merge 3.4 + # Fix any conflicts (e.g. ``hg revert -r default Misc/NEWS``); compile; run the test suite + hg ci -m '#12345: merge with 3.4.' + +If you are not using the share extension, you will need to use +``hg pull ../3.4`` before being able to merge. + +.. note:: + Even when porting an already committed patch, you should *still* check the + test suite runs successfully before committing the patch to another branch. + Subtle differences between two branches sometimes make a patch bogus if + ported without any modifications. + + +Porting changesets between the two major Python versions (2.x and 3.x) +---------------------------------------------------------------------- + +Assume you just committed something on ``2.7``, and want to port it to ``3.4``. +You can use ``hg graft`` as follow:: + + cd ../3.4 + hg graft 2.7 + +This will port the latest changeset committed in the 2.7 clone to the 3.4 clone. +``hg graft`` always commits automatically, except in case of conflicts, when +you have to resolve them and run ``hg graft --continue`` afterwards. +Instead of the branch name you can also specify a changeset id, and you can +also graft changesets from 3.x to 2.7. + +On older version of Mercurial where ``hg graft`` is not available, you can use:: + + cd ../3.4 + hg export 2.7 | hg import - + +The result will be the same, but in case of conflict this will create ``.rej`` +files rather than using Mercurial merge capabilities. + +A third option is to apply manually the patch on ``3.4``. This is convenient +when there are too many differences with ``2.7`` or when there is already a +specific patch for ``3.4``. + +.. warning:: + Never use ``hg merge`` to port changes between 2.x and 3.x (or vice versa). + + +Long-term development of features +--------------------------------- + +If you want to work on a feature long-term (perhaps you're implementing a +PEP), you will probably want to publish your work in a dedicated repository. +The following instructions will help you do so on `hg.python.org +`_'s infrastructure without requiring a lot of upload +bandwidth. + +Go to the main repository's Web page (http://hg.python.org/jython/); there +you find a button labelled "server-side clone", which you can click on to +display a Web form. Enter the relative path of the repository you want to +create on the server, for example ``features/mywork``; and press the button. +A new repository gets created on the server with all the changesets of the +original repository (it will seem very fast; this is normal). + +You can now do a local clone of this repository on your disk:: + + $ hg clone ssh://hg@hg.python.org/features/mywork + $ cd mywork + +.. commented out: old instructions (without server-side cloning) + + First create a public (empty) repository on hg.python.org:: + + $ hg init ssh://hg@hg.python.org/features/mywork + + And do a local clone of that repository on your disk:: + + $ hg clone ssh://hg@hg.python.org/features/mywork + $ cd mywork + + There, pull all the contents from the main repository, either from a local + clone:: + + $ hg pull ../jython + $ hg update + + or directly from the network (which is of course slower):: + + $ hg pull http://hg.python.org/jython + $ hg update + +It is recommended that you create a new `named branch`_ for your work, so as +to easily track changes. That named branch will exist in your feature +repository, but not in the main repository:: + + $ hg branch mywork + $ hg commit -m "Creating branch mywork" + $ hg push --new-branch + +You can now work on your feature, commit changes as you will, and push them +when desired:: + + $ hg push + +When you push them, they will land in the public repository at +``ssh://hg@hg.python.org/features/mywork`` (or +``http://hg.python.org/features/mywork`` for the read-only URL). Other +people can clone the public repository and work on the code too. + +When you want to synchronize with CPython's upstream changes, you can pull +from the main repository, either from its remote URL:: + + $ hg pull http://hg.python.org/jython + +or from a local clone that you may have on your disk (which is of course +faster):: + + $ hg pull ../jython + +and merge all new changes from branch ``default`` to branch ``mywork``:: + + $ hg branch + mywork + $ hg merge default + +Rather than using a clone on ``python.org`` (which isn't particularly useful +for collaboration with folks that don't already have CPython commit rights), +Bitbucket_ also maintain an `up to date clone`_ of the main ``cpython`` +repository that can be used as the basis for a new clone or patch queue. + +.. _named branch: http://mercurial.selenic.com/wiki/NamedBranches +.. _Bitbucket: http://www.bitbucket.org +.. _up to date clone: https://bitbucket.org/mirror/cpython/overview + + +Uploading a patch for review +'''''''''''''''''''''''''''' + +In this scheme, your work will probably consist of many commits (some of +them merges). If you want to upload a patch for review somewhere, you need +a single aggregate patch. This is where having a dedicated named branch +``mywork`` gets handy. + +First ensure that you have pulled *and merged* all changes from the main +repository, as explained above. Then, assuming your :ref:`currently checked +out branch ` is still ``mywork``, simply do:: + + $ hg diff -r default > mywork.patch + +This will write to ``mywork.patch`` all the changes between ``default`` and +``mywork``. diff --git a/communication.rst b/communication.rst index 5f323f9997..36328afe19 100644 --- a/communication.rst +++ b/communication.rst @@ -1,3 +1,6 @@ +.. This file is derived from a file of the same name in the CPython devguide + and will receive updates from the CPython guide by merging. + .. _communication: Following Python's Development @@ -5,11 +8,31 @@ Following Python's Development Python's development is communicated through a myriad of ways, mostly through mailing lists, but also other forms. +These are all relevant to Jython (even sometimes the content specific to C), +and Jython has some dedicated channels too. .. _mailinglists: -Mailing Lists -------------- +Jython Mailing Lists +-------------------- + +The mailing list for discussion of Jython's development is jython-dev_. +The list is open to the public and the Jython core developers subscribe, +along with others interested in Jython's development. + +Questions about using Jython should not be asked here but on jython-users_. + +Jython-checkins_ sends out an email for every commit to Jython's various +repositories from http://hg.python.org. +All core developers subscribe to this list. +There is a mailing list related to issues on the `Jython issue tracker`_. +Subscribe to jython-bugs_ if you want an email for all changes made to any +issue. + +Python and CPython Mailing Lists +-------------------------------- +Developers working on Jython will be interested in the development of Python +(the language) and its reference implementation. python-dev_ is the primary mailing list for discussions about Python's development. The list is open to the public and is subscribed to by all core @@ -43,11 +66,6 @@ subscribe to this list and are known to reply to these emails to make comments about various issues they catch in the commit. Replies get redirected to python-dev. -There are two mailing lists related to issues on the `issue tracker`_. If you -only want an email for when a new issue is open, subscribe to -new-bugs-announce_. If you would rather receive an email for all changes made -to any issue, subscribe to python-bugs-list_. - General Python questions should go to `python-list`_ or `tutor`_ or similar resources, such as StackOverflow_ or the ``#python`` IRC channel on Freenode_. @@ -75,6 +93,13 @@ RSS feed readers. .. _Freenode: http://freenode.net/ +.. _Jython issue tracker: http://bugs.jython.org +.. _jython-bugs: https://lists.sourceforge.net/lists/listinfo/jython-bugs +.. _jython-checkins: https://lists.sourceforge.net/lists/listinfo/jython-checkins +.. _jython-dev: https://lists.sourceforge.net/lists/listinfo/jython-dev +.. _jython-users: https://lists.sourceforge.net/lists/listinfo/jython-users + + IRC --- diff --git a/compiler_jy.rst b/compiler_jy.rst new file mode 100644 index 0000000000..18768f2e5d --- /dev/null +++ b/compiler_jy.rst @@ -0,0 +1,325 @@ +.. Jython companion to compiler.rst + +.. _compiler-jy: + +Design of Jython's Compiler +=========================== + +.. warning:: At present, this is not much more than a copy of the CPython original + with the obviously inapplicable crudely hacked out. + +Abstract +-------- + +In CPython, the compilation from source code to bytecode involves several steps: + +1. Parse source code into a parse tree (:file:`Parser/pgen.c`) +2. Transform parse tree into an Abstract Syntax Tree (:file:`Python/ast.c`) +3. Transform AST into a Control Flow Graph (:file:`Python/compile.c`) +4. Emit bytecode based on the Control Flow Graph (:file:`Python/compile.c`) + +The purpose of this document is to outline how these steps of the process work. + +This document does not touch on how parsing works beyond what is needed +to explain what is needed for compilation. It is also not exhaustive +in terms of the how the entire system works. You will most likely need +to read some source to have an exact understanding of all details. + + +Parse Trees +----------- + +Python's grammar parser is designed for an LL(1) parser mostly based off of the +implementation laid out in the Dragon Book [Aho86]_. + +Jython uses the ANTLR parser-generator, +which is capable of much more than LL(1), +but we like to keep the same source grammar. + +At the time of writing, we are still using an obsolete version of ANTLR. +ANTLR 4 changed substantially, and significant work is needed to follow suit. + +The grammar file for Jython can be found in :file:`grammar/Python.g`. + + + +Abstract Syntax Trees (AST) +--------------------------- + + +.. sidebar:: Green Tree Snakes + + See also `Green Tree Snakes - the missing Python AST docs + `_ by Thomas Kluyver. + +The abstract syntax tree (AST) is a high-level representation of the +program structure without the necessity of containing the source code; +it can be thought of as an abstract representation of the source code. The +specification of the AST nodes is specified using the Zephyr Abstract +Syntax Definition Language (ASDL) [Wang97]_. + +The definition of the AST nodes for Python is found in the file +:file:`ast/Python.asdl`. + +Each AST node (representing statements, expressions, and several +specialized types, like list comprehensions and exception handlers) is +defined by the ASDL. Most definitions in the AST correspond to a +particular source construct, such as an 'if' statement or an attribute +lookup. The definition is independent of its realization in any +particular programming language. + +The following fragment of the Python ASDL construct demonstrates the +approach and syntax:: + + module Python + { + stmt = FunctionDef(identifier name, arguments args, stmt* body, + expr* decorators) + | Return(expr? value) | Yield(expr value) + attributes (int lineno) + } + +The preceding example describes three different kinds of statements; +function definitions, return statements, and yield statements. All +three kinds are considered of type ``stmt`` as shown by ``|`` separating the +various kinds. They all take arguments of various kinds and amounts. + +Modifiers on the argument type specify the number of values needed; ``?`` +means it is optional, ``*`` means 0 or more, while no modifier means only one +value for the argument and it is required. ``FunctionDef``, for instance, +takes an ``identifier`` for the *name*, ``arguments`` for *args*, zero or more +``stmt`` arguments for *body*, and zero or more ``expr`` arguments for +*decorators*. + +Do notice that something like 'arguments', which is a node type, is +represented as a single AST node and not as a sequence of nodes as with +stmt as one might expect. + +All three kinds also have an 'attributes' argument; this is shown by the +fact that 'attributes' lacks a '|' before it. + + +.. note:: something needed here about AST classes supporting the parser. + + +Memory Management +----------------- + + + + +Parse Tree to AST +----------------- + +.. note:: not sure how much of this is true for Jython + +The AST is generated from the parse tree ... + +The function begins a tree walk of the parse tree, creating various AST +nodes as it goes along. It does this by allocating all new nodes it +needs, calling the proper AST node creation functions for any required +supporting functions, and connecting them as needed. + +Do realize that there is no automated nor symbolic connection between +the grammar specification and the nodes in the parse tree. No help is +directly provided by the parse tree as in yacc. + +For instance, one must keep track of which node in the parse tree +one is working with (e.g., if you are working with an 'if' statement +you need to watch out for the ':' token to find the end of the conditional). + +The functions called to generate AST nodes from the parse tree all have +the name ``ast_for_xx`` where *xx* is the grammar rule that the function +handles (``alias_for_import_name`` is the exception to this). These in turn +call the constructor functions as defined by the ASDL grammar and +contained in :file:`Python/Python-ast.c` (which was generated by +:file:`Parser/asdl_c.py`) to create the nodes of the AST. This all leads to a +sequence of AST nodes stored in ``asdl_seq`` structs. + + +Function and macros for creating and using ``asdl_seq *`` types as found +in :file:`Python/asdl.c` and :file:`Include/asdl.h` are as follows: + +``_Py_asdl_seq_new(Py_ssize_t, PyArena *)`` + Allocate memory for an ``asdl_seq`` for the specified length +``asdl_seq_GET(asdl_seq *, int)`` + Get item held at a specific position in an ``asdl_seq`` +``asdl_seq_SET(asdl_seq *, int, stmt_ty)`` + Set a specific index in an ``asdl_seq`` to the specified value +``asdl_seq_LEN(asdl_seq *)`` + Return the length of an ``asdl_seq`` + +If you are working with statements, you must also worry about keeping +track of what line number generated the statement. Currently the line +number is passed as the last parameter to each ``stmt_ty`` function. + + +Control Flow Graphs +------------------- + +A control flow graph (often referenced by its acronym, CFG) is a +directed graph that models the flow of a program using basic blocks that +contain the intermediate representation (abbreviated "IR", and in this +case is Python bytecode) within the blocks. Basic blocks themselves are +a block of IR that has a single entry point but possibly multiple exit +points. The single entry point is the key to basic blocks; it all has +to do with jumps. An entry point is the target of something that +changes control flow (such as a function call or a jump) while exit +points are instructions that would change the flow of the program (such +as jumps and 'return' statements). What this means is that a basic +block is a chunk of code that starts at the entry point and runs to an +exit point or the end of the block. + +As an example, consider an 'if' statement with an 'else' block. The +guard on the 'if' is a basic block which is pointed to by the basic +block containing the code leading to the 'if' statement. The 'if' +statement block contains jumps (which are exit points) to the true body +of the 'if' and the 'else' body (which may be ``NULL``), each of which are +their own basic blocks. Both of those blocks in turn point to the +basic block representing the code following the entire 'if' statement. + +CFGs are usually one step away from final code output. Code is directly +generated from the basic blocks (with jump targets adjusted based on the +output order) by doing a post-order depth-first search on the CFG +following the edges. + + +AST to CFG to Bytecode +---------------------- + +.. note:: names wrong here but are we close in organisation? + +With the AST created, the next step is to create the CFG. The first step +is to convert the AST to Python bytecode without having jump targets +resolved to specific offsets (this is calculated when the CFG goes to +final bytecode). Essentially, this transforms the AST into Python +bytecode with control flow represented by the edges of the CFG. + +Conversion is done in two passes. The first creates the namespace +(variables can be classified as local, free/cell for closures, or +global). With that done, the second pass essentially flattens the CFG +into a list and calculates jump offsets for final output of bytecode. + +The conversion process is initiated by a call to the function +``PyAST_Compile()`` in :file:`Python/compile.c`. This function does both the +conversion of the AST to a CFG and outputting final bytecode from the CFG. +The AST to CFG step is handled mostly by two functions called by +``PyAST_Compile()``; ``PySymtable_Build()`` and ``compiler_mod()``. The former +is in :file:`Python/symtable.c` while the latter is in :file:`Python/compile.c`. + +``PySymtable_Build()`` begins by entering the starting code block for the +AST (passed-in) and then calling the proper ``symtable_visit_xx`` function +(with *xx* being the AST node type). Next, the AST tree is walked with +the various code blocks that delineate the reach of a local variable +as blocks are entered and exited using ``symtable_enter_block()`` and +``symtable_exit_block()``, respectively. + +Once the symbol table is created, it is time for CFG creation, whose +code is in :file:`Python/compile.c`. This is handled by several functions +that break the task down by various AST node types. The functions are +all named ``compiler_visit_xx`` where *xx* is the name of the node type (such +as stmt, expr, etc.). Each function receives a ``struct compiler *`` +and ``xx_ty`` where *xx* is the AST node type. Typically these functions +consist of a large 'switch' statement, branching based on the kind of +node type passed to it. Simple things are handled inline in the +'switch' statement with more complex transformations farmed out to other +functions named ``compiler_xx`` with *xx* being a descriptive name of what is +being handled. + +When transforming an arbitrary AST node, use the ``VISIT()`` macro. +The appropriate ``compiler_visit_xx`` function is called, based on the value +passed in for (so ``VISIT(c, expr, node)`` calls +``compiler_visit_expr(c, node)``). The ``VISIT_SEQ`` macro is very similar, +but is called on AST node sequences (those values that were created as +arguments to a node that used the '*' modifier). There is also +``VISIT_SLICE()`` just for handling slices. + +Emission of bytecode is handled by the asm library. + +.. note:: names again? + +Several helper functions that will emit bytecode and are named +``compiler_xx()`` where *xx* is what the function helps with (list, boolop, +etc.). A rather useful one is ``compiler_nameop()``. +This function looks up the scope of a variable and, based on the +expression context, emits the proper opcode to load, store, or delete +the variable. + +As for handling the line number on which a statement is defined, this is +handled by ``compiler_visit_stmt()`` and thus is not a worry. + +In addition to emitting bytecode based on the AST node, handling the +creation of basic blocks must be done. Below are the macros and +functions used for managing basic blocks: + +``NEXT_BLOCK(struct compiler *)`` + create an an implicit jump from the current block + to the new block +``compiler_new_block(struct compiler *)`` + create a block but don't use it (used for generating jumps) + +Once the CFG is created, it must be flattened and then final emission of +bytecode occurs. Flattening is handled using a post-order depth-first +search. Once flattened, jump offsets are backpatched based on the +flattening and then a ``PyCodeObject`` is created. All of this is +handled by calling ``assemble()``. + + +Introducing JVM Bytecode +------------------------ + + + +Code Objects +------------ + +The result of ``PyAST_Compile()`` is a ``PyCodeObject`` which is defined in +:file:`Include/code.h`. And with that you now have executable Python bytecode! + +The code objects (byte code) are executed in :file:`Python/ceval.c`. This file +will also need a new case statement for the new opcode in the big switch +statement in ``PyEval_EvalFrameDefault()``. + + +Important Files +--------------- + ++ ast/ + + Python.asdl + ASDL syntax file + + asdl.py + Parser for ASDL definition files. Reads in an ASDL description + and parses it into an AST that describes it. + + asdl_java.py + "Generate Java code from an ASDL description." Generates + :file:`...` . + + + +Known Compiler-related Experiments +---------------------------------- + +This section lists known experiments involving the compiler (including +bytecode). + + + + +References +---------- + +.. .. [Aho86] Alfred V. Aho, Ravi Sethi, Jeffrey D. Ullman. + `Compilers: Principles, Techniques, and Tools`, + https://www.amazon.com/exec/obidos/tg/detail/-/0201100886/104-0162389-6419108 + +.. .. [Wang97] Daniel C. Wang, Andrew W. Appel, Jeff L. Korn, and Chris + S. Serra. `The Zephyr Abstract Syntax Description Language.`_ + In Proceedings of the Conference on Domain-Specific Languages, pp. + 213--227, 1997. + +.. _The Zephyr Abstract Syntax Description Language.: + http://www.cs.princeton.edu/research/techreps/TR-554-97 + diff --git a/conf.py b/conf.py index 29658e94ef..725b2c98c3 100644 --- a/conf.py +++ b/conf.py @@ -46,7 +46,7 @@ master_doc = 'index' # General information about the project. -project = u'Python Developer\'s Guide' +project = u'Jython Developer\'s Guide' copyright = u'2011-%s, Python Software Foundation' % time.strftime('%Y') # The version info for the project you're documenting, acts as replacement for @@ -179,7 +179,7 @@ #html_file_suffix = None # Output file base name for HTML help builder. -htmlhelp_basename = 'PythonDevelopersGuidedoc' +htmlhelp_basename = 'JythonDevelopersGuidedoc' # -- Options for LaTeX output -------------------------------------------------- @@ -193,8 +193,8 @@ # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, author, documentclass [howto/manual]). latex_documents = [ - ('index', 'PythonDevelopersGuide.tex', u'Python Developer\'s Guide Documentation', - u'Brett Cannon', 'manual'), + ('index', 'JythonDevelopersGuide.tex', u'Jython Developer\'s Guide Documentation', + u'The Jython & Python Developers', 'manual'), ] # The name of an image file (relative to this directory) to place at the top of @@ -226,6 +226,6 @@ # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ - ('index', 'pythondevelopersguide', u"Python Developer's Guide Documentation", - [u'Brett Cannon'], 1) + ('index', 'jythondevelopersguide', u"Jython Developer's Guide Documentation", + [u'The Jython & Python Developers'], 1) ] diff --git a/coredev.rst b/coredev.rst index 3b50ca9e80..d77651c7ab 100644 --- a/coredev.rst +++ b/coredev.rst @@ -1,16 +1,23 @@ +.. This file is derived from a file of the same name in the CPython devguide + and will receive updates from the CPython guide by merging. + .. _coredev: How to Become a Core Developer ============================== +.. warning:: This is the merge of the hg-era Jython devguide with the current + CPython devguide. + What it Takes ------------- When you have consistently contributed patches which meet quality standards without requiring extensive rewrites prior to being committed, -you may qualify for commit privileges and become a core developer of Python. -You must also work well with other core developers (and people in general) -as you become an ambassador for the Python project. +you may qualify for +commit privileges and become a core developer of Jython. You must also work +well with other core developers (and people in general) as you become an +ambassador for the Jython project. Typically a core developer will offer you the chance to gain commit privilege. The person making the offer will become your mentor and watch your commits for @@ -60,21 +67,20 @@ in the :ref:`experts` and :ref:`developers`). Gaining Commit Privileges ------------------------- -When you have been extended an official offer to become a Python core +When you have been extended an official offer to become a Jython core developer, there are several things you must do. Mailing Lists ''''''''''''' -You are expected to subscribe to python-committers, python-dev, -python-checkins, and one of new-bugs-announce or python-bugs-list. See -:ref:`communication` for links to these mailing lists. - +You are expected to subscribe to jython-dev, jython-checkins, and +jython-bugs. See :ref:`communication` for links to these mailing lists. Issue Tracker ''''''''''''' -If you did not gain the Developer role in the `issue tracker`_ before gaining +If you did not gain the Developer role in +the `Jython issue tracker`_ before gaining commit privileges, please say so. This will allow issues to be assigned to you. A tracker admin should also flip your "is committer" bit in the tracker's account screen. @@ -84,15 +90,36 @@ It is expected that on the issue tracker you have a username in the form of form, please change it. This is so that it is easier to assign issues to the right person. +.. _Jython issue tracker: https://bugs.jython.org/ + +SSH (Mercurial) +''''''''''''''' + +You need to generate an SSH 2 RSA key to be able to commit code. You may have +multiple keys if you wish (e.g., for work and home). Send your key as an +attachment in an email to hgaccounts@python.org. Help in generating an SSH key +can be found in the :ref:`faq`. + +Your SSH key will be set to a username in the form of "first_name.last_name". +This should match your username on the issue tracker. + +You can verify your commit access by looking at +http://hg.python.org/committers.txt which lists all core developers by +username. If you want to practice, there is a `test repository +`_ where you can freely commit and push any +changes you like:: + + hg clone ssh://hg@hg.python.org/test/ hgtest + GitHub '''''' -You will be added to the ``Python core team`` on GitHub. This will give you -rights to commit to various repositories under the `Python organization`_ +You will be added to the ``Jython core team`` on GitHub. This will give you +rights to commit to various repositories under the `Jython organization`_ on GitHub. When you are initially added you will be emailed by GitHub with an invitation to join the team. Please accept the invite in the email or -go to https://github.com/python and accept the invite there. +go to https://github.com/jython and accept the invite there. An entry in the :ref:`developers` should also be entered for you. Typically the person who sponsored your application to become a core developer @@ -100,6 +127,7 @@ makes sure an entry is created for you. .. _Python organization: https://github.com/python +.. _Jython organization: https://github.com/jython .. _contributor_agreement: @@ -117,13 +145,29 @@ done this yet, it is best to do this ASAP, probably before you even do your first commit so as to not forget. Also do not forget to enter your GitHub username into your details on the issue tracker. +.. _contributor form for Python: http://www.python.org/psf/contrib/ + +Read/Write Checkout (Mercurial) +''''''''''''''''''''''''''''''' + +With your commit privileges working and your contributor form submitted, you +can now get a read/write checkout of the code. URLs for read/write checkouts +are different than those for read-only checkouts as SSH is used instead of +HTTP. + +You can clone the repository (which contains all active branches) with:: + + hg clone ssh://hg@hg.python.org/jython -.. _contributor form for Python: https://www.python.org/psf/contrib/ +The default branch in that repository is the current development branch. +You can of course switch your working copy to one of the maintenance branches, +for example:: + hg update 2.7 -Pull Request merging -'''''''''''''''''''' +Pull Request merging (GitHub) +''''''''''''''''''''''''''''' Once you have your commit privileges on GitHub you will be able to accept pull requests on GitHub. You should plan to continue to submit your own @@ -138,8 +182,8 @@ Responsibilities As a core developer, there are certain things that are expected of you. First and foremost, be a good person. This might sound melodramatic, but you -are now a member of the Python project and thus represent the project and your -fellow core developers whenever you discuss Python with anyone. We have a +are now a member of the Jython project and thus represent the project and your +fellow core developers whenever you discuss Jython with anyone. We have a reputation for being a very nice group of people and we would like to keep it that way. Core developers responsibilities include following the `PSF Code of Conduct`_. diff --git a/coverage_jy.rst b/coverage_jy.rst new file mode 100644 index 0000000000..5b5c81ed45 --- /dev/null +++ b/coverage_jy.rst @@ -0,0 +1,129 @@ +.. Jython companion to coverage.rst + +.. _coverage-jy: + +Test Coverage in Jython +======================= + +In broad groups, we have three things to test: + +#. The Python Standard Library modules implemented in Python, +#. The Java implementation of modules in the Standard Library, and +#. The core of Jython implemented in Java. + +In each case, Jython relies heavily on the tests provided by the CPython +reference implementation. + +Jython adopts the Python Standard Library wholesale, and +its quality is assured by the CPython project. +Jython makes some small changes to its copy +to replace or add to tests and library modules. +We aim to minimise the Jython-specific code in both the +standard library and our tests. + +Python development follows the practice that all semantic changes and additions +to the language and :abbr:`stdlib (standard library)` are accompanied by +appropriate unit tests. +This is the aim of ensuring sufficient :doc:`test coverage `. +To the extent that CPython achieves this, a large part of Jython developement +may be driven by these freely provided tests. + +Improvements to the library and its test coverage may emerge from Jython, +but the strategy depends on contributing them +:doc:`upstream to CPython `. + +.. _coverage-by-regrtest-jy: + + +Test Coverage Challenges +------------------------ + +There are places where Jython cannot depend on tests provided by CPython: + +#. Java integration (calling and being called by Java). +#. Concurrency (where CPython's :abbr:`GIL (Global Interpreter Lock)` saves it + but not Jython). +#. Tests dependent on garbage collection (sometimes inadvertently). +#. Tests that expose implementation details that differ. + +For these cases we create tests in Python (to supersede or supplement the +ones in the CPython :mod:`test` module) and tests in Java. + + +Java Integration +'''''''''''''''' + +Java integration is unique to Jython, so we must have our own tests. +Where possible, tests should be in Python, using :mod:`unittest`. +Where tests have to be in Java, we recommend the JUnit_ framework. + + +Concurrency +''''''''''' + +The JVM has strong support for concurrency. +There are many constructs in the Java library to support concurrency, +which may be used directly from Python. + +Python also has support for concurrency, and a rich library, +but in the CPython interpreter, +only one thread may be executing CPython bytecode at a time. +Concurrency is limited to releasing the global lock during blocking operations +such as i/o. +If a task is CPU-bound, +one must use multiple CPython interpreters in order to scale it. + +By contrast, the JVM may switch threads at any time, +part-way through what in CPython would be an atomic operation. +(JVM instructions may be atomic, but this barely helps us at all.) +Re-use or translation of implementations used in CPython needs to bear this +difference in mind, and additional tests are likely to be necessary to +assure us of correctness. + +Some Jython applications will run in an environments with high concurrency, +and Java thread pools (such as a web-service), +where faulty concurrency is more likely to be exposed than in typical uses of +CPython. + + +Garbage Collection +'''''''''''''''''' + +Many older tests create objects they assume are garbage-collected immediately +they go out of scope. +Examples: + +* Files assumed closed as soon as dereferenced. +* The timing of weak reference invalidation. + +These tests may need adaptation to Java. +There is no way to force garbage collection in Java, only to "suggest" it. +Tests that open files are a particular problem on Windows, +where files that remain open when they pass out of scope, +cannot be deleted. + + +Jython Internals +'''''''''''''''' + +Tests of internal methods not accessible to Python are necessarily in Java, +usually as JUnit_ tests. +Examples: + +* The buffer interface. +* Type exposure. + + +Filing the Issue +---------------- + +Once you have increased coverage, you need to create an issue on the +`Jython issue tracker`_ and a patch there. +(In the future process, you will probably submit a +:doc:`pull request `.) +On the issue set the "Components" to "Test" and "Versions" to the version of +Jython you worked on (i.e., the in-development version). + +.. _Jython issue tracker: http://bugs.jython.org +.. _Junit: http://junit.org + diff --git a/devcycle.rst b/devcycle.rst index d7b6eba0c4..d65f3bbdeb 100644 --- a/devcycle.rst +++ b/devcycle.rst @@ -1,13 +1,22 @@ +.. This file is derived from a file of the same name in the CPython devguide + and will receive updates from the CPython guide by merging. + .. _devcycle: Development Cycle ================= +.. warning:: At present, this includes all the material from the Jython guide, + but predominantly describes the CPython process with CPython examples. + The process is the one Jython will move to: some applies already and the + rest is important for us to get clear. + It's just that the examples are a bit incongruous for Jython. + The responsibilities of a core developer shift based on what kind of branch of -Python a developer is working on and what stage the branch is in. +Jython a developer is working on and what stage the branch is in. -To clarify terminology, Python uses a ``major.minor.micro`` nomenclature -for production-ready releases. So for Python 3.1.2 final, that is a *major +To clarify terminology, Jython uses a ``major.minor.micro`` nomenclature +for production-ready releases. So for Jython 3.1.2 final, that is a *major version* of 3, a *minor version* of 1, and a *micro version* of 2. * new *major versions* are exceptional; they only come when strongly @@ -31,7 +40,7 @@ Branches '''''''' There is a branch for each *feature version*, whether released or not (e.g. -2.7, 3.6). Development is handled separately for Python 2 and Python 3: +2.7, 3.5). Development is handled separately for Python 2 and Python 3: no merging happens between 2.x and 3.x branches. @@ -44,6 +53,9 @@ The ``master`` branch is the branch for the next feature release; it is under active development for all kinds of changes: new features, semantic changes, performance improvements, bug fixes. +In a process based on Mercurial, the branch is identifies as ``default``. +It is the branch :ref:`checked out ` by default by Mercurial. + At some point during the life-cycle of a release, a new :ref:`maintenance branch ` is created to host all bug fixing activity for further micro versions in a feature version (3.6.1, 3.6.2, etc.). @@ -112,6 +124,8 @@ Summary There are 5 open branches right now in the Git repository: +.. warning:: CPython-specific + - the ``master`` branch accepts features and bugs fixes for the future 3.7.0 feature release (RM: Ned Deily) - the ``3.6`` branch accepts bug fixes for future 3.6.x maintenance releases @@ -123,8 +137,14 @@ There are 5 open branches right now in the Git repository: - the ``2.7`` branch accepts bug fixes for future 2.7.x maintenance releases (RM: Benjamin Peterson) +.. warning:: Jython information out of date. + See also the :ref:`Status of Python branches `. +- the ``default`` branch holds the future 2.7 version and descends from ``2.5`` +- the ``2.5`` branch holds bug fixes for future 2.5.x maintenance releases and + descends from ``2.2`` +- the ``2.2`` branch holds security fixes for future 2.2.x security releases .. _stages: diff --git a/docquality.rst b/docquality.rst index a3f1b86f81..d6c4f1ee90 100644 --- a/docquality.rst +++ b/docquality.rst @@ -1,8 +1,18 @@ +.. This file is derived from a file of the same name in the CPython devguide + and will receive updates from the CPython guide by merging. + .. _docquality: Helping with Documentation ========================== +.. warning:: At present, this is not much modified from the CPython base. + +.. I've not pulled in the old devguide Python to Jython changes as the section + is about Python, not CPython. The original (as improved upstream) is valid. + And we don't really produce much documentation. (Javadoc is a different + kettle of fish.) + Python is known for having good documentation. But maintaining all of it and keeping a high level of quality takes a lot of effort. Help is always appreciated with the documentation, and it requires little programming @@ -107,6 +117,6 @@ Changes to the devguide are normally published within a day, on a schedule that may be different from the main documentation. .. _separate repository: -.. _devguide repo: https://github.com/python/devguide -.. _the GitHub tracker: https://github.com/python/devguide/issues +.. _devguide repo: https://github.com/jython/devguide +.. _the GitHub tracker: https://github.com/jython/devguide/issues .. _Sphinx: http://www.sphinx-doc.org/ diff --git a/documenting.rst b/documenting.rst index 2be755af5d..ac21c5e3ef 100644 --- a/documenting.rst +++ b/documenting.rst @@ -1,9 +1,16 @@ +.. This file is derived from a file of the same name in the CPython devguide + and will receive updates from the CPython guide by merging. + .. _documenting: ================== Documenting Python ================== +.. warning:: At present, this is not much modified from the CPython base. + But that's probably ok, as Jython documentation is largely from CPython, + and the section is predominantly a guide to ReStructuredText. + The Python language has a substantial body of documentation, much of it contributed by various authors. The markup used for the Python documentation is `reStructuredText`_, developed by the `docutils`_ project, amended by custom diff --git a/experts.rst b/experts.rst index 93af0cafb0..1210d129c0 100644 --- a/experts.rst +++ b/experts.rst @@ -1,8 +1,13 @@ +.. This file is derived from a file of the same name in the CPython devguide + and will receive updates from the CPython guide by merging. + .. _experts: Experts Index ============= +.. warning:: At present, this is not much modified from the CPython base. + This document has tables that list Python Modules, Tools, Platforms and Interest Areas and names for each item that indicate a maintainer or an expert in the field. This list is intended to be used by issue submitters, diff --git a/exploring_jy.rst b/exploring_jy.rst new file mode 100644 index 0000000000..052a21cc69 --- /dev/null +++ b/exploring_jy.rst @@ -0,0 +1,25 @@ +.. Jython companion to exploring.rst + +.. _exploring-jy: + +Exploring Jython's Internals +============================ + +.. warning:: At present, this is not much more than a copy of the CPython original + with the obviously inapplicable crudely hacked out. + +This is a quick guide for people who are interested in learning more about +Jython's internals. +It provides a summary of the source code structure +and contains references to resources providing a more in-depth view. +Much may also be learned by :doc:`exploring CPython's internals `. + + +Jython Source Code Layout +------------------------- + + +Additional References +--------------------- + +... may be found in the corresponding part of the CPython devguide. diff --git a/faq_hg_jy.rst b/faq_hg_jy.rst new file mode 100644 index 0000000000..0e24306565 --- /dev/null +++ b/faq_hg_jy.rst @@ -0,0 +1,946 @@ +.. Jython and Mercurial-specific file included to satisfy some references, + derived from the former CPython devguide file raq.rst + +:tocdepth: 2 + +.. _faq: + +Jython Developer FAQ +~~~~~~~~~~~~~~~~~~~~ + +.. note:: + + The "frequently asked questions" in the CPython devguide were redistributed + to the sections of the guide that deal with the subject of the question, + under the a PR to `delete the FAQ`_. + In the process we lose the Jython and Mercurial answers, + so we're keeping it in the Jython version for the time being. + +.. _delete the FAQ: https://github.com/python/devguide/issues/27 + +.. contents:: + :local: + +Communications +============== + + +Where should I ask general Jython questions? +-------------------------------------------- + +General Jython questions should still go to `jython-users`_ or `tutor`_ +or similar resources, such as StackOverflow_ or the ``#jython`` IRC channel +on Freenode_. + +.. _jython-users: https://lists.sourceforge.net/lists/listinfo/jython-users +.. _tutor: http://mail.python.org/mailman/listinfo/tutor +.. _StackOverflow: http://stackoverflow.com/ +.. _Freenode: http://freenode.net/ + + +.. index:: + single: PEP process; in FAQ + +.. _suggesting-changes-faq: + +Where should I suggest new features and language changes? +--------------------------------------------------------- + +The `python-ideas`_ mailing list is specifically intended for discussion of +new features and language changes. Please don't be disappointed if your +idea isn't met with universal approval: as the long list of Rejected and +Withdrawn PEPs in the `PEP Index`_ attests, and as befits a reasonably mature +programming language, getting significant changes into Python isn't a simple +task. + +If the idea is reasonable, someone will suggest posting it as a feature +request on the `issue tracker`_, or, for larger changes, writing it up as +a `draft PEP`_. + +Sometimes core developers will differ in opinion, or merely be collectively +unconvinced. When there isn't an obvious victor then the +`Status Quo Wins a Stalemate`_ as outlined in the linked post. + +For some examples on language changes that were accepted please read +`Justifying Python Language Changes`_. + +See also the :ref:`langchanges` section of this guide. + +.. _python-ideas: http://mail.python.org/mailman/listinfo/python-ideas +.. _issue tracker: http://bugs.python.org +.. _PEP Index: http://www.python.org/dev/peps/ +.. _draft PEP: http://www.python.org/dev/peps/pep-0001/ +.. _Status Quo Wins a Stalemate: http://www.curiousefficiency.org/posts/2011/02/status-quo-wins-stalemate.html +.. _Justifying Python Language Changes: http://www.curiousefficiency.org/posts/2011/02/justifying-python-language-changes.html + +Where should I ask general questions about contributing to Jython? +------------------------------------------------------------------ + +Jython development is generally discussed on the jython-dev_ list. +There is also the `Python Mentors`_ program which is specifically about +encouraging developers and others that would like to contribute to Python and +Jython development in general, rather than necessarily being focused on one +particular issue. Some core developers are also available on the ``#jython`` +IRC channel on Freenode_. + +.. _jython-dev: https://lists.sourceforge.net/lists/listinfo/jython-dev +.. _Python Mentors: http://pythonmentors.com + + +Where should I report specific problems? +---------------------------------------- + +Specific problems should be posted to the `Jython issue tracker`_. + +.. _Jython issue tracker: http://bugs.jython.org + +What if I'm not sure it is a bug? +--------------------------------- + +The general Python help locations listed above are the best place to start +with that kind of question. If they agree it looks like a bug, then the +next step is to either post it to the `Jython issue tracker`_ or else to ask further +on the core development mailing list, `jython-dev`_. + +.. _jython-dev: https://lists.sourceforge.net/lists/listinfo/jython-dev + + +What if I disagree with an issue resolution on the tracker? +----------------------------------------------------------- + +First, take some time to consider any comments made in association with the +resolution of the tracker issue. On reflection, they may seem more reasonable +than they first appeared. + +If you still feel the resolution is incorrect, then raise the question on +`jython-dev`_. + +.. _jython-dev: https://lists.sourceforge.net/lists/listinfo/jython-dev + +How do I tell who is and isn't a core developer? +------------------------------------------------ + +As Jython core developers are on the same list as CPython core developers, you +can check their name against the `full list of developers`_ with commit +rights to the main source control repository. + +On the `Jython issue tracker`_, most core developers will have the Python logo +appear next to their name. (This is not yet true on the Jython tracker, we're +working on it). + +.. _full list of developers: http://hg.python.org/committers.txt + + +What standards of behaviour are expected in these communication channels? +------------------------------------------------------------------------- + +We try to foster environments of mutual respect, tolerance and encouragement, +as described in the PSF's `Diversity Statement`_. Abiding by the guidelines +in this document and asking questions or posting suggestions in the +appropriate channels are an excellent way to get started on the mutual respect +part, greatly increasing the chances of receiving tolerance and encouragement +in return. + +.. _Diversity Statement: http://www.python.org/psf/diversity/ + + +Version Control +=============== + +For everyone +------------ + +The following FAQs are intended for both core developers and contributors. + +Where can I learn about the version control system used, Mercurial (hg)? +'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' + +Mercurial_'s (also known as ``hg``) official web site is at +http://mercurial.selenic.com/. A book on Mercurial published by +`O'Reilly Media`_, `Mercurial: The Definitive Guide`_, is available +for free online. Another resource is `Hg Init: a Mercurial tutorial`_ +by Joel Spolsky. + +With Mercurial installed, you can run the help tool that comes with +Mercurial to get help:: + + hg help + +The `man page`_ for ``hg`` provides a quick refresher on the details of +various commands, but doesn't provide any guidance on overall +workflow. + +.. _Mercurial: http://mercurial.selenic.com/ +.. _O'Reilly Media: http://www.oreilly.com/ +.. _Mercurial\: The Definitive Guide: http://hgbook.red-bean.com/ +.. _man page: http://www.selenic.com/mercurial/hg.1.html +.. _Hg Init\: a Mercurial tutorial: http://hginit.com/ + + +I already know how to use Git, can I use that instead? +'''''''''''''''''''''''''''''''''''''''''''''''''''''' + +.. warning:: Specific to CPython + +While the main workflow for core developers requires Mercurial, if +you just want to generate patches with ``git diff`` and post them to the +`issue tracker`_, Petri Lehtinen maintains a `git mirror`_ of the main +`CPython repository`_. To create a local clone based on this mirror rather +than the main repository:: + + git clone git://github.com/akheron/cpython + +The mirror's master branch tracks the main repository's default branch, +while the maintenance branch names (``2.7``, ``3.3``, etc) are mapped +directly. + +.. _git mirror: http://github.com/akheron/cpython +.. _CPython repository: http://hg.python.org/cpython + +Please only use this approach if you're already an experienced Git user and +don't require assistance with the specifics of version control commands. All +other parts of this developer's guide assume the use of Mercurial for local +version control. + + +What do I need to use Mercurial? +'''''''''''''''''''''''''''''''' + +UNIX +^^^^ + +First, you need to `download Mercurial`_. Most UNIX-based operating systems +have binary packages available. Most package management systems also +have native Mercurial packages available. + +If you have push rights, you need OpenSSH_. This is needed to verify +your identity when performing commits. As with Mercurial, binary packages +are typically available either online or through the platform's package +management system. + +Mercurial does not use its own compression via SSH +because it is better to enable compression at the SSH level. Enabling +SSH compression can make cloning a remote repository much faster. +You can configure it in your ``~/.ssh/config`` file; for example:: + + Host hg.python.org + Compression yes + +.. _download Mercurial: http://mercurial.selenic.com/downloads +.. _OpenSSH: http://www.openssh.org/ + + +Windows +^^^^^^^ + +The recommended option on Windows is to `download TortoiseHg`_ which +integrates with Windows Explorer and also bundles the command line client +(meaning you can type ``hg`` in a DOS box). Note that most +entries in this FAQ only cover the command line client in detail - refer +to the TortoiseHg documentation for assistance with its graphical interface. + +If you have push rights, you need to configure Mercurial to work with +your SSH keys. For that, open your Mercurial configuration file +(you can do so by opening the TortoiseHg Global Settings dialog and then +clicking *"Edit File"*). If there is no ``[ui]`` section, create it by +typing just that on a line by itself. Then add the following line:: + + ssh = TortoisePlink.exe -ssh -2 -C -i C:\path\to\yourkey.ppk + +where ``C:\path\to\yourkey.ppk`` should be replaced with the actual path +to your SSH private key. + +.. note:: + If your private key is in OpenSSH format, you must first convert it to + PuTTY format by loading it into `PuTTYgen`_. + +.. _download TortoiseHg: http://tortoisehg.bitbucket.org/download/index.html + + +What's a working copy? What's a repository? +''''''''''''''''''''''''''''''''''''''''''' + +Mercurial is a "distributed" version control system. This means that each +participant, even casual contributors, download a complete copy (called a +*clone*, since it is obtained by calling ``hg clone``) of the central +repository which can be treated as a stand-alone repository for all purposes. +That copy is called in the FAQ the *local repository*, to differentiate +with any *remote repository* you might also interact with. + +But you don't modify files directly in the local repository; Mercurial doesn't +allow for it. You modify files in what's called the *working copy* associated +with your local repository: you also run compilations and tests there. +Once you are satisfied with your changes, you can :ref:`commit them `; +committing records the changes as a new *revision* in the *local repository*. + +Changes in your *local repository* don't get automatically shared with the +rest of the world. Mercurial ensures that you have to do so explicitly +(this allows you to experiment quite freely with multiple branches of +development, all on your private computer). The main commands for doing +so are ``hg pull`` and ``hg push``. + + +Which branches are in my local repository? +'''''''''''''''''''''''''''''''''''''''''' + +Typing ``hg branches`` displays the open branches in your local repository:: + + $ hg branches + default 7083:358986e4d053 + 2.5 7065:5ce837b1a1d8 + 2.2 6412:bed9f9de4ef3 + +.. _hg-current-branch: + +Which branch is currently checked out in my working copy? +''''''''''''''''''''''''''''''''''''''''''''''''''''''''' + +Use:: + + $ hg branch + default + +Or to get more information:: + + $ hg summary + parent: 7083:358986e4d053 tip + Merge forward. + branch: default + commit: 17 unknown (clean) + update: (current) + + +.. _hg-switch-branches: + +How do I switch between branches inside my working copy? +'''''''''''''''''''''''''''''''''''''''''''''''''''''''' + +Simply use ``hg update`` to checkout another branch in the current directory:: + + $ hg branch + default + $ hg update 3.3 + 86 files updated, 0 files merged, 11 files removed, 0 files unresolved + $ hg branch + 3.3 + +Adding the ``-v`` option to ``hg update`` will list all updated files. + + +I want to keep a separate working copy per development branch, is it possible? +'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' + +There are two ways: + +1) Use the "`share extension`_" as described in the :ref:`multiple-clones` + section; +2) Create several clones of your local repository; + +If you want to use the second way, you can do:: + + $ hg clone jython jy33 + updating to branch default + 3434 files updated, 0 files merged, 0 files removed, 0 files unresolved + $ cd jy33 + $ hg update 3.3 + 86 files updated, 0 files merged, 11 files removed, 0 files unresolved + +The current branch in a working copy is "sticky": if you pull in some new +changes, ``hg update`` will update to the head of the *current branch*. + +.. _share extension: http://mercurial.selenic.com/wiki/ShareExtension + + +.. _hg-paths: + +How do I link my local repository to a particular remote repository? +'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' + +Your local repository is linked by default to the remote repository it +was *cloned* from. If you created it from scratch, however, it is not linked +to any remote repository. In ``.hg/hgrc`` file for the local repository, add +or modify the following section:: + + [paths] + default = ssh://hg@hg.python.org/jython-docs/devguide + +This example is for a local repository that mirrors the ``devguide`` repository +on ``hg.python.org``. The same approach works for other remote repositories. + +Anywhere that ```` is used in the commands in this +FAQ, ``hg`` will use the default remote repository if you omit the parameter. + + +How do I create a shorthand alias for a remote repository? +'''''''''''''''''''''''''''''''''''''''''''''''''''''''''' + +In your global ``.hgrc`` file add a section similar to the following:: + + [paths] + dg = ssh://hg@hg.python.org/jython-docs/devguide + +This example creates a ``dg`` alias for this ``devguide`` repository +on ``hg.python.org``. This allows "dg" to be entered instead of the +full URL for commands taking a repository argument (e.g. ``hg pull dg`` or +``hg outgoing dg``). + +Anywhere that ```` is used in the commands in this +FAQ, ``hg`` should accept an alias in place of a complete remote URL. + + +How do I compare my local repository to a remote repository? +'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' + +To display the list of changes that are in your local repository, but not +in the remote, use:: + + hg outgoing + +This is the list of changes that will be sent if you call +``hg push ``. It does **not** include any :ref:`uncommitted +changes ` in your working copy! + +Conversely, for the list of changes that are in the remote repository but +not in the local, use:: + + hg incoming + +This is the list of changes that will be retrieved if you call +``hg pull ``. + +.. note:: + In most daily use, you will work against the default remote repository, + and therefore simply type ``hg outgoing`` and ``hg incoming``. + + In this case, you can also get a synthetic summary using + ``hg summary --remote``. + + +How do I update my local repository to be in sync with a remote repository? +''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' + +Run:: + + hg pull + +from the repository you wish to pull the latest changes into. Most of the +time, that repository is a clone of the repository you want to pull from, +so you can simply type:: + + hg pull + +This doesn't update your working copy, though. See below: + + +How do I update my working copy with the latest changes? +'''''''''''''''''''''''''''''''''''''''''''''''''''''''' + +Do:: + + hg update + +This will update your working copy with the latest changes on the +:ref:`current branch `. If you had :ref:`uncommitted +changes ` in your working copy, they will be merged in. + +If you find yourself typing often ``hg pull`` followed by ``hg update``, +be aware that you can combine them in a single command:: + + hg pull -u + + +.. _hg-local-workflow: + +How do I apply a patch? +''''''''''''''''''''''' + +If you want to try out or review a patch generated using Mercurial, do:: + + patch -p1 < somework.patch + +This will apply the changes in your working copy without committing them. +If the patch was not created by Mercurial (for example, a patch created by +Subversion and thus lacking any ``a``/``b`` directory prefixes in the patch), +replace ``-p1`` with ``-p0``. + +If the patch contains renames, deletions or copies, and you intend committing +it after your review, you might prefer using:: + + hg import --no-commit somework.patch + +If you want to work on the patch using mq_ (Mercurial Queues), type instead:: + + hg qimport somework.patch + +This will create a patch in your queue with a name that matches the filename. +You can use the ``-n`` argument to specify a different name. To have the +patch applied to the working copy, type:: + + hg qpush + +Finally, to delete the patch, first un-apply it if necessary using ``hg qpop``, +then do:: + + hg qdelete somework.patch + +.. _extended diff format: http://www.selenic.com/mercurial/hg.1.html#diffs +.. _mq: http://mercurial.selenic.com/wiki/MqExtension + + +.. _merge-patch: + +How do I solve conflicts when applying a patch fails? +''''''''''''''''''''''''''''''''''''''''''''''''''''' + +The standard ``patch`` command, as well as ``hg import``, will produce +unhelpful ``*.rej`` files when it fails applying parts of a patch. +We suggest you try the mpatch_ utility, which can help resolve a number of +common causes of patch rejects. + +To make use of ``mpatch`` transparent, you can define a shell alias in one +of your startup files. For example, if you want it to open the ``kdiff3`` +merge program to fix failing patch hunks:: + + alias patch='mpatch --merge=kdiff3' + +or if you want it to automatically solve conflicts by using heuristics:: + + alias patch='mpatch --auto --no-merge' + +.. _mpatch: http://oss.oracle.com/~mason/mpatch/ + + +How do I add a file or directory to the repository? +''''''''''''''''''''''''''''''''''''''''''''''''''' + +Simply specify the path to the file or directory to add and run:: + + hg add PATH + +If ``PATH`` is a directory, Mercurial will recursively add any files in that +directory and its descendants. + +If you want Mercurial to figure out by itself which files should be added +and/or removed, just run:: + + hg addremove + +**Be careful** though, as it might add some files that are not desired in +the repository (such as build products, cache files, or other data). + +You will then need to run ``hg commit`` (as discussed below) to commit +the file(s) to your local repository. + + +What's the best way to split a file into several files? +''''''''''''''''''''''''''''''''''''''''''''''''''''''' + +To split a file into several files (e.g. a module converted to a package or a +long doc file divided in two separate documents) use ``hg copy``:: + + hg copy module.rst module2.rst + +and then remove the parts that are not necessary from ``module.rst`` and +``module2.rst``. This allows Mercurial to know that the content of +``module2.rst`` used to be in ``module.rst``, and will make subsequent merges +easier. If necessary, you can also use ``hg copy`` several times. + +If you simply create ``module2.rst``, add it with ``hg add``, and copy part of +the content from ``module.rst``, Mercurial won't know that the two file are +related. + + +How do I delete a file or directory in the repository? +'''''''''''''''''''''''''''''''''''''''''''''''''''''' + +Specify the path to be removed with:: + + hg remove PATH + +This will remove the file or the directory from your working copy; you will +have to :ref:`commit your changes ` for the removal to be recorded +in your local repository. + + +.. _hg-status: + +What files are modified in my working copy? +''''''''''''''''''''''''''''''''''''''''''' + +Running:: + + hg status + +will list any pending changes in the working copy. These changes will get +committed to the local repository if you issue an ``hg commit`` without +specifying any path. + +Some +key indicators that can appear in the first column of output are: + + = =========================== + A Scheduled to be added + R Scheduled to be removed + M Modified locally + ? Not under version control + = =========================== + +If you want a line-by-line listing of the differences, use:: + + hg diff + + +How do I revert a file I have modified back to the version in the repository? +''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' + +Running:: + + hg revert PATH + +will revert ``PATH`` to its version in the repository, throwing away any +changes you made locally. If you run:: + + hg revert -a + +from the root of your working copy it will recursively restore everything +to match up with the repository. + + +How do I find out who edited or what revision changed a line last? +'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' + +You want:: + + hg annotate PATH + +This will output to stdout every line of the file along with which revision +last modified that line. When you have the revision number, it is then +easy to :ref:`display it in detail `. + + +.. _hg-log: + +How can I see a list of log messages for a file or specific revision? +''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' + +To see the history of changes for a specific file, run:: + + hg log -v [PATH] + +That will list all messages of revisions which modified the file specified +in ``PATH``. If ``PATH`` is omitted, all revisions are listed. + +If you want to display line-by-line differences for each revision as well, +add the ``-p`` option:: + + hg log -vp [PATH] + +.. _hg-log-rev: + +If you want to view the differences for a specific revision, run:: + + hg log -vp -r + + +How can I see the changeset graph in my repository? +''''''''''''''''''''''''''''''''''''''''''''''''''' + +In Mercurial repositories, changesets don't form a simple list, but rather +a graph: every changeset has one or two parents (it's called a merge changeset +in the latter case), and can have any number of children. + +The graphlog_ extension is very useful for examining the structure of the +changeset graph. It is bundled with Mercurial. + +Graphical tools, such as TortoiseHG, will display the changeset graph +by default. + +.. _graphlog: http://mercurial.selenic.com/wiki/GraphlogExtension + + +How do I update to a specific release tag? +'''''''''''''''''''''''''''''''''''''''''' + +Run:: + + hg tags + +to get a list of tags. To update your working copy to a specific tag, use:: + + hg update + + +How do I find which changeset introduced a bug or regression? +''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' + +``hg bisect``, as the name indicates, helps you do a bisection of a range of +changesets. + +You need two changesets to start the search: one that is "good" +(doesn't have the bug), and one that is "bad" (has the bug). Usually, you +have just noticed the bug in your working copy, so you can start with:: + + hg bisect --bad + +Then you must ``update`` to a previous changeset that doesn't have the bug. +You can conveniently choose a faraway changeset (for example a former release), +and check that it is indeed "good". Then type:: + + hg bisect --good + +Mercurial will automatically bisect so as to narrow the range of possible +culprits, until a single changeset is isolated. Each time Mercurial presents +you with a new changeset, re-compile Python and run the offending test, for +example:: + + make -j2 + ./python -m test -uall test_sometest + +Then, type either ``hg bisect --good`` or ``hg bisect --bad`` depending on +whether the test succeeded or failed. + + +How come feature XYZ isn't available in Mercurial? +'''''''''''''''''''''''''''''''''''''''''''''''''' + +Mercurial comes with many bundled extensions which can be explicitly enabled. +You can get a list of them by typing ``hg help extensions``. Some of these +extensions, such as ``color``, can prettify output; others, such as ``fetch`` +or ``graphlog``, add new Mercurial commands. + +There are also many `configuration options`_ to tweak various aspects of the +command line and other Mercurial behaviour; typing `man hgrc`_ displays +their documentation inside your terminal. + +In the end, please refer to the Mercurial `wiki`_, especially the pages about +`extensions`_ (including third-party ones) and the `tips and tricks`_. + + +.. _man hgrc: http://www.selenic.com/mercurial/hgrc.5.html +.. _wiki: http://mercurial.selenic.com/wiki/ +.. _extensions: http://mercurial.selenic.com/wiki/UsingExtensions +.. _tips and tricks: http://mercurial.selenic.com/wiki/TipsAndTricks +.. _configuration options: http://www.selenic.com/mercurial/hgrc.5.html + + +.. _core-devs-faqs: + +For core developers +------------------- + +These FAQs are intended mainly for core developers. + + +.. _hg-commit: + +How do I commit a change to a file? +''''''''''''''''''''''''''''''''''' + +To commit any changes to a file (which includes adding a new file or deleting +an existing one), you use the command:: + + hg commit [PATH] + +``PATH`` is optional: if it is omitted, all changes in your working copy +will be committed to the local repository. When you commit, be sure that all +changes are desired by :ref:`reviewing them first `; +also, when making commits that you intend to push to public repositories, +you should **not** commit together unrelated changes. + +To abort a commit that you are in the middle of, leave the message +empty (i.e., close the text editor without adding any text for the +message). Mercurial will then abort the commit operation so that you can +try again later. + +Once a change is committed to your local repository, it is still only visible +by you. This means you are free to experiment with as many local commits +you feel like. + +.. note:: + If you do not like the default text editor Mercurial uses for + entering commit messages, you may specify a different editor, + either by changing the ``EDITOR`` environment variable or by setting + a Mercurial-specific editor in your global ``.hgrc`` with the ``editor`` + option in the ``[ui]`` section. + + +.. _hg-merge-conflicts: + +How do I solve merge conflicts? +''''''''''''''''''''''''''''''' + +The easiest way is to install KDiff3 --- Mercurial will open it automatically +in case of conflicts, and you can then use it to solve the conflicts and +save the resulting file(s). KDiff3 will also take care of marking the +conflicts as resolved. + +If you don't use a merge tool, you can use ``hg resolve --list`` to list the +conflicting files, resolve the conflicts manually, and the use +``hg resolve --mark `` to mark these conflicts as resolved. +You can also use ``hg resolve -am`` to mark all the conflicts as resolved. + +.. note:: + Mercurial will use KDiff3 automatically if it's installed and it can find + it --- you don't need to change any settings. KDiff3 is also already + included in the installer of TortoiseHg. For more information, see + http://mercurial.selenic.com/wiki/KDiff3. + + +.. _hg-null-merge: + +How do I make a null merge? +''''''''''''''''''''''''''' + +If you committed something (e.g. on 3.3) that shouldn't be ported on newer +branches (e.g. on default), you have to do a *null merge*:: + + cd 3.x + hg merge 3.3 + hg revert -ar default + hg resolve -am # needed only if the merge created conflicts + hg ci -m '#12345: null merge with 3.3.' + +Before committing, ``hg status`` should list all the merged files as ``M``, +but ``hg diff`` should produce no output. This will record the merge without +actually changing the content of the files. + + +.. _hg-heads-merge: + +I got "abort: push creates new remote heads!" while pushing, what do I do? +'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' + +If you see this message while pushing, it means that you committed something +on a clone that was not up to date, thus creating a new head. +This usually happens for two reasons: + +1. You forgot to run ``hg pull`` and/or ``hg up`` before committing; +2. Someone else pushed on the main repo just before you, causing a push race; + +First of all you should pull the new changesets using ``hg pull``. Then you can +use ``hg heads`` to see which branches have multiple heads. + +If only one branch has multiple heads, you can do:: + + cd default + hg heads . + hg up csid-of-the-other-head + hg merge + hg ci -m 'Merge heads.' + +``hg heads .`` will show you the two heads of the current branch: the one you +pulled and the one you created with your commit (you can also specify a branch +with ``hg heads ``). While not strictly necessary, it is highly +recommended to switch to the other head before merging. This way you will be +merging only your changeset with the rest, and in case of conflicts it will be +a lot easier. + +If more than one branch has multiple heads, you have to repeat these steps for +each branch. Since this creates new changesets, you will also have to +:ref:`merge them between branches `. +For example, if both ``3.3`` +and ``default`` have multiple heads, you should first merge heads in ``3.3``, +then merge heads in ``default``, and finally merge ``3.3`` with ``default`` +using ``hg merge 3.3`` as usual. + +In order to avoid this, you should *always remember to pull and update before +committing*. + + +How do I undo the changes made in a recent commit? +'''''''''''''''''''''''''''''''''''''''''''''''''' + +First, this should not happen if you take the habit of :ref:`reviewing changes +` before committing them. + +In any case, run:: + + hg backout + +This will modify your working copy so that all changes in ```` +(including added or deleted files) are undone. You then need to :ref:`commit +` these changes so that the backout gets permanently recorded. + +.. note:: + These instructions are for Mercurial 1.7 and higher. ``hg backout`` has + a slightly different behaviour in versions before 1.7. + + +SSH +=== + +How do I generate an SSH 2 public key? +-------------------------------------- + +All generated SSH keys should be sent to hgaccounts@python.org for +adding to the list of keys. + +UNIX +'''' + +Run:: + + ssh-keygen -t rsa + +This will generate two files; your public key and your private key. Your +public key is the file ending in ``.pub``. + +Windows +''''''' + +Use PuTTYgen_ to generate your public key. Choose the "SSH2 DSA" radio button, +have it create an OpenSSH formatted key, choose a password, and save the private +key to a file. Copy the section with the public key (using Alt-P) to a file; +that file now has your public key. + +.. _PuTTYgen: http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html + + +Is there a way to avoid having to constantly enter my password for my SSH 2 public key? +--------------------------------------------------------------------------------------- + +UNIX +'''' + +Use ``ssh-agent`` and ``ssh-add`` to register your private key with SSH for +your current session. The simplest solution, though, is to use KeyChain_, +which is a shell script that will handle ``ssh-agent`` and ``ssh-add`` for you +once per login instead of per session. + +.. _KeyChain: http://www.gentoo.org/proj/en/keychain/ + + +.. _pageant: + +Windows +''''''' + +The Pageant program is bundled with TortoiseHg. You can find it in its +installation directory (usually ``C:\Program Files (x86)\TortoiseHg\``); +you can also `download it separately +`_. + +Running Pageant will prevent you from having to type your password constantly. +If you add a shortcut to Pageant to your Autostart group and edit the shortcut +so that the command line includes an argument to your private key then Pageant +will load the key every time you log in. + + +Can I make commits from machines other than the one I generated the keys on? +---------------------------------------------------------------------------- + +You can :ref:`make commits ` from any machine, since they will be +recorded in your *local repository*. + +However, to push these changes to the remote server, you will need proper +credentials. All you need is to make sure that the machine you want to +push changes from has both the public and private keys in the standard +place that ssh will look for them (i.e. ~/.ssh on Unix machines). +Please note that although the key file ending in .pub contains your +user name and machine name in it, that information is not used by the +verification process, therefore these key files can be moved to a +different computer and used for verification. Please guard your keys +and never share your private key with anyone. If you lose the media +on which your keys are stored or the machine on which your keys are +stored, be sure to report this to pydotorg@python.org at the same time +that you change your keys. diff --git a/fixingissues.rst b/fixingissues.rst index 7821d41205..b8d1635104 100644 --- a/fixingissues.rst +++ b/fixingissues.rst @@ -1,8 +1,15 @@ +.. This file is derived from a file of the same name in the CPython devguide + and will receive updates from the CPython guide by merging. + .. _fixingissues: Fixing "easy" Issues (and Beyond) ================================= +.. warning:: At present, this is not much modified from the CPython base. + We have intended for a long time to update our tracker so that CPython + instructions would in fact apply, but they don't yet. + When you feel comfortable enough to want to help tackle issues by trying to create a patch to fix an issue, you can start by looking at the `"easy" issues`_. These issues *should* be ones where it should take no longer than a @@ -20,6 +27,6 @@ discussion to help bring it to a close). Regardless of why the issue is open, you can also always provide useful comments if you do attempt a fix, successful or not. -.. _"easy" issues: https://bugs.python.org/issue?status=1&@sort=-activity&@dispname=Easy%20issues&@startwith=0&@filter=&@group=priority&@columns=id,activity,title,creator,status&keywords=6&@action=search&@pagesize=50 +.. _"easy" issues: https://bugs.jython.org/issue?status=1&@sort=-activity&@dispname=Easy%20issues&@startwith=0&@filter=&@group=priority&@columns=id,activity,title,creator,status&keywords=6&@action=search&@pagesize=50 .. TODO: add something about no active core developer for the area? diff --git a/gitbootcamp.rst b/gitbootcamp.rst index e7ef341fa4..8874ca2311 100644 --- a/gitbootcamp.rst +++ b/gitbootcamp.rst @@ -1,3 +1,6 @@ +.. This file is derived from a file of the same name in the CPython devguide + and will receive updates from the CPython guide by merging. + .. highlight:: console .. _gitbootcamp: @@ -5,11 +8,15 @@ Git Bootcamp and Cheat Sheet ============================ +.. warning:: At present, this is not much modified from the CPython base. + In this section, we'll go over some commonly used Git commands that are relevant to CPython's workflow. .. contents:: +.. Keep the cpython target for setup.rst. +.. _fork-jython: .. _fork-cpython: Forking CPython GitHub Repository diff --git a/grammar_jy.rst b/grammar_jy.rst new file mode 100644 index 0000000000..4514341513 --- /dev/null +++ b/grammar_jy.rst @@ -0,0 +1,27 @@ +.. Jython companion to grammar.rst + +.. _grammar-jy: + +Changing Jython's Grammar +========================= + +.. warning:: At present, this is not much more than a copy of the CPython original + with the obviously inapplicable crudely hacked out. + +Abstract +-------- + +There's more to changing Python's grammar than editing +:file:` grammar/Python.g`. +For a start, we only do this in order to track a change developeed on +the reference implementation CPython. +This document aims to be a checklist of places that must also be fixed +when such a change is adopted by Jython. + +It is probably incomplete. If you see omissions, submit a bug or patch. + + + +Checklist +--------- + diff --git a/help.rst b/help.rst index ea23f8d6a6..f6020e0ebe 100644 --- a/help.rst +++ b/help.rst @@ -1,9 +1,12 @@ +.. This file is derived from a file of the same name in the CPython devguide + and will receive updates from the CPython guide by merging. + .. _help: Where to Get Help ================= -If you are working on Python it is very possible you will come across an issue +If you are working on Jython it is very possible you will come across an issue where you need some assistance to solve it (this happens to core developers all the time). @@ -13,47 +16,33 @@ usage then please check the rest of this guide first as it should answer your question. -Ask #python-dev ---------------- - -If you are comfortable with IRC you can try asking on ``#python-dev`` (on -the `freenode`_ network). Typically there are a number of experienced -developers, ranging from triagers to core developers, who can answer -questions about developing for Python. Just remember that ``#python-dev`` -is for questions involving the development *of* Python whereas ``#python`` -is for questions concerning development *with* Python. - -.. _freenode: http://freenode.net/ - - Core Mentorship --------------- -If you are interested in improving Python and contributing to its development, +If you are interested in improving Jython and contributing to its development, but don’t yet feel entirely comfortable with the public channels mentioned -above, `Python Mentors`_ are here to help you. Python is fortunate to have a -community of volunteer core developers willing to mentor anyone wishing to -contribute code, work on bug fixes or improve documentation. Everyone is -welcomed and encouraged to contribute. +above, `Python Mentors`_ may be there to help you. The Python Mentors list was +originally created to help with CPython development, but since Jython an +implementation acknowledgef by the PSF, it may be a good place to find help for +Jython as well. Python is fortunate to have a community of volunteer core +developers willing to mentor anyone wishing to contribute code, work on bug +fixes or improve documentation. Everyone is welcomed and encouraged to +contribute. -.. _Python Mentors: https://www.python.org/dev/core-mentorship/ +.. _Python Mentors: http://pythonmentors.com +.. FIXME: would Python Mentors count themselves a good place to go for Jython? Mailing Lists ------------- -Further options for seeking assistance include the `python-ideas`_ and -`python-dev`_ mailing lists. Python-ideas contains discussion of speculative -Python language ideas for possible inclusion into the language. If an idea -gains traction it can then be discussed and honed to the point of becoming a -solid proposal and presented on python-dev. Python-dev contains discussion -of current Python design issues, release mechanics, and maintenance of -existing releases. As with ``#python-dev``, these mailing lists are for -questions involving the development *of* Python, **not** for development -*with* Python. - -.. _python-ideas: https://mail.python.org/mailman/listinfo/python-ideas -.. _python-dev: https://mail.python.org/mailman/listinfo/python-dev +Further options for seeking assistance include the `jython-users`_ and +`jython-dev`_ mailing lists. +Jython-users contains discussion of software developement *using* Jython. +Jython-dev contains discussion of current Jython design +issues, release mechanics, and maintenance of existing releases. Just remember +that ``jython-dev`` is for questions involving the development *of* Jython +whereas ``jython-users`` is for questions concerning development *with* Jython. File a Bug @@ -61,9 +50,13 @@ File a Bug If you strongly suspect you have stumbled on a bug (be it in the build process, in the test suite, or in other areas), then open an issue on the -`issue tracker`_. As with every bug report it is strongly advised that +`Jython GitHub repository`_. As with every bug report it is strongly advised that you detail which conditions triggered it (including the OS name and version, -and what you were trying to do), as well as the exact error message you -encountered. +which JVM and version, and what you were trying to do), as well as the exact +error message you encountered. + +.. _Jython GitHub repository: https://github.com/jython/jython +.. _Jython issue tracker: http://bugs.jython.org -.. _issue tracker: https://bugs.python.org +.. _jython-users: https://lists.sourceforge.net/lists/listinfo/jython-users +.. _jython-dev: https://lists.sourceforge.net/lists/listinfo/jython-dev diff --git a/index.rst b/index.rst index 18e44d72c7..f6b5a85845 100644 --- a/index.rst +++ b/index.rst @@ -1,113 +1,145 @@ +.. This file has been extensively modified from the CPython version. + Normally this would mean a new file index_jy.rst, but index.rst is the + starting point for the documentation. + We therefore keep it as modified, although merging changes from the CPython + devguide will be difficult for this file. + ======================== -Python Developer's Guide +Jython Developer's Guide ======================== This guide is a comprehensive resource for :ref:`contributing ` -to Python_ -- for both new and experienced contributors. It is -:ref:`maintained ` by the same community -that maintains Python. We welcome your contributions to Python! +to Jython_ -- for both new and experienced contributors. +It has been (somewhat incompletely) +adapted from the CPython Developer's Guide and the reader should +bear in mind that: + +* The Jython process, like CPython's, is based GitHub PRs, + but Jython does not have the same set of tool integrations. +* Jython migrated from Mercurial in 2020, much later than CPython, + so Jython-specific parts of the guide may refer to the old process. +* Statements about "Python" should apply to both CPython and Jython. + The adaptation is imperfect: parts of the guide will say (or mean) CPython. + +This guide is :ref:`maintained ` by the same +community that maintains CPython and Jython. +We welcome your contributions to either implementation of Python! Quick Reference --------------- -Here are the basic steps needed to get :ref:`set up ` and contribute a -patch. This is meant as a checklist, once you know the basics. For complete -instructions please see the :ref:`setup guide `. +Mercurial +^^^^^^^^^ + +Mercurial is no longer used. +The previous official repository at ``https://hg.python.org/jython`` is not current. -1. Install and set up :ref:`Git ` and other dependencies - (see the :ref:`Get Setup ` page for detailed information). -2. Fork `the CPython repository `_ - to your GitHub account and :ref:`get the source code ` using:: +GitHub +^^^^^^ +.. highlight:: bash - git clone https://github.com//cpython +Here are the basic steps needed to get :ref:`set up ` and contribute a +patch. This is meant as a checklist, once you know the basics. For complete +instructions please see the :ref:`setup guide `. -3. Build Python, on UNIX and Mac OS use:: +1. Install and set up :ref:`Git ` and other dependencies + (see the :ref:`Get Setup ` page for detailed information). - ./configure --with-pydebug && make -j +2. Fork the `Jython GitHub repository`_ + to your GitHub account and :ref:`get the source code ` using:: - and on Windows use:: + git clone https://github.com//jython - PCbuild\build.bat -e -d +3. Build Jython using:: - See also :ref:`more detailed instructions `, - :ref:`how to build dependencies `, and the - plaform-specific pages for :ref:`UNIX `, - :ref:`Mac OS `, and :ref:`Windows `. + ant -4. :doc:`Run the tests `:: + issued in the base check-out directory. + The built application will be in subdirectory ``dist``. + See also :ref:`more detailed instructions `, + and :ref:`how to build dependencies `. - ./python -m test -j3 +4. :doc:`Run the tests `:: - On :ref:`most ` Mac OS X systems, replace :file:`./python` - with :file:`./python.exe`. On Windows, use :file:`python.bat`. With Python - 2.7, replace ``test`` with ``test.regrtest``. + dist/bin/jython -m test.regrtest -e + + This applies for Jython 2.7. + To test Jython 3, when it exists, replace ``test.regrtest`` with ``test``. 5. Create a new branch where your work for the issue will go, e.g.:: git checkout -b fix-issue-12345 master - If an issue does not already exist, please `create it - `_. Trivial issues (e.g. typo fixes) do not - require any issue to be created. + If an issue does not already exist, please create it on + the `Jython GitHub repository`_. + The legacy `bug tracker `_ still operates + (for Jython 2.7 only) but is not preferred. + Trivial issues (e.g. typo fixes) do not require any issue to be created: + create a PR directly. -6. Once you fixed the issue, run the tests, run ``make patchcheck``, and if +6. Once you fixed the issue, run the tests, and if everything is ok, commit. -7. Push the branch on your fork on GitHub and :doc:`create a pull request - `. Include the issue number using ``bpo-NNNN`` in the - pull request description. For example:: +7. Push the branch to your fork on GitHub:: + + git push -u origin fix-issue-12345 + + and :doc:`create a pull request + `. + Include the issue number using GitHub conventions in the pull request + description. + (Use ``bjo-NNNN`` if it is from the legacy tracker.) + +.. note:: - bpo-12345: Fix some bug in spam module + bpo-NNNN is used by CPython for for bugs.python.org + and is picked up by CPython's GitHub tools. + For Jython we need our own naming convention, but cannot yet re-use the + CPython tools. + We intend to move away from bugs.jython.org to GitHub issues more aggressively + that CPython has made the switch. .. note:: First time contributors will need to sign the Contributor Licensing Agreement (CLA) as described in the :ref:`Licensing ` section of this guide. + This requires an account on the CPython's legacy bug tracker. + Quick Links ----------- Here are some links that you probably will reference frequently while -contributing to Python: +contributing to Jython: -* `Issue tracker`_ -* `Buildbot status`_ +* `Jython GitHub repository`_ (or maybe the `Jython issue tracker`_) * :doc:`help` * PEPs_ (Python Enhancement Proposals) * :doc:`gitbootcamp` .. _branchstatus: -Status of Python branches +Status of Jython branches ------------------------- -+------------------+--------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ -| Branch | Schedule | Status | First release | End-of-life | Comment | -+==================+==============+=============+================+================+============================================================================+ -| master | :pep:`537` | features | *2018-06-15* | *2023-06-15* | The master branch is currently the future version Python 3.7. | -+------------------+--------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ -| 3.6 | :pep:`494` | bugfix | 2016-12-23 | *2021-12-23* | `Most recent binary release: Python 3.6.3 | -| | | | | | `_ | -+------------------+--------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ -| 2.7 | :pep:`373` | bugfix | 2010-07-03 | *2020-01-01* | The support has been extended to 2020 (1). | -| | | | | | `Most recent binary release: Python 2.7.13 | -| | | | | | `_ | -+------------------+--------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ -| 3.5 | :pep:`478` | security | 2015-09-13 | *2020-09-13* | `Most recent binary release: Python 3.5.4 | -| | | | | | `_ | -+------------------+--------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ -| 3.4 | :pep:`429` | security | 2014-03-16 | *2019-03-16* | `Most recent security release: Python 3.4.7 | -| | | | | | `_ | -+------------------+--------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ - -(1) The exact date of Python 2.7 end-of-life has not been decided yet. It will -be decided by Python 2.7 release manager, Benjamin Peterson, who will update -the :pep:`373`. Read also the `[Python-Dev] Exact date of Python 2 EOL? -`_ thread -on python-dev (March 2017). + ++--------+----------+-------------+---------------+-------------+-----------------------------------------------------------------------------+ +| Branch | Schedule | Status | First release | End-of-life | Comment | ++========+==========+=============+===============+=============+=============================================================================+ +| master | | bugfix | | | PSF support for Python 2 ended in 2020 (see :pep:`373`) but | +| | | | | | the most recent release of Jython is still `Jython 2.7.2 | +| | | | | | `_ | ++--------+----------+-------------+---------------+-------------+-----------------------------------------------------------------------------+ +| 3.8 | | features | | | This branch is the future of Jython. | +| | | | | | (It should be master, but is just a gleam in the eye at present.) | ++--------+----------+-------------+---------------+-------------+-----------------------------------------------------------------------------+ +| 2.5 | | end-of-life | | | Final release: `Jython 2.5.3 | +| | | | | | `_ | ++--------+----------+-------------+---------------+-------------+-----------------------------------------------------------------------------+ + Status: @@ -124,40 +156,21 @@ Dates in *italic* are scheduled and can be adjusted. By default, the end-of-life is scheduled 5 years after the first release. It can be adjusted by the release manager of each branch. Versions older than 2.7 have reached end-of-life. +The Jython project follows the Python Software Foundation in the naming of +versions of the language (e.g. Jython 2.7 implements Python 2.7), and whether an +implementation of that version has reached reached end-of-life. +The third element (micro) has bears no relationship to CPython micro-versions. See also :ref:`Security branches `. -Each release of Python is tagged in the source repo with a tag of the form +Jython names its releases in the same scheme as CPython. +Each release of Jython is tagged in the source repo with a tag of the form ``vX.Y.ZTN``, where ``X`` is the major version, ``Y`` is the minor version, ``Z`` is the micro version, ``T`` is the release level (``a`` for alpha releases, ``b`` for beta, ``rc`` release candidate, and *null* for final releases), and ``N`` is the release serial number. Some examples of release tags: ``v3.7.0a1``, ``v3.6.3``, ``v2.7.14rc1``. -The code base for a release cycle which has reached end-of-life status -is frozen and no longer has a branch in the repo. The final state of -the end-of-lifed branch is recorded as a tag with the same name as the -former branch, e.g. ``3.3`` or ``2.6``. For reference, here are the -most recently end-of-lifed release cycles: - -+------------------+--------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ -| Tag | Schedule | Status | First release | End-of-life | Comment | -+==================+==============+=============+================+================+============================================================================+ -| 3.3 | :pep:`398` | end-of-life | 2012-09-29 | 2017-09-29 | `Final release: Python 3.3.7 | -| | | | | | `_ | -+------------------+--------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ -| 3.2 | :pep:`392` | end-of-life | 2011-02-20 | 2016-02-20 | `Final release: Python 3.2.6 | -| | | | | | `_ | -+------------------+--------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ -| 3.1 | :pep:`375` | end-of-life | 2009-06-27 | 2012-04-11 | `Final release: Python 3.1.5 | -| | | | | | `_ | -+------------------+--------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ -| 3.0 | :pep:`361` | end-of-life | 2008-12-03 | 2009-01-13 | `Final release: Python 3.0.1 | -| | | | | | `_ | -+------------------+--------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ -| 2.6 | :pep:`361` | end-of-life | 2008-10-01 | 2013-10-29 | `Final release: Python 2.6.9 | -| | | | | | `_ | -+------------------+--------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ .. _contributing: @@ -169,6 +182,8 @@ developer's guide. If you still have questions after reviewing the material in this guide, then the `Python Mentors`_ group is available to help guide new contributors through the process. +.. FIXME: do Python Mentors help Jython contributors with process? + A number of individuals from the Python community have contributed to a series of excellent guides at `Open Source Guides `_. @@ -177,30 +192,31 @@ Core developers and contributors alike will find the following guides useful: * `How to Contribute to Open Source `_ * `Building Welcoming Communities `_ -Guide for contributing to Python: +Guide for contributing to Jython: -* :doc:`setup` +* :doc:`setup_jy` * :doc:`help` -* :doc:`pullrequest` -* :doc:`runtests` +* :doc:`patch_hg_jy` (existing process) +* :doc:`pullrequest` (partly usable future process) +* :doc:`runtests_jy` * Beginner tasks to become familiar with the development process * :doc:`docquality` * :doc:`coverage` * Advanced tasks for once you are comfortable * :doc:`silencewarnings` - * Fixing issues found by the :doc:`buildbots ` + * Fixing issues found by the :doc:`buildbots ` * Helping out with reviewing `open pull requests`_. See :ref:`how to review a Pull Request `. * :doc:`fixingissues` * :ref:`tracker` and :ref:`helptriage` - * :doc:`triaging` + * :doc:`triaging_jy` * :doc:`experts` * :doc:`communication` * :doc:`coredev` + * :doc:`committing_hg_jy` * :doc:`committing` * :doc:`devcycle` - * :doc:`buildbots` - * :doc:`coverity` + * :doc:`buildbots_jy` * :doc:`gitbootcamp` It is **recommended** that the above documents be read in the order listed. You @@ -231,19 +247,19 @@ happen and that process is also described as part of this guide: Other Interpreter Implementations --------------------------------- -This guide is specifically for contributing to the Python reference interpreter, -also known as CPython (while most of the standard library is written in Python, -the interpreter core is written in C and integrates most easily with the C and -C++ ecosystems). +This guide is specifically for contributing to Jython, that is, +Python on the JVM. (While most of the standard library is written in Python, +the interpreter core is written in Java and integrates most easily with the Java +SE and EE ecosystems). There are other Python implementations, each with a different focus. Like -CPython, they always have more things they would like to do than they have +Jython, they always have more things they would like to do than they have developers to work on them. Some major example that may be of interest are: +* CPython_: The reference implementation of Python implemented in C, + and the main focus of language development. * PyPy_: A Python interpreter focused on high speed (JIT-compiled) operation on major platforms -* Jython_: A Python interpreter focused on good integration with the Java - Virtual Machine (JVM) environment * IronPython_: A Python interpreter focused on good integration with the Common Language Runtime (CLR) provided by .NET and Mono * Stackless_: A Python interpreter focused on providing lightweight @@ -255,22 +271,13 @@ Key Resources ------------- * Coding style guides - * :PEP:`7` (Style Guide for C Code) + * Jython's `Java coding standard`_ * :PEP:`8` (Style Guide for Python Code) -* `Issue tracker`_ - * `Meta tracker `_ (issue - tracker for the issue tracker) - * :doc:`experts` - * `Firefox search engine plug-in`_ -* `Buildbot status`_ * Source code - * `Browse online `_ - * `Snapshot of the *master* branch `_ - * `Daily OS X installer `_ + * `Browse in GitHub `_ + * PEPs_ (Python Enhancement Proposals) * :doc:`help` -* :doc:`developers` - .. _resources: @@ -280,16 +287,12 @@ Additional Resources * Anyone can clone the sources for this guide. See :ref:`helping-with-the-developers-guide`. * Help with ... - * :doc:`exploring` - * :doc:`grammar` - * :doc:`compiler` + * :doc:`exploring_jy` + * :doc:`grammar_jy` + * :doc:`compiler_jy` * Tool support - * :doc:`gdb` - * :doc:`clang` - * Various tools with configuration files as found in the `Misc directory`_ * Information about editors and their configurations can be found in the `wiki `_ -* `python.org maintenance`_ * :ref:`Search this guide ` @@ -314,27 +317,54 @@ Full Table of Contents .. toctree:: :numbered: - setup + setup_jy help + patch_hg_jy pullrequest - runtests - coverage + runtests_jy + coverage_jy docquality documenting silencewarnings fixingissues tracker - triaging + triaging_jy communication - porting coredev - developers + committing_hg_jy committing devcycle - buildbots + buildbots_jy stdlibchanges langchanges experts + release_jy + exploring_jy + grammar_jy + compiler_jy + gitbootcamp + faq_hg_jy + +Specific to CPython_ +-------------------- + +These are sections from the CPython guide, retained for reference. +A comparison with the CPython implementation of a feature will sometimes help +you understand the Jython one. + +.. We're also keeping these files for the technical reason that changes from + CPython upstream can only be merged if these CPython files continue to exist. + +.. toctree:: + :maxdepth: 1 + + setup + runtests + coverage + triaging + porting + developers + buildbots gdb exploring grammar @@ -343,7 +373,6 @@ Full Table of Contents clang buildslave motivations - gitbootcamp .. _Buildbot status: https://www.python.org/dev/buildbot/ @@ -352,10 +381,15 @@ Full Table of Contents .. _PEPs: https://www.python.org/dev/peps/ .. _python.org maintenance: https://pythondotorg.readthedocs.io/ .. _Python: https://www.python.org/ +.. _CPython: https://www.python.org/ .. _Python Mentors: https://www.python.org/dev/core-mentorship/ .. _PyPy: http://www.pypy.org/ .. _Jython: http://www.jython.org/ .. _IronPython: http://ironpython.net/ .. _Stackless: http://www.stackless.com/ .. _Issue tracker: https://bugs.python.org/ -.. _open pull requests: https://github.com/python/cpython/pulls?utf8=%E2%9C%93&q=is%3Apr%20is%3Aopen%20label%3A%22awaiting%20review%22 +.. _open pull requests: https://github.com/jython/jython/pulls?utf8=%E2%9C%93&q=is%3Apr%20is%3Aopen%20label%3A%22awaiting%20review%22 +.. _Jython issue tracker: https://bugs.jython.org/ +.. _Jython GitHub repository: https://github.com/jython/jython +.. _Java coding standard: https://wiki.python.org/jython/CodingStandards +.. _Style Guide for Java code: http://www.oracle.com/technetwork/java/codeconvtoc-136057.html diff --git a/langchanges.rst b/langchanges.rst index 6deede937c..9ec6cbdbc5 100644 --- a/langchanges.rst +++ b/langchanges.rst @@ -1,7 +1,14 @@ +.. This file is derived from a file of the same name in the CPython devguide + and will receive updates from the CPython guide by merging. + .. _langchanges: Changing the Python Language ============================ + +.. warning:: At present, this is not much modified from the CPython base. + As it is about Python the language, this may be entirely appropriate. + On occasion people come up with an idea on how to change or improve Python as a programming language. This document is meant to explain exactly what changes have a reasonable chance of being considered and what the process is to propose diff --git a/patch_hg_jy.rst b/patch_hg_jy.rst new file mode 100644 index 0000000000..9fca691fe5 --- /dev/null +++ b/patch_hg_jy.rst @@ -0,0 +1,200 @@ +.. Jython and Mercurial-specific companion to pullrequest.rst, + derived from the former CPython devguide file patch.rst + +.. _patch-hg-jy: + +Lifecycle of a Patch (Mercurial) +================================ + +.. note:: + This has been adapted for Jython from an old version of the CPython devguide, + to describe the Jython process based on Mercurial. + We retain this (for the time being) for reference. + +Creating +-------- + +Tool Usage +'''''''''' + +.. _workflow: + +Mercurial allows for various workflows according to each person's or +project's preference. It is out of this guide's scope to present them all, +so we will stick to a basic workflow where you work on a patch in your +working copy without ever making any local commits. + +If you use this workflow, and your work adds or removes files to the +source tree, you will have to temporarily ``hg add`` or ``hg remove`` them, +respectively, before generating a patch. + +To generate a patch, just invoke ``hg diff`` which will print out a +patch of the working copy's changes against the current revision:: + + hg diff > mywork.patch + +If you want to undo your changes, you can revert them from the working copy:: + + hg revert -a + +You can later re-apply the changes if you want to continue working on the +patch:: + + hg import --no-commit mywork.patch + + +.. seealso:: + Refer to the :ref:`FAQ ` for + :ref:`more information on using Mercurial `. + + +Preparation +''''''''''' + +When creating a patch for submission, there are several things that you should +do to help ensure that your patch is accepted. + +First, make sure to follow Python's style guidelines. For Python code you +should follow :PEP:`8`, and for Java code you should follow Jython's +`Java coding standard `_. +If you have one or two discrepancies those can be +fixed by the core developer who commits your patch. But if you have systematic +deviations from the style guides your patch will be put on hold until you fix +the formatting issues. + +Second, be aware of backwards-compatibility considerations. While the core +developer who eventually handles your patch will make the final call on whether +something is acceptable, thinking about backwards-compatibility early +will help prevent having your patch rejected on these grounds. Put yourself in +the shoes of someone whose code will be broken by the change(s) introduced by +the patch. It is quite likely that any change made will break +someone's code, so you need to have a good reason to make a change as you will +be forcing someone to update their code. (This obviously does not apply to new +classes or functions; new arguments should be optional and have default values +which maintain the existing behavior.) If in doubt, have a look at +:PEP:`387` or :ref:`discuss ` the issue with experienced +developers. + +Third, make sure you have proper tests to verify your patch works as expected. +Patches will not be accepted without the proper tests! + +Fourth, make sure the entire test suite :ref:`runs ` **without +failure** because of your changes. It is not sufficient to only run whichever +test seems impacted by your changes, because there might be interferences +unknown to you between your changes and some other part of the interpreter. + +Fifth, proper :ref:`documentation ` +additions/changes should be included. + + +.. _patch-generation-hg-jy: + +Generation +'''''''''' + +.. + XXX: could we adapt this from CPython to Jython? + To perform a quick sanity check on your patch, you can run:: + + make patchcheck + + This will check and/or fix various common things people forget to do for + patches, such as adding any new files needed for the patch to work (note + that not all checks apply to non-core developers). On Windows, use this + command:: + + ./python.exe Tools/scripts/patchcheck.py + +Assuming you are using the :ref:`basic approach ` suggested earlier, +just type the following:: + + hg diff > mywork.patch + +If you are using another approach, you probably need to find out the right +invocation of ``hg diff`` for your purposes; see ``hg help diff`` and ``hg +help revisions``. Just please make sure that you generate a +**single, condensed** patch rather than a series of several changesets. + + +Licensing +--------- + +For non-trivial changes, we must have your formal approval for distributing +your work under the `PSF license`_. Therefore, you need to fill out a +`contributor form`_ which allows the `Python Software Foundation`_ to +license your code for use with Python (you retain the copyright). + +.. note:: + You only have to sign this document once, it will then apply to all + your further contributions to Python. + + +.. _PSF license: http://docs.python.org/3.4/license.html#terms-and-conditions-for-accessing-or-otherwise-using-python +.. _contributor form: http://www.python.org/psf/contrib/ +.. _Python Software Foundation: http://www.python.org/psf/ + + +Submitting +---------- + +If this is a patch in response to a pre-existing issue on the `issue tracker`_, +attach the patch to the issue; use the ``Choose File`` button on the tracker +web page for the issue to upload your patch file. Please provide any details +about your patch that +would be relevant to the discussion of the issue or your patch. + +If this is a patch for an unreported issue (assuming you already performed a +search on the issue tracker for a pre-existing issue), create a new issue and +attach your patch. Please fill in as much relevant detail as possible to +prevent patch reviewers from having to delay reviewing your patch because of +lack of information. + + +.. _issue tracker: http://bugs.jython.org + + +Reviewing +--------- + +To begin with, please be patient! There are many more people submitting patches +than there are people capable of reviewing your patch. Getting your patch +reviewed requires a reviewer to have the spare time and motivation to +look at your patch (we cannot force anyone to review patches). If your patch has +not received any notice from reviewers (i.e., no comment made) after one +month, first "ping" the issue on the `issue tracker`_ to remind the nosy list +that the patch needs a review. If you don't get a response within a few days +after pinging the issue, then you can try emailing jython-dev asking +for someone to review your patch. + +When someone does manage to find the time to look at your patch they will most +likely make comments about how it can be improved (don't worry, even core +developers of Python have their patches sent back to them for changes). It +is then expected that you post a new patch addressing these comments, and the +review process will thus iterate until a satisfactory solution has emerged. + + +Committing/Rejecting +-------------------- + +Once your patch has reached an acceptable state (and thus considered +"accepted"), it will either be committed or rejected. If it is rejected, please +do not take it personally! Your work is still appreciated regardless of whether +your patch is committed. Balancing what *does* and *does not* go into Jython +is tricky and we simply cannot accept everyone's contributions. + +But if your patch is committed it will then go into Jython's +:abbr:`VCS (version control system)` to be released +with the next major release of Jython. It may also be backported to older +versions of Jython as a bugfix if the core developer doing the commit believes +it is warranted. + + +.. _Style Guide for Java code: http://www.oracle.com/technetwork/java/codeconvtoc-136057.html + +Crediting +--------- + +Non-trivial contributions are credited in the ``Misc/ACKS`` file (and, most +often, in a contribution's ``Misc/NEWS`` entry as well). This is something +the core developer will do when committing your patch, you don't have to +propose the addition by yourself. diff --git a/pullrequest.rst b/pullrequest.rst index 62c9a5aa6c..3524a476b2 100644 --- a/pullrequest.rst +++ b/pullrequest.rst @@ -1,16 +1,22 @@ +.. This file is derived from a file of the same name in the CPython devguide + and will receive updates from the CPython guide by merging. + .. _patch: Lifecycle of a Pull Request =========================== +.. warning:: At present, this is not much modified from the CPython base. + This describes the PR process used by CPython on GitHub, + which works for Jython with some adjustment. Introduction ------------ -CPython uses a workflow based on pull requests. What this means is +Jython uses a workflow based on pull requests. What this means is that you create a branch in Git, make your changes, push those changes to your fork on GitHub (``origin``), and then create a pull request against -the official CPython repository (``upstream``). +the official Jython repository (``upstream``). .. _pullrequest-quickguide: @@ -19,9 +25,9 @@ Quick Guide ----------- `Clear communication`_ is key to contributing to any project, especially an -`Open Source`_ project like CPython. +`Open Source`_ project like Jython. -Here is a quick overview of how you can contribute to CPython: +Here is a quick overview of how you can contribute to Jython: #. `Create an issue`_ that describes your change [*]_ @@ -29,7 +35,7 @@ Here is a quick overview of how you can contribute to CPython: #. Work on changes (e.g. fix a bug or add a new feature) -#. :ref:`Run tests ` and ``make patchcheck`` +#. :ref:`Run tests ` and ``make patchcheck`` #. :ref:`Commit ` and :ref:`push ` changes to your GitHub fork @@ -41,14 +47,15 @@ Here is a quick overview of how you can contribute to CPython: #. When your changes are merged, you can :ref:`delete the PR branch ` -#. Celebrate contributing to CPython! :) +#. Celebrate contributing to Jython! :) .. [*] If an issue is trivial (e.g. typo fixes), or if an issue already exists, you can skip this step. .. _Clear communication: https://opensource.guide/how-to-contribute/#how-to-submit-a-contribution .. _Open Source: https://opensource.guide/ -.. _create an issue: https://bugs.python.org/ +.. _create an issue: https://github.com/jython/jython/issues +.. _Jython: https://github.com/jython/jython .. _CPython: https://github.com/python/cpython .. _use HTTPS: https://help.github.com/articles/which-remote-url-should-i-use/ .. _Create Pull Request: https://help.github.com/articles/creating-a-pull-request/ @@ -71,12 +78,11 @@ You should have already :ref:`set up your system `, (Learn more about :ref:`good-prs`) -* Make sure the changes are fine and don't cause any test failure:: +* Make sure the changes don't cause any test failure:: - make patchcheck - ./python -m test + ./python -m test.regrtest -e - (Learn more about :ref:`patchcheck` and about :doc:`runtests`) + (Learn more about :doc:`runtests_jy`) * Once you are satisfied with the changes, add the files and commit them:: @@ -95,9 +101,9 @@ You should have already :ref:`set up your system `, git rebase upstream/master git push --force origin -* Finally go on :samp:`https://github.com/{}/cpython`: you will +* Finally go on :samp:`https://github.com/{}/jython`: you will see a box with the branch you just pushed and a green button that allows - you to create a pull request against the official CPython repository. + you to create a pull request against the official Jython repository. * When people start adding review comments, you can address them by switching to your branch, making more changes, committing them, and pushing them to @@ -129,7 +135,8 @@ When creating a pull request for submission, there are several things that you should do to help ensure that your pull request is accepted. First, make sure to follow Python's style guidelines. For Python code you -should follow :PEP:`8`, and for C code you should follow :PEP:`7`. If you have +should follow :PEP:`8`, and for Java code you should follow +the `Java coding standard`_. If you have one or two discrepancies those can be fixed by the core developer who merges your pull request. But if you have systematic deviations from the style guides your pull request will be put on hold until you fix the formatting issues. @@ -150,7 +157,7 @@ developers. Third, make sure you have proper tests to verify your pull request works as expected. Pull requests will not be accepted without the proper tests! -Fourth, make sure the entire test suite :ref:`runs ` **without +Fourth, make sure the entire test suite :ref:`runs ` **without failure** because of your changes. It is not sufficient to only run whichever test seems impacted by your changes, because there might be interferences unknown to you between your changes and some other part of the interpreter. @@ -158,41 +165,7 @@ unknown to you between your changes and some other part of the interpreter. Fifth, proper :ref:`documentation ` additions/changes should be included. - -.. _patchcheck: - -``patchcheck`` --------------- - -``patchcheck`` is a simple automated patch checklist that guides a developer -through the common patch generation checks. To run ``patchcheck``: - - On *UNIX* (including Mac OS X):: - - make patchcheck - - On *Windows* (after any successful build):: - - python.bat Tools/scripts/patchcheck.py - -The automated patch checklist runs through: - -* Are there any whitespace problems in Python files? - (using ``Tools/scripts/reindent.py``) -* Are there any whitespace problems in C files? -* Are there any whitespace problems in the documentation? - (using ``Tools/scripts/reindent-rst.py``) -* Has the documentation been updated? -* Has the test suite been updated? -* Has an entry under ``Misc/NEWS.d/next`` been added? -* Has ``Misc/ACKS`` been updated? -* Has ``configure`` been regenerated, if necessary? -* Has ``pyconfig.h.in`` been regenerated, if necessary? - -The automated patch check doesn't actually *answer* all of these -questions. Aside from the whitespace checks, the tool is -a memory aid for the various elements that can go into -making a complete patch. +.. _Java coding standard: https://wiki.python.org/jython/CodingStandards .. _good-commits: @@ -210,7 +183,7 @@ and for each pull request there may be several commits. In particular: Commit messages should follow the following structure:: - bpo-42: the spam module is now more spammy. (GH-NNNN) + The spam module is now more spammy. (GH-NNNN) The spam module sporadically came up short on spam. This change raises the amount of spam in the module by making it more spammy. @@ -220,6 +193,8 @@ of what the purpose of the commit is. If this is not enough detail for a commit, a new paragraph(s) can be added to explain in proper depth what has happened (detail should be good enough that a core developer reading the commit message understands the justification for the change). +Brevity (<70 characters) in the first line and a blank one following it +are important to certain tools, including the GitHub format for change sets. Check :ref:`the git bootcamp ` for further instructions on how the commit should look like when merging a pull request. @@ -237,7 +212,7 @@ license your code for use with Python (you retain the copyright). .. note:: You only have to sign this document once, it will then apply to all - your further contributions to Python. + your further contributions to Jython and CPython. Here are the steps needed in order to sign the CLA: @@ -365,7 +340,7 @@ to be useful, a code review has to be perfect. This is not the case at all! It is helpful to just test the pull request and/or play around with the code and leave comments in the pull request or issue tracker. -1. If you have not already done so, get a copy of the CPython repository +1. If you have not already done so, get a copy of the Jython repository by following the :ref:`setup guide `, build it and run the tests. 2. Check the bug tracker to see what steps are necessary to reproduce diff --git a/release_jy.rst b/release_jy.rst new file mode 100644 index 0000000000..a65cb87fcc --- /dev/null +++ b/release_jy.rst @@ -0,0 +1,908 @@ +===================== +How To Release Jython +===================== + +These are the steps needed to make a public release of Jython. +In the case of a public release, +you can only do this once the hard work of development and debugging is done, +and there is a consensus in the project that adequate quality has been achieved. + +These notes will also be a useful guide if you intend to make a snapshot (private) release. +The process may be tested, without making a real release, +for example to check that the scripts still work: +go through the steps but without pushing upstream or publishing. + +Delete the workspace soon afterwards, +so as not to leave change sets tagged as a release, +that might be accidentally built upon or pushed at a later time. + + +Before you start +================ + +Is Jython Ready? +---------------- + +These things are better dealt with in the development environment, +by issues and PRs, +rather than during the release process: + +* Are the JARs we reference or embed sufficiently up to date? + The security vunverabilities (CVEs) reported against Jython come + almost entirely from the JARs we cite or embed. +* Is the Unicode reference data sufficiently up to date? + See ``~/Misc/makeucnhashdat.py`` and ``unchash.dat``. +* ``src/org/python/core/imp.java``: If there has been any compiler change, + increment the magic number ``APIVersion``. + This magic declares old compiled files incompatible, forcing a fresh compilation for users. + (Maybe do it anyway, if it's been a long time.) + +Consider *not* doing some of these actions until a release candidate is imminent. +The way we manage dependencies means the repository history contains +a copy of every version of every JAR we ever used. +This is a lot of space. +``unchash.dat`` is also large. +Refreshing a dependency will seldom impact our own code materially, +but if it will, it is fine to de-risk the change earlier. + + +Local Necessities +----------------- + +To complete a public release you need the following things: + +* Two JDBC driver JARs that we do not track with version control (licensing restrictions): + + * Informix (currently ``jdbc-4.50.11.jar`` for Java 8). + * Oracle (currently ``ojdbc8-23.4.0.24.05.jar`` for Java 8). + +.. Padding. See https://github.com/sphinx-doc/sphinx/issues/2258 + +* Commit rights to the Jython repository (to push the tagged version). +* The right to publish Jython at Sonatype_. +* A PGP signing key pair (generated with ``gpg --gen-key``). +* Access to the channels where we announce releases (e.g. Twitter). +* Access to modify the bug-tracker configuration. + +You can dry-run this process with only the first pre-requisite (driver JARs), +and a Git clone of the official repository. +In that case, be careful not to push any changes. +(Hint: if you clone from ``https://github.com/jython/jython.git``, +that will prevent an unintended push.) + +.. _Sonatype: https://oss.sonatype.org + + +Making a Releasable Jython +========================== + +Start in the Right Place +------------------------ + +``cd`` to a directory where you have permission to make sub-directories. +The path to this directory can get embedded in published files, +so aim for: + +* short (i.e. near a file system root), in the examples ``D:\git``. +* impersonal (not containing company or personal names). +* ASCII (even though Jython is pretty good with Unicode now). + +The examples in this text were mostly made in Windows PowerShell, +but Git remote operations are in Git Bash. + + +Tool Check +---------- + +We must build with the right version of Java. +(At the time of writing we target Java 8.) +At the same time, let's check that we have the tools we need on the path: + +.. code-block:: ps1con + + PS git> java -version + java version "1.8.0_321" + PS git> ant -version + Apache Ant(TM) version 1.10.14 compiled on August 16 2023 + PS git> gpg --version + gpg (GnuPG) 2.3.3 + libgcrypt 1.9.4 + PS git> git --version + git version 2.39.0.windows.2 + + +Clone the Repository +-------------------- + +Clone the repository to a named sub-directory and ``cd`` into it (Bash): + +.. code-block:: bash + + $ git clone git@github.com:jython/jython.git work + Cloning into 'work'... + $ cd work + $ git describe --all + heads/master + +And in Powershell, +the last commits should be the same as in the project repository: + +.. code-block:: ps1con + + PS work> git log --oneline --graph -4 + * d04ff7f62 (HEAD -> master, origin/master, origin/HEAD) Begin to identify as v2.7.4rc2 + * 3562755e5 (tag: v2.7.4rc1) Prepare for 2.7.4rc1 release. + * 245deba51 Now with sensible timeouts. + * 66600ad7e Impose timeout on regrtest workflows + +.. _changes-preparing-for-a-release: + +Changes Preparing for a Release +------------------------------- + +The following files may need to be updated to match the version you are about to release: + +* ``build.xml``: The version number appears piece by piece in the target ``common-config``. + Update these properties: + + * ``jython.major_version``, + * ``jython.minor_version``, + * ``jython.micro_version``, + * ``jython.release_level``, and + * ``jython.release_serial``. + + In the language of these properties, + version 2.7.4 final is spelled ``2``, ``7``, ``4``, ``${PY_RELEASE_LEVEL_FINAL}``, ``0``. + Every other expression needing a version number is derived from these 5 values. +* ``build.gradle``: The version number appears as a simple string property ``version``, + near the top of the file. + Version 2.7.4 is simply set like this: ``version = '2.7.4'``. +* ``README.txt``: It is possible no change is needed at all, + and if a change is needed, it will probably only be to the running text. + A copy of this file is made during the build, + in which information from ``build.xml`` replaces the place-holders. + (The place-holders look like ``@jython.version@``, etc..) + The resulting text is what a user sees when installing interactively. + It automatically includes a prominent banner when making a snapshot build. +* ``NEWS``: First try to ensure we have listed all issues closed since the last release. + The top of this file may look like: + + .. code-block:: text + + Jython Bugs fixed and features added + - [ NNNN ] ... + + Replace the first line with the release you are building + e.g. "Jython 2.7.4". + For a final release, + it will probably say it is the same as the release candidate, + rather than listing bugs fixed. + Add anything necessary to the section "New Features". + After publication (not now), + we will add a new, empty, section for the version then under development. + +These version-settings may already have been made correctly, +to match the identity of the next release. +The build script ensures that, until we actually tag a change set as a release, +the version numbers set here will always appear with a "snapshot" suffix. + +You should run the ``ant javatest`` and ``ant regrtest`` targets at this point. +These should run clean, or at least failures be explained and acceptable, +e.g. known to be attributable to limitations in your network environment. +If bugs are discovered that you need to fix, +it would be best to abandon work on this repository and +fix them in your usual development workbench. + +.. note:: You can run the ``ant bugtest`` target, but it is deprecated. + (We haven't maintained it as Jython changed.) + It produces some failures known to be spurious. + It also creates files you have to clean up manually before you can build for a release. + +If you changed anything, commit this set of changes locally: + +.. code-block:: bash + + $ git add --all + $ git status + On branch master + Your branch is up to date with 'origin/master'. + + Changes to be committed: + (use "git restore --staged ..." to unstage) + modified: NEWS + modified: build.gradle + modified: build.xml + + $ git commit -m"Prepare for 2.7.4 release." + [master 3f256f4a7] Prepare for 2.7.4 release. + 3 files changed, 4 insertions(+), 6 deletions(-) + + +Get the JARs +------------ + +Find the database driver JARs from reputable sources. + +* The Informix driver may be obtained from Maven Central. + Version ``jdbc-4.50.11.jar`` is known to work on Java 8. + +* The Oracle JDBC driver may also be found at Maven Central. + (The Oracle JARs on Maven Central are now official.) + For Java 8 use the ``ojdbc8`` JARs. + +Let's assume we put the JARs in ``D:\git\support``. +Create an ``ant.properties`` correspondingly: + +.. code-block:: properties + + # Ant properties defined externally to the release build. + informix.jar = ../support/jdbc-4.50.11.jar + oracle.jar = ../support/ojdbc8-23.4.0.24.05.jar + +Note that this file is ephemeral and local: +it is ignored by Git because it is named in ``.gitignore``. + + +Check the Configuration of the Build +------------------------------------ + +Run the ``full-check`` target, which does some simple checks on the repository: + +.. code-block:: ps1con + + PS work> ant full-check + Buildfile: D:\git\work\build.xml + + force-snapshot-if-polluted: + [echo] + [echo] Change set 3f256f4a7 is not tagged 'v2.7.4' - build is a snapshot. + + dump: + [echo] --- build Jython version --- + [echo] jython.version.short = '2.7.4' + [echo] jython.release = '2.7.4' + [echo] jython.version = '2.7.4-SNAPSHOT' + [echo] --- optional libraries --- + [echo] informix = '../support/jdbc-4.50.11.jar' + [echo] oracle = '../support/ojdbc8-23.4.0.24.05.jar' + +It makes an extensive dump, +in which lines like those above matter particularly. +See that ``build.xml`` has worked out the version string correctly, +and that it is a snapshot build, +as it must be because you haven't tagged the release yet. +Check that the rest of this dump looks like what you ordered +(version of Java correct?) +and that it ends with ``BUILD SUCCESSFUL``. + +You could do a complete dry-run at this point. +It would create a snapshot build that identifies itself by the version string above. +If you want something other than "SNAPSHOT" as the qualifier, +define the property ``snapshot.name`` on the ``ant`` command line or in ``ant.properties``. + +If you see a message along the lines "Workspace contains uncontrolled files" +then the files listed must be removed (or possibly added to version control) before continuing. +They may be test-droppings or the by-product of your last-minute changes. + + +Tag the Release +--------------- + +Ensure you have committed any outstanding changes (none in this example) +and tag the final state as the release, +being careful to observe the conventional pattern +(there *is* a "v" and there are *two* dots): + +.. code-block:: ps1con + + PS work> git tag -a -s v2.7.4 -m"Jython 2.7.4 final" + +This may open a pop-up from GPG +that requires a password to unlock your signing key +(see `PGP-signing`_). + +Note that ``git tag -a`` creates a sort of commit. +It will need to be pushed eventually, +but the current state of your repository is still at the change set tagged. +If something goes wrong after this point but before the eventual push to the repository, +that requires changes and a fresh commit, +it is possible to delete the tag with ``git tag -d v2.7.4``, +and make it again at the new tip when you're ready. +The Git book explains why you should not `delete a tag after the push`_. + +We follow CPython in signing the tag with GPG as indicated in :pep:`101` +and the `CPython release-tools`_. +See the section :ref:`PGP-signing` for how to generate a key. +(If you are doing a dry-run you can avoid the signing by dropping the `-s` option.) + +As explained in `signing Git commits with GPG`_, +``gpg`` as supplied with *Git for Windows* +and *GnuPG for Windows* disagree about the location of your keys. +In order for signing to work, +it may be necessary to prepare your installation of Git (one time only) +to select the full version of *GnuPG for Windows* as follows. + +.. code-block:: ps1con + + git config --global gpg.program $env:localappdata\gnupg\bin\gpg.exe + + +.. _signing Git commits with GPG: https://jamesmckay.net/2016/02/signing-git-commits-with-gpg-on-windows/ +.. _CPython release-tools: https://github.com/python/release-tools +.. _delete a tag after the push: https://git-scm.com/docs/git-tag#_discussion + + +Ant Build for Release +--------------------- + +Run the ``full-check`` target again: + +.. code-block:: ps1con + + PS work> ant full-check + Buildfile: D:\git\work\build.xml + + [echo] Build is for release of 2.7.4. + + [echo] jython.version = '2.7.4' + +This time the script confirms it is a release +and the version appears without the "SNAPSHOT" qualifier. + +If all remains well with the properties dumped, run the ``full-build`` target. +This outputs the same dump as ``full-check`` and goes on to build the release artifacts. +``build.xml`` does not force a snapshot build on you now +because the source tree is clean and the tag corresponds to the version. + +The artifacts of interest are produced in the ``./dist`` directory and they are: + +#. ``jython.jar`` +#. ``jython-installer.jar`` +#. ``jython-standalone.jar`` +#. ``sources.jar`` +#. ``javadoc.jar`` + +.. note:: At the time of writing, the ``javadoc`` sub-target produces many warnings. + Java 8 is much stricter than Java 7 about correct Javadoc. + These are not fatal to the build: + they are a sign that our documentation is a bit shabby (and always was secretly). + + +Gradle Build for Release +------------------------ + +We can also build a slim JAR (one *not* containing its dependencies) using Gradle. +The Gradle build was released experimentally in Jython 2.7.2. +Now users have a little experience using this JAR for applications, +we consider it a normal part of the build. + +Gradle operates a build entirely parallel to the Ant build, +where everything is regenerated from source, +working in folder ``./build2``. + +.. code-block:: ps1con + + PS work> .\gradlew --console=plain publish + > Task :generateVersionInfo + This build is for v2.7.4. + + > Task :generateGrammarSource + ... + > Task :compileJava + > Task :expose + > Task :mergeExposed + > Task :mergePythonLib + > Task :copyLib + > Task :processResources + > Task :classes + > Task :pycompile + > Task :jar + > Task :generateMetadataFileForMainPublication + > Task :generatePomFileForMainPublication + > Task :javadoc + ... + > Task :javadocJar + > Task :sourcesJar + > Task :publishMainPublicationToStagingRepoRepository + > Task :publish + + BUILD SUCCESSFUL in 6m 41s + 16 actionable tasks: 16 executed + +Don't worry, this doesn't actually *publish* Jython. +When the build finishes, a JAR that is potentially fit to publish, +and its subsidiary artifacts (source, javadoc, checksums), +will have been created in ``./build2/stagingRepo/org/python/jython-slim/2.7.4``. + +It can also be "published" to your local Maven cache (usually ``~/.m2/repository`` +with the task ``publishMainPublicationToMavenLocal``. +This need not be done as part of a release, +but can be useful in verification using a Gradle or Maven build that references it +(see the section :ref:`jython-slim-regrtest`). + +.. _test-what-you-built: + +Test what you built +------------------- + +At this point, take the stand-alone and installer JARs to an empty directory elsewhere, +and try to use them in a new shell session. +In the example, the local directory ``inst`` is chosen as the target in the installer. +Let's use Java 11, different from the version we built with. + +.. code-block:: ps1con + + PS 274-trial> mkdir kit + PS 274-trial> copy "D:\git\work\dist\jython*.jar" .\kit + PS 274-trial> java -jar kit\jython-installer.jar + WARNING: An illegal reflective access operation has occurred + ... + DEPRECATION: A future version of pip will drop support for Python 2.7. + ... + Successfully installed pip-19.1 setuptools-41.0.1 + +It is worth checking the manifests: + +.. code-block:: ps1con + + PS 274-trial> jar -xf .\kit\jython-standalone.jar META-INF + PS 274-trial> cat .\META-INF\MANIFEST.MF + Manifest-Version: 1.0 + Ant-Version: Apache Ant 1.10.14 + Created-By: 1.8.0_321-b07 (Oracle Corporation) + Main-Class: org.python.util.jython + Built-By: Jeff + Automatic-Module-Name: org.python.jython2.standalone + Implementation-Vendor: Python Software Foundation + Implementation-Title: Jython fat jar with stdlib + Implementation-Version: 2.7.4 + + Name: Build-Info + version: 2.7.4 + git-build: true + oracle: true + informix: true + build-compiler: modern + jdk-target-version: 1.8 + debug: true + +And similarly in other JARs ``inst\jython.jar``, ``kit\jython-installer.jar``. + + +Installation ``regrtest`` +^^^^^^^^^^^^^^^^^^^^^^^^^ + +The real test consists in running the regression tests: + +.. code-block:: ps1con + + PS 274-trial> inst\bin\jython -m test.regrtest -e + == 2.7.4 (tags/v2.7.4:3f256f4a7, Aug 18 2024, 10:30:53) + == [Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] + == platform: java11.0.22 + == encodings: stdin=ms936, stdout=ms936, FS=utf-8 + == locale: default=('en_GB', 'windows-1252'), actual=(None, None) + test_grammar + test_opcodes + test_dict + ... + 4 fails unexpected: + test___all__ test_gc_jy test_import_jy test_ssl_jy + +These failures are false alarms. + +* ``test___all__``, ``test_gc_jy`` and ``test_import_jy`` fail, + and others are skipped, + because we (deliberately) do not include certain test resources. +* ``test_ssl_jy`` fails because of `bjo issue 2858`_. +* ``test_sort`` also fails intermittently on later versions of Java. + +.. _bjo issue 2858: https://bugs.jython.org/issue2858 + + +Stand-alone ``regrtest`` +^^^^^^^^^^^^^^^^^^^^^^^^ + +The stand-alone JAR does not include the tests, +but one may run them by supplying a copy of the test modules as below. +The point of copying (only) the test directory to ``TestLib/test``, +rather than putting ``inst/Lib`` on the path, +is to ensure that other modules are tested from the stand-alone JAR itself. +There will be many failures. +When the author last tried, they were these: + +.. code-block:: ps1con + + PS 274-trial> copy -r inst\Lib\test TestLib\test + PS 274-trial> $env:JYTHONPATH = ".\TestLib" + PS 274-trial> java -jar kit\jython-standalone.jar -m test.regrtest -e + == 2.7.4 (tags/v2.7.4:3f256f4a7, Aug 18 2024, 10:30:53) + == [Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] + == platform: java11.0.22 + == encodings: stdin=ms936, stdout=ms936, FS=utf-8 + == locale: default=('en_GB', 'windows-1252'), actual=(None, None) + test_grammar + test_opcodes + ... + test_zlib + test_zlib_jy + 338 tests OK. + 17 tests skipped: + test_codecmaps_hk test_coerce_jy test_curses test_dict2java + test_exceptions_jy test_java_integration test_java_subclasses + test_java_visibility test_jbasic test_joverload test_jy_internals + test_set_jy test_smtpnet test_socketserver test_subprocess + test_urllib2net test_urllibnet + 10 skips unexpected: + test_coerce_jy test_dict2java test_exceptions_jy + test_java_integration test_java_subclasses test_java_visibility + test_jbasic test_joverload test_jy_internals test_set_jy + 33 tests failed: + test_argparse test_classpathimporter test_cmd_line + test_cmd_line_script test_codecs_jy test_compile_jy test_email_jy + test_email_renamed test_gc_jy test_httpservers test_import + test_import_jy test_json test_jython_initializer + test_jython_launcher test_lib2to3 test_linecache test_marshal + test_os_jy test_pdb test_platform test_popen test_quopri test_repr + test_site test_site_jy test_ssl_jy test_sys test_sys_jy + test_threading test_urllib2 test_warnings test_zipimport_support + 33 fails unexpected: + test_argparse test_classpathimporter test_cmd_line + test_cmd_line_script test_codecs_jy test_compile_jy test_email_jy + test_email_renamed test_gc_jy test_httpservers test_import + test_import_jy test_json test_jython_initializer + test_jython_launcher test_lib2to3 test_linecache test_marshal + test_os_jy test_pdb test_platform test_popen test_quopri test_repr + test_site test_site_jy test_ssl_jy test_sys test_sys_jy + test_threading test_urllib2 test_warnings test_zipimport_support + +Most of these failures are in tests that assume +the library is a real file system. +Others arise because we do not include certain JARs needed for the test. +It is necessary to pick through the failures carefully +to detect which are real. + +.. note:: We could probably do this better through skips in the tests, + sensitive to running stand-alone, + or (widely useful) a broader interpretation of "file path" in Jython, + reflecting the importance of the JAR file system in Java. + + We should do this occasionally, and not just when trying to release. + Some of the failures are genuine problems, + by chance revealed only in the stand-alone version. + + +.. _jython-slim-regrtest: + +Slim (Gradle) ``regrtest`` +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +There is not currently a pre-prepared way to test +the Gradle-built JAR (``jython-slim``), +but it is not difficult to create something. +For this, it is necessary to publish to a local repository, +such as your personal Maven cache: + +.. code-block:: ps1con + + PS work> .\gradlew --console=plain publishMainPublicationToMavenLocal + +This will deliver build artifacts to +``~/.m2/repository/org/python/jython-slim/2.7.4``. +One can construct an application to run with that as a dependency like this: + +.. code-block:: groovy + + // Application importing the jython-slim JAR. + plugins { + id 'application' + } + + repositories { + mavenLocal() + mavenCentral() + } + + dependencies { + implementation 'org.python:jython-slim:2.7.4' + } + + application { + mainClass = 'uk.co.farowl.jython.slimdemo.RegressionTest' + } + + +The following executes ``test.regrtest`` +using the same local copy of the tests +prepared for the stand-alone Jython. + +.. code-block:: java + + package uk.co.farowl.jython.slimdemo; + import org.python.util.PythonInterpreter; + public class RegressionTest { + public static void main(String[] args) { + try (PythonInterpreter interp = new PythonInterpreter()) { + interp.exec("import sys, os"); + interp.exec("sys.path[0] = os.sep.join(['.', 'TestLib'])"); + interp.exec("sys.argv[1:] = ['-e']"); + interp.exec("from test import regrtest as rt"); + interp.exec("rt.main()"); + } + } + } + +Tests have about the same success rate as for the stand-alone Jython JAR. +Notably ``test_ssl_jy`` passes here because a genuine (not wrapped) +Bouncy Castle JAR is on the path. + +Tests end with a failure status under Gradle, even when all tests pass, +because ``regrtest`` calls ``sys.exit``, +which raises ``SystemExit``. +It looks like: + +.. code-block:: text + + 333 tests OK. + ... + 33 tests failed: + ... + Exception in thread "MainThread" Traceback (most recent call last): + File "", line 1, in + File ".\TestLib\test\regrtest.py", line 521, in main + sys.exit(surprises > 0) + SystemExit: True + +One could improve the driver program, but it is complicated to do properly. + + +Build the Bundles to Publish +---------------------------- + +The artifacts for Maven are built using a separate script ``maven/build.xml``. + +.. code-block:: text + + PS work> ant -f maven\build.xml + Buildfile: D:\git\work\maven\build.xml + ... + validate-template-pom: + [xmlvalidate] 1 file(s) have been successfully validated. + ... + BUILD SUCCESSFUL + Total time: 2 minutes 27 seconds + +During the build, ``gpg`` may prompt you (in a dialogue box) +for the pass-phrase that protects your private signing key. +This leaves the following new artifacts in ``./publications``: + +* ``jython-2.7.4-bundle.jar`` +* ``jython-standalone-2.7.4-bundle.jar`` +* ``jython-installer-2.7.4-bundle.jar`` +* ``jython-slim-2.7.4-bundle.jar`` + + +Publication +=========== + +Account +------- + +In order to publish the bundles created in ``./publications``, +it is necessary to have an account with access to ``groupId`` ``org.python``, +which Sonatype will grant given the support of an existing owner. +(This is a human process administered through JIRA.) +There is an extensive `Sonatype OSSRH Guide`_ +about getting and using an account. + +.. _PGP-signing: + +PGP Signing +----------- + +You need a PGP signing key pair (generated with ``gpg --gen-key``) +on the computer where you are working. +This must be published through the pool of PGP key servers +for Sonatype to pick up, +and so reassure users that +this release of Jython is really from the project. + +The infrastructure of PGP has been overhauled +since the previous version of these notes was written. +Follow the Sonatype guide `Working with PGP Signatures`_, +which now appears to have been updated with the changes. + +.. code-block:: text + + PS work> gpg --list-secret-keys + C:\Users\Jeff\AppData\Roaming\gnupg\pubring.kbx + ----------------------------------------------- + sec rsa2048 2019-10-20 [SC] [expires: 2028-02-26] + C8C4B9DC1E031F788B12882B875C3EF9DC4638E3 + uid [ultimate] Jeff Allen + ssb rsa2048 2019-10-20 [E] [expires: 2028-02-26] + +The `OpenPGP key server`_ provides an interface to query +a PGP public key. +PGP servers form a pool. +It may take a few hours for your key to wash up at the machine +Sonatype consults. + +Generation and publication of a key are one-time actions, +except that the key has a finite lifetime with possible extensions. +(The key here has been extended twice.) +See `Working with PGP Signatures`_ for how to extend the life of a key. + +.. note:: You may decide to create a new key for signing future releases. + The key that was used to sign past releases should remain valid + so that users can still validate those past releases. + Renewing an old key is a valid and useful thing to do. + (An exception might occur when the old *private* key is thought + to have been lost.) + +.. _Sonatype OSSRH Guide: https://central.sonatype.org/pages/ossrh-guide.html +.. _Working with PGP Signatures: https://central.sonatype.org/publish/requirements/gpg/ +.. _OpenPGP key server: https://keys.openpgp.org + + +Publication via Sonatype +------------------------ + +You are now ready to upload bundles acceptable to Sonatype. + +* Go to the Sonatype_ repository manager and log in. +* Under "Build Promotion" select "Staging Upload". +* On the "Staging Upload" tab, and the Upload Mode drop-down, + select "Artifact Bundle". +* Navigate to the ``./publications`` folder and upload in turn: + + * ``jython-slim-2.7.4-bundle.jar`` + * ``jython-2.7.4-bundle.jar`` + * ``jython-standalone-2.7.4-bundle.jar`` + * ``jython-installer-2.7.4-bundle.jar`` + + For some reason (privacy?) the display shows a fake file path + but the name is correct. + Each upload creates a "staging repository". + +.. note:: You may get a report (e-mail) from Sonatype Lift at this point + reporting potential vulnerabilities in dependencies. + (It seems only to work on the ``-slim`` JAR, which is why we upload it first.) + If any vulnerability is sufficiently serious to warrant upgrading JARs, + treat this as a late test failure: + fix it in your normal development environment with a PR and repeat the process. + Assuming you have deferred pushing the tag no publicly visible harm has been done. + (See :ref:`jython-push-with-tag` below.) + If you already pushed the tag, + repeat the release process with an appropriate increment on the version number. + +You may discard (drop) Repositories that you decide not to publish +from the "Staging Repositories" tab in the repository manager. + +* Under "Build Promotion" select the "Staging Repositories" tab. +* Check (on the "Activity" tab) + that the upload reached "Close" with good status, + If not, it should tell you what is lacking and you have to go back and fix it. +* In a fresh directory, + download the (as yet unreleased) artifacts from Sonatype and test them, + repeating the section :ref:`test-what-you-built`. + A staging URL has form: + ``https://oss.sonatype.org/content/repositories/orgpython-1105`` + where the final number increments with each upload. +* When you are absolutely satisfied ... "Release" the bundles. + This will cause them to appear in the Maven `Central Repository`_ + (takes an hour or two). + +.. warning:: Release at Sonatype is irreversible. + +.. _Central Repository: https://search.maven.org/ + +.. _jython-push-with-tag: + +Only now is it safe to ``git push`` +----------------------------------- + +If testing convinces you this is a build we should let loose +on an unsuspecting public, +it is time to push these changes and the tag you made +upstream to the Jython repository. +Back in the place where the release was built (and using Bash): + +.. code-block:: bash + + $ git push --follow-tags + +Try very hard not to push a tag you later regret +(e.g. on the wrong change set or a version still needing a fix). +It is problematic to `delete a tag after the push`_. +It is better to increment the version, +which is painless if it is an ``a``, ``b``, or ``rc`` release. + + +Announcement +------------ + +.. note:: This section is slightly modified from Frank's notes, + untested since recent changes. + +* update files in (or make a PR against) the `website repository`_ + that reference the current release: + + * Add to the `website news page`_ (``news.md``) + * Ensure links on the `website front page`_ (``index.md``) + and `website download page`_ (``download.md``) reflect: + + * the latest stable release + * the current alpha, beta, or candidate release (if any to be advertised) + + Exactly what you do here will depend on the kind of release you just made. + +* change the ``#jython`` irc channel topic +* announce on twitter (as jython), irc channel, mailing lists, blog ... +* In the bug tracker: + + * add the new version, against which to report bugs. + * add a new milestone (future version), against which to plan delivery. + +.. _website repository: https://github.com/jython/jython.github.io +.. _website front page: https://www.jython.org/index +.. _website news page: https://www.jython.org/news +.. _website download page: https://www.jython.org/download + + + +Ready for new work +================== + +After a release, +Jython in the development environment +should no longer identify itself as the version just released, so we increment the version string. +We do not know for sure the version next to be publicly released, +so we use the smallest increment that results in a valid version number. + +After an alpha, beta or release candidate, +assume the successor version to be a one-up serial of the *same* release level, +incrementing ``jython.release_serial``. +After a final release, +assume the successor to be an alpha of the next micro-release. +For example, ``2.7.2b2`` is followed by ``2.7.2b3``, +and ``2.7.2`` by ``2.7.3a1``. + +If the version under development is ostensibly ``2.7.4b3``, +the build system will label the code as ``2.7.4b3-DEV`` in builds. +If you build an installer, or dry-run a release, it will be ``2.7.4b3-SNAPSHOT``. +You can read this as a version that "may eventually become" ``2.7.4b3`` etc.. + +The version under development in this scheme will often be one that never sees a release. +E.g. when we are apparently working on ``2.7.4b3``, +the next release is quite likely to be ``2.7.4rc1`` instead. +It's a harmless idiosyncrasy of the process that +the version may only be chosen accurately when the time comes to release it. + +Make this change in both ``build.xml`` and ``build.gradle``. +See the section :ref:`changes-preparing-for-a-release` for details. + +In ``NEWS``, add a new, empty, section in the development history that looks like this: + +.. code-block:: text + + Jython Bugs fixed + +If you've just built a final release, the new material will look like: + +.. code-block:: text + + ============================================================================== + Jython + ============================================================================== + + New Features + + Jython a1 Bugs fixed + + +Commit and push this change upstream. + +.. note:: The description of a new feature is associated with + the prospective final release, + not the alpha or beta that introduced it. diff --git a/runtests_jy.rst b/runtests_jy.rst new file mode 100644 index 0000000000..4583075c9f --- /dev/null +++ b/runtests_jy.rst @@ -0,0 +1,100 @@ +.. Jython companion to runtests.rst + +.. _runtests-jy: + +Running & Writing Tests (Jython) +================================ + +.. highlight:: console + +.. note:: + + This document assumes you are working from an + :ref:`in-development ` checkout of Jython. If you + are not then some things presented here may not work as they may depend + on new features not available in earlier versions of Jython. + + The commands are given for Jython 2.7, and may be different in Jython 3. + +Running +------- + +The shortest, simplest way of running the test suite is the following command +from the root directory of your checkout:: + + $ ant regrtest + +This will run the majority of tests, but exclude a small portion of them; +these excluded tests are known failures on the current platform or +use special kinds of resources: for example, accessing the +Internet, or trying to play a sound or to display a graphical interface on +your desktop. They are disabled by default so that running the test suite +is not too intrusive. To enable some of these additional tests (and for +other flags which can help debug various issues such as reference leaks), read +the help text:: + + $ dist/bin/jython -m test.regrtest -h + +If you want to run a single test file, simply specify the test file name +(without the extension) as an argument. You also probably want to enable +verbose mode (using ``-v``), so that individual failures are detailed:: + + $ dist/bin/jython -m test.regrtest -v test_abc + +This form may be used to run multiple tests by name, +and in a the order specified, or even repeat to tests:: + + $ dist/bin/jython -m test.regrtest test_abc test_int test_abc + +To run a single test case, use the ``unittest`` module, providing the import +path to the test case:: + + $ dist/bin/jython -m unittest -v test.test_abc.TestABC + + +Unexpected Skips +^^^^^^^^^^^^^^^^ + +Sometimes when running the test suite, you will see "unexpected skips" +reported. These represent cases where an entire test module has been +skipped, but the test suite normally expects the tests in that module to +be executed on that platform. + +Often, the cause is that an optional module hasn't been built due to missing +build dependencies. In these cases, the missing module reported when the test +is skipped should match one of the modules reported as failing to build when +:ref:`compiling-jy`. + +In other cases, the skip message should provide enough detail to help figure +out and resolve the cause of the problem (for example, the default security +settings on some platforms will disallow some tests) + + +Writing +------- + +Writing tests for Python is much like writing tests for your own code. Tests +need to be thorough, fast, isolated, consistently repeatable, and as simple as +possible. We try to have tests both for normal behaviour and for error +conditions. Tests live in the ``Lib/test`` directory, where every file that +includes tests has a ``test_`` prefix. + +One difference with ordinary testing is that you are encouraged to rely on the +:py:mod:`test.support` module. It contains various helpers that are tailored to +Python's test suite and help smooth out common problems such as platform +differences, resource consumption and cleanup, or warnings management. +That module is not suitable for use outside of the standard library. + +When you are adding tests to an existing test file, it is also recommended +that you study the other tests in that file; it will teach you which precautions +you have to take to make your tests robust and portable. + + +Benchmarks +---------- +Benchmarking is useful to test that a change does not degrade performance. + +`The Python Benchmark Suite `_ +has a collection of benchmarks for all Python implementations. Documentation +about running the benchmarks is in the `README.txt +`_ of the repo. diff --git a/setup_jy.rst b/setup_jy.rst new file mode 100644 index 0000000000..57402cb982 --- /dev/null +++ b/setup_jy.rst @@ -0,0 +1,285 @@ +.. Jython companion to setup.rst + +=============== +Getting Started +=============== + +.. highlight:: console + +These instructions cover how to get a working copy of the source code and a +compiled version of the Jython interpreter. (Jython is the version of Python +available from http://www.jython.org.) It also gives an overview of the +directory structure of the Jython source code. + +OpenHatch has a partly-relevant `setup guide`_ for CPython for people who are +completely new to contributing to open source. + +.. _setup guide: http://wiki.openhatch.org/Contributing_to_Python + + +.. _setup-jy: + +Getting Set Up +============== + + +.. _vcsetup-jy: + +Version Control Setup (Mercurial) +--------------------------------- + +Jython is developed using `Mercurial `_. The Mercurial +command line program is named ``hg``; this is also used to refer to Mercurial +itself. Mercurial is easily available for common Unix systems by way of the +standard package manager; under Windows, you might want to use the +`TortoiseHg `_ graphical client. + +Version Control Setup (GitHub) +------------------------------ + +Jython will be developed using `git `_. The git +command line program is named ``git``. +git is easily available for all common operating systems. As the +Jython repo will be hosted on GitHub, please refer to either the +`GitHub setup instructions `_ +or the `git project instructions `_ for step-by-step +installation directions. You may also want to consider a graphical client +such as `TortoiseGit `_ or +`GitHub Desktop `_. + +You may also wish to set up :ref:`your name and email ` +and `an SSH key +`_ +that will allow you to interact with GitHub without typing a username +and password each time you execute a command, such as ``git pull``, +``git push``, or ``git fetch``. On Windows, you should also +:ref:`enable autocrlf `. + +.. _checkout-jy: + +Getting the Source Code +----------------------- + +Mercurial +^^^^^^^^^ +One should always work from a working copy of the Jython source code. +While it may +be tempting to work from the copy of Jython you already have installed on your +machine, it is very likely that you will be working from out-of-date code as +the Jython core developers are constantly updating and fixing things in their +:abbr:`VCS (version control system)`. It also means you will have better tool +support through the VCS as it will provide a diff tool, etc. + +To get a working copy of the :ref:`in-development ` branch of +Jython (core developers use a different URL as outlined in :ref:`coredev`), +run:: + + $ hg clone http://hg.python.org/jython + +If you want a working copy of an already-released version of Jython, +i.e., a version in :ref:`maintenance mode `, you can update your +working copy. For instance, to update your working copy to Jython 2.5, do:: + + $ hg update 2.5 + +GitHub +^^^^^^ +In order to get a copy of the source code you should first :ref:`fork the +Jython repository on GitHub ` and then :ref:`create a local +clone of your private fork and configure the remotes `. + +If you want a working copy of an already-released version of Python, +i.e., a version in :ref:`maintenance mode `, you can checkout +a release branch. For instance, to checkout a working copy of Jython 2.5, +do ``git checkout 2.5``. + +You will need to re-compile Jython when you do such an update. + +.. _compiling-jy: + +Compiling +--------- + +Compiling Jython is fairly simple, from the top level of a source checkout do:: + + $ ant + +Each time you issue this command, ``ant`` builds incrementally, +by compiling those Java source files that have changed, +and copying the Python and other files that have changed +to the :file:`dist` directory. +Several other useful targets may be named to ``ant``, +in particular, ``ant clean`` will delete the :file:`dist` and :file:`build` +directories so that a subsequent plain ``ant`` will rebuild everything. +The command:: + + $ ant -p + +lists the top-level targets. + +.. _build-dependencies-jy: + +Build dependencies +^^^^^^^^^^^^^^^^^^ + +The core Jython interpreter depends on a number of jars that are found in +directory ``extlibs``, and enumerated in ``build.xml``. + +Running +^^^^^^^ + +Once Jython is done building you will then have a working build +that can be run in-place from ``./dist/bin/jython``. +There is normally no need to install your built copy of Jython. +The interpreter will adapt to the environment it is being run from +and use the files found in the working copy. + +If you edit Jython's source code in your working copy, +changes to Python code will be picked up by the interpreter for immediate +use and testing. (If you change Java code, you will need to recompile the +affected files as described above.) + +.. _issue tracker: http://bugs.jython.org + +Editors and Tools +================= + +Python is used widely enough that practically all code editors have some form +of support for writing Python code. Various coding tools also include Python +support. + +Jython specific support is less common but supported in several IDEs. Many of +the core developers do pretty well with Emacs or Vim :) + +TODO: Eclipse + +TODO: Netbeans + +For editors and tools which the core developers have felt some special comment +is needed for coding *in* Python, see :ref:`resources`. + + +Directory Structure +=================== + +There are several top-level directories in the Jython source tree. Knowing what +each one is meant to hold will help you find where a certain piece of +functionality is implemented. Do realize, though, there are always exceptions to +every rule. + +``Demo`` + Outdated Jython demo code. + + TODO: fix this to make it current. + +``Doc`` + Outdated Jython website docs. + + TODO: Find an approach to the documentation given that most of it is + exactly as produced by CPython and some of it must be totally different. + +``lib-python`` + A recent copy of the Python standard library for the target version of + Python, taken from the reference implementation CPython. + +``Lib`` + The parts of the standard library we cannot simply adopt from CPython. + These supersede or add to the modules in ``lib-python``. + +``Misc`` + Things that do not belong elsewhere. Typically this is varying kinds of + developer-specific tools and documentation. + +``ast`` + Code related to the parser and the definition of the AST nodes. + + TODO: It may be a good idea to rename this directory "Parser" to match + CPython. + +``bugtests`` + Outdated framework for testing Jython. + + TODO: tests that remain here should be ported to Lib/tests/ + +``extlibs`` + External Java dependencies for Jython. + +``grammar`` + ANTLR source for building Jython's grammar. + +``maven`` + Source for including Jython in Maven. + +``src`` + The part of the Jython interpreter that is implemented in Java. + +``tests`` + Tests for Jython implemented in Java. + +``Tools`` + Various tools that are (or have been) used to maintain Jython. + + + +Manually regenerated files +========================== + +Some files are programmatically generated, but not by ``ant``, +nor destroyed by ``ant clean``. +These must be regenerated by the developer when necessary, +and new versions checked-in like source files. + +Derived Java source +------------------- + +Some Java source that supports subclassing of built-in types +is generated using Python scripts. +These files need to be refreshed only when the signatures of exposed methods +of the corresponding types change. +A new one must be created when a new type is added. +Notes about this are currently on the Jython Wiki at +`Generating the *Derived classes `_. + +The launcher ``jython.exe`` +--------------------------- + +.. highlight:: ps1con + +:file:`src/shell/jython.exe` is the Windows Jython launcher. +It is copied during the ``ant`` build to :file:`dist/bin`. +However, it is derived from :file:`src/shell/jython.py` using PyInstaller_ +by the following process. + +If it is not already installed, install ``virtualenv`` +with the command ``pip install virtualenv``. +In any convenient working directory, create a virtual environment, activate it, +and install ``PyInstaller``:: + + > virtualenv venv + New python executable in ... venv\Scripts\python.exe + Installing setuptools, pip, wheel...done. + > .\venv\Scripts\activate + (venv) > pip install pyinstaller + Collecting pyinstaller + ... + Installing collected packages: future, pypiwin32, ..., pyinstaller + Successfully installed ... pyinstaller-3.3 + +The above set-up need only be performed once. + +Copy :file:`src/shell/jython.py` to this working directory. +Use ``PyInstaller`` to create a single-file executable, +and copy that back to :file:`src/shell`:: + + (venv) > copy \src\shell\jython.py . + (venv) > pyinstaller --onefile jython.py + ... + (venv) > copy .\dist\jython.exe \src\shell + +Above, ```` stands for the root directory of the Jython source. +You *could* do all this in the source tree at :file:`src/shell`, +but the virtual environment and ``PyInstaller`` leave a lot of +working material behind, so it is best done elsewhere. + +.. _PyInstaller: http://www.pyinstaller.org + diff --git a/silencewarnings.rst b/silencewarnings.rst index 97f4a36b35..0afed885a9 100644 --- a/silencewarnings.rst +++ b/silencewarnings.rst @@ -1,8 +1,13 @@ +.. This file is derived from a file of the same name in the CPython devguide + and will receive updates from the CPython guide by merging. + .. _silencewarnings: Silence Warnings From the Test Suite ==================================== +.. warning:: At present, this is not much modified from the CPython base. + When running Python's test suite, no warnings should result when you run it under :ref:`strenuous testing conditions ` (you can ignore the extra flags passed to ``test`` that cause randomness and parallel execution diff --git a/stdlibchanges.rst b/stdlibchanges.rst index 1f7ed4848c..4e74a492f5 100644 --- a/stdlibchanges.rst +++ b/stdlibchanges.rst @@ -1,8 +1,13 @@ +.. This file is derived from a file of the same name in the CPython devguide + and will receive updates from the CPython guide by merging. + .. _stdlibchanges: Adding to the Stdlib ==================== +.. warning:: At present, this is not much modified from the CPython base. + While the stdlib contains a great amount of useful code, sometimes you want more than is provided. This document is meant to explain how you can get either a new addition to a pre-existing module in the stdlib or add an entirely new diff --git a/tracker.rst b/tracker.rst index eeedf5223a..8e7c6c8309 100644 --- a/tracker.rst +++ b/tracker.rst @@ -1,7 +1,17 @@ +.. This file is derived from a file of the same name in the CPython devguide + and will receive updates from the CPython guide by merging. + ============== Issue Tracking ============== +.. warning:: + Thes Jython tracker is (slowly) undergoing an upgrade, after which most + of the following will apply. Until then some of the features mentioned + below are missing as this is a slightly modified copy from + `CPython devguide `_ that needs to + be fully evaluated. + .. _tracker: Using the Issue Tracker @@ -40,7 +50,7 @@ Being now logged in, you can submit a bug by clicking on the "Create New" link in the sidebar. The submission form has a number of fields, and they are described in detail -in the :ref:`triaging` page. This is a short summary: +in the :ref:`triaging-jy` page. This is a short summary: * in the **Title** field, enter a *very* short description of the problem; less than ten words is good; @@ -212,7 +222,7 @@ instructions on the `Tracker Development`_ page. Discussions about the bug tracker. -.. _issue tracker: https://bugs.python.org/ +.. _issue tracker: http://bugs.jython.org/ .. _meta tracker: http://psf.upfronthosting.co.za/roundup/meta/ -.. _advanced search: https://bugs.python.org/issue?@template=search -.. _Tracker Development: https://wiki.python.org/moin/TrackerDevelopment +.. _advanced search: http://bugs.jython.org/issue?@template=search +.. _Tracker Development: http://wiki.python.org/moin/TrackerDevelopment diff --git a/triaging_jy.rst b/triaging_jy.rst new file mode 100644 index 0000000000..a2eb074985 --- /dev/null +++ b/triaging_jy.rst @@ -0,0 +1,338 @@ +.. Jython companion to triaging.rst + +.. _triaging-jy: + +Triaging a Jython Issue +======================= + +.. warning:: At present, this is not much more than a copy of the CPython original. + +When you have the Developer role on the `issue tracker`_ you are able to triage +issues directly without any assistance. + +Fields +------ + +Title +''''' +Should be properly descriptive of what the issue is about. Occasionally +people file an issue that either has too generic of a title or end up thinking +they filed about X but in fact it turns out to be about Y and thus the +title is now wrong. + +Type +'''' + +.. warning:: Old material. Jython tracker has different categories. + +Describes the type of issue. If something does not fit within any +specific type then simply do not set it. + +behavior + Wrong or unexpected behavior, result, or exception. This includes most of + the bugs. +crash + Hard crashes of the Python interpreter -- possibly with a core + dump or a Windows error box. +compile error + Errors reported by the compiler while compiling Python. +resource usage + Situations where too many resources (e.g. memory) are used. +security + Issues that might have security implications. If you think the issue + should not be made public, please report it to security@python.org instead. +performance + Situations where too much time is necessary to complete the task. +enhancement + Issues that propose the addition of new functionality, such as new + functions, classes, modules, or even new arguments for existing functions. + Also used for improvements in the documentation and test suite and for + other refactorings. + +Stage +''''' +.. warning:: Old material. Jython tracker has different categories. + +What is needed next to advance the issue. The *stage* needn't be set until +it is clear that the issue warrants fixing. + +test needed + The bug reporter should post a script or instructions to let a triager or + developer reproduce the issue. +needs patch + The issue lacks a patch to solve the problem (i.e. fixing the bug, or + adding the requested improvement). +patch review + There is a patch, but it needs reviewing or is in the process of being + reviewed. This can be done by any triager as well as a core developer. +commit review + A triager performed a patch review and it looks good to them, but a core + developer needs to commit the patch (and do a quick once-over to make sure + nothing was overlooked). +resolved + The issue is considered closed and dealt with. + +Components +'''''''''' +.. warning:: Old material. Jython tracker has different categories. + +What part of Python is affected by the issue. This is a multi-select field. +Be aware that what component is chosen may cause the issue to be auto-assigned, +i.e. the issue tracker may automatically fill in the `Assigned To`_ field +after you press ``Submit changes``. + +The following component(s) should be selected if the issue applies to: + +2to3 (2.x to 3.0 conversion tool) + The 2to3 conversion tool in `Lib/lib2to3`_. +Build + The build process. +ctypes + The ctypes package in `Lib/ctypes`_. +Demos and Tools + The files in Tools_ and `Tools/demo`_. +Distutils + The distutils package in `Lib/distutils`_. +Documentation + The documentation in Doc_ (used to build the HTML doc at https://docs.python.org/). +email + The email package and related modules. +Extension Modules + C modules in Modules_. +IDLE + The `Lib/idlelib`_ package. +Installation + The installation process. +Interpreter Core + The interpreter core, the built-in objects in `Objects`_, the `Python`_, + `Grammar`_ and `Parser`_ dirs. +IO + The I/O system, `Lib/io.py`_ and `Modules/_io`_. +Library (Lib) + Python modules in Lib_. +Macintosh + The Mac OS X operating system. +Regular Expressions + The `Lib/re.py`_ and `Modules/_sre.c`_ modules. +Tests + The unittest and doctest frameworks in `Lib/unittest`_ and + `Lib/doctest.py`_. + + The CPython tests in `Lib/test`_, the test runner in `Lib/test/regrtest.py`_ + and the `Lib/test/support`_ package. +Tkinter + The `Lib/tkinter`_ package. +Unicode + Unicode, codecs, str vs bytes, `Objects/unicodeobject.c`_. +Windows + The Windows operating system. +XML + The `Lib/xml`_ package. + +Versions +'''''''' +The known versions of Python that the issue affects and should be fixed for. +Thus if an issue for a new feature is assigned for e.g., Python 3.3 but is not +applied before Python 3.3.0 is released, this field should be updated to say +Python 3.4 as the version and drop Python 3.3. + +Priority +'''''''' +.. warning:: Old material. Jython tracker has different categories. + +How important is this issue? + +low + This is for low-impact bugs, or feature requests of little utility. +normal + The default value for most issues, which deserve fixing but without + any urgency to do so. +high + Make some effort to fix the issue before the next final release. +critical + This issue should definitely be fixed before the next final release. +deferred blocker + The issue will not hold up the next release, but will be promoted to a + release blocker for the following release, e.g., won't block the next + release of a1 but will block a2. +release blocker + The issue must be fixed before *any* release is made, e.g., will block the + next release even if it is an alpha release. + +As a guideline, *critical* and above are usually reserved for crashes, +serious regressions or breakage of very important APIs. Whether a bug +is a *release blocker* is a decision better left to the release manager so, +in any doubt, add him or her to the *nosy list*. + +Keywords +'''''''' +Various flags about the issue. Multiple values are possible. + +.. warning:: Old material. Jython tracker has different categories. + +buildbot + A buildbot triggered the issue being reported. +easy + Fixing the issue should not take longer than a day for someone new to + contributing to Python to solve. +gsoc + The issue would fit as, or is related to, a GSoC_ project. +needs review + The patch attached to the issue is in need of a review. +patch + There is a patch attached to the issue. +3.3regression + The issue is a regression in 3.3. + +Nosy List +''''''''' +A list of people who may be interested in an issue. It is acceptable to add +someone to the nosy list if you think the issue should be brought to their +attention. Use the :ref:`experts` to know who wants to be added to the nosy +list for issues targeting specific areas. + + +If you are logged in and have JavaScript enabled, you can use the ``[+]`` +button to add yourself to the nosy list (remember to click on +"Submit Changes" afterwards). Note that you are added to the nosy +automatically when you submit a message. +The nosy list also has an autocomplete that lets you search from the lists of +developers. The search is case-insensitive and +works for real names, modules, interest areas, etc., and only adds the +username(s) to the nosy once an entry is selected. + +Assigned To +''''''''''' +Who is expected to take the next step in resolving the issue. It is acceptable +to assign an issue to someone if the issue cannot move forward without their +help, e.g., they need to make a technical decision to allow the issue to move +forward. Also consult the :ref:`experts` as certain stdlib modules should +always be assigned to a specific person. + +Dependencies +'''''''''''' +The issue requires the listed issue(s) to be resolved first before it can move +forward. + +Superseder +'''''''''' +The issue is a duplicate of the listed issue(s). + +Status +'''''' +.. warning:: Old material. Jython tracker has different categories. + +open + Issue is not resolved. +languishing + The issue has no clear solution , e.g., no agreement on a technical + solution or if it is even a problem worth fixing. +pending + The issue is blocked until someone (often the + :abbr:`OP (original poster)`) provides some critical information; + the issue will be closed after a set amount of time if no reply comes in. + Useful when someone opens an issue that lacks enough information to + reproduce the bug reported. Requesting additional information and setting + status to *pending* indicates that the issue should be closed if the + necessary information is never provided. +closed + The issue has been resolved (somehow). + +Resolution +'''''''''' +Why the issue is in its current state (not usually used for "open"). + +duplicate + Duplicate of another issue; should have the Superseder field filled out. +fixed + A fix for the issue was committed. +later + Issue is to be worked on at a later date. +not a bug + For some reason the issue is invalid (e.g. the perceived problem is not + a bug in Python). +out of date + The issue has already been fixed, or the problem doesn't exist anymore + for other reasons. +postponed + Issue will not be worked on at the moment. +rejected + Issue was rejected (especially for feature requests). +remind + The issue is acting as a reminder for someone. +wont fix + Issue will not be fixed, typically because it would cause a + backwards-compatibility problem. +works for me + Bug cannot be reproduced. + +Mercurial Repository +'''''''''''''''''''' + + +Generating Special Links in a Comment +------------------------------------- + +.. warning:: This is how CPython's automation works. + Jython needs something similar but doesn't have it yet. + + +Comments can automatically generate a link to various web pages if formatted +properly. + +* ``#``, ``issue``, or ``issue `` links to the + tracker issue ````. +* ``msg`` links to the tracker message ````. +* ``PR ``, ``PR``, and ``pull request `` can be used + to link to `GitHub pull requests `_. +* a 10-, 11-, 12-, or 40-digit hex ```` is assumed to be a Git or + Mercurial changeset identifier and generates a link to changeset ```` + on GitHub or https://hg.python.org/. The ``git`` and ``hg`` prefixes can + also be used to disambiguate, and must precede the number without spaces. +* ``r``, ``rev``, or ``revision `` is assumed to be + a legacy Subversion revision number, a reference to a changeset that was + checked in prior to 2011-03-05 when the official Python source code + repositories were migrated from the :abbr:`svn (Subversion)` + :abbr:`VCS (version control system)` to Mercurial. + The issue tracker automatically translates the legacy svn revision + ```` to its corresponding Mercurial changeset identifier. +* ``Dir/file.ext`` and ``Dir/file.ext:NNN`` generate links to files in the + `Python source code repositories `_, + possibly linking to the line number specified after the ``:``. + ``3.6/Dir/file.ext`` will generate a link with ``3.6`` as branch. +* ``PEP `` and ``PEP`` link to the + :abbr:`PEP (Python Enhancement Proposal)` ````. +* ``devguide`` (lowercase), ``devguide/triaging``, and + ``devguide/triaging#generating-special-links-in-a-comment`` generate links to + the Devguide, this page, and this section respectively. + + +.. _CPython: https://github.com/python/cpython/ +.. _Doc: https://github.com/python/cpython/tree/master/Doc/ +.. _Grammar: https://github.com/python/cpython/tree/master/Grammar/ +.. _Lib: https://github.com/python/cpython/tree/master/Lib/ +.. _Lib/lib2to3: https://github.com/python/cpython/tree/master/Lib/lib2to3/ +.. _Lib/ctypes: https://github.com/python/cpython/tree/master/Lib/ctypes/ +.. _Lib/distutils: https://github.com/python/cpython/tree/master/Lib/distutils/ +.. _Lib/doctest.py: https://github.com/python/cpython/tree/master/Lib/doctest.py +.. _Lib/idlelib: https://github.com/python/cpython/tree/master/Lib/idlelib/ +.. _Lib/io.py: https://github.com/python/cpython/tree/master/Lib/io.py +.. _Lib/re.py: https://github.com/python/cpython/tree/master/Lib/re.py +.. _Lib/test: https://github.com/python/cpython/tree/master/Lib/test/ +.. _Lib/test/regrtest.py: https://github.com/python/cpython/tree/master/Lib/test/regrtest.py +.. _Lib/test/support: https://github.com/python/cpython/tree/master/Lib/test/support/ +.. _Lib/tkinter: https://github.com/python/cpython/tree/master/Lib/tkinter/ +.. _Lib/unittest: https://github.com/python/cpython/tree/master/Lib/unittest/ +.. _Lib/xml: https://github.com/python/cpython/tree/master/Lib/xml/ +.. _Modules: https://github.com/python/cpython/tree/master/Modules/ +.. _Modules/_io: https://github.com/python/cpython/tree/master/Modules/_io/ +.. _Modules/_sre.c: https://github.com/python/cpython/tree/master/Modules/_sre.c +.. _Objects: https://github.com/python/cpython/tree/master/Objects/ +.. _Objects/unicodeobject.c: https://github.com/python/cpython/tree/master/Objects/unicodeobject.c +.. _Parser: https://github.com/python/cpython/tree/master/Parser/ +.. _Python: https://github.com/python/cpython/tree/master/Python/ +.. _Tools: https://github.com/python/cpython/tree/master/Tools/ +.. _Tools/demo: https://github.com/python/cpython/tree/master/Tools/demo/ +.. _Developer's guide: https://github.com/python/devguide/ +.. _GSoC: https://developers.google.com/open-source/gsoc/ +.. _issue tracker: https://bugs.jython.org