Review/finalize Python approach for mid 2020 #20716
Conversation
|
P.S.: For the record, this whole thing was kicked off after I posted an issue about not being able to install Atom on the Ubuntu 20.04 development branch. That issue was #20356. I have learned much since then, so while the discussion there might be of some historical/archival interest (maybe?) the commentary there is almost entirely outdated. |
|
Timeline (background):
Timeline (from the start of my involvement to this PR.)
|
Python is only needed for apm --> npm --> node-gyp. (For building Atom packages that include native C/C++ code.) The rest of Atom/apm works 100%, even with no Python installed. With Python 2 soon to be dropped from the Debian/Ubuntu repos, having a hard dependency on `python` or `python2` is a problem. None of the other OSes/platforms have an install-time requirement of having Python on the system, so this is in line with Atom packaging for the other platforms.
|
It occurs to me that while "there is no Python dependency in the Atom packaging for any other OS/platform besides Debian+Ubuntu"... That's mostly because there is no mechanism to do so on Windows or macOS. In which case, whatever we decide should just be consistent across Ubuntu and Fedora, which both have package management systems with dependency resolution, and all that. (They both have pretty much identical systems with "Depends," "Recommends," and "Suggests," too.) |
|
Thanks @DeeDeeG for the summary and the timeline, it's definitely helpful to have the whole context in one place. I would not assume that an average user doesn't need to install native modules, as atom depends on multiple native modules and normally atom users installs a handful of packages (both native and non-native).
So keeping the above in mind, I wanted to summarize the big picture to make sure we are on the same page. I would assume the only affected users by dropping python as a required dependency, would be the ones with minimalist distros, but mainstream ones like Ubuntu, Fedora, etc should not be affected, since these OS-es come bundled with python, and apm 2.5 now supports up to python 3. |
|
Basically correct, yes. Regarding this summary:
|
|
More research and details (which confirm and reinforce the overall gist of the above summary), but I'm not sure if anyone needs this level of detail, so putting behind a "click to expand" thing: More details (click to expand).
These native modules are built and bundled with Atom. End-users get a warning message if they try to reinstall them manually. These modules end up as some sort of native compiled target during the Atom build process, and whatever form that takes, it doesn't require a Python executable after building; Python's only needed at build time. Since they're bundled with Atom, end-users should never need Python for these packages. $ apm install tree-view
The tree-view package is bundled with Atom and should not be explicitly installed.
You can run `apm uninstall tree-view` to uninstall it and then the version bundled
with Atom will be used.
Installing tree-view to /home/[user]/.atom/packages ✓But yeah, I agree some people do need Python to build native modules, like users who install the "terminal-in-Atom" type of packages.
Fedora isn't affected by this PR at all, since that OS uses
I have only found / confirmed one edition each of Debian/Ubuntu that ships without Python, and that is the Docker images for "ubuntu" and "debian". All the following that I tested DO come with Python:
[END OF EXPANDED DETAILS] Briefly: Native modules that Atom depends on are built and bundled during Atom's packaging process. End-users aren't supposed to reinstall or rebuild these; They get a warning message that they shouldn't do that, if they try. So, users shouldn't need Python for these packages. This PR doesn't affect Atom's Fedora package (so it doesn't affect Fedora users), but it does coincidentally make Atom's Debian package match what Atom already is doing for Fedora. (No Python dependency). From my testing, only extremely stripped-down environments like the tiny (~75 to 125 MB) Docker images of Debian/Ubuntu don't ship with Python. In fact it might exclusively be the Docker images; I can't find anything else that doesn't ship with Python. |
|
I would like to see in the
Edit: A reasonable deadline to do this by (getting it into the stable version of Atom) is October 22. |
Requirements for Contributing a Bug Fix
Identify the Bug
Addresses most of #20585
(Some small additional things are left to do, in order to adjust the repo for Python 3. See: #20585 (comment) for details.)
The following PR was rolled into this PR, so it should be automatically closed if this is merged:
Closes #20670
Description of the Change
This was a multi-part fix. Some of it has already been merged. This PR exists to review/finalize the approach taken. The goal is to ensure the Atom project supports Python 3, and can keep operating entirely without needing Python 2. That said, the approach taken here does maintain full backward compatibility with Python 2.
apmwith newernode-gypthat supports Python 2 AND Python 3.node-gypis the only reason end users would need Python in order to use Atom/apm. It builds Atom packages that include native c/c++ code.gypis written in Python, so runningnode-gyprequires a Python interpreter on the system.npm(and its dependencynode-gyp) in thescript/directory, to ensure Atom itself can be BUILT with either Python 2 or Python 3.pythondependency from the Debian/Ubuntu.debpackage.Alternate Designs
The main, most-important thing that was accomplished with regard to Python and Atom, was updating to a newer version of
node-gypwhich supports both Python 3 and Python 2.See: #20703 and atom/apm#887 for the implementation.
As far as I am concerned, and as far as I can tell, that part is not a controversial change, as there is no obvious downside, and as such there is no proposed alternate design 10000 for that part.
Everything else is at least an order of magnitude less important, IMO.
The subjective parts left to decide on are:
.debfile?.debfile, it should by Python 3 aware, i.e.python3 | python2 | python. (versionlesspythonshould be last, as the versionlesspythonpackage is being deprecated in upstream Debian/Ubuntu.).debfile, should it beDepends:(hard requirement),Recommends:(optional but installed by defatult), orSuggests:(optional, not installed by default)?Possible Drawbacks
For the core change, updating
npmandnode-gyp, there are no drawbacks that I am aware of, whatsoever. This is a minor version bump ofnpm, and a well-tested major version bump ofnode-gypthat has been working well so far inapm. The new version ofnode-gypis included in the latest upstream releases ofnpm, so is likely to see wide usage, testing and attention for any bugs.apm2.5.0 also bundles newer Node 10.20.1, a minor version bump over Node 10.2.1. Again, this is a minor version bump of highly-used/tested upstream Node.js. Bugs or issues in the latest v10 Long-Term-Service version of Node are not expected, and in fact could reasonably be expected to include fixes for miscellaneous bugs.As for the subjective situation with the
.debpackaging, every available approach is somewhat arbitrary.Users don't strictly need Python in order to use Atom/apm, unless they install one of a handful of Atom packages that include native code. See: #20406 (comment) for an investigation into this.
Putting Python in the
.debpackage's dependencies would purport to make sure everyone is prepared to install that handful of packages. (Conversely, removing the python dependency might be considered a "breaking change" for users of that handful of packages. But only in the rare circumstances where a user has no Python installed, which as far as I know is not possible on Debian/Ubuntu under a graphical desktop environment; Various Debian/Ubuntu desktop components depend on one version or other of Python. It's only possible to not have Python installed on headless server/docker variants of Debian/Ubuntu, I think.)Furthermore: Other than the
.debpackage, no other official Atom package has Python as an install-time dependency. Atom users on other OSes have apparently been okay all this time without an install-time dependency on Python.node-gyphas additional requirements that currently are NOT specified as install-time dependencies in any platform's packaging, particularlymake,cc/gccandc++/g++. So users without those installed need to read thenode-gyperror messages they encounter, and understand the message well enough to realize that they must install a C compiler and a C++ compiler to their system.As such, the Python dependency in the
.debfile is not sufficient on its own to ensure Atom users can successfully build packages withnode-gyp.Admittedly, using the
python3 | python2 | pythondependency logic would work on all Debian/Ubuntu releases past and for the foreseeable future. But the question remains: Why go out of our way to ensure the user has Python when few users will need it, and the users who need it must also manually install certain dependencies likegccandg++?By the way, users of currently supported desktop-oriented builds of Debian/Ubuntu will have both Python 2 AND Python 3 installed, by default. Only minimal/headless server/Docker images of Debian/Ubuntu are at all likely to have neither Python 2 nor Python 3 installed.
Verification Process
Manually built
apmandatomwith every variation of these changes discussed. Checked whether the built atom.debfile could be installed on various distros. Checked whether the resultingapmcommand works, e.g. by runningapm install tree-view. Saw that CI passes.Release Notes
N/A
(Not user-facing? But this could work: "- Atom no longer requires Python in order to install on Debian/Ubuntu." "- Atom and apm now support Python 3.")