8000 Configure cjdk behavior via scyjava.config by ctrueden · Pull Request #83 · scijava/scyjava · GitHub
[go: up one dir, main page]

Skip to content

Configure cjdk behavior via scyjava.config #83

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 30, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions src/scyjava/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,22 @@
+++oo*OO######O**oo+++++oo*OO######O**oo+++++oo*OO######O**oo+++
+++oo*OO######OO*oo+++++oo*OO######OO*oo+++++oo*OO######OO*oo+++

Bootstrap a Java installation:

>>> from scyjava import config, jimport
>>> config.set_java_constraints(fetch=True, vendor='zulu', version='17')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, I was about to request you as a reviewer, but... 🚀

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i noticed the brief addition of add_kwargs ... I assume it's coming elsewhere. my only comment was gonna be whether that might be too generic? (though I know it matches add_options). perhaps add_jvm_kwargs?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, rather than deprecate the term options in favor of jvm_options, I decided to just leave it and call the kwargs kwargs, to maintain a foolish consistency. We could of course deprecate both in favor of clearer names.

The reason it appeared only "briefly" is because I ended up moving a bunch of unrelated (to this PR) commits over to main, which included the kwargs addition.

Note that I screwed up releasing scyjava 1.11.0: it does not include this PR, even though it had been merged, because I forgot to fast-forward my local main branch. So I then immediately did so and released 1.12.0 as well. The only "harm" there is that 1.11.0 now contains the fetch_java keyword argument to start_jvm which I then removed, breaking SemVer. But hopefully no one will use 1.11.0 and then complain about this. 😅

>>> System = jimport('java.lang.System')
cjdk: Installing JDK zulu:17.0.15 to /home/chuckles/.cache/cjdk
Download 100% of 189.4 MiB |##########| Elapsed Time: 0:00:02 Time: 0:00:02
Extract | | # | 714 Elapsed Time: 0:00:01
cjdk: Installing Maven to /home/chuckles/.cache/cjdk
Download 100% of 8.7 MiB |##########| Elapsed Time: 0:00:00 Time: 0:00:00
Extract | |# | 102 Elapsed Time: 0:00:00
>>> System.getProperty('java.vendor')
'Azul Systems, Inc.'
>>> System.getProperty('java.version')
'17.0.15'

Convert Java collections to Python:

>>> from scyjava import jimport
Expand Down
48 changes: 28 additions & 20 deletions src/scyjava/_cjdk_fetch.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
"""
Utility functions for fetching JDK/JRE and Maven.
"""

from __future__ import annotations

import logging
Expand All @@ -9,21 +13,23 @@
import cjdk
import jpype

import scyjava.config

if TYPE_CHECKING:
from pathlib import Path

_logger = logging.getLogger(__name__)
_DEFAULT_MAVEN_URL = "tgz+https://dlcdn.apache.org/maven/maven-3/3.9.9/binaries/apache-maven-3.9.9-bin.tar.gz" # noqa: E501
_DEFAULT_MAVEN_SHA = "a555254d6b53d267965a3404ecb14e53c3827c09c3b94b5678835887ab404556bfaf78dcfe03ba76fa2508649dca8531c74bca4d5846513522404d48e8c4ac8b" # noqa: E501
_DEFAULT_JAVA_VENDOR = "zulu-jre"
_DEFAULT_JAVA_VERSION = "11"


def ensure_jvm_available() -> None:
"""Ensure that the JVM is available and Maven is installed."""
if not is_jvm_available():
fetch = scyjava.config.get_fetch_java()
if fetch == "never":
# Not allowed to use cjdk.
return
if fetch == "always" or not is_jvm_available():
cjdk_fetch_java()
if not shutil.which("mvn"):
if fetch == "always" or not shutil.which("mvn"):
cjdk_fetch_maven()


Expand All @@ -47,27 +53,27 @@ def _silent_check_output(*args, **kwargs):
return True


def cjdk_fetch_java(vendor: str = "", version: str = "") -> None:
def cjdk_fetch_java(vendor: str | None = None, version: str | None = None) -> None:
"""Fetch java using cjdk and add it to the PATH."""
if not vendor:
vendor = os.getenv("JAVA_VENDOR", _DEFAULT_JAVA_VENDOR)
version = os.getenv("JAVA_VERSION", _DEFAULT_JAVA_VERSION)
if vendor is None:
vendor = scyjava.config.get_java_vendor()
if version is None:
version = scyjava.config.get_java_version()

_logger.info(f"No JVM found, fetching {vendor}:{version} using cjdk...")
home = cjdk.java_home(vendor=vendor, version=version)
_add_to_path(str(home / "bin"))
os.environ["JAVA_HOME"] = str(home)
_logger.info(f"Fetching {vendor}:{version} using cjdk...")
java_home = cjdk.java_home(vendor=vendor, version=version)
_logger.debug(f"java_home -> {java_home}")
_add_to_path(str(java_home / "bin"), front=True)
os.environ["JAVA_HOME"] = str(java_home)


def cjdk_fetch_maven(url: str = "", sha: str = "") -> None:
"""Fetch Maven using cjdk and add it to the PATH."""
# if url was passed as an argument, or env_var, use it with provided sha
# if url was passed as an argument, use it with provided sha
# otherwise, use default values for both
if url := url or os.getenv("MAVEN_URL", ""):
sha = sha or os.getenv("MAVEN_SHA", "")
else:
url = _DEFAULT_MAVEN_URL
sha = _DEFAULT_MAVEN_SHA
if not url:
url = scyjava.config.get_maven_url()
sha = scyjava.config.get_maven_sha()

# fix urls to have proper prefix for cjdk
if url.startswith("http"):
Expand All @@ -88,7 +94,9 @@ def cjdk_fetch_maven(url: str = "", sha: str = "") -> None:
)
kwargs = {sha_lengths[sha_len]: sha}

_logger.info("Fetching Maven using cjdk...")
maven_dir = cjdk.cache_package("Maven", url, **kwargs)
_logger.debug(f"maven_dir -> {maven_dir}")
if maven_bin := next(maven_dir.rglob("apache-maven-*/**/mvn"), None):
_add_to_path(maven_bin.parent, front=True)
else: # pragma: no cover
Expand Down
23 changes: 5 additions & 18 deletions src/scyjava/_jvm.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,15 @@
from functools import lru_cache
from importlib import import_module
from pathlib import Path
from typing import Sequence

import jpype
import jpype.config
from jgo import jgo

import scyjava.config
from scyjava.config import Mode, mode
from scyjava._cjdk_fetch import ensure_jvm_available

_logger = logging.getLogger(__name__)

Expand Down Expand Up @@ -106,7 +108,7 @@ def jvm_version() -> tuple[int, ...]:
return tuple(map(int, m.group(1).split(".")))


def start_jvm(options=None, *, fetch_java: bool = True) -> None:
def start_jvm(options: Sequence[str] = None) -> None:
"""
Explicitly connect to the Java virtual machine (JVM). Only one JVM can
be active; does nothing if the JVM has already been started. Calling
Expand All @@ -118,19 +120,6 @@ def start_jvm(options=None, *, fetch_java: bool = True) -> None:
List of options to pass to the JVM.
For example: ['-Dfoo=bar', '-XX:+UnlockExperimentalVMOptions']
See also scyjava.config.add_options.
:param fetch_java:
If True (default), when a JVM/or maven cannot be located on the system,
[`cjdk`](https://github.com/cachedjdk/cjdk) will be used to download
a JRE distribution and set up the JVM. The following environment variables
may be used to configure the JRE and Maven distributions to download:
* `JAVA_VENDOR`: The vendor of the JRE distribution to download.
Defaults to "zulu-jre".
* `JAVA_VERSION`: The version of the JRE distribution to download.
Defaults to "11".
* `MAVEN_URL`: The URL of the Maven distribution to download.
Defaults to https://dlcdn.apache.org/maven/maven-3/3.9.9/
* `MAVEN_SHA`: The SHA512 hash of the Maven distribution to download, if
providing a custom MAVEN_URL.
"""
# if JVM is already running -- break
if jvm_started():
Expand All @@ -147,10 +136,8 @@ def start_jvm(options=None, *, fetch_java: bool = True) -> None:
# use the logger to notify user that endpoints are being added
_logger.debug("Adding jars from endpoints {0}".format(endpoints))

if fetch_java:
from scyjava._cjdk_fetch import ensure_jvm_available

ensure_jvm_available()
# download JDK/JRE and Maven as appropriate
ensure_jvm_available()

# get endpoints and add to JPype class path
if len(endpoints) > 0:
Expand Down
110 changes: 110 additions & 0 deletions src/scyjava/config.py
< 9113 /tr>
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,13 @@

_logger = _logging.getLogger(__name__)

# Constraints on the Java installation to be used.
_fetch_java: str = "auto"
_java_vendor: str = "zulu-jre"
_java_version: str = "11"
_maven_url: str = "tgz+https://dlcdn.apache.org/maven/maven-3/3.9.9/binaries/apache-maven-3.9.9-bin.tar.gz" # noqa: E501
_maven_sha: str = "a555254d6b53d267965a3404ecb14e53c3827c09c3b94b5678835887ab404556bfaf78dcfe03ba76fa2508649dca8531c74bca4d5846513522404d48e8c4ac8b" # noqa: E501

endpoints: list[str] = []

_repositories = {"scijava.public": _scijava_public()}
Expand All @@ -37,6 +44,109 @@ class Mode(_enum.Enum):
mode = Mode.JPYPE


def set_java_constraints(
fetch: str | bool | None = None,
vendor: str | None = None,
version: str | None = None,
maven_url: str | None = None,
maven_sha: str | None = None,
) -> None:
"""
Set constraints on the version of Java to be used.

:param fetch:
If "auto" (default), when a JVM/or maven cannot be located on the system,
[`cjdk`](https://github.com/cachedjdk/cjdk) will be used to download
a JDK/JRE distribution and set up the JVM.
If "always", cjdk will always be used; if "never", cjdk will never be used.
:param vendor:
The vendor of the JDK/JRE distribution for cjdk to download and cache.
Defaults to "zulu-jre". See the cjdk documentation for details.
:param version:
Expression defining the Java version for cjdk to download and cache.
Defaults to "11". See the cjdk documentation for details.
:param maven_url:
URL of the Maven distribution for cjdk to download and cache.
Defaults to the Maven 3.9.9 binary distribution from dlcdn.apache.org.
:param maven_sha:
The SHA512 (or SHA256 or SHA1) hash of the Maven distribution to download,
if providing a custom maven_url.
"""
global _fetch_java, _java_vendor, _java_version, _maven_url, _maven_sha
if fetch is not None:
if isinstance(fetch, bool):
# Be nice and allow boolean values as a convenience.
fetch = "always" if fetch else "never"
expected = ["auto", "always", "never"]
if fetch not in expected:
raise ValueError(f"Fetch mode {fetch} is not one of {expected}")
_fetch_java = fetch
if vendor is not None:
_java_vendor = vendor
if version is not None:
_java_version = version
if maven_url is not None:
_maven_url = maven_url
_maven_sha = ""
if maven_sha is not None:
_maven_sha = maven_sha


def get_fetch_java() -> str:
"""
Get whether [`cjdk`](https://github.com/cachedjdk/cjdk)
will be used to download a JDK/JRE distribution and set up the JVM.
To set this value, see set_java_constraints.

:return:
"always" for cjdk to obtain the JDK/JRE;
"never" for cjdk *not* to obtain a JDK/JRE;
"auto" for cjdk to be used only when a JVM/or Maven is not on the system path.
"""
return _fetch_java


def get_java_vendor() -> str:
"""
Get the vendor of the JDK/JRE distribution to download.
Vendor of the Java installation for cjdk to download and cache.
To set this value, see set_java_constraints.

:return: String defining the desired JDK/JRE vendor for downloaded JDK/JREs.
"""
return _java_vendor


def get_java_version() -> str:
"""
Expression defining the Java version for cjdk to download and cache.
To set this value, see set_java_constraints.

:return: String defining the desired JDK/JRE version for downloaded JDK/JREs.
"""
return _java_version


def get_maven_url() -> str:
"""
The URL of the Maven distribution to download.
To set this value, see set_java_constraints.

:return: URL pointing to the Maven distribution.
"""
return _maven_url


def get_maven_sha() -> str:
"""
The SHA512 (or SHA256 or SHA1) hash of the Maven distribution to download,
if providing a custom maven_url. To set this value, see set_java_constraints.

:return: Hash value of the Maven distribution, or empty string to skip hash check.
"""
return _maven_sha


def add_repositories(*args, **kwargs) -> None:
"""
Add one or more Maven repositories to be used by jgo for downloading dependencies.
Expand Down
Loading
0