8000 Rework Java installation by viren-nadkarni · Pull Request #11139 · localstack/localstack · GitHub
[go: up one dir, main page]

Skip to content

Rework Java installation #11139

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 22 commits into from
Sep 12, 2024
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
439aedb
Use Java 21
viren-nadkarni Jul 4, 2024
410822b
Merge branch 'master' into update-java-21
viren-nadkarni Jul 30, 2024
c3e7b6e
Add Java package
viren-nadkarni Jul 31, 2024
8fbe169
Fix comments
viren-nadkarni Jul 31, 2024
52d9c72
Refactor Java installer to remove OSPackageInstaller
viren-nadkarni Jul 31, 2024
316b49f
Explicitly specify Java 11 as DDBLocal dependency
viren-nadkarni Jul 31, 2024
378a7a2
Install JRE via LPM during Docker build
viren-nadkarni Jul 31, 2024
f1f8f93
Revert "Explicitly specify Java 11 as DDBLocal dependency"
viren-nadkarni Jul 31, 2024
00e6433
Add comment
viren-nadkarni Jul 31, 2024
2da97ff
Merge branch 'master' into update-java-21
viren-nadkarni Sep 3, 2024
c108cb5
Remove docker-java-home executable
viren-nadkarni Sep 3, 2024
a325dfc
Optimise JRE for minimise disk footprint
viren-nadkarni Sep 3, 2024
88c8736
Determine the latest stable build
viren-nadkarni Sep 3, 2024
f1062cf
Improve already-installed detection
viren-nadkarni Sep 3, 2024
1ac97fc
Improve cross-arch compatibility
viren-nadkarni Sep 3, 2024
c76b429
Add Thai locale
viren-nadkarni Sep 3, 2024
5657058
Minor rearrangement
viren-nadkarni Sep 4, 2024
a286af0
Add binutils package which provides objcopy command
viren-nadkarni Sep 5, 2024
f0d6bf7
Use distinct levels when joining paths
viren-nadkarni Sep 10, 2024
4b8f57f
Fix missing f-string
viren-nadkarni Sep 10, 2024
8689130
Merge branch 'master' into update-java-21
viren-nadkarni Sep 10, 2024
680a58a
Define and use LocalStack user-agent string
viren-nadkarni Sep 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 18 additions & 44 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,34 +1,6 @@
# java-builder: Stage to build a custom JRE (with jlink)
FROM eclipse-temurin:11@sha256:f4ab40c615ec2bd6a09992f6ceebeca796752e58ea778ccc6439087a728ba3b3 AS java-builder

# create a custom, minimized JRE via jlink
RUN jlink --add-modules \
# include required modules
java.base,java.desktop,java.instrument,java.management,java.naming,java.scripting,java.sql,java.xml,jdk.compiler,\
# jdk.unsupported contains sun.misc.Unsafe which is required by certain dependencies
jdk.unsupported,\
# add additional cipher suites
jdk.crypto.cryptoki,\
# add ability to open ZIP/JAR files
jdk.zipfs,\
# OpenSearch needs some jdk modules
jdk.httpserver,jdk.management,\
# MQ Broker requires management agent
jdk.management.agent,\
# required for Spark/Hadoop
java.security.jgss,jdk.security.auth,\
# Elasticsearch 7+ crashes without Thai Segmentation support
jdk.localedata --include-locales en,th \
--compress 2 --strip-debug --no-header-files --no-man-pages --output /usr/lib/jvm/java-11 && \
cp ${JAVA_HOME}/bin/javac /usr/lib/jvm/java-11/bin/javac && \
cp -r ${JAVA_HOME}/include /usr/lib/jvm/java-11/include && \
mv /usr/lib/jvm/java-11/lib/modules /usr/lib/jvm/java-11/lib/modules.bk; \
cp -r ${JAVA_HOME}/lib/* /usr/lib/jvm/java-11/lib/; \
mv /usr/lib/jvm/java-11/lib/modules.bk /usr/lib/jvm/java-11/lib/modules; \
rm -rf /usr/bin/java ${JAVA_HOME} && ln -s /usr/lib/jvm/java-11/bin/java /usr/bin/java


#
# base: Stage which installs necessary runtime dependencies (OS packages, java,...)
#
FROM python:3.11.9-slim-bookworm@sha256:3f3c35617e79276c5f6a2e6a13cdbabdd10257332df963c90c986858b26fad5e AS base
ARG TARGETARCH

Expand Down Expand Up @@ -84,18 +56,7 @@ RUN ARCH= && dpkgArch="$(dpkg --print-architecture)" \
&& test ! $(which python3.9)

SHELL [ "/bin/bash", "-c" ]

# Install Java 11
ENV LANG=C.UTF-8
RUN { \
echo '#!/bin/sh'; echo 'set -e'; echo; \
echo 'dirname "$(dirname "$(readlink -f "$(which javac || which java)")")"'; \
} > /usr/local/bin/docker-java-home \
&& chmod +x /usr/local/bin/docker-java-home
ENV JAVA_HOME=/usr/lib/jvm/java-11
COPY --from=java-builder /usr/lib/jvm/java-11 $JAVA_HOME
RUN ln -s $JAVA_HOME/bin/java /usr/bin/java
ENV PATH="${PATH}:${JAVA_HOME}/bin"
ENV LANG C.UTF-8

# set workdir
RUN mkdir -p /opt/code/localstack
Expand Down Expand Up @@ -137,8 +98,9 @@ RUN --mount=type=cache,target=/root/.cache \
pip3 install --upgrade awscli awscli-local requests



#
# builder: Stage which installs the dependencies of LocalStack Community
#
FROM base AS builder
ARG TARGETARCH

Expand Down Expand Up @@ -166,8 +128,9 @@ RUN --mount=type=cache,target=/root/.cache \
RUN . .venv/bin/activate && pip3 freeze -l > requirements-runtime.txt



#
# final stage: Builds upon base stage and copies resources from builder stages
#
FROM base
COPY --from=builder /opt/code/localstack/.venv /opt/code/localstack/.venv

Expand All @@ -186,12 +149,23 @@ RUN make entrypoints
RUN --mount=type=cache,target=/root/.cache \
--mount=type=cache,target=/var/lib/localstack/cache \
source .venv/bin/activate && \
python -m localstack.cli.lpm install java --version 11 && \
python -m localstack.cli.lpm install \
lambda-runtime \
dynamodb-local && \
chown -R localstack:localstack /usr/lib/localstack && \
chmod -R 777 /usr/lib/localstack

# Set up Java
ENV JAVA_HOME /usr/lib/localstack/java/11
RUN { \
echo '#!/bin/sh'; echo 'set -e'; echo; \
echo 'dirname "$(dirname "$(readlink -f "$(which javac || which java)")")"'; \
} > /usr/local/bin/docker-java-home \
&& chmod +x /usr/local/bin/docker-java-home
RUN ln -s $JAVA_HOME/bin/java /usr/bin/java
ENV PATH="${PATH}:${JAVA_HOME}/bin"

# link the python package installer virtual environments into the localstack venv
RUN echo /var/lib/localstack/lib/python-packages/lib/python3.11/site-packages > localstack-var-python-packages-venv.pth && \
mv localstack-var-python-packages-venv.pth .venv/lib/python*/site-packages/
Expand Down
70 changes: 70 additions & 0 deletions localstack-core/localstack/packages/java.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
import logging
import os
from typing import List

from localstack.packages import Package
from localstack.packages.core import ArchiveDownloadAndExtractInstaller
from localstack.utils.platform import Arch, get_arch, is_linux, is_mac_os

LOG = logging.getLogger(__name__)

# Default version if not specified
DEFAULT_JAVA_VERSION = "11"

# Supported Java LTS versions mapped with Eclipse Temurin build semvers
JAVA_VERSIONS = {
"8": "8u422-b05",
"11": "11.0.24+8",
"17": "17.0.12+7",
"21": "21.0.4+7",
}

JRE_DISTRIB_URL = "https://github.com/adoptium/temurin{version}-binaries/releases/download/{tag_slug}/OpenJDK{version}U-jre_{arch}_{os}_hotspot_{semver_safe}.tar.gz"


class JavaPackageInstaller(ArchiveDownloadAndExtractInstaller):
def __init__(self, version: str):
super().__init__("java", version, extract_single_directory=True)

self.semver = JAVA_VERSIONS[version]

def _get_install_marker_path(self, install_dir: str) -> str:
return os.path.join(install_dir, self._get_archive_subdir())

def _get_download_url(self) -> str:
os = "linux" if is_linux() else "mac" if is_mac_os() else None
arch = (
"x64" if get_arch() == Arch.amd64 else "aarch64" if get_arch() == Arch.arm64 else None
)

tag_slug = f"jdk-{self.semver}"
semver_safe = self.semver.replace("+", "_")

# v8 uses a different tag and version scheme
if self.version == "8":
semver_safe = semver_safe.replace("-", "")
tag_slug = f"jdk{self.semver}"

return JRE_DISTRIB_URL.format(
version=self.version, tag_slug=tag_slug, os=os, arch=arch, semver_safe=semver_safe
)

def _get_archive_subdir(self) -> str | None:
return f"jdk-{self.semver}-jre"

def get_java_home(self) -> str:
return self.get_installed_dir()


class JavaPackage(Package):
def __init__(self, default_version: str = DEFAULT_JAVA_VERSION):
super().__init__(name="Java", default_version=default_version)

def get_versions(self) -> List[str]:
return list(JAVA_VERSIONS.keys())

def _get_installer(self, version):
return JavaPackageInstaller(version)


java_package = JavaPackage()
7 changes: 7 additions & 0 deletions localstack-core/localstack/packages/plugins.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,10 @@ def ffmpeg_package() -> Package:
from localstack.packages.ffmpeg import ffmpeg_package

return ffmpeg_package


@package(name="java")
def java_package() -> Package:
from localstack.packages.java import java_package

return java_package
Loading
0