8339480: Build static-jdk image with a statically linked launcher #20837

magicus · 2024-09-03T12:50:01Z

As a prerequisite for Hermetic Java, we need a statically linked java launcher. It should behave like the normal, dynamically linked java launcher, except that all JDK native libraries should be statically, not dynamically, linked.

This patch is the first step towards this goal. It will generate a static-jdk image with a statically linked launcher. This launcher is missing several native libs, however, and does therefore not behave like a proper dynamic java. One of the reasons for this is that local symbol hiding in static libraries are not implemented yet, which causes symbol clashes when linking all static libraries together. This will be addressed in an upcoming patch.

All changes in the src directory are copied from, or inspired by, changes made in the hermetic-java-runtime branch in Project Leyden.

Progress

Change must not contain extraneous whitespace
Commit message must refer to an issue
Change must be properly reviewed (3 reviews required, with at least 1 Reviewer, 2 Authors)

Issue

JDK-8339480: Build static-jdk image with a statically linked launcher (Enhancement - P2)

Reviewers

David Holmes (@dholmes-ora - Reviewer)
Erik Joelsson (@erikj79 - Reviewer)
Coleen Phillimore (@coleenp - Reviewer)

Contributors

Magnus Ihse Bursie <[email protected]>
Jiangli Zhou <[email protected]>

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/20837/head:pull/20837
$ git checkout pull/20837

Update a local copy of the PR:
$ git checkout pull/20837
$ git pull https://git.openjdk.org/jdk.git pull/20837/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 20837

View PR using the GUI difftool:
$ git pr show -t 20837

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/20837.diff

Using Webrev

Link to Webrev Comment

…ning a static launcher

magicus · 2024-09-03T12:50:49Z

/contributor add ihse
/contributor add jiangli

bridgekeeper · 2024-09-03T12:50:52Z

👋 Welcome back ihse! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

magicus · 2024-09-03T12:51:13Z

@jianglizhou Can you please check if there are any other contributors that should be acknowledged?

openjdk · 2024-09-03T12:51:23Z

@magicus This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8339480: Build static-jdk image with a statically linked launcher

Co-authored-by: Magnus Ihse Bursie <[email protected]>
Co-authored-by: Jiangli Zhou <[email protected]>
Reviewed-by: dholmes, erikj, coleenp

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 87 new commits pushed to the master branch:

d589baf: 8345218: Clean out references to windows-x86 in jib profiles
b823398: 8345267: Fix memory leak in JVMCIEnv dtor
9a48e4d: 8345177: RISC-V: Add gtests for cmpxchg
e3b679a: 8345176: Add tests to verify java.net.Socket constructors close the socket on failure
c7be41e: 8340133: Add concise usage message to the java executable
0b0f83c: 8345220: Serial: Refactor TenuredGeneration::promotion_attempt_is_safe
dfcbfb5: 8345236: RISC-V: Remove revb_h_h_u and revb_h_w_u macro assembler routines
ac2fede: 8344041: Re-enable external specs page
1c4c653: 8345247: Deproblemlist test/jdk/javax/swing/JRadioButton/8075609/bug8075609.java
f5ebda4: 8345173: BlockLocationPrinter::print_location misses a ResourceMark
... and 77 more: https://git.openjdk.org/jdk/compare/c329f97f3211bc14aa4211461bb9a7abb073296e...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

openjdk · 2024-09-03T12:51:47Z

@magicus
Contributor Magnus Ihse Bursie <[email protected]> successfully added.

openjdk · 2024-09-03T12:52:07Z

@magicus
Contributor Jiangli Zhou <[email protected]> successfully added.

openjdk · 2024-09-03T12:52:28Z

@magicus The following labels will be automatically applied to this pull request:

build
client
core-libs
hotspot

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

magicus · 2024-09-03T12:54:20Z

The static-jdk image is enough to run a simple HelloWorld java program; however it cannot yet run JTReg tests. The reason for this is, afaict, that the javac launcher is missing. I am planning to examine if it is possible to add a small shim javac launcher that calls the static java with the proper main class. (And similarly for the other launchers.) The goal here must, after all, be that we should be able to run the normal jtreg tests on the static-jdk image.

mlbridge · 2024-09-03T13:02:55Z

Webrevs

erikj79 · 2024-09-03T18:10:06Z

I tried to take this for a spin on my m1 mac laptop. I ran configure and then make static-jdk-image. The build failed with the following:

chmod: /Users/erik/dev/jdk/build/macosx-aarch64/jdk/lib/server/libjsig.dylib: No such file or directory
ModuleWrapper.gmk:81: recipe for target '/Users/erik/dev/jdk/build/macosx-aarch64/jdk/lib/server/libjsig.dylib' failed
make[3]: *** [/Users/erik/dev/jdk/build/macosx-aarch64/jdk/lib/server/libjsig.dylib] Error 1
make[3]: *** Waiting for unfinished jobs....
make/Main.gmk:191: recipe for target 'java.base-libs' failed
make[2]: *** [java.base-libs] Error 2

I'm guessing this would work if I built the regular image first, or at least at the same time.

magicus · 2024-09-03T19:48:38Z

I'm guessing this would work if I built the regular image first, or at least at the same time.

No, I don't think that should matter. static-jdk-image depends on exploded-image, and the files in your error message resides in jdk, not images/jdk. I have never seen this problem before. I'm not even sure why or when we try to chmod libjsig?

magicus · 2024-09-03T19:50:59Z

I wonder if it is related to JDK-8336498?

make/StaticLibs.gmk

make/ModuleWrapper.gmk

make/common/JdkNativeCompilation.gmk

make/StaticLibs.gmk

make/modules/java.desktop/lib/AwtLibraries.gmk

src/java.base/unix/native/libjli/java_md.c

src/hotspot/share/classfile/classLoader.cpp

src/java.base/unix/native/libjli/java_md.c

jianglizhou · 2024-09-05T00:25:01Z

@jianglizhou Can you please check if there are any other contributors that should be acknowledged?

Thanks for asking! I checked all the related changes in https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch. Following are the related commits from the branch corresponding to the extracted changes included in this PR. There are no other authors for those changes. However, all changes have gone through prior reviews by my teammates. I didn't ask it for the earlier integration PR, is there a way to mention the reviewer contributions?

magicus · 2024-09-05T09:59:57Z

The /contributor is supposed to add attribution to whomever wrote the code. There is no way to document any prior reviewing for code; but they are of course welcome to review this PR, and then it will be documented.

src/java.base/unix/native/libjli/java_md.c

bridgekeeper · 2024-10-03T21:04:18Z

@magicus This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

magicus · 2024-10-15T17:08:21Z

Bot, keep it alive.

jianglizhou · 2024-11-19T16:51:47Z

@jianglizhou

I think we would need to decide what is the dll_dir with the static JDK support and hermetic support.

From my understanding, _sun_boot_library_path does not have any meaning when running with static builds. As I said before, I think we really ought to take a step back, consider the wider picture, and refactor the way we interact with internal native libraries. Determining how to handle _sun_boot_library_path for static builds are definitely part of that work.

There might still be some usages of _sun_boot_library_path with static JDK. I recently made changes (in our local copy and not yet in https://github.com/openjdk/leyden/tree/hermetic-java-runtime) to use _sun_boot_library_path to locate the jspawnhelper in <jdk>/lib when spawning child process is needed, for supporting specific hermetic Java testing mode (without building a full hermetic image). At some point, we want to call out the supported execution modes (e.g. hermetic mode, dynamic/tradition mode, hermetic testing mode, etc) in https://docs.google.com/document/d/1LWy9vFDis5-hbJwrFUBH1rc8w8f3oL6O3sOzQXjXVUM/edit?usp=sharing as part of the ongoing discussions with Alan, Ron and others. Agreed that _sun_boot_library_path should be part of the bigger picture design/discussion.

make/StaticLibs.gmk

make/autoconf/buildjdk-spec.gmk.template

make/common/JdkNativeCompilation.gmk

make/modules/java.desktop/lib/ClientLibraries.gmk

make/modules/jdk.accessibility/Lib.gmk

…to $(MODULE)_INCLUDED_LIBS and module-libs.txt to module-included-libs.txt

dholmes-ora · 2024-11-26T21:14:45Z

/reviewers 3

openjdk · 2024-11-26T21:15:37Z

@dholmes-ora
The total number of required reviews for this PR (including the jcheck configuration and the last /reviewers command) is now set to 3 (with at least 1 Reviewer, 2 Authors).

dholmes-ora

Hotspot changes look fine.

Thanks

coleenp

Hotspot code looks okay.

src/hotspot/os/linux/os_linux.cpp

magicus · 2024-12-02T15:12:09Z

/integrate

magicus · 2024-12-02T15:12:24Z

@dholmes-ora @coleenp @erikj79 Thanks for your reviews!

openjdk · 2024-12-02T15:12:28Z

Going to push as commit 1ca7644.
Since your change was applied there have been 87 commits pushed to the master branch:

d589baf: 8345218: Clean out references to windows-x86 in jib profiles
b823398: 8345267: Fix memory leak in JVMCIEnv dtor
9a48e4d: 8345177: RISC-V: Add gtests for cmpxchg
e3b679a: 8345176: Add tests to verify java.net.Socket constructors close the socket on failure
c7be41e: 8340133: Add concise usage message to the java executable
0b0f83c: 8345220: Serial: Refactor TenuredGeneration::promotion_attempt_is_safe
dfcbfb5: 8345236: RISC-V: Remove revb_h_h_u and revb_h_w_u macro assembler routines
ac2fede: 8344041: Re-enable external specs page
1c4c653: 8345247: Deproblemlist test/jdk/javax/swing/JRadioButton/8075609/bug8075609.java
f5ebda4: 8345173: BlockLocationPrinter::print_location misses a ResourceMark
... and 77 more: https://git.openjdk.org/jdk/compare/c329f97f3211bc14aa4211461bb9a7abb073296e...master

Your commit was automatically rebased without conflicts.

openjdk · 2024-12-02T15:12:39Z

@magicus Pushed as commit 1ca7644.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

jianglizhou · 2024-12-02T18:43:41Z

@magicus Thanks for integrating the changes, particularly reworking and making the statically linked launcher build changes cleaner!

magicus added 2 commits September 3, 2024 14:39

Incorporate changes from leyden/hermetic-java-runtime that allows run…

6073c8e

…ning a static launcher

Makefile changes needed for static-launcher and static-jdk-image targets

43f57b8

openjdk bot added build [email protected] hotspot [email protected] client [email protected] core-libs [email protected] labels Sep 3, 2024

magicus marked this pull request as ready for review September 3, 2024 12:57

openjdk bot added the rfr Pull request is ready for review label Sep 3, 2024

erikj79 reviewed Sep 3, 2024

View reviewed changes

make/StaticLibs.gmk Outdated Show resolved Hide resolved

make/ModuleWrapper.gmk Outdated Show resolved Hide resolved

make/common/JdkNativeCompilation.gmk Outdated Show resolved Hide resolved

jianglizhou reviewed Sep 4, 2024

View reviewed changes

jianglizhou reviewed Sep 5, 2024

View reviewed changes

src/hotspot/share/classfile/classLoader.cpp Show resolved Hide resolved

jianglizhou reviewed Sep 5, 2024

View reviewed changes

src/hotspot/share/classfile/classLoader.cpp Show resolved Hide resolved

jianglizhou reviewed Sep 5, 2024

View reviewed changes

src/java.base/unix/native/libjli/java_md.c Outdated Show resolved Hide resolved

AlanBateman reviewed Sep 5, 2024

View reviewed changes

src/java.base/unix/native/libjli/java_md.c Outdated Show resolved Hide resolved

magicus mentioned this pull request Nov 18, 2024

8333268: Fixes for static build #19478

Closed

3 tasks

erikj79 reviewed Nov 19, 2024

View reviewed changes

bridgekeeper bot added the oca Needs verification of OCA signatory status label Nov 19, 2024

openjdk bot removed the rfr Pull request is ready for review label Nov 19, 2024

bridgekeeper bot removed the oca Needs verification of OCA signatory status label Nov 19, 2024

openjdk bot added the rfr Pull request is ready for review label Nov 19, 2024

magicus added 6 commits November 26, 2024 16:43

Just exclude the a11y libraries from static builds

658be7d

Find LIBJAVA_JPEG_OBJS automatically

2b3ff7e

Find LIBZIP_OBJS automatically

a1b0213

Rename EXCLUDE_FROM_STATIC_LIBS to ONLY_EXPORTED, $(MODULE)_JDK_LIBS …

7e836d0

…to $(MODULE)_INCLUDED_LIBS and module-libs.txt to module-included-libs.txt

Remove LDFLAGS_STATIC_JDK

557dc34

Merge branch 'master' into static-jdk-image

8b1217a

magicus requested a review from erikj79 November 26, 2024 17:13

dholmes-ora approved these changes Nov 26, 2024

View reviewed changes

erikj79 approved these changes Dec 2, 2024

View reviewed changes

coleenp approved these changes Dec 2, 2024

View reviewed changes

src/hotspot/os/linux/os_linux.cpp Show resolved Hide resolved

openjdk bot added the ready Pull request is ready to be integrated label Dec 2, 2024

openjdk bot added the integrated Pull request has been integrated label Dec 2, 2024

openjdk bot closed this Dec 2, 2024

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Dec 2, 2024

magicus deleted the static-jdk-image branch December 2, 2024 15:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

8339480: Build static-jdk image with a statically linked launcher #20837

8339480: Build static-jdk image with a statically linked launcher #20837

magicus commented Sep 3, 2024 •

edited by openjdk bot

Loading

magicus commented Sep 3, 2024 •

edited by bridgekeeper bot

Loading

bridgekeeper bot commented Sep 3, 2024

magicus commented Sep 3, 2024 •

edited by bridgekeeper bot

Loading

openjdk bot commented Sep 3, 2024 •

edited

Loading

openjdk bot commented Sep 3, 2024

openjdk bot commented Sep 3, 2024

openjdk bot commented Sep 3, 2024

magicus commented Sep 3, 2024 •

edited by bridgekeeper bot

Loading

mlbridge bot commented Sep 3, 2024 •

edited

Loading

erikj79 commented Sep 3, 2024 •

edited by bridgekeeper bot

Loading

magicus commented Sep 3, 2024 •

edited by bridgekeeper bot

Loading

magicus commented Sep 3, 2024 •

edited by bridgekeeper bot

Loading

jianglizhou commented Sep 5, 2024 •

edited by bridgekeeper bot

Loading

magicus commented Sep 5, 2024 •

edited by bridgekeeper bot

Loading

bridgekeeper bot commented Oct 3, 2024

magicus commented Oct 15, 2024 •

edited by bridgekeeper bot

Loading

jianglizhou commented Nov 19, 2024 •

edited by bridgekeeper bot

Loading

dholmes-ora commented Nov 26, 2024

openjdk bot commented Nov 26, 2024

dholmes-ora left a comment

coleenp left a comment

magicus commented Dec 2, 2024

magicus commented Dec 2, 2024

openjdk bot commented Dec 2, 2024

openjdk bot commented Dec 2, 2024

jianglizhou commented Dec 2, 2024

8339480: Build static-jdk image with a statically linked launcher #20837

8339480: Build static-jdk image with a statically linked launcher #20837

Conversation

magicus commented Sep 3, 2024 • edited by openjdk bot Loading

Progress

Issue

Reviewers

Contributors

Reviewing

magicus commented Sep 3, 2024 • edited by bridgekeeper bot Loading

bridgekeeper bot commented Sep 3, 2024

magicus commented Sep 3, 2024 • edited by bridgekeeper bot Loading

openjdk bot commented Sep 3, 2024 • edited Loading

openjdk bot commented Sep 3, 2024

openjdk bot commented Sep 3, 2024

openjdk bot commented Sep 3, 2024

magicus commented Sep 3, 2024 • edited by bridgekeeper bot Loading

mlbridge bot commented Sep 3, 2024 • edited Loading

Webrevs

erikj79 commented Sep 3, 2024 • edited by bridgekeeper bot Loading

magicus commented Sep 3, 2024 • edited by bridgekeeper bot Loading

magicus commented Sep 3, 2024 • edited by bridgekeeper bot Loading

jianglizhou commented Sep 5, 2024 • edited by bridgekeeper bot Loading

magicus commented Sep 5, 2024 • edited by bridgekeeper bot Loading

bridgekeeper bot commented Oct 3, 2024

magicus commented Oct 15, 2024 • edited by bridgekeeper bot Loading

jianglizhou commented Nov 19, 2024 • edited by bridgekeeper bot Loading

dholmes-ora commented Nov 26, 2024

openjdk bot commented Nov 26, 2024

dholmes-ora left a comment

Choose a reason for hiding this comment

coleenp left a comment

Choose a reason for hiding this comment

magicus commented Dec 2, 2024

magicus commented Dec 2, 2024

openjdk bot commented Dec 2, 2024

openjdk bot commented Dec 2, 2024

jianglizhou commented Dec 2, 2024

magicus commented Sep 3, 2024 •

edited by openjdk bot

Loading

magicus commented Sep 3, 2024 •

edited by bridgekeeper bot

Loading

magicus commented Sep 3, 2024 •

edited by bridgekeeper bot

Loading

openjdk bot commented Sep 3, 2024 •

edited

Loading

magicus commented Sep 3, 2024 •

edited by bridgekeeper bot

Loading

mlbridge bot commented Sep 3, 2024 •

edited

Loading

erikj79 commented Sep 3, 2024 •

edited by bridgekeeper bot

Loading

magicus commented Sep 3, 2024 •

edited by bridgekeeper bot

Loading

magicus commented Sep 3, 2024 •

edited by bridgekeeper bot

Loading

jianglizhou commented Sep 5, 2024 •

edited by bridgekeeper bot

Loading

magicus commented Sep 5, 2024 •

edited by bridgekeeper bot

Loading

magicus commented Oct 15, 2024 •

edited by bridgekeeper bot

Loading

jianglizhou commented Nov 19, 2024 •

edited by bridgekeeper bot

Loading