diff --git a/doc/building.html b/doc/building.html index cbc7edd7e10..09ea7dc7f72 100644 --- a/doc/building.html +++ b/doc/building.html @@ -890,30 +890,28 @@ spawn failed

If you need general help or advice about developing for the JDK, you can also contact the Adoption Group. See the section on Contributing to OpenJDK for more information.

Reproducible Builds

Build reproducibility is the property of getting exactly the same bits out when building, every time, independent on who builds the product, or where. This is for many reasons a harder goal than it initially appears, but it is an important goal, for security reasons and others. Please see Reproducible Builds for more information about the background and reasons for reproducible builds.

-

Currently, it is not possible to build OpenJDK fully reproducibly, but getting there is an ongoing effort. There are some things you can do to minimize non-determinism and make a larger part of the build reproducible:

+

Currently, it is not possible to build OpenJDK fully reproducibly, but getting there is an ongoing effort.

+

An absolute prerequisite for building reproducible is to speficy a fixed build time, since time stamps are embedded in many file formats. This is done by setting the SOURCE_DATE_EPOCH environment variable, which is an industry standard, that many tools, such as gcc, recognize, and use in place of the current time when generating output.

+

To generate reproducible builds, you must set SOURCE_DATE_EPOCH before running configure. The value in SOURCE_DATE_EPOCH will be stored in the configuration, and used by make. Setting SOURCE_DATE_EPOCH before running make will have no effect on the build.

+

You must also make sure your build does not rely on configure's default adhoc version strings. Default adhoc version strings OPT segment include user name and source directory. You can either override just the OPT segment using --with-version-opt=<any fixed string>, or you can specify the entire version string using --with-version-string=<your version>.

+

This is a typical example of how to build the JDK in a reproducible way:

+
export SOURCE_DATE_EPOCH=946684800
+bash configure --with-version-opt=adhoc
+make
+

Note that regardless if you specify a source date for configure or not, the JDK build system will set SOURCE_DATE_EPOCH for all build tools when building. If --with-source-date has the value updated (which is the default unless SOURCE_DATE_EPOCH is found by in the environment by configure), the source date value will be determined at build time.

+

There are several aspects of reproducible builds that can be individually adjusted by configure arguments. If any of these are given, they will override the value derived from SOURCE_DATE_EPOCH. These arguments are:

-

Add the flag --enable-reproducible-build to your configure command line. This will turn on support for reproducible builds where it could otherwise be lacking.

- -

Default adhoc version strings OPT segment include user name, source directory and timestamp. You can either override just the OPT segment using --with-version-opt=<any fixed string>, or you can specify the entire version string using --with-version-string=<your version>.

- -

The JDK build system will set the SOURCE_DATE_EPOCH environment variable during building, depending on the value of the --with-source-date option for configure. The default value is updated, which means that SOURCE_DATE_EPOCH will be set to the current time each time you are running make.

-

The SOURCE_DATE_EPOCH environment variable is an industry standard, that many tools, such as gcc, recognize, and use in place of the current time when generating output.

-

For reproducible builds, you need to set this to a fixed value. You can use the special value version which will use the nominal release date for the current JDK version, or a value describing a date, either an epoch based timestamp as an integer, or a valid ISO-8601 date.

-

Hint: If your build environment already sets SOURCE_DATE_EPOCH, you can propagate this using --with-source-date=$SOURCE_DATE_EPOCH.

- -

Set a fixed hotspot build time. This will be included in the hotspot library (libjvm.so or jvm.dll) and defaults to the current time when building hotspot. Use --with-hotspot-build-time=<any fixed string> for reproducible builds. It's a string so you don't need to format it specifically, so e.g. n/a will do. Another solution is to use the SOURCE_DATE_EPOCH variable, e.g. --with-hotspot-build-time=$(date --date=@$SOURCE_DATE_EPOCH).

- -

The copyright year in some generated text files are normally set to the current year. This can be overridden by --with-copyright-year=<year>. For fully reproducible builds, this needs to be set to a fixed value.

Hints and Suggestions for Advanced Users

Bash Completion

The configure and make commands tries to play nice with bash command-line completion (using <tab> or <tab><tab>). To use this functionality, make sure you enable completion in your ~/.bashrc (see instructions for bash in your operating system).

diff --git a/doc/building.md b/doc/building.md index f71e4e2e730..e83c2bada21 100644 --- a/doc/building.md +++ b/doc/building.md @@ -1527,57 +1527,85 @@ https://reproducible-builds.org) for more information about the background and reasons for reproducible builds. Currently, it is not possible to build OpenJDK fully reproducibly, but getting -there is an ongoing effort. There are some things you can do to minimize -non-determinism and make a larger part of the build reproducible: +there is an ongoing effort. - * Turn on build system support for reproducible builds +An absolute prerequisite for building reproducible is to speficy a fixed build +time, since time stamps are embedded in many file formats. This is done by +setting the `SOURCE_DATE_EPOCH` environment variable, which is an [industry +standard]( https://reproducible-builds.org/docs/source-date-epoch/), that many +tools, such as gcc, recognize, and use in place of the current time when +generating output. -Add the flag `--enable-reproducible-build` to your `configure` command line. -This will turn on support for reproducible builds where it could otherwise be -lacking. +To generate reproducible builds, you must set `SOURCE_DATE_EPOCH` before running +`configure`. The value in `SOURCE_DATE_EPOCH` will be stored in the +configuration, and used by `make`. Setting `SOURCE_DATE_EPOCH` before running +`make` will have no effect on the build. - * Do not rely on `configure`'s default adhoc version strings - -Default adhoc version strings OPT segment include user name, source directory -and timestamp. You can either override just the OPT segment using +You must also make sure your build does not rely on `configure`'s default adhoc +version strings. Default adhoc version strings `OPT` segment include user name +and source directory. You can either override just the `OPT` segment using `--with-version-opt=`, or you can specify the entire version string using `--with-version-string=`. - * Specify how the build sets `SOURCE_DATE_EPOCH` +This is a typical example of how to build the JDK in a reproducible way: -The JDK build system will set the `SOURCE_DATE_EPOCH` environment variable -during building, depending on the value of the `--with-source-date` option for -`configure`. The default value is `updated`, which means that -`SOURCE_DATE_EPOCH` will be set to the current time each time you are running -`make`. +``` +export SOURCE_DATE_EPOCH=946684800 +bash configure --with-version-opt=adhoc +make +``` -The [`SOURCE_DATE_EPOCH` environment variable]( -https://reproducible-builds.org/docs/source-date-epoch/) is an industry -standard, that many tools, such as gcc, recognize, and use in place of the -current time when generating output. +Note that regardless if you specify a source date for `configure` or not, the +JDK build system will set `SOURCE_DATE_EPOCH` for all build tools when building. +If `--with-source-date` has the value `updated` (which is the default unless +`SOURCE_DATE_EPOCH` is found by in the environment by `configure`), the source +date value will be determined at build time. -For reproducible builds, you need to set this to a fixed value. You can use the -special value `version` which will use the nominal release date for the current -JDK version, or a value describing a date, either an epoch based timestamp as an -integer, or a valid ISO-8601 date. +There are several aspects of reproducible builds that can be individually +adjusted by `configure` arguments. If any of these are given, they will override +the value derived from `SOURCE_DATE_EPOCH`. These arguments are: -**Hint:** If your build environment already sets `SOURCE_DATE_EPOCH`, you can -propagate this using `--with-source-date=$SOURCE_DATE_EPOCH`. + * `--with-source-date` - * Specify a hotspot build time + This option controls how the JDK build sets `SOURCE_DATE_EPOCH` when + building. It can be set to a value describing a date, either an epoch based + timestamp as an integer, or a valid ISO-8601 date. -Set a fixed hotspot build time. This will be included in the hotspot library -(`libjvm.so` or `jvm.dll`) and defaults to the current time when building -hotspot. Use `--with-hotspot-build-time=` for reproducible -builds. It's a string so you don't need to format it specifically, so e.g. `n/a` -will do. Another solution is to use the `SOURCE_DATE_EPOCH` variable, e.g. -`--with-hotspot-build-time=$(date --date=@$SOURCE_DATE_EPOCH)`. + It can also be set to one of the special values `current`, `updated` or + `version`. `current` means that the time of running `configure` will be + used. `version` will use the nominal release date for the current JDK + version. `updated`, which means that `SOURCE_DATE_EPOCH` will be set to the + current time each time you are running `make`. All choices, except for + `updated`, will set a fixed value for the source date timestamp. - * Copyright year + When `SOURCE_DATE_EPOCH` is set, the default value for `--with-source-date` + will be the value given by `SOURCE_DATE_EPOCH`. Otherwise, the default value + is `updated`. -The copyright year in some generated text files are normally set to the current -year. This can be overridden by `--with-copyright-year=`. For fully -reproducible builds, this needs to be set to a fixed value. + * `--with-hotspot-build-time` + + This option controls the build time string that will be included in the + hotspot library (`libjvm.so` or `jvm.dll`). When the source date is fixed + (e.g. by setting `SOURCE_DATE_EPOCH`), the default value for + `--with-hotspot-build-time` will be an ISO 8601 representation of that time + stamp. Otherwise the default value will be the current time when building + hotspot. + + * `--with-copyright-year` + + This option controls the copyright year in some generated text files. When + the source date is fixed (e.g. by setting `SOURCE_DATE_EPOCH`), the default + value for `--with-copyright-year` will be the year of that time stamp. + Otherwise the default is the current year at the time of running configure. + This can be overridden by `--with-copyright-year=`. + + * `--enable-reproducible-build` + + This option controls some additional behavior needed to make the build + reproducible. When the source date is fixed (e.g. by setting + `SOURCE_DATE_EPOCH`), this flag will be turned on by default. Otherwise, the + value is determined by heuristics. If it is explicitly turned off, the build + might not be reproducible. ## Hints and Suggestions for Advanced Users diff --git a/make/InitSupport.gmk b/make/InitSupport.gmk index d2291c50f21..62fdc438c8a 100644 --- a/make/InitSupport.gmk +++ b/make/InitSupport.gmk @@ -1,5 +1,5 @@ # -# Copyright (c) 2011, 2021, Oracle and/or its affiliates. All rights reserved. +# Copyright (c) 2011, 2022, Oracle and/or its affiliates. All rights reserved. # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. # # This code is free software; you can redistribute it and/or modify it @@ -310,17 +310,16 @@ else # $(HAS_SPEC)=true # level of reproducible builds define SetupReproducibleBuild ifeq ($$(SOURCE_DATE), updated) - SOURCE_DATE := $$(shell $$(DATE) +"%s") - endif - export SOURCE_DATE_EPOCH := $$(SOURCE_DATE) - ifeq ($$(IS_GNU_DATE), yes) - export SOURCE_DATE_ISO_8601 := $$(shell $$(DATE) --utc \ - --date="@$$(SOURCE_DATE_EPOCH)" \ - +"%Y-%m-%dT%H:%M:%SZ" 2> /dev/null) - else - export SOURCE_DATE_ISO_8601 := $$(shell $$(DATE) -u \ - -j -f "%s" "$$(SOURCE_DATE_EPOCH)" \ - +"%Y-%m-%dT%H:%M:%SZ" 2> /dev/null) + # For static values of SOURCE_DATE (not "updated"), these are set in spec.gmk + export SOURCE_DATE_EPOCH := $$(shell $$(DATE) +"%s") + ifeq ($$(IS_GNU_DATE), yes) + export SOURCE_DATE_ISO_8601 := $$(shell $$(DATE) --utc \ + --date="@$$(SOURCE_DATE_EPOCH)" +"$$(ISO_8601_FORMAT_STRING)" \ + 2> /dev/null) + else + export SOURCE_DATE_ISO_8601 := $$(shell $$(DATE) -u -j -f "%s" \ + "$$(SOURCE_DATE_EPOCH)" +"$$(ISO_8601_FORMAT_STRING)" 2> /dev/null) + endif endif endef diff --git a/make/autoconf/configure.ac b/make/autoconf/configure.ac index d52ad2c4315..5e58696e34d 100644 --- a/make/autoconf/configure.ac +++ b/make/autoconf/configure.ac @@ -152,6 +152,7 @@ BOOTJDK_SETUP_DOCS_REFERENCE_JDK # ############################################################################### +JDKOPT_SETUP_REPRODUCIBLE_BUILD JDKOPT_SETUP_JDK_OPTIONS ############################################################################### @@ -207,7 +208,6 @@ PLATFORM_SETUP_OPENJDK_TARGET_BITS PLATFORM_SETUP_OPENJDK_TARGET_ENDIANNESS # Configure flags for the tools. Need to know if we should build reproducible. -JDKOPT_SETUP_REPRODUCIBLE_BUILD FLAGS_SETUP_FLAGS # Setup debug symbols (need objcopy from the toolchain for that) diff --git a/make/autoconf/help.m4 b/make/autoconf/help.m4 index 486e78d4fcf..3d6963c7d4d 100644 --- a/make/autoconf/help.m4 +++ b/make/autoconf/help.m4 @@ -296,6 +296,13 @@ AC_DEFUN_ONCE([HELP_PRINT_SUMMARY_AND_WARNINGS], printf "* OpenJDK target: OS: $OPENJDK_TARGET_OS, CPU architecture: $OPENJDK_TARGET_CPU_ARCH, address length: $OPENJDK_TARGET_CPU_BITS\n" printf "* Version string: $VERSION_STRING ($VERSION_SHORT)\n" + if test "x$SOURCE_DATE" != xupdated; then + source_date_info="$SOURCE_DATE ($SOURCE_DATE_ISO_8601)" + else + source_date_info="Determined at build time" + fi + printf "* Source date: $source_date_info\n" + printf "\n" printf "Tools summary:\n" if test "x$OPENJDK_BUILD_OS" = "xwindows"; then diff --git a/make/autoconf/hotspot.m4 b/make/autoconf/hotspot.m4 index 1cac6bb00c6..18f46036fe5 100644 --- a/make/autoconf/hotspot.m4 +++ b/make/autoconf/hotspot.m4 @@ -1,5 +1,5 @@ # -# Copyright (c) 2011, 2020, Oracle and/or its affiliates. All rights reserved. +# Copyright (c) 2011, 2022, Oracle and/or its affiliates. All rights reserved. # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. # # This code is free software; you can redistribute it and/or modify it @@ -114,12 +114,26 @@ AC_DEFUN_ONCE([HOTSPOT_SETUP_MISC], HOTSPOT_TARGET_CPU_ARCH=zero fi + AC_ARG_WITH([hotspot-build-time], [AS_HELP_STRING([--with-hotspot-build-time], - [timestamp to use in hotspot version string, empty for on-the-fly @<:@empty@:>@])]) + [timestamp to use in hotspot version string, empty means determined at build time @<:@source-date/empty@:>@])]) + + AC_MSG_CHECKING([what hotspot build time to use]) if test "x$with_hotspot_build_time" != x; then HOTSPOT_BUILD_TIME="$with_hotspot_build_time" + AC_MSG_RESULT([$HOTSPOT_BUILD_TIME (from --with-hotspot-build-time)]) + else + if test "x$SOURCE_DATE" = xupdated; then + HOTSPOT_BUILD_TIME="" + AC_MSG_RESULT([determined at build time (default)]) + else + # If we have a fixed value for SOURCE_DATE, use it as default + HOTSPOT_BUILD_TIME="$SOURCE_DATE_ISO_8601" + AC_MSG_RESULT([$HOTSPOT_BUILD_TIME (from --with-source-date)]) + fi fi + AC_SUBST(HOTSPOT_BUILD_TIME) diff --git a/make/autoconf/jdk-options.m4 b/make/autoconf/jdk-options.m4 index 70806149203..2034934cd73 100644 --- a/make/autoconf/jdk-options.m4 +++ b/make/autoconf/jdk-options.m4 @@ -211,16 +211,16 @@ AC_DEFUN_ONCE([JDKOPT_SETUP_JDK_OPTIONS], # Setup default copyright year. Mostly overridden when building close to a new year. AC_ARG_WITH(copyright-year, [AS_HELP_STRING([--with-copyright-year], - [Set copyright year value for build @<:@current year@:>@])]) + [Set copyright year value for build @<:@current year/source-date@:>@])]) if test "x$with_copyright_year" = xyes; then AC_MSG_ERROR([Copyright year must have a value]) elif test "x$with_copyright_year" != x; then COPYRIGHT_YEAR="$with_copyright_year" - elif test "x$SOURCE_DATE_EPOCH" != x; then + elif test "x$SOURCE_DATE" != xupdated; then if test "x$IS_GNU_DATE" = xyes; then - COPYRIGHT_YEAR=`date --date=@$SOURCE_DATE_EPOCH +%Y` + COPYRIGHT_YEAR=`$DATE --date=@$SOURCE_DATE +%Y` else - COPYRIGHT_YEAR=`date -j -f %s $SOURCE_DATE_EPOCH +%Y` + COPYRIGHT_YEAR=`$DATE -j -f %s $SOURCE_DATE +%Y` fi else COPYRIGHT_YEAR=`$DATE +'%Y'` @@ -662,15 +662,28 @@ AC_DEFUN([JDKOPT_ALLOW_ABSOLUTE_PATHS_IN_OUTPUT], AC_DEFUN_ONCE([JDKOPT_SETUP_REPRODUCIBLE_BUILD], [ AC_ARG_WITH([source-date], [AS_HELP_STRING([--with-source-date], - [how to set SOURCE_DATE_EPOCH ('updated', 'current', 'version' a timestamp or an ISO-8601 date) @<:@updated@:>@])], + [how to set SOURCE_DATE_EPOCH ('updated', 'current', 'version' a timestamp or an ISO-8601 date) @<:@updated/value of SOURCE_DATE_EPOCH@:>@])], [with_source_date_present=true], [with_source_date_present=false]) + if test "x$SOURCE_DATE_EPOCH" != x && test "x$with_source_date" != x; then + AC_MSG_WARN([--with-source-date will override SOURCE_DATE_EPOCH]) + fi + AC_MSG_CHECKING([what source date to use]) if test "x$with_source_date" = xyes; then AC_MSG_ERROR([--with-source-date must have a value]) - elif test "x$with_source_date" = xupdated || test "x$with_source_date" = x; then - # Tell the makefiles to update at each build + elif test "x$with_source_date" = x; then + if test "x$SOURCE_DATE_EPOCH" != x; then + SOURCE_DATE=$SOURCE_DATE_EPOCH + with_source_date_present=true + AC_MSG_RESULT([$SOURCE_DATE, from SOURCE_DATE_EPOCH]) + else + # Tell the makefiles to update at each build + SOURCE_DATE=updated + AC_MSG_RESULT([determined at build time (default)]) + fi + elif test "x$with_source_date" = xupdated; then SOURCE_DATE=updated AC_MSG_RESULT([determined at build time, from 'updated']) elif test "x$with_source_date" = xcurrent; then @@ -702,6 +715,18 @@ AC_DEFUN_ONCE([JDKOPT_SETUP_REPRODUCIBLE_BUILD], fi fi + ISO_8601_FORMAT_STRING="%Y-%m-%dT%H:%M:%SZ" + if test "x$SOURCE_DATE" != xupdated; then + # If we have a fixed value for SOURCE_DATE, we need to set SOURCE_DATE_EPOCH + # for the rest of configure. + SOURCE_DATE_EPOCH="$SOURCE_DATE" + if test "x$IS_GNU_DATE" = xyes; then + SOURCE_DATE_ISO_8601=`$DATE --utc --date="@$SOURCE_DATE" +"$ISO_8601_FORMAT_STRING" 2> /dev/null` + else + SOURCE_DATE_ISO_8601=`$DATE -u -j -f "%s" "$SOURCE_DATE" +"$ISO_8601_FORMAT_STRING" 2> /dev/null` + fi + fi + REPRODUCIBLE_BUILD_DEFAULT=$with_source_date_present if test "x$OPENJDK_BUILD_OS" = xwindows && \ @@ -726,4 +751,6 @@ AC_DEFUN_ONCE([JDKOPT_SETUP_REPRODUCIBLE_BUILD], AC_SUBST(SOURCE_DATE) AC_SUBST(ENABLE_REPRODUCIBLE_BUILD) + AC_SUBST(ISO_8601_FORMAT_STRING) + AC_SUBST(SOURCE_DATE_ISO_8601) ]) diff --git a/make/autoconf/spec.gmk.in b/make/autoconf/spec.gmk.in index 01bcc32a24e..5671d4a9f3e 100644 --- a/make/autoconf/spec.gmk.in +++ b/make/autoconf/spec.gmk.in @@ -130,6 +130,13 @@ RELEASE_FILE_LIBC:=@RELEASE_FILE_LIBC@ SOURCE_DATE := @SOURCE_DATE@ ENABLE_REPRODUCIBLE_BUILD := @ENABLE_REPRODUCIBLE_BUILD@ +ISO_8601_FORMAT_STRING := @ISO_8601_FORMAT_STRING@ + +ifneq ($(SOURCE_DATE), updated) + # For "updated" source date value, these are set in InitSupport.gmk + export SOURCE_DATE_EPOCH := $(SOURCE_DATE) + SOURCE_DATE_ISO_8601 := @SOURCE_DATE_ISO_8601@ +endif LIBM:=@LIBM@ LIBDL:=@LIBDL@