8210958: Rename "make run-test" to "make test"
Reviewed-by: erikj
This commit is contained in:
Magnus Ihse Bursie
parent
80aa38305d
commit
53ecc07aaa
@ -18,12 +18,13 @@
|
||||
</header>
|
||||
<nav id="TOC">
|
||||
<ul>
|
||||
<li><a href="#using-the-run-test-framework">Using the run-test framework</a><ul>
|
||||
<li><a href="#using-make-test-the-run-test-framework">Using "make test" (the run-test framework)</a><ul>
|
||||
<li><a href="#configuration">Configuration</a></li>
|
||||
</ul></li>
|
||||
<li><a href="#test-selection">Test selection</a><ul>
|
||||
<li><a href="#jtreg">JTReg</a></li>
|
||||
<li><a href="#gtest">Gtest</a></li>
|
||||
<li><a href="#special-tests">Special tests</a></li>
|
||||
</ul></li>
|
||||
<li><a href="#test-results-and-summary">Test results and summary</a></li>
|
||||
<li><a href="#test-suite-control">Test suite control</a><ul>
|
||||
@ -32,22 +33,23 @@
|
||||
</ul></li>
|
||||
</ul>
|
||||
</nav>
|
||||
<h2 id="using-the-run-test-framework">Using the run-test framework</h2>
|
||||
<h2 id="using-make-test-the-run-test-framework">Using "make test" (the run-test framework)</h2>
|
||||
<p>This new way of running tests is developer-centric. It assumes that you have built a JDK locally and want to test it. Running common test targets is simple, and more complex ad-hoc combination of tests is possible. The user interface is forgiving, and clearly report errors it cannot resolve.</p>
|
||||
<p>The main target "run-test" uses the jdk-image as the tested product. There is also an alternate target "exploded-run-test" that uses the exploded image instead. Not all tests will run successfully on the exploded image, but using this target can greatly improve rebuild times for certain workflows.</p>
|
||||
<p>The main target <code>test</code> uses the jdk-image as the tested product. There is also an alternate target <code>exploded-test</code> that uses the exploded image instead. Not all tests will run successfully on the exploded image, but using this target can greatly improve rebuild times for certain workflows.</p>
|
||||
<p>Previously, <code>make test</code> was used invoke an old system for running test, and <code>make run-test</code> was used for the new test framework. For backward compatibility with scripts and muscle memory, <code>run-test</code> (and variants like <code>exploded-run-test</code> or <code>run-test-tier1</code>) are kept as aliases. The old system can still be accessed for some time using <code>cd test && make</code>.</p>
|
||||
<p>Some example command-lines:</p>
|
||||
<pre><code>$ make run-test-tier1
|
||||
$ make run-test-jdk_lang JTREG="JOBS=8"
|
||||
$ make run-test TEST=jdk_lang
|
||||
$ make run-test-only TEST="gtest:LogTagSet gtest:LogTagSetDescriptions" GTEST="REPEAT=-1"
|
||||
$ make run-test TEST="hotspot:hotspot_gc" JTREG="JOBS=1;TIMEOUT=8;VM_OPTIONS=-XshowSettings -Xlog:gc+ref=debug"
|
||||
$ make run-test TEST="jtreg:test/hotspot:hotspot_gc test/hotspot/jtreg/native_sanity/JniVersion.java"
|
||||
$ make exploded-run-test TEST=tier2</code></pre>
|
||||
<pre><code>$ make test-tier1
|
||||
$ make test-jdk_lang JTREG="JOBS=8"
|
||||
$ make test TEST=jdk_lang
|
||||
$ make test-only TEST="gtest:LogTagSet gtest:LogTagSetDescriptions" GTEST="REPEAT=-1"
|
||||
$ make test TEST="hotspot:hotspot_gc" JTREG="JOBS=1;TIMEOUT=8;VM_OPTIONS=-XshowSettings -Xlog:gc+ref=debug"
|
||||
$ make test TEST="jtreg:test/hotspot:hotspot_gc test/hotspot/jtreg/native_sanity/JniVersion.java"
|
||||
$ make exploded-test TEST=tier2</code></pre>
|
||||
<h3 id="configuration">Configuration</h3>
|
||||
<p>To be able to run JTReg tests, <code>configure</code> needs to know where to find the JTReg test framework. If it is not picked up automatically by configure, use the <code>--with-jtreg=<path to jtreg home></code> option to point to the JTReg framework. Note that this option should point to the JTReg home, i.e. the top directory, containing <code>lib/jtreg.jar</code> etc. (An alternative is to set the <code>JT_HOME</code> environment variable to point to the JTReg home before running <code>configure</code>.)</p>
|
||||
<h2 id="test-selection">Test selection</h2>
|
||||
<p>All functionality is available using the run-test make target. In this use case, the test or tests to be executed is controlled using the <code>TEST</code> variable. To speed up subsequent test runs with no source code changes, run-test-only can be used instead, which do not depend on the source and test image build.</p>
|
||||
<p>For some common top-level tests, direct make targets have been generated. This includes all JTReg test groups, the hotspot gtest, and custom tests (if present). This means that <code>make run-test-tier1</code> is equivalent to <code>make run-test TEST="tier1"</code>, but the latter is more tab-completion friendly. For more complex test runs, the <code>run-test TEST="x"</code> solution needs to be used.</p>
|
||||
<p>All functionality is available using the <code>test</code> make target. In this use case, the test or tests to be executed is controlled using the <code>TEST</code> variable. To speed up subsequent test runs with no source code changes, <code>test-only</code> can be used instead, which do not depend on the source and test image build.</p>
|
||||
<p>For some common top-level tests, direct make targets have been generated. This includes all JTReg test groups, the hotspot gtest, and custom tests (if present). This means that <code>make test-tier1</code> is equivalent to <code>make test TEST="tier1"</code>, but the latter is more tab-completion friendly. For more complex test runs, the <code>test TEST="x"</code> solution needs to be used.</p>
|
||||
<p>The test specifications given in <code>TEST</code> is parsed into fully qualified test descriptors, which clearly and unambigously show which tests will be run. As an example, <code>:tier1</code> will expand to <code>jtreg:$(TOPDIR)/test/hotspot/jtreg:tier1 jtreg:$(TOPDIR)/test/jdk:tier1 jtreg:$(TOPDIR)/test/langtools:tier1 jtreg:$(TOPDIR)/test/nashorn:tier1 jtreg:$(TOPDIR)/test/jaxp:tier1</code>. You can always submit a list of fully qualified test descriptors in the <code>TEST</code> variable if you want to shortcut the parser.</p>
|
||||
<h3 id="jtreg">JTReg</h3>
|
||||
<p>JTReg tests can be selected either by picking a JTReg test group, or a selection of files or directories containing JTReg tests.</p>
|
||||
@ -59,6 +61,14 @@ $ make exploded-run-test TEST=tier2</code></pre>
|
||||
<p>Since the Hotspot Gtest suite is so quick, the default is to run all tests. This is specified by just <code>gtest</code>, or as a fully qualified test descriptor <code>gtest:all</code>.</p>
|
||||
<p>If you want, you can single out an individual test or a group of tests, for instance <code>gtest:LogDecorations</code> or <code>gtest:LogDecorations.level_test_vm</code>. This can be particularly useful if you want to run a shaky test repeatedly.</p>
|
||||
<p>For Gtest, there is a separate test suite for each JVM variant. The JVM variant is defined by adding <code>/<variant></code> to the test descriptor, e.g. <code>gtest:Log/client</code>. If you specify no variant, gtest will run once for each JVM variant present (e.g. server, client). So if you only have the server JVM present, then <code>gtest:all</code> will be equivalent to <code>gtest:all/server</code>.</p>
|
||||
<h3 id="special-tests">Special tests</h3>
|
||||
<p>A handful of odd tests that are not covered by any other testing framework are accessible using the <code>special:</code> test descriptor. Currently, this includes <code>hotspot-internal</code>, <code>failure-handler</code> and <code>make</code>.</p>
|
||||
<ul>
|
||||
<li><p>Hotspot legacy internal testing (run using <code>-XX:+ExecuteInternalVMTests</code>) is run using <code>special:hotspot-internal</code> or just <code>hotspot-internal</code> as test descriptor, and will only work on a debug JVM.</p></li>
|
||||
<li><p>Failure handler testing is run using <code>special:failure-handler</code> or just <code>failure-handler</code> as test descriptor.</p></li>
|
||||
<li><p>Tests for the build system, including both makefiles and related functionality, is run using <code>special:make</code> or just <code>make</code> as test descriptor. This is equivalent to <code>special:make:all</code>.</p>
|
||||
<p>A specific make test can be run by supplying it as argument, e.g. <code>special:make:idea</code>. As a special syntax, this can also be expressed as <code>make-idea</code>, which allows for command lines as <code>make test-make-idea</code>.</p></li>
|
||||
</ul>
|
||||
<h2 id="test-results-and-summary">Test results and summary</h2>
|
||||
<p>At the end of the test run, a summary of all tests run will be presented. This will have a consistent look, regardless of what test suites were used. This is a sample summary:</p>
|
||||
<pre><code>==============================
|
||||
@ -72,7 +82,7 @@ Test summary
|
||||
TEST FAILURE</code></pre>
|
||||
<p>Tests where the number of TOTAL tests does not equal the number of PASSed tests will be considered a test failure. These are marked with the <code>>> ... <<</code> marker for easy identification.</p>
|
||||
<p>The classification of non-passed tests differs a bit between test suites. In the summary, ERROR is used as a catch-all for tests that neither passed nor are classified as failed by the framework. This might indicate test framework error, timeout or other problems.</p>
|
||||
<p>In case of test failures, <code>make run-test</code> will exit with a non-zero exit value.</p>
|
||||
<p>In case of test failures, <code>make test</code> will exit with a non-zero exit value.</p>
|
||||
<p>All tests have their result stored in <code>build/$BUILD/test-results/$TEST_ID</code>, where TEST_ID is a path-safe conversion from the fully qualified test descriptor, e.g. for <code>jtreg:jdk/test:tier1</code> the TEST_ID is <code>jtreg_jdk_test_tier1</code>. This path is also printed in the log at the end of the test run.</p>
|
||||
<p>Additional work data is stored in <code>build/$BUILD/test-support/$TEST_ID</code>. For some frameworks, this directory might contain information that is useful in determining the cause of a failed test.</p>
|
||||
<h2 id="test-suite-control">Test suite control</h2>
|
||||
|
||||
@ -1,26 +1,32 @@
|
||||
% Testing the JDK
|
||||
|
||||
## Using the run-test framework
|
||||
## Using "make test" (the run-test framework)
|
||||
|
||||
This new way of running tests is developer-centric. It assumes that you have
|
||||
built a JDK locally and want to test it. Running common test targets is simple,
|
||||
and more complex ad-hoc combination of tests is possible. The user interface is
|
||||
forgiving, and clearly report errors it cannot resolve.
|
||||
|
||||
The main target "run-test" uses the jdk-image as the tested product. There is
|
||||
also an alternate target "exploded-run-test" that uses the exploded image
|
||||
The main target `test` uses the jdk-image as the tested product. There is
|
||||
also an alternate target `exploded-test` that uses the exploded image
|
||||
instead. Not all tests will run successfully on the exploded image, but using
|
||||
this target can greatly improve rebuild times for certain workflows.
|
||||
|
||||
Previously, `make test` was used invoke an old system for running test, and
|
||||
`make run-test` was used for the new test framework. For backward compatibility
|
||||
with scripts and muscle memory, `run-test` (and variants like
|
||||
`exploded-run-test` or `run-test-tier1`) are kept as aliases. The old system
|
||||
can still be accessed for some time using `cd test && make`.
|
||||
|
||||
Some example command-lines:
|
||||
|
||||
$ make run-test-tier1
|
||||
$ make run-test-jdk_lang JTREG="JOBS=8"
|
||||
$ make run-test TEST=jdk_lang
|
||||
$ make run-test-only TEST="gtest:LogTagSet gtest:LogTagSetDescriptions" GTEST="REPEAT=-1"
|
||||
$ make run-test TEST="hotspot:hotspot_gc" JTREG="JOBS=1;TIMEOUT=8;VM_OPTIONS=-XshowSettings -Xlog:gc+ref=debug"
|
||||
$ make run-test TEST="jtreg:test/hotspot:hotspot_gc test/hotspot/jtreg/native_sanity/JniVersion.java"
|
||||
$ make exploded-run-test TEST=tier2
|
||||
$ make test-tier1
|
||||
$ make test-jdk_lang JTREG="JOBS=8"
|
||||
$ make test TEST=jdk_lang
|
||||
$ make test-only TEST="gtest:LogTagSet gtest:LogTagSetDescriptions" GTEST="REPEAT=-1"
|
||||
$ make test TEST="hotspot:hotspot_gc" JTREG="JOBS=1;TIMEOUT=8;VM_OPTIONS=-XshowSettings -Xlog:gc+ref=debug"
|
||||
$ make test TEST="jtreg:test/hotspot:hotspot_gc test/hotspot/jtreg/native_sanity/JniVersion.java"
|
||||
$ make exploded-test TEST=tier2
|
||||
|
||||
### Configuration
|
||||
|
||||
@ -33,16 +39,16 @@ environment variable to point to the JTReg home before running `configure`.)
|
||||
|
||||
## Test selection
|
||||
|
||||
All functionality is available using the run-test make target. In this use
|
||||
case, the test or tests to be executed is controlled using the `TEST` variable.
|
||||
To speed up subsequent test runs with no source code changes, run-test-only can
|
||||
be used instead, which do not depend on the source and test image build.
|
||||
All functionality is available using the `test` make target. In this use case,
|
||||
the test or tests to be executed is controlled using the `TEST` variable. To
|
||||
speed up subsequent test runs with no source code changes, `test-only` can be
|
||||
used instead, which do not depend on the source and test image build.
|
||||
|
||||
For some common top-level tests, direct make targets have been generated. This
|
||||
includes all JTReg test groups, the hotspot gtest, and custom tests (if
|
||||
present). This means that `make run-test-tier1` is equivalent to `make run-test
|
||||
present). This means that `make test-tier1` is equivalent to `make test
|
||||
TEST="tier1"`, but the latter is more tab-completion friendly. For more complex
|
||||
test runs, the `run-test TEST="x"` solution needs to be used.
|
||||
test runs, the `test TEST="x"` solution needs to be used.
|
||||
|
||||
The test specifications given in `TEST` is parsed into fully qualified test
|
||||
descriptors, which clearly and unambigously show which tests will be run. As an
|
||||
@ -98,6 +104,27 @@ is defined by adding `/<variant>` to the test descriptor, e.g.
|
||||
variant present (e.g. server, client). So if you only have the server JVM
|
||||
present, then `gtest:all` will be equivalent to `gtest:all/server`.
|
||||
|
||||
### Special tests
|
||||
|
||||
A handful of odd tests that are not covered by any other testing framework are
|
||||
accessible using the `special:` test descriptor. Currently, this includes
|
||||
`hotspot-internal`, `failure-handler` and `make`.
|
||||
|
||||
* Hotspot legacy internal testing (run using `-XX:+ExecuteInternalVMTests`)
|
||||
is run using `special:hotspot-internal` or just `hotspot-internal` as test
|
||||
descriptor, and will only work on a debug JVM.
|
||||
|
||||
* Failure handler testing is run using `special:failure-handler` or just
|
||||
`failure-handler` as test descriptor.
|
||||
|
||||
* Tests for the build system, including both makefiles and related
|
||||
functionality, is run using `special:make` or just `make` as test
|
||||
descriptor. This is equivalent to `special:make:all`.
|
||||
|
||||
A specific make test can be run by supplying it as argument, e.g.
|
||||
`special:make:idea`. As a special syntax, this can also be expressed as
|
||||
`make-idea`, which allows for command lines as `make test-make-idea`.
|
||||
|
||||
## Test results and summary
|
||||
|
||||
At the end of the test run, a summary of all tests run will be presented. This
|
||||
@ -123,7 +150,7 @@ the summary, ERROR is used as a catch-all for tests that neither passed nor are
|
||||
classified as failed by the framework. This might indicate test framework
|
||||
error, timeout or other problems.
|
||||
|
||||
In case of test failures, `make run-test` will exit with a non-zero exit value.
|
||||
In case of test failures, `make test` will exit with a non-zero exit value.
|
||||
|
||||
All tests have their result stored in `build/$BUILD/test-results/$TEST_ID`,
|
||||
where TEST_ID is a path-safe conversion from the fully qualified test
|
||||
|
||||
@ -1,5 +1,5 @@
|
||||
#
|
||||
# Copyright (c) 2012, 2017, Oracle and/or its affiliates. All rights reserved.
|
||||
# Copyright (c) 2012, 2018, Oracle and/or its affiliates. All rights reserved.
|
||||
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
|
||||
#
|
||||
# This code is free software; you can redistribute it and/or modify it
|
||||
@ -55,10 +55,10 @@ help:
|
||||
$(info $(_) make install # Install the generated images locally)
|
||||
$(info $(_) make reconfigure # Rerun configure with the same arguments as last time)
|
||||
$(info $(_) make help # Give some help on using make)
|
||||
$(info $(_) make test # Run tests, default is all tests (see TEST below))
|
||||
$(info $(_) make run-test-<test> # Run test, e.g. run-test-tier1)
|
||||
$(info $(_) make run-test TEST=<t> # Run test(s) given by TEST specification)
|
||||
$(info $(_) make exploded-run-test TEST=<t> # Run test(s) on the exploded image instead of)
|
||||
$(info $(_) make check # Run basic testing (currently tier1))
|
||||
$(info $(_) make test-<test> # Run test, e.g. test-tier1)
|
||||
$(info $(_) make test TEST=<t> # Run test(s) given by TEST specification)
|
||||
$(info $(_) make exploded-test TEST=<t> # Run test(s) on the exploded image instead of)
|
||||
$(info $(_) # the full jdk image)
|
||||
$(info )
|
||||
$(info Targets for cleaning)
|
||||
@ -99,10 +99,12 @@ help:
|
||||
$(info $(_) TEST_JOBS=<n> # Run <n> parallel test jobs)
|
||||
$(info $(_) CONF_CHECK=<method> # What to do if spec file is out of date)
|
||||
$(info $(_) # method is 'auto', 'ignore' or 'fail' (default))
|
||||
$(info $(_) make test TEST=<test> # Only run the given test or tests, e.g.)
|
||||
$(info $(_) # make test TEST="jdk_lang jdk_net")
|
||||
$(info $(_) JTREG="OPT1=x;OPT2=y" # Control the JTREG test harness for run-test)
|
||||
$(info $(_) GTEST="OPT1=x;OPT2=y" # Control the GTEST test harness for run-test)
|
||||
$(info $(_) TEST="test1 ..." # Use the given test descriptor(s) for testing, e.g.)
|
||||
$(info $(_) # make test TEST="jdk_lang gtest:all")
|
||||
$(info $(_) JTREG="OPT1=x;OPT2=y" # Control the JTREG test harness)
|
||||
$(info $(_) GTEST="OPT1=x;OPT2=y" # Control the GTEST test harness)
|
||||
$(info $(_) TEST_OPTS="OPT1=x;..." # Generic control of all test harnesses)
|
||||
$(info $(_) TEST_VM_OPTS="ARG ..." # Same as setting TEST_OPTS to VM_OPTIONS="ARG ...")
|
||||
$(info )
|
||||
$(if $(all_confs), $(info Available configurations in $(build_dir):) $(foreach var,$(all_confs),$(info * $(var))),\
|
||||
$(info No configurations were found in $(build_dir).) $(info Run 'bash configure' to create a configuration.))
|
||||
|
||||
101
make/Main.gmk
101
make/Main.gmk
@ -473,10 +473,10 @@ ALL_TARGETS += $(INTERIM_JMOD_TARGETS) interim-image generate-link-opt-data
|
||||
#
|
||||
|
||||
define DeclareRunTestRecipe
|
||||
run-test-$1:
|
||||
test-$1:
|
||||
+($(CD) $(TOPDIR)/make && $(MAKE) $(MAKE_ARGS) -f RunTests.gmk run-test TEST="$1")
|
||||
|
||||
exploded-run-test-$1:
|
||||
exploded-test-$1:
|
||||
+($(CD) $(TOPDIR)/make && $(MAKE) $(MAKE_ARGS) -f RunTests.gmk run-test \
|
||||
TEST="$1" JDK_IMAGE_DIR=$(JDK_OUTPUTDIR))
|
||||
|
||||
@ -484,8 +484,8 @@ endef
|
||||
|
||||
# ALL_NAMED_TESTS is defined in FindTests.gmk
|
||||
$(foreach t, $(ALL_NAMED_TESTS), $(eval $(call DeclareRunTestRecipe,$t)))
|
||||
ALL_TEST_TARGETS := $(addprefix run-test-, $(ALL_NAMED_TESTS))
|
||||
ALL_EXPLODED_TEST_TARGETS := $(addprefix exploded-run-test-, $(ALL_NAMED_TESTS))
|
||||
ALL_TEST_TARGETS := $(addprefix test-, $(ALL_NAMED_TESTS))
|
||||
ALL_EXPLODED_TEST_TARGETS := $(addprefix exploded-test-, $(ALL_NAMED_TESTS))
|
||||
|
||||
ALL_TARGETS += $(ALL_TEST_TARGETS) $(ALL_EXPLODED_TEST_TARGETS)
|
||||
|
||||
@ -520,13 +520,6 @@ test-image-hotspot-jtreg-graal:
|
||||
+($(CD) $(TOPDIR)/make/test && $(MAKE) $(MAKE_ARGS) -f JtregGraalUnit.gmk \
|
||||
test-image-hotspot-jtreg-graal)
|
||||
|
||||
run-test:
|
||||
+($(CD) $(TOPDIR)/make && $(MAKE) $(MAKE_ARGS) -f RunTests.gmk run-test TEST="$(TEST)")
|
||||
|
||||
exploded-run-test:
|
||||
+($(CD) $(TOPDIR)/make && $(MAKE) $(MAKE_ARGS) -f RunTests.gmk run-test \
|
||||
TEST="$(TEST)" JDK_IMAGE_DIR=$(JDK_OUTPUTDIR))
|
||||
|
||||
ifeq ($(BUILD_GTEST), true)
|
||||
test-image-hotspot-gtest:
|
||||
+($(CD) $(TOPDIR)/make/hotspot/test && $(MAKE) $(MAKE_ARGS) -f GtestImage.gmk)
|
||||
@ -541,11 +534,6 @@ ifeq ($(BUILD_FAILURE_HANDLER), true)
|
||||
+($(CD) $(TOPDIR)/make/test && $(MAKE) $(MAKE_ARGS) \
|
||||
-f BuildFailureHandler.gmk build)
|
||||
|
||||
# Runs the tests for the failure handler jtreg extension
|
||||
test-failure-handler:
|
||||
+($(CD) $(TOPDIR)/make/test && $(MAKE) $(MAKE_ARGS) \
|
||||
-f BuildFailureHandler.gmk test)
|
||||
|
||||
# Copies the failure handler jtreg extension into the test image
|
||||
test-image-failure-handler:
|
||||
+($(CD) $(TOPDIR)/make/test && $(MAKE) $(MAKE_ARGS) \
|
||||
@ -556,40 +544,19 @@ ALL_TARGETS += prepare-test-image build-test-hotspot-jtreg-native \
|
||||
test-image-hotspot-jtreg-native build-test-jdk-jtreg-native \
|
||||
test-image-jdk-jtreg-native build-test-lib build-test-failure-handler \
|
||||
test-failure-handler test-image-failure-handler test-image-hotspot-gtest \
|
||||
test-image-hotspot-jtreg-graal build-test-hotspot-jtreg-graal \
|
||||
run-test exploded-run-test
|
||||
test-image-hotspot-jtreg-graal build-test-hotspot-jtreg-graal
|
||||
|
||||
################################################################################
|
||||
# Run tests
|
||||
|
||||
# Run tests specified by $(TEST), or the default test set.
|
||||
test:
|
||||
$(call RunTests, $(TEST), $(JDK_IMAGE_DIR))
|
||||
+($(CD) $(TOPDIR)/make && $(MAKE) $(MAKE_ARGS) -f RunTests.gmk run-test TEST="$(TEST)")
|
||||
|
||||
test-hotspot-jtreg:
|
||||
$(call RunTests, "hotspot_all", $(JDK_IMAGE_DIR))
|
||||
exploded-test:
|
||||
+($(CD) $(TOPDIR)/make && $(MAKE) $(MAKE_ARGS) -f RunTests.gmk run-test \
|
||||
TEST="$(TEST)" JDK_IMAGE_DIR=$(JDK_OUTPUTDIR))
|
||||
|
||||
test-hotspot-jtreg-native:
|
||||
$(call RunTests, "hotspot_native_sanity", $(JDK_IMAGE_DIR))
|
||||
|
||||
test-hotspot-internal:
|
||||
$(call RunTests, "hotspot_internal", $(JDK_OUTPUTDIR))
|
||||
|
||||
test-hotspot-gtest:
|
||||
$(call RunTests, "hotspot_gtest", $(JDK_OUTPUTDIR))
|
||||
|
||||
test-jdk-jtreg-native:
|
||||
$(call RunTests, "jdk_native_sanity", $(JDK_IMAGE_DIR))
|
||||
|
||||
test-make:
|
||||
($(CD) $(TOPDIR)/test/make && $(MAKE) $(MAKE_ARGS) -f TestMake.gmk $(TEST_TARGET))
|
||||
|
||||
test-compile-commands:
|
||||
($(CD) $(TOPDIR)/test/make && $(MAKE) $(MAKE_ARGS) -f TestMake.gmk test-compile-commands)
|
||||
|
||||
ALL_TARGETS += test test-hotspot-jtreg test-hotspot-jtreg-native \
|
||||
test-hotspot-internal test-hotspot-gtest test-jdk-jtreg-native test-make \
|
||||
test-compile-commands
|
||||
ALL_TARGETS += test exploded-test
|
||||
|
||||
################################################################################
|
||||
# Bundles
|
||||
@ -874,14 +841,18 @@ else
|
||||
|
||||
docs-zip: docs-jdk
|
||||
|
||||
# Tests
|
||||
test: jdk-image test-image
|
||||
|
||||
run-test: jdk-image test-image
|
||||
exploded-run-test: exploded-image test-image
|
||||
exploded-test: exploded-image test-image
|
||||
|
||||
test-make: clean-test-make compile-commands
|
||||
|
||||
test-make-compile-commands: compile-commands
|
||||
|
||||
# Declare dependency for all generated test targets
|
||||
$(foreach t, $(ALL_TEST_TARGETS), $(eval $t: jdk-image test-image))
|
||||
$(foreach t, $(ALL_EXPLODED_TEST_TARGETS), $(eval $t: exploded-image test-image))
|
||||
$(foreach t, $(filter-out test-make%, $(ALL_TEST_TARGETS)), $(eval $t: jdk-image test-image))
|
||||
$(foreach t, $(filter-out exploded-test-make%, $(ALL_EXPLODED_TEST_TARGETS)), $(eval $t: exploded-image test-image))
|
||||
|
||||
create-buildjdk-copy: jdk.jlink-java java.base-gendata \
|
||||
$(addsuffix -java, $(INTERIM_IMAGE_MODULES))
|
||||
@ -890,16 +861,10 @@ else
|
||||
|
||||
interim-image: $(INTERIM_JMOD_TARGETS)
|
||||
|
||||
test-make: clean-test-make
|
||||
|
||||
test-compile-commands: compile-commands
|
||||
|
||||
build-test-lib: exploded-image-optimize
|
||||
|
||||
build-test-failure-handler: interim-langtools
|
||||
|
||||
test-failure-handler: build-test-failure-handler
|
||||
|
||||
test-image-failure-handler: build-test-failure-handler
|
||||
|
||||
build-test-hotspot-jtreg-native: buildtools-jdk \
|
||||
@ -917,12 +882,6 @@ else
|
||||
|
||||
test-image-hotspot-gtest: hotspot
|
||||
|
||||
test-hotspot-internal: exploded-image
|
||||
|
||||
test-hotspot-jtreg: jdk-image test-image
|
||||
|
||||
test-hotspot-gtest: exploded-image test-image-hotspot-gtest
|
||||
|
||||
install: product-images
|
||||
|
||||
product-bundles: product-images
|
||||
@ -1091,6 +1050,30 @@ all: all-images
|
||||
|
||||
ALL_TARGETS += default jdk images docs bundles all
|
||||
|
||||
# Aliases used for running tests.
|
||||
|
||||
# Let "run-test" be an alias for "test"
|
||||
$(foreach t, $(ALL_NAMED_TESTS), $(eval run-test-$t: test-$t))
|
||||
$(foreach t, $(ALL_NAMED_TESTS), $(eval exploded-run-test-$t: exploded-test-$t))
|
||||
RUN_TEST_TARGETS := $(addprefix run-test-, $(ALL_NAMED_TESTS)) \
|
||||
$(addprefix exploded-run-test-, $(ALL_NAMED_TESTS))
|
||||
|
||||
run-test: test
|
||||
exploded-run-test: exploded-test
|
||||
|
||||
# "make check" is a common idiom for running basic testing
|
||||
check: test-tier1
|
||||
|
||||
# Keep some old names as aliases
|
||||
test-hotspot-jtreg: test-hotspot_all
|
||||
test-hotspot-jtreg-native: test-hotspot_native_sanity
|
||||
test-hotspot-gtest: exploded-test-gtest
|
||||
test-jdk-jtreg-native: test-jdk_native_sanity
|
||||
|
||||
ALL_TARGETS += $(RUN_TEST_TARGETS) run-test exploded-run-test check \
|
||||
test-hotspot-jtreg test-hotspot-jtreg-native test-hotspot-gtest \
|
||||
test-jdk-jtreg-native
|
||||
|
||||
################################################################################
|
||||
################################################################################
|
||||
#
|
||||
|
||||
@ -1,5 +1,5 @@
|
||||
#
|
||||
# Copyright (c) 2011, 2017, Oracle and/or its affiliates. All rights reserved.
|
||||
# Copyright (c) 2011, 2018, Oracle and/or its affiliates. All rights reserved.
|
||||
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
|
||||
#
|
||||
# This code is free software; you can redistribute it and/or modify it
|
||||
@ -30,17 +30,6 @@
|
||||
ifndef _MAINSUPPORT_GMK
|
||||
_MAINSUPPORT_GMK := 1
|
||||
|
||||
# Run the tests specified by $1, with PRODUCT_HOME specified by $2
|
||||
# JT_JAVA is picked up by the jtreg launcher and used to run Jtreg itself.
|
||||
define RunTests
|
||||
($(CD) $(TOPDIR)/test && $(MAKE) $(MAKE_ARGS) -j1 -k MAKEFLAGS= \
|
||||
JT_HOME=$(JT_HOME) PRODUCT_HOME=$(strip $2) \
|
||||
TEST_IMAGE_DIR=$(TEST_IMAGE_DIR) \
|
||||
ALT_OUTPUTDIR=$(OUTPUTDIR) TEST_JOBS=$(TEST_JOBS) \
|
||||
JT_JAVA=$(BOOT_JDK) JIB_JAR=$(JIB_JAR) \
|
||||
JOBS=$(JOBS) $1) || true
|
||||
endef
|
||||
|
||||
define CleanDocs
|
||||
@$(PRINTF) "Cleaning docs ..."
|
||||
@$(PRINTF) "\n" $(LOG_DEBUG)
|
||||
|
||||
@ -1,5 +1,5 @@
|
||||
#
|
||||
# Copyright (c) 2016, 2017, Oracle and/or its affiliates. All rights reserved.
|
||||
# Copyright (c) 2016, 2018, Oracle and/or its affiliates. All rights reserved.
|
||||
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
|
||||
#
|
||||
# This code is free software; you can redistribute it and/or modify it
|
||||
@ -300,16 +300,32 @@ endef
|
||||
|
||||
# Helper function to determine if a test specification is a special test
|
||||
#
|
||||
# It is a special test if it is "special:" followed by a test name.
|
||||
# It is a special test if it is "special:" followed by a test name,
|
||||
# if it is "make:" or "make-" followed by a make test, or any of the special test names
|
||||
# as a single word.
|
||||
define ParseSpecialTestSelection
|
||||
$(if $(filter special:%, $1), \
|
||||
$1 \
|
||||
) \
|
||||
$(if $(filter make%, $1), \
|
||||
$(if $(filter make:%, $1), \
|
||||
special:$(strip $1) \
|
||||
) \
|
||||
$(if $(filter make-%, $1), \
|
||||
special:$(patsubst make-%,make:%, $1) \
|
||||
) \
|
||||
$(if $(filter make, $1), \
|
||||
special:make:all \
|
||||
)
|
||||
) \
|
||||
$(if $(filter hotspot-internal failure-handler, $1), \
|
||||
special:$(strip $1) \
|
||||
)
|
||||
endef
|
||||
|
||||
ifeq ($(TEST), )
|
||||
$(info No test selection given in TEST!)
|
||||
$(info Please use e.g. 'run-test TEST=tier1' or 'run-test-tier1')
|
||||
$(info Please use e.g. 'make test TEST=tier1' or 'make test-tier1')
|
||||
$(info See doc/testing.[md|html] for help)
|
||||
$(error Cannot continue)
|
||||
endif
|
||||
@ -663,9 +679,13 @@ define SetupRunSpecialTestBody
|
||||
$$(JDK_IMAGE_DIR)/bin/java -XX:+ExecuteInternalVMTests \
|
||||
-XX:+ShowMessageBoxOnError -version
|
||||
else ifeq ($$($1_TEST_NAME), failure-handler)
|
||||
$1_TEST_COMMAND_LINE := \
|
||||
($(CD) $(TOPDIR)/make/test && $(MAKE) $(MAKE_ARGS) -f \
|
||||
BuildFailureHandler.gmk test)
|
||||
ifeq ($(BUILD_FAILURE_HANDLER), true)
|
||||
$1_TEST_COMMAND_LINE := \
|
||||
($(CD) $(TOPDIR)/make/test && $(MAKE) $(MAKE_ARGS) -f \
|
||||
BuildFailureHandler.gmk test)
|
||||
else
|
||||
$$(error Cannot test failure handler if it is not built)
|
||||
endif
|
||||
else ifeq ($$($1_TEST_NAME), make)
|
||||
$1_TEST_COMMAND_LINE := \
|
||||
($(CD) $(TOPDIR)/test/make && $(MAKE) $(MAKE_ARGS) -f \
|
||||
|
||||
@ -1,5 +1,5 @@
|
||||
#
|
||||
# Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved.
|
||||
# Copyright (c) 2017, 2018, Oracle and/or its affiliates. All rights reserved.
|
||||
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
|
||||
#
|
||||
# This code is free software; you can redistribute it and/or modify it
|
||||
@ -72,6 +72,15 @@ ALL_NAMED_TESTS += $(JTREG_TEST_GROUPS)
|
||||
# Add Gtest
|
||||
ALL_NAMED_TESTS += gtest
|
||||
|
||||
# Find make test targets
|
||||
MAKE_TEST_TARGETS := $(shell $(MAKE) -s --no-print-directory $(MAKE_ARGS) \
|
||||
SPEC=$(SPEC) -f $(TOPDIR)/test/make/TestMake.gmk print-targets)
|
||||
|
||||
ALL_NAMED_TESTS += $(addprefix make-, $(MAKE_TEST_TARGETS))
|
||||
|
||||
# Add special tests
|
||||
ALL_NAMED_TESTS += hotspot-internal failure-handler make
|
||||
|
||||
################################################################################
|
||||
|
||||
endif # _FIND_TESTS_GMK
|
||||
|
||||
@ -759,7 +759,7 @@ var getJibProfilesProfiles = function (input, common, data) {
|
||||
testedProfile + ".test"
|
||||
],
|
||||
src: "src.conf",
|
||||
make_args: [ "run-test-prebuilt", "LOG_CMDLINES=true", "JTREG_VERBOSE=fail,error,time" ],
|
||||
make_args: [ "test-prebuilt", "LOG_CMDLINES=true", "JTREG_VERBOSE=fail,error,time" ],
|
||||
environment: {
|
||||
"BOOT_JDK": common.boot_jdk_home,
|
||||
"JDK_IMAGE_DIR": input.get(testedProfile + ".jdk", "home_path"),
|
||||
|
||||
@ -1,5 +1,5 @@
|
||||
#
|
||||
# Copyright (c) 2010, 2015, Oracle and/or its affiliates. All rights reserved.
|
||||
# Copyright (c) 2010, 2018, Oracle and/or its affiliates. All rights reserved.
|
||||
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
|
||||
#
|
||||
# This code is free software; you can redistribute it and/or modify it
|
||||
@ -27,11 +27,16 @@
|
||||
# Makefile to run tests from multiple sibling directories
|
||||
#
|
||||
|
||||
$(info WARNING: This way of running tests ("cd test && make") is deprecated)
|
||||
$(info Please use "make test TEST=..." instead. See doc/testing.md for details)
|
||||
|
||||
# Macro to run a test target in a subdir
|
||||
define SUBDIR_TEST # subdirectory target
|
||||
if [ -d $1 ] ; then \
|
||||
if [ -r $1/Makefile ] ; then \
|
||||
$(MAKE) --no-print-directory -k -C $1 $2 ; \
|
||||
echo 'WARNING: This way of running tests ("cd test && make") is deprecated' ; \
|
||||
echo 'Please use "make test TEST=..." instead. See doc/testing.md for details' ; \
|
||||
else \
|
||||
echo "ERROR: File does not exist: $1/Makefile"; \
|
||||
exit 1; \
|
||||
|
||||
@ -1,5 +1,5 @@
|
||||
#
|
||||
# Copyright (c) 2014, Oracle and/or its affiliates. All rights reserved.
|
||||
# Copyright (c) 2014, 2018, Oracle and/or its affiliates. All rights reserved.
|
||||
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
|
||||
#
|
||||
# This code is free software; you can redistribute it and/or modify it
|
||||
@ -36,13 +36,17 @@ java-compilation:
|
||||
copy-files:
|
||||
+$(MAKE) -f TestCopyFiles.gmk $(TEST_SUBTARGET)
|
||||
|
||||
test-idea:
|
||||
idea:
|
||||
+$(MAKE) -f TestIdea.gmk $(TEST_SUBTARGET)
|
||||
|
||||
test-compile-commands:
|
||||
compile-commands:
|
||||
+$(MAKE) -f TestCompileCommands.gmk $(TEST_SUBTARGET)
|
||||
|
||||
TARGETS += make-base java-compilation copy-files idea compile-commands
|
||||
|
||||
all: make-base java-compilation copy-files test-idea
|
||||
all: $(TARGETS)
|
||||
|
||||
.PHONY: default all make-base java-compilation copy-files test-idea test-compile-commands
|
||||
print-targets:
|
||||
$(ECHO) "$(TARGETS)"
|
||||
|
||||
.PHONY: default all $(TARGETS)
|
||||
|
||||
@ -344,7 +344,7 @@ $(eval $(call assert-equals, \
|
||||
KWBASE := APA=banan;GURKA=tomat;COUNT=1%202%203%204%205;SUM=1+2+3+4+5;MANY_WORDS=I have the best words.
|
||||
|
||||
$(eval $(call ParseKeywordVariable, KWBASE, \
|
||||
KEYWORDS := APA GURKA SUM, \
|
||||
SINGLE_KEYWORDS := APA GURKA SUM, \
|
||||
STRING_KEYWORDS := COUNT MANY_WORDS, \
|
||||
))
|
||||
|
||||
@ -377,7 +377,7 @@ override KWBASE_WEIRD_GURKA := paprika
|
||||
KWBASE_WEIRD := ;;APA=banan;;;GURKA=apelsin;APA=skansen;;
|
||||
|
||||
$(eval $(call ParseKeywordVariable, KWBASE_WEIRD, \
|
||||
KEYWORDS := APA GURKA SUM, \
|
||||
SINGLE_KEYWORDS := APA GURKA SUM, \
|
||||
STRING_KEYWORDS := COUNT, \
|
||||
))
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user