stan/jdk-24
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

8272558: IR Test Framework README misses some flags

Browse Source 
Reviewed-by: chagedorn
This commit is contained in:
Tobias Hartmann 2021-08-18 04:47:58 +00:00
parent ec63957f9d
commit 5189047d09
1 changed files with 17 additions and 15 deletions

32
test/hotspot/jtreg/compiler/lib/ir_framework/README.md

@ -10,17 +10,17 @@ The framework is intended to be used in JTreg tests. The JTreg header of the tes
* @library /test/lib /
* @run driver my.package.MySimpleTest
*/
package my.package;
import compiler.lib.ir_framework.*;
public class MySimpleTest {
public static void main(String[] args) {
TestFramework.run(); // The framework runs all tests of this class.
}
@Test
@IR(failOn = IRNode.STORE) // Fail if the IR of myTest() contains any stores.
public void myTest() {
@ -29,14 +29,14 @@ The framework is intended to be used in JTreg tests. The JTreg header of the tes
}
There are various ways how to set up and run a test within the `main()` method of a JTreg test. These are described and can be found in the [TestFramework](./TestFramework.java) class.
## 2. Features
The framework offers various annotations and flags to control how your test code should be invoked and being checked. This section gives an overview over all these features.
### 2.1 Different Tests
There are three kinds of tests depending on how much control is needed over the test invocation.
#### Base Tests
The simplest form of testing provides a single `@Test` annotated method which the framework will invoke as part of the testing. The test method has no or well-defined arguments that the framework can automatically provide.
The simplest form of testing provides a single `@Test` annotated method which the framework will invoke as part of the testing. The test method has no or well-defined arguments that the framework can automatically provide.
More information on base tests with a precise definition can be found in the Javadocs of [Test](./Test.java). Concrete examples on how to specify a base test can be found in [BaseTestsExample](../../../testlibrary_tests/ir_framework/examples/BaseTestExample.java).
@ -57,12 +57,12 @@ The user has the possibility to add an additional `@IR` annotation to any `@Test
- A `failOn` check that verifies that the provided regex is not matched in the C2 IR.
- A `counts` check that verifies that the provided regex is matched a user defined number of times in the C2 IR.
A regex can either be a custom string or any of the default regexes provided by the framework in [IRNode](./IRNode.java) for some commonly used IR nodes (also provides the possibility of composite regexes).
An IR verification cannot always be performed. For example, a JTreg test could be run with _-Xint_ or not a debug build (_-XX:+PrintIdeal_ and _-XX:+PrintOptoAssembly_ are debug build flags). But also CI tier testing could add additional JTreg VM and Javaoptions flags which could make an IR rule unstable.
An IR verification cannot always be performed. For example, a JTreg test could be run with _-Xint_ or not a debug build (_-XX:+PrintIdeal_ and _-XX:+PrintOptoAssembly_ are debug build flags). But also CI tier testing could add additional JTreg VM and Javaoptions flags which could make an IR rule unstable.
In general, the framework will only perform IR verification if the used VM flags allow a C2 compilation and if non-critical additional JTreg VM and Javaoptions are provided (see whiteflag list in [TestFramework](./TestFramework.java)). The user test code, however, can specify any flags which still allow an IR verification to be performed if a C2 compilation is done (expected flags by user defined `@IR` annotations).
In general, the framework will only perform IR verification if the used VM flags allow a C2 compilation and if non-critical additional JTreg VM and Javaoptions are provided (see whiteflag list in [TestFramework](./TestFramework.java)). The user test code, however, can specify any flags which still allow an IR verification to be performed if a C2 compilation is done (expected flags by user defined `@IR` annotations).
An `@IR` annotation allows additional preconditions/restrictions on the currently present VM flags to enable or disable rules when certain flags are present or have a specific value (see `applyIfXX` properties of an `@IR` annotation).
@ -90,6 +90,7 @@ The framework provides various stress and debug flags. They should mainly be use
- `-DExclude=test3`: Provide a list of `@Test` method names which should be excluded from execution.
- `-DScenarios=1,2`: Provide a list of scenario indexes to specify which scenarios should be executed.
- `-DWarmup=200`: Provide a new default value of the number of warm-up iterations (framework default is 2000). This might have an influence on the resulting IR and could lead to matching failures (the user can also set a fixed default warm-up value in a test with `testFrameworkObject.setDefaultWarmup(200)`).
- `-DReportStdout=true`: Print the standard output of the test VM.
- `-DVerbose=true`: Enable more fain-grained logging (slows the execution down).
- `-DReproduce=true`: Flag to use when directly running a test VM to bypass dependencies to the driver VM state (for example, when reproducing an issue).
- `-DPrintTimes=true`: Print the execution time measurements of each executed test.
@ -99,16 +100,17 @@ The framework provides various stress and debug flags. They should mainly be use
- `-DShuffleTests=false`: Disables the random execution order of all tests (such a shuffling is always done by default).
- `-DDumpReplay=true`: Add the `DumpReplay` directive to the test VM.
- `-DGCAfter=true`: Perform `System.gc()` after each test (slows the execution down).
- `-TestCompilationTimeout=20`: Change the default waiting time (default: 10s) for a compilation of a normal `@Test` annotated method.
- `-DTestCompilationTimeout=20`: Change the default waiting time (default: 10s) for a compilation of a normal `@Test` annotated method.
- `-DWaitForCompilationTimeout=20`: Change the default waiting time (default: 10s) for a compilation of a `@Test` annotated method with compilation level [WAIT\_FOR\_COMPILATION](./CompLevel.java).
- `-DIgnoreCompilerControls=false`: Ignore all compiler controls applied in the framework. This includes any compiler control annotations (`@DontCompile`, `@DontInline`, `@ForceCompile`, `@ForceInline`, `@ForceCompileStaticInitializer`), the exclusion of `@Run` and `@Check` methods from compilation, and the directive to not inline `@Test` annotated methods.
- `-DIgnoreCompilerControls=true`: Ignore all compiler controls applied in the framework. This includes any compiler control annotations (`@DontCompile`, `@DontInline`, `@ForceCompile`, `@ForceInline`, `@ForceCompileStaticInitializer`), the exclusion of `@Run` and `@Check` methods from compilation, and the directive to not inline `@Test` annotated methods.
- `-DExcludeRandom=true`: Randomly exclude some methods from compilation.
- `-DPreferCommandLineFlags=true`: Prefer flags set via the command line over flags specified by the tests.
## 3. Test Framework Execution
This section gives an overview of how the framework is executing a JTreg test that calls the framework from within its `main()` method.
The framework will spawn a new "test VM" to execute the user defined tests. The test VM collects all tests of the test class specified by the user code in `main()` and ensures that there is no violation of the required format by the framework. In a next step, the framework does the following for each test in general:
1. Warm the test up for a predefined number of times (default 2000). This can also be adapted for all tests by using `testFrameworkobject.setDefaultWarmup(100)` or for individual tests with an additional [@Warmup](./Warmup.java) annotation.
1. Warm the test up for a predefined number of times (default 2000). This can also be adapted for all tests by using `testFrameworkobject.setDefaultWarmup(100)` or for individual tests with an additional [@Warmup](./Warmup.java) annotation.
2. After the warm-up is finished, the framework compiles the associated `@Test` annotated method at the specified compilation level (default: C2).
3. After the compilation, the test is invoked one more time.
@ -116,8 +118,8 @@ Once the test VM terminates, IR verification (if possible) is performed on the o
Some of the steps above can be different due to the kind of the test or due to using non-default annotation properties. These details and differences are described in the Javadocs for the three tests (see section 2.1 Different Tests).
More information about the internals and the workflow of the framework can be found in the Javadocs of [TestFramework](./TestFramework.java).
More information about the internals and the workflow of the framework can be found in the Javadocs of [TestFramework](./TestFramework.java).
## 4. Internal Framework Tests
There are various tests to verify the correctness of the test framework. These tests can be found in [ir_framework](../../../testlibrary_tests/ir_framework) and can directly be run with JTreg. The tests are part of the normal JTreg tests of HotSpot and should be run upon changing the framework code as a minimal form of testing.