stan/jdk-24
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

8296240: Augment discussion of test tiers in doc/testing.md

Browse Source 
Reviewed-by: alanb
This commit is contained in:
Joe Darcy 2023-11-02 20:28:16 +00:00
parent ffaecd4aa2
commit c788160f8a
2 changed files with 31 additions and 13 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

23
doc/testing.html
View File

@ -207,13 +207,22 @@ changed, and the first N tiers they can afford to run, but at least
tier1.</p>
<p>A brief description of the tiered test groups:</p>
<ul>
<li><p><code>tier1</code>: This is the lowest test tier. Multiple
developers run these tests every day. Because of the widespread use, the
tests in <code>tier1</code> are carefully selected and optimized to run
fast, and to run in the most stable manner. The test failures in
<code>tier1</code> are usually followed up on quickly, either with
fixes, or adding relevant tests to problem list. GitHub Actions
workflows, if enabled, run <code>tier1</code> tests.</p></li>
<li><p><code>tier1</code>: This is the most fundamental test tier.
Roughly speaking, a failure of a test in this tier has the potential to
indicate a problem that would affect many Java programs. Tests in
<code>tier1</code> include tests of HotSpot, core APIs in the
<code>java.base</code> module, and the <code>javac</code> compiler.
Multiple developers run these tests every day. Because of the widespread
use, the tests in <code>tier1</code> are carefully selected and
optimized to run fast, and to run in the most stable manner. As a
guideline, nearly all individual tests in <code>tier1</code> are
expected to run to completion in ten seconds or less when run on common
configurations used for development. Long-running tests, even of core
functionality, should occur in higher tiers or be covered in other kinds
of testing. The test failures in <code>tier1</code> are usually followed
up on quickly, either with fixes, or adding relevant tests to problem
list. GitHub Actions workflows, if enabled, run <code>tier1</code>
tests.</p></li>
<li><p><code>tier2</code>: This test group covers even more ground.
These contain, among other things, tests that either run for too long to
be at <code>tier1</code>, or may require special configuration, or tests

21
doc/testing.md
View File

@ -135,12 +135,21 @@ the first N tiers they can afford to run, but at least tier1.
A brief description of the tiered test groups:
- `tier1`: This is the lowest test tier. Multiple developers run these tests
every day. Because of the widespread use, the tests in `tier1` are
carefully selected and optimized to run fast, and to run in the most stable
manner. The test failures in `tier1` are usually followed up on quickly,
either with fixes, or adding relevant tests to problem list. GitHub Actions
workflows, if enabled, run `tier1` tests.
- `tier1`: This is the most fundamental test tier.
Roughly speaking, a failure of a test in this tier has the potential
to indicate a problem that would affect many Java programs. Tests in
`tier1` include tests of HotSpot, core APIs in the `java.base`
module, and the `javac` compiler. Multiple developers run these
tests every day. Because of the widespread use, the tests in `tier1`
are carefully selected and optimized to run fast, and to run in the
most stable manner. As a guideline, nearly all individual tests in
`tier1` are expected to run to completion in ten seconds or less
when run on common configurations used for development. Long-running
tests, even of core functionality, should occur in higher tiers or
be covered in other kinds of testing. The test failures in `tier1`
are usually followed up on quickly, either with fixes, or adding
relevant tests to problem list. GitHub Actions workflows, if
enabled, run `tier1` tests.
- `tier2`: This test group covers even more ground. These contain, among other
things, tests that either run for too long to be at `tier1`, or may require