Bulk zeroing reduction didn't work with G1, because arraycopy would call pre-barriers on uninitialized oops. The solution is to have version of arraycopy stubs that don't have pre-barriers. Also refactored arraycopy stubs generation on SPARC to be more readable and reduced the number of stubs necessary in some cases.
Reviewed-by: jrose, kvn, never
Repurpose the existing flag G1PrintRegionLivenessInfo to print out the liveness distribution across the regions in the heap at the end of marking.
Reviewed-by: iveresov, jwilhelm
The epochs on the card cache array are initialized to 0 and our initial epoch also starts at 0. So, until the first GC, it might be possible to successfully "claim" a card which was in fact never initialized.
Reviewed-by: johnc, iveresov, ysr
7023151: G1: refactor the code that operates on _cur_alloc_region to be re-used for allocs by the GC threads
7018286: G1: humongous allocation attempts should take the GC locker into account
First, this change replaces the asymmetric locking scheme in the G1 slow alloc path by a summetric one. Second, it factors out the code that operates on _cur_alloc_region so that it can be re-used for allocations by the GC threads in the future.
Reviewed-by: stefank, brutisso, johnc
Manage allocation/freeing of the card cache counts and epochs arrays directly so that an allocation failure while attempting to expand these arrays does not abort the JVM. Failure to expand these arrays is not fatal.
Reviewed-by: iveresov, tonyp
Changed default value of WorkStealingYieldsBeforeSleep from 1000 to 5000. Added more information to G1 pause logging.
Reviewed-by: jwilhelm, tonyp, jmasa
Fix block_size_if_printezis_bits() so it does not expect the bits, only uses them when available. Fix block_size_no_stall() so it does not stall when the bits are missing such cases, letting the caller deal with zero size returns. Constant pool cache oops do not need to be unparsable or conc_unsafe after their klass pointer is installed. Some cosmetic clean-ups and some assertion checking for conc-usafety which, in the presence of class file redefinition, has no a-priori time boundedness, so all GCs must be able to safely deal with putatively conc-unsafe objects in a stop-world pause.
Reviewed-by: jmasa, johnc
7007446: G1: expand the heap with a single step, not one region at a time
Changed G1CollectedHeap::expand() to expand the committed space by calling VirtualSpace::expand_by() once rather than for every region in the expansion amount. This allows the success or failure of the expansion to be determined before creating any heap regions. Introduced a develop flag G1ExitOnExpansionFailure (false by default) that, when true, will exit the VM if the expansion of the committed space fails. Finally G1CollectedHeap::expand() returns a status back to it's caller so that the caller knows whether to attempt the allocation.
Reviewed-by: brutisso, tonyp
There's a potential deadlock between the concurrent cleanup thread and the GC workers that are trying to allocate and waiting for more free regions to be made available.
Reviewed-by: iveresov, jcoomes
A race between the concurrent cleanup thread and the VM thread while it is processing the "expanded sparse table list" causes both threads to try to free the same sparse table entry and either causes one of the threads to fail or leaves the entry in an inconsistent state. The solution is purge all entries on the expanded list that correspond go regions that are being cleaned up.
Reviewed-by: brutisso, johnc
Two small fixes to deal with a guarantee failure (the marking thread should join the SuspendibleThreadSet before calling a method that does pause prediction work so that said method is never called during a pause) and an assert failure (an assert is too strong).
Reviewed-by: iveresov, johnc
This changeset removes the zero-filling thread from G1 and collapses the two free region lists we had before (the "free" and "unclean" lists) into one. The new free list uses the new heap region sets / lists abstractions that we'll ultimately use it to keep track of all regions in the heap. A heap region set was also introduced for the humongous regions. Finally, this change increases the concurrency between the thread that completes freeing regions (after a cleanup pause) and the rest of the system (before we'd have to wait for said thread to complete before allocating a new region). The changest also includes a lot of refactoring and code simplification.
Reviewed-by: jcoomes, johnc
This changeset introduces filtering of SATB buffers at the point when they are about to be enqueued. If this filtering clears enough entries on each buffer, the buffer can then be re-used and not enqueued. This cuts down the number of SATB buffers that need to be processed by the concurrent marking threads.
Reviewed-by: johnc, ysr
First attempt to allocate out the current alloc region using a CAS instead of taking the Heap_lock (first level of G1's slow allocation path). Only if that fails and it's necessary to replace the current alloc region take the Heap_lock (that's the second level of G1's slow allocation path).
Reviewed-by: johnc, brutisso, ysr
During evacuation failure handling we refine the BOT to reflect the location of all the objects in the regions we scan. The changeset includes some minor cleanup: a) non-product print_on() method on the G1 BOT class, b) added more complete BOT verification during heap / region verification, c) slight modification to the BOT set up for humongous regions to be more consistent with the BOT set up during evac failure handling, and d) removed a couple of unused methods.
Reviewed-by: johnc, ysr
The recorded _sweep_limit may not necessarily remain a block boundary as the old generation expands during a concurrent cycle. Terminal actions inside the sweep closure need to be aware of this as they cross over the limit.
Reviewed-by: johnc, minqi
Age bits need not enter the mark-word preservation calculus; also affected, in addition to CMS, per CR synopsis above, were ParNew (but not DefNew), ParallelScavenge and G1, albeit to a lesser degree than CMS.
Reviewed-by: tonyp, johnc
Initialize the _is_alive_non_header field of G1's reference processor with an instance of the G1CMIsAliveClosure. This will stop adding reference objects with live referents to the discovered reference lists unnecessarily.
Reviewed-by: tonyp, ysr, jwilhelm, brutisso
The concurrent marking thread can complete its operation and increment the full GC counter during a Full GC. This causes the nesting of increments to the start and end of Full GCs that we are expecting to be wrong. the fix is for the marking thread to join the suspendible thread set before incrementing the counter so that it's blocked until the Full GC (or any other safepoint) is finished. The change also includes some minor code cleanup (I renamed a parameter).
Reviewed-by: brutisso, ysr
7002546: regression on SpecJbb2005 on 7b118 comparing to 7b117 on small heaps
Relaxed assertion checking related to incremental_collection_failed flag to allow for ExplicitGCInvokesConcurrent behaviour where we do not want a failing scavenge to bail to a stop-world collection. Parameterized incremental_collection_will_fail() so we can selectively use, or not use, as appropriate, the statistical prediction at specific use sites. This essentially reverts the scavenge bail-out logic to what it was prior to some recent changes that had inadvertently started using the statistical prediction which can be noisy in the presence of bursty loads. Added some associated verbose non-product debugging messages.
Reviewed-by: johnc, tonyp
Enable reference discovery during concurrent marking by setting the reference processor field of the concurrent marking closure. Keep reference objects on the discovered reference lists alive during incremental evacuation pauses until they are processed at the end of concurrent marking.
Reviewed-by: ysr, tonyp
Added an ifndef DONT_USE_PRECOMPILED_HEADER to precompiled.hpp. Set up DONT_USE_PRECOMPILED_HEADER when compiling with Sun Studio or when the user specifies USE_PRECOMPILED_HEADER=0. Fixed broken include dependencies.
Reviewed-by: coleenp, kvn
