Since we are at a stop-world pause, the existing CMS-phase checks are sufficient for safety, and the locking check can be safely elided. Elaborated documentation comment to the case where class unloading and verification are disabled, and the query happens when we aren't in the sweeping phase, where the answer "false" would be (almost everywhere) too pessimistic.
Reviewed-by: jmasa, johnc, tonyp
Deprecated HandlePromotionFailure, removing the ability to turn off that feature, did away with one epoch look-ahead when deciding if a scavenge is likely to fail, relying on current data.
Reviewed-by: jmasa, johnc, poonam
Closed a timing hole during which concurrent full gc requests can be missed. The hole can increase the latency of the response to a full gc request by up to the value of CMSWaitDuration. If CMSWaitDuration=0 is, as currently, interpreted as an unbounded wait, suitable in certain tuning scenarios, the application can potentially hang. Made two obscure tunables, including CMSWaitDuration, manageable.
Reviewed-by: jcoomes, tonyp
There is a race between the concurrent refinement threads and the humongous object allocation that can cause the concurrent refinement threads to corrupt the part of the BOT that it is being initialized by the humongous object allocation operation. The solution is to do the humongous object allocation in careful steps to ensure that the concurrent refinement threads always have a consistent view over the BOT, region contents, and top. The fix includes some very minor tidying up in sparsePRT.
Reviewed-by: jcoomes, johnc, ysr
The changes from 6941395 did not clear the CMTask::_aborted_region fields when concurrent marking aborted because of overflow. As a result, the next time around we could see a memory region whose start address was above the global finger and the assertion tripped. Moved the clearing of the aborted regions to ConcurrentMark::clear_marking_state, which is executed on all of the exit paths.
Reviewed-by: tonyp, ysr, jmasa
The remembered set iterator for sparse tables incorrectly assumes that index 0 corresponds to the bottom of the heap, not address 0 as it is the case.
Reviewed-by: ysr, jmasa
Under certain circumstances a safepoint could happen between a JavaThread object being created and that object being added to the Java threads list. This could cause the active field of that thread's SATB queue to get out-of-sync with respect to the other Java threads. The solution is to activate the SATB queue, when necessary, before adding the thread to the Java threads list, not when the JavaThread object is created. The changeset also includes a small fix to rename the surrogate locker thread from "Surrogate Locker Thread (CMS)" to "Surrogate Locker Thread (Concurrent GC)" since it's also used in G1.
Reviewed-by: iveresov, ysr, johnc, jcoomes
Clear the concurrent marking "in progress" flag while the FullGCCount_lock is held. This avoids a race that can cause back to back System.gc() calls, when ExplicitGCInvokesConcurrent is enabled, to fail to initiate a marking cycle causing the requesting thread to hang.
Reviewed-by: tonyp, ysr
Concurrent collectors should expand the perm gen without a full STW GC, but possibly by triggering a concurrent collection. Temporary band-aid for G1 where no concurrent collection is kicked off since the perm gen is not collected concurrently.
Reviewed-by: johnc
Inserted missing yield(check)s in closures used during the work-stealing phase of parallel concurrent marking, a missing synchronous yield-request in the cms perm gen allocation path, and a terminator-terminator for the offer_termination invocation that monitors the yield status of the concurrent marking task. Elaborated some documentation comments and made some task queue termination loop flags configurable at start-up to aid debugging in the field.
Reviewed-by: jmasa, johnc, poonam
Re-enable use of the lock-free versions of region stack push() and pop() by recording aborted regions in a thread-local structure, which are then processed when scanning of the region stack restarts. The previous locking versions of these routines are retained for diagnostic purposes.
Reviewed-by: tonyp, ysr
This change revamps the slow allocation path of G1. Improvements include the following: a) Allocations directly to old regions are now totally banned. G1 now only allows allocations out of young regions (with the only exception being humongous regions). b) The thread that allocates a new region (which is now guaranteed to be young) does not dirty all its cards. Each thread that successfully allocates out of a young region is now responsible for dirtying the cards that corresponding to the "block" that just got allocated. c) allocate_new_tlab() and mem_allocate() are now implemented differently and TLAB allocations are only done by allocate_new_tlab(). d) If a thread schedules an evacuation pause in order to satisfy an allocation request, it will perform the allocation at the end of the safepoint so that the thread that initiated the GC also gets "first pick" of any space made available by the GC. e) If a thread is unable to allocate a humongous object it will schedule an evacuation pause in case it reclaims enough regions so that the humongous allocation can be satisfied aftewards. f) The G1 policy is more careful to set the young list target length to be the survivor number +1. g) Lots of code tidy up, removal, refactoring to make future changes easier.
Reviewed-by: johnc, ysr
Weakened a too-strong, off-by-one assert; added code to keep track of and report any overflows at appropriate level of verbosity.
Reviewed-by: jcoomes, tonyp
Allow for the possibility (when the heap is expanding) that the sweep might skip over and past, rather than necessarily step on, the sweep limit determined at the beginning of a concurrent marking cycle.
Reviewed-by: jmasa, tonyp
We're renaming "evacuation failure" to "to-space overflow". I'm also piggy-backing a small additional change which removes the "Mark closure took..." output.
Reviewed-by: ysr, johnc
There are a few issues in the code that calculates whether to resize the heap and by how much: a) some calculations can overflow 32-bit size_t's, b) min_desired_capacity is not bounded by the max heap size, and c) the assrt that fires is in the wrong place. The fix also includes some tidying up of the related verbose code.
Reviewed-by: ysr, jmasa
GC workers now recognize an intermediate transient state of blocks which are allocated but have not yet completed initialization. blk_start() calls do not attempt to determine the size of a block in the transient state, rather waiting for the block to become initialized so that it is safe to query its size. Audited and ensured the order of initialization of object fields (klass, free bit and size) to respect block state transition protocol. Also included some new assertion checking code enabled in debug mode.
Reviewed-by: chrisphi, johnc, poonam
During RSet updating, when ParallelGCThreads is zero, references that point into the collection set are added directly the referenced region's RSet. This can cause the sparse table in the RSet to expand. RSet scanning and the "occupied" routine will then operate on different instances of the sparse table causing the assert to trip. This may also cause some cards added post expansion to be missed during RSet scanning. When ParallelGCThreads is non-zero such references are recorded on the "references to be scanned" queue and the card containing the reference is recorded in a dirty card queue for use in the event of an evacuation failure. Employ the parallel code in the serial case to avoid expanding the RSets of regions in the collection set.
Reviewed-by: iveresov, ysr, tonyp
As part of 6944166 we disabled the concept of abandoned pauses (i.e., if the collection set is empty, we would still try to do a pause even if it is to update the RSets and scan the roots). This changeset removes the code and structures associated with abandoned pauses.
Reviewed-by: iveresov, johnc
The per-worker _new_refs array is used to hold references that point into the collection set. It is populated during RSet updating and subsequently processed. In the event of an evacuation failure it processed again to recreate the RSets of regions in the collection set. Remove the per-worker _new_refs array by processing the references directly. Use a DirtyCardQueue to hold the cards containing the references so that the RSets of regions in the collection set can be recreated when handling an evacuation failure.
Reviewed-by: iveresov, jmasa, tonyp
During concurrent refinment, filter cards in young regions after it has been determined that the region has been allocated from and the young type of the region has been set.
Reviewed-by: iveresov, tonyp, jcoomes
G1 was not handling explicit GCs correctly in many ways. It does now. See the CR for the list of improvements contained in this changeset.
Reviewed-by: iveresov, ysr, johnc
