Under certain circumstances a safepoint could happen between a JavaThread object being created and that object being added to the Java threads list. This could cause the active field of that thread's SATB queue to get out-of-sync with respect to the other Java threads. The solution is to activate the SATB queue, when necessary, before adding the thread to the Java threads list, not when the JavaThread object is created. The changeset also includes a small fix to rename the surrogate locker thread from "Surrogate Locker Thread (CMS)" to "Surrogate Locker Thread (Concurrent GC)" since it's also used in G1.
Reviewed-by: iveresov, ysr, johnc, jcoomes
Clear the concurrent marking "in progress" flag while the FullGCCount_lock is held. This avoids a race that can cause back to back System.gc() calls, when ExplicitGCInvokesConcurrent is enabled, to fail to initiate a marking cycle causing the requesting thread to hang.
Reviewed-by: tonyp, ysr
Concurrent collectors should expand the perm gen without a full STW GC, but possibly by triggering a concurrent collection. Temporary band-aid for G1 where no concurrent collection is kicked off since the perm gen is not collected concurrently.
Reviewed-by: johnc
Inserted missing yield(check)s in closures used during the work-stealing phase of parallel concurrent marking, a missing synchronous yield-request in the cms perm gen allocation path, and a terminator-terminator for the offer_termination invocation that monitors the yield status of the concurrent marking task. Elaborated some documentation comments and made some task queue termination loop flags configurable at start-up to aid debugging in the field.
Reviewed-by: jmasa, johnc, poonam
Re-enable use of the lock-free versions of region stack push() and pop() by recording aborted regions in a thread-local structure, which are then processed when scanning of the region stack restarts. The previous locking versions of these routines are retained for diagnostic purposes.
Reviewed-by: tonyp, ysr
This change revamps the slow allocation path of G1. Improvements include the following: a) Allocations directly to old regions are now totally banned. G1 now only allows allocations out of young regions (with the only exception being humongous regions). b) The thread that allocates a new region (which is now guaranteed to be young) does not dirty all its cards. Each thread that successfully allocates out of a young region is now responsible for dirtying the cards that corresponding to the "block" that just got allocated. c) allocate_new_tlab() and mem_allocate() are now implemented differently and TLAB allocations are only done by allocate_new_tlab(). d) If a thread schedules an evacuation pause in order to satisfy an allocation request, it will perform the allocation at the end of the safepoint so that the thread that initiated the GC also gets "first pick" of any space made available by the GC. e) If a thread is unable to allocate a humongous object it will schedule an evacuation pause in case it reclaims enough regions so that the humongous allocation can be satisfied aftewards. f) The G1 policy is more careful to set the young list target length to be the survivor number +1. g) Lots of code tidy up, removal, refactoring to make future changes easier.
Reviewed-by: johnc, ysr
Weakened a too-strong, off-by-one assert; added code to keep track of and report any overflows at appropriate level of verbosity.
Reviewed-by: jcoomes, tonyp
Allow for the possibility (when the heap is expanding) that the sweep might skip over and past, rather than necessarily step on, the sweep limit determined at the beginning of a concurrent marking cycle.
Reviewed-by: jmasa, tonyp
We're renaming "evacuation failure" to "to-space overflow". I'm also piggy-backing a small additional change which removes the "Mark closure took..." output.
Reviewed-by: ysr, johnc
There are a few issues in the code that calculates whether to resize the heap and by how much: a) some calculations can overflow 32-bit size_t's, b) min_desired_capacity is not bounded by the max heap size, and c) the assrt that fires is in the wrong place. The fix also includes some tidying up of the related verbose code.
Reviewed-by: ysr, jmasa
GC workers now recognize an intermediate transient state of blocks which are allocated but have not yet completed initialization. blk_start() calls do not attempt to determine the size of a block in the transient state, rather waiting for the block to become initialized so that it is safe to query its size. Audited and ensured the order of initialization of object fields (klass, free bit and size) to respect block state transition protocol. Also included some new assertion checking code enabled in debug mode.
Reviewed-by: chrisphi, johnc, poonam
During RSet updating, when ParallelGCThreads is zero, references that point into the collection set are added directly the referenced region's RSet. This can cause the sparse table in the RSet to expand. RSet scanning and the "occupied" routine will then operate on different instances of the sparse table causing the assert to trip. This may also cause some cards added post expansion to be missed during RSet scanning. When ParallelGCThreads is non-zero such references are recorded on the "references to be scanned" queue and the card containing the reference is recorded in a dirty card queue for use in the event of an evacuation failure. Employ the parallel code in the serial case to avoid expanding the RSets of regions in the collection set.
Reviewed-by: iveresov, ysr, tonyp
As part of 6944166 we disabled the concept of abandoned pauses (i.e., if the collection set is empty, we would still try to do a pause even if it is to update the RSets and scan the roots). This changeset removes the code and structures associated with abandoned pauses.
Reviewed-by: iveresov, johnc
The per-worker _new_refs array is used to hold references that point into the collection set. It is populated during RSet updating and subsequently processed. In the event of an evacuation failure it processed again to recreate the RSets of regions in the collection set. Remove the per-worker _new_refs array by processing the references directly. Use a DirtyCardQueue to hold the cards containing the references so that the RSets of regions in the collection set can be recreated when handling an evacuation failure.
Reviewed-by: iveresov, jmasa, tonyp
During concurrent refinment, filter cards in young regions after it has been determined that the region has been allocated from and the young type of the region has been set.
Reviewed-by: iveresov, tonyp, jcoomes
G1 was not handling explicit GCs correctly in many ways. It does now. See the CR for the list of improvements contained in this changeset.
Reviewed-by: iveresov, ysr, johnc
Rather than changing the default value of the G1UseFixedWindowMMUTracker, the flag and associated guarantee have been removed.
Reviewed-by: jcoomes, tonyp, ysr
When using compressed oops, use compressed promoted pointers in b63:b31 of the mark word, so as not to interfere with the CMS "freeness bit" at b7. Updated mark-word layout documentation.
Reviewed-by: minqi, poonam, jmasa, coleenp
Moved PromotionInfo and friends into new files promotionInfo.{h,c}pp from their previous compactibleFreeListSpace.{h,c}pp home.
Reviewed-by: apetrusenko
On sun4v/CMT avoid use of memset() in BOT updates so as to prevent concurrent BOT readers from seeing the phantom zeros arising from memset()'s use of BIS.
Reviewed-by: jmasa, johnc, minqi, poonam, tonyp
