There is a high number of mispredicted branches associated with calling BitMap::iteratate() from within CMBitMapRO::iterate(). Implement a version of CMBitMapRO::iterate() directly using inline-able routines.
Reviewed-by: tonyp, iveresov
This change simplifies the interaction between GC and concurrent marking. By disabling survivor spaces during the initial-mark pause we don't need to propagate marks of objects we copy during each GC (since we never need to copy an explicitly marked object).
Reviewed-by: johnc, brutisso
Store the "next chunk start index" in the length field of the to-space object, instead of the from-space object, so that we can always reliably read the size of all from-space objects.
Reviewed-by: johnc, ysr, jmasa
Parallelize the removal of self forwarding pointers etc. by wrapping in a HeapRegion closure, which is then wrapped inside an AbstractGangTask.
Reviewed-by: tonyp, iveresov
Introduce a flag that is set when a heap expansion attempt during a GC fails so that we do not consantly attempt to expand the heap when it's going to fail anyway. This not only prevents the excessive ergo output (which is generated when a region allocation fails) but also avoids excessive and ultimately unsuccessful expansion attempts.
Reviewed-by: jmasa, johnc
Change variables representing the number of GC workers to uint from int and size_t. Change the parameter in work(int i) to work(uint worker_id).
Reviewed-by: brutisso, tonyp
Serialize the worker threads that are generating output during parallel heap verification to make sure the output is consistent.
Reviewed-by: brutisso, johnc, jmasa
Running G1 with ParallelGCThreads==0 results in various crashes and asserts. Most of these are caused by unguarded references to the worker threads array or an incorrect number of active workers.
Reviewed-by: jmasa, tonyp
Renamed partially-young GCs as mixed and fully-young GCs as young. Change all external output that includes those terms (GC log and GC ergo log) as well as any comments, fields, methods, etc. The changeset also includes very minor code tidying up (added some curly brackets).
Reviewed-by: johnc, brutisso
Cache workers' calculated starting heap region, used for parallel iteration over the collcection set, for subsequent reuse.
Reviewed-by: tonyp, brutisso
As a result of the changes for 4965777, the G1 reference field scanning closure could be applied to the discovered field of a reference object twice. The failing assert is too strong if the result of the first application of the closure is stolen, and the referenced object, evacuated by another worker thread.
Reviewed-by: ysr, tonyp
Parallelize the serial code that was used to mark objects reachable from survivor objects in the collection set. Some minor improvments in the timers used to track the freeing of the collection set along with some tweaks to PrintGCDetails.
Reviewed-by: tonyp, brutisso
Race between the VM thread reading G1CollectedHeap::_mark_in_progress and it being set by the concurrent mark thread when concurrent marking is aborted by a full GC. Have the concurrent mark thread join the SuspendibleThreadSet before changing the marking state.
Reviewed-by: tonyp, brutisso
Various cleanups and formatting changes identified during a code walk through of the changes for 6484965 ("G1: piggy-back liveness accounting phase on marking").
Reviewed-by: brutisso, tonyp
Major cleanup of the G1CollectorPolicy class. It removes a lot of unused fields and methods and also consolidates replicated information (mainly various ways of counting the number of CSet regions) into one copy.
Reviewed-by: johnc, brutisso
7102445: G1: Unnecessary Resource allocations during RSet scanning
Add a new per-worker thread line in the PrintGCDetails output. GC Worker Other is the difference between the elapsed time for the parallel phase of the evacuation pause and the sum of the times of the sub-phases (external root scanning, mark stack scanning, RSet updating, RSet scanning, object copying, and termination) for that worker. During RSet scanning, stack allocate DirtyCardToOopClosure objects; allocating these in a resource area was causing abnormally high GC Worker Other times while the worker thread freed ResourceArea chunks.
Reviewed-by: tonyp, jwilhelm, brutisso
Acquire the pending list lock in the prologue method of G1's concurrent VM_Operation and release the lock in the epilogue() method. The locking/unlocking order of the pending list lock and the Heap_lock should match that in the prologue and epilogue methods of VM_GC_Operation.
Reviewed-by: tonyp, ysr
Add a new, separate, pointer to the base of the array of discovered reference lists and use this new pointer in places where we iterate over the entire array.
Reviewed-by: ysr, brutisso
Removed unused fields and methods, removed the G1CollectoryPolicy_BestRegionsFirst class and folded its functionality into the G1CollectorPolicy class.
Reviewed-by: ysr, brutisso, jcoomes
Removed the code for skipping over NULL regions in _markedRegions, replacing it with an assertion that a NULL region is never encountered; removed dead methods, remove() and remove_region(), and inlined a simplified addRegion() directly into fillCache().
Reviewed-by: brutisso, tonyp
Co-authored-by: Greg Lewis <glewis@eyesbeyond.com>
Co-authored-by: Jung-uk Kim <jkim@freebsd.org>
Co-authored-by: Christos Zoulas <christos@zoulas.com>
Co-authored-by: Landon Fuller <landonf@plausible.coop>
Co-authored-by: The FreeBSD Foundation <board@freebsdfoundation.org>
Co-authored-by: Michael Franz <mvfranz@gmail.com>
Co-authored-by: Roger Hoover <rhoover@apple.com>
Co-authored-by: Alexander Strange <astrange@apple.com>
Reviewed-by: kvn, twisti, jrose
When refactoring the code for a previous fix, a condition was not correctly negated which prevents the G1 policy from adding the correct number of old regions to the CSet when the young gen size is fixed. The changeset also fixes a small syntactical issue in g1ErgoVerbose.hpp which is causing compiler warnings.
Reviewed-by: brutisso, ysr
During remembered set scanning, the reference processor could discover a reference object whose referent was in the process of being copied and so may not be completely initialized. Do not perform reference discovery during remembered set scanning.
Reviewed-by: tonyp, ysr
Some objects may not be marked in the event of an evacuation failure in a partially young GC, during a marking cycle. Avoid this situation by not allowing partially young GCs during a marking cycle.
Reviewed-by: tonyp, ysr, brutisso
G1 now uses two reference processors - one is used by concurrent marking and the other is used by STW GCs (both full and incremental evacuation pauses). In an evacuation pause, the reference processor is embedded into the closures used to scan objects. Doing so causes causes reference objects to be 'discovered' by the reference processor. At the end of the evacuation pause, these discovered reference objects are processed - preserving (and copying) referent objects (and their reachable graphs) as appropriate.
Reviewed-by: ysr, jwilhelm, brutisso, stefank, tonyp
The desired min / max heap sizes are miscalculated at initialization when NewRatio is used. The changeset also includes an additional small change to turn a print statement into a warning.
Reviewed-by: johnc, jmasa, ysr, brutisso
