GC should support isoheaps
authorfpizlo@apple.com <fpizlo@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 30 Nov 2017 04:39:50 +0000 (04:39 +0000)
committerfpizlo@apple.com <fpizlo@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 30 Nov 2017 04:39:50 +0000 (04:39 +0000)
https://bugs.webkit.org/show_bug.cgi?id=179288

Reviewed by Saam Barati.
Source/JavaScriptCore:

This expands the power of the Subspace API in JSC:

- Everything associated with describing the types of objects is now part of the HeapCellType class.
  We have different HeapCellTypes for different destruction strategies. Any Subspace can use any
  HeapCellType; these are orthogonal things.

- There are now two variants of Subspace: CompleteSubspace, which can allocate any size objects using
  any AlignedMemoryAllocator; and IsoSubspace, which can allocate just one size of object and uses a
  special virtual memory pool for that purpose. Like bmalloc's IsoHeap, IsoSubspace hoards virtual
  pages but releases the physical pages as part of the respective allocator's scavenging policy
  (the Scavenger in bmalloc for IsoHeap and the incremental sweep and full sweep in Riptide for
  IsoSubspace).

So far, this patch just puts subtypes of ExecutableBase in IsoSubspaces. If it works, we can use it
for more things.

This does not have any effect on JetStream (0.18% faster with p = 0.69).

* JavaScriptCore.xcodeproj/project.pbxproj:
* Sources.txt:
* bytecode/AccessCase.cpp:
(JSC::AccessCase::generateImpl):
* bytecode/ObjectAllocationProfileInlines.h:
(JSC::ObjectAllocationProfile::initializeProfile):
* dfg/DFGSpeculativeJIT.cpp:
(JSC::DFG::SpeculativeJIT::emitAllocateRawObject):
(JSC::DFG::SpeculativeJIT::compileMakeRope):
(JSC::DFG::SpeculativeJIT::compileAllocatePropertyStorage):
(JSC::DFG::SpeculativeJIT::compileReallocatePropertyStorage):
* dfg/DFGSpeculativeJIT64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
* ftl/FTLAbstractHeapRepository.h:
* ftl/FTLLowerDFGToB3.cpp:
(JSC::FTL::DFG::LowerDFGToB3::compileMakeRope):
(JSC::FTL::DFG::LowerDFGToB3::compileMaterializeNewObject):
(JSC::FTL::DFG::LowerDFGToB3::allocatePropertyStorageWithSizeImpl):
(JSC::FTL::DFG::LowerDFGToB3::allocateObject):
(JSC::FTL::DFG::LowerDFGToB3::allocatorForSize):
* heap/AlignedMemoryAllocator.cpp:
(JSC::AlignedMemoryAllocator::registerAllocator):
(JSC::AlignedMemoryAllocator::registerSubspace):
* heap/AlignedMemoryAllocator.h:
(JSC::AlignedMemoryAllocator::firstAllocator const):
* heap/AllocationFailureMode.h: Added.
* heap/CompleteSubspace.cpp: Added.
(JSC::CompleteSubspace::CompleteSubspace):
(JSC::CompleteSubspace::~CompleteSubspace):
(JSC::CompleteSubspace::allocatorFor):
(JSC::CompleteSubspace::allocate):
(JSC::CompleteSubspace::allocateNonVirtual):
(JSC::CompleteSubspace::allocatorForSlow):
(JSC::CompleteSubspace::allocateSlow):
(JSC::CompleteSubspace::tryAllocateSlow):
* heap/CompleteSubspace.h: Added.
(JSC::CompleteSubspace::offsetOfAllocatorForSizeStep):
(JSC::CompleteSubspace::allocatorForSizeStep):
(JSC::CompleteSubspace::allocatorForNonVirtual):
* heap/HeapCellType.cpp: Added.
(JSC::HeapCellType::HeapCellType):
(JSC::HeapCellType::~HeapCellType):
(JSC::HeapCellType::finishSweep):
(JSC::HeapCellType::destroy):
* heap/HeapCellType.h: Added.
(JSC::HeapCellType::attributes const):
* heap/IsoAlignedMemoryAllocator.cpp: Added.
(JSC::IsoAlignedMemoryAllocator::IsoAlignedMemoryAllocator):
(JSC::IsoAlignedMemoryAllocator::~IsoAlignedMemoryAllocator):
(JSC::IsoAlignedMemoryAllocator::tryAllocateAlignedMemory):
(JSC::IsoAlignedMemoryAllocator::freeAlignedMemory):
(JSC::IsoAlignedMemoryAllocator::dump const):
* heap/IsoAlignedMemoryAllocator.h: Added.
* heap/IsoSubspace.cpp: Added.
(JSC::IsoSubspace::IsoSubspace):
(JSC::IsoSubspace::~IsoSubspace):
(JSC::IsoSubspace::allocatorFor):
(JSC::IsoSubspace::allocatorForNonVirtual):
(JSC::IsoSubspace::allocate):
(JSC::IsoSubspace::allocateNonVirtual):
* heap/IsoSubspace.h: Added.
(JSC::IsoSubspace::size const):
* heap/MarkedAllocator.cpp:
(JSC::MarkedAllocator::MarkedAllocator):
(JSC::MarkedAllocator::setSubspace):
(JSC::MarkedAllocator::allocateSlowCase):
(JSC::MarkedAllocator::tryAllocateSlowCase): Deleted.
(JSC::MarkedAllocator::allocateSlowCaseImpl): Deleted.
* heap/MarkedAllocator.h:
(JSC::MarkedAllocator::nextAllocatorInAlignedMemoryAllocator const):
(JSC::MarkedAllocator::setNextAllocatorInAlignedMemoryAllocator):
* heap/MarkedAllocatorInlines.h:
(JSC::MarkedAllocator::allocate):
(JSC::MarkedAllocator::tryAllocate): Deleted.
* heap/MarkedBlock.h:
* heap/MarkedBlockInlines.h:
(JSC::MarkedBlock::Handle::finishSweepKnowingHeapCellType):
(JSC::MarkedBlock::Handle::finishSweepKnowingSubspace): Deleted.
* heap/MarkedSpace.cpp:
(JSC::MarkedSpace::addMarkedAllocator):
* heap/MarkedSpace.h:
* heap/Subspace.cpp:
(JSC::Subspace::Subspace):
(JSC::Subspace::initialize):
(JSC::Subspace::finishSweep):
(JSC::Subspace::destroy):
(JSC::Subspace::prepareForAllocation):
(JSC::Subspace::findEmptyBlockToSteal):
(): Deleted.
(JSC::Subspace::allocate): Deleted.
(JSC::Subspace::tryAllocate): Deleted.
(JSC::Subspace::allocatorForSlow): Deleted.
(JSC::Subspace::allocateSlow): Deleted.
(JSC::Subspace::tryAllocateSlow): Deleted.
(JSC::Subspace::didAllocate): Deleted.
* heap/Subspace.h:
(JSC::Subspace::heapCellType const):
(JSC::Subspace::nextSubspaceInAlignedMemoryAllocator const):
(JSC::Subspace::setNextSubspaceInAlignedMemoryAllocator):
(JSC::Subspace::offsetOfAllocatorForSizeStep): Deleted.
(JSC::Subspace::allocatorForSizeStep): Deleted.
(JSC::Subspace::tryAllocatorFor): Deleted.
(JSC::Subspace::allocatorFor): Deleted.
* jit/AssemblyHelpers.h:
(JSC::AssemblyHelpers::emitAllocateJSObjectWithKnownSize):
(JSC::AssemblyHelpers::emitAllocateVariableSized):
(JSC::AssemblyHelpers::emitAllocateVariableSizedCell):
* jit/JITOpcodes.cpp:
(JSC::JIT::emit_op_new_object):
* runtime/ButterflyInlines.h:
(JSC::Butterfly::createUninitialized):
(JSC::Butterfly::tryCreate):
(JSC::Butterfly::growArrayRight):
* runtime/DirectArguments.cpp:
(JSC::DirectArguments::overrideThings):
* runtime/DirectArguments.h:
(JSC::DirectArguments::subspaceFor):
* runtime/DirectEvalExecutable.h:
* runtime/EvalExecutable.h:
* runtime/ExecutableBase.h:
(JSC::ExecutableBase::subspaceFor):
* runtime/FunctionExecutable.h:
* runtime/GenericArgumentsInlines.h:
(JSC::GenericArguments<Type>::initModifiedArgumentsDescriptor):
* runtime/HashMapImpl.h:
(JSC::HashMapBuffer::create):
* runtime/IndirectEvalExecutable.h:
* runtime/JSArray.cpp:
(JSC::JSArray::tryCreateUninitializedRestricted):
(JSC::JSArray::unshiftCountSlowCase):
* runtime/JSArray.h:
(JSC::JSArray::tryCreate):
* runtime/JSArrayBufferView.cpp:
(JSC::JSArrayBufferView::ConstructionContext::ConstructionContext):
* runtime/JSCell.h:
(JSC::subspaceFor):
* runtime/JSCellInlines.h:
(JSC::JSCell::subspaceFor):
(JSC::tryAllocateCellHelper):
(JSC::allocateCell):
(JSC::tryAllocateCell):
* runtime/JSDestructibleObject.h:
(JSC::JSDestructibleObject::subspaceFor):
* runtime/JSDestructibleObjectHeapCellType.cpp: Copied from Source/JavaScriptCore/runtime/JSDestructibleObjectSubspace.cpp.
(JSC::JSDestructibleObjectHeapCellType::JSDestructibleObjectHeapCellType):
(JSC::JSDestructibleObjectHeapCellType::~JSDestructibleObjectHeapCellType):
(JSC::JSDestructibleObjectHeapCellType::finishSweep):
(JSC::JSDestructibleObjectHeapCellType::destroy):
(JSC::JSDestructibleObjectSubspace::JSDestructibleObjectSubspace): Deleted.
(JSC::JSDestructibleObjectSubspace::~JSDestructibleObjectSubspace): Deleted.
(JSC::JSDestructibleObjectSubspace::finishSweep): Deleted.
(JSC::JSDestructibleObjectSubspace::destroy): Deleted.
* runtime/JSDestructibleObjectHeapCellType.h: Copied from Source/JavaScriptCore/runtime/JSDestructibleObjectSubspace.h.
* runtime/JSDestructibleObjectSubspace.cpp: Removed.
* runtime/JSDestructibleObjectSubspace.h: Removed.
* runtime/JSLexicalEnvironment.h:
(JSC::JSLexicalEnvironment::subspaceFor):
* runtime/JSSegmentedVariableObject.h:
(JSC::JSSegmentedVariableObject::subspaceFor):
* runtime/JSSegmentedVariableObjectHeapCellType.cpp: Copied from Source/JavaScriptCore/runtime/JSSegmentedVariableObjectSubspace.cpp.
(JSC::JSSegmentedVariableObjectHeapCellType::JSSegmentedVariableObjectHeapCellType):
(JSC::JSSegmentedVariableObjectHeapCellType::~JSSegmentedVariableObjectHeapCellType):
(JSC::JSSegmentedVariableObjectHeapCellType::finishSweep):
(JSC::JSSegmentedVariableObjectHeapCellType::destroy):
(JSC::JSSegmentedVariableObjectSubspace::JSSegmentedVariableObjectSubspace): Deleted.
(JSC::JSSegmentedVariableObjectSubspace::~JSSegmentedVariableObjectSubspace): Deleted.
(JSC::JSSegmentedVariableObjectSubspace::finishSweep): Deleted.
(JSC::JSSegmentedVariableObjectSubspace::destroy): Deleted.
* runtime/JSSegmentedVariableObjectHeapCellType.h: Copied from Source/JavaScriptCore/runtime/JSSegmentedVariableObjectSubspace.h.
* runtime/JSSegmentedVariableObjectSubspace.cpp: Removed.
* runtime/JSSegmentedVariableObjectSubspace.h: Removed.
* runtime/JSString.h:
(JSC::JSString::subspaceFor):
* runtime/JSStringHeapCellType.cpp: Copied from Source/JavaScriptCore/runtime/JSStringSubspace.cpp.
(JSC::JSStringHeapCellType::JSStringHeapCellType):
(JSC::JSStringHeapCellType::~JSStringHeapCellType):
(JSC::JSStringHeapCellType::finishSweep):
(JSC::JSStringHeapCellType::destroy):
(JSC::JSStringSubspace::JSStringSubspace): Deleted.
(JSC::JSStringSubspace::~JSStringSubspace): Deleted.
(JSC::JSStringSubspace::finishSweep): Deleted.
(JSC::JSStringSubspace::destroy): Deleted.
* runtime/JSStringHeapCellType.h: Copied from Source/JavaScriptCore/runtime/JSStringSubspace.h.
* runtime/JSStringSubspace.cpp: Removed.
* runtime/JSStringSubspace.h: Removed.
* runtime/ModuleProgramExecutable.h:
* runtime/NativeExecutable.h:
* runtime/ProgramExecutable.h:
* runtime/RegExpMatchesArray.h:
(JSC::tryCreateUninitializedRegExpMatchesArray):
* runtime/ScopedArguments.h:
(JSC::ScopedArguments::subspaceFor):
* runtime/VM.cpp:
(JSC::VM::VM):
* runtime/VM.h:
(JSC::VM::gigacageAuxiliarySpace):
* wasm/js/JSWebAssemblyCodeBlock.h:
* wasm/js/JSWebAssemblyCodeBlockHeapCellType.cpp: Copied from Source/JavaScriptCore/wasm/js/JSWebAssemblyCodeBlockSubspace.cpp.
(JSC::JSWebAssemblyCodeBlockHeapCellType::JSWebAssemblyCodeBlockHeapCellType):
(JSC::JSWebAssemblyCodeBlockHeapCellType::~JSWebAssemblyCodeBlockHeapCellType):
(JSC::JSWebAssemblyCodeBlockHeapCellType::finishSweep):
(JSC::JSWebAssemblyCodeBlockHeapCellType::destroy):
(JSC::JSWebAssemblyCodeBlockSubspace::JSWebAssemblyCodeBlockSubspace): Deleted.
(JSC::JSWebAssemblyCodeBlockSubspace::~JSWebAssemblyCodeBlockSubspace): Deleted.
(JSC::JSWebAssemblyCodeBlockSubspace::finishSweep): Deleted.
(JSC::JSWebAssemblyCodeBlockSubspace::destroy): Deleted.
* wasm/js/JSWebAssemblyCodeBlockHeapCellType.h: Copied from Source/JavaScriptCore/wasm/js/JSWebAssemblyCodeBlockSubspace.h.
* wasm/js/JSWebAssemblyCodeBlockSubspace.cpp: Removed.
* wasm/js/JSWebAssemblyCodeBlockSubspace.h: Removed.
* wasm/js/JSWebAssemblyMemory.h:
(JSC::JSWebAssemblyMemory::subspaceFor):

Source/WebCore:

No new tests because no new behavior.

Adopting changes in JSC Subspace API.

* ForwardingHeaders/runtime/JSDestructibleObjectHeapCellType.h: Added.
* ForwardingHeaders/runtime/JSSegmentedVariableObjectHeapCellType.h: Added.
* bindings/js/JSDOMWrapper.cpp:
(WebCore::outputConstraintSubspaceFor):
(WebCore::globalObjectOutputConstraintSubspaceFor):
* bindings/js/JSDOMWrapper.h:
* bindings/js/WebCoreJSClientData.cpp:
(WebCore::JSVMClientData::JSVMClientData):
* bindings/js/WebCoreJSClientData.h:
(WebCore::JSVMClientData::outputConstraintSpace):
(WebCore::JSVMClientData::globalObjectOutputConstraintSpace):
* bindings/scripts/CodeGeneratorJS.pm:
(GenerateHeader):

Source/WTF:

One of my favorite data structures in the GC is a singly-linked list that knows its tail, so that
things get added to it at the end rather that at the beginning. In this patch, I use this to put
the same node on multiple lists, which our existing linked list templates also don't support.

This adds a new linked list that does those things:

- It supports append(). It could also support prepend(), but currently there is no need for it.

- It supports nodes that are on multiple lists. The GC uses std::mem_fn() to create a lambda that the
  list uses to set next.

* WTF.xcodeproj/project.pbxproj:
* wtf/SinglyLinkedListWithTail.h: Added.
(WTF::SinglyLinkedListWithTail::isEmpty const):
(WTF::SinglyLinkedListWithTail::append):
(WTF::SinglyLinkedListWithTail::first const):
(WTF::SinglyLinkedListWithTail::last const):

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@225314 268f45cc-cd09-0410-ab3c-d52691b4dbfc

81 files changed:
Source/JavaScriptCore/ChangeLog
Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj
Source/JavaScriptCore/Sources.txt
Source/JavaScriptCore/bytecode/AccessCase.cpp
Source/JavaScriptCore/bytecode/ObjectAllocationProfileInlines.h
Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp
Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp
Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp
Source/JavaScriptCore/ftl/FTLAbstractHeapRepository.h
Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp
Source/JavaScriptCore/heap/AlignedMemoryAllocator.cpp
Source/JavaScriptCore/heap/AlignedMemoryAllocator.h
Source/JavaScriptCore/heap/AllocationFailureMode.h [new file with mode: 0644]
Source/JavaScriptCore/heap/AllocatorForMode.h [new file with mode: 0644]
Source/JavaScriptCore/heap/CompleteSubspace.cpp [new file with mode: 0644]
Source/JavaScriptCore/heap/CompleteSubspace.h [new file with mode: 0644]
Source/JavaScriptCore/heap/HeapCellType.cpp [new file with mode: 0644]
Source/JavaScriptCore/heap/HeapCellType.h [new file with mode: 0644]
Source/JavaScriptCore/heap/IsoAlignedMemoryAllocator.cpp [new file with mode: 0644]
Source/JavaScriptCore/heap/IsoAlignedMemoryAllocator.h [new file with mode: 0644]
Source/JavaScriptCore/heap/IsoSubspace.cpp [new file with mode: 0644]
Source/JavaScriptCore/heap/IsoSubspace.h [new file with mode: 0644]
Source/JavaScriptCore/heap/MarkedAllocator.cpp
Source/JavaScriptCore/heap/MarkedAllocator.h
Source/JavaScriptCore/heap/MarkedAllocatorInlines.h
Source/JavaScriptCore/heap/MarkedBlock.h
Source/JavaScriptCore/heap/MarkedBlockInlines.h
Source/JavaScriptCore/heap/MarkedSpace.cpp
Source/JavaScriptCore/heap/MarkedSpace.h
Source/JavaScriptCore/heap/Subspace.cpp
Source/JavaScriptCore/heap/Subspace.h
Source/JavaScriptCore/jit/AssemblyHelpers.h
Source/JavaScriptCore/jit/JITOpcodes.cpp
Source/JavaScriptCore/jit/JITOpcodes32_64.cpp
Source/JavaScriptCore/runtime/ButterflyInlines.h
Source/JavaScriptCore/runtime/DirectArguments.cpp
Source/JavaScriptCore/runtime/DirectArguments.h
Source/JavaScriptCore/runtime/DirectEvalExecutable.h
Source/JavaScriptCore/runtime/EvalExecutable.h
Source/JavaScriptCore/runtime/ExecutableBase.h
Source/JavaScriptCore/runtime/FunctionExecutable.h
Source/JavaScriptCore/runtime/GenericArgumentsInlines.h
Source/JavaScriptCore/runtime/HashMapImpl.h
Source/JavaScriptCore/runtime/IndirectEvalExecutable.h
Source/JavaScriptCore/runtime/JSArray.cpp
Source/JavaScriptCore/runtime/JSArray.h
Source/JavaScriptCore/runtime/JSArrayBufferView.cpp
Source/JavaScriptCore/runtime/JSCell.h
Source/JavaScriptCore/runtime/JSCellInlines.h
Source/JavaScriptCore/runtime/JSDestructibleObject.h
Source/JavaScriptCore/runtime/JSDestructibleObjectHeapCellType.cpp [moved from Source/JavaScriptCore/runtime/JSDestructibleObjectSubspace.cpp with 75% similarity]
Source/JavaScriptCore/runtime/JSDestructibleObjectHeapCellType.h [moved from Source/JavaScriptCore/runtime/JSDestructibleObjectSubspace.h with 86% similarity]
Source/JavaScriptCore/runtime/JSLexicalEnvironment.h
Source/JavaScriptCore/runtime/JSSegmentedVariableObject.h
Source/JavaScriptCore/runtime/JSSegmentedVariableObjectHeapCellType.cpp [moved from Source/JavaScriptCore/runtime/JSSegmentedVariableObjectSubspace.cpp with 73% similarity]
Source/JavaScriptCore/runtime/JSSegmentedVariableObjectHeapCellType.h [moved from Source/JavaScriptCore/runtime/JSSegmentedVariableObjectSubspace.h with 85% similarity]
Source/JavaScriptCore/runtime/JSString.h
Source/JavaScriptCore/runtime/JSStringHeapCellType.cpp [moved from Source/JavaScriptCore/runtime/JSStringSubspace.cpp with 77% similarity]
Source/JavaScriptCore/runtime/JSStringHeapCellType.h [moved from Source/JavaScriptCore/runtime/JSStringSubspace.h with 88% similarity]
Source/JavaScriptCore/runtime/ModuleProgramExecutable.h
Source/JavaScriptCore/runtime/NativeExecutable.h
Source/JavaScriptCore/runtime/ProgramExecutable.h
Source/JavaScriptCore/runtime/RegExpMatchesArray.h
Source/JavaScriptCore/runtime/ScopedArguments.h
Source/JavaScriptCore/runtime/VM.cpp
Source/JavaScriptCore/runtime/VM.h
Source/JavaScriptCore/wasm/js/JSWebAssemblyCodeBlock.h
Source/JavaScriptCore/wasm/js/JSWebAssemblyCodeBlockHeapCellType.cpp [moved from Source/JavaScriptCore/wasm/js/JSWebAssemblyCodeBlockSubspace.cpp with 76% similarity]
Source/JavaScriptCore/wasm/js/JSWebAssemblyCodeBlockHeapCellType.h [moved from Source/JavaScriptCore/wasm/js/JSWebAssemblyCodeBlockSubspace.h with 88% similarity]
Source/JavaScriptCore/wasm/js/JSWebAssemblyMemory.h
Source/WTF/ChangeLog
Source/WTF/WTF.xcodeproj/project.pbxproj
Source/WTF/wtf/SinglyLinkedListWithTail.h [new file with mode: 0644]
Source/WebCore/ChangeLog
Source/WebCore/ForwardingHeaders/runtime/JSDestructibleObjectHeapCellType.h [new file with mode: 0644]
Source/WebCore/ForwardingHeaders/runtime/JSSegmentedVariableObjectHeapCellType.h [new file with mode: 0644]
Source/WebCore/bindings/js/JSDOMWrapper.cpp
Source/WebCore/bindings/js/JSDOMWrapper.h
Source/WebCore/bindings/js/WebCoreJSClientData.cpp
Source/WebCore/bindings/js/WebCoreJSClientData.h
Source/WebCore/bindings/scripts/CodeGeneratorJS.pm

index 777834c..78661e8 100644 (file)
@@ -1,3 +1,240 @@
+2017-11-29  Filip Pizlo  <fpizlo@apple.com>
+
+        GC should support isoheaps
+        https://bugs.webkit.org/show_bug.cgi?id=179288
+
+        Reviewed by Saam Barati.
+        
+        This expands the power of the Subspace API in JSC:
+        
+        - Everything associated with describing the types of objects is now part of the HeapCellType class.
+          We have different HeapCellTypes for different destruction strategies. Any Subspace can use any
+          HeapCellType; these are orthogonal things.
+        
+        - There are now two variants of Subspace: CompleteSubspace, which can allocate any size objects using
+          any AlignedMemoryAllocator; and IsoSubspace, which can allocate just one size of object and uses a
+          special virtual memory pool for that purpose. Like bmalloc's IsoHeap, IsoSubspace hoards virtual
+          pages but releases the physical pages as part of the respective allocator's scavenging policy
+          (the Scavenger in bmalloc for IsoHeap and the incremental sweep and full sweep in Riptide for
+          IsoSubspace).
+        
+        So far, this patch just puts subtypes of ExecutableBase in IsoSubspaces. If it works, we can use it
+        for more things.
+        
+        This does not have any effect on JetStream (0.18% faster with p = 0.69).
+
+        * JavaScriptCore.xcodeproj/project.pbxproj:
+        * Sources.txt:
+        * bytecode/AccessCase.cpp:
+        (JSC::AccessCase::generateImpl):
+        * bytecode/ObjectAllocationProfileInlines.h:
+        (JSC::ObjectAllocationProfile::initializeProfile):
+        * dfg/DFGSpeculativeJIT.cpp:
+        (JSC::DFG::SpeculativeJIT::emitAllocateRawObject):
+        (JSC::DFG::SpeculativeJIT::compileMakeRope):
+        (JSC::DFG::SpeculativeJIT::compileAllocatePropertyStorage):
+        (JSC::DFG::SpeculativeJIT::compileReallocatePropertyStorage):
+        * dfg/DFGSpeculativeJIT64.cpp:
+        (JSC::DFG::SpeculativeJIT::compile):
+        * ftl/FTLAbstractHeapRepository.h:
+        * ftl/FTLLowerDFGToB3.cpp:
+        (JSC::FTL::DFG::LowerDFGToB3::compileMakeRope):
+        (JSC::FTL::DFG::LowerDFGToB3::compileMaterializeNewObject):
+        (JSC::FTL::DFG::LowerDFGToB3::allocatePropertyStorageWithSizeImpl):
+        (JSC::FTL::DFG::LowerDFGToB3::allocateObject):
+        (JSC::FTL::DFG::LowerDFGToB3::allocatorForSize):
+        * heap/AlignedMemoryAllocator.cpp:
+        (JSC::AlignedMemoryAllocator::registerAllocator):
+        (JSC::AlignedMemoryAllocator::registerSubspace):
+        * heap/AlignedMemoryAllocator.h:
+        (JSC::AlignedMemoryAllocator::firstAllocator const):
+        * heap/AllocationFailureMode.h: Added.
+        * heap/CompleteSubspace.cpp: Added.
+        (JSC::CompleteSubspace::CompleteSubspace):
+        (JSC::CompleteSubspace::~CompleteSubspace):
+        (JSC::CompleteSubspace::allocatorFor):
+        (JSC::CompleteSubspace::allocate):
+        (JSC::CompleteSubspace::allocateNonVirtual):
+        (JSC::CompleteSubspace::allocatorForSlow):
+        (JSC::CompleteSubspace::allocateSlow):
+        (JSC::CompleteSubspace::tryAllocateSlow):
+        * heap/CompleteSubspace.h: Added.
+        (JSC::CompleteSubspace::offsetOfAllocatorForSizeStep):
+        (JSC::CompleteSubspace::allocatorForSizeStep):
+        (JSC::CompleteSubspace::allocatorForNonVirtual):
+        * heap/HeapCellType.cpp: Added.
+        (JSC::HeapCellType::HeapCellType):
+        (JSC::HeapCellType::~HeapCellType):
+        (JSC::HeapCellType::finishSweep):
+        (JSC::HeapCellType::destroy):
+        * heap/HeapCellType.h: Added.
+        (JSC::HeapCellType::attributes const):
+        * heap/IsoAlignedMemoryAllocator.cpp: Added.
+        (JSC::IsoAlignedMemoryAllocator::IsoAlignedMemoryAllocator):
+        (JSC::IsoAlignedMemoryAllocator::~IsoAlignedMemoryAllocator):
+        (JSC::IsoAlignedMemoryAllocator::tryAllocateAlignedMemory):
+        (JSC::IsoAlignedMemoryAllocator::freeAlignedMemory):
+        (JSC::IsoAlignedMemoryAllocator::dump const):
+        * heap/IsoAlignedMemoryAllocator.h: Added.
+        * heap/IsoSubspace.cpp: Added.
+        (JSC::IsoSubspace::IsoSubspace):
+        (JSC::IsoSubspace::~IsoSubspace):
+        (JSC::IsoSubspace::allocatorFor):
+        (JSC::IsoSubspace::allocatorForNonVirtual):
+        (JSC::IsoSubspace::allocate):
+        (JSC::IsoSubspace::allocateNonVirtual):
+        * heap/IsoSubspace.h: Added.
+        (JSC::IsoSubspace::size const):
+        * heap/MarkedAllocator.cpp:
+        (JSC::MarkedAllocator::MarkedAllocator):
+        (JSC::MarkedAllocator::setSubspace):
+        (JSC::MarkedAllocator::allocateSlowCase):
+        (JSC::MarkedAllocator::tryAllocateSlowCase): Deleted.
+        (JSC::MarkedAllocator::allocateSlowCaseImpl): Deleted.
+        * heap/MarkedAllocator.h:
+        (JSC::MarkedAllocator::nextAllocatorInAlignedMemoryAllocator const):
+        (JSC::MarkedAllocator::setNextAllocatorInAlignedMemoryAllocator):
+        * heap/MarkedAllocatorInlines.h:
+        (JSC::MarkedAllocator::allocate):
+        (JSC::MarkedAllocator::tryAllocate): Deleted.
+        * heap/MarkedBlock.h:
+        * heap/MarkedBlockInlines.h:
+        (JSC::MarkedBlock::Handle::finishSweepKnowingHeapCellType):
+        (JSC::MarkedBlock::Handle::finishSweepKnowingSubspace): Deleted.
+        * heap/MarkedSpace.cpp:
+        (JSC::MarkedSpace::addMarkedAllocator):
+        * heap/MarkedSpace.h:
+        * heap/Subspace.cpp:
+        (JSC::Subspace::Subspace):
+        (JSC::Subspace::initialize):
+        (JSC::Subspace::finishSweep):
+        (JSC::Subspace::destroy):
+        (JSC::Subspace::prepareForAllocation):
+        (JSC::Subspace::findEmptyBlockToSteal):
+        (): Deleted.
+        (JSC::Subspace::allocate): Deleted.
+        (JSC::Subspace::tryAllocate): Deleted.
+        (JSC::Subspace::allocatorForSlow): Deleted.
+        (JSC::Subspace::allocateSlow): Deleted.
+        (JSC::Subspace::tryAllocateSlow): Deleted.
+        (JSC::Subspace::didAllocate): Deleted.
+        * heap/Subspace.h:
+        (JSC::Subspace::heapCellType const):
+        (JSC::Subspace::nextSubspaceInAlignedMemoryAllocator const):
+        (JSC::Subspace::setNextSubspaceInAlignedMemoryAllocator):
+        (JSC::Subspace::offsetOfAllocatorForSizeStep): Deleted.
+        (JSC::Subspace::allocatorForSizeStep): Deleted.
+        (JSC::Subspace::tryAllocatorFor): Deleted.
+        (JSC::Subspace::allocatorFor): Deleted.
+        * jit/AssemblyHelpers.h:
+        (JSC::AssemblyHelpers::emitAllocateJSObjectWithKnownSize):
+        (JSC::AssemblyHelpers::emitAllocateVariableSized):
+        (JSC::AssemblyHelpers::emitAllocateVariableSizedCell):
+        * jit/JITOpcodes.cpp:
+        (JSC::JIT::emit_op_new_object):
+        * runtime/ButterflyInlines.h:
+        (JSC::Butterfly::createUninitialized):
+        (JSC::Butterfly::tryCreate):
+        (JSC::Butterfly::growArrayRight):
+        * runtime/DirectArguments.cpp:
+        (JSC::DirectArguments::overrideThings):
+        * runtime/DirectArguments.h:
+        (JSC::DirectArguments::subspaceFor):
+        * runtime/DirectEvalExecutable.h:
+        * runtime/EvalExecutable.h:
+        * runtime/ExecutableBase.h:
+        (JSC::ExecutableBase::subspaceFor):
+        * runtime/FunctionExecutable.h:
+        * runtime/GenericArgumentsInlines.h:
+        (JSC::GenericArguments<Type>::initModifiedArgumentsDescriptor):
+        * runtime/HashMapImpl.h:
+        (JSC::HashMapBuffer::create):
+        * runtime/IndirectEvalExecutable.h:
+        * runtime/JSArray.cpp:
+        (JSC::JSArray::tryCreateUninitializedRestricted):
+        (JSC::JSArray::unshiftCountSlowCase):
+        * runtime/JSArray.h:
+        (JSC::JSArray::tryCreate):
+        * runtime/JSArrayBufferView.cpp:
+        (JSC::JSArrayBufferView::ConstructionContext::ConstructionContext):
+        * runtime/JSCell.h:
+        (JSC::subspaceFor):
+        * runtime/JSCellInlines.h:
+        (JSC::JSCell::subspaceFor):
+        (JSC::tryAllocateCellHelper):
+        (JSC::allocateCell):
+        (JSC::tryAllocateCell):
+        * runtime/JSDestructibleObject.h:
+        (JSC::JSDestructibleObject::subspaceFor):
+        * runtime/JSDestructibleObjectHeapCellType.cpp: Copied from Source/JavaScriptCore/runtime/JSDestructibleObjectSubspace.cpp.
+        (JSC::JSDestructibleObjectHeapCellType::JSDestructibleObjectHeapCellType):
+        (JSC::JSDestructibleObjectHeapCellType::~JSDestructibleObjectHeapCellType):
+        (JSC::JSDestructibleObjectHeapCellType::finishSweep):
+        (JSC::JSDestructibleObjectHeapCellType::destroy):
+        (JSC::JSDestructibleObjectSubspace::JSDestructibleObjectSubspace): Deleted.
+        (JSC::JSDestructibleObjectSubspace::~JSDestructibleObjectSubspace): Deleted.
+        (JSC::JSDestructibleObjectSubspace::finishSweep): Deleted.
+        (JSC::JSDestructibleObjectSubspace::destroy): Deleted.
+        * runtime/JSDestructibleObjectHeapCellType.h: Copied from Source/JavaScriptCore/runtime/JSDestructibleObjectSubspace.h.
+        * runtime/JSDestructibleObjectSubspace.cpp: Removed.
+        * runtime/JSDestructibleObjectSubspace.h: Removed.
+        * runtime/JSLexicalEnvironment.h:
+        (JSC::JSLexicalEnvironment::subspaceFor):
+        * runtime/JSSegmentedVariableObject.h:
+        (JSC::JSSegmentedVariableObject::subspaceFor):
+        * runtime/JSSegmentedVariableObjectHeapCellType.cpp: Copied from Source/JavaScriptCore/runtime/JSSegmentedVariableObjectSubspace.cpp.
+        (JSC::JSSegmentedVariableObjectHeapCellType::JSSegmentedVariableObjectHeapCellType):
+        (JSC::JSSegmentedVariableObjectHeapCellType::~JSSegmentedVariableObjectHeapCellType):
+        (JSC::JSSegmentedVariableObjectHeapCellType::finishSweep):
+        (JSC::JSSegmentedVariableObjectHeapCellType::destroy):
+        (JSC::JSSegmentedVariableObjectSubspace::JSSegmentedVariableObjectSubspace): Deleted.
+        (JSC::JSSegmentedVariableObjectSubspace::~JSSegmentedVariableObjectSubspace): Deleted.
+        (JSC::JSSegmentedVariableObjectSubspace::finishSweep): Deleted.
+        (JSC::JSSegmentedVariableObjectSubspace::destroy): Deleted.
+        * runtime/JSSegmentedVariableObjectHeapCellType.h: Copied from Source/JavaScriptCore/runtime/JSSegmentedVariableObjectSubspace.h.
+        * runtime/JSSegmentedVariableObjectSubspace.cpp: Removed.
+        * runtime/JSSegmentedVariableObjectSubspace.h: Removed.
+        * runtime/JSString.h:
+        (JSC::JSString::subspaceFor):
+        * runtime/JSStringHeapCellType.cpp: Copied from Source/JavaScriptCore/runtime/JSStringSubspace.cpp.
+        (JSC::JSStringHeapCellType::JSStringHeapCellType):
+        (JSC::JSStringHeapCellType::~JSStringHeapCellType):
+        (JSC::JSStringHeapCellType::finishSweep):
+        (JSC::JSStringHeapCellType::destroy):
+        (JSC::JSStringSubspace::JSStringSubspace): Deleted.
+        (JSC::JSStringSubspace::~JSStringSubspace): Deleted.
+        (JSC::JSStringSubspace::finishSweep): Deleted.
+        (JSC::JSStringSubspace::destroy): Deleted.
+        * runtime/JSStringHeapCellType.h: Copied from Source/JavaScriptCore/runtime/JSStringSubspace.h.
+        * runtime/JSStringSubspace.cpp: Removed.
+        * runtime/JSStringSubspace.h: Removed.
+        * runtime/ModuleProgramExecutable.h:
+        * runtime/NativeExecutable.h:
+        * runtime/ProgramExecutable.h:
+        * runtime/RegExpMatchesArray.h:
+        (JSC::tryCreateUninitializedRegExpMatchesArray):
+        * runtime/ScopedArguments.h:
+        (JSC::ScopedArguments::subspaceFor):
+        * runtime/VM.cpp:
+        (JSC::VM::VM):
+        * runtime/VM.h:
+        (JSC::VM::gigacageAuxiliarySpace):
+        * wasm/js/JSWebAssemblyCodeBlock.h:
+        * wasm/js/JSWebAssemblyCodeBlockHeapCellType.cpp: Copied from Source/JavaScriptCore/wasm/js/JSWebAssemblyCodeBlockSubspace.cpp.
+        (JSC::JSWebAssemblyCodeBlockHeapCellType::JSWebAssemblyCodeBlockHeapCellType):
+        (JSC::JSWebAssemblyCodeBlockHeapCellType::~JSWebAssemblyCodeBlockHeapCellType):
+        (JSC::JSWebAssemblyCodeBlockHeapCellType::finishSweep):
+        (JSC::JSWebAssemblyCodeBlockHeapCellType::destroy):
+        (JSC::JSWebAssemblyCodeBlockSubspace::JSWebAssemblyCodeBlockSubspace): Deleted.
+        (JSC::JSWebAssemblyCodeBlockSubspace::~JSWebAssemblyCodeBlockSubspace): Deleted.
+        (JSC::JSWebAssemblyCodeBlockSubspace::finishSweep): Deleted.
+        (JSC::JSWebAssemblyCodeBlockSubspace::destroy): Deleted.
+        * wasm/js/JSWebAssemblyCodeBlockHeapCellType.h: Copied from Source/JavaScriptCore/wasm/js/JSWebAssemblyCodeBlockSubspace.h.
+        * wasm/js/JSWebAssemblyCodeBlockSubspace.cpp: Removed.
+        * wasm/js/JSWebAssemblyCodeBlockSubspace.h: Removed.
+        * wasm/js/JSWebAssemblyMemory.h:
+        (JSC::JSWebAssemblyMemory::subspaceFor):
+
 2017-11-29  Saam Barati  <sbarati@apple.com>
 
         Remove pointer caging for double arrays
index c29130b..0f50e1c 100644 (file)
                0F2FCCFF18A60070001A27F8 /* DFGThreadData.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F2FCCF818A60070001A27F8 /* DFGThreadData.h */; };
                0F300B7818AB051100A6D72E /* DFGNodeOrigin.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F300B7718AB051100A6D72E /* DFGNodeOrigin.h */; };
                0F300B7C18AB1B1400A6D72E /* DFGIntegerCheckCombiningPhase.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F300B7A18AB1B1400A6D72E /* DFGIntegerCheckCombiningPhase.h */; };
+               0F30CB5E1FCE4E37004B5323 /* AllocatorForMode.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F30CB5D1FCE46B4004B5323 /* AllocatorForMode.h */; settings = {ATTRIBUTES = (Private, ); }; };
                0F30D7C01D95D6320053089D /* CPU.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F30D7BF1D95D62F0053089D /* CPU.h */; settings = {ATTRIBUTES = (Private, ); }; };
                0F30FB611DC2DE99003124F2 /* ArrayBufferSharingMode.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F30FB601DC2DE96003124F2 /* ArrayBufferSharingMode.h */; settings = {ATTRIBUTES = (Private, ); }; };
                0F32BD111BB34F190093A57F /* HeapHelperPool.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F32BD0F1BB34F190093A57F /* HeapHelperPool.h */; };
                0F4C91661C29F4F2004341A6 /* B3OriginDump.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F4C91651C29F4F2004341A6 /* B3OriginDump.h */; };
                0F4DE1CF1C4C1B54004D6C11 /* AirFixObviousSpills.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F4DE1CD1C4C1B54004D6C11 /* AirFixObviousSpills.h */; };
                0F4F29E018B6AD1C0057BC15 /* DFGStaticExecutionCountEstimationPhase.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F4F29DE18B6AD1C0057BC15 /* DFGStaticExecutionCountEstimationPhase.h */; };
-               0F4F82881E2FFDE00075184C /* JSSegmentedVariableObjectSubspace.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F4F82861E2FFDDB0075184C /* JSSegmentedVariableObjectSubspace.h */; settings = {ATTRIBUTES = (Private, ); }; };
+               0F4F82881E2FFDE00075184C /* JSSegmentedVariableObjectHeapCellType.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F4F82861E2FFDDB0075184C /* JSSegmentedVariableObjectHeapCellType.h */; settings = {ATTRIBUTES = (Private, ); }; };
                0F4F828C1E31B9760075184C /* StochasticSpaceTimeMutatorScheduler.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F4F828A1E31B9710075184C /* StochasticSpaceTimeMutatorScheduler.h */; };
                0F50AF3C193E8B3900674EE8 /* DFGStructureClobberState.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F50AF3B193E8B3900674EE8 /* DFGStructureClobberState.h */; };
                0F5513A61D5A682C00C32BD8 /* FreeList.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F5513A51D5A682A00C32BD8 /* FreeList.h */; settings = {ATTRIBUTES = (Private, ); }; };
                0F7DF1351E2970DC0095951B /* MarkedSpaceInlines.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F7DF1301E2970D50095951B /* MarkedSpaceInlines.h */; settings = {ATTRIBUTES = (Private, ); }; };
                0F7DF1371E2970E10095951B /* Subspace.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F7DF1321E2970D50095951B /* Subspace.h */; settings = {ATTRIBUTES = (Private, ); }; };
                0F7DF1381E2970E40095951B /* SubspaceInlines.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F7DF1331E2970D50095951B /* SubspaceInlines.h */; settings = {ATTRIBUTES = (Private, ); }; };
-               0F7DF13C1E2971130095951B /* JSDestructibleObjectSubspace.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F7DF13A1E29710E0095951B /* JSDestructibleObjectSubspace.h */; settings = {ATTRIBUTES = (Private, ); }; };
-               0F7DF13F1E2AFC4D0095951B /* JSStringSubspace.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F7DF13E1E2AFC4B0095951B /* JSStringSubspace.h */; settings = {ATTRIBUTES = (Private, ); }; };
+               0F7DF13C1E2971130095951B /* JSDestructibleObjectHeapCellType.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F7DF13A1E29710E0095951B /* JSDestructibleObjectHeapCellType.h */; settings = {ATTRIBUTES = (Private, ); }; };
+               0F7DF13F1E2AFC4D0095951B /* JSStringHeapCellType.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F7DF13E1E2AFC4B0095951B /* JSStringHeapCellType.h */; settings = {ATTRIBUTES = (Private, ); }; };
                0F7DF1461E2BEF6A0095951B /* MarkedAllocatorInlines.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F7DF1451E2BEF680095951B /* MarkedAllocatorInlines.h */; settings = {ATTRIBUTES = (Private, ); }; };
                0F7F988C1D9596C800F4F12E /* DFGStoreBarrierClusteringPhase.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F7F988A1D9596C300F4F12E /* DFGStoreBarrierClusteringPhase.h */; };
                0F8023EA1613832B00A0BA45 /* ByValInfo.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F8023E91613832300A0BA45 /* ByValInfo.h */; settings = {ATTRIBUTES = (Private, ); }; };
                0FDB2CCA173DA523007B3C1B /* FTLValueFromBlock.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FDB2CC8173DA51E007B3C1B /* FTLValueFromBlock.h */; settings = {ATTRIBUTES = (Private, ); }; };
                0FDB2CE8174830A2007B3C1B /* DFGWorklist.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FDB2CE6174830A2007B3C1B /* DFGWorklist.h */; };
                0FDB2CEA174896C7007B3C1B /* ConcurrentJSLock.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FDB2CE9174896C7007B3C1B /* ConcurrentJSLock.h */; settings = {ATTRIBUTES = (Private, ); }; };
+               0FDCE11C1FAE6209006F3901 /* AllocationFailureMode.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FDCE11B1FAE61F4006F3901 /* AllocationFailureMode.h */; settings = {ATTRIBUTES = (Private, ); }; };
+               0FDCE1221FAE858C006F3901 /* HeapCellType.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FDCE11F1FAE8587006F3901 /* HeapCellType.h */; settings = {ATTRIBUTES = (Private, ); }; };
+               0FDCE12A1FAFA85F006F3901 /* CompleteSubspace.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FDCE1281FAFA859006F3901 /* CompleteSubspace.h */; settings = {ATTRIBUTES = (Private, ); }; };
+               0FDCE12D1FAFB4E5006F3901 /* IsoSubspace.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FDCE12B1FAFB4DE006F3901 /* IsoSubspace.h */; settings = {ATTRIBUTES = (Private, ); }; };
+               0FDCE1321FB11DA4006F3901 /* IsoAlignedMemoryAllocator.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FDCE1301FB11D9D006F3901 /* IsoAlignedMemoryAllocator.h */; settings = {ATTRIBUTES = (Private, ); }; };
                0FDDBFB61666EEDA00C55FEF /* DFGVariableAccessDataDump.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FDDBFB31666EED500C55FEF /* DFGVariableAccessDataDump.h */; };
                0FDE87FC1DFE6E510064C390 /* SpaceTimeMutatorScheduler.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FDE87FB1DFE6E500064C390 /* SpaceTimeMutatorScheduler.h */; };
                0FDF67D21D9C6D27001B9825 /* B3Kind.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FDF67D11D9C6086001B9825 /* B3Kind.h */; };
                79DAE27A1E03C82200B526AA /* WasmExceptionType.h in Headers */ = {isa = PBXBuildFile; fileRef = 79DAE2791E03C82200B526AA /* WasmExceptionType.h */; settings = {ATTRIBUTES = (Private, ); }; };
                79DFCBDB1D88C59600527D03 /* HasOwnPropertyCache.h in Headers */ = {isa = PBXBuildFile; fileRef = 79DFCBDA1D88C59600527D03 /* HasOwnPropertyCache.h */; settings = {ATTRIBUTES = (Private, ); }; };
                79EE0C001B4AFB85000385C9 /* VariableEnvironment.h in Headers */ = {isa = PBXBuildFile; fileRef = 79EE0BFE1B4AFB85000385C9 /* VariableEnvironment.h */; settings = {ATTRIBUTES = (Private, ); }; };
-               79EFD4841EBC045C00F3DFEA /* JSWebAssemblyCodeBlockSubspace.h in Headers */ = {isa = PBXBuildFile; fileRef = 79EFD4821EBC045C00F3DFEA /* JSWebAssemblyCodeBlockSubspace.h */; settings = {ATTRIBUTES = (Private, ); }; };
+               79EFD4841EBC045C00F3DFEA /* JSWebAssemblyCodeBlockHeapCellType.h in Headers */ = {isa = PBXBuildFile; fileRef = 79EFD4821EBC045C00F3DFEA /* JSWebAssemblyCodeBlockHeapCellType.h */; settings = {ATTRIBUTES = (Private, ); }; };
                79F8FC1F1B9FED0F00CA66AB /* DFGMaximalFlushInsertionPhase.h in Headers */ = {isa = PBXBuildFile; fileRef = 79F8FC1D1B9FED0F00CA66AB /* DFGMaximalFlushInsertionPhase.h */; };
                79FC8A081E32E9F000D88F0E /* DFGRegisteredStructure.h in Headers */ = {isa = PBXBuildFile; fileRef = 79FC8A071E32E9F000D88F0E /* DFGRegisteredStructure.h */; settings = {ATTRIBUTES = (Private, ); }; };
                7BC547D31B6959A100959B58 /* WasmFormat.h in Headers */ = {isa = PBXBuildFile; fileRef = 7BC547D21B69599B00959B58 /* WasmFormat.h */; settings = {ATTRIBUTES = (Private, ); }; };
                0F300B7718AB051100A6D72E /* DFGNodeOrigin.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGNodeOrigin.h; path = dfg/DFGNodeOrigin.h; sourceTree = "<group>"; };
                0F300B7918AB1B1400A6D72E /* DFGIntegerCheckCombiningPhase.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGIntegerCheckCombiningPhase.cpp; path = dfg/DFGIntegerCheckCombiningPhase.cpp; sourceTree = "<group>"; };
                0F300B7A18AB1B1400A6D72E /* DFGIntegerCheckCombiningPhase.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGIntegerCheckCombiningPhase.h; path = dfg/DFGIntegerCheckCombiningPhase.h; sourceTree = "<group>"; };
+               0F30CB5D1FCE46B4004B5323 /* AllocatorForMode.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = AllocatorForMode.h; sourceTree = "<group>"; };
                0F30D7BF1D95D62F0053089D /* CPU.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = CPU.h; sourceTree = "<group>"; };
                0F30FB601DC2DE96003124F2 /* ArrayBufferSharingMode.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ArrayBufferSharingMode.h; sourceTree = "<group>"; };
                0F32BD0E1BB34F190093A57F /* HeapHelperPool.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = HeapHelperPool.cpp; sourceTree = "<group>"; };
                0F4DE1D01C4D764B004D6C11 /* B3OriginDump.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = B3OriginDump.cpp; path = b3/B3OriginDump.cpp; sourceTree = "<group>"; };
                0F4F29DD18B6AD1C0057BC15 /* DFGStaticExecutionCountEstimationPhase.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGStaticExecutionCountEstimationPhase.cpp; path = dfg/DFGStaticExecutionCountEstimationPhase.cpp; sourceTree = "<group>"; };
                0F4F29DE18B6AD1C0057BC15 /* DFGStaticExecutionCountEstimationPhase.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGStaticExecutionCountEstimationPhase.h; path = dfg/DFGStaticExecutionCountEstimationPhase.h; sourceTree = "<group>"; };
-               0F4F82851E2FFDDB0075184C /* JSSegmentedVariableObjectSubspace.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = JSSegmentedVariableObjectSubspace.cpp; sourceTree = "<group>"; };
-               0F4F82861E2FFDDB0075184C /* JSSegmentedVariableObjectSubspace.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSSegmentedVariableObjectSubspace.h; sourceTree = "<group>"; };
+               0F4F82851E2FFDDB0075184C /* JSSegmentedVariableObjectHeapCellType.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = JSSegmentedVariableObjectHeapCellType.cpp; sourceTree = "<group>"; };
+               0F4F82861E2FFDDB0075184C /* JSSegmentedVariableObjectHeapCellType.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSSegmentedVariableObjectHeapCellType.h; sourceTree = "<group>"; };
                0F4F82891E31B9710075184C /* StochasticSpaceTimeMutatorScheduler.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = StochasticSpaceTimeMutatorScheduler.cpp; sourceTree = "<group>"; };
                0F4F828A1E31B9710075184C /* StochasticSpaceTimeMutatorScheduler.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = StochasticSpaceTimeMutatorScheduler.h; sourceTree = "<group>"; };
                0F50AF3B193E8B3900674EE8 /* DFGStructureClobberState.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGStructureClobberState.h; path = dfg/DFGStructureClobberState.h; sourceTree = "<group>"; };
                0F7DF1311E2970D50095951B /* Subspace.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = Subspace.cpp; sourceTree = "<group>"; };
                0F7DF1321E2970D50095951B /* Subspace.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = Subspace.h; sourceTree = "<group>"; };
                0F7DF1331E2970D50095951B /* SubspaceInlines.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = SubspaceInlines.h; sourceTree = "<group>"; };
-               0F7DF1391E29710E0095951B /* JSDestructibleObjectSubspace.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = JSDestructibleObjectSubspace.cpp; sourceTree = "<group>"; };
-               0F7DF13A1E29710E0095951B /* JSDestructibleObjectSubspace.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSDestructibleObjectSubspace.h; sourceTree = "<group>"; };
-               0F7DF13D1E2AFC4B0095951B /* JSStringSubspace.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = JSStringSubspace.cpp; sourceTree = "<group>"; };
-               0F7DF13E1E2AFC4B0095951B /* JSStringSubspace.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSStringSubspace.h; sourceTree = "<group>"; };
+               0F7DF1391E29710E0095951B /* JSDestructibleObjectHeapCellType.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = JSDestructibleObjectHeapCellType.cpp; sourceTree = "<group>"; };
+               0F7DF13A1E29710E0095951B /* JSDestructibleObjectHeapCellType.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSDestructibleObjectHeapCellType.h; sourceTree = "<group>"; };
+               0F7DF13D1E2AFC4B0095951B /* JSStringHeapCellType.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = JSStringHeapCellType.cpp; sourceTree = "<group>"; };
+               0F7DF13E1E2AFC4B0095951B /* JSStringHeapCellType.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSStringHeapCellType.h; sourceTree = "<group>"; };
                0F7DF1451E2BEF680095951B /* MarkedAllocatorInlines.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = MarkedAllocatorInlines.h; sourceTree = "<group>"; };
                0F7F98891D9596C300F4F12E /* DFGStoreBarrierClusteringPhase.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGStoreBarrierClusteringPhase.cpp; path = dfg/DFGStoreBarrierClusteringPhase.cpp; sourceTree = "<group>"; };
                0F7F988A1D9596C300F4F12E /* DFGStoreBarrierClusteringPhase.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGStoreBarrierClusteringPhase.h; path = dfg/DFGStoreBarrierClusteringPhase.h; sourceTree = "<group>"; };
                0FDB2CE5174830A2007B3C1B /* DFGWorklist.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGWorklist.cpp; path = dfg/DFGWorklist.cpp; sourceTree = "<group>"; };
                0FDB2CE6174830A2007B3C1B /* DFGWorklist.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGWorklist.h; path = dfg/DFGWorklist.h; sourceTree = "<group>"; };
                0FDB2CE9174896C7007B3C1B /* ConcurrentJSLock.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ConcurrentJSLock.h; sourceTree = "<group>"; };
+               0FDCE11B1FAE61F4006F3901 /* AllocationFailureMode.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = AllocationFailureMode.h; sourceTree = "<group>"; };
+               0FDCE11F1FAE8587006F3901 /* HeapCellType.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = HeapCellType.h; sourceTree = "<group>"; };
+               0FDCE1201FAE8587006F3901 /* HeapCellType.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = HeapCellType.cpp; sourceTree = "<group>"; };
+               0FDCE1271FAFA859006F3901 /* CompleteSubspace.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = CompleteSubspace.cpp; sourceTree = "<group>"; };
+               0FDCE1281FAFA859006F3901 /* CompleteSubspace.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = CompleteSubspace.h; sourceTree = "<group>"; };
+               0FDCE12B1FAFB4DE006F3901 /* IsoSubspace.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = IsoSubspace.h; sourceTree = "<group>"; };
+               0FDCE12C1FAFB4DE006F3901 /* IsoSubspace.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = IsoSubspace.cpp; sourceTree = "<group>"; };
+               0FDCE12F1FB11D9D006F3901 /* IsoAlignedMemoryAllocator.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = IsoAlignedMemoryAllocator.cpp; sourceTree = "<group>"; };
+               0FDCE1301FB11D9D006F3901 /* IsoAlignedMemoryAllocator.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = IsoAlignedMemoryAllocator.h; sourceTree = "<group>"; };
                0FDDBFB21666EED500C55FEF /* DFGVariableAccessDataDump.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGVariableAccessDataDump.cpp; path = dfg/DFGVariableAccessDataDump.cpp; sourceTree = "<group>"; };
                0FDDBFB31666EED500C55FEF /* DFGVariableAccessDataDump.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGVariableAccessDataDump.h; path = dfg/DFGVariableAccessDataDump.h; sourceTree = "<group>"; };
                0FDE87F81DFD0C6D0064C390 /* CellContainer.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = CellContainer.cpp; sourceTree = "<group>"; };
                79DFCBDA1D88C59600527D03 /* HasOwnPropertyCache.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = HasOwnPropertyCache.h; sourceTree = "<group>"; };
                79EE0BFD1B4AFB85000385C9 /* VariableEnvironment.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = VariableEnvironment.cpp; sourceTree = "<group>"; };
                79EE0BFE1B4AFB85000385C9 /* VariableEnvironment.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = VariableEnvironment.h; sourceTree = "<group>"; };
-               79EFD4811EBC045C00F3DFEA /* JSWebAssemblyCodeBlockSubspace.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = JSWebAssemblyCodeBlockSubspace.cpp; path = js/JSWebAssemblyCodeBlockSubspace.cpp; sourceTree = "<group>"; };
-               79EFD4821EBC045C00F3DFEA /* JSWebAssemblyCodeBlockSubspace.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = JSWebAssemblyCodeBlockSubspace.h; path = js/JSWebAssemblyCodeBlockSubspace.h; sourceTree = "<group>"; };
+               79EFD4811EBC045C00F3DFEA /* JSWebAssemblyCodeBlockHeapCellType.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = JSWebAssemblyCodeBlockHeapCellType.cpp; path = js/JSWebAssemblyCodeBlockHeapCellType.cpp; sourceTree = "<group>"; };
+               79EFD4821EBC045C00F3DFEA /* JSWebAssemblyCodeBlockHeapCellType.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = JSWebAssemblyCodeBlockHeapCellType.h; path = js/JSWebAssemblyCodeBlockHeapCellType.h; sourceTree = "<group>"; };
                79F8FC1C1B9FED0F00CA66AB /* DFGMaximalFlushInsertionPhase.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGMaximalFlushInsertionPhase.cpp; path = dfg/DFGMaximalFlushInsertionPhase.cpp; sourceTree = "<group>"; };
                79F8FC1D1B9FED0F00CA66AB /* DFGMaximalFlushInsertionPhase.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGMaximalFlushInsertionPhase.h; path = dfg/DFGMaximalFlushInsertionPhase.h; sourceTree = "<group>"; };
                79FC8A071E32E9F000D88F0E /* DFGRegisteredStructure.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGRegisteredStructure.h; path = dfg/DFGRegisteredStructure.h; sourceTree = "<group>"; };
                                0FEC3C501F33A41600F59B6C /* AlignedMemoryAllocator.cpp */,
                                0FEC3C511F33A41600F59B6C /* AlignedMemoryAllocator.h */,
                                0FA7620A1DB959F600B7A2FD /* AllocatingScope.h */,
+                               0FDCE11B1FAE61F4006F3901 /* AllocationFailureMode.h */,
                                0F9630351D4192C3005609D9 /* AllocatorAttributes.cpp */,
                                0F9630361D4192C3005609D9 /* AllocatorAttributes.h */,
+                               0F30CB5D1FCE46B4004B5323 /* AllocatorForMode.h */,
                                0FDE87F81DFD0C6D0064C390 /* CellContainer.cpp */,
                                0F070A421D543A89006E7232 /* CellContainer.h */,
                                0F070A431D543A89006E7232 /* CellContainerInlines.h */,
                                0FA762011DB9242300B7A2FD /* CollectionScope.h */,
                                0FD0E5E51E43D3470006AB08 /* CollectorPhase.cpp */,
                                0FD0E5E61E43D3470006AB08 /* CollectorPhase.h */,
+                               0FDCE1271FAFA859006F3901 /* CompleteSubspace.cpp */,
+                               0FDCE1281FAFA859006F3901 /* CompleteSubspace.h */,
                                146B14DB12EB5B12001BEC1B /* ConservativeRoots.cpp */,
                                149DAAF212EB559D0083B12B /* ConservativeRoots.h */,
                                0F7DF12F1E2970D50095951B /* ConstraintVolatility.h */,
                                DC3D2B0B1D34376E00BA918C /* HeapCell.cpp */,
                                DC3D2B091D34316100BA918C /* HeapCell.h */,
                                0F070A441D543A89006E7232 /* HeapCellInlines.h */,
+                               0FDCE1201FAE8587006F3901 /* HeapCellType.cpp */,
+                               0FDCE11F1FAE8587006F3901 /* HeapCellType.h */,
                                0F0CAEFD1EC4DA8500970D12 /* HeapFinalizerCallback.cpp */,
                                0F0CAEFE1EC4DA8500970D12 /* HeapFinalizerCallback.h */,
                                0F32BD0E1BB34F190093A57F /* HeapHelperPool.cpp */,
                                0FADE6721D4D23BC00768457 /* HeapUtil.h */,
                                C25F8BCB157544A900245B71 /* IncrementalSweeper.cpp */,
                                C25F8BCC157544A900245B71 /* IncrementalSweeper.h */,
+                               0FDCE12F1FB11D9D006F3901 /* IsoAlignedMemoryAllocator.cpp */,
+                               0FDCE1301FB11D9D006F3901 /* IsoAlignedMemoryAllocator.h */,
+                               0FDCE12C1FAFB4DE006F3901 /* IsoSubspace.cpp */,
+                               0FDCE12B1FAFB4DE006F3901 /* IsoSubspace.h */,
                                0F766D2915A8CC34008F363E /* JITStubRoutineSet.cpp */,
                                0F766D2A15A8CC34008F363E /* JITStubRoutineSet.h */,
                                0F070A451D543A89006E7232 /* LargeAllocation.cpp */,
                                9788FC221471AD0C0068CE2D /* JSDateMath.cpp */,
                                9788FC231471AD0C0068CE2D /* JSDateMath.h */,
                                C2A7F687160432D400F76B98 /* JSDestructibleObject.h */,
-                               0F7DF1391E29710E0095951B /* JSDestructibleObjectSubspace.cpp */,
-                               0F7DF13A1E29710E0095951B /* JSDestructibleObjectSubspace.h */,
+                               0F7DF1391E29710E0095951B /* JSDestructibleObjectHeapCellType.cpp */,
+                               0F7DF13A1E29710E0095951B /* JSDestructibleObjectHeapCellType.h */,
                                A7B4ACAE1484C9CE00B38A36 /* JSExportMacros.h */,
                                798937761DCAB57300F8D4FB /* JSFixedArray.cpp */,
                                798937771DCAB57300F8D4FB /* JSFixedArray.h */,
                                E38D060B1F8E814100649CF2 /* JSScriptFetchParameters.h */,
                                0F919D0E157F3327004A4E7D /* JSSegmentedVariableObject.cpp */,
                                0F919D0F157F3327004A4E7D /* JSSegmentedVariableObject.h */,
-                               0F4F82851E2FFDDB0075184C /* JSSegmentedVariableObjectSubspace.cpp */,
-                               0F4F82861E2FFDDB0075184C /* JSSegmentedVariableObjectSubspace.h */,
+                               0F4F82851E2FFDDB0075184C /* JSSegmentedVariableObjectHeapCellType.cpp */,
+                               0F4F82861E2FFDDB0075184C /* JSSegmentedVariableObjectHeapCellType.h */,
                                A7299D9B17D12837005F5FF9 /* JSSet.cpp */,
                                A7299D9C17D12837005F5FF9 /* JSSet.h */,
                                A790DD69182F499700588807 /* JSSetIterator.cpp */,
                                70EC0EBD1AA0D7DA00B6AAFA /* JSStringIterator.h */,
                                2600B5A4152BAAA70091EE5F /* JSStringJoiner.cpp */,
                                2600B5A5152BAAA70091EE5F /* JSStringJoiner.h */,
-                               0F7DF13D1E2AFC4B0095951B /* JSStringSubspace.cpp */,
-                               0F7DF13E1E2AFC4B0095951B /* JSStringSubspace.h */,
+                               0F7DF13D1E2AFC4B0095951B /* JSStringHeapCellType.cpp */,
+                               0F7DF13E1E2AFC4B0095951B /* JSStringHeapCellType.h */,
                                0F919D09157EE09D004A4E7D /* JSSymbolTableObject.cpp */,
                                0F919D0A157EE09D004A4E7D /* JSSymbolTableObject.h */,
                                70ECA6001AFDBEA200449739 /* JSTemplateRegistryKey.cpp */,
                                ADD09AF31F62482E001313C2 /* JSWebAssembly.h */,
                                5383AA2F1E65E8A100A532FC /* JSWebAssemblyCodeBlock.cpp */,
                                AD9E852E1E8A0C6E008DE39E /* JSWebAssemblyCodeBlock.h */,
-                               79EFD4811EBC045C00F3DFEA /* JSWebAssemblyCodeBlockSubspace.cpp */,
-                               79EFD4821EBC045C00F3DFEA /* JSWebAssemblyCodeBlockSubspace.h */,
+                               79EFD4811EBC045C00F3DFEA /* JSWebAssemblyCodeBlockHeapCellType.cpp */,
+                               79EFD4821EBC045C00F3DFEA /* JSWebAssemblyCodeBlockHeapCellType.h */,
                                AD2FCBA61DB58DA400B3E736 /* JSWebAssemblyCompileError.cpp */,
                                AD2FCBA71DB58DA400B3E736 /* JSWebAssemblyCompileError.h */,
                                796FB4391DFF8C3F0039C95D /* JSWebAssemblyHelpers.h */,
                                86D3B2C410156BDE002865E7 /* ARMAssembler.h in Headers */,
                                86ADD1450FDDEA980006EEC2 /* ARMv7Assembler.h in Headers */,
                                65C0285D1717966800351E35 /* ARMv7DOpcode.h in Headers */,
+                               0FDCE1321FB11DA4006F3901 /* IsoAlignedMemoryAllocator.h in Headers */,
                                0F8335B81639C1EA001443B5 /* ArrayAllocationProfile.h in Headers */,
                                A7A8AF3517ADB5F3005AB174 /* ArrayBuffer.h in Headers */,
                                0FFC99D5184EE318009C10AB /* ArrayBufferNeuteringWatchpoint.h in Headers */,
                                0FEC85061BDACDAC0080FF74 /* B3CheckSpecial.h in Headers */,
                                0FEC85081BDACDAC0080FF74 /* B3CheckValue.h in Headers */,
                                0FEC850A1BDACDAC0080FF74 /* B3Common.h in Headers */,
+                               0FDCE12D1FAFB4E5006F3901 /* IsoSubspace.h in Headers */,
                                0FEC850C1BDACDAC0080FF74 /* B3Commutativity.h in Headers */,
                                0F338E0C1BF0276C0013C88F /* B3Compilation.h in Headers */,
                                7919B7801E03559C005BEED8 /* B3Compile.h in Headers */,
                                A54982041891D0B00081E5B8 /* EventLoop.h in Headers */,
                                FE1C0FFD1B193E9800B53FCA /* Exception.h in Headers */,
                                FE6029D91D6E1E4F0030204D /* ExceptionEventLocation.h in Headers */,
+                               0F30CB5E1FCE4E37004B5323 /* AllocatorForMode.h in Headers */,
                                0F12DE101979D5FD0006FF4E /* ExceptionFuzz.h in Headers */,
                                BC18C4000E16F5CD00B34460 /* ExceptionHelpers.h in Headers */,
                                FE6491371D78F01D00A694D4 /* ExceptionScope.h in Headers */,
                                C4703CD5192844CC0013FBEA /* generator_templates.py in Headers */,
                                70B791951C024A28002481E2 /* GeneratorFunctionConstructor.h in Headers */,
                                70B791971C024A29002481E2 /* GeneratorFunctionPrototype.h in Headers */,
+                               0FDCE12A1FAFA85F006F3901 /* CompleteSubspace.h in Headers */,
                                70B791991C024A29002481E2 /* GeneratorPrototype.h in Headers */,
                                70B7919D1C024A56002481E2 /* GeneratorPrototype.lut.h in Headers */,
                                0FE050191AA9091100D33B33 /* GenericArguments.h in Headers */,
                                996B731C1BDA08DD00331B84 /* JSDataViewPrototype.lut.h in Headers */,
                                978801411471AD920041B016 /* JSDateMath.h in Headers */,
                                C2A7F688160432D400F76B98 /* JSDestructibleObject.h in Headers */,
-                               0F7DF13C1E2971130095951B /* JSDestructibleObjectSubspace.h in Headers */,
+                               0F7DF13C1E2971130095951B /* JSDestructibleObjectHeapCellType.h in Headers */,
                                FE384EE61ADDB7AD0055DE2C /* JSDollarVM.h in Headers */,
                                86E3C614167BABD7006D760A /* JSExport.h in Headers */,
                                A7B4ACAF1484C9CE00B38A36 /* JSExportMacros.h in Headers */,
                                E3D239C91B829C1C00BBEF67 /* JSModuleEnvironment.h in Headers */,
                                D9722752DC54459B9125B539 /* JSModuleLoader.h in Headers */,
                                E318CBC11B8AEF5100A2929D /* JSModuleNamespaceObject.h in Headers */,
+                               0FDCE11C1FAE6209006F3901 /* AllocationFailureMode.h in Headers */,
                                E39DA4A71B7E8B7C0084F33A /* JSModuleRecord.h in Headers */,
                                E33E8D1D1B9013C300346B52 /* JSNativeStdFunction.h in Headers */,
                                BC18C4240E16F5CD00B34460 /* JSObject.h in Headers */,
                                E3201C1D1F8E82360076A032 /* JSScriptFetchParameters.h in Headers */,
                                A7C0C4AC168103020017011D /* JSScriptRefPrivate.h in Headers */,
                                0F919D11157F332C004A4E7D /* JSSegmentedVariableObject.h in Headers */,
-                               0F4F82881E2FFDE00075184C /* JSSegmentedVariableObjectSubspace.h in Headers */,
+                               0F4F82881E2FFDE00075184C /* JSSegmentedVariableObjectHeapCellType.h in Headers */,
                                A7299D9E17D12837005F5FF9 /* JSSet.h in Headers */,
                                A790DD70182F499700588807 /* JSSetIterator.h in Headers */,
                                BDFCB2BBE90F41349E1B0BED /* JSSourceCode.h in Headers */,
                                BC18C4280E16F5CD00B34460 /* JSStringRef.h in Headers */,
                                BC18C4290E16F5CD00B34460 /* JSStringRefCF.h in Headers */,
                                1A28D4A8177B71C80007FA3C /* JSStringRefPrivate.h in Headers */,
-                               0F7DF13F1E2AFC4D0095951B /* JSStringSubspace.h in Headers */,
+                               0F7DF13F1E2AFC4D0095951B /* JSStringHeapCellType.h in Headers */,
                                0F919D0D157EE0A2004A4E7D /* JSSymbolTableObject.h in Headers */,
                                70ECA6061AFDBEA200449739 /* JSTemplateRegistryKey.h in Headers */,
                                AD5C36EA1F75AD6A000BCAAF /* JSToWasm.h in Headers */,
                                709FB8681AE335C60039D069 /* JSWeakSet.h in Headers */,
                                AD5C36EB1F75AD73000BCAAF /* JSWebAssembly.h in Headers */,
                                AD9E852F1E8A0C7C008DE39E /* JSWebAssemblyCodeBlock.h in Headers */,
-                               79EFD4841EBC045C00F3DFEA /* JSWebAssemblyCodeBlockSubspace.h in Headers */,
+                               79EFD4841EBC045C00F3DFEA /* JSWebAssemblyCodeBlockHeapCellType.h in Headers */,
                                AD2FCBE31DB58DAD00B3E736 /* JSWebAssemblyCompileError.h in Headers */,
                                796FB43A1DFF8C3F0039C95D /* JSWebAssemblyHelpers.h in Headers */,
                                AD2FCBE51DB58DAD00B3E736 /* JSWebAssemblyInstance.h in Headers */,
                                0FFB6C391AF48DDC00DB1BF7 /* TypeofType.h in Headers */,
                                52C952B719A289850069B386 /* TypeProfiler.h in Headers */,
                                0F2D4DEC19832DC4007D4B19 /* TypeProfilerLog.h in Headers */,
+                               0FDCE1221FAE858C006F3901 /* HeapCellType.h in Headers */,
                                0F2D4DF019832DD6007D4B19 /* TypeSet.h in Headers */,
                                0FF4274B158EBE91004CB9FF /* udis86.h in Headers */,
                                0FF42741158EBE8D004CB9FF /* udis86_decode.h in Headers */,
index 0a9e71b..a45487e 100644 (file)
@@ -470,6 +470,7 @@ heap/CellContainer.cpp
 heap/CodeBlockSet.cpp
 heap/CollectionScope.cpp
 heap/CollectorPhase.cpp
+heap/CompleteSubspace.cpp
 heap/ConservativeRoots.cpp
 heap/DeferGC.cpp
 heap/DestructionMode.cpp
@@ -486,12 +487,15 @@ heap/HandleSet.cpp
 heap/HandleStack.cpp
 heap/Heap.cpp
 heap/HeapCell.cpp
+heap/HeapCellType.cpp
 heap/HeapFinalizerCallback.cpp
 heap/HeapHelperPool.cpp
 heap/HeapProfiler.cpp
 heap/HeapSnapshot.cpp
 heap/HeapSnapshotBuilder.cpp
 heap/IncrementalSweeper.cpp
+heap/IsoAlignedMemoryAllocator.cpp
+heap/IsoSubspace.cpp
 heap/JITStubRoutineSet.cpp
 heap/LargeAllocation.cpp
 heap/MachineStackMarker.cpp
@@ -771,7 +775,7 @@ runtime/JSCustomGetterSetterFunction.cpp
 runtime/JSDataView.cpp
 runtime/JSDataViewPrototype.cpp
 runtime/JSDateMath.cpp
-runtime/JSDestructibleObjectSubspace.cpp
+runtime/JSDestructibleObjectHeapCellType.cpp
 runtime/JSFixedArray.cpp
 runtime/JSFunction.cpp
 runtime/JSGeneratorFunction.cpp
@@ -806,14 +810,14 @@ runtime/JSScope.cpp
 runtime/JSScriptFetcher.cpp
 runtime/JSScriptFetchParameters.cpp
 runtime/JSSegmentedVariableObject.cpp
-runtime/JSSegmentedVariableObjectSubspace.cpp
+runtime/JSSegmentedVariableObjectHeapCellType.cpp
 runtime/JSSet.cpp
 runtime/JSSetIterator.cpp
 runtime/JSSourceCode.cpp
 runtime/JSString.cpp
 runtime/JSStringIterator.cpp
 runtime/JSStringJoiner.cpp
-runtime/JSStringSubspace.cpp
+runtime/JSStringHeapCellType.cpp
 runtime/JSSymbolTableObject.cpp
 runtime/JSTemplateRegistryKey.cpp
 runtime/JSTypedArrayConstructors.cpp
@@ -970,7 +974,7 @@ wasm/js/JSToWasm.cpp
 wasm/js/JSToWasm.h
 wasm/js/JSWebAssembly.cpp
 wasm/js/JSWebAssemblyCodeBlock.cpp
-wasm/js/JSWebAssemblyCodeBlockSubspace.cpp
+wasm/js/JSWebAssemblyCodeBlockHeapCellType.cpp
 wasm/js/JSWebAssemblyCompileError.cpp
 wasm/js/JSWebAssemblyInstance.cpp
 wasm/js/JSWebAssemblyLinkError.cpp
index 984a0b3..d405696 100644 (file)
@@ -955,7 +955,7 @@ void AccessCase::generateImpl(AccessGenerationState& state)
             size_t newSize = newStructure()->outOfLineCapacity() * sizeof(JSValue);
 
             if (allocatingInline) {
-                MarkedAllocator* allocator = vm.jsValueGigacageAuxiliarySpace.allocatorFor(newSize);
+                MarkedAllocator* allocator = vm.jsValueGigacageAuxiliarySpace.allocatorFor(newSize, AllocatorForMode::AllocatorIfExists);
 
                 if (!allocator) {
                     // Yuck, this case would suck!
index 70125f9..ae5e3ae 100644 (file)
@@ -99,7 +99,7 @@ ALWAYS_INLINE void ObjectAllocationProfile::initializeProfile(VM& vm, JSGlobalOb
     ASSERT(inlineCapacity <= JSFinalObject::maxInlineCapacity());
 
     size_t allocationSize = JSFinalObject::allocationSize(inlineCapacity);
-    MarkedAllocator* allocator = vm.cellSpace.allocatorFor(allocationSize);
+    MarkedAllocator* allocator = vm.cellSpace.allocatorForNonVirtual(allocationSize, AllocatorForMode::EnsureAllocator);
 
     // Take advantage of extra inline capacity available in the size class.
     if (allocator) {
index ec968e4..4880be1 100644 (file)
@@ -110,7 +110,7 @@ void SpeculativeJIT::emitAllocateRawObject(GPRReg resultGPR, RegisteredStructure
     m_jit.move(TrustedImmPtr(0), storageGPR);
     
     if (size) {
-        if (MarkedAllocator* allocator = m_jit.vm()->jsValueGigacageAuxiliarySpace.allocatorFor(size)) {
+        if (MarkedAllocator* allocator = m_jit.vm()->jsValueGigacageAuxiliarySpace.allocatorForNonVirtual(size, AllocatorForMode::AllocatorIfExists)) {
             m_jit.move(TrustedImmPtr(allocator), scratchGPR);
             m_jit.emitAllocate(storageGPR, allocator, scratchGPR, scratch2GPR, slowCases);
             
@@ -125,7 +125,7 @@ void SpeculativeJIT::emitAllocateRawObject(GPRReg resultGPR, RegisteredStructure
     }
 
     size_t allocationSize = JSFinalObject::allocationSize(inlineCapacity);
-    MarkedAllocator* allocatorPtr = subspaceFor<JSFinalObject>(*m_jit.vm())->allocatorFor(allocationSize);
+    MarkedAllocator* allocatorPtr = subspaceFor<JSFinalObject>(*m_jit.vm())->allocatorForNonVirtual(allocationSize, AllocatorForMode::AllocatorIfExists);
     if (allocatorPtr) {
         m_jit.move(TrustedImmPtr(allocatorPtr), scratchGPR);
         emitAllocateJSObject(resultGPR, allocatorPtr, scratchGPR, TrustedImmPtr(structure), storageGPR, scratch2GPR, slowCases);
@@ -4249,8 +4249,7 @@ void SpeculativeJIT::compileMakeRope(Node* node)
     GPRReg scratchGPR = scratch.gpr();
     
     JITCompiler::JumpList slowPath;
-    MarkedAllocator* markedAllocator = subspaceFor<JSString>(*m_jit.vm())->allocatorFor(sizeof(JSRopeString));
-    RELEASE_ASSERT(markedAllocator);
+    MarkedAllocator* markedAllocator = subspaceFor<JSString>(*m_jit.vm())->allocatorForNonVirtual(sizeof(JSRopeString), AllocatorForMode::MustAlreadyHaveAllocator);
     m_jit.move(TrustedImmPtr(markedAllocator), allocatorGPR);
     emitAllocateJSCell(resultGPR, markedAllocator, allocatorGPR, TrustedImmPtr(m_jit.graph().registerStructure(m_jit.vm()->stringStructure.get())), scratchGPR, slowPath);
         
@@ -8428,7 +8427,7 @@ void SpeculativeJIT::compileAllocatePropertyStorage(Node* node)
     
     size_t size = initialOutOfLineCapacity * sizeof(JSValue);
 
-    MarkedAllocator* allocator = m_jit.vm()->jsValueGigacageAuxiliarySpace.allocatorFor(size);
+    MarkedAllocator* allocator = m_jit.vm()->jsValueGigacageAuxiliarySpace.allocatorForNonVirtual(size, AllocatorForMode::AllocatorIfExists);
 
     if (!allocator || node->transition()->previous->couldHaveIndexingHeader()) {
         SpeculateCellOperand base(this, node->child1());
@@ -8473,7 +8472,7 @@ void SpeculativeJIT::compileReallocatePropertyStorage(Node* node)
     size_t newSize = oldSize * outOfLineGrowthFactor;
     ASSERT(newSize == node->transition()->next->outOfLineCapacity() * sizeof(JSValue));
     
-    MarkedAllocator* allocator = m_jit.vm()->jsValueGigacageAuxiliarySpace.allocatorFor(newSize);
+    MarkedAllocator* allocator = m_jit.vm()->jsValueGigacageAuxiliarySpace.allocatorForNonVirtual(newSize, AllocatorForMode::AllocatorIfExists);
 
     if (!allocator || node->transition()->previous->couldHaveIndexingHeader()) {
         SpeculateCellOperand base(this, node->child1());
index bc973f0..bfda638 100644 (file)
@@ -4115,7 +4115,7 @@ void SpeculativeJIT::compile(Node* node)
         
         RegisteredStructure structure = node->structure();
         size_t allocationSize = JSFinalObject::allocationSize(structure->inlineCapacity());
-        MarkedAllocator* allocatorPtr = subspaceFor<JSFinalObject>(*m_jit.vm())->allocatorFor(allocationSize);
+        MarkedAllocator* allocatorPtr = subspaceFor<JSFinalObject>(*m_jit.vm())->allocatorForNonVirtual(allocationSize, AllocatorForMode::AllocatorIfExists);
 
         m_jit.move(TrustedImmPtr(allocatorPtr), allocatorGPR);
         emitAllocateJSObject(resultGPR, allocatorPtr, allocatorGPR, TrustedImmPtr(structure), TrustedImmPtr(0), scratchGPR, slowPath);
index 891d03b..0fde482 100644 (file)
@@ -4321,7 +4321,7 @@ void SpeculativeJIT::compile(Node* node)
 
         RegisteredStructure structure = node->structure();
         size_t allocationSize = JSFinalObject::allocationSize(structure->inlineCapacity());
-        MarkedAllocator* allocatorPtr = subspaceFor<JSFinalObject>(*m_jit.vm())->allocatorFor(allocationSize);
+        MarkedAllocator* allocatorPtr = subspaceFor<JSFinalObject>(*m_jit.vm())->allocatorForNonVirtual(allocationSize, AllocatorForMode::AllocatorIfExists);
 
         m_jit.move(TrustedImmPtr(allocatorPtr), allocatorGPR);
         emitAllocateJSObject(resultGPR, allocatorPtr, allocatorGPR, TrustedImmPtr(structure), TrustedImmPtr(0), scratchGPR, slowPath);
index 676bd3e..1654b59 100644 (file)
@@ -123,12 +123,12 @@ namespace JSC { namespace FTL {
 
 #define FOR_EACH_INDEXED_ABSTRACT_HEAP(macro) \
     macro(ArrayStorage_vector, ArrayStorage::vectorOffset(), sizeof(WriteBarrier<Unknown>)) \
+    macro(CompleteSubspace_allocatorForSizeStep, CompleteSubspace::offsetOfAllocatorForSizeStep(), sizeof(MarkedAllocator*)) \
     macro(DirectArguments_storage, DirectArguments::storageOffset(), sizeof(EncodedJSValue)) \
     macro(JSLexicalEnvironment_variables, JSLexicalEnvironment::offsetOfVariables(), sizeof(EncodedJSValue)) \
     macro(JSPropertyNameEnumerator_cachedPropertyNamesVectorContents, 0, sizeof(WriteBarrier<JSString>)) \
     macro(JSRopeString_fibers, JSRopeString::offsetOfFibers(), sizeof(WriteBarrier<JSString>)) \
     macro(ScopedArguments_overflowStorage, ScopedArguments::overflowStorageOffset(), sizeof(EncodedJSValue)) \
-    macro(Subspace_allocatorForSizeStep, Subspace::offsetOfAllocatorForSizeStep(), sizeof(MarkedAllocator*)) \
     macro(WriteBarrierBuffer_bufferContents, 0, sizeof(JSCell*)) \
     macro(characters8, 0, sizeof(LChar)) \
     macro(characters16, 0, sizeof(UChar)) \
index 967bda0..50e72d7 100644 (file)
@@ -5783,8 +5783,7 @@ private:
         
         LBasicBlock lastNext = m_out.insertNewBlocksBefore(slowPath);
         
-        MarkedAllocator* allocator = subspaceFor<JSRopeString>(vm())->allocatorFor(sizeof(JSRopeString));
-        DFG_ASSERT(m_graph, m_node, allocator);
+        MarkedAllocator* allocator = subspaceFor<JSRopeString>(vm())->allocatorForNonVirtual(sizeof(JSRopeString), AllocatorForMode::MustAlreadyHaveAllocator);
         
         LValue result = allocateCell(
             m_out.constIntPtr(allocator), vm().stringStructure.get(), slowPath);
@@ -9690,8 +9689,7 @@ private:
             
             if (structure->outOfLineCapacity() || hasIndexedProperties(structure->indexingType())) {
                 size_t allocationSize = JSFinalObject::allocationSize(structure->inlineCapacity());
-                MarkedAllocator* cellAllocator = subspaceFor<JSFinalObject>(vm())->allocatorFor(allocationSize);
-                DFG_ASSERT(m_graph, m_node, cellAllocator);
+                MarkedAllocator* cellAllocator = subspaceFor<JSFinalObject>(vm())->allocatorForNonVirtual(allocationSize, AllocatorForMode::MustAlreadyHaveAllocator);
 
                 bool hasIndexingHeader = hasIndexedProperties(structure->indexingType());
                 unsigned indexingHeaderSize = 0;
@@ -10686,7 +10684,7 @@ private:
         LBasicBlock lastNext = m_out.insertNewBlocksBefore(slowPath);
 
         size_t sizeInBytes = sizeInValues * sizeof(JSValue);
-        MarkedAllocator* allocator = vm().jsValueGigacageAuxiliarySpace.allocatorFor(sizeInBytes);
+        MarkedAllocator* allocator = vm().jsValueGigacageAuxiliarySpace.allocatorForNonVirtual(sizeInBytes, AllocatorForMode::AllocatorIfExists);
         LValue startOfStorage = allocateHeapCell(m_out.constIntPtr(allocator), slowPath);
         ValueFromBlock fastButterfly = m_out.anchor(
             m_out.add(m_out.constIntPtr(sizeInBytes + sizeof(IndexingHeader)), startOfStorage));
@@ -11830,7 +11828,7 @@ private:
     LValue allocateObject(
         size_t size, StructureType structure, LValue butterfly, LBasicBlock slowPath)
     {
-        MarkedAllocator* allocator = subspaceFor<ClassType>(vm())->allocatorFor(size);
+        MarkedAllocator* allocator = subspaceFor<ClassType>(vm())->allocatorForNonVirtual(size, AllocatorForMode::AllocatorIfExists);
         return allocateObject(m_out.constIntPtr(allocator), structure, butterfly, slowPath);
     }
     
@@ -11847,10 +11845,10 @@ private:
         
         // Try to do some constant-folding here.
         if (subspace->hasIntPtr() && size->hasIntPtr()) {
-            Subspace* actualSubspace = bitwise_cast<Subspace*>(subspace->asIntPtr());
+            CompleteSubspace* actualSubspace = bitwise_cast<CompleteSubspace*>(subspace->asIntPtr());
             size_t actualSize = size->asIntPtr();
             
-            MarkedAllocator* actualAllocator = actualSubspace->allocatorFor(actualSize);
+            MarkedAllocator* actualAllocator = actualSubspace->allocatorForNonVirtual(actualSize, AllocatorForMode::AllocatorIfExists);
             if (!actualAllocator) {
                 LBasicBlock continuation = m_out.newBlock();
                 LBasicBlock lastNext = m_out.insertNewBlocksBefore(continuation);
@@ -11880,11 +11878,11 @@ private:
         
         return m_out.loadPtr(
             m_out.baseIndex(
-                m_heaps.Subspace_allocatorForSizeStep,
+                m_heaps.CompleteSubspace_allocatorForSizeStep,
                 subspace, m_out.sub(sizeClassIndex, m_out.intPtrOne)));
     }
     
-    LValue allocatorForSize(Subspace& subspace, LValue size, LBasicBlock slowPath)
+    LValue allocatorForSize(CompleteSubspace& subspace, LValue size, LBasicBlock slowPath)
     {
         return allocatorForSize(m_out.constIntPtr(&subspace), size, slowPath);
     }
@@ -11910,7 +11908,7 @@ private:
     LValue allocateObject(RegisteredStructure structure)
     {
         size_t allocationSize = JSFinalObject::allocationSize(structure.get()->inlineCapacity());
-        MarkedAllocator* allocator = subspaceFor<JSFinalObject>(vm())->allocatorFor(allocationSize);
+        MarkedAllocator* allocator = subspaceFor<JSFinalObject>(vm())->allocatorForNonVirtual(allocationSize, AllocatorForMode::AllocatorIfExists);
         
         // FIXME: If the allocator is null, we could simply emit a normal C call to the allocator
         // instead of putting it on the slow path.
index baf7571..97fb863 100644 (file)
@@ -26,6 +26,9 @@
 #include "config.h"
 #include "AlignedMemoryAllocator.h"
 
+#include "MarkedAllocator.h"
+#include "Subspace.h"
+
 namespace JSC { 
 
 AlignedMemoryAllocator::AlignedMemoryAllocator()
@@ -36,6 +39,24 @@ AlignedMemoryAllocator::~AlignedMemoryAllocator()
 {
 }
 
+void AlignedMemoryAllocator::registerAllocator(MarkedAllocator* allocator)
+{
+    RELEASE_ASSERT(!allocator->nextAllocatorInAlignedMemoryAllocator());
+    
+    if (m_allocators.isEmpty()) {
+        for (Subspace* subspace = m_subspaces.first(); subspace; subspace = subspace->nextSubspaceInAlignedMemoryAllocator())
+            subspace->didCreateFirstAllocator(allocator);
+    }
+    
+    m_allocators.append(std::mem_fn(&MarkedAllocator::setNextAllocatorInAlignedMemoryAllocator), allocator);
+}
+
+void AlignedMemoryAllocator::registerSubspace(Subspace* subspace)
+{
+    RELEASE_ASSERT(!subspace->nextSubspaceInAlignedMemoryAllocator());
+    m_subspaces.append(std::mem_fn(&Subspace::setNextSubspaceInAlignedMemoryAllocator), subspace);
+}
+
 } // namespace JSC
 
 
index c3129f3..a688144 100644 (file)
 #pragma once
 
 #include <wtf/PrintStream.h>
+#include <wtf/SinglyLinkedListWithTail.h>
 
 namespace JSC {
 
+class MarkedAllocator;
+class Subspace;
+
 class AlignedMemoryAllocator {
     WTF_MAKE_NONCOPYABLE(AlignedMemoryAllocator);
     WTF_MAKE_FAST_ALLOCATED;
@@ -40,6 +44,15 @@ public:
     virtual void freeAlignedMemory(void*) = 0;
     
     virtual void dump(PrintStream&) const = 0;
+
+    void registerAllocator(MarkedAllocator*);
+    MarkedAllocator* firstAllocator() const { return m_allocators.first(); }
+
+    void registerSubspace(Subspace*);
+
+private:
+    SinglyLinkedListWithTail<MarkedAllocator> m_allocators;
+    SinglyLinkedListWithTail<Subspace> m_subspaces;
 };
 
 } // namespace WTF
diff --git a/Source/JavaScriptCore/heap/AllocationFailureMode.h b/Source/JavaScriptCore/heap/AllocationFailureMode.h
new file mode 100644 (file)
index 0000000..c322b05
--- /dev/null
@@ -0,0 +1,36 @@
+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#pragma once
+
+namespace JSC {
+
+enum class AllocationFailureMode {
+    Assert,
+    ReturnNull
+};
+
+} // namespace JSC
+
diff --git a/Source/JavaScriptCore/heap/AllocatorForMode.h b/Source/JavaScriptCore/heap/AllocatorForMode.h
new file mode 100644 (file)
index 0000000..a5a03c7
--- /dev/null
@@ -0,0 +1,38 @@
+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#pragma once
+
+namespace JSC {
+
+// This is the mode of the allocatorFor function.
+enum class AllocatorForMode {
+    MustAlreadyHaveAllocator,
+    EnsureAllocator,
+    AllocatorIfExists
+};
+
+} // namespace JSC
+
diff --git a/Source/JavaScriptCore/heap/CompleteSubspace.cpp b/Source/JavaScriptCore/heap/CompleteSubspace.cpp
new file mode 100644 (file)
index 0000000..685ec1d
--- /dev/null
@@ -0,0 +1,152 @@
+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#include "config.h"
+#include "Subspace.h"
+
+#include "JSCInlines.h"
+#include "MarkedAllocatorInlines.h"
+#include "MarkedBlockInlines.h"
+#include "PreventCollectionScope.h"
+#include "SubspaceInlines.h"
+
+namespace JSC {
+
+CompleteSubspace::CompleteSubspace(CString name, Heap& heap, HeapCellType* heapCellType, AlignedMemoryAllocator* alignedMemoryAllocator)
+    : Subspace(name, heap)
+{
+    initialize(heapCellType, alignedMemoryAllocator);
+    for (size_t i = MarkedSpace::numSizeClasses; i--;)
+        m_allocatorForSizeStep[i] = nullptr;
+}
+
+CompleteSubspace::~CompleteSubspace()
+{
+}
+
+MarkedAllocator* CompleteSubspace::allocatorFor(size_t size, AllocatorForMode mode)
+{
+    return allocatorForNonVirtual(size, mode);
+}
+
+void* CompleteSubspace::allocate(size_t size, GCDeferralContext* deferralContext, AllocationFailureMode failureMode)
+{
+    return allocateNonVirtual(size, deferralContext, failureMode);
+}
+
+void* CompleteSubspace::allocateNonVirtual(size_t size, GCDeferralContext* deferralContext, AllocationFailureMode failureMode)
+{
+    void *result;
+    if (MarkedAllocator* allocator = allocatorForNonVirtual(size, AllocatorForMode::AllocatorIfExists))
+        result = allocator->allocate(deferralContext, failureMode);
+    else
+        result = allocateSlow(size, deferralContext, failureMode);
+    return result;
+}
+
+MarkedAllocator* CompleteSubspace::allocatorForSlow(size_t size)
+{
+    size_t index = MarkedSpace::sizeClassToIndex(size);
+    size_t sizeClass = MarkedSpace::s_sizeClassForSizeStep[index];
+    if (!sizeClass)
+        return nullptr;
+    
+    // This is written in such a way that it's OK for the JIT threads to end up here if they want
+    // to generate code that uses some allocator that hadn't been used yet. Note that a possibly-
+    // just-as-good solution would be to return null if we're in the JIT since the JIT treats null
+    // allocator as "please always take the slow path". But, that could lead to performance
+    // surprises and the algorithm here is pretty easy. Only this code has to hold the lock, to
+    // prevent simultaneously MarkedAllocator creations from multiple threads. This code ensures
+    // that any "forEachAllocator" traversals will only see this allocator after it's initialized
+    // enough: it will have 
+    auto locker = holdLock(m_space.allocatorLock());
+    if (MarkedAllocator* allocator = m_allocatorForSizeStep[index])
+        return allocator;
+
+    if (false)
+        dataLog("Creating marked allocator for ", m_name, ", ", m_attributes, ", ", sizeClass, ".\n");
+    std::unique_ptr<MarkedAllocator> uniqueAllocator =
+        std::make_unique<MarkedAllocator>(m_space.heap(), sizeClass);
+    MarkedAllocator* allocator = uniqueAllocator.get();
+    m_allocators.append(WTFMove(uniqueAllocator));
+    allocator->setSubspace(this);
+    m_space.addMarkedAllocator(locker, allocator);
+    index = MarkedSpace::sizeClassToIndex(sizeClass);
+    for (;;) {
+        if (MarkedSpace::s_sizeClassForSizeStep[index] != sizeClass)
+            break;
+
+        m_allocatorForSizeStep[index] = allocator;
+        
+        if (!index--)
+            break;
+    }
+    allocator->setNextAllocatorInSubspace(m_firstAllocator);
+    m_alignedMemoryAllocator->registerAllocator(allocator);
+    WTF::storeStoreFence();
+    m_firstAllocator = allocator;
+    return allocator;
+}
+
+void* CompleteSubspace::allocateSlow(size_t size, GCDeferralContext* deferralContext, AllocationFailureMode failureMode)
+{
+    void* result = tryAllocateSlow(size, deferralContext);
+    if (failureMode == AllocationFailureMode::Assert)
+        RELEASE_ASSERT(result);
+    return result;
+}
+
+void* CompleteSubspace::tryAllocateSlow(size_t size, GCDeferralContext* deferralContext)
+{
+    sanitizeStackForVM(m_space.heap()->vm());
+    
+    if (MarkedAllocator* allocator = allocatorFor(size, AllocatorForMode::EnsureAllocator))
+        return allocator->allocate(deferralContext, AllocationFailureMode::ReturnNull);
+    
+    if (size <= Options::largeAllocationCutoff()
+        && size <= MarkedSpace::largeCutoff) {
+        dataLog("FATAL: attampting to allocate small object using large allocation.\n");
+        dataLog("Requested allocation size: ", size, "\n");
+        RELEASE_ASSERT_NOT_REACHED();
+    }
+    
+    m_space.heap()->collectIfNecessaryOrDefer(deferralContext);
+    
+    size = WTF::roundUpToMultipleOf<MarkedSpace::sizeStep>(size);
+    LargeAllocation* allocation = LargeAllocation::tryCreate(*m_space.m_heap, size, this);
+    if (!allocation)
+        return nullptr;
+    
+    m_space.m_largeAllocations.append(allocation);
+    m_space.m_heap->didAllocate(size);
+    m_space.m_capacity += size;
+    
+    m_largeAllocations.append(allocation);
+        
+    return allocation->cell();
+}
+
+} // namespace JSC
+
diff --git a/Source/JavaScriptCore/heap/CompleteSubspace.h b/Source/JavaScriptCore/heap/CompleteSubspace.h
new file mode 100644 (file)
index 0000000..942d87f
--- /dev/null
@@ -0,0 +1,83 @@
+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#pragma once
+
+#include "Subspace.h"
+
+namespace JSC {
+
+class CompleteSubspace : public Subspace {
+public:
+    JS_EXPORT_PRIVATE CompleteSubspace(CString name, Heap&, HeapCellType*, AlignedMemoryAllocator*);
+    JS_EXPORT_PRIVATE ~CompleteSubspace();
+
+    // In some code paths, we need it to be a compile error to call the virtual version of one of
+    // these functions. That's why we do final methods the old school way.
+    
+    MarkedAllocator* allocatorFor(size_t, AllocatorForMode) override;
+    MarkedAllocator* allocatorForNonVirtual(size_t, AllocatorForMode);
+    
+    void* allocate(size_t, GCDeferralContext*, AllocationFailureMode) override;
+    JS_EXPORT_PRIVATE void* allocateNonVirtual(size_t, GCDeferralContext*, AllocationFailureMode);
+    
+    static ptrdiff_t offsetOfAllocatorForSizeStep() { return OBJECT_OFFSETOF(CompleteSubspace, m_allocatorForSizeStep); }
+    
+    MarkedAllocator** allocatorForSizeStep() { return &m_allocatorForSizeStep[0]; }
+
+private:
+    MarkedAllocator* allocatorForSlow(size_t);
+    
+    // These slow paths are concerned with large allocations and allocator creation.
+    void* allocateSlow(size_t, GCDeferralContext*, AllocationFailureMode);
+    void* tryAllocateSlow(size_t, GCDeferralContext*);
+    
+    std::array<MarkedAllocator*, MarkedSpace::numSizeClasses> m_allocatorForSizeStep;
+    Vector<std::unique_ptr<MarkedAllocator>> m_allocators;
+};
+
+ALWAYS_INLINE MarkedAllocator* CompleteSubspace::allocatorForNonVirtual(size_t size, AllocatorForMode mode)
+{
+    if (size <= MarkedSpace::largeCutoff) {
+        MarkedAllocator* result = m_allocatorForSizeStep[MarkedSpace::sizeClassToIndex(size)];
+        switch (mode) {
+        case AllocatorForMode::MustAlreadyHaveAllocator:
+            RELEASE_ASSERT(result);
+            break;
+        case AllocatorForMode::EnsureAllocator:
+            if (!result)
+                return allocatorForSlow(size);
+            break;
+        case AllocatorForMode::AllocatorIfExists:
+            break;
+        }
+        return result;
+    }
+    RELEASE_ASSERT(mode != AllocatorForMode::MustAlreadyHaveAllocator);
+    return nullptr;
+}
+
+} // namespace JSC
+
diff --git a/Source/JavaScriptCore/heap/HeapCellType.cpp b/Source/JavaScriptCore/heap/HeapCellType.cpp
new file mode 100644 (file)
index 0000000..473deeb
--- /dev/null
@@ -0,0 +1,74 @@
+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#include "config.h"
+#include "HeapCellType.h"
+
+#include "JSCInlines.h"
+#include "MarkedBlockInlines.h"
+
+namespace JSC {
+
+namespace {
+
+// Writing it this way ensures that when you pass this as a functor, the callee is specialized for
+// this callback. If you wrote this as a normal function then the callee would be specialized for
+// the function's type and it would have indirect calls to that function. And unlike a lambda, it's
+// possible to mark this ALWAYS_INLINE.
+struct DestroyFunc {
+    ALWAYS_INLINE void operator()(VM& vm, JSCell* cell) const
+    {
+        ASSERT(cell->structureID());
+        ASSERT(cell->inlineTypeFlags() & StructureIsImmortal);
+        Structure* structure = cell->structure(vm);
+        const ClassInfo* classInfo = structure->classInfo();
+        MethodTable::DestroyFunctionPtr destroy = classInfo->methodTable.destroy;
+        destroy(cell);
+    }
+};
+
+} // anonymous namespace
+
+HeapCellType::HeapCellType(AllocatorAttributes attributes)
+    : m_attributes(attributes)
+{
+}
+
+HeapCellType::~HeapCellType()
+{
+}
+
+void HeapCellType::finishSweep(MarkedBlock::Handle& block, FreeList* freeList)
+{
+    block.finishSweepKnowingHeapCellType(freeList, DestroyFunc());
+}
+
+void HeapCellType::destroy(VM& vm, JSCell* cell)
+{
+    DestroyFunc()(vm, cell);
+}
+
+} // namespace JSC
+
diff --git a/Source/JavaScriptCore/heap/HeapCellType.h b/Source/JavaScriptCore/heap/HeapCellType.h
new file mode 100644 (file)
index 0000000..ade4364
--- /dev/null
@@ -0,0 +1,53 @@
+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#pragma once
+
+#include "MarkedBlock.h"
+
+namespace JSC {
+
+class HeapCellType {
+    WTF_MAKE_NONCOPYABLE(HeapCellType);
+    WTF_MAKE_FAST_ALLOCATED;
+public:
+    JS_EXPORT_PRIVATE HeapCellType(AllocatorAttributes);
+    JS_EXPORT_PRIVATE virtual ~HeapCellType();
+
+    const AllocatorAttributes& attributes() const { return m_attributes; }
+
+    // The purpose of overriding this is to specialize the sweep for your destructors. This won't
+    // be called for no-destructor blocks. This must call MarkedBlock::finishSweepKnowingSubspace.
+    virtual void finishSweep(MarkedBlock::Handle&, FreeList*);
+
+    // These get called for large objects.
+    virtual void destroy(VM&, JSCell*);
+
+private:
+    AllocatorAttributes m_attributes;
+};
+
+} // namespace JSC
+
diff --git a/Source/JavaScriptCore/heap/IsoAlignedMemoryAllocator.cpp b/Source/JavaScriptCore/heap/IsoAlignedMemoryAllocator.cpp
new file mode 100644 (file)
index 0000000..0cc8141
--- /dev/null
@@ -0,0 +1,84 @@
+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#include "config.h"
+#include "IsoAlignedMemoryAllocator.h"
+
+namespace JSC {
+
+IsoAlignedMemoryAllocator::IsoAlignedMemoryAllocator()
+{
+}
+
+IsoAlignedMemoryAllocator::~IsoAlignedMemoryAllocator()
+{
+}
+
+void* IsoAlignedMemoryAllocator::tryAllocateAlignedMemory(size_t alignment, size_t size)
+{
+    // Since this is designed specially for IsoSubspace, we know that we will only be asked to
+    // allocate MarkedBlocks.
+    RELEASE_ASSERT(alignment == MarkedBlock::blockSize);
+    RELEASE_ASSERT(size == MarkedBlock::blockSize);
+    
+    auto locker = holdLock(m_lock);
+    
+    m_firstUncommitted = m_committed.findBit(m_firstUncommitted, false);
+    if (m_firstUncommitted < m_blocks.size()) {
+        m_committed[m_firstUncommitted] = true;
+        void* result = m_blocks[m_firstUncommitted];
+        OSAllocator::commit(result, MarkedBlock::blockSize, true, false);
+        return result;
+    }
+    
+    void* result = fastAlignedMalloc(MarkedBlock::blockSize, MarkedBlock::blockSize);
+    unsigned index = m_blocks.size();
+    m_blocks.append(result);
+    m_blockIndices.add(result, index);
+    if (m_blocks.capacity() != m_committed.size())
+        m_committed.resize(m_blocks.capacity());
+    m_committed[index] = true;
+    return result;
+}
+
+void IsoAlignedMemoryAllocator::freeAlignedMemory(void* basePtr)
+{
+    auto locker = holdLock(m_lock);
+    
+    auto iter = m_blockIndices.find(basePtr);
+    RELEASE_ASSERT(iter != m_blockIndices.end());
+    unsigned index = iter->value;
+    m_committed[index] = false;
+    m_firstUncommitted = std::min(index, m_firstUncommitted);
+    OSAllocator::decommit(basePtr, MarkedBlock::blockSize);
+}
+
+void IsoAlignedMemoryAllocator::dump(PrintStream& out) const
+{
+    out.print("Iso(", RawPointer(this), ")");
+}
+
+} // namespace JSC
+
diff --git a/Source/JavaScriptCore/heap/IsoAlignedMemoryAllocator.h b/Source/JavaScriptCore/heap/IsoAlignedMemoryAllocator.h
new file mode 100644 (file)
index 0000000..09073b9
--- /dev/null
@@ -0,0 +1,51 @@
+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#pragma once
+
+#include "AlignedMemoryAllocator.h"
+
+namespace JSC {
+
+class IsoAlignedMemoryAllocator : public AlignedMemoryAllocator {
+public:
+    IsoAlignedMemoryAllocator();
+    ~IsoAlignedMemoryAllocator();
+
+    void* tryAllocateAlignedMemory(size_t alignment, size_t size) override;
+    void freeAlignedMemory(void*) override;
+
+    void dump(PrintStream&) const override;
+
+private:
+    Lock m_lock;
+    Vector<void*> m_blocks;
+    HashMap<void*, unsigned> m_blockIndices;
+    FastBitVector m_committed;
+    unsigned m_firstUncommitted { 0 };
+};
+
+} // namespace JSC
+
diff --git a/Source/JavaScriptCore/heap/IsoSubspace.cpp b/Source/JavaScriptCore/heap/IsoSubspace.cpp
new file mode 100644 (file)
index 0000000..3263ee7
--- /dev/null
@@ -0,0 +1,71 @@
+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#include "config.h"
+#include "IsoSubspace.h"
+
+#include "IsoAlignedMemoryAllocator.h"
+#include "MarkedAllocatorInlines.h"
+
+namespace JSC {
+
+IsoSubspace::IsoSubspace(CString name, Heap& heap, HeapCellType* heapCellType, size_t size)
+    : Subspace(name, heap)
+    , m_size(size)
+    , m_allocator(&heap, WTF::roundUpToMultipleOf<MarkedBlock::atomSize>(size))
+    , m_isoAlignedMemoryAllocator(std::make_unique<IsoAlignedMemoryAllocator>())
+{
+    initialize(heapCellType, m_isoAlignedMemoryAllocator.get());
+
+    auto locker = holdLock(m_space.allocatorLock());
+    m_allocator.setSubspace(this);
+    m_space.addMarkedAllocator(locker, &m_allocator);
+    m_alignedMemoryAllocator->registerAllocator(&m_allocator);
+    m_firstAllocator = &m_allocator;
+}
+
+IsoSubspace::~IsoSubspace()
+{
+}
+
+MarkedAllocator* IsoSubspace::allocatorFor(size_t size, AllocatorForMode mode)
+{
+    return allocatorForNonVirtual(size, mode);
+}
+
+void* IsoSubspace::allocate(size_t size, GCDeferralContext* deferralContext, AllocationFailureMode failureMode)
+{
+    return allocateNonVirtual(size, deferralContext, failureMode);
+}
+
+void* IsoSubspace::allocateNonVirtual(size_t size, GCDeferralContext* deferralContext, AllocationFailureMode failureMode)
+{
+    RELEASE_ASSERT(size == this->size());
+    void* result = m_allocator.allocate(deferralContext, failureMode);
+    return result;
+}
+
+} // namespace JSC
+
diff --git a/Source/JavaScriptCore/heap/IsoSubspace.h b/Source/JavaScriptCore/heap/IsoSubspace.h
new file mode 100644 (file)
index 0000000..ade8c9f
--- /dev/null
@@ -0,0 +1,63 @@
+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#pragma once
+
+#include "MarkedAllocator.h"
+#include "Subspace.h"
+
+namespace JSC {
+
+class IsoAlignedMemoryAllocator;
+
+class IsoSubspace : public Subspace {
+public:
+    JS_EXPORT_PRIVATE IsoSubspace(CString name, Heap&, HeapCellType*, size_t size);
+    JS_EXPORT_PRIVATE ~IsoSubspace();
+
+    size_t size() const { return m_size; }
+
+    MarkedAllocator* allocatorFor(size_t, AllocatorForMode) override;
+    MarkedAllocator* allocatorForNonVirtual(size_t size, AllocatorForMode);
+
+    void* allocate(size_t, GCDeferralContext*, AllocationFailureMode) override;
+    JS_EXPORT_PRIVATE void* allocateNonVirtual(size_t size, GCDeferralContext* deferralContext, AllocationFailureMode failureMode);
+
+private:
+    size_t m_size;
+    MarkedAllocator m_allocator;
+    std::unique_ptr<IsoAlignedMemoryAllocator> m_isoAlignedMemoryAllocator;
+};
+
+inline MarkedAllocator* IsoSubspace::allocatorForNonVirtual(size_t size, AllocatorForMode)
+{
+    RELEASE_ASSERT(size == this->size());
+    return &m_allocator;
+}
+
+#define ISO_SUBSPACE_INIT(heap, heapCellType, type) ("Isolated " #type " Space", (heap), (heapCellType), sizeof(type))
+
+} // namespace JSC
+
index aa2144c..7cbbfc6 100644 (file)
@@ -41,17 +41,21 @@ namespace JSC {
 
 static constexpr bool tradeDestructorBlocks = true;
 
-MarkedAllocator::MarkedAllocator(Heap* heap, Subspace* subspace, size_t cellSize)
+MarkedAllocator::MarkedAllocator(Heap* heap, size_t cellSize)
     : m_freeList(cellSize)
     , m_currentBlock(0)
     , m_lastActiveBlock(0)
     , m_cellSize(static_cast<unsigned>(cellSize))
-    , m_attributes(subspace->attributes())
     , m_heap(heap)
-    , m_subspace(subspace)
 {
 }
 
+void MarkedAllocator::setSubspace(Subspace* subspace)
+{
+    m_attributes = subspace->attributes();
+    m_subspace = subspace;
+}
+
 bool MarkedAllocator::isPagedOut(double deadline)
 {
     unsigned itersSinceLastTimeCheck = 0;
@@ -175,19 +179,7 @@ ALWAYS_INLINE void MarkedAllocator::doTestCollectionsIfNeeded(GCDeferralContext*
         allocationCount = 0;
 }
 
-void* MarkedAllocator::allocateSlowCase(GCDeferralContext* deferralContext)
-{
-    bool crashOnFailure = true;
-    return allocateSlowCaseImpl(deferralContext, crashOnFailure);
-}
-
-void* MarkedAllocator::tryAllocateSlowCase(GCDeferralContext* deferralContext)
-{
-    bool crashOnFailure = false;
-    return allocateSlowCaseImpl(deferralContext, crashOnFailure);
-}
-
-void* MarkedAllocator::allocateSlowCaseImpl(GCDeferralContext* deferralContext, bool crashOnFailure)
+void* MarkedAllocator::allocateSlowCase(GCDeferralContext* deferralContext, AllocationFailureMode failureMode)
 {
     SuperSamplerScope superSamplerScope(false);
     ASSERT(m_heap->vm()->currentThreadIsHoldingAPILock());
@@ -204,11 +196,8 @@ void* MarkedAllocator::allocateSlowCaseImpl(GCDeferralContext* deferralContext,
     
     // Goofy corner case: the GC called a callback and now this allocator has a currentBlock. This only
     // happens when running WebKit tests, which inject a callback into the GC's finalization.
-    if (UNLIKELY(m_currentBlock)) {
-        if (crashOnFailure)
-            return allocate(deferralContext);
-        return tryAllocate(deferralContext);
-    }
+    if (UNLIKELY(m_currentBlock))
+        return allocate(deferralContext, failureMode);
     
     void* result = tryAllocateWithoutCollecting();
     
@@ -217,7 +206,7 @@ void* MarkedAllocator::allocateSlowCaseImpl(GCDeferralContext* deferralContext,
     
     MarkedBlock::Handle* block = tryAllocateBlock();
     if (!block) {
-        if (crashOnFailure)
+        if (failureMode == AllocationFailureMode::Assert)
             RELEASE_ASSERT_NOT_REACHED();
         else
             return nullptr;
index e8a7e9f..c169d9d 100644 (file)
@@ -25,6 +25,7 @@
 
 #pragma once
 
+#include "AllocationFailureMode.h"
 #include "AllocatorAttributes.h"
 #include "FreeList.h"
 #include "MarkedBlock.h"
@@ -67,13 +68,17 @@ class LLIntOffsetsExtractor;
 // https://bugs.webkit.org/show_bug.cgi?id=162121
 
 class MarkedAllocator {
+    WTF_MAKE_NONCOPYABLE(MarkedAllocator);
+    WTF_MAKE_FAST_ALLOCATED;
+    
     friend class LLIntOffsetsExtractor;
 
 public:
     static ptrdiff_t offsetOfFreeList();
     static ptrdiff_t offsetOfCellSize();
 
-    MarkedAllocator(Heap*, Subspace*, size_t cellSize);
+    MarkedAllocator(Heap*, size_t cellSize);
+    void setSubspace(Subspace*);
     void lastChanceToFinalize();
     void prepareForAllocation();
     void stopAllocating();
@@ -90,8 +95,7 @@ public:
     bool needsDestruction() const { return m_attributes.destruction == NeedsDestruction; }
     DestructionMode destruction() const { return m_attributes.destruction; }
     HeapCell::Kind cellKind() const { return m_attributes.cellKind; }
-    void* allocate(GCDeferralContext* = nullptr);
-    void* tryAllocate(GCDeferralContext* = nullptr);
+    void* allocate(GCDeferralContext*, AllocationFailureMode);
     Heap* heap() { return m_heap; }
 
     bool isFreeListedCell(const void* target) const;
@@ -136,9 +140,11 @@ public:
     
     MarkedAllocator* nextAllocator() const { return m_nextAllocator; }
     MarkedAllocator* nextAllocatorInSubspace() const { return m_nextAllocatorInSubspace; }
+    MarkedAllocator* nextAllocatorInAlignedMemoryAllocator() const { return m_nextAllocatorInAlignedMemoryAllocator; }
     
     void setNextAllocator(MarkedAllocator* allocator) { m_nextAllocator = allocator; }
     void setNextAllocatorInSubspace(MarkedAllocator* allocator) { m_nextAllocatorInSubspace = allocator; }
+    void setNextAllocatorInAlignedMemoryAllocator(MarkedAllocator* allocator) { m_nextAllocatorInAlignedMemoryAllocator = allocator; }
     
     MarkedBlock::Handle* findEmptyBlockToSteal();
     
@@ -155,9 +161,7 @@ public:
 private:
     friend class MarkedBlock;
     
-    JS_EXPORT_PRIVATE void* allocateSlowCase(GCDeferralContext*);
-    JS_EXPORT_PRIVATE void* tryAllocateSlowCase(GCDeferralContext*);
-    void* allocateSlowCaseImpl(GCDeferralContext*, bool crashOnFailure);
+    JS_EXPORT_PRIVATE void* allocateSlowCase(GCDeferralContext*, AllocationFailureMode failureMode);
     void didConsumeFreeList();
     void* tryAllocateWithoutCollecting();
     MarkedBlock::Handle* tryAllocateBlock();
@@ -192,10 +196,11 @@ private:
     AllocatorAttributes m_attributes;
     // FIXME: All of these should probably be references.
     // https://bugs.webkit.org/show_bug.cgi?id=166988
-    Heap* m_heap;
-    Subspace* m_subspace;
+    Heap* m_heap { nullptr };
+    Subspace* m_subspace { nullptr };
     MarkedAllocator* m_nextAllocator { nullptr };
     MarkedAllocator* m_nextAllocatorInSubspace { nullptr };
+    MarkedAllocator* m_nextAllocatorInAlignedMemoryAllocator { nullptr };
 };
 
 inline ptrdiff_t MarkedAllocator::offsetOfFreeList()
index dbcc7c1..369d675 100644 (file)
@@ -35,21 +35,12 @@ inline bool MarkedAllocator::isFreeListedCell(const void* target) const
     return m_freeList.contains(bitwise_cast<HeapCell*>(target));
 }
 
-ALWAYS_INLINE void* MarkedAllocator::tryAllocate(GCDeferralContext* deferralContext)
+ALWAYS_INLINE void* MarkedAllocator::allocate(GCDeferralContext* deferralContext, AllocationFailureMode failureMode)
 {
     return m_freeList.allocate(
         [&] () -> HeapCell* {
             sanitizeStackForVM(heap()->vm());
-            return static_cast<HeapCell*>(tryAllocateSlowCase(deferralContext));
-        });
-}
-
-ALWAYS_INLINE void* MarkedAllocator::allocate(GCDeferralContext* deferralContext)
-{
-    return m_freeList.allocate(
-        [&] () -> HeapCell* {
-            sanitizeStackForVM(heap()->vm());
-            return static_cast<HeapCell*>(allocateSlowCase(deferralContext));
+            return static_cast<HeapCell*>(allocateSlowCase(deferralContext, failureMode));
         });
 }
 
index 5555069..e0f0de2 100644 (file)
@@ -132,7 +132,7 @@ public:
         
         // This is to be called by Subspace.
         template<typename DestroyFunc>
-        void finishSweepKnowingSubspace(FreeList*, const DestroyFunc&);
+        void finishSweepKnowingHeapCellType(FreeList*, const DestroyFunc&);
         
         void unsweepWithNoNewlyAllocated();
         
index 0019ba8..4e0b9bc 100644 (file)
@@ -272,7 +272,7 @@ void MarkedBlock::Handle::specializedSweep(FreeList* freeList, MarkedBlock::Hand
 }
 
 template<typename DestroyFunc>
-void MarkedBlock::Handle::finishSweepKnowingSubspace(FreeList* freeList, const DestroyFunc& destroyFunc)
+void MarkedBlock::Handle::finishSweepKnowingHeapCellType(FreeList* freeList, const DestroyFunc& destroyFunc)
 {
     SweepMode sweepMode = freeList ? SweepToFreeList : SweepOnly;
     SweepDestructionMode destructionMode = this->sweepDestructionMode();
index 2714e1b..fce6070 100644 (file)
@@ -132,9 +132,10 @@ const Vector<size_t>& sizeClasses()
             }
 
             // Manually inject size classes for objects we know will be allocated in high volume.
+            // FIXME: All of these things should have IsoSubspaces.
+            // https://bugs.webkit.org/show_bug.cgi?id=179876
             add(sizeof(UnlinkedFunctionExecutable));
             add(sizeof(UnlinkedFunctionCodeBlock));
-            add(sizeof(FunctionExecutable));
             add(sizeof(FunctionCodeBlock));
             add(sizeof(JSString));
             add(sizeof(JSFunction));
@@ -551,25 +552,13 @@ void MarkedSpace::dumpBits(PrintStream& out)
         });
 }
 
-MarkedAllocator* MarkedSpace::addMarkedAllocator(
-    const AbstractLocker&, Subspace* subspace, size_t sizeClass)
+void MarkedSpace::addMarkedAllocator(const AbstractLocker&, MarkedAllocator* allocator)
 {
-    MarkedAllocator* allocator = m_bagOfAllocators.add(heap(), subspace, sizeClass);
     allocator->setNextAllocator(nullptr);
     
     WTF::storeStoreFence();
 
-    if (!m_firstAllocator) {
-        m_firstAllocator = allocator;
-        m_lastAllocator = allocator;
-        for (Subspace* subspace : m_subspaces)
-            subspace->didCreateFirstAllocator(allocator);
-    } else {
-        m_lastAllocator->setNextAllocator(allocator);
-        m_lastAllocator = allocator;
-    }
-    
-    return allocator;
+    m_allocators.append(std::mem_fn(&MarkedAllocator::setNextAllocator), allocator);
 }
 
 } // namespace JSC
index 5013eaf..a6528b3 100644 (file)
 #include <wtf/Noncopyable.h>
 #include <wtf/RetainPtr.h>
 #include <wtf/SentinelLinkedList.h>
+#include <wtf/SinglyLinkedListWithTail.h>
 #include <wtf/Vector.h>
 
 namespace JSC {
 
+class CompleteSubspace;
 class Heap;
 class HeapIterationScope;
 class LLIntOffsetsExtractor;
@@ -155,10 +157,10 @@ public:
     LargeAllocation** largeAllocationsForThisCollectionEnd() const { return m_largeAllocationsForThisCollectionEnd; }
     unsigned largeAllocationsForThisCollectionSize() const { return m_largeAllocationsForThisCollectionSize; }
     
-    MarkedAllocator* firstAllocator() const { return m_firstAllocator; }
+    MarkedAllocator* firstAllocator() const { return m_allocators.first(); }
     
     Lock& allocatorLock() { return m_allocatorLock; }
-    MarkedAllocator* addMarkedAllocator(const AbstractLocker&, Subspace*, size_t cellSize);
+    void addMarkedAllocator(const AbstractLocker&, MarkedAllocator*);
     
     // When this is true it means that we have flipped but the mark bits haven't converged yet.
     bool isMarking() const { return m_isMarking; }
@@ -168,14 +170,12 @@ public:
     JS_EXPORT_PRIVATE static std::array<size_t, numSizeClasses> s_sizeClassForSizeStep;
     
 private:
+    friend class CompleteSubspace;
     friend class LLIntOffsetsExtractor;
     friend class JIT;
     friend class WeakSet;
     friend class Subspace;
     
-    void* allocateSlow(Subspace&, GCDeferralContext*, size_t);
-    void* tryAllocateSlow(Subspace&, GCDeferralContext*, size_t);
-
     // Use this version when calling from within the GC where we know that the allocators
     // have already been stopped.
     template<typename Functor> void forEachLiveCell(const Functor&);
@@ -210,9 +210,7 @@ private:
     SentinelLinkedList<WeakSet, BasicRawSentinelNode<WeakSet>> m_newActiveWeakSets;
 
     Lock m_allocatorLock;
-    Bag<MarkedAllocator> m_bagOfAllocators;
-    MarkedAllocator* m_firstAllocator { nullptr };
-    MarkedAllocator* m_lastAllocator { nullptr };
+    SinglyLinkedListWithTail<MarkedAllocator> m_allocators;
 
     friend class HeapVerifier;
 };
@@ -229,7 +227,7 @@ template <typename Functor> inline void MarkedSpace::forEachBlock(const Functor&
 template <typename Functor>
 void MarkedSpace::forEachAllocator(const Functor& functor)
 {
-    for (MarkedAllocator* allocator = m_firstAllocator; allocator; allocator = allocator->nextAllocator()) {
+    for (MarkedAllocator* allocator = m_allocators.first(); allocator; allocator = allocator->nextAllocator()) {
         if (functor(*allocator) == IterationStatus::Done)
             return;
     }
index e7188b0..6247da6 100644 (file)
@@ -26,6 +26,8 @@
 #include "config.h"
 #include "Subspace.h"
 
+#include "AlignedMemoryAllocator.h"
+#include "HeapCellType.h"
 #include "JSCInlines.h"
 #include "MarkedAllocatorInlines.h"
 #include "MarkedBlockInlines.h"
 
 namespace JSC {
 
-namespace {
-
-// Writing it this way ensures that when you pass this as a functor, the callee is specialized for
-// this callback. If you wrote this as a normal function then the callee would be specialized for
-// the function's type and it would have indirect calls to that function. And unlike a lambda, it's
-// possible to mark this ALWAYS_INLINE.
-struct DestroyFunc {
-    ALWAYS_INLINE void operator()(VM& vm, JSCell* cell) const
-    {
-        ASSERT(cell->structureID());
-        ASSERT(cell->inlineTypeFlags() & StructureIsImmortal);
-        Structure* structure = cell->structure(vm);
-        const ClassInfo* classInfo = structure->classInfo();
-        MethodTable::DestroyFunctionPtr destroy = classInfo->methodTable.destroy;
-        destroy(cell);
-    }
-};
-
-} // anonymous namespace
-
-Subspace::Subspace(CString name, Heap& heap, AllocatorAttributes attributes, AlignedMemoryAllocator* alignedMemoryAllocator)
+Subspace::Subspace(CString name, Heap& heap)
     : m_space(heap.objectSpace())
     , m_name(name)
-    , m_attributes(attributes)
-    , m_alignedMemoryAllocator(alignedMemoryAllocator)
-    , m_allocatorForEmptyAllocation(m_space.firstAllocator())
 {
-    // It's remotely possible that we're GCing right now even if the client is careful to only
-    // create subspaces right after VM creation, since collectContinuously (and probably other
-    // things) could cause a GC to be launched at pretty much any time and it's not 100% obvious
-    // that all clients would be able to ensure that there are zero safepoints between when they
-    // create VM and when they do this. Preventing GC while we're creating the Subspace ensures
-    // that we don't have to worry about whether it's OK for the GC to ever see a brand new
-    // subspace.
+}
+
+void Subspace::initialize(HeapCellType* heapCellType, AlignedMemoryAllocator* alignedMemoryAllocator)
+{
+    m_attributes = heapCellType->attributes();
+    m_heapCellType = heapCellType;
+    m_alignedMemoryAllocator = alignedMemoryAllocator;
+    m_allocatorForEmptyAllocation = m_alignedMemoryAllocator->firstAllocator();
+
+    Heap& heap = *m_space.heap();
     PreventCollectionScope preventCollectionScope(heap);
     heap.objectSpace().m_subspaces.append(this);
-    
-    for (size_t i = MarkedSpace::numSizeClasses; i--;)
-        m_allocatorForSizeStep[i] = nullptr;
+    m_alignedMemoryAllocator->registerSubspace(this);
 }
 
 Subspace::~Subspace()
@@ -81,60 +61,12 @@ Subspace::~Subspace()
 
 void Subspace::finishSweep(MarkedBlock::Handle& block, FreeList* freeList)
 {
-    block.finishSweepKnowingSubspace(freeList, DestroyFunc());
+    m_heapCellType->finishSweep(block, freeList);
 }
 
 void Subspace::destroy(VM& vm, JSCell* cell)
 {
-    DestroyFunc()(vm, cell);
-}
-
-// The reason why we distinguish between allocate and tryAllocate is to minimize the number of
-// checks on the allocation path in both cases. Likewise, the reason why we have overloads with and
-// without deferralContext is to minimize the amount of code for calling allocate when you don't
-// need the deferralContext.
-void* Subspace::allocate(size_t size)
-{
-    void* result;
-    if (MarkedAllocator* allocator = tryAllocatorFor(size))
-        result = allocator->allocate();
-    else
-        result = allocateSlow(nullptr, size);
-    didAllocate(result);
-    return result;
-}
-
-void* Subspace::allocate(GCDeferralContext* deferralContext, size_t size)
-{
-    void *result;
-    if (MarkedAllocator* allocator = tryAllocatorFor(size))
-        result = allocator->allocate(deferralContext);
-    else
-        result = allocateSlow(deferralContext, size);
-    didAllocate(result);
-    return result;
-}
-
-void* Subspace::tryAllocate(size_t size)
-{
-    void* result;
-    if (MarkedAllocator* allocator = tryAllocatorFor(size))
-        result = allocator->tryAllocate();
-    else
-        result = tryAllocateSlow(nullptr, size);
-    didAllocate(result);
-    return result;
-}
-
-void* Subspace::tryAllocate(GCDeferralContext* deferralContext, size_t size)
-{
-    void* result;
-    if (MarkedAllocator* allocator = tryAllocatorFor(size))
-        result = allocator->tryAllocate(deferralContext);
-    else
-        result = tryAllocateSlow(deferralContext, size);
-    didAllocate(result);
-    return result;
+    m_heapCellType->destroy(vm, cell);
 }
 
 void Subspace::prepareForAllocation()
@@ -144,104 +76,17 @@ void Subspace::prepareForAllocation()
             allocator.prepareForAllocation();
         });
 
-    m_allocatorForEmptyAllocation = m_space.firstAllocator();
+    m_allocatorForEmptyAllocation = m_alignedMemoryAllocator->firstAllocator();
 }
 
 MarkedBlock::Handle* Subspace::findEmptyBlockToSteal()
 {
-    for (; m_allocatorForEmptyAllocation; m_allocatorForEmptyAllocation = m_allocatorForEmptyAllocation->nextAllocator()) {
-        Subspace* otherSubspace = m_allocatorForEmptyAllocation->subspace();
-        if (otherSubspace->alignedMemoryAllocator() != alignedMemoryAllocator())
-            continue;
-        
+    for (; m_allocatorForEmptyAllocation; m_allocatorForEmptyAllocation = m_allocatorForEmptyAllocation->nextAllocatorInAlignedMemoryAllocator()) {
         if (MarkedBlock::Handle* block = m_allocatorForEmptyAllocation->findEmptyBlockToSteal())
             return block;
     }
     return nullptr;
 }
 
-MarkedAllocator* Subspace::allocatorForSlow(size_t size)
-{
-    size_t index = MarkedSpace::sizeClassToIndex(size);
-    size_t sizeClass = MarkedSpace::s_sizeClassForSizeStep[index];
-    if (!sizeClass)
-        return nullptr;
-    
-    // This is written in such a way that it's OK for the JIT threads to end up here if they want
-    // to generate code that uses some allocator that hadn't been used yet. Note that a possibly-
-    // just-as-good solution would be to return null if we're in the JIT since the JIT treats null
-    // allocator as "please always take the slow path". But, that could lead to performance
-    // surprises and the algorithm here is pretty easy. Only this code has to hold the lock, to
-    // prevent simultaneously MarkedAllocator creations from multiple threads. This code ensures
-    // that any "forEachAllocator" traversals will only see this allocator after it's initialized
-    // enough: it will have 
-    auto locker = holdLock(m_space.allocatorLock());
-    if (MarkedAllocator* allocator = m_allocatorForSizeStep[index])
-        return allocator;
-
-    if (false)
-        dataLog("Creating marked allocator for ", m_name, ", ", m_attributes, ", ", sizeClass, ".\n");
-    MarkedAllocator* allocator = m_space.addMarkedAllocator(locker, this, sizeClass);
-    index = MarkedSpace::sizeClassToIndex(sizeClass);
-    for (;;) {
-        if (MarkedSpace::s_sizeClassForSizeStep[index] != sizeClass)
-            break;
-
-        m_allocatorForSizeStep[index] = allocator;
-        
-        if (!index--)
-            break;
-    }
-    allocator->setNextAllocatorInSubspace(m_firstAllocator);
-    WTF::storeStoreFence();
-    m_firstAllocator = allocator;
-    return allocator;
-}
-
-void* Subspace::allocateSlow(GCDeferralContext* deferralContext, size_t size)
-{
-    void* result = tryAllocateSlow(deferralContext, size);
-    RELEASE_ASSERT(result);
-    return result;
-}
-
-void* Subspace::tryAllocateSlow(GCDeferralContext* deferralContext, size_t size)
-{
-    sanitizeStackForVM(m_space.heap()->vm());
-    
-    if (MarkedAllocator* allocator = allocatorFor(size))
-        return allocator->tryAllocate(deferralContext);
-    
-    if (size <= Options::largeAllocationCutoff()
-        && size <= MarkedSpace::largeCutoff) {
-        dataLog("FATAL: attampting to allocate small object using large allocation.\n");
-        dataLog("Requested allocation size: ", size, "\n");
-        RELEASE_ASSERT_NOT_REACHED();
-    }
-    
-    m_space.heap()->collectIfNecessaryOrDefer(deferralContext);
-    
-    size = WTF::roundUpToMultipleOf<MarkedSpace::sizeStep>(size);
-    LargeAllocation* allocation = LargeAllocation::tryCreate(*m_space.m_heap, size, this);
-    if (!allocation)
-        return nullptr;
-    
-    m_space.m_largeAllocations.append(allocation);
-    m_space.m_heap->didAllocate(size);
-    m_space.m_capacity += size;
-    
-    m_largeAllocations.append(allocation);
-        
-    return allocation->cell();
-}
-
-ALWAYS_INLINE void Subspace::didAllocate(void* ptr)
-{
-    UNUSED_PARAM(ptr);
-    
-    // This is useful for logging allocations, or doing other kinds of debugging hacks. Just make
-    // sure you JSC_forceGCSlowPaths=true.
-}
-
 } // namespace JSC
 
index 899ae3c..ffcbe85 100644 (file)
@@ -25,6 +25,8 @@
 
 #pragma once
 
+#include "AllocationFailureMode.h"
+#include "AllocatorForMode.h"
 #include "MarkedBlock.h"
 #include "MarkedSpace.h"
 #include <wtf/text/CString.h>
 namespace JSC {
 
 class AlignedMemoryAllocator;
+class HeapCellType;
 
 // The idea of subspaces is that you can provide some custom behavior for your objects if you
 // allocate them from a custom Subspace in which you override some of the virtual methods. This
-// class is the baseclass of Subspaces and it provides a reasonable default implementation, where
-// sweeping assumes immortal structure. The common ways of overriding this are:
-//
-// - Provide customized destructor behavior. You can change how the destructor is called. You can
-//   also specialize the destructor call in the loop.
-//
-// - Use the Subspace as a quick way to iterate all of the objects in that subspace.
+// class is the baseclass of Subspaces. Usually you will use either Subspace or FixedSizeSubspace.
 class Subspace {
     WTF_MAKE_NONCOPYABLE(Subspace);
     WTF_MAKE_FAST_ALLOCATED;
 public:
-    JS_EXPORT_PRIVATE Subspace(CString name, Heap&, AllocatorAttributes, AlignedMemoryAllocator*);
+    JS_EXPORT_PRIVATE Subspace(CString name, Heap&);
     JS_EXPORT_PRIVATE virtual ~Subspace();
-    
+
     const char* name() const { return m_name.data(); }
     MarkedSpace& space() const { return m_space; }
     
     const AllocatorAttributes& attributes() const { return m_attributes; }
+    HeapCellType* heapCellType() const { return m_heapCellType; }
     AlignedMemoryAllocator* alignedMemoryAllocator() const { return m_alignedMemoryAllocator; }
     
-    // The purpose of overriding this is to specialize the sweep for your destructors. This won't
-    // be called for no-destructor blocks. This must call MarkedBlock::finishSweepKnowingSubspace.
-    virtual void finishSweep(MarkedBlock::Handle&, FreeList*);
-    
-    // These get called for large objects.
-    virtual void destroy(VM&, JSCell*);
-    
-    MarkedAllocator* tryAllocatorFor(size_t);
-    MarkedAllocator* allocatorFor(size_t);
-    
-    JS_EXPORT_PRIVATE void* allocate(size_t);
-    JS_EXPORT_PRIVATE void* allocate(GCDeferralContext*, size_t);
-    
-    JS_EXPORT_PRIVATE void* tryAllocate(size_t);
-    JS_EXPORT_PRIVATE void* tryAllocate(GCDeferralContext*, size_t);
+    void finishSweep(MarkedBlock::Handle&, FreeList*);
+    void destroy(VM&, JSCell*);
+
+    virtual MarkedAllocator* allocatorFor(size_t, AllocatorForMode) = 0;
+    virtual void* allocate(size_t, GCDeferralContext*, AllocationFailureMode) = 0;
     
     void prepareForAllocation();
     
@@ -96,48 +84,25 @@ public:
     template<typename Func>
     void forEachLiveCell(const Func&);
     
-    static ptrdiff_t offsetOfAllocatorForSizeStep() { return OBJECT_OFFSETOF(Subspace, m_allocatorForSizeStep); }
-    
-    MarkedAllocator** allocatorForSizeStep() { return &m_allocatorForSizeStep[0]; }
+    Subspace* nextSubspaceInAlignedMemoryAllocator() const { return m_nextSubspaceInAlignedMemoryAllocator; }
+    void setNextSubspaceInAlignedMemoryAllocator(Subspace* subspace) { m_nextSubspaceInAlignedMemoryAllocator = subspace; }
 
-private:
-    MarkedAllocator* allocatorForSlow(size_t);
-    
-    // These slow paths are concerned with large allocations and allocator creation.
-    void* allocateSlow(GCDeferralContext*, size_t);
-    void* tryAllocateSlow(GCDeferralContext*, size_t);
-    
-    void didAllocate(void*);
+protected:
+    void initialize(HeapCellType*, AlignedMemoryAllocator*);
     
     MarkedSpace& m_space;
     
     CString m_name;
     AllocatorAttributes m_attributes;
+
+    HeapCellType* m_heapCellType { nullptr };
+    AlignedMemoryAllocator* m_alignedMemoryAllocator { nullptr };
     
-    AlignedMemoryAllocator* m_alignedMemoryAllocator;
-    
-    std::array<MarkedAllocator*, MarkedSpace::numSizeClasses> m_allocatorForSizeStep;
     MarkedAllocator* m_firstAllocator { nullptr };
     MarkedAllocator* m_allocatorForEmptyAllocation { nullptr }; // Uses the MarkedSpace linked list of blocks.
     SentinelLinkedList<LargeAllocation, BasicRawSentinelNode<LargeAllocation>> m_largeAllocations;
+    Subspace* m_nextSubspaceInAlignedMemoryAllocator { nullptr };
 };
 
-ALWAYS_INLINE MarkedAllocator* Subspace::tryAllocatorFor(size_t size)
-{
-    if (size <= MarkedSpace::largeCutoff)
-        return m_allocatorForSizeStep[MarkedSpace::sizeClassToIndex(size)];
-    return nullptr;
-}
-
-ALWAYS_INLINE MarkedAllocator* Subspace::allocatorFor(size_t size)
-{
-    if (size <= MarkedSpace::largeCutoff) {
-        if (MarkedAllocator* result = m_allocatorForSizeStep[MarkedSpace::sizeClassToIndex(size)])
-            return result;
-        return allocatorForSlow(size);
-    }
-    return nullptr;
-}
-
 } // namespace JSC
 
index f13fc93..c83efb7 100644 (file)
@@ -1572,7 +1572,7 @@ public:
         VM& vm, GPRReg resultGPR, StructureType structure, StorageType storage, GPRReg scratchGPR1,
         GPRReg scratchGPR2, JumpList& slowPath, size_t size)
     {
-        MarkedAllocator* allocator = subspaceFor<ClassType>(vm)->allocatorFor(size);
+        MarkedAllocator* allocator = subspaceFor<ClassType>(vm)->allocatorForNonVirtual(size, AllocatorForMode::AllocatorIfExists);
         if (!allocator) {
             slowPath.append(jump());
             return;
@@ -1589,7 +1589,7 @@ public:
     
     // allocationSize can be aliased with any of the other input GPRs. If it's not aliased then it
     // won't be clobbered.
-    void emitAllocateVariableSized(GPRReg resultGPR, Subspace& subspace, GPRReg allocationSize, GPRReg scratchGPR1, GPRReg scratchGPR2, JumpList& slowPath)
+    void emitAllocateVariableSized(GPRReg resultGPR, CompleteSubspace& subspace, GPRReg allocationSize, GPRReg scratchGPR1, GPRReg scratchGPR2, JumpList& slowPath)
     {
         static_assert(!(MarkedSpace::sizeStep & (MarkedSpace::sizeStep - 1)), "MarkedSpace::sizeStep must be a power of two.");
         
@@ -1607,7 +1607,7 @@ public:
     template<typename ClassType, typename StructureType>
     void emitAllocateVariableSizedCell(VM& vm, GPRReg resultGPR, StructureType structure, GPRReg allocationSize, GPRReg scratchGPR1, GPRReg scratchGPR2, JumpList& slowPath)
     {
-        Subspace& subspace = *subspaceFor<ClassType>(vm);
+        CompleteSubspace& subspace = *subspaceFor<ClassType>(vm);
         emitAllocateVariableSized(resultGPR, subspace, allocationSize, scratchGPR1, scratchGPR2, slowPath);
         emitStoreStructureWithTypeInfo(structure, resultGPR, scratchGPR2);
     }
index fb5d10f..4892c50 100644 (file)
@@ -81,7 +81,7 @@ void JIT::emit_op_new_object(Instruction* currentInstruction)
 {
     Structure* structure = currentInstruction[3].u.objectAllocationProfile->structure();
     size_t allocationSize = JSFinalObject::allocationSize(structure->inlineCapacity());
-    MarkedAllocator* allocator = subspaceFor<JSFinalObject>(*m_vm)->allocatorFor(allocationSize);
+    MarkedAllocator* allocator = subspaceFor<JSFinalObject>(*m_vm)->allocatorForNonVirtual(allocationSize, AllocatorForMode::AllocatorIfExists);
 
     RegisterID resultReg = regT0;
     RegisterID allocatorReg = regT1;
index 1706e01..8045128 100644 (file)
@@ -79,7 +79,7 @@ void JIT::emit_op_new_object(Instruction* currentInstruction)
 {
     Structure* structure = currentInstruction[3].u.objectAllocationProfile->structure();
     size_t allocationSize = JSFinalObject::allocationSize(structure->inlineCapacity());
-    MarkedAllocator* allocator = subspaceFor<JSFinalObject>(*m_vm)->allocatorFor(allocationSize);
+    MarkedAllocator* allocator = subspaceFor<JSFinalObject>(*m_vm)->allocatorForNonVirtual(allocationSize, AllocatorForMode::AllocatorIfExists);
 
     RegisterID resultReg = returnValueGPR;
     RegisterID allocatorReg = regT1;
index 2aa26c4..367e774 100644 (file)
@@ -62,7 +62,7 @@ ALWAYS_INLINE unsigned Butterfly::optimalContiguousVectorLength(Structure* struc
 inline Butterfly* Butterfly::createUninitialized(VM& vm, JSCell*, size_t preCapacity, size_t propertyCapacity, bool hasIndexingHeader, size_t indexingPayloadSizeInBytes)
 {
     size_t size = totalSize(preCapacity, propertyCapacity, hasIndexingHeader, indexingPayloadSizeInBytes);
-    void* base = vm.jsValueGigacageAuxiliarySpace.allocate(size);
+    void* base = vm.jsValueGigacageAuxiliarySpace.allocateNonVirtual(size, nullptr, AllocationFailureMode::Assert);
     Butterfly* result = fromBase(base, preCapacity, propertyCapacity);
     return result;
 }
@@ -70,7 +70,7 @@ inline Butterfly* Butterfly::createUninitialized(VM& vm, JSCell*, size_t preCapa
 inline Butterfly* Butterfly::tryCreate(VM& vm, JSCell*, size_t preCapacity, size_t propertyCapacity, bool hasIndexingHeader, const IndexingHeader& indexingHeader, size_t indexingPayloadSizeInBytes)
 {
     size_t size = totalSize(preCapacity, propertyCapacity, hasIndexingHeader, indexingPayloadSizeInBytes);
-    void* base = vm.jsValueGigacageAuxiliarySpace.tryAllocate(size);
+    void* base = vm.jsValueGigacageAuxiliarySpace.allocateNonVirtual(size, nullptr, AllocationFailureMode::ReturnNull);
     if (!base)
         return nullptr;
     Butterfly* result = fromBase(base, preCapacity, propertyCapacity);
@@ -148,7 +148,7 @@ inline Butterfly* Butterfly::growArrayRight(
     void* theBase = base(0, propertyCapacity);
     size_t oldSize = totalSize(0, propertyCapacity, hadIndexingHeader, oldIndexingPayloadSizeInBytes);
     size_t newSize = totalSize(0, propertyCapacity, true, newIndexingPayloadSizeInBytes);
-    void* newBase = vm.jsValueGigacageAuxiliarySpace.tryAllocate(newSize);
+    void* newBase = vm.jsValueGigacageAuxiliarySpace.allocateNonVirtual(newSize, nullptr, AllocationFailureMode::ReturnNull);
     if (!newBase)
         return nullptr;
     // FIXME: This probably shouldn't be a memcpy.
index 2dd435d..aa84b59 100644 (file)
@@ -118,8 +118,7 @@ void DirectArguments::overrideThings(VM& vm)
     putDirect(vm, vm.propertyNames->callee, m_callee.get(), static_cast<unsigned>(PropertyAttribute::DontEnum));
     putDirect(vm, vm.propertyNames->iteratorSymbol, globalObject()->arrayProtoValuesFunction(), static_cast<unsigned>(PropertyAttribute::DontEnum));
     
-    void* backingStore = vm.gigacageAuxiliarySpace(m_mappedArguments.kind).tryAllocate(mappedArgumentsSize());
-    RELEASE_ASSERT(backingStore);
+    void* backingStore = vm.gigacageAuxiliarySpace(m_mappedArguments.kind).allocateNonVirtual(mappedArgumentsSize(), nullptr, AllocationFailureMode::Assert);
     bool* overrides = static_cast<bool*>(backingStore);
     m_mappedArguments.set(vm, this, overrides);
     for (unsigned i = m_length; i--;)
index d99b494..8a14d30 100644 (file)
@@ -47,7 +47,7 @@ private:
     
 public:
     template<typename CellType>
-    static Subspace* subspaceFor(VM& vm)
+    static CompleteSubspace* subspaceFor(VM& vm)
     {
         RELEASE_ASSERT(!CellType::needsDestruction);
         return &vm.jsValueGigacageCellSpace;
index e48019f..74182fd 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2016 Apple Inc. All rights reserved.
+ * Copyright (C) 2016-2017 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -31,6 +31,12 @@ namespace JSC {
 
 class DirectEvalExecutable final : public EvalExecutable {
 public:
+    template<typename CellType>
+    static IsoSubspace* subspaceFor(VM& vm)
+    {
+        return &vm.directEvalExecutableSpace;
+    }
+
     static DirectEvalExecutable* create(ExecState*, const SourceCode&, bool isInStrictContext, DerivedContextType, bool isArrowFunctionContext, EvalContextType, const VariableEnvironment*);
 private:
     DirectEvalExecutable(ExecState*, const SourceCode&, bool inStrictContext, DerivedContextType, bool isArrowFunctionContext, EvalContextType);
index 976541b..7604bbd 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2009, 2010, 2013-2016 Apple Inc. All rights reserved.
+ * Copyright (C) 2009-2017 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -37,7 +37,7 @@ public:
     static const unsigned StructureFlags = Base::StructureFlags | StructureIsImmortal;
 
     static void destroy(JSCell*);
-
+    
     EvalCodeBlock* codeBlock()
     {
         return m_evalCodeBlock.get();
index 48eae3e..dbda0a3 100644 (file)
@@ -81,6 +81,10 @@ public:
 
     static const bool needsDestruction = true;
     static void destroy(JSCell*);
+    
+    // Force subclasses to override this.
+    template<typename>
+    static void subspaceFor(VM&) { }
         
     CodeBlockHash hashFor(CodeSpecializationKind) const;
 
index 3f6686c..7782c42 100644 (file)
@@ -38,6 +38,12 @@ public:
     typedef ScriptExecutable Base;
     static const unsigned StructureFlags = Base::StructureFlags | StructureIsImmortal;
 
+    template<typename CellType>
+    static IsoSubspace* subspaceFor(VM& vm)
+    {
+        return &vm.functionExecutableSpace;
+    }
+
     static FunctionExecutable* create(
         VM& vm, const SourceCode& source, UnlinkedFunctionExecutable* unlinkedExecutable, 
         unsigned lastLine, unsigned endColumn, Intrinsic intrinsic)
index 5893cd8..d90e59f 100644 (file)
@@ -263,8 +263,7 @@ void GenericArguments<Type>::initModifiedArgumentsDescriptor(VM& vm, unsigned ar
     RELEASE_ASSERT(!m_modifiedArgumentsDescriptor);
 
     if (argsLength) {
-        void* backingStore = vm.gigacageAuxiliarySpace(m_modifiedArgumentsDescriptor.kind).tryAllocate(WTF::roundUpToMultipleOf<8>(argsLength));
-        RELEASE_ASSERT(backingStore);
+        void* backingStore = vm.gigacageAuxiliarySpace(m_modifiedArgumentsDescriptor.kind).allocateNonVirtual(WTF::roundUpToMultipleOf<8>(argsLength), nullptr, AllocationFailureMode::Assert);
         bool* modifiedArguments = static_cast<bool*>(backingStore);
         m_modifiedArgumentsDescriptor.set(vm, this, modifiedArguments);
         for (unsigned i = argsLength; i--;)
index d0ab1fc..a6f1f7d 100644 (file)
@@ -207,7 +207,7 @@ public:
     {
         auto scope = DECLARE_THROW_SCOPE(vm);
         size_t allocationSize = HashMapBuffer::allocationSize(capacity);
-        void* data = vm.jsValueGigacageAuxiliarySpace.tryAllocate(allocationSize);
+        void* data = vm.jsValueGigacageAuxiliarySpace.allocateNonVirtual(allocationSize, nullptr, AllocationFailureMode::ReturnNull);
         if (!data) {
             throwOutOfMemoryError(exec, scope);
             return nullptr;
index 4f175e8..f64601c 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2016 Apple Inc. All rights reserved.
+ * Copyright (C) 2016-2017 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -31,6 +31,12 @@ namespace JSC {
 
 class IndirectEvalExecutable final : public EvalExecutable {
 public:
+    template<typename CellType>
+    static IsoSubspace* subspaceFor(VM& vm)
+    {
+        return &vm.indirectEvalExecutableSpace;
+    }
+
     static IndirectEvalExecutable* create(ExecState*, const SourceCode&, bool isInStrictContext, DerivedContextType, bool isArrowFunctionContext, EvalContextType);
 private:
     IndirectEvalExecutable(ExecState*, const SourceCode&, bool inStrictContext, DerivedContextType, bool isArrowFunctionContext, EvalContextType);
index ca69969..35deceb 100644 (file)
@@ -81,7 +81,7 @@ JSArray* JSArray::tryCreateUninitializedRestricted(ObjectInitializationScope& sc
             || hasContiguous(indexingType));
 
         unsigned vectorLength = Butterfly::optimalContiguousVectorLength(structure, initialLength);
-        void* temp = vm.jsValueGigacageAuxiliarySpace.tryAllocate(deferralContext, Butterfly::totalSize(0, outOfLineStorage, true, vectorLength * sizeof(EncodedJSValue)));
+        void* temp = vm.jsValueGigacageAuxiliarySpace.allocateNonVirtual(Butterfly::totalSize(0, outOfLineStorage, true, vectorLength * sizeof(EncodedJSValue)), deferralContext, AllocationFailureMode::ReturnNull);
         if (UNLIKELY(!temp))
             return nullptr;
         butterfly = Butterfly::fromBase(temp, 0, outOfLineStorage);
@@ -98,7 +98,7 @@ JSArray* JSArray::tryCreateUninitializedRestricted(ObjectInitializationScope& sc
     } else {
         static const unsigned indexBias = 0;
         unsigned vectorLength = ArrayStorage::optimalVectorLength(indexBias, structure, initialLength);
-        void* temp = vm.jsValueGigacageAuxiliarySpace.tryAllocate(deferralContext, Butterfly::totalSize(indexBias, outOfLineStorage, true, ArrayStorage::sizeFor(vectorLength)));
+        void* temp = vm.jsValueGigacageAuxiliarySpace.allocateNonVirtual(Butterfly::totalSize(indexBias, outOfLineStorage, true, ArrayStorage::sizeFor(vectorLength)), deferralContext, AllocationFailureMode::ReturnNull);
         if (UNLIKELY(!temp))
             return nullptr;
         butterfly = Butterfly::fromBase(temp, indexBias, outOfLineStorage);
@@ -368,7 +368,7 @@ bool JSArray::unshiftCountSlowCase(const AbstractLocker&, VM& vm, DeferGC&, bool
         allocatedNewStorage = false;
     } else {
         size_t newSize = Butterfly::totalSize(0, propertyCapacity, true, ArrayStorage::sizeFor(desiredCapacity));
-        newAllocBase = vm.jsValueGigacageAuxiliarySpace.tryAllocate(newSize);
+        newAllocBase = vm.jsValueGigacageAuxiliarySpace.allocateNonVirtual(newSize, nullptr, AllocationFailureMode::ReturnNull);
         if (!newAllocBase)
             return false;
         newStorageCapacity = desiredCapacity;
index abf4ea1..9ea3977 100644 (file)
@@ -237,7 +237,9 @@ inline JSArray* JSArray::tryCreate(VM& vm, Structure* structure, unsigned initia
             return nullptr;
 
         unsigned vectorLength = Butterfly::optimalContiguousVectorLength(structure, vectorLengthHint);
-        void* temp = vm.jsValueGigacageAuxiliarySpace.tryAllocate(nullptr, Butterfly::totalSize(0, outOfLineStorage, true, vectorLength * sizeof(EncodedJSValue)));
+        void* temp = vm.jsValueGigacageAuxiliarySpace.allocateNonVirtual(
+            Butterfly::totalSize(0, outOfLineStorage, true, vectorLength * sizeof(EncodedJSValue)),
+            nullptr, AllocationFailureMode::ReturnNull);
         if (!temp)
             return nullptr;
         butterfly = Butterfly::fromBase(temp, 0, outOfLineStorage);
index 31de566..28168e9 100644 (file)
@@ -66,7 +66,7 @@ JSArrayBufferView::ConstructionContext::ConstructionContext(
         void* temp;
         size_t size = sizeOf(length, elementSize);
         if (size) {
-            temp = vm.primitiveGigacageAuxiliarySpace.tryAllocate(nullptr, size);
+            temp = vm.primitiveGigacageAuxiliarySpace.allocateNonVirtual(size, nullptr, AllocationFailureMode::ReturnNull);
             if (!temp)
                 return;
         } else
index 9fc0c9b..2e91973 100644 (file)
@@ -37,6 +37,7 @@
 
 namespace JSC {
 
+class CompleteSubspace;
 class CopyVisitor;
 class GCDeferralContext;
 class ExecState;
@@ -50,11 +51,6 @@ class PropertyName;
 class PropertyNameArray;
 class Structure;
 
-enum class AllocationFailureMode {
-    ShouldAssertOnFailure,
-    ShouldNotAssertOnFailure
-};
-
 enum class GCDeferralContextArgPresense {
     HasArg,
     DoesNotHaveArg
@@ -86,8 +82,8 @@ template<typename T> void* tryAllocateCell(Heap&, GCDeferralContext*, size_t = s
 class JSCell : public HeapCell {
     friend class JSValue;
     friend class MarkedBlock;
-    template<typename T, AllocationFailureMode, GCDeferralContextArgPresense>
-    friend void* tryAllocateCellHelper(Heap&, GCDeferralContext*, size_t);
+    template<typename T>
+    friend void* tryAllocateCellHelper(Heap&, size_t, GCDeferralContext*, AllocationFailureMode);
 
 public:
     static const unsigned StructureFlags = 0;
@@ -98,7 +94,7 @@ public:
     // FIXME: Refer to Subspace by reference.
     // https://bugs.webkit.org/show_bug.cgi?id=166988
     template<typename CellType>
-    static Subspace* subspaceFor(VM&);
+    static CompleteSubspace* subspaceFor(VM&);
 
     static JSCell* seenMultipleCalleeObjects() { return bitwise_cast<JSCell*>(static_cast<uintptr_t>(1)); }
 
@@ -320,7 +316,7 @@ inline To jsDynamicCast(VM& vm, JSValue from)
 // FIXME: Refer to Subspace by reference.
 // https://bugs.webkit.org/show_bug.cgi?id=166988
 template<typename Type>
-inline Subspace* subspaceFor(VM& vm)
+inline auto subspaceFor(VM& vm)
 {
     return Type::template subspaceFor<Type>(vm);
 }
index 570f094..6736f5f 100644 (file)
@@ -135,30 +135,21 @@ ALWAYS_INLINE VM& ExecState::vm() const
 }
 
 template<typename CellType>
-Subspace* JSCell::subspaceFor(VM& vm)
+CompleteSubspace* JSCell::subspaceFor(VM& vm)
 {
     if (CellType::needsDestruction)
         return &vm.destructibleCellSpace;
     return &vm.cellSpace;
 }
 
-template<typename T, AllocationFailureMode mode, GCDeferralContextArgPresense deferralContextArgPresence>
-ALWAYS_INLINE void* tryAllocateCellHelper(Heap& heap, GCDeferralContext* deferralContext, size_t size)
+template<typename T>
+ALWAYS_INLINE void* tryAllocateCellHelper(Heap& heap, size_t size, GCDeferralContext* deferralContext, AllocationFailureMode failureMode)
 {
     ASSERT(deferralContext || !DisallowGC::isInEffectOnCurrentThread());
     ASSERT(size >= sizeof(T));
-    JSCell* result;
-    if (mode == AllocationFailureMode::ShouldAssertOnFailure) {
-        result = (deferralContextArgPresence == GCDeferralContextArgPresense::HasArg)
-            ? static_cast<JSCell*>(subspaceFor<T>(*heap.vm())->allocate(deferralContext, size))
-            : static_cast<JSCell*>(subspaceFor<T>(*heap.vm())->allocate(size));
-    } else {
-        result = (deferralContextArgPresence == GCDeferralContextArgPresense::HasArg)
-            ? static_cast<JSCell*>(subspaceFor<T>(*heap.vm())->tryAllocate(deferralContext, size))
-            : static_cast<JSCell*>(subspaceFor<T>(*heap.vm())->tryAllocate(size));
-        if (UNLIKELY(!result))
-            return nullptr;
-    }
+    JSCell* result = static_cast<JSCell*>(subspaceFor<T>(*heap.vm())->allocateNonVirtual(size, deferralContext, failureMode));
+    if (failureMode == AllocationFailureMode::ReturnNull && !result)
+        return nullptr;
 #if ENABLE(GC_VALIDATION)
     ASSERT(!heap.vm()->isInitializingObject());
     heap.vm()->setInitializingObjectClass(T::info());
@@ -170,25 +161,25 @@ ALWAYS_INLINE void* tryAllocateCellHelper(Heap& heap, GCDeferralContext* deferra
 template<typename T>
 void* allocateCell(Heap& heap, size_t size)
 {
-    return tryAllocateCellHelper<T, AllocationFailureMode::ShouldAssertOnFailure, GCDeferralContextArgPresense::DoesNotHaveArg>(heap, nullptr, size);
+    return tryAllocateCellHelper<T>(heap, size, nullptr, AllocationFailureMode::Assert);
 }
 
 template<typename T>
 void* tryAllocateCell(Heap& heap, size_t size)
 {
-    return tryAllocateCellHelper<T, AllocationFailureMode::ShouldNotAssertOnFailure, GCDeferralContextArgPresense::DoesNotHaveArg>(heap, nullptr, size);
+    return tryAllocateCellHelper<T>(heap, size, nullptr, AllocationFailureMode::ReturnNull);
 }
 
 template<typename T>
 void* allocateCell(Heap& heap, GCDeferralContext* deferralContext, size_t size)
 {
-    return tryAllocateCellHelper<T, AllocationFailureMode::ShouldAssertOnFailure, GCDeferralContextArgPresense::HasArg>(heap, deferralContext, size);
+    return tryAllocateCellHelper<T>(heap, size, deferralContext, AllocationFailureMode::Assert);
 }
 
 template<typename T>
 void* tryAllocateCell(Heap& heap, GCDeferralContext* deferralContext, size_t size)
 {
-    return tryAllocateCellHelper<T, AllocationFailureMode::ShouldNotAssertOnFailure, GCDeferralContextArgPresense::HasArg>(heap, deferralContext, size);
+    return tryAllocateCellHelper<T>(heap, size, deferralContext, AllocationFailureMode::ReturnNull);
 }
 
 inline bool JSCell::isObject() const
index 0c10f2d..effe71e 100644 (file)
@@ -38,7 +38,7 @@ public:
     static const bool needsDestruction = true;
     
     template<typename CellType>
-    static Subspace* subspaceFor(VM& vm)
+    static CompleteSubspace* subspaceFor(VM& vm)
     {
         return &vm.destructibleObjectSpace;
     }
  */
 
 #include "config.h"
-#include "JSDestructibleObjectSubspace.h"
+#include "JSDestructibleObjectHeapCellType.h"
 
 #include "MarkedBlockInlines.h"
 #include "JSCInlines.h"
-#include "SubspaceInlines.h"
 
 namespace JSC {
 
@@ -43,21 +42,21 @@ struct DestroyFunc {
 
 } // anonymous namespace
 
-JSDestructibleObjectSubspace::JSDestructibleObjectSubspace(CString name, Heap& heap, AlignedMemoryAllocator* allocator)
-    : Subspace(name, heap, AllocatorAttributes(NeedsDestruction, HeapCell::JSCell), allocator)
+JSDestructibleObjectHeapCellType::JSDestructibleObjectHeapCellType()
+    : HeapCellType(AllocatorAttributes(NeedsDestruction, HeapCell::JSCell))
 {
 }
 
-JSDestructibleObjectSubspace::~JSDestructibleObjectSubspace()
+JSDestructibleObjectHeapCellType::~JSDestructibleObjectHeapCellType()
 {
 }
 
-void JSDestructibleObjectSubspace::finishSweep(MarkedBlock::Handle& handle, FreeList* freeList)
+void JSDestructibleObjectHeapCellType::finishSweep(MarkedBlock::Handle& handle, FreeList* freeList)
 {
-    handle.finishSweepKnowingSubspace(freeList, DestroyFunc());
+    handle.finishSweepKnowingHeapCellType(freeList, DestroyFunc());
 }
 
-void JSDestructibleObjectSubspace::destroy(VM& vm, JSCell* cell)
+void JSDestructibleObjectHeapCellType::destroy(VM& vm, JSCell* cell)
 {
     DestroyFunc()(vm, cell);
 }
 
 #pragma once
 
-#include "Subspace.h"
+#include "HeapCellType.h"
 
 namespace JSC {
 
-class JSDestructibleObjectSubspace : public Subspace {
+class JSDestructibleObjectHeapCellType : public HeapCellType {
 public:
-    JS_EXPORT_PRIVATE JSDestructibleObjectSubspace(CString name, Heap&, AlignedMemoryAllocator*);
-    JS_EXPORT_PRIVATE virtual ~JSDestructibleObjectSubspace();
+    JS_EXPORT_PRIVATE JSDestructibleObjectHeapCellType();
+    JS_EXPORT_PRIVATE virtual ~JSDestructibleObjectHeapCellType();
     
     void finishSweep(MarkedBlock::Handle&, FreeList*) override;
     void destroy(VM&, JSCell*) override;
index 0a9326c..ee015e2 100644 (file)
@@ -41,7 +41,7 @@ class JSLexicalEnvironment : public JSSymbolTableObject {
     friend class LLIntOffsetsExtractor;
 public:
     template<typename CellType>
-    static Subspace* subspaceFor(VM& vm)
+    static CompleteSubspace* subspaceFor(VM& vm)
     {
         RELEASE_ASSERT(!CellType::needsDestruction);
         return &vm.jsValueGigacageCellSpace;
index 93fb987..07e9c04 100644 (file)
@@ -89,7 +89,7 @@ public:
     static void destroy(JSCell*);
     
     template<typename>
-    static Subspace* subspaceFor(VM& vm)
+    static CompleteSubspace* subspaceFor(VM& vm)
     {
         return &vm.segmentedVariableObjectSpace;
     }
  */
 
 #include "config.h"
-#include "JSSegmentedVariableObjectSubspace.h"
+#include "JSSegmentedVariableObjectHeapCellType.h"
 
 #include "JSCInlines.h"
 #include "MarkedBlockInlines.h"
-#include "SubspaceInlines.h"
 
 namespace JSC {
 
@@ -43,21 +42,21 @@ struct DestroyFunc {
 
 } // anonymous namespace
 
-JSSegmentedVariableObjectSubspace::JSSegmentedVariableObjectSubspace(CString name, Heap& heap, AlignedMemoryAllocator* allocator)
-    : Subspace(name, heap, AllocatorAttributes(NeedsDestruction, HeapCell::JSCell), allocator)
+JSSegmentedVariableObjectHeapCellType::JSSegmentedVariableObjectHeapCellType()
+    : HeapCellType(AllocatorAttributes(NeedsDestruction, HeapCell::JSCell))
 {
 }
 
-JSSegmentedVariableObjectSubspace::~JSSegmentedVariableObjectSubspace()
+JSSegmentedVariableObjectHeapCellType::~JSSegmentedVariableObjectHeapCellType()
 {
 }
 
-void JSSegmentedVariableObjectSubspace::finishSweep(MarkedBlock::Handle& handle, FreeList* freeList)
+void JSSegmentedVariableObjectHeapCellType::finishSweep(MarkedBlock::Handle& handle, FreeList* freeList)
 {
-    handle.finishSweepKnowingSubspace(freeList, DestroyFunc());
+    handle.finishSweepKnowingHeapCellType(freeList, DestroyFunc());
 }
 
-void JSSegmentedVariableObjectSubspace::destroy(VM& vm, JSCell* cell)
+void JSSegmentedVariableObjectHeapCellType::destroy(VM& vm, JSCell* cell)
 {
     DestroyFunc()(vm, cell);
 }
 
 #pragma once
 
-#include "Subspace.h"
+#include "HeapCellType.h"
 
 namespace JSC {
 
-class JSSegmentedVariableObjectSubspace : public Subspace {
+class JSSegmentedVariableObjectHeapCellType : public HeapCellType {
 public:
-    JS_EXPORT_PRIVATE JSSegmentedVariableObjectSubspace(CString name, Heap&, AlignedMemoryAllocator*);
-    JS_EXPORT_PRIVATE virtual ~JSSegmentedVariableObjectSubspace();
+    JS_EXPORT_PRIVATE JSSegmentedVariableObjectHeapCellType();
+    JS_EXPORT_PRIVATE virtual ~JSSegmentedVariableObjectHeapCellType();
     
     void finishSweep(MarkedBlock::Handle&, FreeList*) override;
     void destroy(VM&, JSCell*) override;
index 2c7c9dc..144d9f6 100644 (file)
@@ -91,7 +91,7 @@ public:
     // We specialize the string subspace to get the fastest possible sweep. This wouldn't be
     // necessary if JSString didn't have a destructor.
     template<typename>
-    static Subspace* subspaceFor(VM& vm)
+    static CompleteSubspace* subspaceFor(VM& vm)
     {
         return &vm.stringSpace;
     }
  */
 
 #include "config.h"
-#include "JSStringSubspace.h"
+#include "JSStringHeapCellType.h"
 
 #include "MarkedBlockInlines.h"
 #include "JSCInlines.h"
-#include "SubspaceInlines.h"
 
 namespace JSC {
 
@@ -43,21 +42,21 @@ struct DestroyFunc {
 
 } // anonymous namespace
 
-JSStringSubspace::JSStringSubspace(CString name, Heap& heap, AlignedMemoryAllocator* allocator)
-    : Subspace(name, heap, AllocatorAttributes(NeedsDestruction, HeapCell::JSCell), allocator)
+JSStringHeapCellType::JSStringHeapCellType()
+    : HeapCellType(AllocatorAttributes(NeedsDestruction, HeapCell::JSCell))
 {
 }
 
-JSStringSubspace::~JSStringSubspace()
+JSStringHeapCellType::~JSStringHeapCellType()
 {
 }
 
-void JSStringSubspace::finishSweep(MarkedBlock::Handle& handle, FreeList* freeList)
+void JSStringHeapCellType::finishSweep(MarkedBlock::Handle& handle, FreeList* freeList)
 {
-    handle.finishSweepKnowingSubspace(freeList, DestroyFunc());
+    handle.finishSweepKnowingHeapCellType(freeList, DestroyFunc());
 }
 
-void JSStringSubspace::destroy(VM& vm, JSCell* cell)
+void JSStringHeapCellType::destroy(VM& vm, JSCell* cell)
 {
     DestroyFunc()(vm, cell);
 }
 
 #pragma once
 
-#include "Subspace.h"
+#include "HeapCellType.h"
 
 namespace JSC {
 
-class JSStringSubspace : public Subspace {
+class JSStringHeapCellType : public HeapCellType {
 public:
-    JS_EXPORT_PRIVATE JSStringSubspace(CString name, Heap&, AlignedMemoryAllocator*);
-    JS_EXPORT_PRIVATE virtual ~JSStringSubspace();
+    JS_EXPORT_PRIVATE JSStringHeapCellType();
+    JS_EXPORT_PRIVATE virtual ~JSStringHeapCellType();
     
     void finishSweep(MarkedBlock::Handle&, FreeList*) override;
     void destroy(VM&, JSCell*) override;
index f424bad..40b8d96 100644 (file)
@@ -35,6 +35,12 @@ public:
     typedef ScriptExecutable Base;
     static const unsigned StructureFlags = Base::StructureFlags | StructureIsImmortal;
 
+    template<typename CellType>
+    static IsoSubspace* subspaceFor(VM& vm)
+    {
+        return &vm.moduleProgramExecutableSpace;
+    }
+
     static ModuleProgramExecutable* create(ExecState*, const SourceCode&);
 
     static void destroy(JSCell*);
index aa56cf4..7656ce1 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2009, 2010, 2013-2016 Apple Inc. All rights reserved.
+ * Copyright (C) 2009-2017 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -42,6 +42,12 @@ public:
     static NativeExecutable* create(VM&, Ref<JITCode>&& callThunk, NativeFunction function, Ref<JITCode>&& constructThunk, NativeFunction constructor, Intrinsic, const DOMJIT::Signature*, const String& name);
 
     static void destroy(JSCell*);
+    
+    template<typename CellType>
+    static IsoSubspace* subspaceFor(VM& vm)
+    {
+        return &vm.nativeExecutableSpace;
+    }
 
     CodeBlockHash hashFor(CodeSpecializationKind) const;
 
index d17482a..240f763 100644 (file)
@@ -35,6 +35,12 @@ public:
     typedef ScriptExecutable Base;
     static const unsigned StructureFlags = Base::StructureFlags | StructureIsImmortal;
 
+    template<typename CellType>
+    static IsoSubspace* subspaceFor(VM& vm)
+    {
+        return &vm.programExecutableSpace;
+    }
+
     static ProgramExecutable* create(ExecState* exec, const SourceCode& source)
     {
         VM& vm = exec->vm();
index af6c55b..a077868 100644 (file)
@@ -42,7 +42,7 @@ ALWAYS_INLINE JSArray* tryCreateUninitializedRegExpMatchesArray(ObjectInitializa
 
     JSGlobalObject* globalObject = structure->globalObject();
     bool createUninitialized = globalObject->isOriginalArrayStructure(structure);
-    void* temp = vm.jsValueGigacageAuxiliarySpace.tryAllocate(deferralContext, Butterfly::totalSize(0, structure->outOfLineCapacity(), true, vectorLength * sizeof(EncodedJSValue)));
+    void* temp = vm.jsValueGigacageAuxiliarySpace.allocateNonVirtual(Butterfly::totalSize(0, structure->outOfLineCapacity(), true, vectorLength * sizeof(EncodedJSValue)), deferralContext, AllocationFailureMode::ReturnNull);
     if (UNLIKELY(!temp))
         return nullptr;
     Butterfly* butterfly = Butterfly::fromBase(temp, 0, structure->outOfLineCapacity());
index 859f7e7..66f5efe 100644 (file)
@@ -43,7 +43,7 @@ private:
 
 public:
     template<typename CellType>
-    static Subspace* subspaceFor(VM& vm)
+    static CompleteSubspace* subspaceFor(VM& vm)
     {
         RELEASE_ASSERT(!CellType::needsDestruction);
         return &vm.jsValueGigacageCellSpace;
index b200857..5abdfbb 100644 (file)
@@ -39,6 +39,7 @@
 #include "CommonSlowPaths.h"
 #include "CustomGetterSetter.h"
 #include "DFGWorklist.h"
+#include "DirectEvalExecutable.h"
 #include "Disassembler.h"
 #include "ErrorInstance.h"
 #include "EvalCodeBlock.h"
@@ -47,6 +48,7 @@
 #include "FastMallocAlignedMemoryAllocator.h"
 #include "FunctionCodeBlock.h"
 #include "FunctionConstructor.h"
+#include "FunctionExecutable.h"
 #include "GCActivityCallback.h"
 #include "GetterSetter.h"
 #include "GigacageAlignedMemoryAllocator.h"
@@ -57,6 +59,7 @@
 #include "HostCallReturnValue.h"
 #include "Identifier.h"
 #include "IncrementalSweeper.h"
+#include "IndirectEvalExecutable.h"
 #include "InferredTypeTable.h"
 #include "Interpreter.h"
 #include "JITCode.h"
@@ -64,6 +67,7 @@
 #include "JSAPIValueWrapper.h"
 #include "JSArray.h"
 #include "JSCInlines.h"
+#include "JSDestructibleObjectHeapCellType.h"
 #include "JSFixedArray.h"
 #include "JSFunction.h"
 #include "JSGlobalObjectFunctions.h"
 #include "JSMapIterator.h"
 #include "JSPromiseDeferred.h"
 #include "JSPropertyNameEnumerator.h"
+#include "JSSegmentedVariableObjectHeapCellType.h"
 #include "JSScriptFetchParameters.h"
 #include "JSScriptFetcher.h"
 #include "JSSet.h"
 #include "JSSetIterator.h"
 #include "JSSourceCode.h"
+#include "JSStringHeapCellType.h"
 #include "JSTemplateRegistryKey.h"
 #include "JSWebAssembly.h"
+#include "JSWebAssemblyCodeBlockHeapCellType.h"
 #include "JSWithScope.h"
 #include "LLIntData.h"
 #include "Lexer.h"
 #include "Lookup.h"
 #include "MinimumReservedZoneSize.h"
 #include "ModuleProgramCodeBlock.h"
+#include "ModuleProgramExecutable.h"
+#include "NativeExecutable.h"
 #include "NativeStdFunctionCell.h"
 #include "Nodes.h"
 #include "Parser.h"
 #include "ProfilerDatabase.h"
 #include "ProgramCodeBlock.h"
+#include "ProgramExecutable.h"
 #include "PromiseDeferredTimer.h"
 #include "PropertyMapHashTable.h"
 #include "RegExpCache.h"
@@ -171,18 +181,33 @@ VM::VM(VMType vmType, HeapType heapType)
     , fastMallocAllocator(std::make_unique<FastMallocAlignedMemoryAllocator>())
     , primitiveGigacageAllocator(std::make_unique<GigacageAlignedMemoryAllocator>(Gigacage::Primitive))
     , jsValueGigacageAllocator(std::make_unique<GigacageAlignedMemoryAllocator>(Gigacage::JSValue))
-    , primitiveGigacageAuxiliarySpace("Primitive Gigacage Auxiliary", heap, AllocatorAttributes(DoesNotNeedDestruction, HeapCell::Auxiliary), primitiveGigacageAllocator.get())
-    , jsValueGigacageAuxiliarySpace("JSValue Gigacage Auxiliary", heap, AllocatorAttributes(DoesNotNeedDestruction, HeapCell::Auxiliary), jsValueGigacageAllocator.get())
-    , cellSpace("JSCell", heap, AllocatorAttributes(DoesNotNeedDestruction, HeapCell::JSCell), fastMallocAllocator.get())
-    , jsValueGigacageCellSpace("JSValue Gigacage JSCell", heap, AllocatorAttributes(DoesNotNeedDestruction, HeapCell::JSCell), jsValueGigacageAllocator.get())
-    , destructibleCellSpace("Destructible JSCell", heap, AllocatorAttributes(NeedsDestruction, HeapCell::JSCell), fastMallocAllocator.get())
-    , stringSpace("JSString", heap, fastMallocAllocator.get())
-    , destructibleObjectSpace("JSDestructibleObject", heap, fastMallocAllocator.get())
-    , eagerlySweptDestructibleObjectSpace("Eagerly Swept JSDestructibleObject", heap, fastMallocAllocator.get())
-    , segmentedVariableObjectSpace("JSSegmentedVariableObjectSpace", heap, fastMallocAllocator.get())
+    , auxiliaryHeapCellType(std::make_unique<HeapCellType>(AllocatorAttributes(DoesNotNeedDestruction, HeapCell::Auxiliary)))
+    , cellHeapCellType(std::make_unique<HeapCellType>(AllocatorAttributes(DoesNotNeedDestruction, HeapCell::JSCell)))
+    , destructibleCellHeapCellType(std::make_unique<HeapCellType>(AllocatorAttributes(NeedsDestruction, HeapCell::JSCell)))
+    , stringHeapCellType(std::make_unique<JSStringHeapCellType>())
+    , destructibleObjectHeapCellType(std::make_unique<JSDestructibleObjectHeapCellType>())
+    , segmentedVariableObjectHeapCellType(std::make_unique<JSSegmentedVariableObjectHeapCellType>())
 #if ENABLE(WEBASSEMBLY)
-    , webAssemblyCodeBlockSpace("JSWebAssemblyCodeBlockSpace", heap, fastMallocAllocator.get())
+    , webAssemblyCodeBlockHeapCellType(std::make_unique<JSWebAssemblyCodeBlockHeapCellType>())
 #endif
+    , primitiveGigacageAuxiliarySpace("Primitive Gigacage Auxiliary", heap, auxiliaryHeapCellType.get(), primitiveGigacageAllocator.get())
+    , jsValueGigacageAuxiliarySpace("JSValue Gigacage Auxiliary", heap, auxiliaryHeapCellType.get(), jsValueGigacageAllocator.get())
+    , cellSpace("JSCell", heap, cellHeapCellType.get(), fastMallocAllocator.get())
+    , jsValueGigacageCellSpace("JSValue Gigacage JSCell", heap, cellHeapCellType.get(), jsValueGigacageAllocator.get())
+    , destructibleCellSpace("Destructible JSCell", heap, destructibleCellHeapCellType.get(), fastMallocAllocator.get())
+    , stringSpace("JSString", heap, stringHeapCellType.get(), fastMallocAllocator.get())
+    , destructibleObjectSpace("JSDestructibleObject", heap, destructibleObjectHeapCellType.get(), fastMallocAllocator.get())
+    , eagerlySweptDestructibleObjectSpace("Eagerly Swept JSDestructibleObject", heap, destructibleObjectHeapCellType.get(), fastMallocAllocator.get())
+    , segmentedVariableObjectSpace("JSSegmentedVariableObjectSpace", heap, segmentedVariableObjectHeapCellType.get(), fastMallocAllocator.get())
+#if ENABLE(WEBASSEMBLY)
+    , webAssemblyCodeBlockSpace("JSWebAssemblyCodeBlockSpace", heap, webAssemblyCodeBlockHeapCellType.get(), fastMallocAllocator.get())
+#endif
+    , nativeExecutableSpace ISO_SUBSPACE_INIT(heap, destructibleCellHeapCellType.get(), NativeExecutable)
+    , directEvalExecutableSpace ISO_SUBSPACE_INIT(heap, destructibleCellHeapCellType.get(), DirectEvalExecutable)
+    , indirectEvalExecutableSpace ISO_SUBSPACE_INIT(heap, destructibleCellHeapCellType.get(), IndirectEvalExecutable)
+    , functionExecutableSpace ISO_SUBSPACE_INIT(heap, destructibleCellHeapCellType.get(), FunctionExecutable)
+    , moduleProgramExecutableSpace ISO_SUBSPACE_INIT(heap, destructibleCellHeapCellType.get(), ModuleProgramExecutable)
+    , programExecutableSpace ISO_SUBSPACE_INIT(heap, destructibleCellHeapCellType.get(), ProgramExecutable)
     , vmType(vmType)
     , clientData(0)
     , topEntryFrame(nullptr)
index b554b35..015f7e4 100644 (file)
@@ -30,6 +30,7 @@
 
 #include "CallData.h"
 #include "CodeSpecializationKind.h"
+#include "CompleteSubspace.h"
 #include "ConcurrentJSLock.h"
 #include "ControlFlowProfiler.h"
 #include "DateInstanceCache.h"
 #include "FunctionHasExecutedCache.h"
 #include "Heap.h"
 #include "Intrinsic.h"
+#include "IsoSubspace.h"
 #include "JITThunks.h"
 #include "JSCJSValue.h"
-#include "JSDestructibleObjectSubspace.h"
 #include "JSLock.h"
-#include "JSSegmentedVariableObjectSubspace.h"
-#include "JSStringSubspace.h"
-#include "JSWebAssemblyCodeBlockSubspace.h"
 #include "MacroAssemblerCodeRef.h"
 #include "Microtask.h"
 #include "NumericStrings.h"
 #include "SmallStrings.h"
 #include "Strong.h"
 #include "StructureCache.h"
-#include "Subspace.h"
 #include "TemplateRegistryKeyTable.h"
 #include "VMEntryRecord.h"
 #include "VMTraps.h"
@@ -108,9 +105,13 @@ class HeapProfiler;
 class Identifier;
 class Interpreter;
 class JSCustomGetterSetterFunction;
+class JSDestructibleObjectHeapCellType;
 class JSGlobalObject;
 class JSObject;
 class JSRunLoopTimer;
+class JSSegmentedVariableObjectHeapCellType;
+class JSStringHeapCellType;
+class JSWebAssemblyCodeBlockHeapCellType;
 class JSWebAssemblyInstance;
 class LLIntOffsetsExtractor;
 class NativeExecutable;
@@ -290,9 +291,19 @@ public:
     std::unique_ptr<FastMallocAlignedMemoryAllocator> fastMallocAllocator;
     std::unique_ptr<GigacageAlignedMemoryAllocator> primitiveGigacageAllocator;
     std::unique_ptr<GigacageAlignedMemoryAllocator> jsValueGigacageAllocator;
+
+    std::unique_ptr<HeapCellType> auxiliaryHeapCellType;
+    std::unique_ptr<HeapCellType> cellHeapCellType;
+    std::unique_ptr<HeapCellType> destructibleCellHeapCellType;
+    std::unique_ptr<JSStringHeapCellType> stringHeapCellType;
+    std::unique_ptr<JSDestructibleObjectHeapCellType> destructibleObjectHeapCellType;
+    std::unique_ptr<JSSegmentedVariableObjectHeapCellType> segmentedVariableObjectHeapCellType;
+#if ENABLE(WEBASSEMBLY)
+    std::unique_ptr<JSWebAssemblyCodeBlockHeapCellType> webAssemblyCodeBlockHeapCellType;
+#endif
     
-    Subspace primitiveGigacageAuxiliarySpace; // Typed arrays, strings, bitvectors, etc go here.
-    Subspace jsValueGigacageAuxiliarySpace; // Butterflies, arrays of JSValues, etc go here.
+    CompleteSubspace primitiveGigacageAuxiliarySpace; // Typed arrays, strings, bitvectors, etc go here.
+    CompleteSubspace jsValueGigacageAuxiliarySpace; // Butterflies, arrays of JSValues, etc go here.
 
     // We make cross-cutting assumptions about typed arrays being in the primitive Gigacage and butterflies
     // being in the JSValue gigacage. For some types, it's super obvious where they should go, and so we
@@ -300,7 +311,7 @@ public:
     // constant somewhere.
     // FIXME: Maybe it would be better if everyone abstracted this?
     // https://bugs.webkit.org/show_bug.cgi?id=175248
-    ALWAYS_INLINE Subspace& gigacageAuxiliarySpace(Gigacage::Kind kind)
+    ALWAYS_INLINE CompleteSubspace& gigacageAuxiliarySpace(Gigacage::Kind kind)
     {
         switch (kind) {
         case Gigacage::Primitive:
@@ -315,16 +326,23 @@ public:
     }
     
     // Whenever possible, use subspaceFor<CellType>(vm) to get one of these subspaces.
-    Subspace cellSpace;
-    Subspace jsValueGigacageCellSpace;
-    Subspace destructibleCellSpace;
-    JSStringSubspace stringSpace;
-    JSDestructibleObjectSubspace destructibleObjectSpace;
-    JSDestructibleObjectSubspace eagerlySweptDestructibleObjectSpace;
-    JSSegmentedVariableObjectSubspace segmentedVariableObjectSpace;
+    CompleteSubspace cellSpace;
+    CompleteSubspace jsValueGigacageCellSpace;
+    CompleteSubspace destructibleCellSpace;
+    CompleteSubspace stringSpace;
+    CompleteSubspace destructibleObjectSpace;
+    CompleteSubspace eagerlySweptDestructibleObjectSpace;
+    CompleteSubspace segmentedVariableObjectSpace;
 #if ENABLE(WEBASSEMBLY)
-    JSWebAssemblyCodeBlockSubspace webAssemblyCodeBlockSpace;
+    CompleteSubspace webAssemblyCodeBlockSpace;
 #endif
+    
+    IsoSubspace nativeExecutableSpace;
+    IsoSubspace directEvalExecutableSpace;
+    IsoSubspace indirectEvalExecutableSpace;
+    IsoSubspace functionExecutableSpace;
+    IsoSubspace moduleProgramExecutableSpace;
+    IsoSubspace programExecutableSpace;
 
     VMType vmType;
     ClientData* clientData;
index 890e272..77fe12f 100644 (file)
@@ -58,7 +58,7 @@ public:
     }
 
     template<typename CellType>
-    static Subspace* subspaceFor(VM& vm)
+    static CompleteSubspace* subspaceFor(VM& vm)
     {
         return &vm.webAssemblyCodeBlockSpace;
     }
  */
 
 #include "config.h"
-#include "JSWebAssemblyCodeBlockSubspace.h"
+#include "JSWebAssemblyCodeBlockHeapCellType.h"
 
 #if ENABLE(WEBASSEMBLY)
 
 #include "JSCInlines.h"
 #include "JSWebAssemblyCodeBlock.h"
 #include "MarkedBlockInlines.h"
-#include "SubspaceInlines.h"
 
 namespace JSC {
 
@@ -47,21 +46,21 @@ struct DestroyFunc {
 
 } // anonymous namespace
 
-JSWebAssemblyCodeBlockSubspace::JSWebAssemblyCodeBlockSubspace(CString name, Heap& heap, AlignedMemoryAllocator* allocator)
-    : Subspace(name, heap, AllocatorAttributes(NeedsDestruction, HeapCell::JSCell), allocator)
+JSWebAssemblyCodeBlockHeapCellType::JSWebAssemblyCodeBlockHeapCellType()
+    : HeapCellType(AllocatorAttributes(NeedsDestruction, HeapCell::JSCell))
 {
 }
 
-JSWebAssemblyCodeBlockSubspace::~JSWebAssemblyCodeBlockSubspace()
+JSWebAssemblyCodeBlockHeapCellType::~JSWebAssemblyCodeBlockHeapCellType()
 {
 }
 
-void JSWebAssemblyCodeBlockSubspace::finishSweep(MarkedBlock::Handle& handle, FreeList* freeList)
+void JSWebAssemblyCodeBlockHeapCellType::finishSweep(MarkedBlock::Handle& handle, FreeList* freeList)
 {
-    handle.finishSweepKnowingSubspace(freeList, DestroyFunc());
+    handle.finishSweepKnowingHeapCellType(freeList, DestroyFunc());
 }
 
-void JSWebAssemblyCodeBlockSubspace::destroy(VM& vm, JSCell* cell)
+void JSWebAssemblyCodeBlockHeapCellType::destroy(VM& vm, JSCell* cell)
 {
     DestroyFunc()(vm, cell);
 }
 
 #if ENABLE(WEBASSEMBLY)
 
-#include "Subspace.h"
+#include "HeapCellType.h"
 
 namespace JSC {
 
-class JSWebAssemblyCodeBlockSubspace : public Subspace {
+class JSWebAssemblyCodeBlockHeapCellType : public HeapCellType {
 public:
-    JSWebAssemblyCodeBlockSubspace(CString name, Heap&, AlignedMemoryAllocator*);
-    virtual ~JSWebAssemblyCodeBlockSubspace();
+    JSWebAssemblyCodeBlockHeapCellType();
+    virtual ~JSWebAssemblyCodeBlockHeapCellType();
     
     void finishSweep(MarkedBlock::Handle&, FreeList*) override;
     void destroy(VM&, JSCell*) override;
index 4bbbccf..e03e7b0 100644 (file)
@@ -42,7 +42,7 @@ public:
     typedef JSDestructibleObject Base;
 
     template<typename CellType>
-    static Subspace* subspaceFor(VM& vm)
+    static CompleteSubspace* subspaceFor(VM& vm)
     {
         // We hold onto a lot of memory, so it makes a lot of sense to be swept eagerly.
         return &vm.eagerlySweptDestructibleObjectSpace;
index 2f751bc..75fcfd5 100644 (file)
@@ -1,3 +1,28 @@
+2017-11-29  Filip Pizlo  <fpizlo@apple.com>
+
+        GC should support isoheaps
+        https://bugs.webkit.org/show_bug.cgi?id=179288
+
+        Reviewed by Saam Barati.
+        
+        One of my favorite data structures in the GC is a singly-linked list that knows its tail, so that
+        things get added to it at the end rather that at the beginning. In this patch, I use this to put
+        the same node on multiple lists, which our existing linked list templates also don't support.
+        
+        This adds a new linked list that does those things:
+        
+        - It supports append(). It could also support prepend(), but currently there is no need for it.
+        
+        - It supports nodes that are on multiple lists. The GC uses std::mem_fn() to create a lambda that the
+          list uses to set next.
+
+        * WTF.xcodeproj/project.pbxproj:
+        * wtf/SinglyLinkedListWithTail.h: Added.
+        (WTF::SinglyLinkedListWithTail::isEmpty const):
+        (WTF::SinglyLinkedListWithTail::append):
+        (WTF::SinglyLinkedListWithTail::first const):
+        (WTF::SinglyLinkedListWithTail::last const):
+
 2017-11-29  Stanislav Ocovaj  <stanislav.ocovaj@rt-rk.com>
 
         [DFG][MIPS] Enable DFG JIT on MIPS.
index b427808..e446847 100644 (file)
                0F43D8F01DB5ADDC00108FB6 /* AutomaticThread.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = AutomaticThread.h; sourceTree = "<group>"; };
                0F4570421BE5B58F0062A629 /* Dominators.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = Dominators.h; sourceTree = "<group>"; };
                0F4570441BE834410062A629 /* BubbleSort.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = BubbleSort.h; sourceTree = "<group>"; };
+               0F4D8C711FC1E7CE001D32AC /* SinglyLinkedListWithTail.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = SinglyLinkedListWithTail.h; sourceTree = "<group>"; };
                0F5BF1651F2317830029D91D /* NaturalLoops.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = NaturalLoops.h; sourceTree = "<group>"; };
                0F5BF1741F23D49A0029D91D /* Gigacage.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = Gigacage.cpp; sourceTree = "<group>"; };
                0F5BF1751F23D49A0029D91D /* Gigacage.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = Gigacage.h; sourceTree = "<group>"; };
                                A8A4730A151A825B004123FF /* SimpleStats.h */,
                                795212021F42588800BD6421 /* SingleRootGraph.h */,
                                A8A4730B151A825B004123FF /* SinglyLinkedList.h */,
+                               0F4D8C711FC1E7CE001D32AC /* SinglyLinkedListWithTail.h */,
                                A748744F17A0BDAE00FA04CB /* SixCharacterHash.cpp */,
                                A748745017A0BDAE00FA04CB /* SixCharacterHash.h */,
                                A8A4730C151A825B004123FF /* SizeLimits.cpp */,
diff --git a/Source/WTF/wtf/SinglyLinkedListWithTail.h b/Source/WTF/wtf/SinglyLinkedListWithTail.h
new file mode 100644 (file)
index 0000000..fb1543c
--- /dev/null
@@ -0,0 +1,63 @@
+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#pragma once
+
+#include <wtf/Assertions.h>
+#include <wtf/StdLibExtras.h>
+
+namespace WTF {
+
+template<typename T>
+class SinglyLinkedListWithTail {
+public:
+    bool isEmpty() const { return !m_first; }
+    
+    template<typename SetNextFunc>
+    void append(SetNextFunc&& setNextFunc, T* node)
+    {
+        if (!m_first) {
+            RELEASE_ASSERT(!m_last);
+            m_first = node;
+            m_last = node;
+            return;
+        }
+        
+        std::forward<SetNextFunc>(setNextFunc)(m_last, node);
+        m_last = node;
+    }
+    
+    T* first() const { return m_first; }
+    T* last() const { return m_last; }
+    
+private:
+    T* m_first { nullptr };
+    T* m_last { nullptr };
+};
+
+} // namespace WTF
+
+using WTF::SinglyLinkedListWithTail;
+
index b2191ef..7132df1 100644 (file)
@@ -1,3 +1,28 @@
+2017-11-29  Filip Pizlo  <fpizlo@apple.com>
+
+        GC should support isoheaps
+        https://bugs.webkit.org/show_bug.cgi?id=179288
+
+        Reviewed by Saam Barati.
+
+        No new tests because no new behavior.
+        
+        Adopting changes in JSC Subspace API.
+
+        * ForwardingHeaders/runtime/JSDestructibleObjectHeapCellType.h: Added.
+        * ForwardingHeaders/runtime/JSSegmentedVariableObjectHeapCellType.h: Added.
+        * bindings/js/JSDOMWrapper.cpp:
+        (WebCore::outputConstraintSubspaceFor):
+        (WebCore::globalObjectOutputConstraintSubspaceFor):
+        * bindings/js/JSDOMWrapper.h:
+        * bindings/js/WebCoreJSClientData.cpp:
+        (WebCore::JSVMClientData::JSVMClientData):
+        * bindings/js/WebCoreJSClientData.h:
+        (WebCore::JSVMClientData::outputConstraintSpace):
+        (WebCore::JSVMClientData::globalObjectOutputConstraintSpace):
+        * bindings/scripts/CodeGeneratorJS.pm:
+        (GenerateHeader):
+
 2017-11-29  Eric Carlson  <eric.carlson@apple.com>
 
         [MediaStream] Clean up audio and video capture factories
diff --git a/Source/WebCore/ForwardingHeaders/runtime/JSDestructibleObjectHeapCellType.h b/Source/WebCore/ForwardingHeaders/runtime/JSDestructibleObjectHeapCellType.h
new file mode 100644 (file)
index 0000000..a054e66
--- /dev/null
@@ -0,0 +1,2 @@
+#pragma once
+#include <JavaScriptCore/JSDestructibleObjectHeapCellType.h>
diff --git a/Source/WebCore/ForwardingHeaders/runtime/JSSegmentedVariableObjectHeapCellType.h b/Source/WebCore/ForwardingHeaders/runtime/JSSegmentedVariableObjectHeapCellType.h
new file mode 100644 (file)
index 0000000..7c0395b
--- /dev/null
@@ -0,0 +1,2 @@
+#pragma once
+#include <JavaScriptCore/JSSegmentedVariableObjectHeapCellType.h>
index c658a6b..8ef50e8 100644 (file)
@@ -45,12 +45,12 @@ JSDOMWindow& JSDOMObject::domWindow() const
     return *domWindow;
 }
 
-Subspace* outputConstraintSubspaceFor(VM& vm)
+CompleteSubspace* outputConstraintSubspaceFor(VM& vm)
 {
     return &static_cast<JSVMClientData*>(vm.clientData)->outputConstraintSpace();
 }
 
-Subspace* globalObjectOutputConstraintSubspaceFor(VM& vm)
+CompleteSubspace* globalObjectOutputConstraintSubspaceFor(VM& vm)
 {
     return &static_cast<JSVMClientData*>(vm.clientData)->globalObjectOutputConstraintSpace();
 }
index 11d439c..5ae9233 100644 (file)
@@ -73,8 +73,8 @@ protected:
     }
 };
 
-WEBCORE_EXPORT JSC::Subspace* outputConstraintSubspaceFor(JSC::VM&);
-WEBCORE_EXPORT JSC::Subspace* globalObjectOutputConstraintSubspaceFor(JSC::VM&);
+WEBCORE_EXPORT JSC::CompleteSubspace* outputConstraintSubspaceFor(JSC::VM&);
+WEBCORE_EXPORT JSC::CompleteSubspace* globalObjectOutputConstraintSubspaceFor(JSC::VM&);
 
 template<typename ImplementationClass> class JSDOMWrapper : public JSDOMObject {
 public:
index 13543e8..6daeb0f 100644 (file)
@@ -34,6 +34,8 @@
 #include <heap/MarkedBlockInlines.h>
 #include <heap/SubspaceInlines.h>
 #include <heap/VisitingTimeout.h>
+#include <runtime/JSDestructibleObjectHeapCellType.h>
+#include <runtime/JSSegmentedVariableObjectHeapCellType.h>
 #include <runtime/VM.h>
 #include <wtf/MainThread.h>
 
@@ -44,8 +46,8 @@ using namespace JSC;
 JSVMClientData::JSVMClientData(VM& vm)
     : m_builtinFunctions(vm)
     , m_builtinNames(&vm)
-    , m_outputConstraintSpace("WebCore Wrapper w/ Output Constraint", vm.heap, vm.fastMallocAllocator.get())
-    , m_globalObjectOutputConstraintSpace("WebCore Global Object w/ Output Constraint", vm.heap, vm.fastMallocAllocator.get())
+    , m_outputConstraintSpace("WebCore Wrapper w/ Output Constraint", vm.heap, vm.destructibleObjectHeapCellType.get(), vm.fastMallocAllocator.get())
+    , m_globalObjectOutputConstraintSpace("WebCore Global Object w/ Output Constraint", vm.heap, vm.segmentedVariableObjectHeapCellType.get(), vm.fastMallocAllocator.get())
 {
 }
 
index aca8a5d..3ac9aa6 100644 (file)
@@ -59,8 +59,8 @@ public:
     WebCoreBuiltinNames& builtinNames() { return m_builtinNames; }
     JSBuiltinFunctions& builtinFunctions() { return m_builtinFunctions; }
     
-    JSC::Subspace& outputConstraintSpace() { return m_outputConstraintSpace; }
-    JSC::Subspace& globalObjectOutputConstraintSpace() { return m_globalObjectOutputConstraintSpace; }
+    JSC::CompleteSubspace& outputConstraintSpace() { return m_outputConstraintSpace; }
+    JSC::CompleteSubspace& globalObjectOutputConstraintSpace() { return m_globalObjectOutputConstraintSpace; }
     
     template<typename Func>
     void forEachOutputConstraintSpace(const Func& func)
@@ -76,8 +76,8 @@ private:
     JSBuiltinFunctions m_builtinFunctions;
     WebCoreBuiltinNames m_builtinNames;
     
-    JSC::JSDestructibleObjectSubspace m_outputConstraintSpace;
-    JSC::JSSegmentedVariableObjectSubspace m_globalObjectOutputConstraintSpace;
+    JSC::CompleteSubspace m_outputConstraintSpace;
+    JSC::CompleteSubspace m_globalObjectOutputConstraintSpace;
 };
 
 } // namespace WebCore
index 92d521f..a828956 100644 (file)
@@ -2735,7 +2735,7 @@ sub GenerateHeader
             # this just calls visitAdditionalChildren, you usually don't have to worry about this.
             push(@headerContent, "    static void visitOutputConstraints(JSCell*, JSC::SlotVisitor&);\n");
             my $subspaceFunc = IsDOMGlobalObject($interface) ? "globalObjectOutputConstraintSubspaceFor" : "outputConstraintSubspaceFor";
-            push(@headerContent, "    template<typename> static JSC::Subspace* subspaceFor(JSC::VM& vm) { return $subspaceFunc(vm); }\n");
+            push(@headerContent, "    template<typename> static JSC::CompleteSubspace* subspaceFor(JSC::VM& vm) { return $subspaceFunc(vm); }\n");
         }
     }