[JSC] Shrink size of VM by lazily allocating IsoSubspaces for non-common types
authorysuzuki@apple.com <ysuzuki@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 5 Feb 2019 06:32:08 +0000 (06:32 +0000)
committerysuzuki@apple.com <ysuzuki@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 5 Feb 2019 06:32:08 +0000 (06:32 +0000)
https://bugs.webkit.org/show_bug.cgi?id=193993

Reviewed by Keith Miller.

Source/JavaScriptCore:

JSC::VM has a lot of IsoSubspaces, and each takes 504B. This unnecessarily makes VM so large.
And some of them are rarely used. We should allocate it lazily.

In this patch, we make some `IsoSubspaces` `std::unique_ptr<IsoSubspace>`. And we add ensureXXXSpace
functions which allocate IsoSubspaces lazily. This function is used by subspaceFor<> in each class.
And we also add subspaceForConcurrently<> function, which is called from concurrent JIT tiers. This
returns nullptr if the subspace is not allocated yet. JSCell::subspaceFor now takes second template
parameter which tells the function whether subspaceFor is concurrently done. If the IsoSubspace is
lazily created, we may return nullptr for the concurrent access. We ensure the space's initialization
by using WTF::storeStoreFence when lazily allocating it.

In GC's constraint solving, we may touch these lazily allocated spaces. At that time, we check the
existence of the space before touching this. This is not racy because the main thread is stopped when
the constraint solving is working.

This changes sizeof(VM) from 64736 to 56472.

Another interesting thing is that we removed `PreventCollectionScope preventCollectionScope(heap);` in
`Subspace::initialize`. This is really dangerous API since it easily causes dead-lock between the
collector and the mutator if IsoSubspace is dynamically created. We do want to make IsoSubspaces
dynamically-created ones since the requirement of the pre-allocation poses a scalability problem
of IsoSubspace adoption because IsoSubspace is large. Registered Subspace is only touched in the
EndPhase, and the peripheries should be stopped when running EndPhase. Thus, as long as the main thread
can run this IsoSubspace code, the collector is never EndPhase. So this is safe.

* API/JSCallbackFunction.h:
* API/ObjCCallbackFunction.h:
(JSC::ObjCCallbackFunction::subspaceFor):
* API/glib/JSCCallbackFunction.h:
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::visitChildren):
(JSC::CodeBlock::finalizeUnconditionally):
* bytecode/CodeBlock.h:
* bytecode/EvalCodeBlock.h:
* bytecode/ExecutableToCodeBlockEdge.h:
* bytecode/FunctionCodeBlock.h:
* bytecode/ModuleProgramCodeBlock.h:
* bytecode/ProgramCodeBlock.h:
* bytecode/UnlinkedFunctionExecutable.cpp:
(JSC::UnlinkedFunctionExecutable::unlinkedCodeBlockFor):
* bytecode/UnlinkedFunctionExecutable.h:
* dfg/DFGSpeculativeJIT.cpp:
(JSC::DFG::SpeculativeJIT::emitAllocateRawObject):
(JSC::DFG::SpeculativeJIT::compileMakeRope):
(JSC::DFG::SpeculativeJIT::compileNewObject):
* ftl/FTLLowerDFGToB3.cpp:
(JSC::FTL::DFG::LowerDFGToB3::compileMakeRope):
(JSC::FTL::DFG::LowerDFGToB3::compileMaterializeNewObject):
(JSC::FTL::DFG::LowerDFGToB3::allocateObject):
(JSC::FTL::DFG::LowerDFGToB3::allocateVariableSizedObject):
(JSC::FTL::DFG::LowerDFGToB3::allocateVariableSizedCell):
* heap/Heap.cpp:
(JSC::Heap::finalizeUnconditionalFinalizers):
(JSC::Heap::deleteAllCodeBlocks):
(JSC::Heap::deleteAllUnlinkedCodeBlocks):
(JSC::Heap::addCoreConstraints):
* heap/Subspace.cpp:
(JSC::Subspace::initialize):
* jit/AssemblyHelpers.h:
(JSC::AssemblyHelpers::emitAllocateJSObjectWithKnownSize):
(JSC::AssemblyHelpers::emitAllocateVariableSizedCell):
* jit/JITOpcodes.cpp:
(JSC::JIT::emit_op_new_object):
* jit/JITOpcodes32_64.cpp:
(JSC::JIT::emit_op_new_object):
* runtime/DirectArguments.h:
* runtime/DirectEvalExecutable.h:
* runtime/ErrorInstance.h:
(JSC::ErrorInstance::subspaceFor):
* runtime/ExecutableBase.h:
* runtime/FunctionExecutable.h:
* runtime/IndirectEvalExecutable.h:
* runtime/InferredValue.cpp:
(JSC::InferredValue::visitChildren):
* runtime/InferredValue.h:
* runtime/InferredValueInlines.h:
(JSC::InferredValue::finalizeUnconditionally):
* runtime/InternalFunction.h:
* runtime/JSAsyncFunction.h:
* runtime/JSAsyncGeneratorFunction.h:
* runtime/JSBoundFunction.h:
* runtime/JSCell.h:
(JSC::subspaceFor):
(JSC::subspaceForConcurrently):
* runtime/JSCellInlines.h:
(JSC::allocatorForNonVirtualConcurrently):
* runtime/JSCustomGetterSetterFunction.h:
* runtime/JSDestructibleObject.h:
* runtime/JSFunction.h:
* runtime/JSGeneratorFunction.h:
* runtime/JSImmutableButterfly.h:
* runtime/JSLexicalEnvironment.h:
(JSC::JSLexicalEnvironment::subspaceFor):
* runtime/JSNativeStdFunction.h:
* runtime/JSSegmentedVariableObject.h:
* runtime/JSString.h:
* runtime/ModuleProgramExecutable.h:
* runtime/NativeExecutable.h:
* runtime/ProgramExecutable.h:
* runtime/PropertyMapHashTable.h:
* runtime/ProxyRevoke.h:
* runtime/ScopedArguments.h:
* runtime/ScriptExecutable.cpp:
(JSC::ScriptExecutable::clearCode):
(JSC::ScriptExecutable::installCode):
* runtime/Structure.h:
* runtime/StructureRareData.h:
* runtime/SubspaceAccess.h: Copied from Source/JavaScriptCore/runtime/InferredValueInlines.h.
* runtime/VM.cpp:
(JSC::VM::VM):
* runtime/VM.h:
(JSC::VM::SpaceAndSet::SpaceAndSet):
(JSC::VM::SpaceAndSet::setFor):
(JSC::VM::forEachScriptExecutableSpace):
(JSC::VM::SpaceAndFinalizerSet::SpaceAndFinalizerSet): Deleted.
(JSC::VM::SpaceAndFinalizerSet::finalizerSetFor): Deleted.
(JSC::VM::ScriptExecutableSpaceAndSet::ScriptExecutableSpaceAndSet): Deleted.
(JSC::VM::ScriptExecutableSpaceAndSet::clearableCodeSetFor): Deleted.
(JSC::VM::UnlinkedFunctionExecutableSpaceAndSet::UnlinkedFunctionExecutableSpaceAndSet): Deleted.
(JSC::VM::UnlinkedFunctionExecutableSpaceAndSet::clearableCodeSetFor): Deleted.
* runtime/WeakMapImpl.h:
(JSC::WeakMapImpl::subspaceFor):
* wasm/js/JSWebAssemblyCodeBlock.h:
* wasm/js/JSWebAssemblyMemory.h:
* wasm/js/WebAssemblyFunction.h:
* wasm/js/WebAssemblyWrapperFunction.h:

Source/WebCore:

* bindings/scripts/CodeGeneratorJS.pm:
(GenerateHeader):
* bridge/runtime_method.h:

Source/WebKit:

* WebProcess/Plugins/Netscape/JSNPMethod.h:
* WebProcess/Plugins/Netscape/JSNPObject.h:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@240965 268f45cc-cd09-0410-ab3c-d52691b4dbfc

71 files changed:
Source/JavaScriptCore/API/JSCallbackFunction.h
Source/JavaScriptCore/API/ObjCCallbackFunction.h
Source/JavaScriptCore/API/glib/JSCCallbackFunction.h
Source/JavaScriptCore/CMakeLists.txt
Source/JavaScriptCore/ChangeLog
Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj
Source/JavaScriptCore/bytecode/CodeBlock.cpp
Source/JavaScriptCore/bytecode/CodeBlock.h
Source/JavaScriptCore/bytecode/EvalCodeBlock.h
Source/JavaScriptCore/bytecode/ExecutableToCodeBlockEdge.h
Source/JavaScriptCore/bytecode/FunctionCodeBlock.h
Source/JavaScriptCore/bytecode/ModuleProgramCodeBlock.h
Source/JavaScriptCore/bytecode/ProgramCodeBlock.h
Source/JavaScriptCore/bytecode/UnlinkedFunctionExecutable.cpp
Source/JavaScriptCore/bytecode/UnlinkedFunctionExecutable.h
Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp
Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp
Source/JavaScriptCore/heap/AlignedMemoryAllocator.cpp
Source/JavaScriptCore/heap/Heap.cpp
Source/JavaScriptCore/heap/MarkedSpace.cpp
Source/JavaScriptCore/heap/Subspace.cpp
Source/JavaScriptCore/jit/AssemblyHelpers.h
Source/JavaScriptCore/jit/JITOpcodes.cpp
Source/JavaScriptCore/jit/JITOpcodes32_64.cpp
Source/JavaScriptCore/runtime/DirectArguments.h
Source/JavaScriptCore/runtime/DirectEvalExecutable.h
Source/JavaScriptCore/runtime/ErrorInstance.h
Source/JavaScriptCore/runtime/ExecutableBase.h
Source/JavaScriptCore/runtime/FunctionExecutable.h
Source/JavaScriptCore/runtime/IndirectEvalExecutable.h
Source/JavaScriptCore/runtime/InferredValue.cpp
Source/JavaScriptCore/runtime/InferredValue.h
Source/JavaScriptCore/runtime/InferredValueInlines.h
Source/JavaScriptCore/runtime/InternalFunction.h
Source/JavaScriptCore/runtime/JSAsyncFunction.h
Source/JavaScriptCore/runtime/JSAsyncGeneratorFunction.h
Source/JavaScriptCore/runtime/JSBoundFunction.h
Source/JavaScriptCore/runtime/JSCell.h
Source/JavaScriptCore/runtime/JSCellInlines.h
Source/JavaScriptCore/runtime/JSCustomGetterSetterFunction.h
Source/JavaScriptCore/runtime/JSDestructibleObject.h
Source/JavaScriptCore/runtime/JSFunction.h
Source/JavaScriptCore/runtime/JSGeneratorFunction.h
Source/JavaScriptCore/runtime/JSImmutableButterfly.h
Source/JavaScriptCore/runtime/JSLexicalEnvironment.h
Source/JavaScriptCore/runtime/JSNativeStdFunction.h
Source/JavaScriptCore/runtime/JSSegmentedVariableObject.h
Source/JavaScriptCore/runtime/JSString.h
Source/JavaScriptCore/runtime/ModuleProgramExecutable.h
Source/JavaScriptCore/runtime/NativeExecutable.h
Source/JavaScriptCore/runtime/ProgramExecutable.h
Source/JavaScriptCore/runtime/PropertyMapHashTable.h
Source/JavaScriptCore/runtime/ProxyRevoke.h
Source/JavaScriptCore/runtime/ScopedArguments.h
Source/JavaScriptCore/runtime/ScriptExecutable.cpp
Source/JavaScriptCore/runtime/Structure.h
Source/JavaScriptCore/runtime/StructureRareData.h
Source/JavaScriptCore/runtime/SubspaceAccess.h [new file with mode: 0644]
Source/JavaScriptCore/runtime/VM.cpp
Source/JavaScriptCore/runtime/VM.h
Source/JavaScriptCore/runtime/WeakMapImpl.h
Source/JavaScriptCore/wasm/js/JSWebAssemblyCodeBlock.h
Source/JavaScriptCore/wasm/js/JSWebAssemblyMemory.h
Source/JavaScriptCore/wasm/js/WebAssemblyFunction.h
Source/JavaScriptCore/wasm/js/WebAssemblyWrapperFunction.h
Source/WebCore/ChangeLog
Source/WebCore/bindings/scripts/CodeGeneratorJS.pm
Source/WebCore/bridge/runtime_method.h
Source/WebKit/ChangeLog
Source/WebKit/WebProcess/Plugins/Netscape/JSNPMethod.h
Source/WebKit/WebProcess/Plugins/Netscape/JSNPObject.h

index 6cf9cfc..7b2e559 100644 (file)
@@ -37,10 +37,10 @@ class JSCallbackFunction final : public InternalFunction {
 public:
     typedef InternalFunction Base;
 
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess mode>
     static IsoSubspace* subspaceFor(VM& vm)
     {
-        return &vm.callbackFunctionSpace;
+        return vm.callbackFunctionSpace<mode>();
     }
 
     static JSCallbackFunction* create(VM&, JSGlobalObject*, JSObjectCallAsFunctionCallback, const String& name);
index c90ff0c..c30c156 100644 (file)
@@ -48,10 +48,10 @@ class ObjCCallbackFunction : public InternalFunction {
 public:
     typedef InternalFunction Base;
 
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess mode>
     static IsoSubspace* subspaceFor(VM& vm)
     {
-        return &vm.objCCallbackFunctionSpace;
+        return vm.objCCallbackFunctionSpace<mode>();
     }
 
     static ObjCCallbackFunction* create(VM&, JSGlobalObject*, const String& name, std::unique_ptr<ObjCCallbackFunctionImpl>);
index 04663fa..2c59b9b 100644 (file)
@@ -40,7 +40,7 @@ class JSCCallbackFunction : public InternalFunction {
 public:
     typedef InternalFunction Base;
 
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess>
     static IsoSubspace* subspaceFor(VM& vm)
     {
         return subspaceForImpl(vm);
index d04740b..be0c214 100644 (file)
@@ -937,6 +937,7 @@ set(JavaScriptCore_PRIVATE_FRAMEWORK_HEADERS
     runtime/StructureRareData.h
     runtime/StructureRareDataInlines.h
     runtime/StructureTransitionTable.h
+    runtime/SubspaceAccess.h
     runtime/Symbol.h
     runtime/SymbolPrototype.h
     runtime/SymbolTable.h
index 27b9869..8599815 100644 (file)
@@ -1,3 +1,139 @@
+2019-02-04  Yusuke Suzuki  <ysuzuki@apple.com>
+
+        [JSC] Shrink size of VM by lazily allocating IsoSubspaces for non-common types
+        https://bugs.webkit.org/show_bug.cgi?id=193993
+
+        Reviewed by Keith Miller.
+
+        JSC::VM has a lot of IsoSubspaces, and each takes 504B. This unnecessarily makes VM so large.
+        And some of them are rarely used. We should allocate it lazily.
+
+        In this patch, we make some `IsoSubspaces` `std::unique_ptr<IsoSubspace>`. And we add ensureXXXSpace
+        functions which allocate IsoSubspaces lazily. This function is used by subspaceFor<> in each class.
+        And we also add subspaceForConcurrently<> function, which is called from concurrent JIT tiers. This
+        returns nullptr if the subspace is not allocated yet. JSCell::subspaceFor now takes second template
+        parameter which tells the function whether subspaceFor is concurrently done. If the IsoSubspace is
+        lazily created, we may return nullptr for the concurrent access. We ensure the space's initialization
+        by using WTF::storeStoreFence when lazily allocating it.
+
+        In GC's constraint solving, we may touch these lazily allocated spaces. At that time, we check the
+        existence of the space before touching this. This is not racy because the main thread is stopped when
+        the constraint solving is working.
+
+        This changes sizeof(VM) from 64736 to 56472.
+
+        Another interesting thing is that we removed `PreventCollectionScope preventCollectionScope(heap);` in
+        `Subspace::initialize`. This is really dangerous API since it easily causes dead-lock between the
+        collector and the mutator if IsoSubspace is dynamically created. We do want to make IsoSubspaces
+        dynamically-created ones since the requirement of the pre-allocation poses a scalability problem
+        of IsoSubspace adoption because IsoSubspace is large. Registered Subspace is only touched in the
+        EndPhase, and the peripheries should be stopped when running EndPhase. Thus, as long as the main thread
+        can run this IsoSubspace code, the collector is never EndPhase. So this is safe.
+
+        * API/JSCallbackFunction.h:
+        * API/ObjCCallbackFunction.h:
+        (JSC::ObjCCallbackFunction::subspaceFor):
+        * API/glib/JSCCallbackFunction.h:
+        * CMakeLists.txt:
+        * JavaScriptCore.xcodeproj/project.pbxproj:
+        * bytecode/CodeBlock.cpp:
+        (JSC::CodeBlock::visitChildren):
+        (JSC::CodeBlock::finalizeUnconditionally):
+        * bytecode/CodeBlock.h:
+        * bytecode/EvalCodeBlock.h:
+        * bytecode/ExecutableToCodeBlockEdge.h:
+        * bytecode/FunctionCodeBlock.h:
+        * bytecode/ModuleProgramCodeBlock.h:
+        * bytecode/ProgramCodeBlock.h:
+        * bytecode/UnlinkedFunctionExecutable.cpp:
+        (JSC::UnlinkedFunctionExecutable::unlinkedCodeBlockFor):
+        * bytecode/UnlinkedFunctionExecutable.h:
+        * dfg/DFGSpeculativeJIT.cpp:
+        (JSC::DFG::SpeculativeJIT::emitAllocateRawObject):
+        (JSC::DFG::SpeculativeJIT::compileMakeRope):
+        (JSC::DFG::SpeculativeJIT::compileNewObject):
+        * ftl/FTLLowerDFGToB3.cpp:
+        (JSC::FTL::DFG::LowerDFGToB3::compileMakeRope):
+        (JSC::FTL::DFG::LowerDFGToB3::compileMaterializeNewObject):
+        (JSC::FTL::DFG::LowerDFGToB3::allocateObject):
+        (JSC::FTL::DFG::LowerDFGToB3::allocateVariableSizedObject):
+        (JSC::FTL::DFG::LowerDFGToB3::allocateVariableSizedCell):
+        * heap/Heap.cpp:
+        (JSC::Heap::finalizeUnconditionalFinalizers):
+        (JSC::Heap::deleteAllCodeBlocks):
+        (JSC::Heap::deleteAllUnlinkedCodeBlocks):
+        (JSC::Heap::addCoreConstraints):
+        * heap/Subspace.cpp:
+        (JSC::Subspace::initialize):
+        * jit/AssemblyHelpers.h:
+        (JSC::AssemblyHelpers::emitAllocateJSObjectWithKnownSize):
+        (JSC::AssemblyHelpers::emitAllocateVariableSizedCell):
+        * jit/JITOpcodes.cpp:
+        (JSC::JIT::emit_op_new_object):
+        * jit/JITOpcodes32_64.cpp:
+        (JSC::JIT::emit_op_new_object):
+        * runtime/DirectArguments.h:
+        * runtime/DirectEvalExecutable.h:
+        * runtime/ErrorInstance.h:
+        (JSC::ErrorInstance::subspaceFor):
+        * runtime/ExecutableBase.h:
+        * runtime/FunctionExecutable.h:
+        * runtime/IndirectEvalExecutable.h:
+        * runtime/InferredValue.cpp:
+        (JSC::InferredValue::visitChildren):
+        * runtime/InferredValue.h:
+        * runtime/InferredValueInlines.h:
+        (JSC::InferredValue::finalizeUnconditionally):
+        * runtime/InternalFunction.h:
+        * runtime/JSAsyncFunction.h:
+        * runtime/JSAsyncGeneratorFunction.h:
+        * runtime/JSBoundFunction.h:
+        * runtime/JSCell.h:
+        (JSC::subspaceFor):
+        (JSC::subspaceForConcurrently):
+        * runtime/JSCellInlines.h:
+        (JSC::allocatorForNonVirtualConcurrently):
+        * runtime/JSCustomGetterSetterFunction.h:
+        * runtime/JSDestructibleObject.h:
+        * runtime/JSFunction.h:
+        * runtime/JSGeneratorFunction.h:
+        * runtime/JSImmutableButterfly.h:
+        * runtime/JSLexicalEnvironment.h:
+        (JSC::JSLexicalEnvironment::subspaceFor):
+        * runtime/JSNativeStdFunction.h:
+        * runtime/JSSegmentedVariableObject.h:
+        * runtime/JSString.h:
+        * runtime/ModuleProgramExecutable.h:
+        * runtime/NativeExecutable.h:
+        * runtime/ProgramExecutable.h:
+        * runtime/PropertyMapHashTable.h:
+        * runtime/ProxyRevoke.h:
+        * runtime/ScopedArguments.h:
+        * runtime/ScriptExecutable.cpp:
+        (JSC::ScriptExecutable::clearCode):
+        (JSC::ScriptExecutable::installCode):
+        * runtime/Structure.h:
+        * runtime/StructureRareData.h:
+        * runtime/SubspaceAccess.h: Copied from Source/JavaScriptCore/runtime/InferredValueInlines.h.
+        * runtime/VM.cpp:
+        (JSC::VM::VM):
+        * runtime/VM.h:
+        (JSC::VM::SpaceAndSet::SpaceAndSet):
+        (JSC::VM::SpaceAndSet::setFor):
+        (JSC::VM::forEachScriptExecutableSpace):
+        (JSC::VM::SpaceAndFinalizerSet::SpaceAndFinalizerSet): Deleted.
+        (JSC::VM::SpaceAndFinalizerSet::finalizerSetFor): Deleted.
+        (JSC::VM::ScriptExecutableSpaceAndSet::ScriptExecutableSpaceAndSet): Deleted.
+        (JSC::VM::ScriptExecutableSpaceAndSet::clearableCodeSetFor): Deleted.
+        (JSC::VM::UnlinkedFunctionExecutableSpaceAndSet::UnlinkedFunctionExecutableSpaceAndSet): Deleted.
+        (JSC::VM::UnlinkedFunctionExecutableSpaceAndSet::clearableCodeSetFor): Deleted.
+        * runtime/WeakMapImpl.h:
+        (JSC::WeakMapImpl::subspaceFor):
+        * wasm/js/JSWebAssemblyCodeBlock.h:
+        * wasm/js/JSWebAssemblyMemory.h:
+        * wasm/js/WebAssemblyFunction.h:
+        * wasm/js/WebAssemblyWrapperFunction.h:
+
 2019-02-04  Keith Miller  <keith_miller@apple.com>
 
         Change llint operand macros to inline functions
index f842f19..f25f336 100644 (file)
                E36CC9472086314F0051FFD6 /* WasmCreationMode.h in Headers */ = {isa = PBXBuildFile; fileRef = E36CC9462086314F0051FFD6 /* WasmCreationMode.h */; settings = {ATTRIBUTES = (Private, ); }; };
                E3794E761B77EB97005543AE /* ModuleAnalyzer.h in Headers */ = {isa = PBXBuildFile; fileRef = E3794E741B77EB97005543AE /* ModuleAnalyzer.h */; settings = {ATTRIBUTES = (Private, ); }; };
                E3893A1D2203A7C600E79A74 /* AsyncFromSyncIteratorPrototype.lut.h in Headers */ = {isa = PBXBuildFile; fileRef = E3893A1C2203A7C600E79A74 /* AsyncFromSyncIteratorPrototype.lut.h */; };
+               E39006212208BFC4001019CF /* SubspaceAccess.h in Headers */ = {isa = PBXBuildFile; fileRef = E39006202208BFC3001019CF /* SubspaceAccess.h */; settings = {ATTRIBUTES = (Private, ); }; };
                E393ADD81FE702D00022D681 /* WeakMapImplInlines.h in Headers */ = {isa = PBXBuildFile; fileRef = E393ADD71FE702CC0022D681 /* WeakMapImplInlines.h */; };
                E39D45F51D39005600B3B377 /* InterpreterInlines.h in Headers */ = {isa = PBXBuildFile; fileRef = E39D9D841D39000600667282 /* InterpreterInlines.h */; settings = {ATTRIBUTES = (Private, ); }; };
                E39DA4A71B7E8B7C0084F33A /* JSModuleRecord.h in Headers */ = {isa = PBXBuildFile; fileRef = E39DA4A51B7E8B7C0084F33A /* JSModuleRecord.h */; settings = {ATTRIBUTES = (Private, ); }; };
                E3794E741B77EB97005543AE /* ModuleAnalyzer.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ModuleAnalyzer.h; sourceTree = "<group>"; };
                E380A76B1DCD7195000F89E6 /* MacroAssemblerHelpers.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = MacroAssemblerHelpers.h; sourceTree = "<group>"; };
                E380D66B1F19249D00A59095 /* BuiltinNames.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = BuiltinNames.cpp; sourceTree = "<group>"; };
-               E3893A1C2203A7C600E79A74 /* AsyncFromSyncIteratorPrototype.lut.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = AsyncFromSyncIteratorPrototype.lut.h; path = AsyncFromSyncIteratorPrototype.lut.h; sourceTree = "<group>"; };
+               E3893A1C2203A7C600E79A74 /* AsyncFromSyncIteratorPrototype.lut.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = AsyncFromSyncIteratorPrototype.lut.h; sourceTree = "<group>"; };
                E38D060B1F8E814100649CF2 /* JSScriptFetchParameters.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSScriptFetchParameters.h; sourceTree = "<group>"; };
                E38D060C1F8E814100649CF2 /* ScriptFetchParameters.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ScriptFetchParameters.h; sourceTree = "<group>"; };
                E38D060D1F8E814100649CF2 /* JSScriptFetchParameters.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = JSScriptFetchParameters.cpp; sourceTree = "<group>"; };
+               E39006202208BFC3001019CF /* SubspaceAccess.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = SubspaceAccess.h; sourceTree = "<group>"; };
                E393ADD71FE702CC0022D681 /* WeakMapImplInlines.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WeakMapImplInlines.h; sourceTree = "<group>"; };
                E3963CEC1B73F75000EB4CE5 /* NodesAnalyzeModule.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = NodesAnalyzeModule.cpp; sourceTree = "<group>"; };
                E39D9D841D39000600667282 /* InterpreterInlines.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = InterpreterInlines.h; sourceTree = "<group>"; };
                                C2FE18A316BAEC4000AF3061 /* StructureRareData.h */,
                                C20BA92C16BB1C1500B3AEA2 /* StructureRareDataInlines.h */,
                                BC9041470EB9250900FE26FA /* StructureTransitionTable.h */,
+                               E39006202208BFC3001019CF /* SubspaceAccess.h */,
                                705B41A31A6E501E00716757 /* Symbol.cpp */,
                                705B41A41A6E501E00716757 /* Symbol.h */,
                                705B41A51A6E501E00716757 /* SymbolConstructor.cpp */,
                                BC9041480EB9250900FE26FA /* StructureTransitionTable.h in Headers */,
                                0F44767020C5E2B4008B2C36 /* StubInfoSummary.h in Headers */,
                                0F7DF1371E2970E10095951B /* Subspace.h in Headers */,
+                               E39006212208BFC4001019CF /* SubspaceAccess.h in Headers */,
                                0F7DF1381E2970E40095951B /* SubspaceInlines.h in Headers */,
                                0F4A38FA1C8E13DF00190318 /* SuperSampler.h in Headers */,
                                530A66CD1FB1346D0026A545 /* SuperSamplerBytecodeScope.h in Headers */,
index 0707789..3837d11 100644 (file)
@@ -997,7 +997,7 @@ void CodeBlock::visitChildren(SlotVisitor& visitor)
     stronglyVisitStrongReferences(locker, visitor);
     stronglyVisitWeakReferences(locker, visitor);
     
-    VM::SpaceAndFinalizerSet::finalizerSetFor(*subspace()).add(this);
+    VM::SpaceAndSet::setFor(*subspace()).add(this);
 }
 
 bool CodeBlock::shouldVisitStrongly(const ConcurrentJSLocker& locker)
@@ -1392,7 +1392,7 @@ void CodeBlock::finalizeUnconditionally(VM&)
     }
 #endif // ENABLE(DFG_JIT)
 
-    VM::SpaceAndFinalizerSet::finalizerSetFor(*subspace()).remove(this);
+    VM::SpaceAndSet::setFor(*subspace()).remove(this);
 }
 
 void CodeBlock::destroy(JSCell* cell)
index b091683..d5a232b 100644 (file)
@@ -114,7 +114,7 @@ public:
     static const unsigned StructureFlags = Base::StructureFlags | StructureIsImmortal;
     static const bool needsDestruction = true;
 
-    template<typename>
+    template<typename, SubspaceAccess>
     static void subspaceFor(VM&) { }
 
     DECLARE_INFO;
index 1166962..e568a8d 100644 (file)
@@ -38,7 +38,7 @@ public:
     typedef GlobalCodeBlock Base;
     DECLARE_INFO;
 
-    template<typename>
+    template<typename, SubspaceAccess>
     static IsoSubspace* subspaceFor(VM& vm)
     {
         return &vm.codeBlockSpace.space;
index 91b6c8a..feadd2b 100644 (file)
@@ -40,7 +40,7 @@ public:
     typedef JSCell Base;
     static const unsigned StructureFlags = Base::StructureFlags | StructureIsImmortal;
 
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess>
     static IsoSubspace* subspaceFor(VM& vm)
     {
         return &vm.executableToCodeBlockEdgeSpace;
index 55540bd..cccb5c5 100644 (file)
@@ -39,7 +39,7 @@ public:
     typedef CodeBlock Base;
     DECLARE_INFO;
 
-    template<typename>
+    template<typename, SubspaceAccess>
     static IsoSubspace* subspaceFor(VM& vm)
     {
         return &vm.codeBlockSpace.space;
index f74784e..467b3d5 100644 (file)
@@ -39,7 +39,7 @@ public:
     typedef GlobalCodeBlock Base;
     DECLARE_INFO;
 
-    template<typename>
+    template<typename, SubspaceAccess>
     static IsoSubspace* subspaceFor(VM& vm)
     {
         return &vm.codeBlockSpace.space;
index be89b6d..e8ba30f 100644 (file)
@@ -39,7 +39,7 @@ public:
     typedef GlobalCodeBlock Base;
     DECLARE_INFO;
 
-    template<typename>
+    template<typename, SubspaceAccess>
     static IsoSubspace* subspaceFor(VM& vm)
     {
         return &vm.codeBlockSpace.space;
index 7b29fc3..ac416e4 100644 (file)
@@ -237,7 +237,7 @@ UnlinkedFunctionCodeBlock* UnlinkedFunctionExecutable::unlinkedCodeBlockFor(
         m_unlinkedCodeBlockForConstruct.set(vm, this, result);
         break;
     }
-    vm.unlinkedFunctionExecutableSpace.clearableCodeSet.add(this);
+    vm.unlinkedFunctionExecutableSpace.set.add(this);
     return result;
 }
 
index d637d68..c8109ee 100644 (file)
@@ -61,7 +61,7 @@ public:
     typedef JSCell Base;
     static const unsigned StructureFlags = Base::StructureFlags | StructureIsImmortal;
 
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess>
     static IsoSubspace* subspaceFor(VM& vm)
     {
         return &vm.unlinkedFunctionExecutableSpace.space;
@@ -120,7 +120,7 @@ public:
     {
         m_unlinkedCodeBlockForCall.clear();
         m_unlinkedCodeBlockForConstruct.clear();
-        vm.unlinkedFunctionExecutableSpace.clearableCodeSet.remove(this);
+        vm.unlinkedFunctionExecutableSpace.set.remove(this);
     }
 
     void recordParse(CodeFeatures features, bool hasCapturedVariables)
index 9ec916f..2fed431 100644 (file)
@@ -130,7 +130,7 @@ void SpeculativeJIT::emitAllocateRawObject(GPRReg resultGPR, RegisteredStructure
     }
 
     size_t allocationSize = JSFinalObject::allocationSize(inlineCapacity);
-    Allocator allocator = subspaceFor<JSFinalObject>(*m_jit.vm())->allocatorForNonVirtual(allocationSize, AllocatorForMode::AllocatorIfExists);
+    Allocator allocator = allocatorForNonVirtualConcurrently<JSFinalObject>(*m_jit.vm(), allocationSize, AllocatorForMode::AllocatorIfExists);
     if (allocator) {
         emitAllocateJSObject(resultGPR, JITAllocator::constant(allocator), scratchGPR, TrustedImmPtr(structure), storageGPR, scratch2GPR, slowCases);
         m_jit.emitInitializeInlineStorage(resultGPR, structure->inlineCapacity());
@@ -4359,7 +4359,7 @@ void SpeculativeJIT::compileMakeRope(Node* node)
     GPRReg scratchGPR = scratch.gpr();
     
     JITCompiler::JumpList slowPath;
-    Allocator allocatorValue = subspaceFor<JSRopeString>(*m_jit.vm())->allocatorForNonVirtual(sizeof(JSRopeString), AllocatorForMode::AllocatorIfExists);
+    Allocator allocatorValue = allocatorForNonVirtualConcurrently<JSRopeString>(*m_jit.vm(), sizeof(JSRopeString), AllocatorForMode::AllocatorIfExists);
     emitAllocateJSCell(resultGPR, JITAllocator::constant(allocatorValue), allocatorGPR, TrustedImmPtr(m_jit.graph().registerStructure(m_jit.vm()->stringStructure.get())), scratchGPR, slowPath);
         
     m_jit.storePtr(TrustedImmPtr(nullptr), JITCompiler::Address(resultGPR, JSString::offsetOfValue()));
@@ -12540,8 +12540,7 @@ void SpeculativeJIT::compileNewObject(Node* node)
 
     RegisteredStructure structure = node->structure();
     size_t allocationSize = JSFinalObject::allocationSize(structure->inlineCapacity());
-    Allocator allocatorValue = subspaceFor<JSFinalObject>(*m_jit.vm())->allocatorForNonVirtual(allocationSize, AllocatorForMode::AllocatorIfExists);
-
+    Allocator allocatorValue = allocatorForNonVirtualConcurrently<JSFinalObject>(*m_jit.vm(), allocationSize, AllocatorForMode::AllocatorIfExists);
     if (!allocatorValue)
         slowPath.append(m_jit.jump());
     else {
index f4764b7..584051d 100644 (file)
@@ -6472,7 +6472,7 @@ private:
         
         LBasicBlock lastNext = m_out.insertNewBlocksBefore(slowPath);
         
-        Allocator allocator = subspaceFor<JSRopeString>(vm())->allocatorForNonVirtual(sizeof(JSRopeString), AllocatorForMode::AllocatorIfExists);
+        Allocator allocator = allocatorForNonVirtualConcurrently<JSRopeString>(vm(), sizeof(JSRopeString), AllocatorForMode::AllocatorIfExists);
         
         LValue result = allocateCell(
             m_out.constIntPtr(allocator.localAllocator()), vm().stringStructure.get(), slowPath);
@@ -10723,7 +10723,7 @@ private:
             
             if (structure->outOfLineCapacity() || hasIndexedProperties(structure->indexingType())) {
                 size_t allocationSize = JSFinalObject::allocationSize(structure->inlineCapacity());
-                Allocator cellAllocator = subspaceFor<JSFinalObject>(vm())->allocatorForNonVirtual(allocationSize, AllocatorForMode::AllocatorIfExists);
+                Allocator cellAllocator = allocatorForNonVirtualConcurrently<JSFinalObject>(vm(), allocationSize, AllocatorForMode::AllocatorIfExists);
 
                 bool hasIndexingHeader = hasIndexedProperties(structure->indexingType());
                 unsigned indexingHeaderSize = 0;
@@ -13190,7 +13190,7 @@ private:
     LValue allocateObject(
         size_t size, StructureType structure, LValue butterfly, LBasicBlock slowPath)
     {
-        Allocator allocator = subspaceFor<ClassType>(vm())->allocatorForNonVirtual(size, AllocatorForMode::AllocatorIfExists);
+        Allocator allocator = allocatorForNonVirtualConcurrently<ClassType>(vm(), size, AllocatorForMode::AllocatorIfExists);
         return allocateObject(
             m_out.constIntPtr(allocator.localAllocator()), structure, butterfly, slowPath);
     }
@@ -13254,7 +13254,9 @@ private:
     LValue allocateVariableSizedObject(
         LValue size, RegisteredStructure structure, LValue butterfly, LBasicBlock slowPath)
     {
-        LValue allocator = allocatorForSize(*subspaceFor<ClassType>(vm()), size, slowPath);
+        CompleteSubspace* subspace = subspaceForConcurrently<ClassType>(vm());
+        RELEASE_ASSERT_WITH_MESSAGE(subspace, "CompleteSubspace is always allocated");
+        LValue allocator = allocatorForSize(*subspace, size, slowPath);
         return allocateObject(allocator, structure, butterfly, slowPath);
     }
 
@@ -13262,14 +13264,16 @@ private:
     LValue allocateVariableSizedCell(
         LValue size, Structure* structure, LBasicBlock slowPath)
     {
-        LValue allocator = allocatorForSize(*subspaceFor<ClassType>(vm()), size, slowPath);
+        CompleteSubspace* subspace = subspaceForConcurrently<ClassType>(vm());
+        RELEASE_ASSERT_WITH_MESSAGE(subspace, "CompleteSubspace is always allocated");
+        LValue allocator = allocatorForSize(*subspace, size, slowPath);
         return allocateCell(allocator, structure, slowPath);
     }
     
     LValue allocateObject(RegisteredStructure structure)
     {
         size_t allocationSize = JSFinalObject::allocationSize(structure.get()->inlineCapacity());
-        Allocator allocator = subspaceFor<JSFinalObject>(vm())->allocatorForNonVirtual(allocationSize, AllocatorForMode::AllocatorIfExists);
+        Allocator allocator = allocatorForNonVirtualConcurrently<JSFinalObject>(vm(), allocationSize, AllocatorForMode::AllocatorIfExists);
         
         // FIXME: If the allocator is null, we could simply emit a normal C call to the allocator
         // instead of putting it on the slow path.
index dabc7a9..9fb6f22 100644 (file)
@@ -27,6 +27,7 @@
 #include "AlignedMemoryAllocator.h"
 
 #include "BlockDirectory.h"
+#include "Heap.h"
 #include "Subspace.h"
 
 namespace JSC { 
@@ -44,6 +45,7 @@ void AlignedMemoryAllocator::registerDirectory(BlockDirectory* directory)
     RELEASE_ASSERT(!directory->nextDirectoryInAlignedMemoryAllocator());
     
     if (m_directories.isEmpty()) {
+        ASSERT(!mayBeGCThread() || directory->heap()->worldIsStopped());
         for (Subspace* subspace = m_subspaces.first(); subspace; subspace = subspace->nextSubspaceInAlignedMemoryAllocator())
             subspace->didCreateFirstDirectory(directory);
     }
index 06ea3b6..5f2bc83 100644 (file)
@@ -567,18 +567,23 @@ void Heap::finalizeMarkedUnconditionalFinalizers(CellSet& cellSet)
 
 void Heap::finalizeUnconditionalFinalizers()
 {
-    finalizeMarkedUnconditionalFinalizers<InferredValue>(vm()->inferredValuesWithFinalizers);
+    if (vm()->m_inferredValueSpace)
+        finalizeMarkedUnconditionalFinalizers<InferredValue>(vm()->m_inferredValueSpace->space);
     vm()->forEachCodeBlockSpace(
         [&] (auto& space) {
-            this->finalizeMarkedUnconditionalFinalizers<CodeBlock>(space.finalizerSet);
+            this->finalizeMarkedUnconditionalFinalizers<CodeBlock>(space.set);
         });
     finalizeMarkedUnconditionalFinalizers<ExecutableToCodeBlockEdge>(vm()->executableToCodeBlockEdgesWithFinalizers);
-    finalizeMarkedUnconditionalFinalizers<JSWeakSet>(vm()->weakSetSpace);
-    finalizeMarkedUnconditionalFinalizers<JSWeakMap>(vm()->weakMapSpace);
-    finalizeMarkedUnconditionalFinalizers<ErrorInstance>(vm()->errorInstanceSpace);
+    if (vm()->m_weakSetSpace)
+        finalizeMarkedUnconditionalFinalizers<JSWeakSet>(*vm()->m_weakSetSpace);
+    if (vm()->m_weakMapSpace)
+        finalizeMarkedUnconditionalFinalizers<JSWeakMap>(*vm()->m_weakMapSpace);
+    if (vm()->m_errorInstanceSpace)
+        finalizeMarkedUnconditionalFinalizers<ErrorInstance>(*vm()->m_errorInstanceSpace);
 
 #if ENABLE(WEBASSEMBLY)
-    finalizeMarkedUnconditionalFinalizers<JSWebAssemblyCodeBlock>(vm()->webAssemblyCodeBlockSpace);
+    if (vm()->m_webAssemblyCodeBlockSpace)
+        finalizeMarkedUnconditionalFinalizers<JSWebAssemblyCodeBlock>(*vm()->m_webAssemblyCodeBlockSpace);
 #endif
 }
 
@@ -880,11 +885,11 @@ void Heap::deleteAllCodeBlocks(DeleteAllCodeEffort effort)
     vm.forEachScriptExecutableSpace(
         [&] (auto& spaceAndSet) {
             HeapIterationScope heapIterationScope(*this);
-            auto& clearableCodeSet = spaceAndSet.clearableCodeSet;
-            clearableCodeSet.forEachLiveCell(
+            auto& set = spaceAndSet.set;
+            set.forEachLiveCell(
                 [&] (HeapCell* cell, HeapCell::Kind) {
                     ScriptExecutable* executable = static_cast<ScriptExecutable*>(cell);
-                    executable->clearCode(clearableCodeSet);
+                    executable->clearCode(set);
                 });
         });
 
@@ -896,11 +901,13 @@ void Heap::deleteAllCodeBlocks(DeleteAllCodeEffort effort)
         // points into a CodeBlock that could be dead. The IC will still succeed because
         // it uses a callee check, but then it will call into dead code.
         HeapIterationScope heapIterationScope(*this);
-        vm.webAssemblyCodeBlockSpace.forEachLiveCell([&] (HeapCell* cell, HeapCell::Kind kind) {
-            ASSERT_UNUSED(kind, kind == HeapCell::JSCell);
-            JSWebAssemblyCodeBlock* codeBlock = static_cast<JSWebAssemblyCodeBlock*>(cell);
-            codeBlock->clearJSCallICs(vm);
-        });
+        if (vm.m_webAssemblyCodeBlockSpace) {
+            vm.m_webAssemblyCodeBlockSpace->forEachLiveCell([&] (HeapCell* cell, HeapCell::Kind kind) {
+                ASSERT_UNUSED(kind, kind == HeapCell::JSCell);
+                JSWebAssemblyCodeBlock* codeBlock = static_cast<JSWebAssemblyCodeBlock*>(cell);
+                codeBlock->clearJSCallICs(vm);
+            });
+        }
     }
 #endif
 }
@@ -916,7 +923,7 @@ void Heap::deleteAllUnlinkedCodeBlocks(DeleteAllCodeEffort effort)
     RELEASE_ASSERT(!m_collectionScope);
 
     HeapIterationScope heapIterationScope(*this);
-    vm.unlinkedFunctionExecutableSpace.clearableCodeSet.forEachLiveCell(
+    vm.unlinkedFunctionExecutableSpace.set.forEachLiveCell(
         [&] (HeapCell* cell, HeapCell::Kind) {
             UnlinkedFunctionExecutable* executable = static_cast<UnlinkedFunctionExecutable*>(cell);
             executable->clearCode(vm);
@@ -2730,7 +2737,8 @@ void Heap::addCoreConstraints()
             };
             
             add(vm.executableToCodeBlockEdgesWithConstraints);
-            add(vm.weakMapSpace);
+            if (vm.m_weakMapSpace)
+                add(*vm.m_weakMapSpace);
         },
         ConstraintVolatility::GreyedByMarking,
         ConstraintParallelism::Parallel);
index ec89a02..16be50f 100644 (file)
@@ -258,6 +258,7 @@ void MarkedSpace::sweepLargeAllocations()
 
 void MarkedSpace::prepareForAllocation()
 {
+    ASSERT(!mayBeGCThread() || m_heap->worldIsStopped());
     for (Subspace* subspace : m_subspaces)
         subspace->prepareForAllocation();
 
index b87052c..de40617 100644 (file)
@@ -50,7 +50,6 @@ void Subspace::initialize(HeapCellType* heapCellType, AlignedMemoryAllocator* al
     m_directoryForEmptyAllocation = m_alignedMemoryAllocator->firstDirectory();
 
     Heap& heap = *m_space.heap();
-    PreventCollectionScope preventCollectionScope(heap);
     heap.objectSpace().m_subspaces.append(this);
     m_alignedMemoryAllocator->registerSubspace(this);
 }
index 283bb03..4b90f3a 100644 (file)
@@ -1752,7 +1752,7 @@ public:
         VM& vm, GPRReg resultGPR, StructureType structure, StorageType storage, GPRReg scratchGPR1,
         GPRReg scratchGPR2, JumpList& slowPath, size_t size)
     {
-        Allocator allocator = subspaceFor<ClassType>(vm)->allocatorForNonVirtual(size, AllocatorForMode::AllocatorIfExists);
+        Allocator allocator = allocatorForNonVirtualConcurrently<ClassType>(vm, size, AllocatorForMode::AllocatorIfExists);
         emitAllocateJSObject(resultGPR, JITAllocator::constant(allocator), scratchGPR1, structure, storage, scratchGPR2, slowPath);
     }
     
@@ -1769,8 +1769,9 @@ public:
     template<typename ClassType, typename StructureType>
     void emitAllocateVariableSizedCell(VM& vm, GPRReg resultGPR, StructureType structure, GPRReg allocationSize, GPRReg scratchGPR1, GPRReg scratchGPR2, JumpList& slowPath)
     {
-        CompleteSubspace& subspace = *subspaceFor<ClassType>(vm);
-        emitAllocateVariableSized(resultGPR, subspace, allocationSize, scratchGPR1, scratchGPR2, slowPath);
+        CompleteSubspace* subspace = subspaceForConcurrently<ClassType>(vm);
+        RELEASE_ASSERT_WITH_MESSAGE(subspace, "CompleteSubspace is always allocated");
+        emitAllocateVariableSized(resultGPR, *subspace, allocationSize, scratchGPR1, scratchGPR2, slowPath);
         emitStoreStructureWithTypeInfo(structure, resultGPR, scratchGPR2);
     }
 
index 378dc50..bd24958 100644 (file)
@@ -96,7 +96,7 @@ void JIT::emit_op_new_object(const Instruction* currentInstruction)
     auto& metadata = bytecode.metadata(m_codeBlock);
     Structure* structure = metadata.m_objectAllocationProfile.structure();
     size_t allocationSize = JSFinalObject::allocationSize(structure->inlineCapacity());
-    Allocator allocator = subspaceFor<JSFinalObject>(*m_vm)->allocatorForNonVirtual(allocationSize, AllocatorForMode::AllocatorIfExists);
+    Allocator allocator = allocatorForNonVirtualConcurrently<JSFinalObject>(*m_vm, allocationSize, AllocatorForMode::AllocatorIfExists);
 
     RegisterID resultReg = regT0;
     RegisterID allocatorReg = regT1;
index f36cd43..0816da4 100644 (file)
@@ -84,7 +84,7 @@ void JIT::emit_op_new_object(const Instruction* currentInstruction)
     auto& metadata = bytecode.metadata(m_codeBlock);
     Structure* structure = metadata.m_objectAllocationProfile.structure();
     size_t allocationSize = JSFinalObject::allocationSize(structure->inlineCapacity());
-    Allocator allocator = subspaceFor<JSFinalObject>(*m_vm)->allocatorForNonVirtual(allocationSize, AllocatorForMode::AllocatorIfExists);
+    Allocator allocator = allocatorForNonVirtualConcurrently<JSFinalObject>(*m_vm, allocationSize, AllocatorForMode::AllocatorIfExists);
 
     RegisterID resultReg = returnValueGPR;
     RegisterID allocatorReg = regT1;
index 2a78fd0..32cdd0b 100644 (file)
@@ -46,10 +46,10 @@ private:
     DirectArguments(VM&, Structure*, unsigned length, unsigned capacity);
     
 public:
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess>
     static CompleteSubspace* subspaceFor(VM& vm)
     {
-        RELEASE_ASSERT(!CellType::needsDestruction);
+        static_assert(!CellType::needsDestruction, "");
         return &vm.jsValueGigacageCellSpace;
     }
 
index 3206708..63e00f4 100644 (file)
@@ -31,10 +31,10 @@ namespace JSC {
 
 class DirectEvalExecutable final : public EvalExecutable {
 public:
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess mode>
     static IsoSubspace* subspaceFor(VM& vm)
     {
-        return &vm.directEvalExecutableSpace.space;
+        return vm.directEvalExecutableSpace<mode>();
     }
 
     static DirectEvalExecutable* create(ExecState*, const SourceCode&, bool isInStrictContext, DerivedContextType, bool isArrowFunctionContext, EvalContextType, const VariableEnvironment*);
index d95cbf8..fbc6f25 100644 (file)
@@ -72,10 +72,10 @@ public:
     bool materializeErrorInfoIfNeeded(VM&);
     bool materializeErrorInfoIfNeeded(VM&, PropertyName);
 
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess mode>
     static IsoSubspace* subspaceFor(VM& vm)
     {
-        return &vm.errorInstanceSpace;
+        return vm.errorInstanceSpace<mode>();
     }
 
     void finalizeUnconditionally(VM&);
index 296da1b..4ba9896 100644 (file)
@@ -84,7 +84,7 @@ public:
     static void destroy(JSCell*);
     
     // Force subclasses to override this.
-    template<typename>
+    template<typename, SubspaceAccess>
     static void subspaceFor(VM&) { }
         
     CodeBlockHash hashFor(CodeSpecializationKind) const;
index 013696a..0e18829 100644 (file)
@@ -40,7 +40,7 @@ public:
     typedef ScriptExecutable Base;
     static const unsigned StructureFlags = Base::StructureFlags | StructureIsImmortal;
 
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess>
     static IsoSubspace* subspaceFor(VM& vm)
     {
         return &vm.functionExecutableSpace.space;
index d868d93..511b447 100644 (file)
@@ -31,10 +31,10 @@ namespace JSC {
 
 class IndirectEvalExecutable final : public EvalExecutable {
 public:
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess mode>
     static IsoSubspace* subspaceFor(VM& vm)
     {
-        return &vm.indirectEvalExecutableSpace.space;
+        return vm.indirectEvalExecutableSpace<mode>();
     }
 
     static IndirectEvalExecutable* create(ExecState*, const SourceCode&, bool isInStrictContext, DerivedContextType, bool isArrowFunctionContext, EvalContextType);
index 989479e..c90f55d 100644 (file)
@@ -63,7 +63,7 @@ void InferredValue::visitChildren(JSCell* cell, SlotVisitor& visitor)
     if (!value.isCell())
         return;
     
-    visitor.vm().inferredValuesWithFinalizers.add(inferredValue);
+    VM::SpaceAndSet::setFor(*inferredValue->subspace()).add(inferredValue);
 }
 
 InferredValue::InferredValue(VM& vm)
index 08028ef..da63e01 100644 (file)
@@ -45,10 +45,10 @@ class InferredValue final : public JSCell {
 public:
     typedef JSCell Base;
     
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess mode>
     static IsoSubspace* subspaceFor(VM& vm)
     {
-        return &vm.inferredValueSpace;
+        return vm.inferredValueSpace<mode>();
     }
 
     static InferredValue* create(VM&);
index 9694b6d..95114e4 100644 (file)
@@ -40,7 +40,7 @@ void InferredValue::finalizeUnconditionally(VM& vm)
         invalidate(vm, StringFireDetail("InferredValue clean-up during GC"));
     }
     
-    vm.inferredValuesWithFinalizers.remove(this);
+    VM::SpaceAndSet::setFor(*subspace()).remove(this);
 }
 
 } // namespace JSC
index 244c4f7..634d9dd 100644 (file)
@@ -38,7 +38,7 @@ public:
     typedef JSDestructibleObject Base;
     static const unsigned StructureFlags = Base::StructureFlags | ImplementsHasInstance | ImplementsDefaultHasInstance | OverridesGetCallData;
 
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess>
     static IsoSubspace* subspaceFor(VM& vm)
     {
         static_assert(sizeof(CellType) == sizeof(InternalFunction), "InternalFunction subclasses that add fields need to override subspaceFor<>()");
index 9b98ea2..ba80bab 100644 (file)
@@ -38,7 +38,7 @@ public:
 
     const static unsigned StructureFlags = Base::StructureFlags;
 
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess>
     static IsoSubspace* subspaceFor(VM& vm)
     {
         return &vm.functionSpace;
index 06c072b..70239b8 100644 (file)
@@ -38,7 +38,7 @@ public:
 
     const static unsigned StructureFlags = Base::StructureFlags;
 
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess>
     static IsoSubspace* subspaceFor(VM& vm)
     {
         return &vm.functionSpace;
index 928ddc5..5814886 100644 (file)
@@ -42,10 +42,10 @@ public:
     const static unsigned StructureFlags = Base::StructureFlags & ~ImplementsDefaultHasInstance;
     static_assert(StructureFlags & ImplementsHasInstance, "");
 
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess mode>
     static IsoSubspace* subspaceFor(VM& vm)
     {
-        return &vm.boundFunctionSpace;
+        return vm.boundFunctionSpace<mode>();
     }
 
     static JSBoundFunction* create(VM&, ExecState*, JSGlobalObject*, JSObject* targetFunction, JSValue boundThis, JSArray* boundArgs, int, const String& name);
index 8cdb873..0f38604 100644 (file)
@@ -32,6 +32,7 @@
 #include "JSLock.h"
 #include "JSTypeInfo.h"
 #include "SlotVisitor.h"
+#include "SubspaceAccess.h"
 #include "TypedArrayType.h"
 #include "WriteBarrier.h"
 
@@ -88,7 +89,7 @@ public:
     // Don't call this directly. Call JSC::subspaceFor<Type>(vm) instead.
     // FIXME: Refer to Subspace by reference.
     // https://bugs.webkit.org/show_bug.cgi?id=166988
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess>
     static CompleteSubspace* subspaceFor(VM&);
 
     static JSCell* seenMultipleCalleeObjects() { return bitwise_cast<JSCell*>(static_cast<uintptr_t>(1)); }
@@ -294,7 +295,13 @@ private:
 template<typename Type>
 inline auto subspaceFor(VM& vm)
 {
-    return Type::template subspaceFor<Type>(vm);
+    return Type::template subspaceFor<Type, SubspaceAccess::OnMainThread>(vm);
+}
+
+template<typename Type>
+inline auto subspaceForConcurrently(VM& vm)
+{
+    return Type::template subspaceFor<Type, SubspaceAccess::Concurrently>(vm);
 }
 
 } // namespace JSC
index f4abba2..332faba 100644 (file)
@@ -25,6 +25,7 @@
 
 #pragma once
 
+#include "AllocatorForMode.h"
 #include "AllocatorInlines.h"
 #include "CompleteSubspaceInlines.h"
 #include "CPU.h"
@@ -145,7 +146,7 @@ ALWAYS_INLINE VM& ExecState::vm() const
     return *callee->markedBlock().vm();
 }
 
-template<typename CellType>
+template<typename CellType, SubspaceAccess>
 CompleteSubspace* JSCell::subspaceFor(VM& vm)
 {
     if (CellType::needsDestruction)
@@ -153,6 +154,14 @@ CompleteSubspace* JSCell::subspaceFor(VM& vm)
     return &vm.cellSpace;
 }
 
+template<typename Type>
+inline Allocator allocatorForNonVirtualConcurrently(VM& vm, size_t allocationSize, AllocatorForMode mode)
+{
+    if (auto* subspace = subspaceForConcurrently<Type>(vm))
+        return subspace->allocatorForNonVirtual(allocationSize, mode);
+    return { };
+}
+
 template<typename T>
 ALWAYS_INLINE void* tryAllocateCellHelper(Heap& heap, size_t size, GCDeferralContext* deferralContext, AllocationFailureMode failureMode)
 {
index 17c7f18..dfac65c 100644 (file)
@@ -40,10 +40,10 @@ public:
 
     static const unsigned StructureFlags = Base::StructureFlags;
 
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess mode>
     static IsoSubspace* subspaceFor(VM& vm)
     {
-        return &vm.customGetterSetterFunctionSpace;
+        return vm.customGetterSetterFunctionSpace<mode>();
     }
 
     static Structure* createStructure(VM& vm, JSGlobalObject* globalObject, JSValue prototype)
index 43828f0..d2617b7 100644 (file)
@@ -37,7 +37,7 @@ public:
 
     static const bool needsDestruction = true;
     
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess>
     static CompleteSubspace* subspaceFor(VM& vm)
     {
         return &vm.destructibleObjectSpace;
index 1e77632..3bf0676 100644 (file)
@@ -62,7 +62,7 @@ class JSFunction : public JSCallee {
 
 public:
     
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess>
     static IsoSubspace* subspaceFor(VM& vm)
     {
         return &vm.functionSpace;
index ee5ac42..99781c5 100644 (file)
@@ -66,7 +66,7 @@ public:
 
     const static unsigned StructureFlags = Base::StructureFlags;
 
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess>
     static IsoSubspace* subspaceFor(VM& vm)
     {
         return &vm.functionSpace;
index 4ac2a6c..b554c5f 100644 (file)
@@ -88,7 +88,7 @@ public:
 
     void copyToArguments(ExecState*, VirtualRegister firstElementDest, unsigned offset, unsigned length);
 
-    template<typename>
+    template<typename, SubspaceAccess>
     static CompleteSubspace* subspaceFor(VM& vm)
     {
         // We allocate out of the JSValue gigacage as other code expects all butterflies to live there.
index cfec956..3b67629 100644 (file)
@@ -40,10 +40,10 @@ class JSLexicalEnvironment : public JSSymbolTableObject {
     friend class JIT;
     friend class LLIntOffsetsExtractor;
 public:
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess>
     static CompleteSubspace* subspaceFor(VM& vm)
     {
-        RELEASE_ASSERT(!CellType::needsDestruction);
+        static_assert(!CellType::needsDestruction, "");
         return &vm.jsValueGigacageCellSpace;
     }
 
index e204149..fab9d6e 100644 (file)
@@ -40,10 +40,10 @@ public:
 
     const static unsigned StructureFlags = Base::StructureFlags;
 
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess mode>
     static IsoSubspace* subspaceFor(VM& vm)
     {
-        return &vm.nativeStdFunctionSpace;
+        return vm.nativeStdFunctionSpace<mode>();
     }
 
     DECLARE_EXPORT_INFO;
index cb5fe37..deef4e5 100644 (file)
@@ -90,7 +90,7 @@ public:
     
     static void destroy(JSCell*);
     
-    template<typename>
+    template<typename, SubspaceAccess>
     static CompleteSubspace* subspaceFor(VM& vm)
     {
         return &vm.segmentedVariableObjectSpace;
index 57d7f90..8b4d209 100644 (file)
@@ -89,7 +89,7 @@ public:
     
     // We specialize the string subspace to get the fastest possible sweep. This wouldn't be
     // necessary if JSString didn't have a destructor.
-    template<typename>
+    template<typename, SubspaceAccess>
     static CompleteSubspace* subspaceFor(VM& vm)
     {
         return &vm.stringSpace;
index f5171d4..ff82c28 100644 (file)
@@ -36,10 +36,10 @@ public:
     typedef ScriptExecutable Base;
     static const unsigned StructureFlags = Base::StructureFlags | StructureIsImmortal;
 
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess mode>
     static IsoSubspace* subspaceFor(VM& vm)
     {
-        return &vm.moduleProgramExecutableSpace.space;
+        return vm.moduleProgramExecutableSpace<mode>();
     }
 
     static ModuleProgramExecutable* create(ExecState*, const SourceCode&);
index 637be6f..91fb35b 100644 (file)
@@ -44,7 +44,7 @@ public:
 
     static void destroy(JSCell*);
     
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess>
     static IsoSubspace* subspaceFor(VM& vm)
     {
         return &vm.nativeExecutableSpace;
index ff84c9e..a1ecd21 100644 (file)
@@ -36,7 +36,7 @@ public:
     typedef ScriptExecutable Base;
     static const unsigned StructureFlags = Base::StructureFlags | StructureIsImmortal;
 
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess>
     static IsoSubspace* subspaceFor(VM& vm)
     {
         return &vm.programExecutableSpace.space;
index 27edadc..70e9a71 100644 (file)
@@ -123,7 +123,7 @@ public:
     typedef JSCell Base;
     static const unsigned StructureFlags = Base::StructureFlags | StructureIsImmortal;
 
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess>
     static IsoSubspace* subspaceFor(VM& vm)
     {
         return &vm.propertyTableSpace;
index 3220965..64bfe3a 100644 (file)
@@ -36,10 +36,10 @@ public:
     typedef InternalFunction Base;
     static const unsigned StructureFlags = Base::StructureFlags;
 
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess mode>
     static IsoSubspace* subspaceFor(VM& vm)
     {
-        return &vm.proxyRevokeSpace;
+        return vm.proxyRevokeSpace<mode>();
     }
 
     static ProxyRevoke* create(VM&, Structure*, ProxyObject*);
index a9c0b24..f36db86 100644 (file)
@@ -43,10 +43,10 @@ private:
     using Base = GenericArguments<ScopedArguments>;
 
 public:
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess>
     static CompleteSubspace* subspaceFor(VM& vm)
     {
-        RELEASE_ASSERT(!CellType::needsDestruction);
+        static_assert(!CellType::needsDestruction, "");
         return &vm.jsValueGigacageCellSpace;
     }
 
index 891c3a1..3251a79 100644 (file)
@@ -68,7 +68,7 @@ void ScriptExecutable::destroy(JSCell* cell)
 void ScriptExecutable::clearCode(IsoCellSet& clearableCodeSet)
 {
     Base::clearCode();
-    ASSERT(&VM::ScriptExecutableSpaceAndSet::clearableCodeSetFor(*subspace()) == &clearableCodeSet);
+    ASSERT(&VM::SpaceAndSet::setFor(*subspace()) == &clearableCodeSet);
     clearableCodeSet.remove(this);
 }
 
@@ -149,7 +149,7 @@ void ScriptExecutable::installCode(VM& vm, CodeBlock* genericCodeBlock, CodeType
         break;
     }
 
-    auto& clearableCodeSet = VM::ScriptExecutableSpaceAndSet::clearableCodeSetFor(*subspace());
+    auto& clearableCodeSet = VM::SpaceAndSet::setFor(*subspace());
     if (hasClearableCode())
         clearableCodeSet.add(this);
     else
index e11a853..3682693 100644 (file)
@@ -132,7 +132,7 @@ public:
 
     ~Structure();
     
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess>
     static IsoSubspace* subspaceFor(VM& vm)
     {
         return &vm.structureSpace;
index 336732d..c4be714 100644 (file)
@@ -43,7 +43,7 @@ public:
     typedef JSCell Base;
     static const unsigned StructureFlags = Base::StructureFlags | StructureIsImmortal;
 
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess>
     static IsoSubspace* subspaceFor(VM& vm)
     {
         return &vm.structureRareDataSpace;
diff --git a/Source/JavaScriptCore/runtime/SubspaceAccess.h b/Source/JavaScriptCore/runtime/SubspaceAccess.h
new file mode 100644 (file)
index 0000000..66e71d8
--- /dev/null
@@ -0,0 +1,35 @@
+/*
+ * Copyright (C) 2019 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+namespace JSC {
+
+enum class SubspaceAccess {
+    OnMainThread,
+    Concurrently,
+};
+
+}
index 30afc6a..4ef7baa 100644 (file)
@@ -287,38 +287,17 @@ VM::VM(VMType vmType, HeapType heapType)
     , destructibleObjectSpace("JSDestructibleObject", heap, destructibleObjectHeapCellType.get(), fastMallocAllocator.get())
     , eagerlySweptDestructibleObjectSpace("Eagerly Swept JSDestructibleObject", heap, destructibleObjectHeapCellType.get(), fastMallocAllocator.get())
     , segmentedVariableObjectSpace("JSSegmentedVariableObjectSpace", heap, segmentedVariableObjectHeapCellType.get(), fastMallocAllocator.get())
-    , boundFunctionSpace ISO_SUBSPACE_INIT(heap, cellHeapCellType.get(), JSBoundFunction)
-    , callbackFunctionSpace ISO_SUBSPACE_INIT(heap, destructibleObjectHeapCellType.get(), JSCallbackFunction)
-    , customGetterSetterFunctionSpace ISO_SUBSPACE_INIT(heap, cellHeapCellType.get(), JSCustomGetterSetterFunction)
     , executableToCodeBlockEdgeSpace ISO_SUBSPACE_INIT(heap, cellHeapCellType.get(), ExecutableToCodeBlockEdge)
     , functionSpace ISO_SUBSPACE_INIT(heap, cellHeapCellType.get(), JSFunction)
-    , inferredValueSpace ISO_SUBSPACE_INIT(heap, destructibleCellHeapCellType.get(), InferredValue)
     , internalFunctionSpace ISO_SUBSPACE_INIT(heap, destructibleObjectHeapCellType.get(), InternalFunction)
     , nativeExecutableSpace ISO_SUBSPACE_INIT(heap, destructibleCellHeapCellType.get(), NativeExecutable)
-    , nativeStdFunctionSpace ISO_SUBSPACE_INIT(heap, cellHeapCellType.get(), JSNativeStdFunction)
-#if JSC_OBJC_API_ENABLED
-    , objCCallbackFunctionSpace ISO_SUBSPACE_INIT(heap, destructibleObjectHeapCellType.get(), ObjCCallbackFunction)
-#endif
     , propertyTableSpace ISO_SUBSPACE_INIT(heap, destructibleCellHeapCellType.get(), PropertyTable)
-    , proxyRevokeSpace ISO_SUBSPACE_INIT(heap, destructibleObjectHeapCellType.get(), ProxyRevoke)
     , structureRareDataSpace ISO_SUBSPACE_INIT(heap, destructibleCellHeapCellType.get(), StructureRareData)
     , structureSpace ISO_SUBSPACE_INIT(heap, destructibleCellHeapCellType.get(), Structure)
-    , weakSetSpace ISO_SUBSPACE_INIT(heap, destructibleObjectHeapCellType.get(), JSWeakSet)
-    , weakMapSpace ISO_SUBSPACE_INIT(heap, destructibleObjectHeapCellType.get(), JSWeakMap)
-    , errorInstanceSpace ISO_SUBSPACE_INIT(heap, destructibleObjectHeapCellType.get(), ErrorInstance)
-#if ENABLE(WEBASSEMBLY)
-    , webAssemblyCodeBlockSpace ISO_SUBSPACE_INIT(heap, webAssemblyCodeBlockHeapCellType.get(), JSWebAssemblyCodeBlock)
-    , webAssemblyFunctionSpace ISO_SUBSPACE_INIT(heap, cellHeapCellType.get(), WebAssemblyFunction)
-    , webAssemblyWrapperFunctionSpace ISO_SUBSPACE_INIT(heap, cellHeapCellType.get(), WebAssemblyWrapperFunction)
-#endif
     , executableToCodeBlockEdgesWithConstraints(executableToCodeBlockEdgeSpace)
     , executableToCodeBlockEdgesWithFinalizers(executableToCodeBlockEdgeSpace)
-    , inferredValuesWithFinalizers(inferredValueSpace)
     , codeBlockSpace ISO_SUBSPACE_INIT(heap, destructibleCellHeapCellType.get(), CodeBlock)
-    , directEvalExecutableSpace ISO_SUBSPACE_INIT(heap, destructibleCellHeapCellType.get(), DirectEvalExecutable)
     , functionExecutableSpace ISO_SUBSPACE_INIT(heap, destructibleCellHeapCellType.get(), FunctionExecutable)
-    , indirectEvalExecutableSpace ISO_SUBSPACE_INIT(heap, destructibleCellHeapCellType.get(), IndirectEvalExecutable)
-    , moduleProgramExecutableSpace ISO_SUBSPACE_INIT(heap, destructibleCellHeapCellType.get(), ModuleProgramExecutable)
     , programExecutableSpace ISO_SUBSPACE_INIT(heap, destructibleCellHeapCellType.get(), ProgramExecutable)
     , unlinkedFunctionExecutableSpace ISO_SUBSPACE_INIT(heap, destructibleCellHeapCellType.get(), UnlinkedFunctionExecutable)
     , vmType(vmType)
@@ -1239,6 +1218,53 @@ void VM::ensureShadowChicken()
     m_shadowChicken = std::make_unique<ShadowChicken>();
 }
 
+#define DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER_SLOW(name, heapCellType, type) \
+    IsoSubspace* VM::name##Slow() \
+    { \
+        ASSERT(!m_##name); \
+        auto space = std::make_unique<IsoSubspace> ISO_SUBSPACE_INIT(heap, heapCellType, type); \
+        WTF::storeStoreFence(); \
+        m_##name = WTFMove(space); \
+        return m_##name.get(); \
+    }
+
+
+DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER_SLOW(boundFunctionSpace, cellHeapCellType.get(), JSBoundFunction)
+DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER_SLOW(callbackFunctionSpace, destructibleObjectHeapCellType.get(), JSCallbackFunction)
+DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER_SLOW(customGetterSetterFunctionSpace, cellHeapCellType.get(), JSCustomGetterSetterFunction)
+DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER_SLOW(errorInstanceSpace, destructibleObjectHeapCellType.get(), ErrorInstance)
+DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER_SLOW(nativeStdFunctionSpace, cellHeapCellType.get(), JSNativeStdFunction)
+DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER_SLOW(proxyRevokeSpace, destructibleObjectHeapCellType.get(), ProxyRevoke)
+DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER_SLOW(weakMapSpace, destructibleObjectHeapCellType.get(), JSWeakMap)
+DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER_SLOW(weakSetSpace, destructibleObjectHeapCellType.get(), JSWeakSet)
+#if JSC_OBJC_API_ENABLED
+DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER_SLOW(objCCallbackFunctionSpace, destructibleObjectHeapCellType.get(), ObjCCallbackFunction)
+#endif
+#if ENABLE(WEBASSEMBLY)
+DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER_SLOW(webAssemblyCodeBlockSpace, webAssemblyCodeBlockHeapCellType.get(), JSWebAssemblyCodeBlock)
+DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER_SLOW(webAssemblyFunctionSpace, cellHeapCellType.get(), WebAssemblyFunction)
+DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER_SLOW(webAssemblyWrapperFunctionSpace, cellHeapCellType.get(), WebAssemblyWrapperFunction)
+#endif
+
+#undef DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER_SLOW
+
+#define DYNAMIC_SPACE_AND_SET_DEFINE_MEMBER_SLOW(name, heapCellType, type) \
+    IsoSubspace* VM::name##Slow() \
+    { \
+        ASSERT(!m_##name); \
+        auto space = std::make_unique<SpaceAndSet> ISO_SUBSPACE_INIT(heap, heapCellType, type); \
+        WTF::storeStoreFence(); \
+        m_##name = WTFMove(space); \
+        return &m_##name->space; \
+    }
+
+DYNAMIC_SPACE_AND_SET_DEFINE_MEMBER_SLOW(inferredValueSpace, destructibleCellHeapCellType.get(), InferredValue)
+DYNAMIC_SPACE_AND_SET_DEFINE_MEMBER_SLOW(directEvalExecutableSpace, destructibleCellHeapCellType.get(), DirectEvalExecutable)
+DYNAMIC_SPACE_AND_SET_DEFINE_MEMBER_SLOW(indirectEvalExecutableSpace, destructibleCellHeapCellType.get(), IndirectEvalExecutable)
+DYNAMIC_SPACE_AND_SET_DEFINE_MEMBER_SLOW(moduleProgramExecutableSpace, destructibleCellHeapCellType.get(), ModuleProgramExecutable)
+
+#undef DYNAMIC_SPACE_AND_SET_DEFINE_MEMBER_SLOW
+
 JSGlobalObject* VM::vmEntryGlobalObject(const CallFrame* callFrame) const
 {
     if (callFrame && callFrame->isGlobalExec()) {
index 8abaa20..2ea8207 100644 (file)
@@ -51,6 +51,7 @@
 #include "SmallStrings.h"
 #include "Strong.h"
 #include "StructureCache.h"
+#include "SubspaceAccess.h"
 #include "VMTraps.h"
 #include "WasmContext.h"
 #include "Watchpoint.h"
@@ -366,56 +367,85 @@ public:
     CompleteSubspace eagerlySweptDestructibleObjectSpace;
     CompleteSubspace segmentedVariableObjectSpace;
     
-    IsoSubspace boundFunctionSpace;
-    IsoSubspace callbackFunctionSpace;
-    IsoSubspace customGetterSetterFunctionSpace;
     IsoSubspace executableToCodeBlockEdgeSpace;
     IsoSubspace functionSpace;
-    IsoSubspace inferredValueSpace;
     IsoSubspace internalFunctionSpace;
     IsoSubspace nativeExecutableSpace;
-    IsoSubspace nativeStdFunctionSpace;
-#if JSC_OBJC_API_ENABLED
-    IsoSubspace objCCallbackFunctionSpace;
-#endif
     IsoSubspace propertyTableSpace;
-    IsoSubspace proxyRevokeSpace;
     IsoSubspace structureRareDataSpace;
     IsoSubspace structureSpace;
-    IsoSubspace weakSetSpace;
-    IsoSubspace weakMapSpace;
-    IsoSubspace errorInstanceSpace;
+
+#define DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER(name) \
+    template<SubspaceAccess mode> \
+    IsoSubspace* name() \
+    { \
+        if (m_##name || mode == SubspaceAccess::Concurrently) \
+            return m_##name.get(); \
+        return name##Slow(); \
+    } \
+    IsoSubspace* name##Slow(); \
+    std::unique_ptr<IsoSubspace> m_##name;
+
+
+#if JSC_OBJC_API_ENABLED
+    DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER(objCCallbackFunctionSpace)
+#endif
+    DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER(boundFunctionSpace)
+    DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER(callbackFunctionSpace)
+    DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER(customGetterSetterFunctionSpace)
+    DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER(errorInstanceSpace)
+    DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER(nativeStdFunctionSpace)
+    DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER(proxyRevokeSpace)
+    DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER(weakSetSpace)
+    DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER(weakMapSpace)
 #if ENABLE(WEBASSEMBLY)
-    IsoSubspace webAssemblyCodeBlockSpace;
-    IsoSubspace webAssemblyFunctionSpace;
-    IsoSubspace webAssemblyWrapperFunctionSpace;
+    DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER(webAssemblyCodeBlockSpace)
+    DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER(webAssemblyFunctionSpace)
+    DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER(webAssemblyWrapperFunctionSpace)
 #endif
+
+#undef DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER
     
     IsoCellSet executableToCodeBlockEdgesWithConstraints;
     IsoCellSet executableToCodeBlockEdgesWithFinalizers;
-    IsoCellSet inferredValuesWithFinalizers;
+
+#define DYNAMIC_SPACE_AND_SET_DEFINE_MEMBER(name) \
+    template<SubspaceAccess mode> \
+    IsoSubspace* name() \
+    { \
+        if (auto* spaceAndSet = m_##name.get()) \
+            return &spaceAndSet->space; \
+        if (mode == SubspaceAccess::Concurrently) \
+            return nullptr; \
+        return name##Slow(); \
+    } \
+    IsoSubspace* name##Slow(); \
+    std::unique_ptr<SpaceAndSet> m_##name;
     
-    struct SpaceAndFinalizerSet {
+    struct SpaceAndSet {
+        WTF_MAKE_STRUCT_FAST_ALLOCATED;
+
         IsoSubspace space;
-        IsoCellSet finalizerSet;
+        IsoCellSet set;
         
         template<typename... Arguments>
-        SpaceAndFinalizerSet(Arguments&&... arguments)
+        SpaceAndSet(Arguments&&... arguments)
             : space(std::forward<Arguments>(arguments)...)
-            , finalizerSet(space)
+            , set(space)
         {
         }
         
-        static IsoCellSet& finalizerSetFor(Subspace& space)
+        static IsoCellSet& setFor(Subspace& space)
         {
             return *bitwise_cast<IsoCellSet*>(
                 bitwise_cast<char*>(&space) -
-                OBJECT_OFFSETOF(SpaceAndFinalizerSet, space) +
-                OBJECT_OFFSETOF(SpaceAndFinalizerSet, finalizerSet));
+                OBJECT_OFFSETOF(SpaceAndSet, space) +
+                OBJECT_OFFSETOF(SpaceAndSet, set));
         }
     };
     
-    SpaceAndFinalizerSet codeBlockSpace;
+    SpaceAndSet codeBlockSpace;
+    DYNAMIC_SPACE_AND_SET_DEFINE_MEMBER(inferredValueSpace)
 
     template<typename Func>
     void forEachCodeBlockSpace(const Func& func)
@@ -425,61 +455,28 @@ public:
         func(codeBlockSpace);
     }
 
-    struct ScriptExecutableSpaceAndSet {
-        IsoSubspace space;
-        IsoCellSet clearableCodeSet;
-
-        template<typename... Arguments>
-        ScriptExecutableSpaceAndSet(Arguments&&... arguments)
-            : space(std::forward<Arguments>(arguments)...)
-            , clearableCodeSet(space)
-        { }
-
-        static IsoCellSet& clearableCodeSetFor(Subspace& space)
-        {
-            return *bitwise_cast<IsoCellSet*>(
-                bitwise_cast<char*>(&space) -
-                OBJECT_OFFSETOF(ScriptExecutableSpaceAndSet, space) +
-                OBJECT_OFFSETOF(ScriptExecutableSpaceAndSet, clearableCodeSet));
-        }
-    };
-
-    ScriptExecutableSpaceAndSet directEvalExecutableSpace;
-    ScriptExecutableSpaceAndSet functionExecutableSpace;
-    ScriptExecutableSpaceAndSet indirectEvalExecutableSpace;
-    ScriptExecutableSpaceAndSet moduleProgramExecutableSpace;
-    ScriptExecutableSpaceAndSet programExecutableSpace;
+    DYNAMIC_SPACE_AND_SET_DEFINE_MEMBER(directEvalExecutableSpace)
+    DYNAMIC_SPACE_AND_SET_DEFINE_MEMBER(indirectEvalExecutableSpace)
+    DYNAMIC_SPACE_AND_SET_DEFINE_MEMBER(moduleProgramExecutableSpace)
+    SpaceAndSet functionExecutableSpace;
+    SpaceAndSet programExecutableSpace;
 
     template<typename Func>
     void forEachScriptExecutableSpace(const Func& func)
     {
-        func(directEvalExecutableSpace);
+        if (m_directEvalExecutableSpace)
+            func(*m_directEvalExecutableSpace);
         func(functionExecutableSpace);
-        func(indirectEvalExecutableSpace);
-        func(moduleProgramExecutableSpace);
+        if (m_indirectEvalExecutableSpace)
+            func(*m_indirectEvalExecutableSpace);
+        if (m_moduleProgramExecutableSpace)
+            func(*m_moduleProgramExecutableSpace);
         func(programExecutableSpace);
     }
 
-    struct UnlinkedFunctionExecutableSpaceAndSet {
-        IsoSubspace space;
-        IsoCellSet clearableCodeSet;
-
-        template<typename... Arguments>
-        UnlinkedFunctionExecutableSpaceAndSet(Arguments&&... arguments)
-            : space(std::forward<Arguments>(arguments)...)
-            , clearableCodeSet(space)
-        { }
-        
-        static IsoCellSet& clearableCodeSetFor(Subspace& space)
-        {
-            return *bitwise_cast<IsoCellSet*>(
-                bitwise_cast<char*>(&space) -
-                OBJECT_OFFSETOF(UnlinkedFunctionExecutableSpaceAndSet, space) +
-                OBJECT_OFFSETOF(UnlinkedFunctionExecutableSpaceAndSet, clearableCodeSet));
-        }
-    };
+    SpaceAndSet unlinkedFunctionExecutableSpace;
 
-    UnlinkedFunctionExecutableSpaceAndSet unlinkedFunctionExecutableSpace;
+#undef DYNAMIC_SPACE_AND_SET_DEFINE_MEMBER
 
     VMType vmType;
     ClientData* clientData;
index 5258cbc..8b45ffe 100644 (file)
@@ -302,12 +302,12 @@ public:
         return std::is_same<WeakMapBucketType, JSC::WeakMapBucket<WeakMapBucketDataKey>>::value;
     }
 
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess mode>
     static IsoSubspace* subspaceFor(VM& vm)
     {
         if (isWeakMap())
-            return &vm.weakMapSpace;
-        return &vm.weakSetSpace;
+            return vm.weakMapSpace<mode>();
+        return vm.weakSetSpace<mode>();
     }
 
     static void visitOutputConstraints(JSCell*, SlotVisitor&);
index 0eb4aa1..bf1ccac 100644 (file)
@@ -59,10 +59,10 @@ public:
         return Structure::create(vm, globalObject, prototype, TypeInfo(CellType, StructureFlags), info());
     }
 
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess mode>
     static IsoSubspace* subspaceFor(VM& vm)
     {
-        return &vm.webAssemblyCodeBlockSpace;
+        return vm.webAssemblyCodeBlockSpace<mode>();
     }
 
     Wasm::CodeBlock& codeBlock() { return m_codeBlock.get(); }
index b03ae04..a145945 100644 (file)
@@ -43,7 +43,7 @@ class JSWebAssemblyMemory final : public JSDestructibleObject {
 public:
     typedef JSDestructibleObject Base;
 
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess>
     static CompleteSubspace* subspaceFor(VM& vm)
     {
         // We hold onto a lot of memory, so it makes a lot of sense to be swept eagerly.
index 3443900..feadec5 100644 (file)
@@ -49,10 +49,10 @@ public:
 
     const static unsigned StructureFlags = Base::StructureFlags;
 
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess mode>
     static IsoSubspace* subspaceFor(VM& vm)
     {
-        return &vm.webAssemblyFunctionSpace;
+        return vm.webAssemblyFunctionSpace<mode>();
     }
 
     DECLARE_EXPORT_INFO;
index 015fef2..12a55d6 100644 (file)
@@ -40,10 +40,10 @@ public:
 
     const static unsigned StructureFlags = Base::StructureFlags;
 
-    template<typename CellType>
+    template<typename CellType, SubspaceAccess mode>
     static IsoSubspace* subspaceFor(VM& vm)
     {
-        return &vm.webAssemblyWrapperFunctionSpace;
+        return vm.webAssemblyWrapperFunctionSpace<mode>();
     }
 
     DECLARE_INFO;
index 232d548..dc74493 100644 (file)
@@ -1,3 +1,14 @@
+2019-02-04  Yusuke Suzuki  <ysuzuki@apple.com>
+
+        [JSC] Shrink size of VM by lazily allocating IsoSubspaces for non-common types
+        https://bugs.webkit.org/show_bug.cgi?id=193993
+
+        Reviewed by Keith Miller.
+
+        * bindings/scripts/CodeGeneratorJS.pm:
+        (GenerateHeader):
+        * bridge/runtime_method.h:
+
 2019-02-04  Simon Fraser  <simon.fraser@apple.com>
 
         Move some macOS/iOS scrolling code into the scrolling/cocoa directory
index 6af893d..b53514b 100644 (file)
@@ -2758,7 +2758,7 @@ sub GenerateHeader
             # this just calls visitAdditionalChildren, you usually don't have to worry about this.
             push(@headerContent, "    static void visitOutputConstraints(JSCell*, JSC::SlotVisitor&);\n");
             my $subspaceFunc = IsDOMGlobalObject($interface) ? "globalObjectOutputConstraintSubspaceFor" : "outputConstraintSubspaceFor";
-            push(@headerContent, "    template<typename> static JSC::CompleteSubspace* subspaceFor(JSC::VM& vm) { return $subspaceFunc(vm); }\n");
+            push(@headerContent, "    template<typename, JSC::SubspaceAccess> static JSC::CompleteSubspace* subspaceFor(JSC::VM& vm) { return $subspaceFunc(vm); }\n");
         }
     }
 
index 2eefd8e..cdc2394 100644 (file)
@@ -37,7 +37,7 @@ public:
     typedef InternalFunction Base;
     static const unsigned StructureFlags = Base::StructureFlags | OverridesGetOwnPropertySlot | OverridesGetCallData;
 
-    template<typename CellType>
+    template<typename CellType, JSC::SubspaceAccess>
     static IsoSubspace* subspaceFor(VM& vm)
     {
         static_assert(sizeof(CellType) == sizeof(RuntimeMethod), "RuntimeMethod subclasses that add fields need to override subspaceFor<>()");
index 937763a..cdfb385 100644 (file)
@@ -1,3 +1,13 @@
+2019-02-04  Yusuke Suzuki  <ysuzuki@apple.com>
+
+        [JSC] Shrink size of VM by lazily allocating IsoSubspaces for non-common types
+        https://bugs.webkit.org/show_bug.cgi?id=193993
+
+        Reviewed by Keith Miller.
+
+        * WebProcess/Plugins/Netscape/JSNPMethod.h:
+        * WebProcess/Plugins/Netscape/JSNPObject.h:
+
 2019-02-04  Simon Fraser  <simon.fraser@apple.com>
 
         Move some macOS/iOS scrolling code into the scrolling/cocoa directory
index 1864bab..28a72bb 100644 (file)
@@ -41,7 +41,7 @@ class JSNPMethod final : public JSC::InternalFunction {
 public:
     typedef JSC::InternalFunction Base;
 
-    template<typename CellType>
+    template<typename CellType, JSC::SubspaceAccess>
     static JSC::IsoSubspace* subspaceFor(JSC::VM& vm)
     {
         return subspaceForImpl(vm);
index 63db339..6f6747f 100644 (file)
@@ -46,7 +46,7 @@ public:
     typedef JSC::JSDestructibleObject Base;
     static const unsigned StructureFlags = Base::StructureFlags | JSC::OverridesGetOwnPropertySlot | JSC::OverridesGetPropertyNames | JSC::OverridesGetCallData;
 
-    template<typename CellType>
+    template<typename CellType, JSC::SubspaceAccess>
     static JSC::IsoSubspace* subspaceFor(JSC::VM& vm)
     {
         return subspaceForImpl(vm);