Support compiling catch in the FTL
authorsbarati@apple.com <sbarati@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 5 Sep 2017 03:21:33 +0000 (03:21 +0000)
committersbarati@apple.com <sbarati@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 5 Sep 2017 03:21:33 +0000 (03:21 +0000)
https://bugs.webkit.org/show_bug.cgi?id=175396

Reviewed by Filip Pizlo.

This patch implements op_catch in the FTL. It extends the DFG implementation
by supporting multiple entrypoints in DFG-SSA. This patch implements this
by introducing an EntrySwitch node. When converting to SSA, we introduce a new
root block with an EntrySwitch that has the previous DFG entrypoints as its
successors. By convention, we pick the zeroth entry point index to be the
op_enter entrypoint. Like in B3, in DFG-SSA, EntrySwitch just acts like a
switch over the entrypoint index argument. DFG::EntrySwitch in the FTL
simply lowers to B3::EntrySwitch. The EntrySwitch in the root block that
SSAConversion creates can not exit because we would both not know where to exit
to in the program: we would not have valid OSR exit state. This design also
mandates that anything we hoist above EntrySwitch in the new root block
can not exit since they also do not have valid OSR exit state.

This patch also adds a new metadata node named InitializeEntrypointArguments.
InitializeEntrypointArguments is a metadata node that initializes the flush format for
the arguments at a given entrypoint. For a given entrypoint index, this node
tells AI and OSRAvailabilityAnalysis what the flush format for each argument
is. This allows each individual entrypoint to have an independent set of
argument types. Currently, this won't happen in practice because ArgumentPosition
unifies flush formats, but this is an implementation detail we probably want
to modify in the future. SSAConversion will add InitializeEntrypointArguments
to the beginning of each of the original DFG entrypoint blocks.

This patch also adds the ability to specify custom prologue code generators in Air.
This allows the FTL to specify a custom prologue for catch entrypoints that
matches the op_catch OSR entry calling convention that the DFG uses. This way,
the baseline JIT code OSR enters into op_catch the same way both in the DFG
and the FTL. In the future, we can use this same mechanism to perform stack
overflow checks instead of using a patchpoint.

* b3/air/AirCode.cpp:
(JSC::B3::Air::Code::isEntrypoint):
(JSC::B3::Air::Code::entrypointIndex):
* b3/air/AirCode.h:
(JSC::B3::Air::Code::setPrologueForEntrypoint):
(JSC::B3::Air::Code::prologueGeneratorForEntrypoint):
* b3/air/AirGenerate.cpp:
(JSC::B3::Air::generate):
* dfg/DFGAbstractInterpreterInlines.h:
(JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects):
* dfg/DFGBasicBlock.h:
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::parseBlock):
(JSC::DFG::ByteCodeParser::parse):
* dfg/DFGCFG.h:
(JSC::DFG::selectCFG):
* dfg/DFGClobberize.h:
(JSC::DFG::clobberize):
* dfg/DFGClobbersExitState.cpp:
(JSC::DFG::clobbersExitState):
* dfg/DFGCommonData.cpp:
(JSC::DFG::CommonData::shrinkToFit):
(JSC::DFG::CommonData::finalizeCatchEntrypoints):
* dfg/DFGCommonData.h:
(JSC::DFG::CommonData::catchOSREntryDataForBytecodeIndex):
(JSC::DFG::CommonData::appendCatchEntrypoint):
* dfg/DFGDoesGC.cpp:
(JSC::DFG::doesGC):
* dfg/DFGFixupPhase.cpp:
(JSC::DFG::FixupPhase::fixupNode):
* dfg/DFGGraph.cpp:
(JSC::DFG::Graph::dump):
(JSC::DFG::Graph::invalidateCFG):
(JSC::DFG::Graph::ensureCPSCFG):
(JSC::DFG::Graph::methodOfGettingAValueProfileFor):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::isEntrypoint):
* dfg/DFGInPlaceAbstractState.cpp:
(JSC::DFG::InPlaceAbstractState::initialize):
(JSC::DFG::InPlaceAbstractState::mergeToSuccessors):
* dfg/DFGJITCode.cpp:
(JSC::DFG::JITCode::shrinkToFit):
(JSC::DFG::JITCode::finalizeOSREntrypoints):
* dfg/DFGJITCode.h:
(JSC::DFG::JITCode::catchOSREntryDataForBytecodeIndex): Deleted.
(JSC::DFG::JITCode::appendCatchEntrypoint): Deleted.
* dfg/DFGJITCompiler.cpp:
(JSC::DFG::JITCompiler::noticeCatchEntrypoint):
(JSC::DFG::JITCompiler::makeCatchOSREntryBuffer):
* dfg/DFGMayExit.cpp:
* dfg/DFGNode.h:
(JSC::DFG::Node::isEntrySwitch):
(JSC::DFG::Node::isTerminal):
(JSC::DFG::Node::entrySwitchData):
(JSC::DFG::Node::numSuccessors):
(JSC::DFG::Node::successor):
(JSC::DFG::Node::entrypointIndex):
* dfg/DFGNodeType.h:
* dfg/DFGOSRAvailabilityAnalysisPhase.cpp:
(JSC::DFG::OSRAvailabilityAnalysisPhase::run):
(JSC::DFG::LocalOSRAvailabilityCalculator::executeNode):
* dfg/DFGOSREntry.cpp:
(JSC::DFG::prepareCatchOSREntry):
* dfg/DFGOSREntry.h:
* dfg/DFGOSREntrypointCreationPhase.cpp:
(JSC::DFG::OSREntrypointCreationPhase::run):
* dfg/DFGPredictionPropagationPhase.cpp:
* dfg/DFGSSAConversionPhase.cpp:
(JSC::DFG::SSAConversionPhase::SSAConversionPhase):
(JSC::DFG::SSAConversionPhase::run):
* dfg/DFGSafeToExecute.h:
(JSC::DFG::safeToExecute):
* dfg/DFGSpeculativeJIT.cpp:
(JSC::DFG::SpeculativeJIT::linkOSREntries):
* dfg/DFGSpeculativeJIT32_64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGSpeculativeJIT64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGStaticExecutionCountEstimationPhase.cpp:
(JSC::DFG::StaticExecutionCountEstimationPhase::run):
* dfg/DFGValidate.cpp:
* ftl/FTLCapabilities.cpp:
(JSC::FTL::canCompile):
* ftl/FTLCompile.cpp:
(JSC::FTL::compile):
* ftl/FTLLowerDFGToB3.cpp:
(JSC::FTL::DFG::LowerDFGToB3::lower):
(JSC::FTL::DFG::LowerDFGToB3::compileNode):
(JSC::FTL::DFG::LowerDFGToB3::compileExtractCatchLocal):
(JSC::FTL::DFG::LowerDFGToB3::compileGetStack):
(JSC::FTL::DFG::LowerDFGToB3::compileEntrySwitch):
(JSC::FTL::DFG::LowerDFGToB3::speculate):
(JSC::FTL::DFG::LowerDFGToB3::appendOSRExitDescriptor):
(JSC::FTL::DFG::LowerDFGToB3::appendOSRExit):
(JSC::FTL::DFG::LowerDFGToB3::blessSpeculation):
* ftl/FTLOutput.cpp:
(JSC::FTL::Output::entrySwitch):
* ftl/FTLOutput.h:
* jit/JITOperations.cpp:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@221602 268f45cc-cd09-0410-ab3c-d52691b4dbfc

41 files changed:
Source/JavaScriptCore/ChangeLog
Source/JavaScriptCore/b3/air/AirCode.cpp
Source/JavaScriptCore/b3/air/AirCode.h
Source/JavaScriptCore/b3/air/AirGenerate.cpp
Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h
Source/JavaScriptCore/dfg/DFGBasicBlock.h
Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp
Source/JavaScriptCore/dfg/DFGCFG.h
Source/JavaScriptCore/dfg/DFGClobberize.h
Source/JavaScriptCore/dfg/DFGClobbersExitState.cpp
Source/JavaScriptCore/dfg/DFGCommonData.cpp
Source/JavaScriptCore/dfg/DFGCommonData.h
Source/JavaScriptCore/dfg/DFGDoesGC.cpp
Source/JavaScriptCore/dfg/DFGFixupPhase.cpp
Source/JavaScriptCore/dfg/DFGGraph.cpp
Source/JavaScriptCore/dfg/DFGGraph.h
Source/JavaScriptCore/dfg/DFGInPlaceAbstractState.cpp
Source/JavaScriptCore/dfg/DFGJITCode.cpp
Source/JavaScriptCore/dfg/DFGJITCode.h
Source/JavaScriptCore/dfg/DFGJITCompiler.cpp
Source/JavaScriptCore/dfg/DFGMayExit.cpp
Source/JavaScriptCore/dfg/DFGNode.h
Source/JavaScriptCore/dfg/DFGNodeType.h
Source/JavaScriptCore/dfg/DFGOSRAvailabilityAnalysisPhase.cpp
Source/JavaScriptCore/dfg/DFGOSREntry.cpp
Source/JavaScriptCore/dfg/DFGOSREntry.h
Source/JavaScriptCore/dfg/DFGOSREntrypointCreationPhase.cpp
Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp
Source/JavaScriptCore/dfg/DFGSSAConversionPhase.cpp
Source/JavaScriptCore/dfg/DFGSafeToExecute.h
Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp
Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp
Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp
Source/JavaScriptCore/dfg/DFGStaticExecutionCountEstimationPhase.cpp
Source/JavaScriptCore/dfg/DFGValidate.cpp
Source/JavaScriptCore/ftl/FTLCapabilities.cpp
Source/JavaScriptCore/ftl/FTLCompile.cpp
Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp
Source/JavaScriptCore/ftl/FTLOutput.cpp
Source/JavaScriptCore/ftl/FTLOutput.h
Source/JavaScriptCore/jit/JITOperations.cpp

index dd0d378..75f4142 100644 (file)
@@ -1,3 +1,140 @@
+2017-09-04  Saam Barati  <sbarati@apple.com>
+
+        Support compiling catch in the FTL
+        https://bugs.webkit.org/show_bug.cgi?id=175396
+
+        Reviewed by Filip Pizlo.
+
+        This patch implements op_catch in the FTL. It extends the DFG implementation
+        by supporting multiple entrypoints in DFG-SSA. This patch implements this
+        by introducing an EntrySwitch node. When converting to SSA, we introduce a new
+        root block with an EntrySwitch that has the previous DFG entrypoints as its
+        successors. By convention, we pick the zeroth entry point index to be the
+        op_enter entrypoint. Like in B3, in DFG-SSA, EntrySwitch just acts like a
+        switch over the entrypoint index argument. DFG::EntrySwitch in the FTL
+        simply lowers to B3::EntrySwitch. The EntrySwitch in the root block that
+        SSAConversion creates can not exit because we would both not know where to exit
+        to in the program: we would not have valid OSR exit state. This design also
+        mandates that anything we hoist above EntrySwitch in the new root block
+        can not exit since they also do not have valid OSR exit state.
+        
+        This patch also adds a new metadata node named InitializeEntrypointArguments.
+        InitializeEntrypointArguments is a metadata node that initializes the flush format for
+        the arguments at a given entrypoint. For a given entrypoint index, this node
+        tells AI and OSRAvailabilityAnalysis what the flush format for each argument
+        is. This allows each individual entrypoint to have an independent set of
+        argument types. Currently, this won't happen in practice because ArgumentPosition
+        unifies flush formats, but this is an implementation detail we probably want
+        to modify in the future. SSAConversion will add InitializeEntrypointArguments
+        to the beginning of each of the original DFG entrypoint blocks.
+        
+        This patch also adds the ability to specify custom prologue code generators in Air.
+        This allows the FTL to specify a custom prologue for catch entrypoints that
+        matches the op_catch OSR entry calling convention that the DFG uses. This way,
+        the baseline JIT code OSR enters into op_catch the same way both in the DFG
+        and the FTL. In the future, we can use this same mechanism to perform stack
+        overflow checks instead of using a patchpoint.
+
+        * b3/air/AirCode.cpp:
+        (JSC::B3::Air::Code::isEntrypoint):
+        (JSC::B3::Air::Code::entrypointIndex):
+        * b3/air/AirCode.h:
+        (JSC::B3::Air::Code::setPrologueForEntrypoint):
+        (JSC::B3::Air::Code::prologueGeneratorForEntrypoint):
+        * b3/air/AirGenerate.cpp:
+        (JSC::B3::Air::generate):
+        * dfg/DFGAbstractInterpreterInlines.h:
+        (JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects):
+        * dfg/DFGBasicBlock.h:
+        * dfg/DFGByteCodeParser.cpp:
+        (JSC::DFG::ByteCodeParser::parseBlock):
+        (JSC::DFG::ByteCodeParser::parse):
+        * dfg/DFGCFG.h:
+        (JSC::DFG::selectCFG):
+        * dfg/DFGClobberize.h:
+        (JSC::DFG::clobberize):
+        * dfg/DFGClobbersExitState.cpp:
+        (JSC::DFG::clobbersExitState):
+        * dfg/DFGCommonData.cpp:
+        (JSC::DFG::CommonData::shrinkToFit):
+        (JSC::DFG::CommonData::finalizeCatchEntrypoints):
+        * dfg/DFGCommonData.h:
+        (JSC::DFG::CommonData::catchOSREntryDataForBytecodeIndex):
+        (JSC::DFG::CommonData::appendCatchEntrypoint):
+        * dfg/DFGDoesGC.cpp:
+        (JSC::DFG::doesGC):
+        * dfg/DFGFixupPhase.cpp:
+        (JSC::DFG::FixupPhase::fixupNode):
+        * dfg/DFGGraph.cpp:
+        (JSC::DFG::Graph::dump):
+        (JSC::DFG::Graph::invalidateCFG):
+        (JSC::DFG::Graph::ensureCPSCFG):
+        (JSC::DFG::Graph::methodOfGettingAValueProfileFor):
+        * dfg/DFGGraph.h:
+        (JSC::DFG::Graph::isEntrypoint):
+        * dfg/DFGInPlaceAbstractState.cpp:
+        (JSC::DFG::InPlaceAbstractState::initialize):
+        (JSC::DFG::InPlaceAbstractState::mergeToSuccessors):
+        * dfg/DFGJITCode.cpp:
+        (JSC::DFG::JITCode::shrinkToFit):
+        (JSC::DFG::JITCode::finalizeOSREntrypoints):
+        * dfg/DFGJITCode.h:
+        (JSC::DFG::JITCode::catchOSREntryDataForBytecodeIndex): Deleted.
+        (JSC::DFG::JITCode::appendCatchEntrypoint): Deleted.
+        * dfg/DFGJITCompiler.cpp:
+        (JSC::DFG::JITCompiler::noticeCatchEntrypoint):
+        (JSC::DFG::JITCompiler::makeCatchOSREntryBuffer):
+        * dfg/DFGMayExit.cpp:
+        * dfg/DFGNode.h:
+        (JSC::DFG::Node::isEntrySwitch):
+        (JSC::DFG::Node::isTerminal):
+        (JSC::DFG::Node::entrySwitchData):
+        (JSC::DFG::Node::numSuccessors):
+        (JSC::DFG::Node::successor):
+        (JSC::DFG::Node::entrypointIndex):
+        * dfg/DFGNodeType.h:
+        * dfg/DFGOSRAvailabilityAnalysisPhase.cpp:
+        (JSC::DFG::OSRAvailabilityAnalysisPhase::run):
+        (JSC::DFG::LocalOSRAvailabilityCalculator::executeNode):
+        * dfg/DFGOSREntry.cpp:
+        (JSC::DFG::prepareCatchOSREntry):
+        * dfg/DFGOSREntry.h:
+        * dfg/DFGOSREntrypointCreationPhase.cpp:
+        (JSC::DFG::OSREntrypointCreationPhase::run):
+        * dfg/DFGPredictionPropagationPhase.cpp:
+        * dfg/DFGSSAConversionPhase.cpp:
+        (JSC::DFG::SSAConversionPhase::SSAConversionPhase):
+        (JSC::DFG::SSAConversionPhase::run):
+        * dfg/DFGSafeToExecute.h:
+        (JSC::DFG::safeToExecute):
+        * dfg/DFGSpeculativeJIT.cpp:
+        (JSC::DFG::SpeculativeJIT::linkOSREntries):
+        * dfg/DFGSpeculativeJIT32_64.cpp:
+        (JSC::DFG::SpeculativeJIT::compile):
+        * dfg/DFGSpeculativeJIT64.cpp:
+        (JSC::DFG::SpeculativeJIT::compile):
+        * dfg/DFGStaticExecutionCountEstimationPhase.cpp:
+        (JSC::DFG::StaticExecutionCountEstimationPhase::run):
+        * dfg/DFGValidate.cpp:
+        * ftl/FTLCapabilities.cpp:
+        (JSC::FTL::canCompile):
+        * ftl/FTLCompile.cpp:
+        (JSC::FTL::compile):
+        * ftl/FTLLowerDFGToB3.cpp:
+        (JSC::FTL::DFG::LowerDFGToB3::lower):
+        (JSC::FTL::DFG::LowerDFGToB3::compileNode):
+        (JSC::FTL::DFG::LowerDFGToB3::compileExtractCatchLocal):
+        (JSC::FTL::DFG::LowerDFGToB3::compileGetStack):
+        (JSC::FTL::DFG::LowerDFGToB3::compileEntrySwitch):
+        (JSC::FTL::DFG::LowerDFGToB3::speculate):
+        (JSC::FTL::DFG::LowerDFGToB3::appendOSRExitDescriptor):
+        (JSC::FTL::DFG::LowerDFGToB3::appendOSRExit):
+        (JSC::FTL::DFG::LowerDFGToB3::blessSpeculation):
+        * ftl/FTLOutput.cpp:
+        (JSC::FTL::Output::entrySwitch):
+        * ftl/FTLOutput.h:
+        * jit/JITOperations.cpp:
+
 2017-09-03  Yusuke Suzuki  <utatane.tea@gmail.com>
 
         [DFG][FTL] Efficiently execute number#toString()
index 0d153b9..599a63c 100644 (file)
@@ -156,6 +156,8 @@ CCallSpecial* Code::cCallSpecial()
 
 bool Code::isEntrypoint(BasicBlock* block) const
 {
+    // Note: This function must work both before and after LowerEntrySwitch.
+
     if (m_entrypoints.isEmpty())
         return !block->index();
     
@@ -166,6 +168,16 @@ bool Code::isEntrypoint(BasicBlock* block) const
     return false;
 }
 
+std::optional<unsigned> Code::entrypointIndex(BasicBlock* block) const
+{
+    RELEASE_ASSERT(m_entrypoints.size());
+    for (unsigned i = 0; i < m_entrypoints.size(); ++i) {
+        if (m_entrypoints[i].block() == block)
+            return i;
+    }
+    return std::nullopt;
+}
+
 void Code::setCalleeSaveRegisterAtOffsetList(RegisterAtOffsetList&& registerAtOffsetList, StackSlot* slot)
 {
     m_uncorrectedCalleeSaveRegisterAtOffsetList = WTFMove(registerAtOffsetList);
index 937d051..2fe9af2 100644 (file)
@@ -53,11 +53,15 @@ namespace Air {
 class BlockInsertionSet;
 class CCallSpecial;
 class CFG;
+class Code;
 class Disassembler;
 
 typedef void WasmBoundsCheckGeneratorFunction(CCallHelpers&, GPRReg);
 typedef SharedTask<WasmBoundsCheckGeneratorFunction> WasmBoundsCheckGenerator;
 
+typedef void PrologueGeneratorFunction(CCallHelpers&, Code&);
+typedef SharedTask<PrologueGeneratorFunction> PrologueGenerator;
+
 // This is an IR that is very close to the bare metal. It requires about 40x more bytes than the
 // generated machine code - for example if you're generating 1MB of machine code, you need about
 // 40MB of Air.
@@ -165,7 +169,18 @@ public:
     const Vector<FrequentedBlock>& entrypoints() const { return m_entrypoints; }
     const FrequentedBlock& entrypoint(unsigned index) const { return m_entrypoints[index]; }
     bool isEntrypoint(BasicBlock*) const;
-    
+    // Note: It is only valid to call this function after LowerEntrySwitch.
+    std::optional<unsigned> entrypointIndex(BasicBlock*) const;
+    void setPrologueForEntrypoint(unsigned entrypointIndex, RefPtr<PrologueGenerator> generator)
+    {
+        // Note: We allow this to be called even before we set m_entrypoints just for convenience to users of this API.
+        m_entrypointIndexToGenerator.set(entrypointIndex, generator);
+    }
+    RefPtr<PrologueGenerator> prologueGeneratorForEntrypoint(unsigned entrypointIndex)
+    {
+        return m_entrypointIndexToGenerator.get(entrypointIndex);
+    }
+
     // This is used by lowerEntrySwitch().
     template<typename Vector>
     void setEntrypoints(Vector&& vector)
@@ -353,6 +368,7 @@ private:
     StackSlot* m_calleeSaveStackSlot { nullptr };
     Vector<FrequentedBlock> m_entrypoints; // This is empty until after lowerEntrySwitch().
     Vector<CCallHelpers::Label> m_entrypointLabels; // This is empty until code generation.
+    HashMap<unsigned, RefPtr<PrologueGenerator>, WTF::IntHash<unsigned>, WTF::UnsignedWithZeroKeyHashTraits<unsigned>> m_entrypointIndexToGenerator;
     RefPtr<WasmBoundsCheckGenerator> m_wasmBoundsCheckGenerator;
     const char* m_lastPhaseName;
     std::unique_ptr<Disassembler> m_disassembler;
index b97910c..cb599e6 100644 (file)
@@ -208,21 +208,28 @@ void generate(Code& code, CCallHelpers& jit)
         if (disassembler)
             disassembler->startBlock(block, jit); 
 
-        if (code.isEntrypoint(block)) {
+        if (std::optional<unsigned> entrypointIndex = code.entrypointIndex(block)) {
+            ASSERT(code.isEntrypoint(block));
+
             if (disassembler)
                 disassembler->startEntrypoint(jit); 
 
-            jit.emitFunctionPrologue();
-            if (code.frameSize()) {
-                AllowMacroScratchRegisterUsageIf allowScratch(jit, isARM64());
-                jit.addPtr(CCallHelpers::TrustedImm32(-code.frameSize()), MacroAssembler::stackPointerRegister);
+            if (RefPtr<PrologueGenerator> prologueGenerator = code.prologueGeneratorForEntrypoint(*entrypointIndex))
+                prologueGenerator->run(jit, code);
+            else {
+                jit.emitFunctionPrologue();
+                if (code.frameSize()) {
+                    AllowMacroScratchRegisterUsageIf allowScratch(jit, isARM64());
+                    jit.addPtr(CCallHelpers::TrustedImm32(-code.frameSize()), MacroAssembler::stackPointerRegister);
+                }
+                
+                jit.emitSave(code.calleeSaveRegisterAtOffsetList());
             }
-            
-            jit.emitSave(code.calleeSaveRegisterAtOffsetList());
 
             if (disassembler)
                 disassembler->endEntrypoint(jit); 
-        }
+        } else
+            ASSERT(!code.isEntrypoint(block));
         
         ASSERT(block->size() >= 1);
         for (unsigned i = 0; i < block->size() - 1; ++i) {
index 9da194b..a8a5695 100644 (file)
@@ -260,7 +260,7 @@ bool AbstractInterpreter<AbstractStateType>::executeEffects(unsigned clobberLimi
         // DFG execution.
         break;
     }
-        
+
     case KillStack: {
         // This is just a hint telling us that the OSR state of the local is no longer inside the
         // flushed data.
@@ -273,7 +273,33 @@ bool AbstractInterpreter<AbstractStateType>::executeEffects(unsigned clobberLimi
         // non-clear value.
         ASSERT(!m_state.variables().operand(node->local()).isClear());
         break;
-        
+
+    case InitializeEntrypointArguments: {
+        unsigned entrypointIndex = node->entrypointIndex();
+        const Vector<FlushFormat>& argumentFormats = m_graph.m_argumentFormats[entrypointIndex];
+        for (unsigned argument = 0; argument < argumentFormats.size(); ++argument) {
+            AbstractValue& value = m_state.variables().argument(argument);
+            switch (argumentFormats[argument]) {
+            case FlushedInt32:
+                value.setType(SpecInt32Only);
+                break;
+            case FlushedBoolean:
+                value.setType(SpecBoolean);
+                break;
+            case FlushedCell:
+                value.setType(m_graph, SpecCell);
+                break;
+            case FlushedJSValue:
+                value.makeBytecodeTop();
+                break;
+            default:
+                DFG_CRASH(m_graph, node, "Bad flush format for argument");
+                break;
+            }
+        }
+        break;
+    }
+
     case LoadVarargs:
     case ForwardVarargs: {
         // FIXME: ForwardVarargs should check if the count becomes known, and if it does, it should turn
@@ -1866,6 +1892,9 @@ bool AbstractInterpreter<AbstractStateType>::executeEffects(unsigned clobberLimi
         break;
     }
 
+    case EntrySwitch:
+        break;
+
     case Return:
         m_state.setIsValid(false);
         break;
index de6abe8..6403eae 100644 (file)
@@ -221,8 +221,6 @@ struct BasicBlock : RefCounted<BasicBlock> {
     
     float executionCount;
     
-    // These fields are reserved for NaturalLoops.
-    
     struct SSAData {
         WTF_MAKE_FAST_ALLOCATED;
     public:
index 279c151..505d1e1 100644 (file)
@@ -5262,9 +5262,7 @@ bool ByteCodeParser::parseBlock(unsigned limit)
                 NEXT_OPCODE(op_catch);
             }
 
-            if (isFTL(m_graph.m_plan.mode)) {
-                // FIXME: Support catch in the FTL.
-                // https://bugs.webkit.org/show_bug.cgi?id=175396
+            if (m_graph.m_plan.mode == FTLForOSREntryMode) {
                 NEXT_OPCODE(op_catch);
             }
 
@@ -6465,8 +6463,6 @@ bool ByteCodeParser::parse()
     m_graph.determineReachability();
     m_graph.killUnreachableBlocks();
 
-    m_graph.m_cpsCFG = std::make_unique<CPSCFG>(m_graph);
-    
     for (BlockIndex blockIndex = m_graph.numBlocks(); blockIndex--;) {
         BasicBlock* block = m_graph.block(blockIndex);
         if (!block)
index a85604a..0e35b2a 100644 (file)
@@ -98,8 +98,7 @@ using SSACFG = CFG;
 template <typename T, typename = typename std::enable_if<std::is_same<T, CPSCFG>::value>::type>
 CPSCFG& selectCFG(Graph& graph)
 {
-    RELEASE_ASSERT(graph.m_cpsCFG);
-    return *graph.m_cpsCFG;
+    return graph.ensureCPSCFG();
 }
 
 template <typename T, typename = typename std::enable_if<std::is_same<T, SSACFG>::value>::type>
index 56c06b3..2eaf2aa 100644 (file)
@@ -426,6 +426,7 @@ void clobberize(Graph& graph, Node* node, const ReadFunctor& read, const WriteFu
     case Jump:
     case Branch:
     case Switch:
+    case EntrySwitch:
     case ForceOSRExit:
     case CheckBadCell:
     case Return:
@@ -437,6 +438,7 @@ void clobberize(Graph& graph, Node* node, const ReadFunctor& read, const WriteFu
     case ProfileType:
     case ProfileControlFlow:
     case PutHint:
+    case InitializeEntrypointArguments:
         write(SideState);
         return;
         
index cc9dcaf..fb94364 100644 (file)
@@ -40,6 +40,7 @@ bool clobbersExitState(Graph& graph, Node* node)
     // There are certain nodes whose effect on the exit state has nothing to do with what they
     // normally clobber.
     switch (node->op()) {
+    case InitializeEntrypointArguments:
     case MovHint:
     case ZombieHint:
     case PutHint:
index 8707788..eb387b2 100644 (file)
@@ -87,6 +87,7 @@ void CommonData::shrinkToFit()
     codeOrigins.shrinkToFit();
     weakReferences.shrinkToFit();
     transitions.shrinkToFit();
+    catchEntrypoints.shrinkToFit();
 }
 
 static StaticLock pcCodeBlockMapLock;
@@ -193,6 +194,17 @@ void CommonData::validateReferences(const TrackedReferences& trackedReferences)
         watchpoint->key().validateReferences(trackedReferences);
 }
 
+void CommonData::finalizeCatchEntrypoints()
+{
+    std::sort(catchEntrypoints.begin(), catchEntrypoints.end(),
+        [] (const CatchEntrypointData& a, const CatchEntrypointData& b) { return a.bytecodeIndex < b.bytecodeIndex; });
+
+#if !ASSERT_DISABLED
+    for (unsigned i = 0; i + 1 < catchEntrypoints.size(); ++i)
+        ASSERT(catchEntrypoints[i].bytecodeIndex <= catchEntrypoints[i + 1].bytecodeIndex);
+#endif
+}
+
 } } // namespace JSC::DFG
 
 #endif // ENABLE(DFG_JIT)
index 294c11f..2408744 100644 (file)
@@ -31,6 +31,7 @@
 #include "DFGAdaptiveInferredPropertyValueWatchpoint.h"
 #include "DFGAdaptiveStructureWatchpoint.h"
 #include "DFGJumpReplacement.h"
+#include "DFGOSREntry.h"
 #include "InlineCallFrameSet.h"
 #include "JSCell.h"
 #include "ProfilerCompilation.h"
@@ -90,6 +91,20 @@ public:
     void installVMTrapBreakpoints(CodeBlock* owner);
     bool isVMTrapBreakpoint(void* address);
 
+    CatchEntrypointData* catchOSREntryDataForBytecodeIndex(unsigned bytecodeIndex)
+    {
+        return tryBinarySearch<CatchEntrypointData, unsigned>(
+            catchEntrypoints, catchEntrypoints.size(), bytecodeIndex,
+            [] (const CatchEntrypointData* item) { return item->bytecodeIndex; });
+    }
+
+    void appendCatchEntrypoint(unsigned bytecodeIndex, void* machineCode, Vector<FlushFormat>&& argumentFormats)
+    {
+        catchEntrypoints.append(CatchEntrypointData { machineCode,  WTFMove(argumentFormats), bytecodeIndex });
+    }
+
+    void finalizeCatchEntrypoints();
+
     unsigned requiredRegisterCountForExecutionAndExit() const
     {
         return std::max(frameRegisterCount, requiredRegisterCountForExit);
@@ -106,11 +121,13 @@ public:
     Vector<WeakReferenceTransition> transitions;
     Vector<WriteBarrier<JSCell>> weakReferences;
     Vector<WriteBarrier<Structure>> weakStructureReferences;
+    Vector<CatchEntrypointData> catchEntrypoints;
     Bag<CodeBlockJettisoningWatchpoint> watchpoints;
     Bag<AdaptiveStructureWatchpoint> adaptiveStructureWatchpoints;
     Bag<AdaptiveInferredPropertyValueWatchpoint> adaptiveInferredPropertyValueWatchpoints;
     Vector<JumpReplacement> jumpReplacements;
     
+    ScratchBuffer* catchOSREntryBuffer;
     RefPtr<Profiler::Compilation> compilation;
     bool livenessHasBeenProved; // Initialized and used on every GC.
     bool allTransitionsHaveBeenMarked; // Initialized and used on every GC.
index 25b1ffc..e0951b8 100644 (file)
@@ -56,6 +56,7 @@ bool doesGC(Graph& graph, Node* node)
     case GetLocal:
     case SetLocal:
     case MovHint:
+    case InitializeEntrypointArguments:
     case ZombieHint:
     case ExitOK:
     case Phantom:
@@ -184,6 +185,7 @@ bool doesGC(Graph& graph, Node* node)
     case Jump:
     case Branch:
     case Switch:
+    case EntrySwitch:
     case Return:
     case TailCall:
     case DirectTailCall:
index 1a3cbc9..1a7559a 100644 (file)
@@ -1549,6 +1549,7 @@ private:
 
         case Phi:
         case Upsilon:
+        case EntrySwitch:
         case GetIndexedPropertyStorage:
         case LastNodeType:
         case CheckTierUpInLoop:
@@ -2037,6 +2038,7 @@ private:
         case ExtractCatchLocal:
         case LoopHint:
         case MovHint:
+        case InitializeEntrypointArguments:
         case ZombieHint:
         case ExitOK:
         case BottomValue:
index ed46b05..ba87f08 100644 (file)
@@ -368,6 +368,11 @@ void Graph::dump(PrintStream& out, const char* prefix, Node* node, DumpContext*
             out.print(comma, inContext(data->cases[i].value, context), ":", data->cases[i].target);
         out.print(comma, "default:", data->fallThrough);
     }
+    if (node->isEntrySwitch()) {
+        EntrySwitchData* data = node->entrySwitchData();
+        for (unsigned i = 0; i < data->cases.size(); ++i)
+            out.print(comma, BranchTarget(data->cases[i]));
+    }
     ClobberSet reads;
     ClobberSet writes;
     addReadsAndWrites(*this, node, reads, writes);
@@ -515,8 +520,10 @@ void Graph::dump(PrintStream& out, DumpContext* context)
     out.print("\n");
     out.print("DFG for ", CodeBlockWithJITType(m_codeBlock, JITCode::DFGJIT), ":\n");
     out.print("  Fixpoint state: ", m_fixpointState, "; Form: ", m_form, "; Unification state: ", m_unificationState, "; Ref count state: ", m_refCountState, "\n");
-    if (m_form == SSA)
-        out.print("  Argument formats: ", listDump(m_argumentFormats), "\n");
+    if (m_form == SSA) {
+        for (unsigned entrypointIndex = 0; entrypointIndex < m_argumentFormats.size(); ++entrypointIndex)
+            out.print("  Argument formats for entrypoint index: ", entrypointIndex, " : ", listDump(m_argumentFormats[entrypointIndex]), "\n");
+    }
     else {
         for (auto pair : m_entrypointToArguments)
             out.print("  Arguments for block#", pair.key->index, ": ", listDump(pair.value), "\n");
@@ -824,6 +831,7 @@ void Graph::invalidateCFG()
     m_controlEquivalenceAnalysis = nullptr;
     m_backwardsDominators = nullptr;
     m_backwardsCFG = nullptr;
+    m_cpsCFG = nullptr;
 }
 
 void Graph::invalidateNodeLiveness()
@@ -1543,6 +1551,14 @@ void Graph::logAssertionFailure(
     logDFGAssertionFailure(*this, toCString("While handling block ", pointerDump(block), "\n\n"), file, line, function, assertion);
 }
 
+CPSCFG& Graph::ensureCPSCFG()
+{
+    RELEASE_ASSERT(m_form != SSA && !m_isInSSAConversion);
+    if (!m_cpsCFG)
+        m_cpsCFG = std::make_unique<CPSCFG>(*this);
+    return *m_cpsCFG;
+}
+
 CPSDominators& Graph::ensureCPSDominators()
 {
     RELEASE_ASSERT(m_form != SSA && !m_isInSSAConversion);
@@ -1610,21 +1626,14 @@ MethodOfGettingAValueProfile Graph::methodOfGettingAValueProfileFor(Node* curren
             CodeBlock* profiledBlock = baselineCodeBlockFor(node->origin.semantic);
 
             if (node->accessesStack(*this)) {
-                ValueProfile* result = [&] () -> ValueProfile* {
-                    if (!node->local().isArgument())
-                        return nullptr;
+                if (m_form != SSA && node->local().isArgument()) {
                     int argument = node->local().toArgument();
+                    Node* argumentNode = m_entrypointToArguments.find(block(0))->value[argument];
                     // FIXME: We should match SetArgument nodes at other entrypoints as well:
                     // https://bugs.webkit.org/show_bug.cgi?id=175841
-                    Node* argumentNode = m_entrypointToArguments.find(block(0))->value[argument];
-                    if (!argumentNode)
-                        return nullptr;
-                    if (node->variableAccessData() != argumentNode->variableAccessData())
-                        return nullptr;
-                    return &profiledBlock->valueProfileForArgument(argument);
-                }();
-                if (result)
-                    return result;
+                    if (argumentNode && node->variableAccessData() == argumentNode->variableAccessData())
+                        return &profiledBlock->valueProfileForArgument(argument);
+                }
 
                 if (node->op() == GetLocal) {
                     return MethodOfGettingAValueProfile::fromLazyOperand(
index 5c39c17..a7e1fed 100644 (file)
@@ -931,6 +931,7 @@ public:
     BackwardsCFG& ensureBackwardsCFG();
     BackwardsDominators& ensureBackwardsDominators();
     ControlEquivalenceAnalysis& ensureControlEquivalenceAnalysis();
+    CPSCFG& ensureCPSCFG();
 
     // These functions only makes sense to call after bytecode parsing
     // because it queries the m_hasExceptionHandlers boolean whose value
@@ -950,6 +951,14 @@ public:
 
     bool isEntrypoint(BasicBlock* block) const
     {
+        ASSERT_WITH_MESSAGE(!m_isInSSAConversion, "This is not written to work during SSA conversion.");
+
+        if (m_form == SSA) {
+            ASSERT(m_entrypoints.size() == 1);
+            ASSERT(m_entrypoints.contains(this->block(0)));
+            return block == this->block(0);
+        }
+
         if (m_entrypoints.size() <= 4) {
             bool result = m_entrypoints.contains(block);
             ASSERT(result == m_entrypointToArguments.contains(block));
@@ -978,12 +987,13 @@ public:
     
     // In CPS, this is all of the SetArgument nodes for the arguments in the machine code block
     // that survived DCE. All of them except maybe "this" will survive DCE, because of the Flush
-    // nodes.
+    // nodes. In SSA, this has no meaning. It's empty.
+    HashMap<BasicBlock*, ArgumentsVector> m_entrypointToArguments;
+
+    // In SSA, this is the argument speculation that we've locked in for an entrypoint block.
     //
-    // In SSA, this is all of the GetStack nodes for the arguments in the machine code block that
-    // may have some speculation in the prologue and survived DCE. Note that to get the speculation
-    // for an argument in SSA, you must use m_argumentFormats, since we still have to speculate
-    // even if the argument got killed. For example:
+    // We must speculate on the argument types at each entrypoint even if operations involving
+    // arguments get killed. For example:
     //
     //     function foo(x) {
     //        var tmp = x + 1;
@@ -1001,10 +1011,20 @@ public:
     //
     // If we DCE the ArithAdd and we remove the int check on x, then this won't do the side
     // effects.
-    HashMap<BasicBlock*, ArgumentsVector> m_entrypointToArguments;
-    
-    // In CPS, this is meaningless. In SSA, this is the argument speculation that we've locked in.
-    Vector<FlushFormat> m_argumentFormats;
+    //
+    // By convention, entrypoint index 0 is used for the CodeBlock's op_enter entrypoint.
+    // So argumentFormats[0] are the argument formats for the normal call entrypoint.
+    Vector<Vector<FlushFormat>> m_argumentFormats;
+
+    // This maps an entrypoint index to a particular op_catch bytecode offset. By convention,
+    // it'll never have zero as a key because we use zero to mean the op_enter entrypoint.
+    HashMap<unsigned, unsigned> m_entrypointIndexToCatchBytecodeOffset;
+
+    // This is the number of logical entrypoints that we're compiling. This is only used
+    // in SSA. Each EntrySwitch node must have numberOfEntrypoints cases. Note, this is
+    // not the same as m_entrypoints.size(). m_entrypoints.size() represents the number
+    // of roots in the CFG. In SSA, m_entrypoints.size() == 1.
+    unsigned m_numberOfEntrypoints { UINT_MAX };
 
     SegmentedVector<VariableAccessData, 16> m_variableAccessData;
     SegmentedVector<ArgumentPosition, 8> m_argumentPositions;
@@ -1060,6 +1080,7 @@ public:
     std::optional<uint32_t> m_maxLocalsForCatchOSREntry;
     std::unique_ptr<FlowIndexing> m_indexingCache;
     std::unique_ptr<FlowMap<AbstractValue>> m_abstractValuesCache;
+    Bag<EntrySwitchData> m_entrySwitchData;
 
     RegisteredStructure stringStructure;
     RegisteredStructure symbolStructure;
index f863120..a0ee0bd 100644 (file)
@@ -100,42 +100,45 @@ void InPlaceAbstractState::initialize()
         entrypoint->cfaStructureClobberStateAtHead = StructuresAreWatched;
         entrypoint->cfaStructureClobberStateAtTail = StructuresAreWatched;
 
-        for (size_t i = 0; i < entrypoint->valuesAtHead.numberOfArguments(); ++i) {
-            entrypoint->valuesAtTail.argument(i).clear();
-
-            FlushFormat format;
-            if (m_graph.m_form == SSA) {
-                // FIXME: When supporting multiple entrypoints in the FTL, we need to change
-                // what we do here: https://bugs.webkit.org/show_bug.cgi?id=175396
-                format = m_graph.m_argumentFormats[i];
-            } else {
-                Node* node = m_graph.m_entrypointToArguments.find(entrypoint)->value[i];
+        if (m_graph.m_form == SSA)  {
+            for (size_t i = 0; i < entrypoint->valuesAtHead.numberOfArguments(); ++i) {
+                entrypoint->valuesAtHead.argument(i).clear();
+                entrypoint->valuesAtTail.argument(i).clear();
+            }
+        } else {
+            const ArgumentsVector& arguments = m_graph.m_entrypointToArguments.find(entrypoint)->value;
+            for (size_t i = 0; i < entrypoint->valuesAtHead.numberOfArguments(); ++i) {
+                entrypoint->valuesAtTail.argument(i).clear();
+
+                FlushFormat format;
+                Node* node = arguments[i];
                 if (!node)
                     format = FlushedJSValue;
                 else {
                     ASSERT(node->op() == SetArgument);
                     format = node->variableAccessData()->flushFormat();
                 }
-            }
-            
-            switch (format) {
-            case FlushedInt32:
-                entrypoint->valuesAtHead.argument(i).setType(SpecInt32Only);
-                break;
-            case FlushedBoolean:
-                entrypoint->valuesAtHead.argument(i).setType(SpecBoolean);
-                break;
-            case FlushedCell:
-                entrypoint->valuesAtHead.argument(i).setType(m_graph, SpecCell);
-                break;
-            case FlushedJSValue:
-                entrypoint->valuesAtHead.argument(i).makeBytecodeTop();
-                break;
-            default:
-                DFG_CRASH(m_graph, nullptr, "Bad flush format for argument");
-                break;
+
+                switch (format) {
+                case FlushedInt32:
+                    entrypoint->valuesAtHead.argument(i).setType(SpecInt32Only);
+                    break;
+                case FlushedBoolean:
+                    entrypoint->valuesAtHead.argument(i).setType(SpecBoolean);
+                    break;
+                case FlushedCell:
+                    entrypoint->valuesAtHead.argument(i).setType(m_graph, SpecCell);
+                    break;
+                case FlushedJSValue:
+                    entrypoint->valuesAtHead.argument(i).makeBytecodeTop();
+                    break;
+                default:
+                    DFG_CRASH(m_graph, nullptr, "Bad flush format for argument");
+                    break;
+                }
             }
         }
+
         for (size_t i = 0; i < entrypoint->valuesAtHead.numberOfLocals(); ++i) {
             entrypoint->valuesAtHead.local(i).clear();
             entrypoint->valuesAtTail.local(i).clear();
@@ -163,6 +166,7 @@ void InPlaceAbstractState::initialize()
             block->valuesAtTail.local(i).clear();
         }
     }
+
     if (m_graph.m_form == SSA) {
         for (BlockIndex blockIndex = 0; blockIndex < m_graph.numBlocks(); ++blockIndex) {
             BasicBlock* block = m_graph.block(blockIndex);
@@ -371,6 +375,14 @@ inline bool InPlaceAbstractState::mergeToSuccessors(BasicBlock* basicBlock)
             changed |= merge(basicBlock, data->cases[i].target.block);
         return changed;
     }
+    
+    case EntrySwitch: {
+        EntrySwitchData* data = terminal->entrySwitchData();
+        bool changed = false;
+        for (unsigned i = data->cases.size(); i--;)
+            changed |= merge(basicBlock, data->cases[i]);
+        return changed;
+    }
 
     case Return:
     case TailCall:
index 6929af1..67c33f0 100644 (file)
@@ -63,7 +63,6 @@ void JITCode::shrinkToFit()
     common.shrinkToFit();
     osrEntry.shrinkToFit();
     osrExit.shrinkToFit();
-    catchEntrypoints.shrinkToFit();
     speculationRecovery.shrinkToFit();
     minifiedDFG.prepareAndShrink();
     variableEventStream.shrinkToFit();
@@ -243,7 +242,6 @@ void JITCode::finalizeOSREntrypoints()
     auto comparator = [] (const auto& a, const auto& b) {
         return a.m_bytecodeIndex < b.m_bytecodeIndex;
     };
-    std::sort(catchEntrypoints.begin(), catchEntrypoints.end(), comparator);
     std::sort(osrEntry.begin(), osrEntry.end(), comparator);
 
 #if !ASSERT_DISABLED
@@ -251,7 +249,6 @@ void JITCode::finalizeOSREntrypoints()
         for (unsigned i = 0; i + 1 < osrVector.size(); ++i)
             ASSERT(osrVector[i].m_bytecodeIndex <= osrVector[i + 1].m_bytecodeIndex);
     };
-    verifyIsSorted(catchEntrypoints);
     verifyIsSorted(osrEntry);
 #endif
 }
index 343eaa6..5507a8a 100644 (file)
@@ -70,20 +70,8 @@ public:
             getOSREntryDataBytecodeIndex);
     }
 
-    CatchEntrypointData* catchOSREntryDataForBytecodeIndex(unsigned bytecodeIndex)
-    {
-        return tryBinarySearch<CatchEntrypointData, unsigned>(
-            catchEntrypoints, catchEntrypoints.size(), bytecodeIndex,
-            [] (const CatchEntrypointData* item) { return item->m_bytecodeIndex; });
-    }
-
     void finalizeOSREntrypoints();
 
-    void appendCatchEntrypoint(unsigned bytecodeIndex, unsigned machineCodeOffset, Vector<FlushFormat>&& argumentFormats)
-    {
-        catchEntrypoints.append(CatchEntrypointData { bytecodeIndex, machineCodeOffset, WTFMove(argumentFormats) });
-    }
-    
     unsigned appendOSRExit(const OSRExit& exit)
     {
         unsigned result = osrExit.size();
@@ -146,12 +134,10 @@ private:
 public:
     CommonData common;
     Vector<DFG::OSREntryData> osrEntry;
-    Vector<CatchEntrypointData> catchEntrypoints;
     SegmentedVector<DFG::OSRExit, 8> osrExit;
     Vector<DFG::SpeculationRecovery> speculationRecovery;
     DFG::VariableEventStream variableEventStream;
     DFG::MinifiedGraph minifiedDFG;
-    ScratchBuffer* catchOSREntryBuffer;
 
 #if ENABLE(FTL_JIT)
     uint8_t neverExecutedEntry { 1 };
index eb0cbce..d19ddb7 100644 (file)
@@ -559,7 +559,7 @@ void JITCompiler::noticeCatchEntrypoint(BasicBlock& basicBlock, JITCompiler::Lab
 {
     RELEASE_ASSERT(basicBlock.isCatchEntrypoint);
     RELEASE_ASSERT(basicBlock.intersectionOfCFAHasVisited); // An entrypoint is reachable by definition.
-    m_jitCode->appendCatchEntrypoint(basicBlock.bytecodeBegin, linkBuffer.offsetOf(blockHead), WTFMove(argumentFormats));
+    m_jitCode->common.appendCatchEntrypoint(basicBlock.bytecodeBegin, linkBuffer.locationOf(blockHead).executableAddress(), WTFMove(argumentFormats));
 }
 
 void JITCompiler::noticeOSREntry(BasicBlock& basicBlock, JITCompiler::Label blockHead, LinkBuffer& linkBuffer)
@@ -687,7 +687,7 @@ void JITCompiler::makeCatchOSREntryBuffer()
 {
     if (m_graph.m_maxLocalsForCatchOSREntry) {
         uint32_t numberOfLiveLocals = std::max(*m_graph.m_maxLocalsForCatchOSREntry, 1u); // Make sure we always allocate a non-null catchOSREntryBuffer.
-        m_jitCode->catchOSREntryBuffer = vm()->scratchBufferForSize(sizeof(JSValue) * numberOfLiveLocals);
+        m_jitCode->common.catchOSREntryBuffer = vm()->scratchBufferForSize(sizeof(JSValue) * numberOfLiveLocals);
     }
 }
 
index 6b299eb..de4ce54 100644 (file)
@@ -53,6 +53,7 @@ ExitMode mayExitImpl(Graph& graph, Node* node, StateType& state)
     case LazyJSConstant:
     case Int52Constant:
     case MovHint:
+    case InitializeEntrypointArguments:
     case SetLocal:
     case Flush:
     case Phantom:
@@ -78,6 +79,7 @@ ExitMode mayExitImpl(Graph& graph, Node* node, StateType& state)
     case PhantomLocal:
     case CountExecution:
     case Jump:
+    case EntrySwitch:
     case Branch:
     case Unreachable:
     case DoubleRep:
index 2e3585d..d58264f 100644 (file)
@@ -203,6 +203,10 @@ struct SwitchData {
     bool didUseJumpTable;
 };
 
+struct EntrySwitchData {
+    Vector<BasicBlock*> cases;
+};
+
 struct CallVarargsData {
     int firstVarArgOffset;
 };
@@ -1315,12 +1319,18 @@ public:
         return op() == Switch;
     }
 
+    bool isEntrySwitch() const
+    {
+        return op() == EntrySwitch;
+    }
+
     bool isTerminal()
     {
         switch (op()) {
         case Jump:
         case Branch:
         case Switch:
+        case EntrySwitch:
         case Return:
         case TailCall:
         case DirectTailCall:
@@ -1382,6 +1392,12 @@ public:
         ASSERT(isSwitch());
         return m_opInfo.as<SwitchData*>();
     }
+
+    EntrySwitchData* entrySwitchData()
+    {
+        ASSERT(isEntrySwitch());
+        return m_opInfo.as<EntrySwitchData*>();
+    }
     
     unsigned numSuccessors()
     {
@@ -1392,6 +1408,8 @@ public:
             return 2;
         case Switch:
             return switchData()->cases.size() + 1;
+        case EntrySwitch:
+            return entrySwitchData()->cases.size();
         default:
             return 0;
         }
@@ -1404,7 +1422,8 @@ public:
                 return switchData()->cases[index].target.block;
             RELEASE_ASSERT(index == switchData()->cases.size());
             return switchData()->fallThrough.block;
-        }
+        } else if (isEntrySwitch())
+            return entrySwitchData()->cases[index];
 
         switch (index) {
         case 0:
@@ -2013,6 +2032,12 @@ public:
         return m_opInfo.as<Profiler::ExecutionCounter*>();
     }
 
+    unsigned entrypointIndex()
+    {
+        ASSERT(op() == InitializeEntrypointArguments);
+        return m_opInfo.as<unsigned>();
+    }
+
     bool shouldGenerate()
     {
         return m_refCount;
index a2cb823..ed74384 100644 (file)
@@ -390,6 +390,7 @@ namespace JSC { namespace DFG {
     macro(Jump, NodeMustGenerate) \
     macro(Branch, NodeMustGenerate) \
     macro(Switch, NodeMustGenerate) \
+    macro(EntrySwitch, NodeMustGenerate) \
     macro(Return, NodeMustGenerate) \
     macro(TailCall, NodeMustGenerate | NodeHasVarArgs) \
     macro(DirectTailCall, NodeMustGenerate | NodeHasVarArgs) \
@@ -440,6 +441,10 @@ namespace JSC { namespace DFG {
     /* Nodes for DOM JIT */\
     macro(CallDOMGetter, NodeResultJS | NodeMustGenerate) \
     macro(CallDOM, NodeResultJS | NodeMustGenerate) \
+    /* Metadata node that initializes the state for flushed argument types at an entrypoint in the program. */ \
+    /* Currently, we only use this for the blocks an EntrySwitch branches to at the root of the program. */ \
+    /* This is only used in SSA. */ \
+    macro(InitializeEntrypointArguments, NodeMustGenerate)
 
 // This enum generates a monotonically increasing id for all Node types,
 // and is used by the subsequent enum to fill out the id (as accessed via the NodeIdMask).
index 3cda687..386403c 100644 (file)
@@ -57,12 +57,9 @@ public:
         
         BasicBlock* root = m_graph.block(0);
         root->ssa->availabilityAtHead.m_locals.fill(Availability::unavailable());
-        for (unsigned argument = m_graph.m_argumentFormats.size(); argument--;) {
-            FlushedAt flushedAt = FlushedAt(
-                m_graph.m_argumentFormats[argument],
-                virtualRegisterForArgument(argument));
-            root->ssa->availabilityAtHead.m_locals.argument(argument) = Availability(flushedAt);
-        }
+
+        for (unsigned argument = 0; argument < m_graph.block(0)->valuesAtHead.numberOfArguments(); ++argument)
+            root->ssa->availabilityAtHead.m_locals.argument(argument) = Availability::unavailable();
 
         // This could be made more efficient by processing blocks in reverse postorder.
         
@@ -86,10 +83,14 @@ public:
                 
                 block->ssa->availabilityAtTail = calculator.m_availability;
                 changed = true;
-                
+
                 for (unsigned successorIndex = block->numSuccessors(); successorIndex--;) {
                     BasicBlock* successor = block->successor(successorIndex);
                     successor->ssa->availabilityAtHead.merge(calculator.m_availability);
+                }
+
+                for (unsigned successorIndex = block->numSuccessors(); successorIndex--;) {
+                    BasicBlock* successor = block->successor(successorIndex);
                     successor->ssa->availabilityAtHead.pruneByLiveness(
                         m_graph, successor->at(0)->origin.forExit);
                 }
@@ -199,6 +200,16 @@ void LocalOSRAvailabilityCalculator::executeNode(Node* node)
         m_availability.m_locals.operand(node->unlinkedLocal()).setNodeUnavailable();
         break;
     }
+
+    case InitializeEntrypointArguments: {
+        unsigned entrypointIndex = node->entrypointIndex();
+        const Vector<FlushFormat>& argumentFormats = m_graph.m_argumentFormats[entrypointIndex];
+        for (unsigned argument = argumentFormats.size(); argument--; ) {
+            FlushedAt flushedAt = FlushedAt(argumentFormats[argument], virtualRegisterForArgument(argument));
+            m_availability.m_locals.argument(argument) = Availability(flushedAt);
+        }
+        break;
+    }
         
     case LoadVarargs:
     case ForwardVarargs: {
index 00134d8..583bf98 100644 (file)
@@ -338,16 +338,19 @@ void* prepareOSREntry(ExecState* exec, CodeBlock* codeBlock, unsigned bytecodeIn
 }
 
 void* prepareCatchOSREntry(ExecState* exec, CodeBlock* codeBlock, unsigned bytecodeIndex)
-{
-    if (!Options::useOSREntryToDFG())
+{ 
+    ASSERT(codeBlock->jitType() == JITCode::DFGJIT || codeBlock->jitType() == JITCode::FTLJIT);
+
+    if (!Options::useOSREntryToDFG() && codeBlock->jitCode()->jitType() == JITCode::DFGJIT)
+        return nullptr;
+    if (!Options::useOSREntryToFTL() && codeBlock->jitCode()->jitType() == JITCode::FTLJIT)
         return nullptr;
 
     VM& vm = exec->vm();
-    ASSERT(codeBlock->jitType() == JITCode::DFGJIT);
-    DFG::JITCode* jitCode = codeBlock->jitCode()->dfg();
-    RELEASE_ASSERT(jitCode);
 
-    DFG::CatchEntrypointData* catchEntrypoint = jitCode->catchOSREntryDataForBytecodeIndex(bytecodeIndex);
+    CommonData* dfgCommon = codeBlock->jitCode()->dfgCommon();
+    RELEASE_ASSERT(dfgCommon);
+    DFG::CatchEntrypointData* catchEntrypoint = dfgCommon->catchOSREntryDataForBytecodeIndex(bytecodeIndex);
     if (!catchEntrypoint) {
         // This can be null under some circumstances. The most common is that we didn't
         // compile this op_catch as an entrypoint since it had never executed when starting
@@ -356,9 +359,9 @@ void* prepareCatchOSREntry(ExecState* exec, CodeBlock* codeBlock, unsigned bytec
     }
 
     // We're only allowed to OSR enter if we've proven we have compatible argument types.
-    for (unsigned argument = 0; argument < catchEntrypoint->m_argumentFormats.size(); ++argument) {
+    for (unsigned argument = 0; argument < catchEntrypoint->argumentFormats.size(); ++argument) {
         JSValue value = exec->uncheckedR(virtualRegisterForArgument(argument)).jsValue();
-        switch (catchEntrypoint->m_argumentFormats[argument]) {
+        switch (catchEntrypoint->argumentFormats[argument]) {
         case DFG::FlushedInt32:
             if (!value.isInt32())
                 return nullptr;
@@ -382,13 +385,13 @@ void* prepareCatchOSREntry(ExecState* exec, CodeBlock* codeBlock, unsigned bytec
         }
     }
 
-    unsigned frameSizeForCheck = jitCode->common.requiredRegisterCountForExecutionAndExit();
+    unsigned frameSizeForCheck = dfgCommon->requiredRegisterCountForExecutionAndExit();
     if (UNLIKELY(!vm.ensureStackCapacityFor(&exec->registers()[virtualRegisterForLocal(frameSizeForCheck).offset()])))
         return nullptr;
 
     ASSERT(Interpreter::getOpcodeID(exec->codeBlock()->instructions()[exec->bytecodeOffset()].u.opcode) == op_catch);
     ValueProfileAndOperandBuffer* buffer = static_cast<ValueProfileAndOperandBuffer*>(exec->codeBlock()->instructions()[exec->bytecodeOffset() + 3].u.pointer);
-    JSValue* dataBuffer = reinterpret_cast<JSValue*>(jitCode->catchOSREntryBuffer->dataBuffer());
+    JSValue* dataBuffer = reinterpret_cast<JSValue*>(dfgCommon->catchOSREntryBuffer->dataBuffer());
     unsigned index = 0;
     buffer->forEach([&] (ValueProfileAndOperand& profile) {
         if (!VirtualRegister(profile.m_operand).isLocal())
@@ -397,9 +400,8 @@ void* prepareCatchOSREntry(ExecState* exec, CodeBlock* codeBlock, unsigned bytec
         ++index;
     });
 
-    jitCode->catchOSREntryBuffer->setActiveLength(sizeof(JSValue) * index);
-
-    return jitCode->executableAddressAtOffset(catchEntrypoint->m_machineCodeOffset);
+    dfgCommon->catchOSREntryBuffer->setActiveLength(sizeof(JSValue) * index);
+    return catchEntrypoint->machineCode;
 }
 
 } } // namespace JSC::DFG
index 712592c..456fa4c 100644 (file)
@@ -71,11 +71,11 @@ inline unsigned getOSREntryDataBytecodeIndex(OSREntryData* osrEntryData)
 }
 
 struct CatchEntrypointData {
-    unsigned m_bytecodeIndex;
-    unsigned m_machineCodeOffset;
     // We use this when doing OSR entry at catch. We prove the arguments
     // are of the expected type before entering at a catch block.
-    Vector<FlushFormat> m_argumentFormats;
+    void* machineCode;
+    Vector<FlushFormat> argumentFormats;
+    unsigned bytecodeIndex;
 };
 
 // Returns a pointer to a data buffer that the OSR entry thunk will recognize and
index d16965b..ac8a5b8 100644 (file)
@@ -147,8 +147,7 @@ public:
         m_graph.m_entrypointToArguments.clear();
         m_graph.m_entrypointToArguments.add(newRoot, newArguments);
 
-        m_graph.m_cpsCFG = std::make_unique<CPSCFG>(m_graph);
-
+        m_graph.invalidateCFG();
         m_graph.resetReachability();
         m_graph.killUnreachableBlocks();
 
index 1088995..c1ec8f9 100644 (file)
@@ -1093,6 +1093,7 @@ private:
             RELEASE_ASSERT_NOT_REACHED();
             break;
             
+        case EntrySwitch:
         case Upsilon:
             // These don't get inserted until we go into SSA.
             RELEASE_ASSERT_NOT_REACHED();
@@ -1156,6 +1157,7 @@ private:
         case ForwardVarargs:
         case PutDynamicVar:
         case NukeStructureAndSetButterfly:
+        case InitializeEntrypointArguments:
             break;
             
         // This gets ignored because it only pretends to produce a value.
index 9dea858..0654905 100644 (file)
@@ -29,6 +29,7 @@
 #if ENABLE(DFG_JIT)
 
 #include "DFGBasicBlockInlines.h"
+#include "DFGBlockInsertionSet.h"
 #include "DFGGraph.h"
 #include "DFGInsertionSet.h"
 #include "DFGPhase.h"
@@ -44,7 +45,6 @@ class SSAConversionPhase : public Phase {
 public:
     SSAConversionPhase(Graph& graph)
         : Phase(graph, "SSA conversion")
-        , m_calculator(graph)
         , m_insertionSet(graph)
     {
     }
@@ -52,9 +52,59 @@ public:
     bool run()
     {
         RELEASE_ASSERT(m_graph.m_form == ThreadedCPS);
+        RELEASE_ASSERT(!m_graph.m_isInSSAConversion);
+        m_graph.m_isInSSAConversion = true;
         
         m_graph.clearReplacements();
         m_graph.clearCPSCFGData();
+
+        HashMap<unsigned, BasicBlock*, WTF::IntHash<unsigned>, WTF::UnsignedWithZeroKeyHashTraits<unsigned>> entrypointIndexToArgumentsBlock;
+
+        {
+            m_graph.m_numberOfEntrypoints = m_graph.m_entrypoints.size();
+
+            BlockInsertionSet blockInsertionSet(m_graph);
+            BasicBlock* newRoot = blockInsertionSet.insert(0, 1.0f);
+
+            EntrySwitchData* entrySwitchData = m_graph.m_entrySwitchData.add();
+            for (unsigned entrypointIndex = 0; entrypointIndex < m_graph.m_numberOfEntrypoints; ++entrypointIndex) {
+                BasicBlock* oldRoot = m_graph.m_entrypoints[entrypointIndex];
+                entrypointIndexToArgumentsBlock.add(entrypointIndex, oldRoot);
+                entrySwitchData->cases.append(oldRoot);
+
+                ASSERT(oldRoot->predecessors.isEmpty());
+                oldRoot->predecessors.append(newRoot);
+
+                if (oldRoot->isCatchEntrypoint) {
+                    ASSERT(!!entrypointIndex);
+                    m_graph.m_entrypointIndexToCatchBytecodeOffset.add(entrypointIndex, oldRoot->bytecodeBegin);
+                }
+
+                NodeOrigin origin = oldRoot->at(0)->origin;
+                m_insertionSet.insertNode(
+                    0, SpecNone, InitializeEntrypointArguments, origin, OpInfo(entrypointIndex));
+                m_insertionSet.insertNode(
+                    0, SpecNone, ExitOK, origin);
+                m_insertionSet.execute(oldRoot);
+            }
+
+            RELEASE_ASSERT(entrySwitchData->cases[0] == m_graph.block(0)); // We strongly assume the normal call entrypoint is the first item in the list.
+
+            m_graph.m_argumentFormats.resize(m_graph.m_numberOfEntrypoints);
+
+            const bool exitOK = false;
+            NodeOrigin origin { CodeOrigin(0), CodeOrigin(0), exitOK };
+            newRoot->appendNode(
+                m_graph, SpecNone, EntrySwitch, origin, OpInfo(entrySwitchData));
+
+            m_graph.m_entrypoints.clear();
+            m_graph.m_entrypoints.append(newRoot);
+
+            blockInsertionSet.execute();
+        }
+
+        SSACalculator calculator(m_graph);
+
         m_graph.ensureSSADominators();
         
         if (verbose) {
@@ -67,7 +117,7 @@ public:
             if (!variable.isRoot())
                 continue;
             
-            SSACalculator::Variable* ssaVariable = m_calculator.newVariable();
+            SSACalculator::Variable* ssaVariable = calculator.newVariable();
             ASSERT(ssaVariable->index() == m_variableForSSAIndex.size());
             m_variableForSSAIndex.append(&variable);
             m_ssaVariableForVariable.add(&variable, ssaVariable);
@@ -103,7 +153,7 @@ public:
                     m_argumentMapping.add(node, childNode);
                 }
                 
-                m_calculator.newDef(
+                calculator.newDef(
                     m_ssaVariableForVariable.get(variable), block, childNode);
             }
             
@@ -112,7 +162,7 @@ public:
         
         // Decide where Phis are to be inserted. This creates the Phi's but doesn't insert them
         // yet. We will later know where to insert based on where SSACalculator tells us to.
-        m_calculator.computePhis(
+        calculator.computePhis(
             [&] (SSACalculator::Variable* ssaVariable, BasicBlock* block) -> Node* {
                 VariableAccessData* variable = m_variableForSSAIndex[ssaVariable->index()];
                 
@@ -167,7 +217,7 @@ public:
             for (unsigned i = 0; i < m_variableForSSAIndex.size(); ++i)
                 dataLog("    ", i, ": ", VariableAccessDataDump(m_graph, m_variableForSSAIndex[i]), "\n");
             dataLog("\n");
-            dataLog("SSA calculator: ", m_calculator, "\n");
+            dataLog("SSA calculator: ", calculator, "\n");
         }
         
         // Do the bulk of the SSA conversion. For each block, this tracks the operand->Node
@@ -220,7 +270,7 @@ public:
                         dataLog("Considering live variable ", VariableAccessDataDump(m_graph, variable), " at head of block ", *block, "\n");
                     
                     SSACalculator::Variable* ssaVariable = m_ssaVariableForVariable.get(variable);
-                    SSACalculator::Def* def = m_calculator.reachingDefAtHead(block, ssaVariable);
+                    SSACalculator::Def* def = calculator.reachingDefAtHead(block, ssaVariable);
                     if (!def) {
                         // If we are required to insert a Phi, then we won't have a reaching def
                         // at head.
@@ -246,7 +296,7 @@ public:
             // valueForOperand with those Phis. For Phis associated with variables that are not
             // flushed, we also insert a MovHint.
             size_t phiInsertionPoint = 0;
-            for (SSACalculator::Def* phiDef : m_calculator.phisForBlock(block)) {
+            for (SSACalculator::Def* phiDef : calculator.phisForBlock(block)) {
                 VariableAccessData* variable = m_variableForSSAIndex[phiDef->variable()->index()];
                 
                 m_insertionSet.insert(phiInsertionPoint, phiDef->value());
@@ -346,7 +396,7 @@ public:
             NodeOrigin upsilonOrigin = terminal.node->origin;
             for (unsigned successorIndex = block->numSuccessors(); successorIndex--;) {
                 BasicBlock* successorBlock = block->successor(successorIndex);
-                for (SSACalculator::Def* phiDef : m_calculator.phisForBlock(successorBlock)) {
+                for (SSACalculator::Def* phiDef : calculator.phisForBlock(successorBlock)) {
                     Node* phiNode = phiDef->value();
                     SSACalculator::Variable* ssaVariable = phiDef->variable();
                     VariableAccessData* variable = m_variableForSSAIndex[ssaVariable->index()];
@@ -383,23 +433,25 @@ public:
             block->ssa = std::make_unique<BasicBlock::SSAData>(block);
         }
 
-        // FIXME: Support multiple entrypoints in DFG SSA:
-        // https://bugs.webkit.org/show_bug.cgi?id=175396
-        RELEASE_ASSERT(m_graph.m_entrypoints.size() == 1);
-        auto& arguments = m_graph.m_entrypointToArguments.find(m_graph.block(0))->value;
-        m_graph.m_argumentFormats.resize(arguments.size());
-        for (unsigned i = arguments.size(); i--;) {
-            FlushFormat format = FlushedJSValue;
-
-            Node* node = m_argumentMapping.get(arguments[i]);
-            
-            RELEASE_ASSERT(node);
-            format = node->stackAccessData()->format;
-            
-            m_graph.m_argumentFormats[i] = format;
-            arguments[i] = node; // Record the load that loads the arguments for the benefit of exit profiling.
+        for (auto& pair : entrypointIndexToArgumentsBlock) {
+            unsigned entrypointIndex = pair.key;
+            BasicBlock* oldRoot = pair.value;
+            ArgumentsVector& arguments = m_graph.m_entrypointToArguments.find(oldRoot)->value;
+            Vector<FlushFormat> argumentFormats;
+            argumentFormats.reserveInitialCapacity(arguments.size());
+            for (unsigned i = 0; i < arguments.size(); ++i) {
+                Node* node = m_argumentMapping.get(arguments[i]);
+                RELEASE_ASSERT(node);
+                argumentFormats.uncheckedAppend(node->stackAccessData()->format);
+            }
+            m_graph.m_argumentFormats[entrypointIndex] = WTFMove(argumentFormats);
         }
-        
+
+        m_graph.m_entrypointToArguments.clear();
+
+        RELEASE_ASSERT(m_graph.m_isInSSAConversion);
+        m_graph.m_isInSSAConversion = false;
+
         m_graph.m_form = SSA;
 
         if (verbose) {
@@ -411,7 +463,6 @@ public:
     }
 
 private:
-    SSACalculator m_calculator;
     InsertionSet m_insertionSet;
     HashMap<VariableAccessData*, SSACalculator::Variable*> m_ssaVariableForVariable;
     HashMap<Node*, Node*> m_argumentMapping;
@@ -421,11 +472,7 @@ private:
 
 bool performSSAConversion(Graph& graph)
 {
-    RELEASE_ASSERT(!graph.m_isInSSAConversion);
-    graph.m_isInSSAConversion = true;
     bool result = runPhase<SSAConversionPhase>(graph);
-    RELEASE_ASSERT(graph.m_isInSSAConversion);
-    graph.m_isInSSAConversion = false;
     return result;
 }
 
index eae374b..9dc2380 100644 (file)
@@ -311,6 +311,7 @@ bool safeToExecute(AbstractStateType& state, Graph& graph, Node* node)
     case Jump:
     case Branch:
     case Switch:
+    case EntrySwitch:
     case Return:
     case TailCall:
     case DirectTailCall:
@@ -399,6 +400,7 @@ bool safeToExecute(AbstractStateType& state, Graph& graph, Node* node)
     case AtomicsSub:
     case AtomicsXor:
     case AtomicsIsLockFree:
+    case InitializeEntrypointArguments:
         return true;
 
     case ArraySlice:
index db8c036..5437aa6 100644 (file)
@@ -1931,6 +1931,7 @@ void SpeculativeJIT::linkOSREntries(LinkBuffer& linkBuffer)
     }
 
     m_jit.jitCode()->finalizeOSREntrypoints();
+    m_jit.jitCode()->common.finalizeCatchEntrypoints();
 
     ASSERT(osrEntryIndex == m_osrEntryHeads.size());
     
index 80710ae..f3a7f31 100644 (file)
@@ -5650,7 +5650,7 @@ void SpeculativeJIT::compile(Node* node)
         GPRReg tagGPR = tag.gpr();
         GPRReg payloadGPR = payload.gpr();
 
-        JSValue* ptr = &reinterpret_cast<JSValue*>(m_jit.jitCode()->catchOSREntryBuffer->dataBuffer())[node->catchOSREntryIndex()];
+        JSValue* ptr = &reinterpret_cast<JSValue*>(m_jit.jitCode()->common.catchOSREntryBuffer->dataBuffer())[node->catchOSREntryIndex()];
         m_jit.move(CCallHelpers::TrustedImmPtr(ptr), tempGPR);
         m_jit.load32(CCallHelpers::Address(tempGPR, TagOffset), tagGPR);
         m_jit.load32(CCallHelpers::Address(tempGPR, PayloadOffset), payloadGPR);
@@ -5703,6 +5703,8 @@ void SpeculativeJIT::compile(Node* node)
     case AtomicsSub:
     case AtomicsXor:
     case IdentityWithProfile:
+    case InitializeEntrypointArguments:
+    case EntrySwitch:
         DFG_CRASH(m_jit.graph(), node, "unexpected node in DFG backend");
         break;
     }
index 309c47b..e0b9f8c 100644 (file)
@@ -6006,7 +6006,7 @@ void SpeculativeJIT::compile(Node* node)
         break;
 
     case ExtractCatchLocal: {
-        JSValue* ptr = &reinterpret_cast<JSValue*>(m_jit.jitCode()->catchOSREntryBuffer->dataBuffer())[node->catchOSREntryIndex()];
+        JSValue* ptr = &reinterpret_cast<JSValue*>(m_jit.jitCode()->common.catchOSREntryBuffer->dataBuffer())[node->catchOSREntryIndex()];
         GPRTemporary temp(this);
         GPRReg tempGPR = temp.gpr();
         m_jit.move(CCallHelpers::TrustedImmPtr(ptr), tempGPR);
@@ -6113,6 +6113,8 @@ void SpeculativeJIT::compile(Node* node)
 #endif // ENABLE(FTL_JIT)
 
     case LastNodeType:
+    case EntrySwitch:
+    case InitializeEntrypointArguments:
     case Phi:
     case Upsilon:
     case ExtractOSREntryLocal:
index 5a104af..b8d16f6 100644 (file)
@@ -80,6 +80,11 @@ public:
                 applyCounts(data->fallThrough);
                 break;
             }
+            
+            case EntrySwitch: {
+                DFG_CRASH(m_graph, terminal, "Unexpected EntrySwitch in CPS form.");
+                break;
+            }
 
             default:
                 break;
index 1bc1da6..81b027c 100644 (file)
@@ -84,6 +84,10 @@ public:
         // in release builds.
 
         VALIDATE((m_graph.block(0)), m_graph.isEntrypoint(m_graph.block(0)));
+        VALIDATE((m_graph.block(0)), m_graph.block(0) == m_graph.m_entrypoints[0]);
+
+        for (BasicBlock* block : m_graph.m_entrypoints)
+            VALIDATE((block), block->predecessors.isEmpty());
 
         // Validate that all local variables at the head of all entrypoints are dead.
         for (BasicBlock* entrypoint : m_graph.m_entrypoints) {
@@ -549,6 +553,8 @@ private:
                 case PutStack:
                 case KillStack:
                 case GetStack:
+                case EntrySwitch:
+                case InitializeEntrypointArguments:
                     VALIDATE((node), !"unexpected node type in CPS");
                     break;
                 case MaterializeNewObject: {
@@ -639,6 +645,13 @@ private:
         // FIXME: Add more things here.
         // https://bugs.webkit.org/show_bug.cgi?id=123471
         
+        VALIDATE((), m_graph.m_entrypoints.size() == 1);
+        VALIDATE((), m_graph.m_entrypoints[0] == m_graph.block(0));
+        VALIDATE((), !m_graph.m_argumentFormats.isEmpty()); // We always have at least one entrypoint.
+
+        for (unsigned entrypointIndex : m_graph.m_entrypointIndexToCatchBytecodeOffset.keys())
+            VALIDATE((), entrypointIndex > 0); // By convention, 0 is the entrypoint index for the op_enter entrypoint, which can not be in a catch.
+
         for (BlockIndex blockIndex = 0; blockIndex < m_graph.numBlocks(); ++blockIndex) {
             BasicBlock* block = m_graph.block(blockIndex);
             if (!block)
@@ -740,6 +753,14 @@ private:
                     break;
                 }
 
+                case EntrySwitch:
+                    VALIDATE((node), node->entrySwitchData()->cases.size() == m_graph.m_numberOfEntrypoints);
+                    break;
+
+                case InitializeEntrypointArguments:
+                    VALIDATE((node), node->entrypointIndex() < m_graph.m_numberOfEntrypoints);
+                    break;
+
                 default:
                     m_graph.doToChildren(
                         node,
@@ -780,6 +801,8 @@ private:
             getLocalPositions.operand(operand) < setLocalPositions.operand(operand));
     }
     
+    void reportValidationContext() { }
+
     void reportValidationContext(Node* node)
     {
         dataLogF("@%u", node->index());
index d1fa639..1f36c45 100644 (file)
@@ -111,6 +111,7 @@ inline CapabilityLevel canCompile(Node* node)
     case Phi:
     case Upsilon:
     case ExtractOSREntryLocal:
+    case ExtractCatchLocal:
     case LoopHint:
     case SkipScope:
     case GetGlobalObject:
@@ -245,6 +246,7 @@ inline CapabilityLevel canCompile(Node* node)
     case GetMyArgumentByVal:
     case GetMyArgumentByValOutOfBounds:
     case ForwardVarargs:
+    case EntrySwitch:
     case Switch:
     case TypeOf:
     case PutById:
@@ -301,6 +303,7 @@ inline CapabilityLevel canCompile(Node* node)
     case AtomicsSub:
     case AtomicsXor:
     case AtomicsIsLockFree:
+    case InitializeEntrypointArguments:
         // These are OK.
         break;
 
index d58570f..ff2ce56 100644 (file)
@@ -142,6 +142,7 @@ void compile(State& state, Safepoint::Result& safepointResult)
         });
 
     state.finalizer->b3CodeLinkBuffer = std::make_unique<LinkBuffer>(jit, codeBlock, JITCompilationCanFail);
+
     if (state.finalizer->b3CodeLinkBuffer->didFailToAllocate()) {
         state.allocationFailed = true;
         return;
@@ -152,9 +153,18 @@ void compile(State& state, Safepoint::Result& safepointResult)
         codeBlock->setPCToCodeOriginMap(std::make_unique<PCToCodeOriginMap>(PCToCodeOriginMapBuilder(vm, WTFMove(originMap)), *state.finalizer->b3CodeLinkBuffer));
 
     state.generatedFunction = bitwise_cast<GeneratedFunction>(
-        state.finalizer->b3CodeLinkBuffer->entrypoint().executableAddress());
+        state.finalizer->b3CodeLinkBuffer->locationOf(state.proc->entrypointLabel(0)));
     state.jitCode->initializeB3Byproducts(state.proc->releaseByproducts());
 
+    for (auto pair : state.graph.m_entrypointIndexToCatchBytecodeOffset) {
+        unsigned catchBytecodeOffset = pair.value;
+        unsigned entrypointIndex = pair.key;
+        Vector<FlushFormat> argumentFormats = state.graph.m_argumentFormats[entrypointIndex];
+        state.jitCode->common.appendCatchEntrypoint(
+            catchBytecodeOffset, state.finalizer->b3CodeLinkBuffer->locationOf(state.proc->entrypointLabel(entrypointIndex)).executableAddress(), WTFMove(argumentFormats));
+    }
+    state.jitCode->common.finalizeCatchEntrypoints();
+
     if (B3::Air::Disassembler* disassembler = state.proc->code().disassembler()) {
         PrintStream& out = WTF::dataFile();
 
index 12c043e..1e777ad 100644 (file)
@@ -28,6 +28,7 @@
 
 #if ENABLE(FTL_JIT)
 
+#include "AirCode.h"
 #include "AirGenerationContext.h"
 #include "AllowMacroScratchRegisterUsage.h"
 #include "AtomicsObject.h"
@@ -151,6 +152,29 @@ public:
                 "_", codeBlock()->hash());
         } else
             name = "jsBody";
+
+        {
+            m_proc.setNumEntrypoints(m_graph.m_numberOfEntrypoints);
+            CodeBlock* codeBlock = m_graph.m_codeBlock;
+
+            RefPtr<B3::Air::PrologueGenerator> catchPrologueGenerator = createSharedTask<B3::Air::PrologueGeneratorFunction>(
+                [codeBlock] (CCallHelpers& jit, B3::Air::Code& code) {
+                    AllowMacroScratchRegisterUsage allowScratch(jit);
+                    jit.addPtr(CCallHelpers::TrustedImm32(-code.frameSize()), GPRInfo::callFrameRegister, CCallHelpers::stackPointerRegister);
+                    jit.emitSave(code.calleeSaveRegisterAtOffsetList());
+                    jit.emitPutToCallFrameHeader(codeBlock, CallFrameSlot::codeBlock);
+                });
+
+            for (unsigned catchEntrypointIndex : m_graph.m_entrypointIndexToCatchBytecodeOffset.keys()) {
+                RELEASE_ASSERT(catchEntrypointIndex != 0);
+                m_proc.code().setPrologueForEntrypoint(catchEntrypointIndex, catchPrologueGenerator);
+            }
+
+            if (m_graph.m_maxLocalsForCatchOSREntry) {
+                uint32_t numberOfLiveLocals = std::max(*m_graph.m_maxLocalsForCatchOSREntry, 1u); // Make sure we always allocate a non-null catchOSREntryBuffer.
+                m_ftlState.jitCode->common.catchOSREntryBuffer = m_graph.m_vm.scratchBufferForSize(sizeof(JSValue) * numberOfLiveLocals);
+            }
+        }
         
         m_graph.ensureSSADominators();
 
@@ -162,7 +186,8 @@ public:
         // We use prologue frequency for all of the initialization code.
         m_out.setFrequency(1);
         
-        m_prologue = m_out.newBlock();
+        LBasicBlock prologue = m_out.newBlock();
+        LBasicBlock callEntrypointArgumentSpeculations = m_out.newBlock();
         m_handleExceptions = m_out.newBlock();
 
         for (BlockIndex blockIndex = 0; blockIndex < m_graph.numBlocks(); ++blockIndex) {
@@ -176,8 +201,8 @@ public:
         // Back to prologue frequency for any bocks that get sneakily created in the initialization code.
         m_out.setFrequency(1);
         
-        m_out.appendTo(m_prologue, m_handleExceptions);
-        m_out.initializeConstants(m_proc, m_prologue);
+        m_out.appendTo(prologue, callEntrypointArgumentSpeculations);
+        m_out.initializeConstants(m_proc, prologue);
         createPhiVariables();
 
         size_t sizeOfCaptured = sizeof(JSValue) * m_graph.m_nextMachineLocal;
@@ -259,51 +284,57 @@ public:
             });
 
         LBasicBlock firstDFGBasicBlock = lowBlock(m_graph.block(0));
-        // Check Arguments.
-        availabilityMap().clear();
-        availabilityMap().m_locals = Operands<Availability>(codeBlock()->numParameters(), 0);
-        for (unsigned i = codeBlock()->numParameters(); i--;) {
-            availabilityMap().m_locals.argument(i) =
-                Availability(FlushedAt(FlushedJSValue, virtualRegisterForArgument(i)));
-        }
-        m_node = nullptr;
-        m_origin = NodeOrigin(CodeOrigin(0), CodeOrigin(0), true);
-        auto& arguments = m_graph.m_entrypointToArguments.find(m_graph.block(0))->value;
-        for (unsigned i = codeBlock()->numParameters(); i--;) {
-            Node* node = arguments[i];
-            m_out.setOrigin(node);
-            VirtualRegister operand = virtualRegisterForArgument(i);
-            
-            LValue jsValue = m_out.load64(addressFor(operand));
-            
-            if (node) {
-                DFG_ASSERT(m_graph, node, operand == node->stackAccessData()->machineLocal);
-                
-                // This is a hack, but it's an effective one. It allows us to do CSE on the
-                // primordial load of arguments. This assumes that the GetLocal that got put in
-                // place of the original SetArgument doesn't have any effects before it. This
-                // should hold true.
-                m_loadedArgumentValues.add(node, jsValue);
+
+        {
+            Vector<LBasicBlock> successors(m_graph.m_numberOfEntrypoints);
+            successors[0] = callEntrypointArgumentSpeculations;
+            for (unsigned i = 1; i < m_graph.m_numberOfEntrypoints; ++i) {
+                // Currently, the only other entrypoint is an op_catch entrypoint.
+                // We do OSR entry at op_catch, and we prove argument formats before
+                // jumping to FTL code, so we don't need to check argument types here
+                // for these entrypoints.
+                successors[i] = firstDFGBasicBlock;
             }
-            
-            switch (m_graph.m_argumentFormats[i]) {
-            case FlushedInt32:
-                speculate(BadType, jsValueValue(jsValue), node, isNotInt32(jsValue));
-                break;
-            case FlushedBoolean:
-                speculate(BadType, jsValueValue(jsValue), node, isNotBoolean(jsValue));
-                break;
-            case FlushedCell:
-                speculate(BadType, jsValueValue(jsValue), node, isNotCell(jsValue));
-                break;
-            case FlushedJSValue:
-                break;
-            default:
-                DFG_CRASH(m_graph, node, "Bad flush format for argument");
-                break;
+
+            m_out.entrySwitch(successors);
+            m_out.appendTo(callEntrypointArgumentSpeculations, m_handleExceptions);
+
+            m_node = nullptr;
+            m_origin = NodeOrigin(CodeOrigin(0), CodeOrigin(0), true);
+
+            // Check Arguments.
+            availabilityMap().clear();
+            availabilityMap().m_locals = Operands<Availability>(codeBlock()->numParameters(), 0);
+            for (unsigned i = codeBlock()->numParameters(); i--;) {
+                availabilityMap().m_locals.argument(i) =
+                    Availability(FlushedAt(FlushedJSValue, virtualRegisterForArgument(i)));
+            }
+
+            for (unsigned i = codeBlock()->numParameters(); i--;) {
+                MethodOfGettingAValueProfile profile(&m_graph.m_profiledBlock->valueProfileForArgument(i));
+                VirtualRegister operand = virtualRegisterForArgument(i);
+                LValue jsValue = m_out.load64(addressFor(operand));
+                
+                switch (m_graph.m_argumentFormats[0][i]) {
+                case FlushedInt32:
+                    speculate(BadType, jsValueValue(jsValue), profile, isNotInt32(jsValue));
+                    break;
+                case FlushedBoolean:
+                    speculate(BadType, jsValueValue(jsValue), profile, isNotBoolean(jsValue));
+                    break;
+                case FlushedCell:
+                    speculate(BadType, jsValueValue(jsValue), profile, isNotCell(jsValue));
+                    break;
+                case FlushedJSValue:
+                    break;
+                default:
+                    DFG_CRASH(m_graph, nullptr, "Bad flush format for argument");
+                    break;
+                }
             }
+            m_out.jump(firstDFGBasicBlock);
         }
-        m_out.jump(firstDFGBasicBlock);
+
 
         m_out.appendTo(m_handleExceptions, firstDFGBasicBlock);
         Box<CCallHelpers::Label> exceptionHandler = state->exceptionHandler;
@@ -512,6 +543,9 @@ private:
         case ExtractOSREntryLocal:
             compileExtractOSREntryLocal();
             break;
+        case ExtractCatchLocal:
+            compileExtractCatchLocal();
+            break;
         case GetStack:
             compileGetStack();
             break;
@@ -940,6 +974,9 @@ private:
         case DFG::Switch:
             compileSwitch();
             break;
+        case DFG::EntrySwitch:
+            compileEntrySwitch();
+            break;
         case DFG::Return:
             compileReturn();
             break;
@@ -1155,6 +1192,7 @@ private:
         case PutHint:
         case BottomValue:
         case KillStack:
+        case InitializeEntrypointArguments:
             break;
         default:
             DFG_CRASH(m_graph, m_node, "Unrecognized node in FTL backend");
@@ -1525,16 +1563,15 @@ private:
             m_ftlState.jitCode->ftlForOSREntry()->entryBuffer()->dataBuffer());
         setJSValue(m_out.load64(m_out.absolute(buffer + m_node->unlinkedLocal().toLocal())));
     }
+
+    void compileExtractCatchLocal()
+    {
+        EncodedJSValue* buffer = static_cast<EncodedJSValue*>(m_ftlState.jitCode->common.catchOSREntryBuffer->dataBuffer());
+        setJSValue(m_out.load64(m_out.absolute(buffer + m_node->catchOSREntryIndex())));
+    }
     
     void compileGetStack()
     {
-        // GetLocals arise only for captured variables and arguments. For arguments, we might have
-        // already loaded it.
-        if (LValue value = m_loadedArgumentValues.get(m_node)) {
-            setJSValue(value);
-            return;
-        }
-        
         StackAccessData* data = m_node->stackAccessData();
         AbstractValue& value = m_state.variables().operand(data->local);
         
@@ -7791,6 +7828,14 @@ private:
         
         DFG_CRASH(m_graph, m_node, "Bad switch kind");
     }
+
+    void compileEntrySwitch()
+    {
+        Vector<LBasicBlock> successors;
+        for (DFG::BasicBlock* successor : m_node->entrySwitchData()->cases)
+            successors.append(lowBlock(successor));
+        m_out.entrySwitch(successors);
+    }
     
     void compileReturn()
     {
@@ -12611,6 +12656,12 @@ private:
     {
         appendOSRExit(kind, lowValue, highValue, failCondition, m_origin);
     }
+
+    void speculate(
+        ExitKind kind, FormattedValue lowValue, const MethodOfGettingAValueProfile& profile, LValue failCondition)
+    {
+        appendOSRExit(kind, lowValue, profile, failCondition, m_origin);
+    }
     
     void terminate(ExitKind kind)
     {
@@ -14135,16 +14186,29 @@ private:
 
     OSRExitDescriptor* appendOSRExitDescriptor(FormattedValue lowValue, Node* highValue)
     {
+        return appendOSRExitDescriptor(lowValue, m_graph.methodOfGettingAValueProfileFor(m_node, highValue));
+    }
+
+    OSRExitDescriptor* appendOSRExitDescriptor(FormattedValue lowValue, const MethodOfGettingAValueProfile& profile)
+    {
         return &m_ftlState.jitCode->osrExitDescriptors.alloc(
-            lowValue.format(), m_graph.methodOfGettingAValueProfileFor(m_node, highValue),
+            lowValue.format(), profile,
             availabilityMap().m_locals.numberOfArguments(),
             availabilityMap().m_locals.numberOfLocals());
     }
-    
+
     void appendOSRExit(
         ExitKind kind, FormattedValue lowValue, Node* highValue, LValue failCondition, 
         NodeOrigin origin, bool isExceptionHandler = false)
     {
+        return appendOSRExit(kind, lowValue, m_graph.methodOfGettingAValueProfileFor(m_node, highValue),
+            failCondition, origin, isExceptionHandler);
+    }
+    
+    void appendOSRExit(
+        ExitKind kind, FormattedValue lowValue, const MethodOfGettingAValueProfile& profile, LValue failCondition, 
+        NodeOrigin origin, bool isExceptionHandler = false)
+    {
         if (verboseCompilationEnabled()) {
             dataLog("    OSR exit #", m_ftlState.jitCode->osrExitDescriptors.size(), " with availability: ", availabilityMap(), "\n");
             if (!m_availableRecoveries.isEmpty())
@@ -14179,12 +14243,17 @@ private:
             return;
 
         blessSpeculation(
-            m_out.speculate(failCondition), kind, lowValue, highValue, origin);
+            m_out.speculate(failCondition), kind, lowValue, profile, origin);
     }
 
     void blessSpeculation(CheckValue* value, ExitKind kind, FormattedValue lowValue, Node* highValue, NodeOrigin origin)
     {
-        OSRExitDescriptor* exitDescriptor = appendOSRExitDescriptor(lowValue, highValue);
+        blessSpeculation(value, kind, lowValue, m_graph.methodOfGettingAValueProfileFor(m_node, highValue), origin);
+    }
+
+    void blessSpeculation(CheckValue* value, ExitKind kind, FormattedValue lowValue, const MethodOfGettingAValueProfile& profile, NodeOrigin origin)
+    {
+        OSRExitDescriptor* exitDescriptor = appendOSRExitDescriptor(lowValue, profile);
         
         value->appendColdAnys(buildExitArguments(exitDescriptor, origin.forExit, lowValue));
 
@@ -14671,7 +14740,6 @@ private:
     Output m_out;
     Procedure& m_proc;
     
-    LBasicBlock m_prologue;
     LBasicBlock m_handleExceptions;
     HashMap<DFG::BasicBlock*, LBasicBlock> m_blocks;
     
@@ -14688,11 +14756,6 @@ private:
     HashMap<Node*, LoweredNodeValue> m_storageValues;
     HashMap<Node*, LoweredNodeValue> m_doubleValues;
     
-    // This is a bit of a hack. It prevents B3 from having to do CSE on loading of arguments.
-    // It's nice to have these optimizations on our end because we can guarantee them a bit better.
-    // Probably also saves B3 compile time.
-    HashMap<Node*, LValue> m_loadedArgumentValues;
-    
     HashMap<Node*, LValue> m_phis;
     
     LocalOSRAvailabilityCalculator m_availabilityCalculator;
index 41b12fa..cc4bb39 100644 (file)
@@ -861,6 +861,14 @@ void Output::addIncomingToPhi(LValue phi, ValueFromBlock value)
         value.value()->as<B3::UpsilonValue>()->setPhi(phi);
 }
 
+void Output::entrySwitch(const Vector<LBasicBlock>& cases)
+{
+    RELEASE_ASSERT(cases.size() == m_proc.numEntrypoints());
+    m_block->appendNew<Value>(m_proc, EntrySwitch, origin());
+    for (LBasicBlock block : cases)
+        m_block->appendSuccessor(FrequentedBlock(block));
+}
+
 } } // namespace JSC::FTL
 
 #endif // ENABLE(FTL_JIT)
index 64dc072..8439c63 100644 (file)
@@ -429,6 +429,8 @@ public:
         }
     }
 
+    void entrySwitch(const Vector<LBasicBlock>&);
+
     void ret(LValue);
 
     void unreachable();
index d8e57a7..54d43a1 100644 (file)
@@ -42,6 +42,7 @@
 #include "ErrorHandlingScope.h"
 #include "EvalCodeBlock.h"
 #include "ExceptionFuzz.h"
+#include "FTLOSREntry.h"
 #include "FrameTracers.h"
 #include "FunctionCodeBlock.h"
 #include "GetterSetter.h"
@@ -1533,10 +1534,14 @@ char* JIT_OPERATION operationTryOSREnterAtCatch(ExecState* exec, uint32_t byteco
     NativeCallFrameTracer tracer(&vm, exec);
 
     CodeBlock* optimizedReplacement = exec->codeBlock()->replacement();
-    if (optimizedReplacement->jitType() != JITCode::DFGJIT)
-        return nullptr;
-
-    return static_cast<char*>(DFG::prepareCatchOSREntry(exec, optimizedReplacement, bytecodeIndex));
+    switch (optimizedReplacement->jitType()) {
+    case JITCode::DFGJIT:
+    case JITCode::FTLJIT:
+        return static_cast<char*>(DFG::prepareCatchOSREntry(exec, optimizedReplacement, bytecodeIndex));
+    default:
+        break;
+    }
+    return nullptr;
 }
 
 char* JIT_OPERATION operationTryOSREnterAtCatchAndValueProfile(ExecState* exec, uint32_t bytecodeIndex)
@@ -1546,8 +1551,14 @@ char* JIT_OPERATION operationTryOSREnterAtCatchAndValueProfile(ExecState* exec,
 
     CodeBlock* codeBlock = exec->codeBlock();
     CodeBlock* optimizedReplacement = codeBlock->replacement();
-    if (optimizedReplacement->jitType() == JITCode::DFGJIT)
+
+    switch (optimizedReplacement->jitType()) {
+    case JITCode::DFGJIT:
+    case JITCode::FTLJIT:
         return static_cast<char*>(DFG::prepareCatchOSREntry(exec, optimizedReplacement, bytecodeIndex));
+    default:
+        break;
+    }
 
     codeBlock->ensureCatchLivenessIsComputedForBytecodeOffset(bytecodeIndex);
     ValueProfileAndOperandBuffer* buffer = static_cast<ValueProfileAndOperandBuffer*>(codeBlock->instructions()[bytecodeIndex + 3].u.pointer);