op_mul/ArithMul(Untyped,Untyped) should be an IC
authorsbarati@apple.com <sbarati@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 25 Jul 2016 19:04:16 +0000 (19:04 +0000)
committersbarati@apple.com <sbarati@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 25 Jul 2016 19:04:16 +0000 (19:04 +0000)
https://bugs.webkit.org/show_bug.cgi?id=160108

Reviewed by Mark Lam.

This patch makes Mul a type based IC in much the same way that we made
Add a type-based IC. I implemented Mul in the same way. I abstracted the
implementation of the Add IC in the various JITs to allow for it to
work over arbitrary IC snippets. This will make adding Div/Sub/Pow in the
future easy. This patch also adds a new boolean argument to the various
snippet generateFastPath() methods to indicate if we should emit result profiling.
I added this because we want this profiling to be emitted for Mul in
the baseline, but not in the DFG. We used to indicate this through passing
in a nullptr for the ArithProfile, but we no longer do that in the upper
JIT tiers. So we are passing an explicit request from the JIT tier about
whether or not it's worth it for the IC to emit profiling.

We now emit much less code for Mul. Here is some data on the average
Mul snippet/IC size:

           |   JetStream  |  Unity 3D  |
     ------| -------------|--------------
      Old  |  ~280 bytes  | ~280 bytes |
     ------| -------------|--------------
      New  |   210  bytes |  185 bytes |
     ------------------------------------

* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::addJITAddIC):
(JSC::CodeBlock::addJITMulIC):
(JSC::CodeBlock::findStubInfo):
* bytecode/CodeBlock.h:
(JSC::CodeBlock::stubInfoBegin):
(JSC::CodeBlock::stubInfoEnd):
* dfg/DFGSpeculativeJIT.cpp:
(JSC::DFG::GPRTemporary::adopt):
(JSC::DFG::FPRTemporary::FPRTemporary):
(JSC::DFG::SpeculativeJIT::compileValueAdd):
(JSC::DFG::SpeculativeJIT::compileMathIC):
(JSC::DFG::SpeculativeJIT::compileArithMul):
* dfg/DFGSpeculativeJIT.h:
(JSC::DFG::SpeculativeJIT::callOperation):
(JSC::DFG::GPRTemporary::GPRTemporary):
(JSC::DFG::GPRTemporary::operator=):
(JSC::DFG::FPRTemporary::~FPRTemporary):
(JSC::DFG::FPRTemporary::fpr):
* ftl/FTLLowerDFGToB3.cpp:
(JSC::FTL::DFG::LowerDFGToB3::compileToThis):
(JSC::FTL::DFG::LowerDFGToB3::compileValueAdd):
(JSC::FTL::DFG::LowerDFGToB3::compileMathIC):
(JSC::FTL::DFG::LowerDFGToB3::compileArithMul):
* jit/JIT.h:
(JSC::JIT::getSlowCase):
* jit/JITAddGenerator.cpp:
(JSC::JITAddGenerator::generateInline):
(JSC::JITAddGenerator::generateFastPath):
* jit/JITAddGenerator.h:
(JSC::JITAddGenerator::JITAddGenerator):
(JSC::JITAddGenerator::isLeftOperandValidConstant):
(JSC::JITAddGenerator::isRightOperandValidConstant):
* jit/JITArithmetic.cpp:
(JSC::JIT::emit_op_add):
(JSC::JIT::emitSlow_op_add):
(JSC::JIT::emitMathICFast):
(JSC::JIT::emitMathICSlow):
(JSC::JIT::emit_op_mul):
(JSC::JIT::emitSlow_op_mul):
(JSC::JIT::emit_op_sub):
* jit/JITInlines.h:
(JSC::JIT::callOperation):
* jit/JITMathIC.h:
(JSC::JITMathIC::slowPathStartLocation):
(JSC::JITMathIC::slowPathCallLocation):
(JSC::JITMathIC::isLeftOperandValidConstant):
(JSC::JITMathIC::isRightOperandValidConstant):
(JSC::JITMathIC::generateInline):
(JSC::JITMathIC::generateOutOfLine):
* jit/JITMathICForwards.h:
* jit/JITMulGenerator.cpp:
(JSC::JITMulGenerator::generateInline):
(JSC::JITMulGenerator::generateFastPath):
* jit/JITMulGenerator.h:
(JSC::JITMulGenerator::JITMulGenerator):
(JSC::JITMulGenerator::isLeftOperandValidConstant):
(JSC::JITMulGenerator::isRightOperandValidConstant):
(JSC::JITMulGenerator::didEmitFastPath): Deleted.
(JSC::JITMulGenerator::endJumpList): Deleted.
(JSC::JITMulGenerator::slowPathJumpList): Deleted.
* jit/JITOperations.cpp:
* jit/JITOperations.h:

git-svn-id: http://svn.webkit.org/repository/webkit/trunk@203693 268f45cc-cd09-0410-ab3c-d52691b4dbfc

17 files changed:
Source/JavaScriptCore/ChangeLog
Source/JavaScriptCore/bytecode/CodeBlock.cpp
Source/JavaScriptCore/bytecode/CodeBlock.h
Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp
Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h
Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp
Source/JavaScriptCore/jit/JIT.h
Source/JavaScriptCore/jit/JITAddGenerator.cpp
Source/JavaScriptCore/jit/JITAddGenerator.h
Source/JavaScriptCore/jit/JITArithmetic.cpp
Source/JavaScriptCore/jit/JITInlines.h
Source/JavaScriptCore/jit/JITMathIC.h
Source/JavaScriptCore/jit/JITMathICForwards.h
Source/JavaScriptCore/jit/JITMulGenerator.cpp
Source/JavaScriptCore/jit/JITMulGenerator.h
Source/JavaScriptCore/jit/JITOperations.cpp
Source/JavaScriptCore/jit/JITOperations.h

index 157fcf4..f5f2e1e 100644 (file)
@@ -1,3 +1,96 @@
+2016-07-25  Saam Barati  <sbarati@apple.com>
+
+        op_mul/ArithMul(Untyped,Untyped) should be an IC
+        https://bugs.webkit.org/show_bug.cgi?id=160108
+
+        Reviewed by Mark Lam.
+
+        This patch makes Mul a type based IC in much the same way that we made
+        Add a type-based IC. I implemented Mul in the same way. I abstracted the
+        implementation of the Add IC in the various JITs to allow for it to
+        work over arbitrary IC snippets. This will make adding Div/Sub/Pow in the
+        future easy. This patch also adds a new boolean argument to the various
+        snippet generateFastPath() methods to indicate if we should emit result profiling.
+        I added this because we want this profiling to be emitted for Mul in
+        the baseline, but not in the DFG. We used to indicate this through passing
+        in a nullptr for the ArithProfile, but we no longer do that in the upper
+        JIT tiers. So we are passing an explicit request from the JIT tier about
+        whether or not it's worth it for the IC to emit profiling.
+
+        We now emit much less code for Mul. Here is some data on the average
+        Mul snippet/IC size:
+
+                   |   JetStream  |  Unity 3D  |
+             ------| -------------|--------------
+              Old  |  ~280 bytes  | ~280 bytes |
+             ------| -------------|--------------
+              New  |   210  bytes |  185 bytes |
+             ------------------------------------
+
+        * bytecode/CodeBlock.cpp:
+        (JSC::CodeBlock::addJITAddIC):
+        (JSC::CodeBlock::addJITMulIC):
+        (JSC::CodeBlock::findStubInfo):
+        * bytecode/CodeBlock.h:
+        (JSC::CodeBlock::stubInfoBegin):
+        (JSC::CodeBlock::stubInfoEnd):
+        * dfg/DFGSpeculativeJIT.cpp:
+        (JSC::DFG::GPRTemporary::adopt):
+        (JSC::DFG::FPRTemporary::FPRTemporary):
+        (JSC::DFG::SpeculativeJIT::compileValueAdd):
+        (JSC::DFG::SpeculativeJIT::compileMathIC):
+        (JSC::DFG::SpeculativeJIT::compileArithMul):
+        * dfg/DFGSpeculativeJIT.h:
+        (JSC::DFG::SpeculativeJIT::callOperation):
+        (JSC::DFG::GPRTemporary::GPRTemporary):
+        (JSC::DFG::GPRTemporary::operator=):
+        (JSC::DFG::FPRTemporary::~FPRTemporary):
+        (JSC::DFG::FPRTemporary::fpr):
+        * ftl/FTLLowerDFGToB3.cpp:
+        (JSC::FTL::DFG::LowerDFGToB3::compileToThis):
+        (JSC::FTL::DFG::LowerDFGToB3::compileValueAdd):
+        (JSC::FTL::DFG::LowerDFGToB3::compileMathIC):
+        (JSC::FTL::DFG::LowerDFGToB3::compileArithMul):
+        * jit/JIT.h:
+        (JSC::JIT::getSlowCase):
+        * jit/JITAddGenerator.cpp:
+        (JSC::JITAddGenerator::generateInline):
+        (JSC::JITAddGenerator::generateFastPath):
+        * jit/JITAddGenerator.h:
+        (JSC::JITAddGenerator::JITAddGenerator):
+        (JSC::JITAddGenerator::isLeftOperandValidConstant):
+        (JSC::JITAddGenerator::isRightOperandValidConstant):
+        * jit/JITArithmetic.cpp:
+        (JSC::JIT::emit_op_add):
+        (JSC::JIT::emitSlow_op_add):
+        (JSC::JIT::emitMathICFast):
+        (JSC::JIT::emitMathICSlow):
+        (JSC::JIT::emit_op_mul):
+        (JSC::JIT::emitSlow_op_mul):
+        (JSC::JIT::emit_op_sub):
+        * jit/JITInlines.h:
+        (JSC::JIT::callOperation):
+        * jit/JITMathIC.h:
+        (JSC::JITMathIC::slowPathStartLocation):
+        (JSC::JITMathIC::slowPathCallLocation):
+        (JSC::JITMathIC::isLeftOperandValidConstant):
+        (JSC::JITMathIC::isRightOperandValidConstant):
+        (JSC::JITMathIC::generateInline):
+        (JSC::JITMathIC::generateOutOfLine):
+        * jit/JITMathICForwards.h:
+        * jit/JITMulGenerator.cpp:
+        (JSC::JITMulGenerator::generateInline):
+        (JSC::JITMulGenerator::generateFastPath):
+        * jit/JITMulGenerator.h:
+        (JSC::JITMulGenerator::JITMulGenerator):
+        (JSC::JITMulGenerator::isLeftOperandValidConstant):
+        (JSC::JITMulGenerator::isRightOperandValidConstant):
+        (JSC::JITMulGenerator::didEmitFastPath): Deleted.
+        (JSC::JITMulGenerator::endJumpList): Deleted.
+        (JSC::JITMulGenerator::slowPathJumpList): Deleted.
+        * jit/JITOperations.cpp:
+        * jit/JITOperations.h:
+
 2016-07-25  Darin Adler  <darin@apple.com>
 
         Speed up make process slightly by improving "list of files" idiom
index ee81916..59c2516 100644 (file)
@@ -3013,6 +3013,11 @@ JITAddIC* CodeBlock::addJITAddIC()
     return m_addICs.add();
 }
 
+JITMulIC* CodeBlock::addJITMulIC()
+{
+    return m_mulICs.add();
+}
+
 StructureStubInfo* CodeBlock::findStubInfo(CodeOrigin codeOrigin)
 {
     for (StructureStubInfo* stubInfo : m_stubInfos) {
index 480a1cb..d065f59 100644 (file)
@@ -250,6 +250,7 @@ public:
 #if ENABLE(JIT)
     StructureStubInfo* addStubInfo(AccessType);
     JITAddIC* addJITAddIC();
+    JITMulIC* addJITMulIC();
     Bag<StructureStubInfo>::iterator stubInfoBegin() { return m_stubInfos.begin(); }
     Bag<StructureStubInfo>::iterator stubInfoEnd() { return m_stubInfos.end(); }
     
@@ -1014,6 +1015,7 @@ private:
     std::unique_ptr<RegisterAtOffsetList> m_calleeSaveRegisters;
     Bag<StructureStubInfo> m_stubInfos;
     Bag<JITAddIC> m_addICs;
+    Bag<JITMulIC> m_mulICs;
     Bag<ByValInfo> m_byValInfos;
     Bag<CallLinkInfo> m_callLinkInfos;
     SentinelLinkedList<CallLinkInfo, BasicRawSentinelNode<CallLinkInfo>> m_incomingCalls;
index 5f8445d..91c5652 100644 (file)
@@ -44,7 +44,6 @@
 #include "JITBitXorGenerator.h"
 #include "JITDivGenerator.h"
 #include "JITLeftShiftGenerator.h"
-#include "JITMathIC.h"
 #include "JITMulGenerator.h"
 #include "JITRightShiftGenerator.h"
 #include "JITSubGenerator.h"
@@ -1317,6 +1316,16 @@ void GPRTemporary::adopt(GPRTemporary& other)
     other.m_gpr = InvalidGPRReg;
 }
 
+FPRTemporary::FPRTemporary(FPRTemporary&& other)
+{
+    ASSERT(other.m_jit);
+    ASSERT(other.m_fpr != InvalidFPRReg);
+    m_jit = other.m_jit;
+    m_fpr = other.m_fpr;
+
+    other.m_jit = nullptr;
+}
+
 FPRTemporary::FPRTemporary(SpeculativeJIT* jit)
     : m_jit(jit)
     , m_fpr(InvalidFPRReg)
@@ -3336,6 +3345,27 @@ void SpeculativeJIT::compileValueAdd(Node* node)
         return;
     }
 
+#if USE(JSVALUE64)
+    bool needsScratchGPRReg = true;
+    bool needsScratchFPRReg = false;
+#else
+    bool needsScratchGPRReg = true;
+    bool needsScratchFPRReg = true;
+#endif
+
+    JITAddIC* addIC = m_jit.codeBlock()->addJITAddIC();
+    auto repatchingFunction = operationValueAddOptimize;
+    auto nonRepatchingFunction = operationValueAdd;
+    
+    compileMathIC(node, addIC, needsScratchGPRReg, needsScratchFPRReg, repatchingFunction, nonRepatchingFunction);
+}
+
+template <typename Generator, typename RepatchingFunction, typename NonRepatchingFunction>
+void SpeculativeJIT::compileMathIC(Node* node, JITMathIC<Generator>* mathIC, bool needsScratchGPRReg, bool needsScratchFPRReg, RepatchingFunction repatchingFunction, NonRepatchingFunction nonRepatchingFunction)
+{
+    Edge& leftChild = node->child1();
+    Edge& rightChild = node->child2();
+
     Optional<JSValueOperand> left;
     Optional<JSValueOperand> right;
 
@@ -3347,19 +3377,29 @@ void SpeculativeJIT::compileValueAdd(Node* node)
     FPRReg leftFPR = leftNumber.fpr();
     FPRReg rightFPR = rightNumber.fpr();
 
+    GPRReg scratchGPR = InvalidGPRReg;
+    FPRReg scratchFPR = InvalidFPRReg;
+
+    Optional<FPRTemporary> fprScratch;
+    if (needsScratchFPRReg) {
+        fprScratch = FPRTemporary(this);
+        scratchFPR = fprScratch->fpr();
+    }
+
 #if USE(JSVALUE64)
+    Optional<GPRTemporary> gprScratch;
+    if (needsScratchGPRReg) {
+        gprScratch = GPRTemporary(this);
+        scratchGPR = gprScratch->gpr();
+    }
     GPRTemporary result(this);
     JSValueRegs resultRegs = JSValueRegs(result.gpr());
-    GPRTemporary scratch(this);
-    GPRReg scratchGPR = scratch.gpr();
-    FPRReg scratchFPR = InvalidFPRReg;
 #else
     GPRTemporary resultTag(this);
     GPRTemporary resultPayload(this);
     JSValueRegs resultRegs = JSValueRegs(resultPayload.gpr(), resultTag.gpr());
-    GPRReg scratchGPR = resultTag.gpr();
-    FPRTemporary fprScratch(this);
-    FPRReg scratchFPR = fprScratch.fpr();
+    if (needsScratchGPRReg)
+        scratchGPR = resultRegs.tagGPR();
 #endif
 
     SnippetOperand leftOperand(m_state.forNode(leftChild).resultType());
@@ -3374,21 +3414,21 @@ void SpeculativeJIT::compileValueAdd(Node* node)
 
     ASSERT(!leftOperand.isConst() || !rightOperand.isConst());
 
-    if (!leftOperand.isConst()) {
+    if (!mathIC->isLeftOperandValidConstant()) {
         left = JSValueOperand(this, leftChild);
         leftRegs = left->jsValueRegs();
     }
-    if (!rightOperand.isConst()) {
+    if (!mathIC->isRightOperandValidConstant()) {
         right = JSValueOperand(this, rightChild);
         rightRegs = right->jsValueRegs();
     }
 
-    JITAddIC* addIC = m_jit.codeBlock()->addJITAddIC();
     Box<MathICGenerationState> addICGenerationState = Box<MathICGenerationState>::create();
     ArithProfile* arithProfile = m_jit.graph().baselineCodeBlockFor(node->origin.semantic)->arithProfileForBytecodeOffset(node->origin.semantic.bytecodeIndex);
-    addIC->m_generator = JITAddGenerator(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs, leftFPR, rightFPR, scratchGPR, scratchFPR, arithProfile);
+    mathIC->m_generator = Generator(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs, leftFPR, rightFPR, scratchGPR, scratchFPR, arithProfile);
 
-    bool generatedInline = addIC->generateInline(m_jit, *addICGenerationState);
+    bool shouldEmitProfiling = false;
+    bool generatedInline = mathIC->generateInline(m_jit, *addICGenerationState, shouldEmitProfiling);
 
     if (generatedInline) {
         ASSERT(!addICGenerationState->slowPathJumps.empty());
@@ -3406,38 +3446,38 @@ void SpeculativeJIT::compileValueAdd(Node* node)
 
             auto innerLeftRegs = leftRegs;
             auto innerRightRegs = rightRegs;
-            if (leftOperand.isConst()) {
+            if (mathIC->isLeftOperandValidConstant()) {
                 innerLeftRegs = resultRegs;
                 m_jit.moveValue(leftChild->asJSValue(), innerLeftRegs);
-            } else if (rightOperand.isConst()) {
+            } else if (mathIC->isRightOperandValidConstant()) {
                 innerRightRegs = resultRegs;
                 m_jit.moveValue(rightChild->asJSValue(), innerRightRegs);
             }
 
             if (addICGenerationState->shouldSlowPathRepatch)
-                addICGenerationState->slowPathCall = callOperation(operationValueAddOptimize, resultRegs, innerLeftRegs, innerRightRegs, TrustedImmPtr(addIC));
+                addICGenerationState->slowPathCall = callOperation(bitwise_cast<J_JITOperation_EJJMic>(repatchingFunction), resultRegs, innerLeftRegs, innerRightRegs, TrustedImmPtr(mathIC));
             else
-                addICGenerationState->slowPathCall = callOperation(operationValueAdd, resultRegs, innerLeftRegs, innerRightRegs);
+                addICGenerationState->slowPathCall = callOperation(nonRepatchingFunction, resultRegs, innerLeftRegs, innerRightRegs);
 
             silentFill(savePlans);
             m_jit.exceptionCheck();
             m_jit.jump().linkTo(done, &m_jit);
 
             m_jit.addLinkTask([=] (LinkBuffer& linkBuffer) {
-                addIC->finalizeInlineCode(*addICGenerationState, linkBuffer);
+                mathIC->finalizeInlineCode(*addICGenerationState, linkBuffer);
             });
         });
     } else {
-        if (leftOperand.isConst()) {
+        if (mathIC->isLeftOperandValidConstant()) {
             left = JSValueOperand(this, leftChild);
             leftRegs = left->jsValueRegs();
-        } else if (rightOperand.isConst()) {
+        } else if (mathIC->isRightOperandValidConstant()) {
             right = JSValueOperand(this, rightChild);
             rightRegs = right->jsValueRegs();
         }
 
         flushRegisters();
-        callOperation(operationValueAdd, resultRegs, leftRegs, rightRegs);
+        callOperation(nonRepatchingFunction, resultRegs, leftRegs, rightRegs);
         m_jit.exceptionCheck();
     }
 
@@ -4156,78 +4196,18 @@ void SpeculativeJIT::compileArithMul(Node* node)
             return;
         }
 
-        Optional<JSValueOperand> left;
-        Optional<JSValueOperand> right;
-
-        JSValueRegs leftRegs;
-        JSValueRegs rightRegs;
-
-        FPRTemporary leftNumber(this);
-        FPRTemporary rightNumber(this);
-        FPRReg leftFPR = leftNumber.fpr();
-        FPRReg rightFPR = rightNumber.fpr();
-
 #if USE(JSVALUE64)
-        GPRTemporary result(this);
-        JSValueRegs resultRegs = JSValueRegs(result.gpr());
-        GPRTemporary scratch(this);
-        GPRReg scratchGPR = scratch.gpr();
-        FPRReg scratchFPR = InvalidFPRReg;
+        bool needsScratchGPRReg = true;
+        bool needsScratchFPRReg = false;
 #else
-        GPRTemporary resultTag(this);
-        GPRTemporary resultPayload(this);
-        JSValueRegs resultRegs = JSValueRegs(resultPayload.gpr(), resultTag.gpr());
-        GPRReg scratchGPR = resultTag.gpr();
-        FPRTemporary fprScratch(this);
-        FPRReg scratchFPR = fprScratch.fpr();
+        bool needsScratchGPRReg = true;
+        bool needsScratchFPRReg = true;
 #endif
-
-        SnippetOperand leftOperand(m_state.forNode(leftChild).resultType());
-        SnippetOperand rightOperand(m_state.forNode(rightChild).resultType());
-
-        // The snippet generator does not support both operands being constant. If the left
-        // operand is already const, we'll ignore the right operand's constness.
-        if (leftChild->isInt32Constant())
-            leftOperand.setConstInt32(leftChild->asInt32());
-        else if (rightChild->isInt32Constant())
-            rightOperand.setConstInt32(rightChild->asInt32());
-
-        ASSERT(!leftOperand.isConst() || !rightOperand.isConst());
-
-        if (!leftOperand.isPositiveConstInt32()) {
-            left = JSValueOperand(this, leftChild);
-            leftRegs = left->jsValueRegs();
-        }
-        if (!rightOperand.isPositiveConstInt32()) {
-            right = JSValueOperand(this, rightChild);
-            rightRegs = right->jsValueRegs();
-        }
-
-        JITMulGenerator gen(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs,
-            leftFPR, rightFPR, scratchGPR, scratchFPR);
-        gen.generateFastPath(m_jit);
-
-        ASSERT(gen.didEmitFastPath());
-        gen.endJumpList().append(m_jit.jump());
-
-        gen.slowPathJumpList().link(&m_jit);
-        silentSpillAllRegisters(resultRegs);
-
-        if (leftOperand.isPositiveConstInt32()) {
-            leftRegs = resultRegs;
-            m_jit.moveValue(leftChild->asJSValue(), leftRegs);
-        } else if (rightOperand.isPositiveConstInt32()) {
-            rightRegs = resultRegs;
-            m_jit.moveValue(rightChild->asJSValue(), rightRegs);
-        }
-
-        callOperation(operationValueMul, resultRegs, leftRegs, rightRegs);
-
-        silentFillAllRegisters(resultRegs);
-        m_jit.exceptionCheck();
-
-        gen.endJumpList().link(&m_jit);
-        jsValueResult(resultRegs, node);
+        JITMulIC* mulIC = m_jit.codeBlock()->addJITMulIC();
+        auto repatchingFunction = operationValueMulOptimize;
+        auto nonRepatchingFunction = operationValueMul;
+        
+        compileMathIC(node, mulIC, needsScratchGPRReg, needsScratchFPRReg, repatchingFunction, nonRepatchingFunction);
         return;
     }
 
index 02d0870..cce3204 100644 (file)
@@ -36,6 +36,7 @@
 #include "DFGOSRExitJumpPlaceholder.h"
 #include "DFGSilentRegisterSavePlan.h"
 #include "DFGValueSource.h"
+#include "JITMathIC.h"
 #include "JITOperations.h"
 #include "MarkedAllocator.h"
 #include "PutKind.h"
@@ -1288,9 +1289,9 @@ public:
 
 #if USE(JSVALUE64)
 
-    JITCompiler::Call callOperation(J_JITOperation_EJJJaic operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, TrustedImmPtr ptr)
+    JITCompiler::Call callOperation(J_JITOperation_EJJMic operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, TrustedImmPtr mathIC)
     {
-        m_jit.setupArgumentsWithExecState(arg1.gpr(), arg2.gpr(), ptr);
+        m_jit.setupArgumentsWithExecState(arg1.gpr(), arg2.gpr(), mathIC);
         return appendCallSetResult(operation, result.gpr());
     }
 
@@ -1711,9 +1712,9 @@ public:
     }
 #else // USE(JSVALUE32_64)
 
-    JITCompiler::Call callOperation(J_JITOperation_EJJJaic operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, TrustedImmPtr ptr)
+    JITCompiler::Call callOperation(J_JITOperation_EJJMic operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, TrustedImmPtr mathIC)
     {
-        m_jit.setupArgumentsWithExecState(EABI_32BIT_DUMMY_ARG arg1.payloadGPR(), arg1.tagGPR(), arg2.payloadGPR(), arg2.tagGPR(), ptr);
+        m_jit.setupArgumentsWithExecState(EABI_32BIT_DUMMY_ARG arg1.payloadGPR(), arg1.tagGPR(), arg2.payloadGPR(), arg2.tagGPR(), mathIC);
         return appendCallSetResult(operation, result.payloadGPR(), result.tagGPR());
     }
 
@@ -2451,6 +2452,9 @@ public:
     void emitUntypedRightShiftBitOp(Node*);
     void compileShiftOp(Node*);
 
+    template <typename Generator, typename RepatchingFunction, typename NonRepatchingFunction>
+    void compileMathIC(Node*, JITMathIC<Generator>*, bool needsScratchGPRReg, bool needsScratchFPRReg, RepatchingFunction, NonRepatchingFunction);
+
     void compileValueAdd(Node*);
     void compileArithAdd(Node*);
     void compileMakeRope(Node*);
@@ -3045,6 +3049,16 @@ public:
 
     GPRTemporary(GPRTemporary& other) = delete;
 
+    GPRTemporary(GPRTemporary&& other)
+    {
+        ASSERT(other.m_jit);
+        ASSERT(other.m_gpr != InvalidGPRReg);
+        m_jit = other.m_jit;
+        m_gpr = other.m_gpr;
+        other.m_jit = nullptr;
+        other.m_gpr = InvalidGPRReg;
+    }
+
     GPRTemporary& operator=(GPRTemporary&& other)
     {
         ASSERT(!m_jit);
@@ -3093,7 +3107,9 @@ private:
 };
 
 class FPRTemporary {
+    WTF_MAKE_NONCOPYABLE(FPRTemporary);
 public:
+    FPRTemporary(FPRTemporary&&);
     FPRTemporary(SpeculativeJIT*);
     FPRTemporary(SpeculativeJIT*, SpeculateDoubleOperand&);
     FPRTemporary(SpeculativeJIT*, SpeculateDoubleOperand&, SpeculateDoubleOperand&);
@@ -3103,11 +3119,13 @@ public:
 
     ~FPRTemporary()
     {
-        m_jit->unlock(fpr());
+        if (LIKELY(m_jit))
+            m_jit->unlock(fpr());
     }
 
     FPRReg fpr() const
     {
+        ASSERT(m_jit);
         ASSERT(m_fpr != InvalidFPRReg);
         return m_fpr;
     }
index 9b7cd9e..d0a8992 100644 (file)
@@ -1536,9 +1536,18 @@ private:
         m_out.appendTo(continuation, lastNext);
         setJSValue(m_out.phi(Int64, fastResult, slowResult));
     }
-    
+
     void compileValueAdd()
     {
+        JITAddIC* addIC = codeBlock()->addJITAddIC();
+        auto repatchingFunction = operationValueAddOptimize;
+        auto nonRepatchingFunction = operationValueAdd;
+        compileMathIC(addIC, repatchingFunction, nonRepatchingFunction);
+    }
+
+    template <typename Generator>
+    void compileMathIC(JITMathIC<Generator>* mathIC, FunctionPtr repatchingFunction, FunctionPtr nonRepatchingFunction)
+    {
         Node* node = m_node;
         
         LValue left = lowJSValue(node->child1());
@@ -1565,42 +1574,42 @@ private:
                 Box<CCallHelpers::JumpList> exceptions =
                     exceptionHandle->scheduleExitCreation(params)->jumps(jit);
 
-                JITAddIC* addIC = jit.codeBlock()->addJITAddIC();
-                Box<MathICGenerationState> addICGenerationState = Box<MathICGenerationState>::create();
+                Box<MathICGenerationState> mathICGenerationState = Box<MathICGenerationState>::create();
                 ArithProfile* arithProfile = state->graph.baselineCodeBlockFor(node->origin.semantic)->arithProfileForBytecodeOffset(node->origin.semantic.bytecodeIndex);
-                addIC->m_generator = JITAddGenerator(leftOperand, rightOperand, JSValueRegs(params[0].gpr()),
+                mathIC->m_generator = Generator(leftOperand, rightOperand, JSValueRegs(params[0].gpr()),
                     JSValueRegs(params[1].gpr()), JSValueRegs(params[2].gpr()), params.fpScratch(0),
                     params.fpScratch(1), params.gpScratch(0), InvalidFPRReg, arithProfile);
 
-                bool generatedInline = addIC->generateInline(jit, *addICGenerationState);
+                bool shouldEmitProfiling = false;
+                bool generatedInline = mathIC->generateInline(jit, *mathICGenerationState, shouldEmitProfiling);
 
                 if (generatedInline) {
-                    ASSERT(!addICGenerationState->slowPathJumps.empty());
+                    ASSERT(!mathICGenerationState->slowPathJumps.empty());
                     auto done = jit.label();
                     params.addLatePath([=] (CCallHelpers& jit) {
                         AllowMacroScratchRegisterUsage allowScratch(jit);
-                        addICGenerationState->slowPathJumps.link(&jit);
-                        addICGenerationState->slowPathStart = jit.label();
+                        mathICGenerationState->slowPathJumps.link(&jit);
+                        mathICGenerationState->slowPathStart = jit.label();
 
-                        if (addICGenerationState->shouldSlowPathRepatch) {
+                        if (mathICGenerationState->shouldSlowPathRepatch) {
                             SlowPathCall call = callOperation(*state, params.unavailableRegisters(), jit, node->origin.semantic, exceptions.get(),
-                                operationValueAddOptimize, params[0].gpr(), params[1].gpr(), params[2].gpr(), CCallHelpers::TrustedImmPtr(addIC));
-                            addICGenerationState->slowPathCall = call.call();
+                                repatchingFunction, params[0].gpr(), params[1].gpr(), params[2].gpr(), CCallHelpers::TrustedImmPtr(mathIC));
+                            mathICGenerationState->slowPathCall = call.call();
                         } else {
                             SlowPathCall call = callOperation(*state, params.unavailableRegisters(), jit, node->origin.semantic,
-                                exceptions.get(), operationValueAdd, params[0].gpr(), params[1].gpr(), params[2].gpr());
-                            addICGenerationState->slowPathCall = call.call();
+                                exceptions.get(), nonRepatchingFunction, params[0].gpr(), params[1].gpr(), params[2].gpr());
+                            mathICGenerationState->slowPathCall = call.call();
                         }
                         jit.jump().linkTo(done, &jit);
 
                         jit.addLinkTask([=] (LinkBuffer& linkBuffer) {
-                            addIC->finalizeInlineCode(*addICGenerationState, linkBuffer);
+                            mathIC->finalizeInlineCode(*mathICGenerationState, linkBuffer);
                         });
                     });
                 } else {
                     callOperation(
                         *state, params.unavailableRegisters(), jit, node->origin.semantic, exceptions.get(),
-                        operationValueAdd, params[0].gpr(), params[1].gpr(), params[2].gpr());
+                        nonRepatchingFunction, params[0].gpr(), params[1].gpr(), params[2].gpr());
                 }
             });
 
@@ -1762,7 +1771,10 @@ private:
         }
 
         case UntypedUse: {
-            emitBinarySnippet<JITMulGenerator>(operationValueMul);
+            JITMulIC* mulIC = codeBlock()->addJITMulIC();
+            auto repatchingFunction = operationValueMulOptimize;
+            auto nonRepatchingFunction = operationValueMul;
+            compileMathIC(mulIC, repatchingFunction, nonRepatchingFunction);
             break;
         }
 
index c94b740..90580bd 100644 (file)
@@ -711,6 +711,12 @@ namespace JSC {
         bool isOperandConstantInt(int src);
         bool isOperandConstantChar(int src);
 
+        template <typename Generator, typename ProfiledFunction, typename NonProfiledFunction>
+        void emitMathICFast(JITMathIC<Generator>*, Instruction*, ProfiledFunction, NonProfiledFunction);
+
+        template <typename Generator, typename ProfiledRepatchFunction, typename ProfiledFunction, typename RepatchFunction>
+        void emitMathICSlow(JITMathIC<Generator>*, Instruction*, ProfiledRepatchFunction, ProfiledFunction, RepatchFunction);
+
         Jump getSlowCase(Vector<SlowCaseEntry>::iterator& iter)
         {
             return iter++->from;
@@ -765,8 +771,8 @@ namespace JSC {
         MacroAssembler::Call callOperation(J_JITOperation_EJJ, int, GPRReg, GPRReg);
         MacroAssembler::Call callOperation(J_JITOperation_EJJArp, JSValueRegs, JSValueRegs, JSValueRegs, ArithProfile*);
         MacroAssembler::Call callOperation(J_JITOperation_EJJ, JSValueRegs, JSValueRegs, JSValueRegs);
-        MacroAssembler::Call callOperation(J_JITOperation_EJJArpJaic, JSValueRegs, JSValueRegs, JSValueRegs, ArithProfile*, JITAddIC*);
-        MacroAssembler::Call callOperation(J_JITOperation_EJJJaic, JSValueRegs, JSValueRegs, JSValueRegs, JITAddIC*);
+        MacroAssembler::Call callOperation(J_JITOperation_EJJArpMic, JSValueRegs, JSValueRegs, JSValueRegs, ArithProfile*, TrustedImmPtr);
+        MacroAssembler::Call callOperation(J_JITOperation_EJJMic, JSValueRegs, JSValueRegs, JSValueRegs, TrustedImmPtr);
         MacroAssembler::Call callOperation(J_JITOperation_EJJAp, int, GPRReg, GPRReg, ArrayProfile*);
         MacroAssembler::Call callOperation(J_JITOperation_EJJBy, int, GPRReg, GPRReg, ByValInfo*);
         MacroAssembler::Call callOperation(Z_JITOperation_EJOJ, GPRReg, GPRReg, GPRReg);
@@ -956,8 +962,8 @@ namespace JSC {
 
         PCToCodeOriginMapBuilder m_pcToCodeOriginMapBuilder;
 
-        HashMap<Instruction*, JITAddIC*> m_instructionToJITAddIC;
-        HashMap<Instruction*, MathICGenerationState> m_instructionToJITAddICGenerationState;
+        HashMap<Instruction*, void*> m_instructionToMathIC;
+        HashMap<Instruction*, MathICGenerationState> m_instructionToMathICGenerationState;
 
         bool m_canBeOptimized;
         bool m_canBeOptimizedOrInlined;
index a9c2603..6515a25 100644 (file)
@@ -43,6 +43,8 @@ JITMathICInlineResult JITAddGenerator::generateInline(CCallHelpers& jit, MathICG
         lhs = m_arithProfile->lhsObservedType();
         rhs = m_arithProfile->rhsObservedType();
         if (lhs.isEmpty() || rhs.isEmpty()) {
+            // FIXME: ICs should be able to repatch without emitting an inline path:
+            // https://bugs.webkit.org/show_bug.cgi?id=160110
             lhs = ObservedType().withInt32();
             rhs = ObservedType().withInt32();
         }
@@ -71,7 +73,7 @@ JITMathICInlineResult JITAddGenerator::generateInline(CCallHelpers& jit, MathICG
     return JITMathICInlineResult::GenerateFullSnippet;
 }
 
-bool JITAddGenerator::generateFastPath(CCallHelpers& jit, CCallHelpers::JumpList& endJumpList, CCallHelpers::JumpList& slowPathJumpList)
+bool JITAddGenerator::generateFastPath(CCallHelpers& jit, CCallHelpers::JumpList& endJumpList, CCallHelpers::JumpList& slowPathJumpList, bool shouldEmitProfiling)
 {
     ASSERT(m_scratchGPR != InvalidGPRReg);
     ASSERT(m_scratchGPR != m_left.payloadGPR());
@@ -166,7 +168,7 @@ bool JITAddGenerator::generateFastPath(CCallHelpers& jit, CCallHelpers::JumpList
 
     // Do doubleVar + doubleVar.
     jit.addDouble(m_rightFPR, m_leftFPR);
-    if (m_arithProfile)
+    if (m_arithProfile && shouldEmitProfiling)
         m_arithProfile->emitSetDouble(jit);
         
     jit.boxDouble(m_leftFPR, m_result);
index b8917d8..b5cb1b1 100644 (file)
@@ -61,7 +61,10 @@ public:
     }
 
     JITMathICInlineResult generateInline(CCallHelpers&, MathICGenerationState&);
-    bool generateFastPath(CCallHelpers&, CCallHelpers::JumpList& endJumpList, CCallHelpers::JumpList& slowPathJumpList);
+    bool generateFastPath(CCallHelpers&, CCallHelpers::JumpList& endJumpList, CCallHelpers::JumpList& slowPathJumpList, bool shouldEmitProfiling);
+
+    bool isLeftOperandValidConstant() const { return m_leftOperand.isConstInt32(); }
+    bool isRightOperandValidConstant() const { return m_rightOperand.isConstInt32(); }
 
 private:
     SnippetOperand m_leftOperand;
index 1e4ad80..0f3c389 100644 (file)
@@ -684,6 +684,22 @@ ALWAYS_INLINE static OperandTypes getOperandTypes(Instruction* instruction)
 
 void JIT::emit_op_add(Instruction* currentInstruction)
 {
+    JITAddIC* addIC = m_codeBlock->addJITAddIC();
+    m_instructionToMathIC.add(currentInstruction, addIC);
+    emitMathICFast(addIC, currentInstruction, operationValueAddProfiled, operationValueAdd);
+}
+
+void JIT::emitSlow_op_add(Instruction* currentInstruction, Vector<SlowCaseEntry>::iterator& iter)
+{
+    linkAllSlowCasesForBytecodeOffset(m_slowCases, iter, m_bytecodeOffset);
+
+    JITAddIC* addIC = bitwise_cast<JITAddIC*>(m_instructionToMathIC.get(currentInstruction));
+    emitMathICSlow(addIC, currentInstruction, operationValueAddProfiledOptimize, operationValueAddProfiled, operationValueAddOptimize);
+}
+
+template <typename Generator, typename ProfiledFunction, typename NonProfiledFunction>
+void JIT::emitMathICFast(JITMathIC<Generator>* mathIC, Instruction* currentInstruction, ProfiledFunction profiledFunction, NonProfiledFunction nonProfiledFunction)
+{
     int result = currentInstruction[1].u.operand;
     int op1 = currentInstruction[2].u.operand;
     int op2 = currentInstruction[3].u.operand;
@@ -692,9 +708,9 @@ void JIT::emit_op_add(Instruction* currentInstruction)
     OperandTypes types = getOperandTypes(copiedInstruction(currentInstruction));
     JSValueRegs leftRegs = JSValueRegs(regT0);
     JSValueRegs rightRegs = JSValueRegs(regT1);
-    JSValueRegs resultRegs = leftRegs;
-    GPRReg scratchGPR = regT2;
-    FPRReg scratchFPR = InvalidFPRReg;
+    JSValueRegs resultRegs = JSValueRegs(regT2);
+    GPRReg scratchGPR = regT3;
+    FPRReg scratchFPR = fpRegT2;
 #else
     OperandTypes types = getOperandTypes(currentInstruction);
     JSValueRegs leftRegs = JSValueRegs(regT1, regT0);
@@ -718,40 +734,35 @@ void JIT::emit_op_add(Instruction* currentInstruction)
 
     RELEASE_ASSERT(!leftOperand.isConst() || !rightOperand.isConst());
 
-    if (!leftOperand.isConst())
+    if (!mathIC->isLeftOperandValidConstant())
         emitGetVirtualRegister(op1, leftRegs);
-    if (!rightOperand.isConst())
+    if (!mathIC->isRightOperandValidConstant())
         emitGetVirtualRegister(op2, rightRegs);
 
-    JITAddIC* addIC = m_codeBlock->addJITAddIC();
-
-    m_instructionToJITAddIC.add(currentInstruction, addIC);
-    MathICGenerationState& addICGenerationState = m_instructionToJITAddICGenerationState.add(currentInstruction, MathICGenerationState()).iterator->value;
+    MathICGenerationState& mathICGenerationState = m_instructionToMathICGenerationState.add(currentInstruction, MathICGenerationState()).iterator->value;
 
-    addIC->m_generator = JITAddGenerator(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs, fpRegT0, fpRegT1, scratchGPR, scratchFPR, arithProfile);
+    mathIC->m_generator = Generator(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs, fpRegT0, fpRegT1, scratchGPR, scratchFPR, arithProfile);
 
-    bool generatedInlineCode = addIC->generateInline(*this, addICGenerationState);
+    bool generatedInlineCode = mathIC->generateInline(*this, mathICGenerationState);
     if (!generatedInlineCode) {
         if (leftOperand.isConst())
             emitGetVirtualRegister(op1, leftRegs);
         else if (rightOperand.isConst())
             emitGetVirtualRegister(op2, rightRegs);
-
         if (arithProfile)
-            callOperation(operationValueAddProfiled, resultRegs, leftRegs, rightRegs, arithProfile);
+            callOperation(profiledFunction, resultRegs, leftRegs, rightRegs, arithProfile);
         else
-            callOperation(operationValueAdd, resultRegs, leftRegs, rightRegs);
+            callOperation(nonProfiledFunction, resultRegs, leftRegs, rightRegs);
     } else
-        addSlowCase(addICGenerationState.slowPathJumps);
+        addSlowCase(mathICGenerationState.slowPathJumps);
     emitPutVirtualRegister(result, resultRegs);
 }
 
-void JIT::emitSlow_op_add(Instruction* currentInstruction, Vector<SlowCaseEntry>::iterator& iter)
+template <typename Generator, typename ProfiledRepatchFunction, typename ProfiledFunction, typename RepatchFunction>
+void JIT::emitMathICSlow(JITMathIC<Generator>* mathIC, Instruction* currentInstruction, ProfiledRepatchFunction profiledRepatchFunction, ProfiledFunction profiledFunction, RepatchFunction repatchFunction)
 {
-    linkAllSlowCasesForBytecodeOffset(m_slowCases, iter, m_bytecodeOffset);
-
-    MathICGenerationState& addICGenerationState = m_instructionToJITAddICGenerationState.find(currentInstruction)->value;
-    addICGenerationState.slowPathStart = label();
+    MathICGenerationState& mathICGenerationState = m_instructionToMathICGenerationState.find(currentInstruction)->value;
+    mathICGenerationState.slowPathStart = label();
 
     int result = currentInstruction[1].u.operand;
     int op1 = currentInstruction[2].u.operand;
@@ -761,7 +772,7 @@ void JIT::emitSlow_op_add(Instruction* currentInstruction, Vector<SlowCaseEntry>
     OperandTypes types = getOperandTypes(copiedInstruction(currentInstruction));
     JSValueRegs leftRegs = JSValueRegs(regT0);
     JSValueRegs rightRegs = JSValueRegs(regT1);
-    JSValueRegs resultRegs = leftRegs;
+    JSValueRegs resultRegs = JSValueRegs(regT2);
 #else
     OperandTypes types = getOperandTypes(currentInstruction);
     JSValueRegs leftRegs = JSValueRegs(regT1, regT0);
@@ -777,25 +788,24 @@ void JIT::emitSlow_op_add(Instruction* currentInstruction, Vector<SlowCaseEntry>
     else if (isOperandConstantInt(op2))
         rightOperand.setConstInt32(getOperandConstantInt(op2));
 
-    if (leftOperand.isConst())
+    if (mathIC->isLeftOperandValidConstant())
         emitGetVirtualRegister(op1, leftRegs);
-    if (rightOperand.isConst())
+    if (mathIC->isRightOperandValidConstant())
         emitGetVirtualRegister(op2, rightRegs);
 
-    JITAddIC* addIC = m_instructionToJITAddIC.get(currentInstruction);
     if (shouldEmitProfiling()) {
         ArithProfile& arithProfile = m_codeBlock->arithProfileForPC(currentInstruction);
-        if (addICGenerationState.shouldSlowPathRepatch)
-            addICGenerationState.slowPathCall = callOperation(operationValueAddProfiledOptimize, resultRegs, leftRegs, rightRegs, &arithProfile, addIC);
+        if (mathICGenerationState.shouldSlowPathRepatch)
+            mathICGenerationState.slowPathCall = callOperation(bitwise_cast<J_JITOperation_EJJArpMic>(profiledRepatchFunction), resultRegs, leftRegs, rightRegs, &arithProfile, TrustedImmPtr(mathIC));
         else
-            addICGenerationState.slowPathCall = callOperation(operationValueAddProfiled, resultRegs, leftRegs, rightRegs, &arithProfile);
+            mathICGenerationState.slowPathCall = callOperation(profiledFunction, resultRegs, leftRegs, rightRegs, &arithProfile);
     } else
-        addICGenerationState.slowPathCall = callOperation(operationValueAddOptimize, resultRegs, leftRegs, rightRegs, addIC);
+        mathICGenerationState.slowPathCall = callOperation(bitwise_cast<J_JITOperation_EJJMic>(repatchFunction), resultRegs, leftRegs, rightRegs, TrustedImmPtr(mathIC));
     emitPutVirtualRegister(result, resultRegs);
 
     addLinkTask([=] (LinkBuffer& linkBuffer) {
-        MathICGenerationState& addICGenerationState = m_instructionToJITAddICGenerationState.find(currentInstruction)->value;
-        addIC->finalizeInlineCode(addICGenerationState, linkBuffer);
+        MathICGenerationState& mathICGenerationState = m_instructionToMathICGenerationState.find(currentInstruction)->value;
+        mathIC->finalizeInlineCode(mathICGenerationState, linkBuffer);
     });
 }
 
@@ -875,112 +885,17 @@ void JIT::emitSlow_op_div(Instruction* currentInstruction, Vector<SlowCaseEntry>
 
 void JIT::emit_op_mul(Instruction* currentInstruction)
 {
-    int result = currentInstruction[1].u.operand;
-    int op1 = currentInstruction[2].u.operand;
-    int op2 = currentInstruction[3].u.operand;
-
-#if USE(JSVALUE64)
-    OperandTypes types = getOperandTypes(copiedInstruction(currentInstruction));
-    JSValueRegs leftRegs = JSValueRegs(regT0);
-    JSValueRegs rightRegs = JSValueRegs(regT1);
-    JSValueRegs resultRegs = JSValueRegs(regT2);
-    GPRReg scratchGPR = regT3;
-    FPRReg scratchFPR = InvalidFPRReg;
-#else
-    OperandTypes types = getOperandTypes(currentInstruction);
-    JSValueRegs leftRegs = JSValueRegs(regT1, regT0);
-    JSValueRegs rightRegs = JSValueRegs(regT3, regT2);
-    JSValueRegs resultRegs = leftRegs;
-    GPRReg scratchGPR = regT4;
-    FPRReg scratchFPR = fpRegT2;
-#endif
-
-    ArithProfile* arithProfile = nullptr;
-    if (shouldEmitProfiling())
-        arithProfile = &m_codeBlock->arithProfileForPC(currentInstruction);
-
-    SnippetOperand leftOperand(types.first());
-    SnippetOperand rightOperand(types.second());
-
-    if (isOperandConstantInt(op1))
-        leftOperand.setConstInt32(getOperandConstantInt(op1));
-    else if (isOperandConstantInt(op2))
-        rightOperand.setConstInt32(getOperandConstantInt(op2));
-
-    RELEASE_ASSERT(!leftOperand.isConst() || !rightOperand.isConst());
-
-    if (!leftOperand.isPositiveConstInt32())
-        emitGetVirtualRegister(op1, leftRegs);
-    if (!rightOperand.isPositiveConstInt32())
-        emitGetVirtualRegister(op2, rightRegs);
-
-    JITMulGenerator gen(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs,
-        fpRegT0, fpRegT1, scratchGPR, scratchFPR, arithProfile);
-
-    gen.generateFastPath(*this);
-
-    if (gen.didEmitFastPath()) {
-        gen.endJumpList().link(this);
-        emitPutVirtualRegister(result, resultRegs);
-
-        addSlowCase(gen.slowPathJumpList());
-    } else {
-        ASSERT(gen.endJumpList().empty());
-        ASSERT(gen.slowPathJumpList().empty());
-        if (arithProfile) {
-            if (leftOperand.isPositiveConstInt32())
-                emitGetVirtualRegister(op1, leftRegs);
-            if (rightOperand.isPositiveConstInt32())
-                emitGetVirtualRegister(op2, rightRegs);
-            callOperation(operationValueMulProfiled, resultRegs, leftRegs, rightRegs, arithProfile);
-            emitPutVirtualRegister(result, resultRegs);
-        } else {
-            JITSlowPathCall slowPathCall(this, currentInstruction, slow_path_mul);
-            slowPathCall.call();
-        }
-    }
+    JITMulIC* mulIC = m_codeBlock->addJITMulIC();
+    m_instructionToMathIC.add(currentInstruction, mulIC);
+    emitMathICFast(mulIC, currentInstruction, operationValueMulProfiled, operationValueMul);
 }
 
 void JIT::emitSlow_op_mul(Instruction* currentInstruction, Vector<SlowCaseEntry>::iterator& iter)
 {
     linkAllSlowCasesForBytecodeOffset(m_slowCases, iter, m_bytecodeOffset);
-    
-    int result = currentInstruction[1].u.operand;
-    int op1 = currentInstruction[2].u.operand;
-    int op2 = currentInstruction[3].u.operand;
-
-#if USE(JSVALUE64)
-    OperandTypes types = getOperandTypes(copiedInstruction(currentInstruction));
-    JSValueRegs leftRegs = JSValueRegs(regT0);
-    JSValueRegs rightRegs = JSValueRegs(regT1);
-    JSValueRegs resultRegs = leftRegs;
-#else
-    OperandTypes types = getOperandTypes(currentInstruction);
-    JSValueRegs leftRegs = JSValueRegs(regT1, regT0);
-    JSValueRegs rightRegs = JSValueRegs(regT3, regT2);
-    JSValueRegs resultRegs = leftRegs;
-#endif
-
-    SnippetOperand leftOperand(types.first());
-    SnippetOperand rightOperand(types.second());
 
-    if (isOperandConstantInt(op1))
-        leftOperand.setConstInt32(getOperandConstantInt(op1));
-    else if (isOperandConstantInt(op2))
-        rightOperand.setConstInt32(getOperandConstantInt(op2));
-
-    if (shouldEmitProfiling()) {
-        if (leftOperand.isPositiveConstInt32())
-            emitGetVirtualRegister(op1, leftRegs);
-        if (rightOperand.isPositiveConstInt32())
-            emitGetVirtualRegister(op2, rightRegs);
-        ArithProfile& arithProfile = m_codeBlock->arithProfileForPC(currentInstruction);
-        callOperation(operationValueMulProfiled, resultRegs, leftRegs, rightRegs, &arithProfile);
-        emitPutVirtualRegister(result, resultRegs);
-    } else {
-        JITSlowPathCall slowPathCall(this, currentInstruction, slow_path_mul);
-        slowPathCall.call();
-    }
+    JITMulIC* mulIC = bitwise_cast<JITMulIC*>(m_instructionToMathIC.get(currentInstruction));
+    emitMathICSlow(mulIC, currentInstruction, operationValueMulProfiledOptimize, operationValueMulProfiled, operationValueMulOptimize);
 }
 
 void JIT::emit_op_sub(Instruction* currentInstruction)
index 31b1d4d..b7b9131 100644 (file)
@@ -432,17 +432,17 @@ ALWAYS_INLINE MacroAssembler::Call JIT::callOperation(J_JITOperation_EJJArp oper
     return call;
 }
 
-ALWAYS_INLINE MacroAssembler::Call JIT::callOperation(J_JITOperation_EJJArpJaic operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, ArithProfile* arithProfile, JITAddIC* addIC)
+ALWAYS_INLINE MacroAssembler::Call JIT::callOperation(J_JITOperation_EJJArpMic operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, ArithProfile* arithProfile, TrustedImmPtr mathIC)
 {
-    setupArgumentsWithExecState(arg1, arg2, TrustedImmPtr(arithProfile), TrustedImmPtr(addIC));
+    setupArgumentsWithExecState(arg1, arg2, TrustedImmPtr(arithProfile), mathIC);
     Call call = appendCallWithExceptionCheck(operation);
     setupResults(result);
     return call;
 }
 
-ALWAYS_INLINE MacroAssembler::Call JIT::callOperation(J_JITOperation_EJJJaic operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, JITAddIC* addIC)
+ALWAYS_INLINE MacroAssembler::Call JIT::callOperation(J_JITOperation_EJJMic operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, TrustedImmPtr mathIC)
 {
-    setupArgumentsWithExecState(arg1, arg2, TrustedImmPtr(addIC));
+    setupArgumentsWithExecState(arg1, arg2, mathIC);
     Call call = appendCallWithExceptionCheck(operation);
     setupResults(result);
     return call;
index 4daae70..78cac77 100644 (file)
@@ -30,6 +30,7 @@
 #include "CCallHelpers.h"
 #include "JITAddGenerator.h"
 #include "JITMathICInlineResult.h"
+#include "JITMulGenerator.h"
 #include "LinkBuffer.h"
 #include "Repatch.h"
 #include "SnippetOperand.h"
@@ -54,7 +55,10 @@ public:
     CodeLocationLabel slowPathStartLocation() { return m_inlineStart.labelAtOffset(m_deltaFromStartToSlowPathStart); }
     CodeLocationCall slowPathCallLocation() { return m_inlineStart.callAtOffset(m_deltaFromStartToSlowPathCallLocation); }
 
-    bool generateInline(CCallHelpers& jit, MathICGenerationState& state)
+    bool isLeftOperandValidConstant() const { return m_generator.isLeftOperandValidConstant(); }
+    bool isRightOperandValidConstant() const { return m_generator.isRightOperandValidConstant(); }
+
+    bool generateInline(CCallHelpers& jit, MathICGenerationState& state, bool shouldEmitProfiling = true)
     {
         state.fastPathStart = jit.label();
         size_t startSize = jit.m_assembler.buffer().codeSize();
@@ -73,7 +77,7 @@ public:
         }
         case JITMathICInlineResult::GenerateFullSnippet: {
             MacroAssembler::JumpList endJumpList;
-            bool result = m_generator.generateFastPath(jit, endJumpList, state.slowPathJumps);
+            bool result = m_generator.generateFastPath(jit, endJumpList, state.slowPathJumps, shouldEmitProfiling);
             if (result) {
                 state.fastPathEnd = jit.label();
                 state.shouldSlowPathRepatch = false;
@@ -104,7 +108,9 @@ public:
 
             MacroAssembler::JumpList endJumpList; 
             MacroAssembler::JumpList slowPathJumpList; 
-            bool emittedFastPath = m_generator.generateFastPath(jit, endJumpList, slowPathJumpList);
+
+            bool shouldEmitProfiling = !JITCode::isOptimizingJIT(codeBlock->jitType());
+            bool emittedFastPath = m_generator.generateFastPath(jit, endJumpList, slowPathJumpList, shouldEmitProfiling);
             if (!emittedFastPath)
                 return;
             endJumpList.append(jit.jump());
@@ -157,6 +163,7 @@ public:
 };
 
 typedef JITMathIC<JITAddGenerator> JITAddIC;
+typedef JITMathIC<JITMulGenerator> JITMulIC;
 
 } // namespace JSC
 
index ad0a429..1d0744c 100644 (file)
@@ -31,8 +31,10 @@ namespace JSC {
 
 template <typename Generator> class JITMathIC;
 class JITAddGenerator;
+class JITMulGenerator;
 
 typedef JITMathIC<JITAddGenerator> JITAddIC;
+typedef JITMathIC<JITMulGenerator> JITMulIC;
 
 } // namespace JSC
 
index 169a260..1c49b01 100644 (file)
 #if ENABLE(JIT)
 
 #include "ArithProfile.h"
+#include "JITMathIC.h"
 
 namespace JSC {
 
-void JITMulGenerator::generateFastPath(CCallHelpers& jit)
+JITMathICInlineResult JITMulGenerator::generateInline(CCallHelpers& jit, MathICGenerationState& state)
+{
+    // We default to speculating int32.
+    ObservedType lhs = ObservedType().withInt32();
+    ObservedType rhs = ObservedType().withInt32();
+
+    if (m_arithProfile) {
+        lhs = m_arithProfile->lhsObservedType();
+        rhs = m_arithProfile->rhsObservedType();
+        if (lhs.isEmpty() || rhs.isEmpty()) {
+            // FIXME: ICs should be able to repatch without emitting an inline path:
+            // https://bugs.webkit.org/show_bug.cgi?id=160110
+            lhs = ObservedType().withInt32();
+            rhs = ObservedType().withInt32();
+        }
+    }
+
+    if (lhs.isOnlyNonNumber() && rhs.isOnlyNonNumber())
+        return JITMathICInlineResult::DontGenerate;
+
+    if (lhs.isOnlyNumber() && rhs.isOnlyNumber()) {
+        if (!jit.supportsFloatingPoint())
+            return JITMathICInlineResult::DontGenerate;
+
+        if (!m_leftOperand.definitelyIsNumber())
+            state.slowPathJumps.append(jit.branchIfNotNumber(m_left, m_scratchGPR));
+        if (!m_rightOperand.definitelyIsNumber())
+            state.slowPathJumps.append(jit.branchIfNotNumber(m_right, m_scratchGPR));
+        state.slowPathJumps.append(jit.branchIfInt32(m_left));
+        state.slowPathJumps.append(jit.branchIfInt32(m_right));
+        jit.unboxDoubleNonDestructive(m_left, m_leftFPR, m_scratchGPR, m_scratchFPR);
+        jit.unboxDoubleNonDestructive(m_right, m_rightFPR, m_scratchGPR, m_scratchFPR);
+        jit.mulDouble(m_rightFPR, m_leftFPR);
+        jit.boxDouble(m_leftFPR, m_result);
+
+        return JITMathICInlineResult::GeneratedFastPath;
+    }
+
+    if ((lhs.isOnlyInt32() || m_leftOperand.isPositiveConstInt32()) && (rhs.isOnlyInt32() || m_rightOperand.isPositiveConstInt32())) {
+        ASSERT(!m_leftOperand.isPositiveConstInt32() || !m_rightOperand.isPositiveConstInt32());
+        if (!m_leftOperand.isPositiveConstInt32())
+            state.slowPathJumps.append(jit.branchIfNotInt32(m_left));
+        if (!m_rightOperand.isPositiveConstInt32())
+            state.slowPathJumps.append(jit.branchIfNotInt32(m_right));
+
+        if (m_leftOperand.isPositiveConstInt32() || m_rightOperand.isPositiveConstInt32()) {
+            JSValueRegs var = m_leftOperand.isPositiveConstInt32() ? m_right : m_left;
+            int32_t constValue = m_leftOperand.isPositiveConstInt32() ? m_leftOperand.asConstInt32() : m_rightOperand.asConstInt32();
+            state.slowPathJumps.append(jit.branchMul32(CCallHelpers::Overflow, var.payloadGPR(), CCallHelpers::Imm32(constValue), m_scratchGPR));
+        } else {
+            state.slowPathJumps.append(jit.branchMul32(CCallHelpers::Overflow, m_right.payloadGPR(), m_left.payloadGPR(), m_scratchGPR));
+            state.slowPathJumps.append(jit.branchTest32(CCallHelpers::Zero, m_scratchGPR)); // Go slow if potential negative zero.
+        }
+        jit.boxInt32(m_scratchGPR, m_result);
+
+        return JITMathICInlineResult::GeneratedFastPath;
+    }
+
+    return JITMathICInlineResult::GenerateFullSnippet;
+}
+
+bool JITMulGenerator::generateFastPath(CCallHelpers& jit, CCallHelpers::JumpList& endJumpList, CCallHelpers::JumpList& slowPathJumpList, bool shouldEmitProfiling)
 {
     ASSERT(m_scratchGPR != InvalidGPRReg);
     ASSERT(m_scratchGPR != m_left.payloadGPR());
@@ -47,12 +109,8 @@ void JITMulGenerator::generateFastPath(CCallHelpers& jit)
 
     ASSERT(!m_leftOperand.isPositiveConstInt32() || !m_rightOperand.isPositiveConstInt32());
 
-    if (!m_leftOperand.mightBeNumber() || !m_rightOperand.mightBeNumber()) {
-        ASSERT(!m_didEmitFastPath);
-        return;
-    }
-
-    m_didEmitFastPath = true;
+    if (!m_leftOperand.mightBeNumber() || !m_rightOperand.mightBeNumber())
+        return false;
 
     if (m_leftOperand.isPositiveConstInt32() || m_rightOperand.isPositiveConstInt32()) {
         JSValueRegs var = m_leftOperand.isPositiveConstInt32() ? m_right : m_left;
@@ -66,20 +124,20 @@ void JITMulGenerator::generateFastPath(CCallHelpers& jit)
         if (multiplyResultGPR == var.payloadGPR())
             multiplyResultGPR = m_scratchGPR;
 
-        m_slowPathJumpList.append(jit.branchMul32(CCallHelpers::Overflow, var.payloadGPR(), CCallHelpers::Imm32(constOpr.asConstInt32()), multiplyResultGPR));
+        slowPathJumpList.append(jit.branchMul32(CCallHelpers::Overflow, var.payloadGPR(), CCallHelpers::Imm32(constOpr.asConstInt32()), multiplyResultGPR));
 
         jit.boxInt32(multiplyResultGPR, m_result);
-        m_endJumpList.append(jit.jump());
+        endJumpList.append(jit.jump());
 
         if (!jit.supportsFloatingPoint()) {
-            m_slowPathJumpList.append(notInt32);
-            return;
+            slowPathJumpList.append(notInt32);
+            return true;
         }
 
         // Try to do doubleVar * double(intConstant).
         notInt32.link(&jit);
         if (!varOpr.definitelyIsNumber())
-            m_slowPathJumpList.append(jit.branchIfNotNumber(var, m_scratchGPR));
+            slowPathJumpList.append(jit.branchIfNotNumber(var, m_scratchGPR));
 
         jit.unboxDoubleNonDestructive(var, m_leftFPR, m_scratchGPR, m_scratchFPR);
 
@@ -98,23 +156,23 @@ void JITMulGenerator::generateFastPath(CCallHelpers& jit)
         leftNotInt = jit.branchIfNotInt32(m_left);
         rightNotInt = jit.branchIfNotInt32(m_right);
 
-        m_slowPathJumpList.append(jit.branchMul32(CCallHelpers::Overflow, m_right.payloadGPR(), m_left.payloadGPR(), m_scratchGPR));
-        m_slowPathJumpList.append(jit.branchTest32(CCallHelpers::Zero, m_scratchGPR)); // Go slow if potential negative zero.
+        slowPathJumpList.append(jit.branchMul32(CCallHelpers::Overflow, m_right.payloadGPR(), m_left.payloadGPR(), m_scratchGPR));
+        slowPathJumpList.append(jit.branchTest32(CCallHelpers::Zero, m_scratchGPR)); // Go slow if potential negative zero.
 
         jit.boxInt32(m_scratchGPR, m_result);
-        m_endJumpList.append(jit.jump());
+        endJumpList.append(jit.jump());
 
         if (!jit.supportsFloatingPoint()) {
-            m_slowPathJumpList.append(leftNotInt);
-            m_slowPathJumpList.append(rightNotInt);
-            return;
+            slowPathJumpList.append(leftNotInt);
+            slowPathJumpList.append(rightNotInt);
+            return true;
         }
 
         leftNotInt.link(&jit);
         if (!m_leftOperand.definitelyIsNumber())
-            m_slowPathJumpList.append(jit.branchIfNotNumber(m_left, m_scratchGPR));
+            slowPathJumpList.append(jit.branchIfNotNumber(m_left, m_scratchGPR));
         if (!m_rightOperand.definitelyIsNumber())
-            m_slowPathJumpList.append(jit.branchIfNotNumber(m_right, m_scratchGPR));
+            slowPathJumpList.append(jit.branchIfNotNumber(m_right, m_scratchGPR));
 
         jit.unboxDoubleNonDestructive(m_left, m_leftFPR, m_scratchGPR, m_scratchFPR);
         CCallHelpers::Jump rightIsDouble = jit.branchIfNotInt32(m_right);
@@ -124,7 +182,7 @@ void JITMulGenerator::generateFastPath(CCallHelpers& jit)
 
         rightNotInt.link(&jit);
         if (!m_rightOperand.definitelyIsNumber())
-            m_slowPathJumpList.append(jit.branchIfNotNumber(m_right, m_scratchGPR));
+            slowPathJumpList.append(jit.branchIfNotNumber(m_right, m_scratchGPR));
 
         jit.convertInt32ToDouble(m_left.payloadGPR(), m_leftFPR);
 
@@ -139,7 +197,7 @@ void JITMulGenerator::generateFastPath(CCallHelpers& jit)
     // Do doubleVar * doubleVar.
     jit.mulDouble(m_rightFPR, m_leftFPR);
 
-    if (!m_arithProfile)
+    if (!m_arithProfile || !shouldEmitProfiling)
         jit.boxDouble(m_leftFPR, m_result);
     else {
         // The Int52 overflow check below intentionally omits 1ll << 51 as a valid negative Int52 value.
@@ -149,6 +207,7 @@ void JITMulGenerator::generateFastPath(CCallHelpers& jit)
         const int64_t negativeZeroBits = 1ll << 63;
 #if USE(JSVALUE64)
         jit.moveDoubleTo64(m_leftFPR, m_result.payloadGPR());
+
         CCallHelpers::Jump notNegativeZero = jit.branch64(CCallHelpers::NotEqual, m_result.payloadGPR(), CCallHelpers::TrustedImm64(negativeZeroBits));
 
         jit.or32(CCallHelpers::TrustedImm32(ArithProfile::NegZeroDouble), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits()));
@@ -186,13 +245,15 @@ void JITMulGenerator::generateFastPath(CCallHelpers& jit)
         
         jit.or32(CCallHelpers::TrustedImm32(ArithProfile::Int52Overflow), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits()));
 
-        m_endJumpList.append(noInt52Overflow);
+        endJumpList.append(noInt52Overflow);
         if (m_scratchGPR == m_result.tagGPR() || m_scratchGPR == m_result.payloadGPR())
             jit.boxDouble(m_leftFPR, m_result);
 
-        m_endJumpList.append(done);
+        endJumpList.append(done);
 #endif
     }
+
+    return true;
 }
 
 } // namespace JSC
index a73e9d9..ef55039 100644 (file)
 #if ENABLE(JIT)
 
 #include "CCallHelpers.h"
+#include "JITMathICInlineResult.h"
 #include "SnippetOperand.h"
 
 namespace JSC {
 
+struct MathICGenerationState;
+
 class JITMulGenerator {
 public:
+    JITMulGenerator() { }
+
     JITMulGenerator(SnippetOperand leftOperand, SnippetOperand rightOperand,
         JSValueRegs result, JSValueRegs left, JSValueRegs right,
         FPRReg leftFPR, FPRReg rightFPR, GPRReg scratchGPR, FPRReg scratchFPR,
@@ -53,11 +58,11 @@ public:
         ASSERT(!m_leftOperand.isPositiveConstInt32() || !m_rightOperand.isPositiveConstInt32());
     }
 
-    void generateFastPath(CCallHelpers&);
+    JITMathICInlineResult generateInline(CCallHelpers&, MathICGenerationState&);
+    bool generateFastPath(CCallHelpers&, CCallHelpers::JumpList& endJumpList, CCallHelpers::JumpList& slowJumpList, bool shouldEmitProfiling);
 
-    bool didEmitFastPath() const { return m_didEmitFastPath; }
-    CCallHelpers::JumpList& endJumpList() { return m_endJumpList; }
-    CCallHelpers::JumpList& slowPathJumpList() { return m_slowPathJumpList; }
+    bool isLeftOperandValidConstant() const { return m_leftOperand.isPositiveConstInt32(); }
+    bool isRightOperandValidConstant() const { return m_rightOperand.isPositiveConstInt32(); }
 
 private:
     SnippetOperand m_leftOperand;
@@ -70,10 +75,6 @@ private:
     GPRReg m_scratchGPR;
     FPRReg m_scratchFPR;
     ArithProfile* m_arithProfile;
-    bool m_didEmitFastPath { false };
-
-    CCallHelpers::JumpList m_endJumpList;
-    CCallHelpers::JumpList m_slowPathJumpList;
 };
 
 } // namespace JSC
index 2e9396a..00a35e6 100644 (file)
@@ -2336,11 +2336,8 @@ EncodedJSValue JIT_OPERATION operationValueAddNoOptimize(ExecState* exec, Encode
     return JSValue::encode(result);
 }
 
-EncodedJSValue JIT_OPERATION operationValueMul(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2)
+ALWAYS_INLINE static EncodedJSValue unprofiledMul(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2)
 {
-    VM* vm = &exec->vm();
-    NativeCallFrameTracer tracer(vm, exec);
-
     JSValue op1 = JSValue::decode(encodedOp1);
     JSValue op2 = JSValue::decode(encodedOp2);
 
@@ -2349,14 +2346,14 @@ EncodedJSValue JIT_OPERATION operationValueMul(ExecState* exec, EncodedJSValue e
     return JSValue::encode(jsNumber(a * b));
 }
 
-EncodedJSValue JIT_OPERATION operationValueMulProfiled(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile* arithProfile)
+ALWAYS_INLINE static EncodedJSValue profiledMul(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile* arithProfile, bool shouldObserveLHSAndRHSTypes = true)
 {
-    VM* vm = &exec->vm();
-    NativeCallFrameTracer tracer(vm, exec);
-
     JSValue op1 = JSValue::decode(encodedOp1);
     JSValue op2 = JSValue::decode(encodedOp2);
 
+    if (shouldObserveLHSAndRHSTypes)
+        arithProfile->observeLHSAndRHS(op1, op2);
+
     double a = op1.toNumber(exec);
     double b = op2.toNumber(exec);
     
@@ -2365,6 +2362,61 @@ EncodedJSValue JIT_OPERATION operationValueMulProfiled(ExecState* exec, EncodedJ
     return JSValue::encode(result);
 }
 
+EncodedJSValue JIT_OPERATION operationValueMul(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2)
+{
+    VM* vm = &exec->vm();
+    NativeCallFrameTracer tracer(vm, exec);
+
+    return unprofiledMul(exec, encodedOp1, encodedOp2);
+}
+
+EncodedJSValue JIT_OPERATION operationValueMulNoOptimize(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, JITMulIC*)
+{
+    VM* vm = &exec->vm();
+    NativeCallFrameTracer tracer(vm, exec);
+
+    return unprofiledMul(exec, encodedOp1, encodedOp2);
+}
+
+EncodedJSValue JIT_OPERATION operationValueMulOptimize(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, JITMulIC* mulIC)
+{
+    VM* vm = &exec->vm();
+    NativeCallFrameTracer tracer(vm, exec);
+
+    auto nonOptimizeVariant = operationValueMulNoOptimize;
+    mulIC->generateOutOfLine(*vm, exec->codeBlock(), nonOptimizeVariant);
+
+    return unprofiledMul(exec, encodedOp1, encodedOp2);
+}
+
+EncodedJSValue JIT_OPERATION operationValueMulProfiled(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile* arithProfile)
+{
+    VM* vm = &exec->vm();
+    NativeCallFrameTracer tracer(vm, exec);
+
+    return profiledMul(exec, encodedOp1, encodedOp2, arithProfile);
+}
+
+EncodedJSValue JIT_OPERATION operationValueMulProfiledOptimize(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile* arithProfile, JITMulIC* mulIC)
+{
+    VM* vm = &exec->vm();
+    NativeCallFrameTracer tracer(vm, exec);
+
+    arithProfile->observeLHSAndRHS(JSValue::decode(encodedOp1), JSValue::decode(encodedOp2));
+    auto nonOptimizeVariant = operationValueMulProfiledNoOptimize;
+    mulIC->generateOutOfLine(*vm, exec->codeBlock(), nonOptimizeVariant);
+
+    return profiledMul(exec, encodedOp1, encodedOp2, arithProfile, false);
+}
+
+EncodedJSValue JIT_OPERATION operationValueMulProfiledNoOptimize(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile* arithProfile, JITMulIC*)
+{
+    VM* vm = &exec->vm();
+    NativeCallFrameTracer tracer(vm, exec);
+
+    return profiledMul(exec, encodedOp1, encodedOp2, arithProfile);
+}
+
 EncodedJSValue JIT_OPERATION operationValueSub(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2)
 {
     VM* vm = &exec->vm();
index ce96ac3..95d6eaa 100644 (file)
@@ -84,7 +84,7 @@ typedef char* UnusedPtr;
     Icf: InlineCallFrame*
     Idc: const Identifier*
     J: EncodedJSValue
-    Jaic: JITAddIC*
+    Mic: JITMathIC* (can be JITAddIC*, JITMulIC*, etc).
     Jcp: const JSValue*
     Jsc: JSScope*
     Jsf: JSFunction*
@@ -137,8 +137,8 @@ typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJAp)(ExecState*, Encoded
 typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJBy)(ExecState*, EncodedJSValue, EncodedJSValue, ByValInfo*);
 typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJJ)(ExecState*, EncodedJSValue, EncodedJSValue, EncodedJSValue);
 typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJArp)(ExecState*, EncodedJSValue, EncodedJSValue, ArithProfile*);
-typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJArpJaic)(ExecState*, EncodedJSValue, EncodedJSValue, ArithProfile*, JITAddIC*);
-typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJJaic)(ExecState*, EncodedJSValue, EncodedJSValue, JITAddIC*);
+typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJArpMic)(ExecState*, EncodedJSValue, EncodedJSValue, ArithProfile*, void*);
+typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJMic)(ExecState*, EncodedJSValue, EncodedJSValue, void*);
 typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJssZ)(ExecState*, JSString*, int32_t);
 typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJssReo)(ExecState*, JSString*, RegExpObject*);
 typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJssReoJss)(ExecState*, JSString*, RegExpObject*, JSString*);
@@ -419,6 +419,10 @@ EncodedJSValue JIT_OPERATION operationValueAddProfiledNoOptimize(ExecState*, Enc
 EncodedJSValue JIT_OPERATION operationValueAddOptimize(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, JITAddIC*) WTF_INTERNAL;
 EncodedJSValue JIT_OPERATION operationValueAddNoOptimize(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, JITAddIC*) WTF_INTERNAL;
 EncodedJSValue JIT_OPERATION operationValueMul(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2) WTF_INTERNAL;
+EncodedJSValue JIT_OPERATION operationValueMulOptimize(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, JITMulIC*) WTF_INTERNAL;
+EncodedJSValue JIT_OPERATION operationValueMulNoOptimize(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, JITMulIC*) WTF_INTERNAL;
+EncodedJSValue JIT_OPERATION operationValueMulProfiledOptimize(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile*, JITMulIC*) WTF_INTERNAL;
+EncodedJSValue JIT_OPERATION operationValueMulProfiledNoOptimize(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile*, JITMulIC*) WTF_INTERNAL;
 EncodedJSValue JIT_OPERATION operationValueMulProfiled(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile*) WTF_INTERNAL;
 EncodedJSValue JIT_OPERATION operationValueSub(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2) WTF_INTERNAL;
 EncodedJSValue JIT_OPERATION operationValueSubProfiled(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile*) WTF_INTERNAL;