op_add/ValueAdd should be an IC in all JIT tiers
authorsbarati@apple.com <sbarati@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 21 Jul 2016 23:41:44 +0000 (23:41 +0000)
committersbarati@apple.com <sbarati@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 21 Jul 2016 23:41:44 +0000 (23:41 +0000)
https://bugs.webkit.org/show_bug.cgi?id=159649

Reviewed by Benjamin Poulain.

Source/JavaScriptCore:

This patch makes Add an IC inside all JIT tiers. It does so in a
simple, but effective, way. We will try to generate an int+int add
that will repatch itself if its type checks fail. Sometimes though,
we have runtime type data saying that the add won't be int+int.
In those cases, we will just generate a full snippet that doesn't patch itself.
Other times, we may generate no inline code and defer to making a C call. A lot
of this patch is just refactoring ResultProfile into what we're now calling ArithProfile.
ArithProfile does everything ResultProfile used to do, and more. It records simple type
data about the LHS/RHS operands it sees. This allows us to determine if an op_add
has only seen int+int operands, etc. ArithProfile will also contain the ResultType
for the LHS/RHS that the parser feeds into op_add. ArithProfile now fits into 32-bits.
This means instead of having a side table like we did for ResultProfile, we just
inject the ArithProfile into the bytecode instruction stream. This makes asking
for ArithProfile faster; we no longer need to lock around this operation.

The size of an Add has gone down on average, but we can still do better.
We still generate a lot of code because we generate calls to the slow path.
I think we can make this better by moving the slow path to a shared thunk
system. This patch mostly lays the foundation for future improvements to Add,
and a framework to move all other arithmetic operations to be typed-based ICs.

Here is some data I took on the average op_add/ValueAdd size on various benchmarks:
           |   JetStream  |  Speedometer |  Unity 3D  |
     ------| -------------|-----------------------------
      Old  |  189 bytes   |  169 bytes   |  192 bytes |
     ------| -------------|-----------------------------
      New  |  148 bytes   |  124 bytes   |  143 bytes |
     ---------------------------------------------------

Making an arithmetic IC is now easy. The JITMathIC class will hold a snippet
generator as a member variable. To make a snippet an IC, you need to implement
a generateInline(.) method, which generates the inline IC. Then, you need to
generate the IC where you used to generate the snippet. When generating the
IC, we need to inform JITMathIC of various data like we do with StructureStubInfo.
We need to tell it about where the slow path starts, where the slow path call is, etc.
When generating a JITMathIC, it may tell you that it didn't generate any code inline.
This is a request to the user of JITMathIC to just generate a C call along the
fast path. JITMathIC may also have the snippet tell it to just generate the full
snippet instead of the int+int path along the fast path.

In subsequent patches, we can improve upon how we decide to generate int+int or
the full snippet. I tried to get clever by having double+double, double+int, int+double,
fast paths, but they didn't work out nearly as well as the int+int fast path. I ended up
generating a lot of code when I did this and ended up using more memory than just generating
the full snippet. There is probably some way we can be clever and generate specialized fast
paths that are more successful than what I tried implementing, but I think that's worth deferring
this to follow up patches once the JITMathIC foundation has landed.

This patch also fixes a bug inside the slow path lambdas in the DFG.
Before, it was not legal to emit an exception check inside them. Now,
it is. So it's now easy to define arbitrary late paths using the DFG
slow path lambda API.

* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* bytecode/ArithProfile.cpp: Added.
(JSC::ArithProfile::emitObserveResult):
(JSC::ArithProfile::shouldEmitSetDouble):
(JSC::ArithProfile::emitSetDouble):
(JSC::ArithProfile::shouldEmitSetNonNumber):
(JSC::ArithProfile::emitSetNonNumber):
(WTF::printInternal):
* bytecode/ArithProfile.h: Added.
(JSC::ObservedType::ObservedType):
(JSC::ObservedType::sawInt32):
(JSC::ObservedType::isOnlyInt32):
(JSC::ObservedType::sawNumber):
(JSC::ObservedType::isOnlyNumber):
(JSC::ObservedType::sawNonNumber):
(JSC::ObservedType::isOnlyNonNumber):
(JSC::ObservedType::isEmpty):
(JSC::ObservedType::bits):
(JSC::ObservedType::withInt32):
(JSC::ObservedType::withNumber):
(JSC::ObservedType::withNonNumber):
(JSC::ObservedType::withoutNonNumber):
(JSC::ObservedType::operator==):
(JSC::ArithProfile::ArithProfile):
(JSC::ArithProfile::fromInt):
(JSC::ArithProfile::lhsResultType):
(JSC::ArithProfile::rhsResultType):
(JSC::ArithProfile::lhsObservedType):
(JSC::ArithProfile::rhsObservedType):
(JSC::ArithProfile::setLhsObservedType):
(JSC::ArithProfile::setRhsObservedType):
(JSC::ArithProfile::tookSpecialFastPath):
(JSC::ArithProfile::didObserveNonInt32):
(JSC::ArithProfile::didObserveDouble):
(JSC::ArithProfile::didObserveNonNegZeroDouble):
(JSC::ArithProfile::didObserveNegZeroDouble):
(JSC::ArithProfile::didObserveNonNumber):
(JSC::ArithProfile::didObserveInt32Overflow):
(JSC::ArithProfile::didObserveInt52Overflow):
(JSC::ArithProfile::setObservedNonNegZeroDouble):
(JSC::ArithProfile::setObservedNegZeroDouble):
(JSC::ArithProfile::setObservedNonNumber):
(JSC::ArithProfile::setObservedInt32Overflow):
(JSC::ArithProfile::setObservedInt52Overflow):
(JSC::ArithProfile::addressOfBits):
(JSC::ArithProfile::observeResult):
(JSC::ArithProfile::lhsSawInt32):
(JSC::ArithProfile::lhsSawNumber):
(JSC::ArithProfile::lhsSawNonNumber):
(JSC::ArithProfile::rhsSawInt32):
(JSC::ArithProfile::rhsSawNumber):
(JSC::ArithProfile::rhsSawNonNumber):
(JSC::ArithProfile::observeLHSAndRHS):
(JSC::ArithProfile::bits):
(JSC::ArithProfile::hasBits):
(JSC::ArithProfile::setBit):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::dumpRareCaseProfile):
(JSC::CodeBlock::dumpArithProfile):
(JSC::CodeBlock::dumpBytecode):
(JSC::CodeBlock::addStubInfo):
(JSC::CodeBlock::addJITAddIC):
(JSC::CodeBlock::findStubInfo):
(JSC::CodeBlock::resetJITData):
(JSC::CodeBlock::shrinkToFit):
(JSC::CodeBlock::dumpValueProfiles):
(JSC::CodeBlock::rareCaseProfileCountForBytecodeOffset):
(JSC::CodeBlock::arithProfileForBytecodeOffset):
(JSC::CodeBlock::arithProfileForPC):
(JSC::CodeBlock::couldTakeSpecialFastCase):
(JSC::CodeBlock::dumpResultProfile): Deleted.
(JSC::CodeBlock::resultProfileForBytecodeOffset): Deleted.
(JSC::CodeBlock::specialFastCaseProfileCountForBytecodeOffset): Deleted.
(JSC::CodeBlock::ensureResultProfile): Deleted.
* bytecode/CodeBlock.h:
(JSC::CodeBlock::stubInfoBegin):
(JSC::CodeBlock::stubInfoEnd):
(JSC::CodeBlock::couldTakeSlowCase):
(JSC::CodeBlock::numberOfResultProfiles): Deleted.
* bytecode/MethodOfGettingAValueProfile.cpp:
(JSC::MethodOfGettingAValueProfile::emitReportValue):
* bytecode/MethodOfGettingAValueProfile.h:
(JSC::MethodOfGettingAValueProfile::MethodOfGettingAValueProfile):
* bytecode/ValueProfile.cpp:
(JSC::ResultProfile::emitDetectNumericness): Deleted.
(JSC::ResultProfile::emitSetDouble): Deleted.
(JSC::ResultProfile::emitSetNonNumber): Deleted.
(WTF::printInternal): Deleted.
* bytecode/ValueProfile.h:
(JSC::getRareCaseProfileBytecodeOffset):
(JSC::ResultProfile::ResultProfile): Deleted.
(JSC::ResultProfile::bytecodeOffset): Deleted.
(JSC::ResultProfile::specialFastPathCount): Deleted.
(JSC::ResultProfile::didObserveNonInt32): Deleted.
(JSC::ResultProfile::didObserveDouble): Deleted.
(JSC::ResultProfile::didObserveNonNegZeroDouble): Deleted.
(JSC::ResultProfile::didObserveNegZeroDouble): Deleted.
(JSC::ResultProfile::didObserveNonNumber): Deleted.
(JSC::ResultProfile::didObserveInt32Overflow): Deleted.
(JSC::ResultProfile::didObserveInt52Overflow): Deleted.
(JSC::ResultProfile::setObservedNonNegZeroDouble): Deleted.
(JSC::ResultProfile::setObservedNegZeroDouble): Deleted.
(JSC::ResultProfile::setObservedNonNumber): Deleted.
(JSC::ResultProfile::setObservedInt32Overflow): Deleted.
(JSC::ResultProfile::setObservedInt52Overflow): Deleted.
(JSC::ResultProfile::addressOfFlags): Deleted.
(JSC::ResultProfile::addressOfSpecialFastPathCount): Deleted.
(JSC::ResultProfile::detectNumericness): Deleted.
(JSC::ResultProfile::hasBits): Deleted.
(JSC::ResultProfile::setBit): Deleted.
(JSC::getResultProfileBytecodeOffset): Deleted.
* bytecompiler/BytecodeGenerator.cpp:
(JSC::BytecodeGenerator::emitBinaryOp):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::makeSafe):
* dfg/DFGGraph.cpp:
(JSC::DFG::Graph::methodOfGettingAValueProfileFor):
* dfg/DFGJITCompiler.cpp:
(JSC::DFG::JITCompiler::exceptionCheck):
* dfg/DFGSlowPathGenerator.h:
(JSC::DFG::SlowPathGenerator::generate):
* dfg/DFGSpeculativeJIT.cpp:
(JSC::DFG::SpeculativeJIT::addSlowPathGenerator):
(JSC::DFG::SpeculativeJIT::runSlowPathGenerators):
(JSC::DFG::SpeculativeJIT::compileValueAdd):
* dfg/DFGSpeculativeJIT.h:
(JSC::DFG::SpeculativeJIT::silentSpillAllRegistersImpl):
(JSC::DFG::SpeculativeJIT::silentSpillAllRegisters):
(JSC::DFG::SpeculativeJIT::callOperation):
* ftl/FTLLowerDFGToB3.cpp:
(JSC::FTL::DFG::LowerDFGToB3::compileValueAdd):
(JSC::FTL::DFG::LowerDFGToB3::compileStrCat):
* jit/CCallHelpers.h:
(JSC::CCallHelpers::setupArgumentsWithExecState):
(JSC::CCallHelpers::setupArguments):
* jit/JIT.h:
* jit/JITAddGenerator.cpp:
(JSC::JITAddGenerator::generateInline):
(JSC::JITAddGenerator::generateFastPath):
* jit/JITAddGenerator.h:
(JSC::JITAddGenerator::JITAddGenerator):
(JSC::JITAddGenerator::didEmitFastPath): Deleted.
(JSC::JITAddGenerator::endJumpList): Deleted.
(JSC::JITAddGenerator::slowPathJumpList): Deleted.
* jit/JITArithmetic.cpp:
(JSC::JIT::emit_op_jless):
(JSC::JIT::emitSlow_op_urshift):
(JSC::getOperandTypes):
(JSC::JIT::emit_op_add):
(JSC::JIT::emitSlow_op_add):
(JSC::JIT::emit_op_div):
(JSC::JIT::emit_op_mul):
(JSC::JIT::emitSlow_op_mul):
(JSC::JIT::emit_op_sub):
(JSC::JIT::emitSlow_op_sub):
* jit/JITDivGenerator.cpp:
(JSC::JITDivGenerator::generateFastPath):
* jit/JITDivGenerator.h:
(JSC::JITDivGenerator::JITDivGenerator):
* jit/JITInlines.h:
(JSC::JIT::callOperation):
* jit/JITMathIC.h: Added.
(JSC::JITMathIC::doneLocation):
(JSC::JITMathIC::slowPathStartLocation):
(JSC::JITMathIC::slowPathCallLocation):
(JSC::JITMathIC::generateInline):
(JSC::JITMathIC::generateOutOfLine):
(JSC::JITMathIC::finalizeInlineCode):
* jit/JITMathICForwards.h: Added.
* jit/JITMathICInlineResult.h: Added.
* jit/JITMulGenerator.cpp:
(JSC::JITMulGenerator::generateFastPath):
* jit/JITMulGenerator.h:
(JSC::JITMulGenerator::JITMulGenerator):
* jit/JITOperations.cpp:
* jit/JITOperations.h:
* jit/JITSubGenerator.cpp:
(JSC::JITSubGenerator::generateFastPath):
* jit/JITSubGenerator.h:
(JSC::JITSubGenerator::JITSubGenerator):
* jit/Repatch.cpp:
(JSC::readCallTarget):
(JSC::ftlThunkAwareRepatchCall):
(JSC::tryCacheGetByID):
(JSC::repatchGetByID):
(JSC::appropriateGenericPutByIdFunction):
(JSC::tryCachePutByID):
(JSC::repatchPutByID):
(JSC::tryRepatchIn):
(JSC::repatchIn):
(JSC::linkSlowFor):
(JSC::resetGetByID):
(JSC::resetPutByID):
(JSC::repatchCall): Deleted.
* jit/Repatch.h:
* llint/LLIntData.cpp:
(JSC::LLInt::Data::performAssertions):
* llint/LowLevelInterpreter.asm:
* llint/LowLevelInterpreter32_64.asm:
* llint/LowLevelInterpreter64.asm:
* parser/ResultType.h:
(JSC::ResultType::ResultType):
(JSC::ResultType::isInt32):
(JSC::ResultType::definitelyIsNumber):
(JSC::ResultType::definitelyIsString):
(JSC::ResultType::definitelyIsBoolean):
(JSC::ResultType::mightBeNumber):
(JSC::ResultType::isNotNumber):
(JSC::ResultType::forBitOp):
(JSC::ResultType::bits):
(JSC::OperandTypes::OperandTypes):
* runtime/CommonSlowPaths.cpp:
(JSC::SLOW_PATH_DECL):
(JSC::updateArithProfileForBinaryArithOp):
(JSC::updateResultProfileForBinaryArithOp): Deleted.
* tests/stress/op-add-exceptions.js: Added.
(assert):
(f1):
(f2):
(f3):
(let.oException.valueOf):
(foo):
(ident):
(bar):

Source/WebCore:

* ForwardingHeaders/jit/JITMathICForwards.h: Added.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@203537 268f45cc-cd09-0410-ab3c-d52691b4dbfc

47 files changed:
Source/JavaScriptCore/CMakeLists.txt
Source/JavaScriptCore/ChangeLog
Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj
Source/JavaScriptCore/bytecode/ArithProfile.cpp [new file with mode: 0644]
Source/JavaScriptCore/bytecode/ArithProfile.h [new file with mode: 0644]
Source/JavaScriptCore/bytecode/CodeBlock.cpp
Source/JavaScriptCore/bytecode/CodeBlock.h
Source/JavaScriptCore/bytecode/MethodOfGettingAValueProfile.cpp
Source/JavaScriptCore/bytecode/MethodOfGettingAValueProfile.h
Source/JavaScriptCore/bytecode/ValueProfile.cpp
Source/JavaScriptCore/bytecode/ValueProfile.h
Source/JavaScriptCore/bytecompiler/BytecodeGenerator.cpp
Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp
Source/JavaScriptCore/dfg/DFGGraph.cpp
Source/JavaScriptCore/dfg/DFGJITCompiler.cpp
Source/JavaScriptCore/dfg/DFGSlowPathGenerator.h
Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp
Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h
Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp
Source/JavaScriptCore/jit/CCallHelpers.h
Source/JavaScriptCore/jit/JIT.h
Source/JavaScriptCore/jit/JITAddGenerator.cpp
Source/JavaScriptCore/jit/JITAddGenerator.h
Source/JavaScriptCore/jit/JITArithmetic.cpp
Source/JavaScriptCore/jit/JITDivGenerator.cpp
Source/JavaScriptCore/jit/JITDivGenerator.h
Source/JavaScriptCore/jit/JITInlines.h
Source/JavaScriptCore/jit/JITMathIC.h [new file with mode: 0644]
Source/JavaScriptCore/jit/JITMathICForwards.h [new file with mode: 0644]
Source/JavaScriptCore/jit/JITMathICInlineResult.h [new file with mode: 0644]
Source/JavaScriptCore/jit/JITMulGenerator.cpp
Source/JavaScriptCore/jit/JITMulGenerator.h
Source/JavaScriptCore/jit/JITOperations.cpp
Source/JavaScriptCore/jit/JITOperations.h
Source/JavaScriptCore/jit/JITSubGenerator.cpp
Source/JavaScriptCore/jit/JITSubGenerator.h
Source/JavaScriptCore/jit/Repatch.cpp
Source/JavaScriptCore/jit/Repatch.h
Source/JavaScriptCore/llint/LLIntData.cpp
Source/JavaScriptCore/llint/LowLevelInterpreter.asm
Source/JavaScriptCore/llint/LowLevelInterpreter32_64.asm
Source/JavaScriptCore/llint/LowLevelInterpreter64.asm
Source/JavaScriptCore/parser/ResultType.h
Source/JavaScriptCore/runtime/CommonSlowPaths.cpp
Source/JavaScriptCore/tests/stress/op-add-exceptions.js [new file with mode: 0644]
Source/WebCore/ChangeLog
Source/WebCore/ForwardingHeaders/jit/JITMathICForwards.h [new file with mode: 0644]

index 6fe0a5f..6b22bd4 100644 (file)
@@ -176,6 +176,7 @@ set(JavaScriptCore_SOURCES
     builtins/BuiltinExecutableCreator.cpp
 
     bytecode/AdaptiveInferredPropertyValueWatchpointBase.cpp
+    bytecode/ArithProfile.cpp
     bytecode/ArrayAllocationProfile.cpp
     bytecode/ArrayProfile.cpp
     bytecode/BytecodeBasicBlock.cpp
index 2366505..2ea9f92 100644 (file)
@@ -1,3 +1,289 @@
+2016-07-21  Saam Barati  <sbarati@apple.com>
+
+        op_add/ValueAdd should be an IC in all JIT tiers
+        https://bugs.webkit.org/show_bug.cgi?id=159649
+
+        Reviewed by Benjamin Poulain.
+
+        This patch makes Add an IC inside all JIT tiers. It does so in a
+        simple, but effective, way. We will try to generate an int+int add
+        that will repatch itself if its type checks fail. Sometimes though,
+        we have runtime type data saying that the add won't be int+int.
+        In those cases, we will just generate a full snippet that doesn't patch itself.
+        Other times, we may generate no inline code and defer to making a C call. A lot
+        of this patch is just refactoring ResultProfile into what we're now calling ArithProfile.
+        ArithProfile does everything ResultProfile used to do, and more. It records simple type
+        data about the LHS/RHS operands it sees. This allows us to determine if an op_add
+        has only seen int+int operands, etc. ArithProfile will also contain the ResultType
+        for the LHS/RHS that the parser feeds into op_add. ArithProfile now fits into 32-bits.
+        This means instead of having a side table like we did for ResultProfile, we just
+        inject the ArithProfile into the bytecode instruction stream. This makes asking
+        for ArithProfile faster; we no longer need to lock around this operation.
+
+        The size of an Add has gone down on average, but we can still do better.
+        We still generate a lot of code because we generate calls to the slow path.
+        I think we can make this better by moving the slow path to a shared thunk
+        system. This patch mostly lays the foundation for future improvements to Add,
+        and a framework to move all other arithmetic operations to be typed-based ICs.
+
+        Here is some data I took on the average op_add/ValueAdd size on various benchmarks:
+                   |   JetStream  |  Speedometer |  Unity 3D  |
+             ------| -------------|-----------------------------
+              Old  |  189 bytes   |  169 bytes   |  192 bytes |
+             ------| -------------|-----------------------------
+              New  |  148 bytes   |  124 bytes   |  143 bytes |
+             ---------------------------------------------------
+
+        Making an arithmetic IC is now easy. The JITMathIC class will hold a snippet
+        generator as a member variable. To make a snippet an IC, you need to implement
+        a generateInline(.) method, which generates the inline IC. Then, you need to
+        generate the IC where you used to generate the snippet. When generating the
+        IC, we need to inform JITMathIC of various data like we do with StructureStubInfo.
+        We need to tell it about where the slow path starts, where the slow path call is, etc.
+        When generating a JITMathIC, it may tell you that it didn't generate any code inline.
+        This is a request to the user of JITMathIC to just generate a C call along the
+        fast path. JITMathIC may also have the snippet tell it to just generate the full
+        snippet instead of the int+int path along the fast path.
+
+        In subsequent patches, we can improve upon how we decide to generate int+int or
+        the full snippet. I tried to get clever by having double+double, double+int, int+double,
+        fast paths, but they didn't work out nearly as well as the int+int fast path. I ended up
+        generating a lot of code when I did this and ended up using more memory than just generating
+        the full snippet. There is probably some way we can be clever and generate specialized fast
+        paths that are more successful than what I tried implementing, but I think that's worth deferring
+        this to follow up patches once the JITMathIC foundation has landed.
+
+        This patch also fixes a bug inside the slow path lambdas in the DFG.
+        Before, it was not legal to emit an exception check inside them. Now,
+        it is. So it's now easy to define arbitrary late paths using the DFG
+        slow path lambda API.
+
+        * CMakeLists.txt:
+        * JavaScriptCore.xcodeproj/project.pbxproj:
+        * bytecode/ArithProfile.cpp: Added.
+        (JSC::ArithProfile::emitObserveResult):
+        (JSC::ArithProfile::shouldEmitSetDouble):
+        (JSC::ArithProfile::emitSetDouble):
+        (JSC::ArithProfile::shouldEmitSetNonNumber):
+        (JSC::ArithProfile::emitSetNonNumber):
+        (WTF::printInternal):
+        * bytecode/ArithProfile.h: Added.
+        (JSC::ObservedType::ObservedType):
+        (JSC::ObservedType::sawInt32):
+        (JSC::ObservedType::isOnlyInt32):
+        (JSC::ObservedType::sawNumber):
+        (JSC::ObservedType::isOnlyNumber):
+        (JSC::ObservedType::sawNonNumber):
+        (JSC::ObservedType::isOnlyNonNumber):
+        (JSC::ObservedType::isEmpty):
+        (JSC::ObservedType::bits):
+        (JSC::ObservedType::withInt32):
+        (JSC::ObservedType::withNumber):
+        (JSC::ObservedType::withNonNumber):
+        (JSC::ObservedType::withoutNonNumber):
+        (JSC::ObservedType::operator==):
+        (JSC::ArithProfile::ArithProfile):
+        (JSC::ArithProfile::fromInt):
+        (JSC::ArithProfile::lhsResultType):
+        (JSC::ArithProfile::rhsResultType):
+        (JSC::ArithProfile::lhsObservedType):
+        (JSC::ArithProfile::rhsObservedType):
+        (JSC::ArithProfile::setLhsObservedType):
+        (JSC::ArithProfile::setRhsObservedType):
+        (JSC::ArithProfile::tookSpecialFastPath):
+        (JSC::ArithProfile::didObserveNonInt32):
+        (JSC::ArithProfile::didObserveDouble):
+        (JSC::ArithProfile::didObserveNonNegZeroDouble):
+        (JSC::ArithProfile::didObserveNegZeroDouble):
+        (JSC::ArithProfile::didObserveNonNumber):
+        (JSC::ArithProfile::didObserveInt32Overflow):
+        (JSC::ArithProfile::didObserveInt52Overflow):
+        (JSC::ArithProfile::setObservedNonNegZeroDouble):
+        (JSC::ArithProfile::setObservedNegZeroDouble):
+        (JSC::ArithProfile::setObservedNonNumber):
+        (JSC::ArithProfile::setObservedInt32Overflow):
+        (JSC::ArithProfile::setObservedInt52Overflow):
+        (JSC::ArithProfile::addressOfBits):
+        (JSC::ArithProfile::observeResult):
+        (JSC::ArithProfile::lhsSawInt32):
+        (JSC::ArithProfile::lhsSawNumber):
+        (JSC::ArithProfile::lhsSawNonNumber):
+        (JSC::ArithProfile::rhsSawInt32):
+        (JSC::ArithProfile::rhsSawNumber):
+        (JSC::ArithProfile::rhsSawNonNumber):
+        (JSC::ArithProfile::observeLHSAndRHS):
+        (JSC::ArithProfile::bits):
+        (JSC::ArithProfile::hasBits):
+        (JSC::ArithProfile::setBit):
+        * bytecode/CodeBlock.cpp:
+        (JSC::CodeBlock::dumpRareCaseProfile):
+        (JSC::CodeBlock::dumpArithProfile):
+        (JSC::CodeBlock::dumpBytecode):
+        (JSC::CodeBlock::addStubInfo):
+        (JSC::CodeBlock::addJITAddIC):
+        (JSC::CodeBlock::findStubInfo):
+        (JSC::CodeBlock::resetJITData):
+        (JSC::CodeBlock::shrinkToFit):
+        (JSC::CodeBlock::dumpValueProfiles):
+        (JSC::CodeBlock::rareCaseProfileCountForBytecodeOffset):
+        (JSC::CodeBlock::arithProfileForBytecodeOffset):
+        (JSC::CodeBlock::arithProfileForPC):
+        (JSC::CodeBlock::couldTakeSpecialFastCase):
+        (JSC::CodeBlock::dumpResultProfile): Deleted.
+        (JSC::CodeBlock::resultProfileForBytecodeOffset): Deleted.
+        (JSC::CodeBlock::specialFastCaseProfileCountForBytecodeOffset): Deleted.
+        (JSC::CodeBlock::ensureResultProfile): Deleted.
+        * bytecode/CodeBlock.h:
+        (JSC::CodeBlock::stubInfoBegin):
+        (JSC::CodeBlock::stubInfoEnd):
+        (JSC::CodeBlock::couldTakeSlowCase):
+        (JSC::CodeBlock::numberOfResultProfiles): Deleted.
+        * bytecode/MethodOfGettingAValueProfile.cpp:
+        (JSC::MethodOfGettingAValueProfile::emitReportValue):
+        * bytecode/MethodOfGettingAValueProfile.h:
+        (JSC::MethodOfGettingAValueProfile::MethodOfGettingAValueProfile):
+        * bytecode/ValueProfile.cpp:
+        (JSC::ResultProfile::emitDetectNumericness): Deleted.
+        (JSC::ResultProfile::emitSetDouble): Deleted.
+        (JSC::ResultProfile::emitSetNonNumber): Deleted.
+        (WTF::printInternal): Deleted.
+        * bytecode/ValueProfile.h:
+        (JSC::getRareCaseProfileBytecodeOffset):
+        (JSC::ResultProfile::ResultProfile): Deleted.
+        (JSC::ResultProfile::bytecodeOffset): Deleted.
+        (JSC::ResultProfile::specialFastPathCount): Deleted.
+        (JSC::ResultProfile::didObserveNonInt32): Deleted.
+        (JSC::ResultProfile::didObserveDouble): Deleted.
+        (JSC::ResultProfile::didObserveNonNegZeroDouble): Deleted.
+        (JSC::ResultProfile::didObserveNegZeroDouble): Deleted.
+        (JSC::ResultProfile::didObserveNonNumber): Deleted.
+        (JSC::ResultProfile::didObserveInt32Overflow): Deleted.
+        (JSC::ResultProfile::didObserveInt52Overflow): Deleted.
+        (JSC::ResultProfile::setObservedNonNegZeroDouble): Deleted.
+        (JSC::ResultProfile::setObservedNegZeroDouble): Deleted.
+        (JSC::ResultProfile::setObservedNonNumber): Deleted.
+        (JSC::ResultProfile::setObservedInt32Overflow): Deleted.
+        (JSC::ResultProfile::setObservedInt52Overflow): Deleted.
+        (JSC::ResultProfile::addressOfFlags): Deleted.
+        (JSC::ResultProfile::addressOfSpecialFastPathCount): Deleted.
+        (JSC::ResultProfile::detectNumericness): Deleted.
+        (JSC::ResultProfile::hasBits): Deleted.
+        (JSC::ResultProfile::setBit): Deleted.
+        (JSC::getResultProfileBytecodeOffset): Deleted.
+        * bytecompiler/BytecodeGenerator.cpp:
+        (JSC::BytecodeGenerator::emitBinaryOp):
+        * dfg/DFGByteCodeParser.cpp:
+        (JSC::DFG::ByteCodeParser::makeSafe):
+        * dfg/DFGGraph.cpp:
+        (JSC::DFG::Graph::methodOfGettingAValueProfileFor):
+        * dfg/DFGJITCompiler.cpp:
+        (JSC::DFG::JITCompiler::exceptionCheck):
+        * dfg/DFGSlowPathGenerator.h:
+        (JSC::DFG::SlowPathGenerator::generate):
+        * dfg/DFGSpeculativeJIT.cpp:
+        (JSC::DFG::SpeculativeJIT::addSlowPathGenerator):
+        (JSC::DFG::SpeculativeJIT::runSlowPathGenerators):
+        (JSC::DFG::SpeculativeJIT::compileValueAdd):
+        * dfg/DFGSpeculativeJIT.h:
+        (JSC::DFG::SpeculativeJIT::silentSpillAllRegistersImpl):
+        (JSC::DFG::SpeculativeJIT::silentSpillAllRegisters):
+        (JSC::DFG::SpeculativeJIT::callOperation):
+        * ftl/FTLLowerDFGToB3.cpp:
+        (JSC::FTL::DFG::LowerDFGToB3::compileValueAdd):
+        (JSC::FTL::DFG::LowerDFGToB3::compileStrCat):
+        * jit/CCallHelpers.h:
+        (JSC::CCallHelpers::setupArgumentsWithExecState):
+        (JSC::CCallHelpers::setupArguments):
+        * jit/JIT.h:
+        * jit/JITAddGenerator.cpp:
+        (JSC::JITAddGenerator::generateInline):
+        (JSC::JITAddGenerator::generateFastPath):
+        * jit/JITAddGenerator.h:
+        (JSC::JITAddGenerator::JITAddGenerator):
+        (JSC::JITAddGenerator::didEmitFastPath): Deleted.
+        (JSC::JITAddGenerator::endJumpList): Deleted.
+        (JSC::JITAddGenerator::slowPathJumpList): Deleted.
+        * jit/JITArithmetic.cpp:
+        (JSC::JIT::emit_op_jless):
+        (JSC::JIT::emitSlow_op_urshift):
+        (JSC::getOperandTypes):
+        (JSC::JIT::emit_op_add):
+        (JSC::JIT::emitSlow_op_add):
+        (JSC::JIT::emit_op_div):
+        (JSC::JIT::emit_op_mul):
+        (JSC::JIT::emitSlow_op_mul):
+        (JSC::JIT::emit_op_sub):
+        (JSC::JIT::emitSlow_op_sub):
+        * jit/JITDivGenerator.cpp:
+        (JSC::JITDivGenerator::generateFastPath):
+        * jit/JITDivGenerator.h:
+        (JSC::JITDivGenerator::JITDivGenerator):
+        * jit/JITInlines.h:
+        (JSC::JIT::callOperation):
+        * jit/JITMathIC.h: Added.
+        (JSC::JITMathIC::doneLocation):
+        (JSC::JITMathIC::slowPathStartLocation):
+        (JSC::JITMathIC::slowPathCallLocation):
+        (JSC::JITMathIC::generateInline):
+        (JSC::JITMathIC::generateOutOfLine):
+        (JSC::JITMathIC::finalizeInlineCode):
+        * jit/JITMathICForwards.h: Added.
+        * jit/JITMathICInlineResult.h: Added.
+        * jit/JITMulGenerator.cpp:
+        (JSC::JITMulGenerator::generateFastPath):
+        * jit/JITMulGenerator.h:
+        (JSC::JITMulGenerator::JITMulGenerator):
+        * jit/JITOperations.cpp:
+        * jit/JITOperations.h:
+        * jit/JITSubGenerator.cpp:
+        (JSC::JITSubGenerator::generateFastPath):
+        * jit/JITSubGenerator.h:
+        (JSC::JITSubGenerator::JITSubGenerator):
+        * jit/Repatch.cpp:
+        (JSC::readCallTarget):
+        (JSC::ftlThunkAwareRepatchCall):
+        (JSC::tryCacheGetByID):
+        (JSC::repatchGetByID):
+        (JSC::appropriateGenericPutByIdFunction):
+        (JSC::tryCachePutByID):
+        (JSC::repatchPutByID):
+        (JSC::tryRepatchIn):
+        (JSC::repatchIn):
+        (JSC::linkSlowFor):
+        (JSC::resetGetByID):
+        (JSC::resetPutByID):
+        (JSC::repatchCall): Deleted.
+        * jit/Repatch.h:
+        * llint/LLIntData.cpp:
+        (JSC::LLInt::Data::performAssertions):
+        * llint/LowLevelInterpreter.asm:
+        * llint/LowLevelInterpreter32_64.asm:
+        * llint/LowLevelInterpreter64.asm:
+        * parser/ResultType.h:
+        (JSC::ResultType::ResultType):
+        (JSC::ResultType::isInt32):
+        (JSC::ResultType::definitelyIsNumber):
+        (JSC::ResultType::definitelyIsString):
+        (JSC::ResultType::definitelyIsBoolean):
+        (JSC::ResultType::mightBeNumber):
+        (JSC::ResultType::isNotNumber):
+        (JSC::ResultType::forBitOp):
+        (JSC::ResultType::bits):
+        (JSC::OperandTypes::OperandTypes):
+        * runtime/CommonSlowPaths.cpp:
+        (JSC::SLOW_PATH_DECL):
+        (JSC::updateArithProfileForBinaryArithOp):
+        (JSC::updateResultProfileForBinaryArithOp): Deleted.
+        * tests/stress/op-add-exceptions.js: Added.
+        (assert):
+        (f1):
+        (f2):
+        (f3):
+        (let.oException.valueOf):
+        (foo):
+        (ident):
+        (bar):
+
 2016-07-21  Csaba Osztrogon√°c  <ossy@webkit.org>
 
         Clarify testing mode names in run-jsc-stress-tests
index 6b7668d..51fc062 100644 (file)
                7905BB691D12050E0019FE57 /* InlineAccess.h in Headers */ = {isa = PBXBuildFile; fileRef = 7905BB671D12050E0019FE57 /* InlineAccess.h */; settings = {ATTRIBUTES = (Private, ); }; };
                79160DBD1C8E3EC8008C085A /* ProxyRevoke.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 79160DBB1C8E3EC8008C085A /* ProxyRevoke.cpp */; };
                79160DBE1C8E3EC8008C085A /* ProxyRevoke.h in Headers */ = {isa = PBXBuildFile; fileRef = 79160DBC1C8E3EC8008C085A /* ProxyRevoke.h */; settings = {ATTRIBUTES = (Private, ); }; };
+               79233C2B1D34715700C5A834 /* JITMathIC.h in Headers */ = {isa = PBXBuildFile; fileRef = 79233C291D34715700C5A834 /* JITMathIC.h */; settings = {ATTRIBUTES = (Private, ); }; };
                792CB3491C4EED5C00D13AF3 /* PCToCodeOriginMap.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 792CB3471C4EED5C00D13AF3 /* PCToCodeOriginMap.cpp */; };
                792CB34A1C4EED5C00D13AF3 /* PCToCodeOriginMap.h in Headers */ = {isa = PBXBuildFile; fileRef = 792CB3481C4EED5C00D13AF3 /* PCToCodeOriginMap.h */; settings = {ATTRIBUTES = (Private, ); }; };
                7964656A1B952FF0003059EE /* GetPutInfo.h in Headers */ = {isa = PBXBuildFile; fileRef = 796465681B952FF0003059EE /* GetPutInfo.h */; settings = {ATTRIBUTES = (Private, ); }; };
                797E07A91B8FCFB9008400BA /* JSGlobalLexicalEnvironment.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 797E07A71B8FCFB9008400BA /* JSGlobalLexicalEnvironment.cpp */; };
                797E07AA1B8FCFB9008400BA /* JSGlobalLexicalEnvironment.h in Headers */ = {isa = PBXBuildFile; fileRef = 797E07A81B8FCFB9008400BA /* JSGlobalLexicalEnvironment.h */; settings = {ATTRIBUTES = (Private, ); }; };
                799EF7C41C56ED96002B0534 /* B3PCToOriginMap.h in Headers */ = {isa = PBXBuildFile; fileRef = 799EF7C31C56ED96002B0534 /* B3PCToOriginMap.h */; settings = {ATTRIBUTES = (Private, ); }; };
+               79A228351D35D71E00D8E067 /* ArithProfile.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 79A228331D35D71E00D8E067 /* ArithProfile.cpp */; };
+               79A228361D35D71F00D8E067 /* ArithProfile.h in Headers */ = {isa = PBXBuildFile; fileRef = 79A228341D35D71E00D8E067 /* ArithProfile.h */; };
+               79AF0BE41D3EFD4C00E95FA5 /* JITMathICInlineResult.h in Headers */ = {isa = PBXBuildFile; fileRef = 79AF0BE31D3EFD4C00E95FA5 /* JITMathICInlineResult.h */; settings = {ATTRIBUTES = (Private, ); }; };
                79B00CBC1C6AB07E0088C65D /* ProxyConstructor.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 79B00CB81C6AB07E0088C65D /* ProxyConstructor.cpp */; };
                79B00CBD1C6AB07E0088C65D /* ProxyConstructor.h in Headers */ = {isa = PBXBuildFile; fileRef = 79B00CB91C6AB07E0088C65D /* ProxyConstructor.h */; settings = {ATTRIBUTES = (Private, ); }; };
                79B00CBE1C6AB07E0088C65D /* ProxyObject.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 79B00CBA1C6AB07E0088C65D /* ProxyObject.cpp */; settings = {COMPILER_FLAGS = "-fno-optimize-sibling-calls"; }; };
                79B00CBF1C6AB07E0088C65D /* ProxyObject.h in Headers */ = {isa = PBXBuildFile; fileRef = 79B00CBB1C6AB07E0088C65D /* ProxyObject.h */; settings = {ATTRIBUTES = (Private, ); }; };
+               79B1788E1D399B8000B1A567 /* JITMathICForwards.h in Headers */ = {isa = PBXBuildFile; fileRef = 79A899FE1D38612E00D18C73 /* JITMathICForwards.h */; settings = {ATTRIBUTES = (Private, ); }; };
                79C4B15D1BA2158F00FD592E /* DFGLiveCatchVariablePreservationPhase.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 79C4B15B1BA2158F00FD592E /* DFGLiveCatchVariablePreservationPhase.cpp */; };
                79C4B15E1BA2158F00FD592E /* DFGLiveCatchVariablePreservationPhase.h in Headers */ = {isa = PBXBuildFile; fileRef = 79C4B15C1BA2158F00FD592E /* DFGLiveCatchVariablePreservationPhase.h */; settings = {ATTRIBUTES = (Private, ); }; };
                79CFC6F01C33B10000C768EA /* LLIntPCRanges.h in Headers */ = {isa = PBXBuildFile; fileRef = 79CFC6EF1C33B10000C768EA /* LLIntPCRanges.h */; settings = {ATTRIBUTES = (Private, ); }; };
                7905BB671D12050E0019FE57 /* InlineAccess.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = InlineAccess.h; sourceTree = "<group>"; };
                79160DBB1C8E3EC8008C085A /* ProxyRevoke.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ProxyRevoke.cpp; sourceTree = "<group>"; };
                79160DBC1C8E3EC8008C085A /* ProxyRevoke.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ProxyRevoke.h; sourceTree = "<group>"; };
+               79233C291D34715700C5A834 /* JITMathIC.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JITMathIC.h; sourceTree = "<group>"; };
                792CB3471C4EED5C00D13AF3 /* PCToCodeOriginMap.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = PCToCodeOriginMap.cpp; sourceTree = "<group>"; };
                792CB3481C4EED5C00D13AF3 /* PCToCodeOriginMap.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = PCToCodeOriginMap.h; sourceTree = "<group>"; };
                796465681B952FF0003059EE /* GetPutInfo.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = GetPutInfo.h; sourceTree = "<group>"; };
                797E07A71B8FCFB9008400BA /* JSGlobalLexicalEnvironment.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = JSGlobalLexicalEnvironment.cpp; sourceTree = "<group>"; };
                797E07A81B8FCFB9008400BA /* JSGlobalLexicalEnvironment.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSGlobalLexicalEnvironment.h; sourceTree = "<group>"; };
                799EF7C31C56ED96002B0534 /* B3PCToOriginMap.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = B3PCToOriginMap.h; path = b3/B3PCToOriginMap.h; sourceTree = "<group>"; };
+               79A228331D35D71E00D8E067 /* ArithProfile.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ArithProfile.cpp; sourceTree = "<group>"; };
+               79A228341D35D71E00D8E067 /* ArithProfile.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ArithProfile.h; sourceTree = "<group>"; };
+               79A899FE1D38612E00D18C73 /* JITMathICForwards.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JITMathICForwards.h; sourceTree = "<group>"; };
+               79AF0BE31D3EFD4C00E95FA5 /* JITMathICInlineResult.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JITMathICInlineResult.h; sourceTree = "<group>"; };
                79B00CB81C6AB07E0088C65D /* ProxyConstructor.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ProxyConstructor.cpp; sourceTree = "<group>"; };
                79B00CB91C6AB07E0088C65D /* ProxyConstructor.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ProxyConstructor.h; sourceTree = "<group>"; };
                79B00CBA1C6AB07E0088C65D /* ProxyObject.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ProxyObject.cpp; sourceTree = "<group>"; };
                                DE5A09FF1BA3AC3E003D4424 /* IntrinsicEmitter.cpp */,
                                1429D92D0ED22D7000B89619 /* JIT.cpp */,
                                1429D92E0ED22D7000B89619 /* JIT.h */,
+                               79233C291D34715700C5A834 /* JITMathIC.h */,
+                               79A899FE1D38612E00D18C73 /* JITMathICForwards.h */,
+                               79AF0BE31D3EFD4C00E95FA5 /* JITMathICInlineResult.h */,
                                FE1220251BE7F5640039E6F2 /* JITAddGenerator.cpp */,
                                FE1220261BE7F5640039E6F2 /* JITAddGenerator.h */,
                                86A90ECF0EE7D51F00AB350D /* JITArithmetic.cpp */,
                                0F8335B51639C1E3001443B5 /* ArrayAllocationProfile.h */,
                                0F63945115D07051006A597C /* ArrayProfile.cpp */,
                                0F63945215D07051006A597C /* ArrayProfile.h */,
+                               79A228331D35D71E00D8E067 /* ArithProfile.cpp */,
+                               79A228341D35D71E00D8E067 /* ArithProfile.h */,
                                C2FCAE0C17A9C24E0034C735 /* BytecodeBasicBlock.cpp */,
                                C2FCAE0D17A9C24E0034C735 /* BytecodeBasicBlock.h */,
                                0F21C27E14BEAA8000ADC64B /* BytecodeConventions.h */,
                                A5EA70E819F5B1010098F5EC /* AugmentableInspectorControllerClient.h in Headers */,
                                0FEC84FF1BDACDAC0080FF74 /* B3ArgumentRegValue.h in Headers */,
                                0FEC85011BDACDAC0080FF74 /* B3BasicBlock.h in Headers */,
+                               79233C2B1D34715700C5A834 /* JITMathIC.h in Headers */,
                                0FEC85021BDACDAC0080FF74 /* B3BasicBlockInlines.h in Headers */,
                                0FEC85031BDACDAC0080FF74 /* B3BasicBlockUtils.h in Headers */,
                                0FEC85041BDACDAC0080FF74 /* B3BlockWorklist.h in Headers */,
                                A7A8AF3A17ADB5F3005AB174 /* Int8Array.h in Headers */,
                                BC11667B0E199C05008066DD /* InternalFunction.h in Headers */,
                                1429D77C0ED20D7300B89619 /* Interpreter.h in Headers */,
+                               79AF0BE41D3EFD4C00E95FA5 /* JITMathICInlineResult.h in Headers */,
                                A1B9E23A1B4E0D6700BC7FED /* IntlCollator.h in Headers */,
                                A1B9E23C1B4E0D6700BC7FED /* IntlCollatorConstructor.h in Headers */,
                                A18193E31B4E0CDB00FC1029 /* IntlCollatorConstructor.lut.h in Headers */,
                                A51007C1187CC3C600B38879 /* JSGlobalObjectInspectorController.h in Headers */,
                                A50E4B6418809DD50068A46D /* JSGlobalObjectRuntimeAgent.h in Headers */,
                                A503FA2A188F105900110F14 /* JSGlobalObjectScriptDebugServer.h in Headers */,
+                               79A228361D35D71F00D8E067 /* ArithProfile.h in Headers */,
                                A513E5C0185BFACC007E95AD /* JSInjectedScriptHost.h in Headers */,
                                A513E5C2185BFACC007E95AD /* JSInjectedScriptHostPrototype.h in Headers */,
                                C442CB251A6CDB8C005D3D7C /* JSInputs.json in Headers */,
                                0FC8150A14043BF500CFA603 /* WriteBarrierSupport.h in Headers */,
                                9688CB160ED12B4E001D649F /* X86Assembler.h in Headers */,
                                9959E92E1BD17FA4001AA413 /* xxd.pl in Headers */,
+                               79B1788E1D399B8000B1A567 /* JITMathICForwards.h in Headers */,
                                451539B912DC994500EF7AC4 /* Yarr.h in Headers */,
                                86704B8512DBA33700A9FE7B /* YarrInterpreter.h in Headers */,
                                86704B8712DBA33700A9FE7B /* YarrJIT.h in Headers */,
                                0FEB3ECF16237F6C00AB67AD /* MacroAssembler.cpp in Sources */,
                                86C568E011A213EE0007F7F0 /* MacroAssemblerARM.cpp in Sources */,
                                FEB137571BB11EF900CD5100 /* MacroAssemblerARM64.cpp in Sources */,
+                               79A228351D35D71E00D8E067 /* ArithProfile.cpp in Sources */,
                                A729009C17976C6000317298 /* MacroAssemblerARMv7.cpp in Sources */,
                                FE68C6381B90DE0B0042BCB3 /* MacroAssemblerPrinter.cpp in Sources */,
                                A7A4AE0817973B26005612B1 /* MacroAssemblerX86Common.cpp in Sources */,
diff --git a/Source/JavaScriptCore/bytecode/ArithProfile.cpp b/Source/JavaScriptCore/bytecode/ArithProfile.cpp
new file mode 100644 (file)
index 0000000..6154306
--- /dev/null
@@ -0,0 +1,143 @@
+/*
+ * Copyright (C) 2016 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#include "config.h"
+#include "ArithProfile.h"
+
+#include "CCallHelpers.h"
+#include "JSCInlines.h"
+
+namespace JSC {
+
+#if ENABLE(JIT)
+void ArithProfile::emitObserveResult(CCallHelpers& jit, JSValueRegs regs, TagRegistersMode mode)
+{
+    if (!shouldEmitSetDouble() && !shouldEmitSetNonNumber())
+        return;
+
+    CCallHelpers::Jump isInt32 = jit.branchIfInt32(regs, mode);
+    CCallHelpers::Jump notDouble = jit.branchIfNotDoubleKnownNotInt32(regs, mode);
+    emitSetDouble(jit);
+    CCallHelpers::Jump done = jit.jump();
+    notDouble.link(&jit);
+    emitSetNonNumber(jit);
+    done.link(&jit);
+    isInt32.link(&jit);
+}
+
+bool ArithProfile::shouldEmitSetDouble() const
+{
+    uint32_t mask = ArithProfile::Int32Overflow | ArithProfile::Int52Overflow | ArithProfile::NegZeroDouble | ArithProfile::NonNegZeroDouble;
+    return (m_bits & mask) != mask;
+}
+
+void ArithProfile::emitSetDouble(CCallHelpers& jit)
+{
+    if (shouldEmitSetDouble())
+        jit.or32(CCallHelpers::TrustedImm32(ArithProfile::Int32Overflow | ArithProfile::Int52Overflow | ArithProfile::NegZeroDouble | ArithProfile::NonNegZeroDouble), CCallHelpers::AbsoluteAddress(addressOfBits()));
+}
+
+bool ArithProfile::shouldEmitSetNonNumber() const
+{
+    uint32_t mask = ArithProfile::NonNumber;
+    return (m_bits & mask) != mask;
+}
+
+void ArithProfile::emitSetNonNumber(CCallHelpers& jit)
+{
+    if (shouldEmitSetNonNumber())
+        jit.or32(CCallHelpers::TrustedImm32(ArithProfile::NonNumber), CCallHelpers::AbsoluteAddress(addressOfBits()));
+}
+#endif // ENABLE(JIT)
+
+} // namespace JSC
+
+namespace WTF {
+    
+using namespace JSC;
+
+void printInternal(PrintStream& out, const ArithProfile& profile)
+{
+    const char* separator = "";
+
+    out.print("Result:<");
+    if (!profile.didObserveNonInt32()) {
+        out.print("Int32");
+        separator = "|";
+    } else {
+        if (profile.didObserveNegZeroDouble()) {
+            out.print(separator, "NegZeroDouble");
+            separator = "|";
+        }
+        if (profile.didObserveNonNegZeroDouble()) {
+            out.print(separator, "NonNegZeroDouble");
+            separator = "|";
+        }
+        if (profile.didObserveNonNumber()) {
+            out.print(separator, "NonNumber");
+            separator = "|";
+        }
+        if (profile.didObserveInt32Overflow()) {
+            out.print(separator, "Int32Overflow");
+            separator = "|";
+        }
+        if (profile.didObserveInt52Overflow()) {
+            out.print(separator, "Int52Overflow");
+            separator = "|";
+        }
+    }
+    if (profile.tookSpecialFastPath())
+        out.print(separator, "Took special fast path.");
+    out.print(">");
+
+    out.print(" LHS ObservedType:<");
+    out.print(profile.lhsObservedType());
+    out.print("> RHS ObservedType:<");
+    out.print(profile.rhsObservedType());
+    out.print(">");
+
+    out.print(" LHS ResultType:<", RawPointer(bitwise_cast<void*>(static_cast<uintptr_t>(profile.lhsResultType().bits()))));
+    out.print("> RHS ResultType:<", RawPointer(bitwise_cast<void*>(static_cast<uintptr_t>(profile.rhsResultType().bits()))));
+    out.print(">");
+}
+
+void printInternal(PrintStream& out, const JSC::ObservedType& observedType)
+{
+    const char* separator = "";
+    if (observedType.sawInt32()) {
+        out.print(separator, "Int32");
+        separator = "|";
+    }
+    if (observedType.sawNumber()) {
+        out.print(separator, "Number");
+        separator = "|";
+    }
+    if (observedType.sawNonNumber()) {
+        out.print(separator, "NonNumber");
+        separator = "|";
+    }
+}
+
+} // namespace WTF
diff --git a/Source/JavaScriptCore/bytecode/ArithProfile.h b/Source/JavaScriptCore/bytecode/ArithProfile.h
new file mode 100644 (file)
index 0000000..1d30f43
--- /dev/null
@@ -0,0 +1,225 @@
+/*
+ * Copyright (C) 2016 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#pragma once
+
+#include "GPRInfo.h"
+#include "JSCJSValue.h"
+#include "ResultType.h"
+#include "TagRegistersMode.h"
+
+namespace JSC {
+
+class CCallHelpers;
+
+struct ObservedType {
+    ObservedType(uint8_t bits = TypeEmpty)
+        : m_bits(bits)
+    { }
+
+    bool sawInt32() const { return m_bits & TypeInt32; }
+    bool isOnlyInt32() const { return m_bits == TypeInt32; }
+    bool sawNumber() const { return m_bits & TypeNumber; }
+    bool isOnlyNumber() const { return m_bits == TypeNumber; }
+    bool sawNonNumber() const { return m_bits & TypeNonNumber; }
+    bool isOnlyNonNumber() const { return m_bits == TypeNonNumber; }
+    bool isEmpty() const { return !m_bits; }
+    uint8_t bits() const { return m_bits; }
+
+    ObservedType withInt32() const { return ObservedType(m_bits | TypeInt32); }
+    ObservedType withNumber() const { return ObservedType(m_bits | TypeNumber); }
+    ObservedType withNonNumber() const { return ObservedType(m_bits | TypeNonNumber); }
+    ObservedType withoutNonNumber() const { return ObservedType(m_bits & ~TypeNonNumber); }
+
+    bool operator==(const ObservedType& other) const { return m_bits == other.m_bits; }
+
+    static const uint8_t TypeEmpty = 0x0;
+    static const uint8_t TypeInt32 = 0x1;
+    static const uint8_t TypeNumber = 0x02;
+    static const uint8_t TypeNonNumber = 0x04;
+
+    static const uint32_t numBitsNeeded = 3;
+
+private:
+    uint8_t m_bits;
+};
+
+struct ArithProfile {
+private:
+    static const uint32_t numberOfFlagBits = 5;
+    static const uint32_t rhsResultTypeShift = numberOfFlagBits;
+    static const uint32_t lhsResultTypeShift = rhsResultTypeShift + ResultType::numBitsNeeded;
+    static const uint32_t rhsObservedTypeShift = lhsResultTypeShift + ResultType::numBitsNeeded;
+    static const uint32_t lhsObservedTypeShift = rhsObservedTypeShift + ObservedType::numBitsNeeded;
+
+    static_assert(ObservedType::numBitsNeeded == 3, "We make a hard assumption about that here.");
+    static const uint32_t clearRhsObservedTypeBitMask = ~((1 << rhsObservedTypeShift) | (1 << (rhsObservedTypeShift + 1)) | (1 << (rhsObservedTypeShift + 2)));
+    static const uint32_t clearLhsObservedTypeBitMask = ~((1 << lhsObservedTypeShift) | (1 << (lhsObservedTypeShift + 1)) | (1 << (lhsObservedTypeShift + 2)));
+
+    static const uint32_t resultTypeMask = (1 << ResultType::numBitsNeeded) - 1;
+    static const uint32_t observedTypeMask = (1 << ObservedType::numBitsNeeded) - 1;
+public:
+    static const uint32_t specialFastPathBit = 1 << (lhsObservedTypeShift + ObservedType::numBitsNeeded);
+    static_assert((lhsObservedTypeShift + ObservedType::numBitsNeeded) <= (sizeof(uint32_t) * 8) - 1, "Should fit in a uint32_t.");
+    static_assert(!(specialFastPathBit & ~clearLhsObservedTypeBitMask), "These bits should not intersect.");
+    static_assert(specialFastPathBit & clearLhsObservedTypeBitMask, "These bits should intersect.");
+    static_assert(specialFastPathBit > ~clearLhsObservedTypeBitMask, "These bits should not intersect and specialFastPathBit should be a higher bit.");
+
+    ArithProfile(ResultType lhs, ResultType rhs)
+    {
+        m_bits = (lhs.bits() << lhsResultTypeShift) | (rhs.bits() << rhsResultTypeShift);
+        ASSERT(lhsResultType().bits() == lhs.bits() && rhsResultType().bits() == rhs.bits());
+        ASSERT(lhsObservedType().isEmpty());
+        ASSERT(rhsObservedType().isEmpty());
+    }
+    ArithProfile() { }
+
+    static ArithProfile fromInt(uint32_t bits)
+    {
+        ArithProfile result;
+        result.m_bits = bits;
+        return result;
+    }
+
+    enum ObservedResults {
+        NonNegZeroDouble = 1 << 0,
+        NegZeroDouble    = 1 << 1,
+        NonNumber        = 1 << 2,
+        Int32Overflow    = 1 << 3,
+        Int52Overflow    = 1 << 4,
+    };
+
+    ResultType lhsResultType() const { return ResultType((m_bits >> lhsResultTypeShift) & resultTypeMask); }
+    ResultType rhsResultType() const { return ResultType((m_bits >> rhsResultTypeShift) & resultTypeMask); }
+
+    ObservedType lhsObservedType() const { return ObservedType((m_bits >> lhsObservedTypeShift) & observedTypeMask); }
+    ObservedType rhsObservedType() const { return ObservedType((m_bits >> rhsObservedTypeShift) & observedTypeMask); }
+    void setLhsObservedType(ObservedType type)
+    {
+        uint32_t bits = m_bits;
+        bits &= clearLhsObservedTypeBitMask;
+        bits |= type.bits() << lhsObservedTypeShift;
+        m_bits = bits;
+        ASSERT(lhsObservedType() == type);
+    }
+
+    void setRhsObservedType(ObservedType type)
+    { 
+        uint32_t bits = m_bits;
+        bits &= clearRhsObservedTypeBitMask;
+        bits |= type.bits() << rhsObservedTypeShift;
+        m_bits = bits;
+        ASSERT(rhsObservedType() == type);
+    }
+
+    bool tookSpecialFastPath() const { return m_bits & specialFastPathBit; }
+
+    bool didObserveNonInt32() const { return hasBits(NonNegZeroDouble | NegZeroDouble | NonNumber); }
+    bool didObserveDouble() const { return hasBits(NonNegZeroDouble | NegZeroDouble); }
+    bool didObserveNonNegZeroDouble() const { return hasBits(NonNegZeroDouble); }
+    bool didObserveNegZeroDouble() const { return hasBits(NegZeroDouble); }
+    bool didObserveNonNumber() const { return hasBits(NonNumber); }
+    bool didObserveInt32Overflow() const { return hasBits(Int32Overflow); }
+    bool didObserveInt52Overflow() const { return hasBits(Int52Overflow); }
+
+    void setObservedNonNegZeroDouble() { setBit(NonNegZeroDouble); }
+    void setObservedNegZeroDouble() { setBit(NegZeroDouble); }
+    void setObservedNonNumber() { setBit(NonNumber); }
+    void setObservedInt32Overflow() { setBit(Int32Overflow); }
+    void setObservedInt52Overflow() { setBit(Int52Overflow); }
+
+    void* addressOfBits() { return &m_bits; }
+
+    void observeResult(JSValue value)
+    {
+        if (value.isInt32())
+            return;
+        if (value.isNumber()) {
+            m_bits |= Int32Overflow | Int52Overflow | NonNegZeroDouble | NegZeroDouble;
+            return;
+        }
+        m_bits |= NonNumber;
+    }
+
+    void lhsSawInt32() { setLhsObservedType(lhsObservedType().withInt32()); }
+    void lhsSawNumber() { setLhsObservedType(lhsObservedType().withNumber()); }
+    void lhsSawNonNumber() { setLhsObservedType(lhsObservedType().withNonNumber()); }
+    void rhsSawInt32() { setRhsObservedType(rhsObservedType().withInt32()); }
+    void rhsSawNumber() { setRhsObservedType(rhsObservedType().withNumber()); }
+    void rhsSawNonNumber() { setRhsObservedType(rhsObservedType().withNonNumber()); }
+
+    void observeLHSAndRHS(JSValue lhs, JSValue rhs)
+    {
+        ArithProfile newProfile = *this;
+        if (lhs.isNumber()) {
+            if (lhs.isInt32())
+                newProfile.lhsSawInt32();
+            else
+                newProfile.lhsSawNumber();
+        } else
+            newProfile.lhsSawNonNumber();
+
+        if (rhs.isNumber()) {
+            if (rhs.isInt32())
+                newProfile.rhsSawInt32();
+            else
+                newProfile.rhsSawNumber();
+        } else
+            newProfile.rhsSawNonNumber();
+
+        m_bits = newProfile.bits();
+    }
+
+#if ENABLE(JIT)    
+    // Sets (Int32Overflow | Int52Overflow | NonNegZeroDouble | NegZeroDouble) if it sees a
+    // double. Sets NonNumber if it sees a non-number.
+    void emitObserveResult(CCallHelpers&, JSValueRegs, TagRegistersMode = HaveTagRegisters);
+    
+    // Sets (Int32Overflow | Int52Overflow | NonNegZeroDouble | NegZeroDouble).
+    bool shouldEmitSetDouble() const;
+    void emitSetDouble(CCallHelpers&);
+    
+    // Sets NonNumber.
+    void emitSetNonNumber(CCallHelpers&);
+    bool shouldEmitSetNonNumber() const;
+#endif // ENABLE(JIT)
+
+    uint32_t bits() const { return m_bits; }
+
+private:
+    bool hasBits(int mask) const { return m_bits & mask; }
+    void setBit(int mask) { m_bits |= mask; }
+
+    uint32_t m_bits { 0 }; // We take care to update m_bits only in a single operation. We don't ever store an inconsistent bit representation to it.
+};
+
+} // namespace JSC
+
+namespace WTF {
+
+void printInternal(PrintStream&, const JSC::ArithProfile&);
+void printInternal(PrintStream&, const JSC::ObservedType&);
+
+} // namespace WTF
index 3f7d2f0..ee81916 100644 (file)
@@ -30,6 +30,7 @@
 #include "config.h"
 #include "CodeBlock.h"
 
+#include "ArithProfile.h"
 #include "BasicBlockLocation.h"
 #include "BytecodeGenerator.h"
 #include "BytecodeUseDef.h"
@@ -45,6 +46,7 @@
 #include "InlineCallFrame.h"
 #include "Interpreter.h"
 #include "JIT.h"
+#include "JITMathIC.h"
 #include "JSCJSValue.h"
 #include "JSFunction.h"
 #include "JSLexicalEnvironment.h"
@@ -769,7 +771,7 @@ void CodeBlock::dumpRareCaseProfile(PrintStream& out, const char* name, RareCase
     out.print(name, profile->m_counter);
 }
 
-void CodeBlock::dumpResultProfile(PrintStream& out, ResultProfile* profile, bool& hasPrintedProfiling)
+void CodeBlock::dumpArithProfile(PrintStream& out, ArithProfile* profile, bool& hasPrintedProfiling)
 {
     if (!profile)
         return;
@@ -1773,8 +1775,7 @@ void CodeBlock::dumpBytecode(
 
     dumpRareCaseProfile(out, "rare case: ", rareCaseProfileForBytecodeOffset(location), hasPrintedProfiling);
     {
-        ConcurrentJITLocker locker(m_lock);
-        dumpResultProfile(out, resultProfileForBytecodeOffset(locker, location), hasPrintedProfiling);
+        dumpArithProfile(out, arithProfileForBytecodeOffset(location), hasPrintedProfiling);
     }
     
 #if ENABLE(DFG_JIT)
@@ -3007,6 +3008,11 @@ StructureStubInfo* CodeBlock::addStubInfo(AccessType accessType)
     return m_stubInfos.add(accessType);
 }
 
+JITAddIC* CodeBlock::addJITAddIC()
+{
+    return m_addICs.add();
+}
+
 StructureStubInfo* CodeBlock::findStubInfo(CodeOrigin codeOrigin)
 {
     for (StructureStubInfo* stubInfo : m_stubInfos) {
@@ -3053,11 +3059,6 @@ void CodeBlock::resetJITData()
     // We can clear this because the DFG's queries to these data structures are guarded by whether
     // there is JIT code.
     m_rareCaseProfiles.clear();
-    
-    // We can clear these because the DFG only accesses members of this data structure when
-    // holding the lock or after querying whether we have JIT code.
-    m_resultProfiles.clear();
-    m_bytecodeOffsetToResultProfileIndexMap = nullptr;
 }
 #endif
 
@@ -3287,7 +3288,6 @@ void CodeBlock::shrinkToFit(ShrinkMode shrinkMode)
     ConcurrentJITLocker locker(m_lock);
 
     m_rareCaseProfiles.shrinkToFit();
-    m_resultProfiles.shrinkToFit();
     
     if (shrinkMode == EarlyShrink) {
         m_constantRegisters.shrinkToFit();
@@ -4163,11 +4163,6 @@ void CodeBlock::dumpValueProfiles()
         RareCaseProfile* profile = rareCaseProfile(i);
         dataLogF("   bc = %d: %u\n", profile->m_bytecodeOffset, profile->m_counter);
     }
-    dataLog("ResultProfile for ", *this, ":\n");
-    for (unsigned i = 0; i < numberOfResultProfiles(); ++i) {
-        const ResultProfile& profile = *resultProfile(i);
-        dataLog("   bc = ", profile.bytecodeOffset(), ": ", profile, "\n");
-    }
 }
 #endif // ENABLE(VERBOSE_VALUE_PROFILE)
 
@@ -4370,56 +4365,55 @@ unsigned CodeBlock::rareCaseProfileCountForBytecodeOffset(int bytecodeOffset)
     return 0;
 }
 
-ResultProfile* CodeBlock::resultProfileForBytecodeOffset(const ConcurrentJITLocker&, int bytecodeOffset)
+ArithProfile* CodeBlock::arithProfileForBytecodeOffset(int bytecodeOffset)
 {
-    if (!m_bytecodeOffsetToResultProfileIndexMap)
-        return nullptr;
-    auto iterator = m_bytecodeOffsetToResultProfileIndexMap->find(bytecodeOffset);
-    if (iterator == m_bytecodeOffsetToResultProfileIndexMap->end())
+    auto opcodeID = vm()->interpreter->getOpcodeID(instructions()[bytecodeOffset].u.opcode);
+    switch (opcodeID) {
+    case op_bitor:
+    case op_bitand:
+    case op_bitxor:
+    case op_add:
+    case op_mul:
+    case op_sub:
+    case op_div:
+        break;
+    default:
         return nullptr;
-    return &m_resultProfiles[iterator->value];
-}
+    }
 
-unsigned CodeBlock::specialFastCaseProfileCountForBytecodeOffset(int bytecodeOffset)
-{
-    ConcurrentJITLocker locker(m_lock);
-    return specialFastCaseProfileCountForBytecodeOffset(locker, bytecodeOffset);
+    return &arithProfileForPC(instructions().begin() + bytecodeOffset);
 }
 
-unsigned CodeBlock::specialFastCaseProfileCountForBytecodeOffset(const ConcurrentJITLocker& locker, int bytecodeOffset)
+ArithProfile& CodeBlock::arithProfileForPC(Instruction* pc)
 {
-    ResultProfile* profile = resultProfileForBytecodeOffset(locker, bytecodeOffset);
-    if (!profile)
-        return 0;
-    return profile->specialFastPathCount();
+    if (!ASSERT_DISABLED) {
+        ASSERT(pc >= instructions().begin() && pc < instructions().end());
+        auto opcodeID = vm()->interpreter->getOpcodeID(pc[0].u.opcode);
+        switch (opcodeID) {
+        case op_bitor:
+        case op_bitand:
+        case op_bitxor:
+        case op_add:
+        case op_mul:
+        case op_sub:
+        case op_div:
+            break;
+        default:
+            ASSERT_NOT_REACHED();
+        }
+    }
+
+    return *bitwise_cast<ArithProfile*>(&pc[4].u.operand);
 }
 
 bool CodeBlock::couldTakeSpecialFastCase(int bytecodeOffset)
 {
     if (!hasBaselineJITProfiling())
         return false;
-    unsigned specialFastCaseCount = specialFastCaseProfileCountForBytecodeOffset(bytecodeOffset);
-    return specialFastCaseCount >= Options::couldTakeSlowCaseMinimumCount();
-}
-
-ResultProfile* CodeBlock::ensureResultProfile(int bytecodeOffset)
-{
-    ConcurrentJITLocker locker(m_lock);
-    return ensureResultProfile(locker, bytecodeOffset);
-}
-
-ResultProfile* CodeBlock::ensureResultProfile(const ConcurrentJITLocker& locker, int bytecodeOffset)
-{
-    ResultProfile* profile = resultProfileForBytecodeOffset(locker, bytecodeOffset);
-    if (!profile) {
-        m_resultProfiles.append(ResultProfile(bytecodeOffset));
-        profile = &m_resultProfiles.last();
-        ASSERT(&m_resultProfiles.last() == &m_resultProfiles[m_resultProfiles.size() - 1]);
-        if (!m_bytecodeOffsetToResultProfileIndexMap)
-            m_bytecodeOffsetToResultProfileIndexMap = std::make_unique<BytecodeOffsetToResultProfileIndexMap>();
-        m_bytecodeOffsetToResultProfileIndexMap->add(bytecodeOffset, m_resultProfiles.size() - 1);
-    }
-    return profile;
+    ArithProfile* profile = arithProfileForBytecodeOffset(bytecodeOffset);
+    if (!profile)
+        return false;
+    return profile->tookSpecialFastPath();
 }
 
 #if ENABLE(JIT)
index b288365..480a1cb 100644 (file)
@@ -51,6 +51,7 @@
 #include "HandlerInfo.h"
 #include "Instruction.h"
 #include "JITCode.h"
+#include "JITMathICForwards.h"
 #include "JITWriteBarrier.h"
 #include "JSCell.h"
 #include "JSGlobalObject.h"
@@ -79,6 +80,7 @@
 namespace JSC {
 
 class ExecState;
+class JITAddGenerator;
 class JSModuleEnvironment;
 class LLIntOffsetsExtractor;
 class PCToCodeOriginMap;
@@ -88,6 +90,8 @@ class TypeLocation;
 
 enum class AccessType : int8_t;
 
+struct ArithProfile;
+
 typedef HashMap<CodeOrigin, StructureStubInfo*, CodeOriginApproximateHash> StubInfoMap;
 
 enum ReoptimizationMode { DontCountReoptimization, CountReoptimization };
@@ -245,6 +249,7 @@ public:
     
 #if ENABLE(JIT)
     StructureStubInfo* addStubInfo(AccessType);
+    JITAddIC* addJITAddIC();
     Bag<StructureStubInfo>::iterator stubInfoBegin() { return m_stubInfos.begin(); }
     Bag<StructureStubInfo>::iterator stubInfoEnd() { return m_stubInfos.end(); }
     
@@ -440,13 +445,8 @@ public:
         return value >= Options::couldTakeSlowCaseMinimumCount();
     }
 
-    ResultProfile* ensureResultProfile(int bytecodeOffset);
-    ResultProfile* ensureResultProfile(const ConcurrentJITLocker&, int bytecodeOffset);
-    unsigned numberOfResultProfiles() { return m_resultProfiles.size(); }
-    ResultProfile* resultProfileForBytecodeOffset(const ConcurrentJITLocker&, int bytecodeOffset);
-
-    unsigned specialFastCaseProfileCountForBytecodeOffset(const ConcurrentJITLocker&, int bytecodeOffset);
-    unsigned specialFastCaseProfileCountForBytecodeOffset(int bytecodeOffset);
+    ArithProfile* arithProfileForBytecodeOffset(int bytecodeOffset);
+    ArithProfile& arithProfileForPC(Instruction*);
 
     bool couldTakeSpecialFastCase(int bytecodeOffset);
 
@@ -957,7 +957,7 @@ private:
     void dumpValueProfiling(PrintStream&, const Instruction*&, bool& hasPrintedProfiling);
     void dumpArrayProfiling(PrintStream&, const Instruction*&, bool& hasPrintedProfiling);
     void dumpRareCaseProfile(PrintStream&, const char* name, RareCaseProfile*, bool& hasPrintedProfiling);
-    void dumpResultProfile(PrintStream&, ResultProfile*, bool& hasPrintedProfiling);
+    void dumpArithProfile(PrintStream&, ArithProfile*, bool& hasPrintedProfiling);
 
     bool shouldVisitStrongly();
     bool shouldJettisonDueToWeakReference();
@@ -1013,6 +1013,7 @@ private:
 #if ENABLE(JIT)
     std::unique_ptr<RegisterAtOffsetList> m_calleeSaveRegisters;
     Bag<StructureStubInfo> m_stubInfos;
+    Bag<JITAddIC> m_addICs;
     Bag<ByValInfo> m_byValInfos;
     Bag<CallLinkInfo> m_callLinkInfos;
     SentinelLinkedList<CallLinkInfo, BasicRawSentinelNode<CallLinkInfo>> m_incomingCalls;
@@ -1029,9 +1030,6 @@ private:
     RefCountedArray<ValueProfile> m_argumentValueProfiles;
     RefCountedArray<ValueProfile> m_valueProfiles;
     SegmentedVector<RareCaseProfile, 8> m_rareCaseProfiles;
-    SegmentedVector<ResultProfile, 8> m_resultProfiles;
-    typedef HashMap<unsigned, unsigned, IntHash<unsigned>, WTF::UnsignedWithZeroKeyHashTraits<unsigned>> BytecodeOffsetToResultProfileIndexMap;
-    std::unique_ptr<BytecodeOffsetToResultProfileIndexMap> m_bytecodeOffsetToResultProfileIndexMap;
     RefCountedArray<ArrayAllocationProfile> m_arrayAllocationProfiles;
     ArrayProfileVector m_arrayProfiles;
     RefCountedArray<ObjectAllocationProfile> m_objectAllocationProfiles;
index 4e059e2..0795009 100644 (file)
@@ -28,6 +28,7 @@
 
 #if ENABLE(DFG_JIT)
 
+#include "ArithProfile.h"
 #include "CCallHelpers.h"
 #include "CodeBlock.h"
 #include "JSCInlines.h"
@@ -65,8 +66,8 @@ void MethodOfGettingAValueProfile::emitReportValue(CCallHelpers& jit, JSValueReg
         return;
     }
         
-    case ResultProfileReady: {
-        u.resultProfile->emitDetectNumericness(jit, regs, DoNotHaveTagRegisters);
+    case ArithProfileReady: {
+        u.arithProfile->emitObserveResult(jit, regs, DoNotHaveTagRegisters);
         return;
     } }
     
index 127eb9a..dcfc511 100644 (file)
@@ -40,7 +40,7 @@ namespace JSC {
 class CCallHelpers;
 class CodeBlock;
 class LazyOperandValueProfileKey;
-struct ResultProfile;
+struct ArithProfile;
 struct ValueProfile;
 
 class MethodOfGettingAValueProfile {
@@ -59,11 +59,11 @@ public:
             m_kind = None;
     }
     
-    MethodOfGettingAValueProfile(ResultProfile* profile)
+    MethodOfGettingAValueProfile(ArithProfile* profile)
     {
         if (profile) {
-            m_kind = ResultProfileReady;
-            u.resultProfile = profile;
+            m_kind = ArithProfileReady;
+            u.arithProfile = profile;
         } else
             m_kind = None;
     }
@@ -79,14 +79,14 @@ private:
     enum Kind {
         None,
         Ready,
-        ResultProfileReady,
+        ArithProfileReady,
         LazyOperand
     };
     
     Kind m_kind;
     union {
         ValueProfile* profile;
-        ResultProfile* resultProfile;
+        ArithProfile* arithProfile;
         struct {
             CodeBlock* codeBlock;
             unsigned bytecodeOffset;
index b145d01..6382a62 100644 (file)
 
 namespace JSC {
 
-#if ENABLE(JIT)
-void ResultProfile::emitDetectNumericness(CCallHelpers& jit, JSValueRegs regs, TagRegistersMode mode)
-{
-    CCallHelpers::Jump isInt32 = jit.branchIfInt32(regs, mode);
-    CCallHelpers::Jump notDouble = jit.branchIfNotDoubleKnownNotInt32(regs, mode);
-    // FIXME: We could be more precise here.
-    emitSetDouble(jit);
-    CCallHelpers::Jump done = jit.jump();
-    notDouble.link(&jit);
-    emitSetNonNumber(jit);
-    done.link(&jit);
-    isInt32.link(&jit);
-}
-
-void ResultProfile::emitSetDouble(CCallHelpers& jit)
-{
-    jit.or32(CCallHelpers::TrustedImm32(ResultProfile::Int32Overflow | ResultProfile::Int52Overflow | ResultProfile::NegZeroDouble | ResultProfile::NonNegZeroDouble), CCallHelpers::AbsoluteAddress(addressOfFlags()));
-}
-
-void ResultProfile::emitSetNonNumber(CCallHelpers& jit)
-{
-    jit.or32(CCallHelpers::TrustedImm32(ResultProfile::NonNumber), CCallHelpers::AbsoluteAddress(addressOfFlags()));
-}
-#endif // ENABLE(JIT)
-
 } // namespace JSC
-
-namespace WTF {
-    
-using namespace JSC;
-
-void printInternal(PrintStream& out, const ResultProfile& profile)
-{
-    const char* separator = "";
-
-    if (!profile.didObserveNonInt32()) {
-        out.print("Int32");
-        separator = "|";
-    } else {
-        if (profile.didObserveNegZeroDouble()) {
-            out.print(separator, "NegZeroDouble");
-            separator = "|";
-        }
-        if (profile.didObserveNonNegZeroDouble()) {
-            out.print("NonNegZeroDouble");
-            separator = "|";
-        }
-        if (profile.didObserveNonNumber()) {
-            out.print("NonNumber");
-            separator = "|";
-        }
-        if (profile.didObserveInt32Overflow()) {
-            out.print("Int32Overflow");
-            separator = "|";
-        }
-        if (profile.didObserveInt52Overflow()) {
-            out.print("Int52Overflow");
-            separator = "|";
-        }
-    }
-    if (profile.specialFastPathCount()) {
-        out.print(" special fast path: ");
-        out.print(profile.specialFastPathCount());
-    }
-}
-
-} // namespace WTF
index fe759ca..0f4957b 100644 (file)
@@ -209,88 +209,6 @@ inline int getRareCaseProfileBytecodeOffset(RareCaseProfile* rareCaseProfile)
     return rareCaseProfile->m_bytecodeOffset;
 }
 
-struct ResultProfile {
-private:
-    static const int numberOfFlagBits = 5;
-
-public:
-    ResultProfile(int bytecodeOffset)
-        : m_bytecodeOffsetAndFlags(bytecodeOffset << numberOfFlagBits)
-    {
-        ASSERT(((bytecodeOffset << numberOfFlagBits) >> numberOfFlagBits) == bytecodeOffset);
-    }
-
-    enum ObservedResults {
-        NonNegZeroDouble = 1 << 0,
-        NegZeroDouble    = 1 << 1,
-        NonNumber        = 1 << 2,
-        Int32Overflow    = 1 << 3,
-        Int52Overflow    = 1 << 4,
-    };
-
-    int bytecodeOffset() const { return m_bytecodeOffsetAndFlags >> numberOfFlagBits; }
-    unsigned specialFastPathCount() const { return m_specialFastPathCount; }
-
-    bool didObserveNonInt32() const { return hasBits(NonNegZeroDouble | NegZeroDouble | NonNumber); }
-    bool didObserveDouble() const { return hasBits(NonNegZeroDouble | NegZeroDouble); }
-    bool didObserveNonNegZeroDouble() const { return hasBits(NonNegZeroDouble); }
-    bool didObserveNegZeroDouble() const { return hasBits(NegZeroDouble); }
-    bool didObserveNonNumber() const { return hasBits(NonNumber); }
-    bool didObserveInt32Overflow() const { return hasBits(Int32Overflow); }
-    bool didObserveInt52Overflow() const { return hasBits(Int52Overflow); }
-
-    void setObservedNonNegZeroDouble() { setBit(NonNegZeroDouble); }
-    void setObservedNegZeroDouble() { setBit(NegZeroDouble); }
-    void setObservedNonNumber() { setBit(NonNumber); }
-    void setObservedInt32Overflow() { setBit(Int32Overflow); }
-    void setObservedInt52Overflow() { setBit(Int52Overflow); }
-
-    void* addressOfFlags() { return &m_bytecodeOffsetAndFlags; }
-    void* addressOfSpecialFastPathCount() { return &m_specialFastPathCount; }
-    
-    void detectNumericness(JSValue value)
-    {
-        if (value.isInt32())
-            return;
-        if (value.isNumber()) {
-            m_bytecodeOffsetAndFlags |= Int32Overflow | Int52Overflow | NonNegZeroDouble | NegZeroDouble;
-            return;
-        }
-        m_bytecodeOffsetAndFlags |= NonNumber;
-    }
-
-#if ENABLE(JIT)    
-    // Sets (Int32Overflow | Int52Overflow | NonNegZeroDouble | NegZeroDouble) if it sees a
-    // double. Sets NonNumber if it sees a non-number.
-    void emitDetectNumericness(CCallHelpers&, JSValueRegs, TagRegistersMode = HaveTagRegisters);
-    
-    // Sets (Int32Overflow | Int52Overflow | NonNegZeroDouble | NegZeroDouble).
-    void emitSetDouble(CCallHelpers&);
-    
-    // Sets NonNumber.
-    void emitSetNonNumber(CCallHelpers&);
-#endif // ENABLE(JIT)
-
-private:
-    bool hasBits(int mask) const { return m_bytecodeOffsetAndFlags & mask; }
-    void setBit(int mask) { m_bytecodeOffsetAndFlags |= mask; }
-
-    int m_bytecodeOffsetAndFlags;
-    unsigned m_specialFastPathCount { 0 };
-};
-
-inline int getResultProfileBytecodeOffset(ResultProfile* profile)
-{
-    return profile->bytecodeOffset();
-}
-
 } // namespace JSC
 
-namespace WTF {
-
-void printInternal(PrintStream&, const JSC::ResultProfile&);
-
-} // namespace WTF
-
 #endif // ValueProfile_h
-
index 0fc8ceb..617f442 100644 (file)
@@ -31,6 +31,7 @@
 #include "config.h"
 #include "BytecodeGenerator.h"
 
+#include "ArithProfile.h"
 #include "BuiltinExecutables.h"
 #include "BytecodeLivenessAnalysis.h"
 #include "Interpreter.h"
@@ -1600,7 +1601,7 @@ RegisterID* BytecodeGenerator::emitBinaryOp(OpcodeID opcodeID, RegisterID* dst,
 
     if (opcodeID == op_bitor || opcodeID == op_bitand || opcodeID == op_bitxor ||
         opcodeID == op_add || opcodeID == op_mul || opcodeID == op_sub || opcodeID == op_div)
-        instructions().append(types.toInt());
+        instructions().append(ArithProfile(types.first(), types.second()).bits());
 
     return dst;
 }
index bcf3692..36d4759 100644 (file)
@@ -28,6 +28,7 @@
 
 #if ENABLE(DFG_JIT)
 
+#include "ArithProfile.h"
 #include "ArrayConstructor.h"
 #include "BasicBlockLocation.h"
 #include "CallLinkStatus.h"
@@ -914,29 +915,28 @@ private:
             return node;
 
         {
-            ConcurrentJITLocker locker(m_inlineStackTop->m_profiledBlock->m_lock);
-            ResultProfile* resultProfile = m_inlineStackTop->m_profiledBlock->resultProfileForBytecodeOffset(locker, m_currentIndex);
-            if (resultProfile) {
+            ArithProfile* arithProfile = m_inlineStackTop->m_profiledBlock->arithProfileForBytecodeOffset(m_currentIndex);
+            if (arithProfile) {
                 switch (node->op()) {
                 case ArithAdd:
                 case ArithSub:
                 case ValueAdd:
-                    if (resultProfile->didObserveDouble())
+                    if (arithProfile->didObserveDouble())
                         node->mergeFlags(NodeMayHaveDoubleResult);
-                    if (resultProfile->didObserveNonNumber())
+                    if (arithProfile->didObserveNonNumber())
                         node->mergeFlags(NodeMayHaveNonNumberResult);
                     break;
                 
                 case ArithMul: {
-                    if (resultProfile->didObserveInt52Overflow())
+                    if (arithProfile->didObserveInt52Overflow())
                         node->mergeFlags(NodeMayOverflowInt52);
-                    if (resultProfile->didObserveInt32Overflow() || m_inlineStackTop->m_exitProfile.hasExitSite(m_currentIndex, Overflow))
+                    if (arithProfile->didObserveInt32Overflow() || m_inlineStackTop->m_exitProfile.hasExitSite(m_currentIndex, Overflow))
                         node->mergeFlags(NodeMayOverflowInt32InBaseline);
-                    if (resultProfile->didObserveNegZeroDouble() || m_inlineStackTop->m_exitProfile.hasExitSite(m_currentIndex, NegativeZero))
+                    if (arithProfile->didObserveNegZeroDouble() || m_inlineStackTop->m_exitProfile.hasExitSite(m_currentIndex, NegativeZero))
                         node->mergeFlags(NodeMayNegZeroInBaseline);
-                    if (resultProfile->didObserveDouble())
+                    if (arithProfile->didObserveDouble())
                         node->mergeFlags(NodeMayHaveDoubleResult);
-                    if (resultProfile->didObserveNonNumber())
+                    if (arithProfile->didObserveNonNumber())
                         node->mergeFlags(NodeMayHaveNonNumberResult);
                     break;
                 }
index 74557ab..a8b2f9c 100644 (file)
@@ -1548,9 +1548,8 @@ MethodOfGettingAValueProfile Graph::methodOfGettingAValueProfileFor(Node* node)
             return profiledBlock->valueProfileForBytecodeOffset(node->origin.semantic.bytecodeIndex);
         
         {
-            ConcurrentJITLocker locker(profiledBlock->m_lock);
             if (profiledBlock->hasBaselineJITProfiling()) {
-                if (ResultProfile* result = profiledBlock->resultProfileForBytecodeOffset(locker, node->origin.semantic.bytecodeIndex))
+                if (ArithProfile* result = profiledBlock->arithProfileForBytecodeOffset(node->origin.semantic.bytecodeIndex))
                     return result;
             }
         }
index d421387..81c0698 100644 (file)
@@ -604,7 +604,7 @@ void JITCompiler::exceptionCheck()
     HandlerInfo* exceptionHandler;
     bool willCatchException = m_graph.willCatchExceptionInMachineFrame(m_speculative->m_currentNode->origin.forExit, opCatchOrigin, exceptionHandler); 
     if (willCatchException) {
-        unsigned streamIndex = m_speculative->m_outOfLineStreamIndex != UINT_MAX ? m_speculative->m_outOfLineStreamIndex : m_speculative->m_stream->size();
+        unsigned streamIndex = m_speculative->m_outOfLineStreamIndex ? *m_speculative->m_outOfLineStreamIndex : m_speculative->m_stream->size();
         MacroAssembler::Jump hadException = emitNonPatchableExceptionCheck();
         // We assume here that this is called after callOpeartion()/appendCall() is called.
         appendExceptionHandlingOSRExit(ExceptionCheck, streamIndex, opCatchOrigin, exceptionHandler, m_jitCode->common.lastCallSite(), hadException);
index 337c3c9..c0fb200 100644 (file)
@@ -52,7 +52,7 @@ public:
         jit->m_outOfLineStreamIndex = m_streamIndex;
         jit->m_origin = m_origin;
         generateInternal(jit);
-        jit->m_outOfLineStreamIndex = UINT_MAX;
+        jit->m_outOfLineStreamIndex = Nullopt;
         if (!ASSERT_DISABLED)
             jit->m_jit.abortWithReason(DFGSlowPathGeneratorFellThrough);
     }
index 5af11f4..164f13f 100644 (file)
@@ -44,6 +44,7 @@
 #include "JITBitXorGenerator.h"
 #include "JITDivGenerator.h"
 #include "JITLeftShiftGenerator.h"
+#include "JITMathIC.h"
 #include "JITMulGenerator.h"
 #include "JITRightShiftGenerator.h"
 #include "JITSubGenerator.h"
@@ -56,6 +57,7 @@
 #include "ScopedArguments.h"
 #include "ScratchRegisterAllocator.h"
 #include "WriteBarrierBuffer.h"
+#include <wtf/Box.h>
 #include <wtf/MathExtras.h>
 
 namespace JSC { namespace DFG {
@@ -362,7 +364,7 @@ void SpeculativeJIT::addSlowPathGenerator(std::unique_ptr<SlowPathGenerator> slo
 
 void SpeculativeJIT::addSlowPathGenerator(std::function<void()> lambda)
 {
-    m_slowPathLambdas.append(std::make_pair(lambda, m_currentNode));
+    m_slowPathLambdas.append(SlowPathLambda{ lambda, m_currentNode, static_cast<unsigned>(m_stream->size()) });
 }
 
 void SpeculativeJIT::runSlowPathGenerators(PCToCodeOriginMapBuilder& pcToCodeOriginMapBuilder)
@@ -371,11 +373,13 @@ void SpeculativeJIT::runSlowPathGenerators(PCToCodeOriginMapBuilder& pcToCodeOri
         pcToCodeOriginMapBuilder.appendItem(m_jit.label(), slowPathGenerator->origin().semantic);
         slowPathGenerator->generate(this);
     }
-    for (auto& generatorPair : m_slowPathLambdas) {
-        Node* currentNode = generatorPair.second;
+    for (auto& slowPathLambda : m_slowPathLambdas) {
+        Node* currentNode = slowPathLambda.currentNode;
         m_currentNode = currentNode;
+        m_outOfLineStreamIndex = slowPathLambda.streamIndex;
         pcToCodeOriginMapBuilder.appendItem(m_jit.label(), currentNode->origin.semantic);
-        generatorPair.first();
+        slowPathLambda.generator();
+        m_outOfLineStreamIndex = Nullopt;
     }
 }
 
@@ -3379,31 +3383,64 @@ void SpeculativeJIT::compileValueAdd(Node* node)
         rightRegs = right->jsValueRegs();
     }
 
-    JITAddGenerator gen(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs,
-        leftFPR, rightFPR, scratchGPR, scratchFPR);
-    gen.generateFastPath(m_jit);
+    JITAddIC* addIC = m_jit.codeBlock()->addJITAddIC();
+    Box<MathICGenerationState> addICGenerationState = Box<MathICGenerationState>::create();
+    ArithProfile* arithProfile = m_jit.graph().baselineCodeBlockFor(node->origin.semantic)->arithProfileForBytecodeOffset(node->origin.semantic.bytecodeIndex);
+    addIC->m_generator = JITAddGenerator(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs, leftFPR, rightFPR, scratchGPR, scratchFPR, arithProfile);
 
-    ASSERT(gen.didEmitFastPath());
-    gen.endJumpList().append(m_jit.jump());
+    bool generatedInline = addIC->generateInline(m_jit, *addICGenerationState);
 
-    gen.slowPathJumpList().link(&m_jit);
+    if (generatedInline) {
+        ASSERT(!addICGenerationState->slowPathJumps.empty());
 
-    silentSpillAllRegisters(resultRegs);
+        Vector<SilentRegisterSavePlan> savePlans;
+        silentSpillAllRegistersImpl(false, savePlans, resultRegs);
 
-    if (leftOperand.isConst()) {
-        leftRegs = resultRegs;
-        m_jit.moveValue(leftChild->asJSValue(), leftRegs);
-    } else if (rightOperand.isConst()) {
-        rightRegs = resultRegs;
-        m_jit.moveValue(rightChild->asJSValue(), rightRegs);
-    }
+        auto done = m_jit.label();
 
-    callOperation(operationValueAdd, resultRegs, leftRegs, rightRegs);
+        addSlowPathGenerator([=, savePlans = WTFMove(savePlans)] () {
+            addICGenerationState->slowPathJumps.link(&m_jit);
+            addICGenerationState->slowPathStart = m_jit.label();
 
-    silentFillAllRegisters(resultRegs);
-    m_jit.exceptionCheck();
+            silentSpill(savePlans);
+
+            auto innerLeftRegs = leftRegs;
+            auto innerRightRegs = rightRegs;
+            if (leftOperand.isConst()) {
+                innerLeftRegs = resultRegs;
+                m_jit.moveValue(leftChild->asJSValue(), innerLeftRegs);
+            } else if (rightOperand.isConst()) {
+                innerRightRegs = resultRegs;
+                m_jit.moveValue(rightChild->asJSValue(), innerRightRegs);
+            }
+
+            if (addICGenerationState->shouldSlowPathRepatch)
+                addICGenerationState->slowPathCall = callOperation(operationValueAddOptimize, resultRegs, innerLeftRegs, innerRightRegs, addIC);
+            else
+                addICGenerationState->slowPathCall = callOperation(operationValueAdd, resultRegs, innerLeftRegs, innerRightRegs);
+
+            silentFill(savePlans);
+            m_jit.exceptionCheck();
+            m_jit.jump().linkTo(done, &m_jit);
+
+            m_jit.addLinkTask([=] (LinkBuffer& linkBuffer) {
+                addIC->finalizeInlineCode(*addICGenerationState, linkBuffer);
+            });
+        });
+    } else {
+        if (leftOperand.isConst()) {
+            left = JSValueOperand(this, leftChild);
+            leftRegs = left->jsValueRegs();
+        } else if (rightOperand.isConst()) {
+            right = JSValueOperand(this, rightChild);
+            rightRegs = right->jsValueRegs();
+        }
+
+        flushRegisters();
+        callOperation(operationValueAdd, resultRegs, leftRegs, rightRegs);
+        m_jit.exceptionCheck();
+    }
 
-    gen.endJumpList().link(&m_jit);
     jsValueResult(resultRegs, node);
     return;
 }
index 47e1228..7f1904b 100644 (file)
@@ -388,13 +388,15 @@ public:
     {
         silentSpillAllRegistersImpl(doSpill, plans, InvalidGPRReg, InvalidGPRReg, exclude);
     }
-#if USE(JSVALUE32_64)
     template<typename CollectionType>
     void silentSpillAllRegistersImpl(bool doSpill, CollectionType& plans, JSValueRegs exclude)
     {
+#if USE(JSVALUE32_64)
         silentSpillAllRegistersImpl(doSpill, plans, exclude.tagGPR(), exclude.payloadGPR());
-    }
+#else
+        silentSpillAllRegistersImpl(doSpill, plans, exclude.gpr());
 #endif
+    }
     
     void silentSpillAllRegisters(GPRReg exclude, GPRReg exclude2 = InvalidGPRReg, FPRReg fprExclude = InvalidFPRReg)
     {
@@ -1285,6 +1287,13 @@ public:
 
 
 #if USE(JSVALUE64)
+
+    JITCompiler::Call callOperation(J_JITOperation_EJJJaic operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, JITAddIC* addIC)
+    {
+        m_jit.setupArgumentsWithExecState(arg1.gpr(), arg2.gpr(), TrustedImmPtr(addIC));
+        return appendCallSetResult(operation, result.gpr());
+    }
+
     JITCompiler::Call callOperation(J_JITOperation_EJJI operation, GPRReg result, GPRReg arg1, GPRReg arg2, UniquedStringImpl* uid)
     {
         m_jit.setupArgumentsWithExecState(arg1, arg2, TrustedImmPtr(uid));
@@ -1702,6 +1711,12 @@ public:
     }
 #else // USE(JSVALUE32_64)
 
+    JITCompiler::Call callOperation(J_JITOperation_EJJJaic operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, JITAddIC* addIC)
+    {
+        m_jit.setupArgumentsWithExecState(EABI_32BIT_DUMMY_ARG arg1.payloadGPR(), arg1.tagGPR(), arg2.payloadGPR(), arg2.tagGPR(), TrustedImmPtr(addIC));
+        return appendCallSetResult(operation, result.payloadGPR(), result.tagGPR());
+    }
+
     JITCompiler::Call callOperation(J_JITOperation_EJJI operation, GPRReg resultTag, GPRReg resultPayload, GPRReg arg1Tag, GPRReg arg1Payload, GPRReg arg2Tag, GPRReg arg2Payload, UniquedStringImpl* uid)
     {
         m_jit.setupArgumentsWithExecState(EABI_32BIT_DUMMY_ARG arg1Payload, arg1Tag, arg2Payload, arg2Tag, TrustedImmPtr(uid));
@@ -2832,9 +2847,14 @@ public:
     MinifiedGraph* m_minifiedGraph;
     
     Vector<std::unique_ptr<SlowPathGenerator>, 8> m_slowPathGenerators;
-    Vector<std::pair<std::function<void()>, Node*>, 8> m_slowPathLambdas;
+    struct SlowPathLambda {
+        std::function<void()> generator;
+        Node* currentNode;
+        unsigned streamIndex;
+    };
+    Vector<SlowPathLambda> m_slowPathLambdas;
     Vector<SilentRegisterSavePlan> m_plans;
-    unsigned m_outOfLineStreamIndex { UINT_MAX };
+    Optional<unsigned> m_outOfLineStreamIndex;
 };
 
 
index 1151f96..9b7cd9e 100644 (file)
@@ -62,6 +62,7 @@
 #include "JITDivGenerator.h"
 #include "JITInlineCacheGenerator.h"
 #include "JITLeftShiftGenerator.h"
+#include "JITMathIC.h"
 #include "JITMulGenerator.h"
 #include "JITRightShiftGenerator.h"
 #include "JITSubGenerator.h"
@@ -1538,7 +1539,72 @@ private:
     
     void compileValueAdd()
     {
-        emitBinarySnippet<JITAddGenerator>(operationValueAdd);
+        Node* node = m_node;
+        
+        LValue left = lowJSValue(node->child1());
+        LValue right = lowJSValue(node->child2());
+
+        SnippetOperand leftOperand(m_state.forNode(node->child1()).resultType());
+        SnippetOperand rightOperand(m_state.forNode(node->child2()).resultType());
+            
+        PatchpointValue* patchpoint = m_out.patchpoint(Int64);
+        patchpoint->appendSomeRegister(left);
+        patchpoint->appendSomeRegister(right);
+        patchpoint->append(m_tagMask, ValueRep::lateReg(GPRInfo::tagMaskRegister));
+        patchpoint->append(m_tagTypeNumber, ValueRep::lateReg(GPRInfo::tagTypeNumberRegister));
+        RefPtr<PatchpointExceptionHandle> exceptionHandle =
+            preparePatchpointForExceptions(patchpoint);
+        patchpoint->numGPScratchRegisters = 1;
+        patchpoint->numFPScratchRegisters = 2;
+        patchpoint->clobber(RegisterSet::macroScratchRegisters());
+        State* state = &m_ftlState;
+        patchpoint->setGenerator(
+            [=] (CCallHelpers& jit, const StackmapGenerationParams& params) {
+                AllowMacroScratchRegisterUsage allowScratch(jit);
+
+                Box<CCallHelpers::JumpList> exceptions =
+                    exceptionHandle->scheduleExitCreation(params)->jumps(jit);
+
+                JITAddIC* addIC = jit.codeBlock()->addJITAddIC();
+                Box<MathICGenerationState> addICGenerationState = Box<MathICGenerationState>::create();
+                ArithProfile* arithProfile = state->graph.baselineCodeBlockFor(node->origin.semantic)->arithProfileForBytecodeOffset(node->origin.semantic.bytecodeIndex);
+                addIC->m_generator = JITAddGenerator(leftOperand, rightOperand, JSValueRegs(params[0].gpr()),
+                    JSValueRegs(params[1].gpr()), JSValueRegs(params[2].gpr()), params.fpScratch(0),
+                    params.fpScratch(1), params.gpScratch(0), InvalidFPRReg, arithProfile);
+
+                bool generatedInline = addIC->generateInline(jit, *addICGenerationState);
+
+                if (generatedInline) {
+                    ASSERT(!addICGenerationState->slowPathJumps.empty());
+                    auto done = jit.label();
+                    params.addLatePath([=] (CCallHelpers& jit) {
+                        AllowMacroScratchRegisterUsage allowScratch(jit);
+                        addICGenerationState->slowPathJumps.link(&jit);
+                        addICGenerationState->slowPathStart = jit.label();
+
+                        if (addICGenerationState->shouldSlowPathRepatch) {
+                            SlowPathCall call = callOperation(*state, params.unavailableRegisters(), jit, node->origin.semantic, exceptions.get(),
+                                operationValueAddOptimize, params[0].gpr(), params[1].gpr(), params[2].gpr(), CCallHelpers::TrustedImmPtr(addIC));
+                            addICGenerationState->slowPathCall = call.call();
+                        } else {
+                            SlowPathCall call = callOperation(*state, params.unavailableRegisters(), jit, node->origin.semantic,
+                                exceptions.get(), operationValueAdd, params[0].gpr(), params[1].gpr(), params[2].gpr());
+                            addICGenerationState->slowPathCall = call.call();
+                        }
+                        jit.jump().linkTo(done, &jit);
+
+                        jit.addLinkTask([=] (LinkBuffer& linkBuffer) {
+                            addIC->finalizeInlineCode(*addICGenerationState, linkBuffer);
+                        });
+                    });
+                } else {
+                    callOperation(
+                        *state, params.unavailableRegisters(), jit, node->origin.semantic, exceptions.get(),
+                        operationValueAdd, params[0].gpr(), params[1].gpr(), params[2].gpr());
+                }
+            });
+
+        setJSValue(patchpoint);
     }
     
     void compileStrCat()
index 677c135..30537fd 100644 (file)
@@ -826,6 +826,18 @@ public:
         addCallArgument(arg6);
     }
 
+    ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, GPRReg arg2, GPRReg arg3, GPRReg arg4, TrustedImmPtr arg5, TrustedImmPtr arg6)
+    {
+        resetCallArguments();
+        addCallArgument(GPRInfo::callFrameRegister);
+        addCallArgument(arg1);
+        addCallArgument(arg2);
+        addCallArgument(arg3);
+        addCallArgument(arg4);
+        addCallArgument(arg5);
+        addCallArgument(arg6);
+    }
+
     ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, GPRReg arg2, GPRReg arg3, GPRReg arg4, GPRReg arg5, GPRReg arg6, TrustedImmPtr arg7)
     {
         resetCallArguments();
@@ -1594,6 +1606,14 @@ public:
         setupArgumentsWithExecState(arg1, arg2, arg3);
     }
 
+    ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, GPRReg arg2, GPRReg arg3, GPRReg arg4, TrustedImmPtr arg5, TrustedImmPtr arg6)
+    {
+        poke(arg6, POKE_ARGUMENT_OFFSET + 2);
+        poke(arg5, POKE_ARGUMENT_OFFSET + 1);
+        poke(arg4, POKE_ARGUMENT_OFFSET);
+        setupArgumentsWithExecState(arg1, arg2, arg3);
+    }
+
     ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, GPRReg arg2, GPRReg arg3, GPRReg arg4, TrustedImmPtr arg5)
     {
         poke(arg5, POKE_ARGUMENT_OFFSET + 1);
@@ -2099,6 +2119,14 @@ public:
         move(GPRInfo::callFrameRegister, GPRInfo::argumentGPR0);
     }
 
+    ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, GPRReg arg2, TrustedImmPtr arg3, TrustedImmPtr arg4)
+    {
+        setupTwoStubArgsGPR<GPRInfo::argumentGPR1, GPRInfo::argumentGPR2>(arg1, arg2);
+        move(arg3, GPRInfo::argumentGPR3);
+        move(arg4, GPRInfo::argumentGPR4);
+        move(GPRInfo::callFrameRegister, GPRInfo::argumentGPR0);
+    }
+
     ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, TrustedImmPtr arg2, TrustedImm32 arg3, GPRReg arg4, GPRReg arg5)
     {
         setupThreeStubArgsGPR<GPRInfo::argumentGPR1, GPRInfo::argumentGPR4, GPRInfo::argumentGPR5>(arg1, arg4, arg5);
@@ -2207,6 +2235,15 @@ public:
     }
 #endif
     
+    void setupArgumentsWithExecState(JSValueRegs arg1, JSValueRegs arg2)
+    {
+#if USE(JSVALUE64)
+        setupArgumentsWithExecState(arg1.gpr(), arg2.gpr());
+#else
+        setupArgumentsWithExecState(EABI_32BIT_DUMMY_ARG arg1.payloadGPR(), arg1.tagGPR(), arg2.payloadGPR(), arg2.tagGPR());
+#endif
+    }
+
     void setupArgumentsWithExecState(JSValueRegs arg1, JSValueRegs arg2, TrustedImmPtr arg3)
     {
 #if USE(JSVALUE64)
@@ -2215,6 +2252,15 @@ public:
         setupArgumentsWithExecState(EABI_32BIT_DUMMY_ARG arg1.payloadGPR(), arg1.tagGPR(), arg2.payloadGPR(), arg2.tagGPR(), arg3);
 #endif
     }
+
+    void setupArgumentsWithExecState(JSValueRegs arg1, JSValueRegs arg2, TrustedImmPtr arg3, TrustedImmPtr arg4)
+    {
+#if USE(JSVALUE64)
+        setupArgumentsWithExecState(arg1.gpr(), arg2.gpr(), arg3, arg4);
+#else
+        setupArgumentsWithExecState(EABI_32BIT_DUMMY_ARG arg1.payloadGPR(), arg1.tagGPR(), arg2.payloadGPR(), arg2.tagGPR(), arg3, arg4);
+#endif
+    }
     
     void setupArguments(JSValueRegs arg1)
     {
index 093d7da..c94b740 100644 (file)
 #include "Interpreter.h"
 #include "JITDisassembler.h"
 #include "JITInlineCacheGenerator.h"
+#include "JITMathIC.h"
 #include "JSInterfaceJIT.h"
 #include "Opcode.h"
 #include "PCToCodeOriginMap.h"
-#include "ResultType.h"
 #include "UnusedPointer.h"
 
 namespace JSC {
@@ -435,7 +435,6 @@ namespace JSC {
         void compileGetByIdHotPath(const Identifier*);
 
         // Arithmetic opcode helpers
-        void emitSub32Constant(int dst, int op, int32_t constant, ResultType opType);
         void emitBinaryDoubleOp(OpcodeID, int dst, int op1, int op2, OperandTypes, JumpList& notInt32Op1, JumpList& notInt32Op2, bool op1IsInRegisters = true, bool op2IsInRegisters = true);
 
 #else // USE(JSVALUE32_64)
@@ -764,7 +763,10 @@ namespace JSC {
 #endif
         MacroAssembler::Call callOperation(J_JITOperation_EJI, int, GPRReg, UniquedStringImpl*);
         MacroAssembler::Call callOperation(J_JITOperation_EJJ, int, GPRReg, GPRReg);
-        MacroAssembler::Call callOperation(J_JITOperation_EJJRp, JSValueRegs, JSValueRegs, JSValueRegs, ResultProfile*);
+        MacroAssembler::Call callOperation(J_JITOperation_EJJArp, JSValueRegs, JSValueRegs, JSValueRegs, ArithProfile*);
+        MacroAssembler::Call callOperation(J_JITOperation_EJJ, JSValueRegs, JSValueRegs, JSValueRegs);
+        MacroAssembler::Call callOperation(J_JITOperation_EJJArpJaic, JSValueRegs, JSValueRegs, JSValueRegs, ArithProfile*, JITAddIC*);
+        MacroAssembler::Call callOperation(J_JITOperation_EJJJaic, JSValueRegs, JSValueRegs, JSValueRegs, JITAddIC*);
         MacroAssembler::Call callOperation(J_JITOperation_EJJAp, int, GPRReg, GPRReg, ArrayProfile*);
         MacroAssembler::Call callOperation(J_JITOperation_EJJBy, int, GPRReg, GPRReg, ByValInfo*);
         MacroAssembler::Call callOperation(Z_JITOperation_EJOJ, GPRReg, GPRReg, GPRReg);
@@ -954,6 +956,9 @@ namespace JSC {
 
         PCToCodeOriginMapBuilder m_pcToCodeOriginMapBuilder;
 
+        HashMap<Instruction*, JITAddIC*> m_instructionToJITAddIC;
+        HashMap<Instruction*, MathICGenerationState> m_instructionToJITAddICGenerationState;
+
         bool m_canBeOptimized;
         bool m_canBeOptimizedOrInlined;
         bool m_shouldEmitProfiling;
index f291243..a9c2603 100644 (file)
 #include "config.h"
 #include "JITAddGenerator.h"
 
+#include "ArithProfile.h"
+#include "JITMathIC.h"
+#include "LinkBuffer.h"
+
 #if ENABLE(JIT)
 
 namespace JSC {
 
-void JITAddGenerator::generateFastPath(CCallHelpers& jit)
+JITMathICInlineResult JITAddGenerator::generateInline(CCallHelpers& jit, MathICGenerationState& state)
+{
+    // We default to speculating int32.
+    ObservedType lhs = ObservedType().withInt32();
+    ObservedType rhs = ObservedType().withInt32();
+    if (m_arithProfile) {
+        lhs = m_arithProfile->lhsObservedType();
+        rhs = m_arithProfile->rhsObservedType();
+        if (lhs.isEmpty() || rhs.isEmpty()) {
+            lhs = ObservedType().withInt32();
+            rhs = ObservedType().withInt32();
+        }
+    }
+
+    if (lhs.isOnlyNonNumber() && rhs.isOnlyNonNumber())
+        return JITMathICInlineResult::DontGenerate;
+
+    if ((lhs.isOnlyInt32() || m_leftOperand.isConstInt32()) && (rhs.isOnlyInt32() || m_rightOperand.isConstInt32())) {
+        ASSERT(!m_leftOperand.isConstInt32() || !m_rightOperand.isConstInt32());
+        if (!m_leftOperand.isConstInt32())
+            state.slowPathJumps.append(jit.branchIfNotInt32(m_left));
+        if (!m_rightOperand.isConstInt32())
+            state.slowPathJumps.append(jit.branchIfNotInt32(m_right));
+
+        if (m_leftOperand.isConstInt32() || m_rightOperand.isConstInt32()) {
+            JSValueRegs var = m_leftOperand.isConstInt32() ? m_right : m_left;
+            int32_t constValue = m_leftOperand.isConstInt32() ? m_leftOperand.asConstInt32() : m_rightOperand.asConstInt32();
+            state.slowPathJumps.append(jit.branchAdd32(CCallHelpers::Overflow, var.payloadGPR(), CCallHelpers::Imm32(constValue), m_scratchGPR));
+        } else
+            state.slowPathJumps.append(jit.branchAdd32(CCallHelpers::Overflow, m_right.payloadGPR(), m_left.payloadGPR(), m_scratchGPR));
+        jit.boxInt32(m_scratchGPR, m_result);
+        return JITMathICInlineResult::GeneratedFastPath;
+    }
+
+    return JITMathICInlineResult::GenerateFullSnippet;
+}
+
+bool JITAddGenerator::generateFastPath(CCallHelpers& jit, CCallHelpers::JumpList& endJumpList, CCallHelpers::JumpList& slowPathJumpList)
 {
     ASSERT(m_scratchGPR != InvalidGPRReg);
     ASSERT(m_scratchGPR != m_left.payloadGPR());
@@ -43,12 +84,8 @@ void JITAddGenerator::generateFastPath(CCallHelpers& jit)
 
     ASSERT(!m_leftOperand.isConstInt32() || !m_rightOperand.isConstInt32());
     
-    if (!m_leftOperand.mightBeNumber() || !m_rightOperand.mightBeNumber()) {
-        ASSERT(!m_didEmitFastPath);
-        return;
-    }
-
-    m_didEmitFastPath = true;
+    if (!m_leftOperand.mightBeNumber() || !m_rightOperand.mightBeNumber())
+        return false;
 
     if (m_leftOperand.isConstInt32() || m_rightOperand.isConstInt32()) {
         JSValueRegs var = m_leftOperand.isConstInt32() ? m_right : m_left;
@@ -58,20 +95,20 @@ void JITAddGenerator::generateFastPath(CCallHelpers& jit)
         // Try to do intVar + intConstant.
         CCallHelpers::Jump notInt32 = jit.branchIfNotInt32(var);
 
-        m_slowPathJumpList.append(jit.branchAdd32(CCallHelpers::Overflow, var.payloadGPR(), CCallHelpers::Imm32(constOpr.asConstInt32()), m_scratchGPR));
+        slowPathJumpList.append(jit.branchAdd32(CCallHelpers::Overflow, var.payloadGPR(), CCallHelpers::Imm32(constOpr.asConstInt32()), m_scratchGPR));
 
         jit.boxInt32(m_scratchGPR, m_result);
-        m_endJumpList.append(jit.jump());
+        endJumpList.append(jit.jump());
 
         if (!jit.supportsFloatingPoint()) {
-            m_slowPathJumpList.append(notInt32);
-            return;
+            slowPathJumpList.append(notInt32);
+            return true;
         }
 
         // Try to do doubleVar + double(intConstant).
         notInt32.link(&jit);
         if (!varOpr.definitelyIsNumber())
-            m_slowPathJumpList.append(jit.branchIfNotNumber(var, m_scratchGPR));
+            slowPathJumpList.append(jit.branchIfNotNumber(var, m_scratchGPR));
 
         jit.unboxDoubleNonDestructive(var, m_leftFPR, m_scratchGPR, m_scratchFPR);
 
@@ -89,22 +126,23 @@ void JITAddGenerator::generateFastPath(CCallHelpers& jit)
         leftNotInt = jit.branchIfNotInt32(m_left);
         rightNotInt = jit.branchIfNotInt32(m_right);
 
-        m_slowPathJumpList.append(jit.branchAdd32(CCallHelpers::Overflow, m_right.payloadGPR(), m_left.payloadGPR(), m_scratchGPR));
+        slowPathJumpList.append(jit.branchAdd32(CCallHelpers::Overflow, m_right.payloadGPR(), m_left.payloadGPR(), m_scratchGPR));
 
         jit.boxInt32(m_scratchGPR, m_result);
-        m_endJumpList.append(jit.jump());
+        endJumpList.append(jit.jump());
+
 
         if (!jit.supportsFloatingPoint()) {
-            m_slowPathJumpList.append(leftNotInt);
-            m_slowPathJumpList.append(rightNotInt);
-            return;
+            slowPathJumpList.append(leftNotInt);
+            slowPathJumpList.append(rightNotInt);
+            return true;
         }
 
         leftNotInt.link(&jit);
         if (!m_leftOperand.definitelyIsNumber())
-            m_slowPathJumpList.append(jit.branchIfNotNumber(m_left, m_scratchGPR));
+            slowPathJumpList.append(jit.branchIfNotNumber(m_left, m_scratchGPR));
         if (!m_rightOperand.definitelyIsNumber())
-            m_slowPathJumpList.append(jit.branchIfNotNumber(m_right, m_scratchGPR));
+            slowPathJumpList.append(jit.branchIfNotNumber(m_right, m_scratchGPR));
 
         jit.unboxDoubleNonDestructive(m_left, m_leftFPR, m_scratchGPR, m_scratchFPR);
         CCallHelpers::Jump rightIsDouble = jit.branchIfNotInt32(m_right);
@@ -114,7 +152,7 @@ void JITAddGenerator::generateFastPath(CCallHelpers& jit)
 
         rightNotInt.link(&jit);
         if (!m_rightOperand.definitelyIsNumber())
-            m_slowPathJumpList.append(jit.branchIfNotNumber(m_right, m_scratchGPR));
+            slowPathJumpList.append(jit.branchIfNotNumber(m_right, m_scratchGPR));
 
         jit.convertInt32ToDouble(m_left.payloadGPR(), m_leftFPR);
 
@@ -128,10 +166,12 @@ void JITAddGenerator::generateFastPath(CCallHelpers& jit)
 
     // Do doubleVar + doubleVar.
     jit.addDouble(m_rightFPR, m_leftFPR);
-    if (m_resultProfile)
-        m_resultProfile->emitSetDouble(jit);
+    if (m_arithProfile)
+        m_arithProfile->emitSetDouble(jit);
         
     jit.boxDouble(m_leftFPR, m_result);
+
+    return true;
 }
 
 } // namespace JSC
index 8bf97ce..b8917d8 100644 (file)
 #if ENABLE(JIT)
 
 #include "CCallHelpers.h"
+#include "JITMathICInlineResult.h"
+#include "JITOperations.h"
 #include "SnippetOperand.h"
 
 namespace JSC {
 
+struct MathICGenerationState;
+
 class JITAddGenerator {
 public:
+    JITAddGenerator()
+    { }
+
     JITAddGenerator(SnippetOperand leftOperand, SnippetOperand rightOperand,
         JSValueRegs result, JSValueRegs left, JSValueRegs right,
         FPRReg leftFPR, FPRReg rightFPR, GPRReg scratchGPR, FPRReg scratchFPR,
-        ResultProfile* resultProfile = nullptr)
+        ArithProfile* arithProfile = nullptr)
         : m_leftOperand(leftOperand)
         , m_rightOperand(rightOperand)
         , m_result(result)
@@ -48,16 +55,13 @@ public:
         , m_rightFPR(rightFPR)
         , m_scratchGPR(scratchGPR)
         , m_scratchFPR(scratchFPR)
-        , m_resultProfile(resultProfile)
+        , m_arithProfile(arithProfile)
     {
         ASSERT(!m_leftOperand.isConstInt32() || !m_rightOperand.isConstInt32());
     }
 
-    void generateFastPath(CCallHelpers&);
-
-    bool didEmitFastPath() const { return m_didEmitFastPath; }
-    CCallHelpers::JumpList& endJumpList() { return m_endJumpList; }
-    CCallHelpers::JumpList& slowPathJumpList() { return m_slowPathJumpList; }
+    JITMathICInlineResult generateInline(CCallHelpers&, MathICGenerationState&);
+    bool generateFastPath(CCallHelpers&, CCallHelpers::JumpList& endJumpList, CCallHelpers::JumpList& slowPathJumpList);
 
 private:
     SnippetOperand m_leftOperand;
@@ -69,11 +73,7 @@ private:
     FPRReg m_rightFPR;
     GPRReg m_scratchGPR;
     FPRReg m_scratchFPR;
-    ResultProfile* m_resultProfile;
-    bool m_didEmitFastPath { false };
-
-    CCallHelpers::JumpList m_endJumpList;
-    CCallHelpers::JumpList m_slowPathJumpList;
+    ArithProfile* m_arithProfile;
 };
 
 } // namespace JSC
index 4855f1d..1e4ad80 100644 (file)
@@ -28,6 +28,7 @@
 #if ENABLE(JIT)
 #include "JIT.h"
 
+#include "ArithProfile.h"
 #include "CodeBlock.h"
 #include "JITAddGenerator.h"
 #include "JITBitAndGenerator.h"
@@ -36,6 +37,7 @@
 #include "JITDivGenerator.h"
 #include "JITInlines.h"
 #include "JITLeftShiftGenerator.h"
+#include "JITMathIC.h"
 #include "JITMulGenerator.h"
 #include "JITNegGenerator.h"
 #include "JITOperations.h"
 #include "JSFunction.h"
 #include "Interpreter.h"
 #include "JSCInlines.h"
+#include "LinkBuffer.h"
 #include "ResultType.h"
 #include "SlowPathCall.h"
 
-
 namespace JSC {
 
 void JIT::emit_op_jless(Instruction* currentInstruction)
@@ -675,20 +677,26 @@ void JIT::emitSlow_op_urshift(Instruction* currentInstruction, Vector<SlowCaseEn
     slowPathCall.call();
 }
 
+ALWAYS_INLINE static OperandTypes getOperandTypes(Instruction* instruction)
+{
+    return OperandTypes(ArithProfile::fromInt(instruction[4].u.operand).lhsResultType(), ArithProfile::fromInt(instruction[4].u.operand).rhsResultType());
+}
+
 void JIT::emit_op_add(Instruction* currentInstruction)
 {
     int result = currentInstruction[1].u.operand;
     int op1 = currentInstruction[2].u.operand;
     int op2 = currentInstruction[3].u.operand;
-    OperandTypes types = OperandTypes::fromInt(currentInstruction[4].u.operand);
 
 #if USE(JSVALUE64)
+    OperandTypes types = getOperandTypes(copiedInstruction(currentInstruction));
     JSValueRegs leftRegs = JSValueRegs(regT0);
     JSValueRegs rightRegs = JSValueRegs(regT1);
     JSValueRegs resultRegs = leftRegs;
     GPRReg scratchGPR = regT2;
     FPRReg scratchFPR = InvalidFPRReg;
 #else
+    OperandTypes types = getOperandTypes(currentInstruction);
     JSValueRegs leftRegs = JSValueRegs(regT1, regT0);
     JSValueRegs rightRegs = JSValueRegs(regT3, regT2);
     JSValueRegs resultRegs = leftRegs;
@@ -696,9 +704,9 @@ void JIT::emit_op_add(Instruction* currentInstruction)
     FPRReg scratchFPR = fpRegT2;
 #endif
 
-    ResultProfile* resultProfile = nullptr;
+    ArithProfile* arithProfile = nullptr;
     if (shouldEmitProfiling())
-        resultProfile = m_codeBlock->ensureResultProfile(m_bytecodeOffset);
+        arithProfile = &m_codeBlock->arithProfileForPC(currentInstruction);
 
     SnippetOperand leftOperand(types.first());
     SnippetOperand rightOperand(types.second());
@@ -715,47 +723,47 @@ void JIT::emit_op_add(Instruction* currentInstruction)
     if (!rightOperand.isConst())
         emitGetVirtualRegister(op2, rightRegs);
 
-    JITAddGenerator gen(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs,
-        fpRegT0, fpRegT1, scratchGPR, scratchFPR, resultProfile);
+    JITAddIC* addIC = m_codeBlock->addJITAddIC();
 
-    gen.generateFastPath(*this);
+    m_instructionToJITAddIC.add(currentInstruction, addIC);
+    MathICGenerationState& addICGenerationState = m_instructionToJITAddICGenerationState.add(currentInstruction, MathICGenerationState()).iterator->value;
 
-    if (gen.didEmitFastPath()) {
-        gen.endJumpList().link(this);
-        emitPutVirtualRegister(result, resultRegs);
-        
-        addSlowCase(gen.slowPathJumpList());
-    } else {
-        ASSERT(gen.endJumpList().empty());
-        ASSERT(gen.slowPathJumpList().empty());
-        if (resultProfile) {
-            if (leftOperand.isConst())
-                emitGetVirtualRegister(op1, leftRegs);
-            if (rightOperand.isConst())
-                emitGetVirtualRegister(op2, rightRegs);
-            callOperation(operationValueAddProfiled, resultRegs, leftRegs, rightRegs, resultProfile);
-            emitPutVirtualRegister(result, resultRegs);
-        } else {
-            JITSlowPathCall slowPathCall(this, currentInstruction, slow_path_add);
-            slowPathCall.call();
-        }
-    }
+    addIC->m_generator = JITAddGenerator(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs, fpRegT0, fpRegT1, scratchGPR, scratchFPR, arithProfile);
+
+    bool generatedInlineCode = addIC->generateInline(*this, addICGenerationState);
+    if (!generatedInlineCode) {
+        if (leftOperand.isConst())
+            emitGetVirtualRegister(op1, leftRegs);
+        else if (rightOperand.isConst())
+            emitGetVirtualRegister(op2, rightRegs);
+
+        if (arithProfile)
+            callOperation(operationValueAddProfiled, resultRegs, leftRegs, rightRegs, arithProfile);
+        else
+            callOperation(operationValueAdd, resultRegs, leftRegs, rightRegs);
+    } else
+        addSlowCase(addICGenerationState.slowPathJumps);
+    emitPutVirtualRegister(result, resultRegs);
 }
 
 void JIT::emitSlow_op_add(Instruction* currentInstruction, Vector<SlowCaseEntry>::iterator& iter)
 {
     linkAllSlowCasesForBytecodeOffset(m_slowCases, iter, m_bytecodeOffset);
 
+    MathICGenerationState& addICGenerationState = m_instructionToJITAddICGenerationState.find(currentInstruction)->value;
+    addICGenerationState.slowPathStart = label();
+
     int result = currentInstruction[1].u.operand;
     int op1 = currentInstruction[2].u.operand;
     int op2 = currentInstruction[3].u.operand;
-    OperandTypes types = OperandTypes::fromInt(currentInstruction[4].u.operand);
 
 #if USE(JSVALUE64)
+    OperandTypes types = getOperandTypes(copiedInstruction(currentInstruction));
     JSValueRegs leftRegs = JSValueRegs(regT0);
     JSValueRegs rightRegs = JSValueRegs(regT1);
     JSValueRegs resultRegs = leftRegs;
 #else
+    OperandTypes types = getOperandTypes(currentInstruction);
     JSValueRegs leftRegs = JSValueRegs(regT1, regT0);
     JSValueRegs rightRegs = JSValueRegs(regT3, regT2);
     JSValueRegs resultRegs = leftRegs;
@@ -769,18 +777,26 @@ void JIT::emitSlow_op_add(Instruction* currentInstruction, Vector<SlowCaseEntry>
     else if (isOperandConstantInt(op2))
         rightOperand.setConstInt32(getOperandConstantInt(op2));
 
+    if (leftOperand.isConst())
+        emitGetVirtualRegister(op1, leftRegs);
+    if (rightOperand.isConst())
+        emitGetVirtualRegister(op2, rightRegs);
+
+    JITAddIC* addIC = m_instructionToJITAddIC.get(currentInstruction);
     if (shouldEmitProfiling()) {
-        if (leftOperand.isConst())
-            emitGetVirtualRegister(op1, leftRegs);
-        if (rightOperand.isConst())
-            emitGetVirtualRegister(op2, rightRegs);
-        ResultProfile* resultProfile = m_codeBlock->ensureResultProfile(m_bytecodeOffset);
-        callOperation(operationValueAddProfiled, resultRegs, leftRegs, rightRegs, resultProfile);
-        emitPutVirtualRegister(result, resultRegs);
-    } else {
-        JITSlowPathCall slowPathCall(this, currentInstruction, slow_path_add);
-        slowPathCall.call();
-    }
+        ArithProfile& arithProfile = m_codeBlock->arithProfileForPC(currentInstruction);
+        if (addICGenerationState.shouldSlowPathRepatch)
+            addICGenerationState.slowPathCall = callOperation(operationValueAddProfiledOptimize, resultRegs, leftRegs, rightRegs, &arithProfile, addIC);
+        else
+            addICGenerationState.slowPathCall = callOperation(operationValueAddProfiled, resultRegs, leftRegs, rightRegs, &arithProfile);
+    } else
+        addICGenerationState.slowPathCall = callOperation(operationValueAddOptimize, resultRegs, leftRegs, rightRegs, addIC);
+    emitPutVirtualRegister(result, resultRegs);
+
+    addLinkTask([=] (LinkBuffer& linkBuffer) {
+        MathICGenerationState& addICGenerationState = m_instructionToJITAddICGenerationState.find(currentInstruction)->value;
+        addIC->finalizeInlineCode(addICGenerationState, linkBuffer);
+    });
 }
 
 void JIT::emit_op_div(Instruction* currentInstruction)
@@ -788,14 +804,15 @@ void JIT::emit_op_div(Instruction* currentInstruction)
     int result = currentInstruction[1].u.operand;
     int op1 = currentInstruction[2].u.operand;
     int op2 = currentInstruction[3].u.operand;
-    OperandTypes types = OperandTypes::fromInt(currentInstruction[4].u.operand);
 
 #if USE(JSVALUE64)
+    OperandTypes types = getOperandTypes(copiedInstruction(currentInstruction));
     JSValueRegs leftRegs = JSValueRegs(regT0);
     JSValueRegs rightRegs = JSValueRegs(regT1);
     JSValueRegs resultRegs = leftRegs;
     GPRReg scratchGPR = regT2;
 #else
+    OperandTypes types = getOperandTypes(currentInstruction);
     JSValueRegs leftRegs = JSValueRegs(regT1, regT0);
     JSValueRegs rightRegs = JSValueRegs(regT3, regT2);
     JSValueRegs resultRegs = leftRegs;
@@ -803,9 +820,9 @@ void JIT::emit_op_div(Instruction* currentInstruction)
 #endif
     FPRReg scratchFPR = fpRegT2;
 
-    ResultProfile* resultProfile = nullptr;
+    ArithProfile* arithProfile = nullptr;
     if (shouldEmitProfiling())
-        resultProfile = m_codeBlock->ensureResultProfile(m_bytecodeOffset);
+        arithProfile = &m_codeBlock->arithProfileForPC(currentInstruction);
 
     SnippetOperand leftOperand(types.first());
     SnippetOperand rightOperand(types.second());
@@ -831,7 +848,7 @@ void JIT::emit_op_div(Instruction* currentInstruction)
         emitGetVirtualRegister(op2, rightRegs);
 
     JITDivGenerator gen(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs,
-        fpRegT0, fpRegT1, scratchGPR, scratchFPR, resultProfile);
+        fpRegT0, fpRegT1, scratchGPR, scratchFPR, arithProfile);
 
     gen.generateFastPath(*this);
 
@@ -861,15 +878,16 @@ void JIT::emit_op_mul(Instruction* currentInstruction)
     int result = currentInstruction[1].u.operand;
     int op1 = currentInstruction[2].u.operand;
     int op2 = currentInstruction[3].u.operand;
-    OperandTypes types = OperandTypes::fromInt(currentInstruction[4].u.operand);
 
 #if USE(JSVALUE64)
+    OperandTypes types = getOperandTypes(copiedInstruction(currentInstruction));
     JSValueRegs leftRegs = JSValueRegs(regT0);
     JSValueRegs rightRegs = JSValueRegs(regT1);
     JSValueRegs resultRegs = JSValueRegs(regT2);
     GPRReg scratchGPR = regT3;
     FPRReg scratchFPR = InvalidFPRReg;
 #else
+    OperandTypes types = getOperandTypes(currentInstruction);
     JSValueRegs leftRegs = JSValueRegs(regT1, regT0);
     JSValueRegs rightRegs = JSValueRegs(regT3, regT2);
     JSValueRegs resultRegs = leftRegs;
@@ -877,9 +895,9 @@ void JIT::emit_op_mul(Instruction* currentInstruction)
     FPRReg scratchFPR = fpRegT2;
 #endif
 
-    ResultProfile* resultProfile = nullptr;
+    ArithProfile* arithProfile = nullptr;
     if (shouldEmitProfiling())
-        resultProfile = m_codeBlock->ensureResultProfile(m_bytecodeOffset);
+        arithProfile = &m_codeBlock->arithProfileForPC(currentInstruction);
 
     SnippetOperand leftOperand(types.first());
     SnippetOperand rightOperand(types.second());
@@ -897,7 +915,7 @@ void JIT::emit_op_mul(Instruction* currentInstruction)
         emitGetVirtualRegister(op2, rightRegs);
 
     JITMulGenerator gen(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs,
-        fpRegT0, fpRegT1, scratchGPR, scratchFPR, resultProfile);
+        fpRegT0, fpRegT1, scratchGPR, scratchFPR, arithProfile);
 
     gen.generateFastPath(*this);
 
@@ -909,12 +927,12 @@ void JIT::emit_op_mul(Instruction* currentInstruction)
     } else {
         ASSERT(gen.endJumpList().empty());
         ASSERT(gen.slowPathJumpList().empty());
-        if (resultProfile) {
+        if (arithProfile) {
             if (leftOperand.isPositiveConstInt32())
                 emitGetVirtualRegister(op1, leftRegs);
             if (rightOperand.isPositiveConstInt32())
                 emitGetVirtualRegister(op2, rightRegs);
-            callOperation(operationValueMulProfiled, resultRegs, leftRegs, rightRegs, resultProfile);
+            callOperation(operationValueMulProfiled, resultRegs, leftRegs, rightRegs, arithProfile);
             emitPutVirtualRegister(result, resultRegs);
         } else {
             JITSlowPathCall slowPathCall(this, currentInstruction, slow_path_mul);
@@ -930,13 +948,14 @@ void JIT::emitSlow_op_mul(Instruction* currentInstruction, Vector<SlowCaseEntry>
     int result = currentInstruction[1].u.operand;
     int op1 = currentInstruction[2].u.operand;
     int op2 = currentInstruction[3].u.operand;
-    OperandTypes types = OperandTypes::fromInt(currentInstruction[4].u.operand);
 
 #if USE(JSVALUE64)
+    OperandTypes types = getOperandTypes(copiedInstruction(currentInstruction));
     JSValueRegs leftRegs = JSValueRegs(regT0);
     JSValueRegs rightRegs = JSValueRegs(regT1);
     JSValueRegs resultRegs = leftRegs;
 #else
+    OperandTypes types = getOperandTypes(currentInstruction);
     JSValueRegs leftRegs = JSValueRegs(regT1, regT0);
     JSValueRegs rightRegs = JSValueRegs(regT3, regT2);
     JSValueRegs resultRegs = leftRegs;
@@ -955,8 +974,8 @@ void JIT::emitSlow_op_mul(Instruction* currentInstruction, Vector<SlowCaseEntry>
             emitGetVirtualRegister(op1, leftRegs);
         if (rightOperand.isPositiveConstInt32())
             emitGetVirtualRegister(op2, rightRegs);
-        ResultProfile* resultProfile = m_codeBlock->ensureResultProfile(m_bytecodeOffset);
-        callOperation(operationValueMulProfiled, resultRegs, leftRegs, rightRegs, resultProfile);
+        ArithProfile& arithProfile = m_codeBlock->arithProfileForPC(currentInstruction);
+        callOperation(operationValueMulProfiled, resultRegs, leftRegs, rightRegs, &arithProfile);
         emitPutVirtualRegister(result, resultRegs);
     } else {
         JITSlowPathCall slowPathCall(this, currentInstruction, slow_path_mul);
@@ -969,15 +988,16 @@ void JIT::emit_op_sub(Instruction* currentInstruction)
     int result = currentInstruction[1].u.operand;
     int op1 = currentInstruction[2].u.operand;
     int op2 = currentInstruction[3].u.operand;
-    OperandTypes types = OperandTypes::fromInt(currentInstruction[4].u.operand);
 
 #if USE(JSVALUE64)
+    OperandTypes types = getOperandTypes(copiedInstruction(currentInstruction));
     JSValueRegs leftRegs = JSValueRegs(regT0);
     JSValueRegs rightRegs = JSValueRegs(regT1);
     JSValueRegs resultRegs = leftRegs;
     GPRReg scratchGPR = regT2;
     FPRReg scratchFPR = InvalidFPRReg;
 #else
+    OperandTypes types = getOperandTypes(currentInstruction);
     JSValueRegs leftRegs = JSValueRegs(regT1, regT0);
     JSValueRegs rightRegs = JSValueRegs(regT3, regT2);
     JSValueRegs resultRegs = leftRegs;
@@ -985,9 +1005,9 @@ void JIT::emit_op_sub(Instruction* currentInstruction)
     FPRReg scratchFPR = fpRegT2;
 #endif
 
-    ResultProfile* resultProfile = nullptr;
+    ArithProfile* arithProfile = nullptr;
     if (shouldEmitProfiling())
-        resultProfile = m_codeBlock->ensureResultProfile(m_bytecodeOffset);
+        arithProfile = &m_codeBlock->arithProfileForPC(currentInstruction);
 
     SnippetOperand leftOperand(types.first());
     SnippetOperand rightOperand(types.second());
@@ -996,7 +1016,7 @@ void JIT::emit_op_sub(Instruction* currentInstruction)
     emitGetVirtualRegister(op2, rightRegs);
 
     JITSubGenerator gen(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs,
-        fpRegT0, fpRegT1, scratchGPR, scratchFPR, resultProfile);
+        fpRegT0, fpRegT1, scratchGPR, scratchFPR, arithProfile);
 
     gen.generateFastPath(*this);
 
@@ -1023,8 +1043,8 @@ void JIT::emitSlow_op_sub(Instruction* currentInstruction, Vector<SlowCaseEntry>
 #endif
 
     if (shouldEmitProfiling()) {
-        ResultProfile* resultProfile = m_codeBlock->ensureResultProfile(m_bytecodeOffset);
-        callOperation(operationValueSubProfiled, resultRegs, leftRegs, rightRegs, resultProfile);
+        ArithProfile& arithProfile = m_codeBlock->arithProfileForPC(currentInstruction);
+        callOperation(operationValueSubProfiled, resultRegs, leftRegs, rightRegs, &arithProfile);
         emitPutVirtualRegister(result, resultRegs);
     } else {
         JITSlowPathCall slowPathCall(this, currentInstruction, slow_path_sub);
index f205fdf..be92d3f 100644 (file)
@@ -28,6 +28,7 @@
 
 #if ENABLE(JIT)
 
+#include "ArithProfile.h"
 #include "JSCJSValueInlines.h"
 #include "MathCommon.h"
 
@@ -101,9 +102,9 @@ void JITDivGenerator::generateFastPath(CCallHelpers& jit)
     }
 
     // Is the result actually an integer? The DFG JIT would really like to know. If it's
-    // not an integer, we increment a count. If this together with the slow case counter
-    // are below threshold then the DFG JIT will compile this division with a speculation
-    // that the remainder is zero.
+    // not an integer, we set a bit. If this together with the slow case counter are below
+    // threshold then the DFG JIT will compile this division with a speculation that the
+    // remainder is zero.
 
     // As well, there are cases where a double result here would cause an important field
     // in the heap to sometimes have doubles in it, resulting in double predictions getting
@@ -128,8 +129,8 @@ void JITDivGenerator::generateFastPath(CCallHelpers& jit)
 
     notDoubleZero.link(&jit);
 #endif
-    if (m_resultProfile)
-        jit.add32(CCallHelpers::TrustedImm32(1), CCallHelpers::AbsoluteAddress(m_resultProfile->addressOfSpecialFastPathCount()));
+    if (m_arithProfile)
+        jit.or32(CCallHelpers::TrustedImm32(ArithProfile::specialFastPathBit), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits()));
     jit.boxDouble(m_leftFPR, m_result);
 }
 
index f9911f3..a5ae088 100644 (file)
@@ -38,7 +38,7 @@ public:
     JITDivGenerator(SnippetOperand leftOperand, SnippetOperand rightOperand,
         JSValueRegs result, JSValueRegs left, JSValueRegs right,
         FPRReg leftFPR, FPRReg rightFPR, GPRReg scratchGPR, FPRReg scratchFPR,
-        ResultProfile* resultProfile = nullptr)
+        ArithProfile* arithProfile = nullptr)
         : m_leftOperand(leftOperand)
         , m_rightOperand(rightOperand)
         , m_result(result)
@@ -48,7 +48,7 @@ public:
         , m_rightFPR(rightFPR)
         , m_scratchGPR(scratchGPR)
         , m_scratchFPR(scratchFPR)
-        , m_resultProfile(resultProfile)
+        , m_arithProfile(arithProfile)
     {
         ASSERT(!m_leftOperand.isConstInt32() || !m_rightOperand.isConstInt32());
     }
@@ -71,7 +71,7 @@ private:
     FPRReg m_rightFPR;
     GPRReg m_scratchGPR;
     FPRReg m_scratchFPR;
-    ResultProfile* m_resultProfile;
+    ArithProfile* m_arithProfile;
     bool m_didEmitFastPath { false };
 
     CCallHelpers::JumpList m_endJumpList;
index 9905f5e..31b1d4d 100644 (file)
@@ -416,9 +416,33 @@ ALWAYS_INLINE MacroAssembler::Call JIT::callOperation(V_JITOperation_ECIZC opera
     return appendCallWithExceptionCheck(operation);
 }
 
-inline MacroAssembler::Call JIT::callOperation(J_JITOperation_EJJRp operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, ResultProfile* resultProfile)
+ALWAYS_INLINE MacroAssembler::Call JIT::callOperation(J_JITOperation_EJJ operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2)
 {
-    setupArgumentsWithExecState(arg1, arg2, TrustedImmPtr(resultProfile));
+    setupArgumentsWithExecState(arg1, arg2);
+    Call call = appendCallWithExceptionCheck(operation);
+    setupResults(result);
+    return call;
+}
+
+ALWAYS_INLINE MacroAssembler::Call JIT::callOperation(J_JITOperation_EJJArp operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, ArithProfile* arithProfile)
+{
+    setupArgumentsWithExecState(arg1, arg2, TrustedImmPtr(arithProfile));
+    Call call = appendCallWithExceptionCheck(operation);
+    setupResults(result);
+    return call;
+}
+
+ALWAYS_INLINE MacroAssembler::Call JIT::callOperation(J_JITOperation_EJJArpJaic operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, ArithProfile* arithProfile, JITAddIC* addIC)
+{
+    setupArgumentsWithExecState(arg1, arg2, TrustedImmPtr(arithProfile), TrustedImmPtr(addIC));
+    Call call = appendCallWithExceptionCheck(operation);
+    setupResults(result);
+    return call;
+}
+
+ALWAYS_INLINE MacroAssembler::Call JIT::callOperation(J_JITOperation_EJJJaic operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, JITAddIC* addIC)
+{
+    setupArgumentsWithExecState(arg1, arg2, TrustedImmPtr(addIC));
     Call call = appendCallWithExceptionCheck(operation);
     setupResults(result);
     return call;
diff --git a/Source/JavaScriptCore/jit/JITMathIC.h b/Source/JavaScriptCore/jit/JITMathIC.h
new file mode 100644 (file)
index 0000000..c6fbfca
--- /dev/null
@@ -0,0 +1,163 @@
+/*
+ * Copyright (C) 2016 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#pragma once
+
+#if ENABLE(JIT)
+
+#include "CCallHelpers.h"
+#include "JITAddGenerator.h"
+#include "JITMathICInlineResult.h"
+#include "LinkBuffer.h"
+#include "Repatch.h"
+#include "SnippetOperand.h"
+
+namespace JSC {
+
+class LinkBuffer;
+
+struct MathICGenerationState {
+    MacroAssembler::Label fastPathStart;
+    MacroAssembler::Label fastPathEnd;
+    MacroAssembler::Label slowPathStart;
+    MacroAssembler::Call slowPathCall;
+    MacroAssembler::JumpList slowPathJumps;
+    bool shouldSlowPathRepatch;
+};
+
+template <typename GeneratorType>
+class JITMathIC {
+public:
+    CodeLocationLabel doneLocation() { return m_inlineStart.labelAtOffset(m_inlineSize); }
+    CodeLocationLabel slowPathStartLocation() { return m_inlineStart.labelAtOffset(m_deltaFromStartToSlowPathStart); }
+    CodeLocationCall slowPathCallLocation() { return m_inlineStart.callAtOffset(m_deltaFromStartToSlowPathCallLocation); }
+
+    bool generateInline(CCallHelpers& jit, MathICGenerationState& state)
+    {
+        state.fastPathStart = jit.label();
+        size_t startSize = jit.m_assembler.buffer().codeSize();
+        JITMathICInlineResult result = m_generator.generateInline(jit, state);
+
+        switch (result) {
+        case JITMathICInlineResult::GeneratedFastPath: {
+            size_t inlineSize = jit.m_assembler.buffer().codeSize() - startSize;
+            if (static_cast<ptrdiff_t>(inlineSize) < MacroAssembler::maxJumpReplacementSize()) {
+                size_t nopsToEmitInBytes = MacroAssembler::maxJumpReplacementSize() - inlineSize;
+                jit.emitNops(nopsToEmitInBytes);
+            }
+            state.shouldSlowPathRepatch = true;
+            state.fastPathEnd = jit.label();
+            return true;
+        }
+        case JITMathICInlineResult::GenerateFullSnippet: {
+            MacroAssembler::JumpList endJumpList;
+            bool result = m_generator.generateFastPath(jit, endJumpList, state.slowPathJumps);
+            if (result) {
+                state.fastPathEnd = jit.label();
+                state.shouldSlowPathRepatch = false;
+                endJumpList.link(&jit);
+                return true;
+            }
+            return false;
+        }
+        case JITMathICInlineResult::DontGenerate: {
+            return false;
+        }
+        default:
+            ASSERT_NOT_REACHED();
+        }
+
+        return false;
+    }
+
+    void generateOutOfLine(VM& vm, CodeBlock* codeBlock, FunctionPtr callReplacement)
+    {
+        // We rewire to the alternate regardless of whether or not we can allocate the out of line path
+        // because if we fail allocating the out of line path, we don't want to waste time trying to
+        // allocate it in the future.
+        ftlThunkAwareRepatchCall(codeBlock, slowPathCallLocation(), callReplacement);
+
+        {
+            CCallHelpers jit(&vm, codeBlock);
+
+            MacroAssembler::JumpList endJumpList; 
+            MacroAssembler::JumpList slowPathJumpList; 
+            bool emittedFastPath = m_generator.generateFastPath(jit, endJumpList, slowPathJumpList);
+            if (!emittedFastPath)
+                return;
+            endJumpList.append(jit.jump());
+
+            LinkBuffer linkBuffer(vm, jit, codeBlock, JITCompilationCanFail);
+            if (linkBuffer.didFailToAllocate())
+                return;
+
+            linkBuffer.link(endJumpList, doneLocation());
+            linkBuffer.link(slowPathJumpList, slowPathStartLocation());
+
+            m_code = FINALIZE_CODE_FOR(
+                codeBlock, linkBuffer, ("JITMathIC: generating out of line IC snippet"));
+        }
+
+        {
+            CCallHelpers jit(&vm, codeBlock);
+            auto jump = jit.jump();
+            // We don't need a nop sled here because nobody should be jumping into the middle of an IC.
+            bool needsBranchCompaction = false;
+            RELEASE_ASSERT(jit.m_assembler.buffer().codeSize() <= static_cast<size_t>(m_inlineSize));
+            LinkBuffer linkBuffer(jit, m_inlineStart.dataLocation(), jit.m_assembler.buffer().codeSize(), JITCompilationMustSucceed, needsBranchCompaction);
+            RELEASE_ASSERT(linkBuffer.isValid());
+            linkBuffer.link(jump, CodeLocationLabel(m_code.code().executableAddress()));
+            FINALIZE_CODE(linkBuffer, ("JITMathIC: linking constant jump to out of line stub"));
+        }
+    }
+
+    void finalizeInlineCode(const MathICGenerationState& state, LinkBuffer& linkBuffer)
+    {
+        CodeLocationLabel start = linkBuffer.locationOf(state.fastPathStart);
+        m_inlineStart = start;
+
+        m_inlineSize = MacroAssembler::differenceBetweenCodePtr(
+            start, linkBuffer.locationOf(state.fastPathEnd));
+        ASSERT(m_inlineSize > 0);
+
+        m_deltaFromStartToSlowPathCallLocation = MacroAssembler::differenceBetweenCodePtr(
+            start, linkBuffer.locationOf(state.slowPathCall));
+        m_deltaFromStartToSlowPathStart = MacroAssembler::differenceBetweenCodePtr(
+            start, linkBuffer.locationOf(state.slowPathStart));
+    }
+
+    MacroAssemblerCodeRef m_code;
+    CodeLocationLabel m_inlineStart;
+    int32_t m_inlineSize;
+    int32_t m_deltaFromStartToSlowPathCallLocation;
+    int32_t m_deltaFromStartToSlowPathStart;
+    GeneratorType m_generator;
+};
+
+typedef JITMathIC<JITAddGenerator> JITAddIC;
+
+} // namespace JSC
+
+#endif // ENABLE(JIT)
diff --git a/Source/JavaScriptCore/jit/JITMathICForwards.h b/Source/JavaScriptCore/jit/JITMathICForwards.h
new file mode 100644 (file)
index 0000000..ad0a429
--- /dev/null
@@ -0,0 +1,39 @@
+/*
+ * Copyright (C) 2016 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#pragma once
+
+#if ENABLE(JIT)
+
+namespace JSC {
+
+template <typename Generator> class JITMathIC;
+class JITAddGenerator;
+
+typedef JITMathIC<JITAddGenerator> JITAddIC;
+
+} // namespace JSC
+
+#endif // ENABLE(JIT)
diff --git a/Source/JavaScriptCore/jit/JITMathICInlineResult.h b/Source/JavaScriptCore/jit/JITMathICInlineResult.h
new file mode 100644 (file)
index 0000000..0189c13
--- /dev/null
@@ -0,0 +1,40 @@
+/*
+ * Copyright (C) 2016 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#pragma once
+
+#if ENABLE(JIT)
+
+namespace JSC {
+
+enum class JITMathICInlineResult {
+    GeneratedFastPath,
+    GenerateFullSnippet,
+    DontGenerate
+};
+
+} // namespace JSC
+
+#endif // ENABLE(JIT)
index e0732be..169a260 100644 (file)
@@ -28,6 +28,8 @@
 
 #if ENABLE(JIT)
 
+#include "ArithProfile.h"
+
 namespace JSC {
 
 void JITMulGenerator::generateFastPath(CCallHelpers& jit)
@@ -137,7 +139,7 @@ void JITMulGenerator::generateFastPath(CCallHelpers& jit)
     // Do doubleVar * doubleVar.
     jit.mulDouble(m_rightFPR, m_leftFPR);
 
-    if (!m_resultProfile)
+    if (!m_arithProfile)
         jit.boxDouble(m_leftFPR, m_result);
     else {
         // The Int52 overflow check below intentionally omits 1ll << 51 as a valid negative Int52 value.
@@ -149,18 +151,18 @@ void JITMulGenerator::generateFastPath(CCallHelpers& jit)
         jit.moveDoubleTo64(m_leftFPR, m_result.payloadGPR());
         CCallHelpers::Jump notNegativeZero = jit.branch64(CCallHelpers::NotEqual, m_result.payloadGPR(), CCallHelpers::TrustedImm64(negativeZeroBits));
 
-        jit.or32(CCallHelpers::TrustedImm32(ResultProfile::NegZeroDouble), CCallHelpers::AbsoluteAddress(m_resultProfile->addressOfFlags()));
+        jit.or32(CCallHelpers::TrustedImm32(ArithProfile::NegZeroDouble), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits()));
         CCallHelpers::Jump done = jit.jump();
 
         notNegativeZero.link(&jit);
-        jit.or32(CCallHelpers::TrustedImm32(ResultProfile::NonNegZeroDouble), CCallHelpers::AbsoluteAddress(m_resultProfile->addressOfFlags()));
+        jit.or32(CCallHelpers::TrustedImm32(ArithProfile::NonNegZeroDouble), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits()));
 
         jit.move(m_result.payloadGPR(), m_scratchGPR);
         jit.urshiftPtr(CCallHelpers::Imm32(52), m_scratchGPR);
         jit.and32(CCallHelpers::Imm32(0x7ff), m_scratchGPR);
         CCallHelpers::Jump noInt52Overflow = jit.branch32(CCallHelpers::LessThanOrEqual, m_scratchGPR, CCallHelpers::TrustedImm32(0x431));
 
-        jit.or32(CCallHelpers::TrustedImm32(ResultProfile::Int52Overflow), CCallHelpers::AbsoluteAddress(m_resultProfile->addressOfFlags()));
+        jit.or32(CCallHelpers::TrustedImm32(ArithProfile::Int52Overflow), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits()));
         noInt52Overflow.link(&jit);
 
         done.link(&jit);
@@ -171,18 +173,18 @@ void JITMulGenerator::generateFastPath(CCallHelpers& jit)
         notNegativeZero.append(jit.branch32(CCallHelpers::NotEqual, m_result.payloadGPR(), CCallHelpers::TrustedImm32(0)));
         notNegativeZero.append(jit.branch32(CCallHelpers::NotEqual, m_result.tagGPR(), CCallHelpers::TrustedImm32(negativeZeroBits >> 32)));
 
-        jit.or32(CCallHelpers::TrustedImm32(ResultProfile::NegZeroDouble), CCallHelpers::AbsoluteAddress(m_resultProfile->addressOfFlags()));
+        jit.or32(CCallHelpers::TrustedImm32(ArithProfile::NegZeroDouble), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits()));
         CCallHelpers::Jump done = jit.jump();
 
         notNegativeZero.link(&jit);
-        jit.or32(CCallHelpers::TrustedImm32(ResultProfile::NonNegZeroDouble), CCallHelpers::AbsoluteAddress(m_resultProfile->addressOfFlags()));
+        jit.or32(CCallHelpers::TrustedImm32(ArithProfile::NonNegZeroDouble), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits()));
 
         jit.move(m_result.tagGPR(), m_scratchGPR);
         jit.urshiftPtr(CCallHelpers::Imm32(52 - 32), m_scratchGPR);
         jit.and32(CCallHelpers::Imm32(0x7ff), m_scratchGPR);
         CCallHelpers::Jump noInt52Overflow = jit.branch32(CCallHelpers::LessThanOrEqual, m_scratchGPR, CCallHelpers::TrustedImm32(0x431));
         
-        jit.or32(CCallHelpers::TrustedImm32(ResultProfile::Int52Overflow), CCallHelpers::AbsoluteAddress(m_resultProfile->addressOfFlags()));
+        jit.or32(CCallHelpers::TrustedImm32(ArithProfile::Int52Overflow), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits()));
 
         m_endJumpList.append(noInt52Overflow);
         if (m_scratchGPR == m_result.tagGPR() || m_scratchGPR == m_result.payloadGPR())
index faa033b..a73e9d9 100644 (file)
@@ -38,7 +38,7 @@ public:
     JITMulGenerator(SnippetOperand leftOperand, SnippetOperand rightOperand,
         JSValueRegs result, JSValueRegs left, JSValueRegs right,
         FPRReg leftFPR, FPRReg rightFPR, GPRReg scratchGPR, FPRReg scratchFPR,
-        ResultProfile* resultProfile = nullptr)
+        ArithProfile* arithProfile = nullptr)
         : m_leftOperand(leftOperand)
         , m_rightOperand(rightOperand)
         , m_result(result)
@@ -48,7 +48,7 @@ public:
         , m_rightFPR(rightFPR)
         , m_scratchGPR(scratchGPR)
         , m_scratchFPR(scratchFPR)
-        , m_resultProfile(resultProfile)
+        , m_arithProfile(arithProfile)
     {
         ASSERT(!m_leftOperand.isPositiveConstInt32() || !m_rightOperand.isPositiveConstInt32());
     }
@@ -69,7 +69,7 @@ private:
     FPRReg m_rightFPR;
     GPRReg m_scratchGPR;
     FPRReg m_scratchFPR;
-    ResultProfile* m_resultProfile;
+    ArithProfile* m_arithProfile;
     bool m_didEmitFastPath { false };
 
     CCallHelpers::JumpList m_endJumpList;
index ad419da..2e9396a 100644 (file)
@@ -28,6 +28,7 @@
 
 #if ENABLE(JIT)
 
+#include "ArithProfile.h"
 #include "ArrayConstructor.h"
 #include "CommonSlowPaths.h"
 #include "DFGCompilationMode.h"
@@ -2247,18 +2248,82 @@ JSCell* JIT_OPERATION operationToIndexString(ExecState* exec, int32_t index)
     return jsString(exec, Identifier::from(exec, index).string());
 }
 
+ALWAYS_INLINE static EncodedJSValue unprofiledAdd(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2)
+{
+    VM* vm = &exec->vm();
+    NativeCallFrameTracer tracer(vm, exec);
+    
+    JSValue op1 = JSValue::decode(encodedOp1);
+    JSValue op2 = JSValue::decode(encodedOp2);
+    
+    return JSValue::encode(jsAdd(exec, op1, op2));
+}
+
+ALWAYS_INLINE static EncodedJSValue profiledAdd(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile* arithProfile)
+{
+    VM* vm = &exec->vm();
+    NativeCallFrameTracer tracer(vm, exec);
+    
+    JSValue op1 = JSValue::decode(encodedOp1);
+    JSValue op2 = JSValue::decode(encodedOp2);
+
+    ASSERT(arithProfile);
+    arithProfile->observeLHSAndRHS(op1, op2);
+
+    JSValue result = jsAdd(exec, op1, op2);
+    arithProfile->observeResult(result);
+
+    return JSValue::encode(result);
+}
+
 EncodedJSValue JIT_OPERATION operationValueAdd(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2)
 {
+    return unprofiledAdd(exec, encodedOp1, encodedOp2);
+}
+
+EncodedJSValue JIT_OPERATION operationValueAddProfiled(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile* arithProfile)
+{
+    return profiledAdd(exec, encodedOp1, encodedOp2, arithProfile);
+}
+
+EncodedJSValue JIT_OPERATION operationValueAddProfiledOptimize(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile* arithProfile, JITAddIC* addIC)
+{
     VM* vm = &exec->vm();
     NativeCallFrameTracer tracer(vm, exec);
     
     JSValue op1 = JSValue::decode(encodedOp1);
     JSValue op2 = JSValue::decode(encodedOp2);
+
+    ASSERT(arithProfile);
+    arithProfile->observeLHSAndRHS(op1, op2);
+    auto nonOptimizeVariant = operationValueAddProfiledNoOptimize;
+    addIC->generateOutOfLine(*vm, exec->codeBlock(), nonOptimizeVariant);
     
+    JSValue result = jsAdd(exec, op1, op2);
+    arithProfile->observeResult(result);
+
+    return JSValue::encode(result);
+}
+
+EncodedJSValue JIT_OPERATION operationValueAddProfiledNoOptimize(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile* arithProfile, JITAddIC*)
+{
+    return profiledAdd(exec, encodedOp1, encodedOp2, arithProfile);
+}
+
+EncodedJSValue JIT_OPERATION operationValueAddOptimize(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, JITAddIC* addIC)
+{
+    VM* vm = &exec->vm();
+    NativeCallFrameTracer tracer(vm, exec);
+
+    auto nonOptimizeVariant = operationValueAddNoOptimize;
+    addIC->generateOutOfLine(*vm, exec->codeBlock(), nonOptimizeVariant);
+
+    JSValue op1 = JSValue::decode(encodedOp1);
+    JSValue op2 = JSValue::decode(encodedOp2);
     return JSValue::encode(jsAdd(exec, op1, op2));
 }
 
-EncodedJSValue JIT_OPERATION operationValueAddProfiled(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ResultProfile* resultProfile)
+EncodedJSValue JIT_OPERATION operationValueAddNoOptimize(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, JITAddIC*)
 {
     VM* vm = &exec->vm();
     NativeCallFrameTracer tracer(vm, exec);
@@ -2267,7 +2332,7 @@ EncodedJSValue JIT_OPERATION operationValueAddProfiled(ExecState* exec, EncodedJ
     JSValue op2 = JSValue::decode(encodedOp2);
     
     JSValue result = jsAdd(exec, op1, op2);
-    resultProfile->detectNumericness(result);
+
     return JSValue::encode(result);
 }
 
@@ -2284,7 +2349,7 @@ EncodedJSValue JIT_OPERATION operationValueMul(ExecState* exec, EncodedJSValue e
     return JSValue::encode(jsNumber(a * b));
 }
 
-EncodedJSValue JIT_OPERATION operationValueMulProfiled(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ResultProfile* resultProfile)
+EncodedJSValue JIT_OPERATION operationValueMulProfiled(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile* arithProfile)
 {
     VM* vm = &exec->vm();
     NativeCallFrameTracer tracer(vm, exec);
@@ -2296,7 +2361,7 @@ EncodedJSValue JIT_OPERATION operationValueMulProfiled(ExecState* exec, EncodedJ
     double b = op2.toNumber(exec);
     
     JSValue result = jsNumber(a * b);
-    resultProfile->detectNumericness(result);
+    arithProfile->observeResult(result);
     return JSValue::encode(result);
 }
 
@@ -2313,7 +2378,7 @@ EncodedJSValue JIT_OPERATION operationValueSub(ExecState* exec, EncodedJSValue e
     return JSValue::encode(jsNumber(a - b));
 }
 
-EncodedJSValue JIT_OPERATION operationValueSubProfiled(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ResultProfile* resultProfile)
+EncodedJSValue JIT_OPERATION operationValueSubProfiled(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile* arithProfile)
 {
     VM* vm = &exec->vm();
     NativeCallFrameTracer tracer(vm, exec);
@@ -2325,7 +2390,7 @@ EncodedJSValue JIT_OPERATION operationValueSubProfiled(ExecState* exec, EncodedJ
     double b = op2.toNumber(exec);
     
     JSValue result = jsNumber(a - b);
-    resultProfile->detectNumericness(result);
+    arithProfile->observeResult(result);
     return JSValue::encode(result);
 }
 
index 0ed1a22..ce96ac3 100644 (file)
@@ -28,6 +28,7 @@
 
 #if ENABLE(JIT)
 
+#include "JITMathICForwards.h"
 #include "MacroAssemblerCodeRef.h"
 #include "PropertyOffset.h"
 #include "SlowPathReturnType.h"
@@ -42,6 +43,7 @@ class ArrayProfile;
 class CallLinkInfo;
 class CodeBlock;
 class ExecState;
+class JITAddGenerator;
 class JSArray;
 class JSFunction;
 class JSLexicalEnvironment;
@@ -54,7 +56,7 @@ class WatchpointSet;
 
 struct ByValInfo;
 struct InlineCallFrame;
-struct ResultProfile;
+struct ArithProfile;
 
 typedef ExecState CallFrame;
 
@@ -69,6 +71,7 @@ typedef char* UnusedPtr;
     A: JSArray*
     Aap: ArrayAllocationProfile*
     Ap: ArrayProfile*
+    Arp: ArithProfile*
     By: ByValInfo*
     C: JSCell*
     Cb: CodeBlock*
@@ -81,6 +84,7 @@ typedef char* UnusedPtr;
     Icf: InlineCallFrame*
     Idc: const Identifier*
     J: EncodedJSValue
+    Jaic: JITAddIC*
     Jcp: const JSValue*
     Jsc: JSScope*
     Jsf: JSFunction*
@@ -92,7 +96,6 @@ typedef char* UnusedPtr;
     Q: int64_t
     R: Register
     Reo: RegExpObject*
-    Rp: ResultProfile*
     S: size_t
     Sprt: SlowPathReturnType
     Ssi: StructureStubInfo*
@@ -133,7 +136,9 @@ typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJJ)(ExecState*, EncodedJ
 typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJAp)(ExecState*, EncodedJSValue, EncodedJSValue, ArrayProfile*);
 typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJBy)(ExecState*, EncodedJSValue, EncodedJSValue, ByValInfo*);
 typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJJ)(ExecState*, EncodedJSValue, EncodedJSValue, EncodedJSValue);
-typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJRp)(ExecState*, EncodedJSValue, EncodedJSValue, ResultProfile*);
+typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJArp)(ExecState*, EncodedJSValue, EncodedJSValue, ArithProfile*);
+typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJArpJaic)(ExecState*, EncodedJSValue, EncodedJSValue, ArithProfile*, JITAddIC*);
+typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJJaic)(ExecState*, EncodedJSValue, EncodedJSValue, JITAddIC*);
 typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJssZ)(ExecState*, JSString*, int32_t);
 typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJssReo)(ExecState*, JSString*, RegExpObject*);
 typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJssReoJss)(ExecState*, JSString*, RegExpObject*, JSString*);
@@ -408,11 +413,15 @@ EncodedJSValue JIT_OPERATION operationNextEnumeratorPname(ExecState*, JSCell*, i
 JSCell* JIT_OPERATION operationToIndexString(ExecState*, int32_t);
 
 EncodedJSValue JIT_OPERATION operationValueAdd(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2) WTF_INTERNAL;
-EncodedJSValue JIT_OPERATION operationValueAddProfiled(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ResultProfile*) WTF_INTERNAL;
+EncodedJSValue JIT_OPERATION operationValueAddProfiled(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile*) WTF_INTERNAL;
+EncodedJSValue JIT_OPERATION operationValueAddProfiledOptimize(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile*, JITAddIC*) WTF_INTERNAL;
+EncodedJSValue JIT_OPERATION operationValueAddProfiledNoOptimize(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile*, JITAddIC*) WTF_INTERNAL;
+EncodedJSValue JIT_OPERATION operationValueAddOptimize(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, JITAddIC*) WTF_INTERNAL;
+EncodedJSValue JIT_OPERATION operationValueAddNoOptimize(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, JITAddIC*) WTF_INTERNAL;
 EncodedJSValue JIT_OPERATION operationValueMul(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2) WTF_INTERNAL;
-EncodedJSValue JIT_OPERATION operationValueMulProfiled(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ResultProfile*) WTF_INTERNAL;
+EncodedJSValue JIT_OPERATION operationValueMulProfiled(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile*) WTF_INTERNAL;
 EncodedJSValue JIT_OPERATION operationValueSub(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2) WTF_INTERNAL;
-EncodedJSValue JIT_OPERATION operationValueSubProfiled(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ResultProfile*) WTF_INTERNAL;
+EncodedJSValue JIT_OPERATION operationValueSubProfiled(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile*) WTF_INTERNAL;
 
 void JIT_OPERATION operationProcessTypeProfilerLog(ExecState*) WTF_INTERNAL;
 void JIT_OPERATION operationProcessShadowChickenLog(ExecState*) WTF_INTERNAL;
index 4cfed8f..4733a8d 100644 (file)
@@ -26,6 +26,8 @@
 #include "config.h"
 #include "JITSubGenerator.h"
 
+#include "ArithProfile.h"
+
 #if ENABLE(JIT)
 
 namespace JSC {
@@ -83,8 +85,8 @@ void JITSubGenerator::generateFastPath(CCallHelpers& jit)
     rightWasInteger.link(&jit);
 
     jit.subDouble(m_rightFPR, m_leftFPR);
-    if (m_resultProfile)
-        m_resultProfile->emitSetDouble(jit);
+    if (m_arithProfile)
+        m_arithProfile->emitSetDouble(jit);
 
     jit.boxDouble(m_leftFPR, m_result);
 }
index 9dd78ef..98bda5d 100644 (file)
@@ -38,7 +38,7 @@ public:
     JITSubGenerator(SnippetOperand leftOperand, SnippetOperand rightOperand,
         JSValueRegs result, JSValueRegs left, JSValueRegs right,
         FPRReg leftFPR, FPRReg rightFPR, GPRReg scratchGPR, FPRReg scratchFPR,
-        ResultProfile* resultProfile = nullptr)
+        ArithProfile* arithProfile = nullptr)
         : m_leftOperand(leftOperand)
         , m_rightOperand(rightOperand)
         , m_result(result)
@@ -48,7 +48,7 @@ public:
         , m_rightFPR(rightFPR)
         , m_scratchGPR(scratchGPR)
         , m_scratchFPR(scratchFPR)
-        , m_resultProfile(resultProfile)
+        , m_arithProfile(arithProfile)
     { }
 
     void generateFastPath(CCallHelpers&);
@@ -67,7 +67,7 @@ private:
     FPRReg m_rightFPR;
     GPRReg m_scratchGPR;
     FPRReg m_scratchFPR;
-    ResultProfile* m_resultProfile;
+    ArithProfile* m_arithProfile;
     bool m_didEmitFastPath { false };
 
     CCallHelpers::JumpList m_endJumpList;
index aacd457..d45939c 100644 (file)
@@ -72,7 +72,7 @@ static FunctionPtr readCallTarget(CodeBlock* codeBlock, CodeLocationCall call)
     return result;
 }
 
-static void repatchCall(CodeBlock* codeBlock, CodeLocationCall call, FunctionPtr newCalleeFunction)
+void ftlThunkAwareRepatchCall(CodeBlock* codeBlock, CodeLocationCall call, FunctionPtr newCalleeFunction)
 {
 #if ENABLE(FTL_JIT)
     if (codeBlock->jitType() == JITCode::FTLJIT) {
@@ -167,7 +167,7 @@ static InlineCacheAction tryCacheGetByID(ExecState* exec, JSValue baseValue, con
 
                 bool generatedCodeInline = InlineAccess::generateArrayLength(*codeBlock->vm(), stubInfo, jsCast<JSArray*>(baseValue));
                 if (generatedCodeInline) {
-                    repatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingGetByIdFunction(kind));
+                    ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingGetByIdFunction(kind));
                     stubInfo.initArrayLength();
                     return RetryCacheLater;
                 }
@@ -220,7 +220,7 @@ static InlineCacheAction tryCacheGetByID(ExecState* exec, JSValue baseValue, con
             if (generatedCodeInline) {
                 LOG_IC((ICEvent::GetByIdSelfPatch, structure->classInfo(), propertyName));
                 structure->startWatchingPropertyForReplacements(vm, slot.cachedOffset());
-                repatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingGetByIdFunction(kind));
+                ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingGetByIdFunction(kind));
                 stubInfo.initGetByIdSelf(codeBlock, structure, slot.cachedOffset());
                 return RetryCacheLater;
             }
@@ -314,7 +314,7 @@ void repatchGetByID(ExecState* exec, JSValue baseValue, const Identifier& proper
     GCSafeConcurrentJITLocker locker(exec->codeBlock()->m_lock, exec->vm().heap);
     
     if (tryCacheGetByID(exec, baseValue, propertyName, slot, stubInfo, kind) == GiveUpOnCache)
-        repatchCall(exec->codeBlock(), stubInfo.slowPathCallLocation(), appropriateGenericGetByIdFunction(kind));
+        ftlThunkAwareRepatchCall(exec->codeBlock(), stubInfo.slowPathCallLocation(), appropriateGenericGetByIdFunction(kind));
 }
 
 static V_JITOperation_ESsiJJI appropriateGenericPutByIdFunction(const PutPropertySlot &slot, PutKind putKind)
@@ -372,7 +372,7 @@ static InlineCacheAction tryCachePutByID(ExecState* exec, JSValue baseValue, Str
                 bool generatedCodeInline = InlineAccess::generateSelfPropertyReplace(vm, stubInfo, structure, slot.cachedOffset());
                 if (generatedCodeInline) {
                     LOG_IC((ICEvent::PutByIdSelfPatch, structure->classInfo(), ident));
-                    repatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingPutByIdFunction(slot, putKind));
+                    ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingPutByIdFunction(slot, putKind));
                     stubInfo.initPutByIdReplace(codeBlock, structure, slot.cachedOffset());
                     return RetryCacheLater;
                 }
@@ -468,7 +468,7 @@ void repatchPutByID(ExecState* exec, JSValue baseValue, Structure* structure, co
     GCSafeConcurrentJITLocker locker(exec->codeBlock()->m_lock, exec->vm().heap);
     
     if (tryCachePutByID(exec, baseValue, structure, propertyName, slot, stubInfo, putKind) == GiveUpOnCache)
-        repatchCall(exec->codeBlock(), stubInfo.slowPathCallLocation(), appropriateGenericPutByIdFunction(slot, putKind));
+        ftlThunkAwareRepatchCall(exec->codeBlock(), stubInfo.slowPathCallLocation(), appropriateGenericPutByIdFunction(slot, putKind));
 }
 
 static InlineCacheAction tryRepatchIn(
@@ -529,7 +529,7 @@ void repatchIn(
 {
     SuperSamplerScope superSamplerScope(false);
     if (tryRepatchIn(exec, base, ident, wasFound, slot, stubInfo) == GiveUpOnCache)
-        repatchCall(exec->codeBlock(), stubInfo.slowPathCallLocation(), operationIn);
+        ftlThunkAwareRepatchCall(exec->codeBlock(), stubInfo.slowPathCallLocation(), operationIn);
 }
 
 static void linkSlowFor(VM*, CallLinkInfo& callLinkInfo, MacroAssemblerCodeRef codeRef)
@@ -901,7 +901,7 @@ void linkPolymorphicCall(
 
 void resetGetByID(CodeBlock* codeBlock, StructureStubInfo& stubInfo, GetByIDKind kind)
 {
-    repatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingGetByIdFunction(kind));
+    ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingGetByIdFunction(kind));
     InlineAccess::rewireStubAsJump(*codeBlock->vm(), stubInfo, stubInfo.slowPathStartLocation());
 }
 
@@ -920,7 +920,7 @@ void resetPutByID(CodeBlock* codeBlock, StructureStubInfo& stubInfo)
         optimizedFunction = operationPutByIdDirectNonStrictOptimize;
     }
 
-    repatchCall(codeBlock, stubInfo.slowPathCallLocation(), optimizedFunction);
+    ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation(), optimizedFunction);
     InlineAccess::rewireStubAsJump(*codeBlock->vm(), stubInfo, stubInfo.slowPathStartLocation());
 }
 
index 204762d..0453be7 100644 (file)
@@ -54,6 +54,7 @@ void linkPolymorphicCall(ExecState*, CallLinkInfo&, CallVariant);
 void resetGetByID(CodeBlock*, StructureStubInfo&, GetByIDKind);
 void resetPutByID(CodeBlock*, StructureStubInfo&);
 void resetIn(CodeBlock*, StructureStubInfo&);
+void ftlThunkAwareRepatchCall(CodeBlock*, CodeLocationCall, FunctionPtr newCalleeFunction);
 
 } // namespace JSC
 
index 5d71b77..ea79e33 100644 (file)
@@ -26,6 +26,7 @@
 #include "config.h"
 #include "LLIntData.h"
 
+#include "ArithProfile.h"
 #include "BytecodeConventions.h"
 #include "CodeBlock.h"
 #include "CodeType.h"
@@ -223,6 +224,47 @@ void Data::performAssertions(VM& vm)
 #endif
 
     ASSERT(StringImpl::s_hashFlag8BitBuffer == 8);
+
+    {
+        uint32_t bits = 0x120000;
+        UNUSED_PARAM(bits);
+        ArithProfile arithProfile;
+        arithProfile.lhsSawInt32();
+        arithProfile.rhsSawInt32();
+        ASSERT(arithProfile.bits() == bits);
+        ASSERT(ArithProfile::fromInt(bits).lhsObservedType().isOnlyInt32());
+        ASSERT(ArithProfile::fromInt(bits).rhsObservedType().isOnlyInt32());
+    }
+    {
+        uint32_t bits = 0x220000;
+        UNUSED_PARAM(bits);
+        ArithProfile arithProfile;
+        arithProfile.lhsSawNumber();
+        arithProfile.rhsSawInt32();
+        ASSERT(arithProfile.bits() == bits);
+        ASSERT(ArithProfile::fromInt(bits).lhsObservedType().isOnlyNumber());
+        ASSERT(ArithProfile::fromInt(bits).rhsObservedType().isOnlyInt32());
+    }
+    {
+        uint32_t bits = 0x240000;
+        UNUSED_PARAM(bits);
+        ArithProfile arithProfile;
+        arithProfile.lhsSawNumber();
+        arithProfile.rhsSawNumber();
+        ASSERT(arithProfile.bits() == bits);
+        ASSERT(ArithProfile::fromInt(bits).lhsObservedType().isOnlyNumber());
+        ASSERT(ArithProfile::fromInt(bits).rhsObservedType().isOnlyNumber());
+    }
+    {
+        uint32_t bits = 0x140000;
+        UNUSED_PARAM(bits);
+        ArithProfile arithProfile;
+        arithProfile.lhsSawInt32();
+        arithProfile.rhsSawNumber();
+        ASSERT(arithProfile.bits() == bits);
+        ASSERT(ArithProfile::fromInt(bits).lhsObservedType().isOnlyInt32());
+        ASSERT(ArithProfile::fromInt(bits).rhsObservedType().isOnlyNumber());
+    }
 }
 #if COMPILER(CLANG)
 #pragma clang diagnostic pop
index 3e0dbd8..32b50c0 100644 (file)
@@ -252,6 +252,12 @@ const IsInvalidated = 2
 # ShadowChicken data
 const ShadowChickenTailMarker = 0x7a11
 
+# ArithProfile data
+const ArithProfileIntInt = 0x120000
+const ArithProfileNumberInt = 0x220000
+const ArithProfileNumberNumber = 0x240000
+const ArithProfileIntNumber = 0x140000
+
 # Some register conventions.
 if JSVALUE64
     # - Use a pair of registers to represent the PC: one register for the
@@ -286,6 +292,10 @@ if JSVALUE64
         loadp offset * 8[PB, PC, 8], dest
     end
     
+    macro storeisToInstruction(value, offset)
+        storei value, offset * 8[PB, PC, 8]
+    end
+
     macro storepToInstruction(value, offset)
         storep value, offset * 8[PB, PC, 8]
     end
@@ -299,6 +309,10 @@ else
     macro loadpFromInstruction(offset, dest)
         loadp offset * 4[PC], dest
     end
+
+    macro storeisToInstruction(value, offset)
+        storei value, offset * 4[PC]
+    end
 end
 
 if X86_64_WIN
index 803de4c..ba59f83 100644 (file)
@@ -1014,6 +1014,9 @@ macro binaryOpCustomStore(integerOperationAndStore, doubleOperation, slowPath)
     loadConstantOrVariable2Reg(t0, t2, t0)
     bineq t2, Int32Tag, .op1NotInt
     bineq t3, Int32Tag, .op2NotInt
+    loadisFromInstruction(4, t5)
+    ori ArithProfileIntInt, t5
+    storeisToInstruction(t5, 4)
     loadi 4[PC], t2
     integerOperationAndStore(t3, t1, t0, .slow, t2)
     dispatch(5)
@@ -1023,10 +1026,16 @@ macro binaryOpCustomStore(integerOperationAndStore, doubleOperation, slowPath)
     bia t2, LowestTag, .slow
     bib t3, LowestTag, .op1NotIntOp2Double
     bineq t3, Int32Tag, .slow
+    loadisFromInstruction(4, t5)
+    ori ArithProfileNumberInt, t5
+    storeisToInstruction(t5, 4)
     ci2d t1, ft1
     jmp .op1NotIntReady
 .op1NotIntOp2Double:
     fii2d t1, t3, ft1
+    loadisFromInstruction(4, t5)
+    ori ArithProfileNumberNumber, t5
+    storeisToInstruction(t5, 4)
 .op1NotIntReady:
     loadi 4[PC], t1
     fii2d t0, t2, ft0
@@ -1038,6 +1047,9 @@ macro binaryOpCustomStore(integerOperationAndStore, doubleOperation, slowPath)
     # First operand is definitely an int, the second operand is definitely not.
     loadi 4[PC], t2
     bia t3, LowestTag, .slow
+    loadisFromInstruction(4, t5)
+    ori ArithProfileIntNumber, t5
+    storeisToInstruction(t5, 4)
     ci2d t0, ft0
     fii2d t1, t3, ft1
     doubleOperation(ft1, ft0)
index 79bb4e3..9ec57c8 100644 (file)
@@ -893,6 +893,9 @@ macro binaryOpCustomStore(integerOperationAndStore, doubleOperation, slowPath)
     bqb t1, tagTypeNumber, .op2NotInt
     loadisFromInstruction(1, t2)
     integerOperationAndStore(t1, t0, .slow, t2)
+    loadisFromInstruction(4, t1)
+    ori ArithProfileIntInt, t1
+    storeisToInstruction(t1, 4)
     dispatch(5)
 
 .op1NotInt:
@@ -902,8 +905,14 @@ macro binaryOpCustomStore(integerOperationAndStore, doubleOperation, slowPath)
     btqz t1, tagTypeNumber, .slow
     addq tagTypeNumber, t1
     fq2d t1, ft1
+    loadisFromInstruction(4, t2)
+    ori ArithProfileNumberNumber, t2
+    storeisToInstruction(t2, 4)
     jmp .op1NotIntReady
 .op1NotIntOp2Int:
+    loadisFromInstruction(4, t2)
+    ori ArithProfileNumberInt, t2
+    storeisToInstruction(t2, 4)
     ci2d t1, ft1
 .op1NotIntReady:
     loadisFromInstruction(1, t2)
@@ -919,6 +928,9 @@ macro binaryOpCustomStore(integerOperationAndStore, doubleOperation, slowPath)
     # First operand is definitely an int, the second is definitely not.
     loadisFromInstruction(1, t2)
     btqz t1, tagTypeNumber, .slow
+    loadisFromInstruction(4, t3)
+    ori ArithProfileIntNumber, t3
+    storeisToInstruction(t3, 4)
     ci2d t0, ft0
     addq tagTypeNumber, t1
     fq2d t1, ft1
index a975fdd..7e53c7d 100644 (file)
@@ -32,46 +32,48 @@ namespace JSC {
     private:
         friend struct OperandTypes;
 
-        typedef char Type;
+        typedef uint8_t Type;
         static const Type TypeInt32 = 1;
-        
-        static const Type TypeMaybeNumber = 0x04;
-        static const Type TypeMaybeString = 0x08;
-        static const Type TypeMaybeNull   = 0x10;
-        static const Type TypeMaybeBool   = 0x20;
-        static const Type TypeMaybeOther  = 0x40;
+        static const Type TypeMaybeNumber = 0x02;
+        static const Type TypeMaybeString = 0x04;
+        static const Type TypeMaybeNull   = 0x08;
+        static const Type TypeMaybeBool   = 0x10;
+        static const Type TypeMaybeOther  = 0x20;
 
         static const Type TypeBits = TypeMaybeNumber | TypeMaybeString | TypeMaybeNull | TypeMaybeBool | TypeMaybeOther;
 
+    public:
+        static const int numBitsNeeded = 6;
+        static_assert((TypeBits & ((1 << numBitsNeeded) - 1)) == TypeBits, "This is necessary for correctness.");
+
         explicit ResultType(Type type)
-            : m_type(type)
+            : m_bits(type)
         {
         }
 
-    public:
         bool isInt32() const
         {
-            return m_type & TypeInt32;
+            return m_bits & TypeInt32;
         }
 
         bool definitelyIsNumber() const
         {
-            return (m_type & TypeBits) == TypeMaybeNumber;
+            return (m_bits & TypeBits) == TypeMaybeNumber;
         }
         
         bool definitelyIsString() const
         {
-            return (m_type & TypeBits) == TypeMaybeString;
+            return (m_bits & TypeBits) == TypeMaybeString;
         }
 
         bool definitelyIsBoolean() const
         {
-            return (m_type & TypeBits) == TypeMaybeBool;
+            return (m_bits & TypeBits) == TypeMaybeBool;
         }
 
         bool mightBeNumber() const
         {
-            return m_type & TypeMaybeNumber;
+            return m_bits & TypeMaybeNumber;
         }
 
         bool isNotNumber() const
@@ -141,8 +143,10 @@ namespace JSC {
             return numberTypeIsInt32();
         }
 
+        Type bits() const { return m_bits; }
+
     private:
-        Type m_type;
+        Type m_bits;
     };
     
     struct OperandTypes
@@ -152,8 +156,8 @@ namespace JSC {
             // We have to initialize one of the int to ensure that
             // the entire struct is initialized.
             m_u.i = 0;
-            m_u.rds.first = first.m_type;
-            m_u.rds.second = second.m_type;
+            m_u.rds.first = first.m_bits;
+            m_u.rds.second = second.m_bits;
         }
         
         union {
index 65fda68..4593b06 100644 (file)
@@ -26,6 +26,7 @@
 #include "config.h"
 #include "CommonSlowPaths.h"
 
+#include "ArithProfile.h"
 #include "ArrayConstructor.h"
 #include "BuiltinNames.h"
 #include "CallFrame.h"
@@ -359,22 +360,21 @@ SLOW_PATH_DECL(slow_path_negate)
 }
 
 #if ENABLE(DFG_JIT)
-static void updateResultProfileForBinaryArithOp(ExecState* exec, Instruction* pc, JSValue result, JSValue left, JSValue right)
+static void updateArithProfileForBinaryArithOp(ExecState* exec, Instruction* pc, JSValue result, JSValue left, JSValue right)
 {
     CodeBlock* codeBlock = exec->codeBlock();
-    unsigned bytecodeOffset = codeBlock->bytecodeOffset(pc);
-    ResultProfile* profile = codeBlock->ensureResultProfile(bytecodeOffset);
+    ArithProfile& profile = codeBlock->arithProfileForPC(pc);
 
     if (result.isNumber()) {
         if (!result.isInt32()) {
             if (left.isInt32() && right.isInt32())
-                profile->setObservedInt32Overflow();
+                profile.setObservedInt32Overflow();
 
             double doubleVal = result.asNumber();
             if (!doubleVal && std::signbit(doubleVal))
-                profile->setObservedNegZeroDouble();
+                profile.setObservedNegZeroDouble();
             else {
-                profile->setObservedNonNegZeroDouble();
+                profile.setObservedNonNegZeroDouble();
 
                 // The Int52 overflow check here intentionally omits 1ll << 51 as a valid negative Int52 value.
                 // Therefore, we will get a false positive if the result is that value. This is intentionally
@@ -382,14 +382,14 @@ static void updateResultProfileForBinaryArithOp(ExecState* exec, Instruction* pc
                 static const int64_t int52OverflowPoint = (1ll << 51);
                 int64_t int64Val = static_cast<int64_t>(std::abs(doubleVal));
                 if (int64Val >= int52OverflowPoint)
-                    profile->setObservedInt52Overflow();
+                    profile.setObservedInt52Overflow();
             }
         }
     } else
-        profile->setObservedNonNumber();
+        profile.setObservedNonNumber();
 }
 #else
-static void updateResultProfileForBinaryArithOp(ExecState*, Instruction*, JSValue, JSValue, JSValue) { }
+static void updateArithProfileForBinaryArithOp(ExecState*, Instruction*, JSValue, JSValue, JSValue) { }
 #endif
 
 SLOW_PATH_DECL(slow_path_to_number)
@@ -407,6 +407,9 @@ SLOW_PATH_DECL(slow_path_add)
     JSValue v2 = OP_C(3).jsValue();
     JSValue result;
 
+    ArithProfile& arithProfile = exec->codeBlock()->arithProfileForPC(pc);
+    arithProfile.observeLHSAndRHS(v1, v2);
+
     if (v1.isString() && !v2.isObject())
         result = jsString(exec, asString(v1), v2.toString(exec));
     else if (v1.isNumber() && v2.isNumber())
@@ -415,7 +418,7 @@ SLOW_PATH_DECL(slow_path_add)
         result = jsAddSlowCase(exec, v1, v2);
 
     RETURN_WITH_PROFILING(result, {
-        updateResultProfileForBinaryArithOp(exec, pc, result, v1, v2);
+        updateArithProfileForBinaryArithOp(exec, pc, result, v1, v2);
     });
 }
 
@@ -432,7 +435,7 @@ SLOW_PATH_DECL(slow_path_mul)
     double b = right.toNumber(exec);
     JSValue result = jsNumber(a * b);
     RETURN_WITH_PROFILING(result, {
-        updateResultProfileForBinaryArithOp(exec, pc, result, left, right);
+        updateArithProfileForBinaryArithOp(exec, pc, result, left, right);
     });
 }
 
@@ -445,7 +448,7 @@ SLOW_PATH_DECL(slow_path_sub)
     double b = right.toNumber(exec);
     JSValue result = jsNumber(a - b);
     RETURN_WITH_PROFILING(result, {
-        updateResultProfileForBinaryArithOp(exec, pc, result, left, right);
+        updateArithProfileForBinaryArithOp(exec, pc, result, left, right);
     });
 }
 
@@ -458,7 +461,7 @@ SLOW_PATH_DECL(slow_path_div)
     double b = right.toNumber(exec);
     JSValue result = jsNumber(a / b);
     RETURN_WITH_PROFILING(result, {
-        updateResultProfileForBinaryArithOp(exec, pc, result, left, right);
+        updateArithProfileForBinaryArithOp(exec, pc, result, left, right);
     });
 }
 
diff --git a/Source/JavaScriptCore/tests/stress/op-add-exceptions.js b/Source/JavaScriptCore/tests/stress/op-add-exceptions.js
new file mode 100644 (file)
index 0000000..fc1ccd6
--- /dev/null
@@ -0,0 +1,79 @@
+function assert(b) {
+    if (!b)
+        throw new Error("Bad!");
+}
+noInline(assert);
+
+function f1() { return "f1"; }
+noInline(f1);
+function f2() { return "f2"; }
+noInline(f2);
+function f3() { return "f3"; }
+noInline(f3);
+
+let oException = {
+    valueOf() { throw new Error(""); }
+};
+
+function foo(arg1, arg2) {
+    let a = f1();
+    let b = f2();
+    let c = f3();
+    try {
+        arg1 + arg2;
+    } catch(e) {
+        assert(arg1 === oException);
+        assert(arg2 === oException);
+    }
+    assert(a === "f1");
+    assert(b === "f2");
+    assert(c === "f3");
+}
+noInline(foo);
+
+for (let i = 0; i < 1000; i++) {
+    foo(i, {});
+    foo({}, i);
+}
+foo(oException, oException);
+for (let i = 0; i < 10000; i++) {
+    foo(i, {});
+    foo({}, i);
+}
+foo(oException, oException);
+
+
+function ident(x) { return x; }
+noInline(ident);
+
+function bar(arg1, arg2) {
+    let a = f1();
+    let b = f2();
+    let c = f3();
+    let x = ident(arg1);
+    let y = ident(arg2);
+
+    try {
+        arg1 + arg2;
+    } catch(e) {
+        assert(arg1 === oException);
+        assert(arg2 === oException);
+        assert(x === oException);
+        assert(y === oException);
+    }
+    assert(a === "f1");
+    assert(b === "f2");
+    assert(c === "f3");
+}
+noInline(bar);
+
+for (let i = 0; i < 1000; i++) {
+    bar(i, {});
+    bar({}, i);
+}
+bar(oException, oException);
+for (let i = 0; i < 10000; i++) {
+    bar(i, {});
+    bar({}, i);
+}
+bar(oException, oException);
index 38b56c4..c38f51f 100644 (file)
@@ -1,3 +1,12 @@
+2016-07-21  Saam Barati  <sbarati@apple.com>
+
+        op_add/ValueAdd should be an IC in all JIT tiers
+        https://bugs.webkit.org/show_bug.cgi?id=159649
+
+        Reviewed by Benjamin Poulain.
+
+        * ForwardingHeaders/jit/JITMathICForwards.h: Added.
+
 2016-07-21  Chris Dumez  <cdumez@apple.com>
 
         Make parameters mandatory for Document.create*() operations
diff --git a/Source/WebCore/ForwardingHeaders/jit/JITMathICForwards.h b/Source/WebCore/ForwardingHeaders/jit/JITMathICForwards.h
new file mode 100644 (file)
index 0000000..c0a9d3a
--- /dev/null
@@ -0,0 +1,4 @@
+#ifndef WebCore_FWD_JITMathICForwards_h
+#define WebCore_FWD_JITMathICForwards_h
+#include <JavaScriptCore/JITMathICForwards.h>
+#endif