https://bugs.webkit.org/show_bug.cgi?id=145092
Reviewed by Filip Pizlo.
When we have a hot loop without OSR Entry inside a slower loop that support OSR Entry,
we get the inside loop driving the tierUpCounter and we have very little chance of
doing a CheckTierUp on the outer loop. In turn, this give almost no opportunity to tier
up in the outer loop and OSR Enter there.
This patches changes CheckTierUp to force its outer loops to do a CheckTierUp themselves.
To do that, CheckTierUp sets a flag "nestedTriggerIsSet" to force the outer loop to
enter their CheckTierUp regardless of the tier-up counter.
* bytecode/ExecutionCounter.cpp:
(JSC::ExecutionCounter<countingVariant>::setThreshold):
This is somewhat unrelated. This assertion is incorrect because it relies on
m_counter, which changes on an other thread.
I have hit it a couple of times with this patch because we are a bit more aggressive
on CheckTierUp. What happens is:
1) ExecutionCounter<countingVariant>::checkIfThresholdCrossedAndSet() first checks
hasCrossedThreshold(), and it is false.
2) On the main thread, the hot loops keeps running and the counter becomes large
enough to cross the threshold.
3) ExecutionCounter<countingVariant>::checkIfThresholdCrossedAndSet() runs the next
test, setThreshold(), where the assertion is. Since the counter is now large enough,
the assertion fails.
* dfg/DFGAbstractInterpreterInlines.h:
(JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects):
* dfg/DFGClobberize.h:
(JSC::DFG::clobberize):
* dfg/DFGDoesGC.cpp:
(JSC::DFG::doesGC):
* dfg/DFGFixupPhase.cpp:
(JSC::DFG::FixupPhase::fixupNode):
* dfg/DFGJITCode.h:
I used a uint8_t instead of a boolean to make the code generation clearer
in DFGSpeculativeJIT64.
* dfg/DFGNodeType.h:
* dfg/DFGOperations.cpp:
* dfg/DFGOperations.h:
* dfg/DFGPredictionPropagationPhase.cpp:
(JSC::DFG::PredictionPropagationPhase::propagate):
This is a bit annoying: we have the NaturalLoops analysis that provides us
everything we need to know about loops, but the TierUpCheck are conservative
and set on LoopHint.
To make the two work together, we first find all the CheckTierUp that cannot
OSR enter and we keep a list of all the natural loops containing them.
Then we do a second pass over the LoopHints, get their NaturalLoop, and check
if it contains a loop that cannot OSR enter.
* dfg/DFGSafeToExecute.h:
(JSC::DFG::safeToExecute):
* dfg/DFGSpeculativeJIT32_64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGSpeculativeJIT64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGTierUpCheckInjectionPhase.cpp:
(JSC::DFG::TierUpCheckInjectionPhase::run):
(JSC::DFG::TierUpCheckInjectionPhase::canOSREnterAtLoopHint):
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@184511
268f45cc-cd09-0410-ab3c-
d52691b4dbfc
+2015-05-18 Benjamin Poulain <benjamin@webkit.org>
+
+ [JSC] When entering a CheckTierUp without OSREntry, force the CheckTierUp for the outer loops with OSR Entry
+ https://bugs.webkit.org/show_bug.cgi?id=145092
+
+ Reviewed by Filip Pizlo.
+
+ When we have a hot loop without OSR Entry inside a slower loop that support OSR Entry,
+ we get the inside loop driving the tierUpCounter and we have very little chance of
+ doing a CheckTierUp on the outer loop. In turn, this give almost no opportunity to tier
+ up in the outer loop and OSR Enter there.
+
+ This patches changes CheckTierUp to force its outer loops to do a CheckTierUp themselves.
+
+ To do that, CheckTierUp sets a flag "nestedTriggerIsSet" to force the outer loop to
+ enter their CheckTierUp regardless of the tier-up counter.
+
+ * bytecode/ExecutionCounter.cpp:
+ (JSC::ExecutionCounter<countingVariant>::setThreshold):
+ This is somewhat unrelated. This assertion is incorrect because it relies on
+ m_counter, which changes on an other thread.
+
+ I have hit it a couple of times with this patch because we are a bit more aggressive
+ on CheckTierUp. What happens is:
+ 1) ExecutionCounter<countingVariant>::checkIfThresholdCrossedAndSet() first checks
+ hasCrossedThreshold(), and it is false.
+ 2) On the main thread, the hot loops keeps running and the counter becomes large
+ enough to cross the threshold.
+ 3) ExecutionCounter<countingVariant>::checkIfThresholdCrossedAndSet() runs the next
+ test, setThreshold(), where the assertion is. Since the counter is now large enough,
+ the assertion fails.
+
+ * dfg/DFGAbstractInterpreterInlines.h:
+ (JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects):
+ * dfg/DFGClobberize.h:
+ (JSC::DFG::clobberize):
+ * dfg/DFGDoesGC.cpp:
+ (JSC::DFG::doesGC):
+ * dfg/DFGFixupPhase.cpp:
+ (JSC::DFG::FixupPhase::fixupNode):
+
+ * dfg/DFGJITCode.h:
+ I used a uint8_t instead of a boolean to make the code generation clearer
+ in DFGSpeculativeJIT64.
+
+ * dfg/DFGNodeType.h:
+ * dfg/DFGOperations.cpp:
+ * dfg/DFGOperations.h:
+
+ * dfg/DFGPredictionPropagationPhase.cpp:
+ (JSC::DFG::PredictionPropagationPhase::propagate):
+ This is a bit annoying: we have the NaturalLoops analysis that provides us
+ everything we need to know about loops, but the TierUpCheck are conservative
+ and set on LoopHint.
+
+ To make the two work together, we first find all the CheckTierUp that cannot
+ OSR enter and we keep a list of all the natural loops containing them.
+
+ Then we do a second pass over the LoopHints, get their NaturalLoop, and check
+ if it contains a loop that cannot OSR enter.
+
+ * dfg/DFGSafeToExecute.h:
+ (JSC::DFG::safeToExecute):
+ * dfg/DFGSpeculativeJIT32_64.cpp:
+ (JSC::DFG::SpeculativeJIT::compile):
+ * dfg/DFGSpeculativeJIT64.cpp:
+ (JSC::DFG::SpeculativeJIT::compile):
+ * dfg/DFGTierUpCheckInjectionPhase.cpp:
+ (JSC::DFG::TierUpCheckInjectionPhase::run):
+ (JSC::DFG::TierUpCheckInjectionPhase::canOSREnterAtLoopHint):
+
2015-05-18 Filip Pizlo <fpizlo@apple.com>
Add a Int-or-Boolean speculation to Branch
return false;
}
- ASSERT(!m_activeThreshold || !hasCrossedThreshold(codeBlock));
-
// Compute the true total count.
double trueTotalCount = count();
}
case CheckTierUpAndOSREnter:
+ case CheckTierUpWithNestedTriggerAndOSREnter:
case LoopHint:
case ZombieHint:
break;
case CheckTierUpInLoop:
case CheckTierUpAtReturn:
case CheckTierUpAndOSREnter:
+ case CheckTierUpWithNestedTriggerAndOSREnter:
case LoopHint:
case Breakpoint:
case ProfileWillCall:
case CheckTierUpInLoop:
case CheckTierUpAtReturn:
case CheckTierUpAndOSREnter:
+ case CheckTierUpWithNestedTriggerAndOSREnter:
case LoopHint:
case StoreBarrier:
case InvalidationPoint:
case CheckTierUpInLoop:
case CheckTierUpAtReturn:
case CheckTierUpAndOSREnter:
+ case CheckTierUpWithNestedTriggerAndOSREnter:
case InvalidationPoint:
case CheckArray:
case CheckInBounds:
DFG::VariableEventStream variableEventStream;
DFG::MinifiedGraph minifiedDFG;
#if ENABLE(FTL_JIT)
+ uint8_t nestedTriggerIsSet { 0 };
UpperTierExecutionCounter tierUpCounter;
RefPtr<CodeBlock> osrEntryBlock;
unsigned osrEntryRetry;
/* Tier-up checks from the DFG to the FTL. */\
macro(CheckTierUpInLoop, NodeMustGenerate) \
macro(CheckTierUpAndOSREnter, NodeMustGenerate) \
+ macro(CheckTierUpWithNestedTriggerAndOSREnter, NodeMustGenerate) \
macro(CheckTierUpAtReturn, NodeMustGenerate) \
\
/* Get the value of a local variable, without linking into the VariableAccessData */\
Operands<JSValue>(), ToFTLDeferredCompilationCallback::create(codeBlock));
}
-void JIT_OPERATION triggerTierUpNow(ExecState* exec)
+static void triggerTierUpNowCommon(ExecState* exec, bool inLoop)
{
VM* vm = &exec->vm();
NativeCallFrameTracer tracer(vm, exec);
*codeBlock, ": Entered triggerTierUpNow with executeCounter = ",
jitCode->tierUpCounter, "\n");
}
-
+ if (inLoop)
+ jitCode->nestedTriggerIsSet = 1;
+
triggerFTLReplacementCompile(vm, codeBlock, jitCode);
}
+void JIT_OPERATION triggerTierUpNow(ExecState* exec)
+{
+ triggerTierUpNowCommon(exec, false);
+}
+
+void JIT_OPERATION triggerTierUpNowInLoop(ExecState* exec)
+{
+ triggerTierUpNowCommon(exec, true);
+}
+
char* JIT_OPERATION triggerOSREntryNow(
ExecState* exec, int32_t bytecodeIndex, int32_t streamIndex)
{
}
JITCode* jitCode = codeBlock->jitCode()->dfg();
+ jitCode->nestedTriggerIsSet = 0;
if (Options::verboseOSR()) {
dataLog(
#if ENABLE(FTL_JIT)
void JIT_OPERATION triggerTierUpNow(ExecState*) WTF_INTERNAL;
+void JIT_OPERATION triggerTierUpNowInLoop(ExecState*) WTF_INTERNAL;
char* JIT_OPERATION triggerOSREntryNow(ExecState*, int32_t bytecodeIndex, int32_t streamIndex) WTF_INTERNAL;
#endif // ENABLE(FTL_JIT)
case CheckTierUpInLoop:
case CheckTierUpAtReturn:
case CheckTierUpAndOSREnter:
+ case CheckTierUpWithNestedTriggerAndOSREnter:
case InvalidationPoint:
case CheckInBounds:
case ValueToInt32:
case CheckTierUpInLoop:
case CheckTierUpAtReturn:
case CheckTierUpAndOSREnter:
+ case CheckTierUpWithNestedTriggerAndOSREnter:
case LoopHint:
case StoreBarrier:
case InvalidationPoint:
case CheckTierUpInLoop:
case CheckTierUpAtReturn:
case CheckTierUpAndOSREnter:
+ case CheckTierUpWithNestedTriggerAndOSREnter:
case Int52Rep:
case FiatInt52:
case Int52Constant:
silentSpillAllRegisters(InvalidGPRReg);
m_jit.setupArgumentsExecState();
- appendCall(triggerTierUpNow);
+ appendCall(triggerTierUpNowInLoop);
silentFillAllRegisters(InvalidGPRReg);
done.link(&m_jit);
break;
}
- case CheckTierUpAndOSREnter: {
+ case CheckTierUpAndOSREnter:
+ case CheckTierUpWithNestedTriggerAndOSREnter: {
ASSERT(!node->origin.semantic.inlineCallFrame);
GPRTemporary temp(this);
GPRReg tempGPR = temp.gpr();
+
+ MacroAssembler::Jump forceOSREntry;
+ if (op == CheckTierUpWithNestedTriggerAndOSREnter)
+ forceOSREntry = m_jit.branchTest8(MacroAssembler::NonZero, MacroAssembler::AbsoluteAddress(&m_jit.jitCode()->nestedTriggerIsSet));
MacroAssembler::Jump done = m_jit.branchAdd32(
MacroAssembler::Signed,
TrustedImm32(Options::ftlTierUpCounterIncrementForLoop()),
MacroAssembler::AbsoluteAddress(&m_jit.jitCode()->tierUpCounter.m_counter));
-
+
+ if (forceOSREntry.isSet())
+ forceOSREntry.link(&m_jit);
silentSpillAllRegisters(tempGPR);
m_jit.setupArgumentsWithExecState(
TrustedImm32(node->origin.semantic.bytecodeIndex),
case CheckTierUpInLoop:
case CheckTierUpAtReturn:
case CheckTierUpAndOSREnter:
+ case CheckTierUpWithNestedTriggerAndOSREnter:
DFG_CRASH(m_jit.graph(), node, "Unexpected tier-up node");
break;
#endif // ENABLE(FTL_JIT)
if (!Options::enableOSREntryToFTL())
level = FTL::CanCompile;
+
+ // First we find all the loops that contain a LoopHint for which we cannot OSR enter.
+ // We use that information to decide if we need CheckTierUpAndOSREnter or CheckTierUpWithNestedTriggerAndOSREnter.
+ NaturalLoops& naturalLoops = m_graph.m_naturalLoops;
+ naturalLoops.computeIfNecessary(m_graph);
+
+ HashSet<const NaturalLoop*> loopsContainingLoopHintWithoutOSREnter = findLoopsContainingLoopHintWithoutOSREnter(naturalLoops, level);
InsertionSet insertionSet(m_graph);
for (BlockIndex blockIndex = m_graph.numBlocks(); blockIndex--;) {
Node* node = block->at(nodeIndex);
if (node->op() != LoopHint)
continue;
-
- // We only put OSR checks for the first LoopHint in the block. Note that
- // more than one LoopHint could happen in cases where we did a lot of CFG
- // simplification in the bytecode parser, but it should be very rare.
-
+
NodeOrigin origin = node->origin;
-
- if (level != FTL::CanCompileAndOSREnter || origin.semantic.inlineCallFrame) {
- insertionSet.insertNode(
- nodeIndex + 1, SpecNone, CheckTierUpInLoop, origin);
- break;
- }
-
- bool isAtTop = true;
- for (unsigned subNodeIndex = nodeIndex; subNodeIndex--;) {
- if (!block->at(subNodeIndex)->isSemanticallySkippable()) {
- isAtTop = false;
- break;
- }
- }
-
- if (!isAtTop) {
- insertionSet.insertNode(
- nodeIndex + 1, SpecNone, CheckTierUpInLoop, origin);
- break;
- }
-
- insertionSet.insertNode(
- nodeIndex + 1, SpecNone, CheckTierUpAndOSREnter, origin);
+ if (canOSREnterAtLoopHint(level, block, nodeIndex)) {
+ const NaturalLoop* loop = naturalLoops.innerMostLoopOf(block);
+ if (loop && loopsContainingLoopHintWithoutOSREnter.contains(loop))
+ insertionSet.insertNode(nodeIndex + 1, SpecNone, CheckTierUpWithNestedTriggerAndOSREnter, origin);
+ else
+ insertionSet.insertNode(nodeIndex + 1, SpecNone, CheckTierUpAndOSREnter, origin);
+ } else
+ insertionSet.insertNode(nodeIndex + 1, SpecNone, CheckTierUpInLoop, origin);
break;
}
return false;
#endif // ENABLE(FTL_JIT)
}
+
+private:
+#if ENABLE(FTL_JIT)
+ bool canOSREnterAtLoopHint(FTL::CapabilityLevel level, const BasicBlock* block, unsigned nodeIndex)
+ {
+ Node* node = block->at(nodeIndex);
+ ASSERT(node->op() == LoopHint);
+
+ NodeOrigin origin = node->origin;
+ if (level != FTL::CanCompileAndOSREnter || origin.semantic.inlineCallFrame)
+ return false;
+
+ // We only put OSR checks for the first LoopHint in the block. Note that
+ // more than one LoopHint could happen in cases where we did a lot of CFG
+ // simplification in the bytecode parser, but it should be very rare.
+ for (unsigned subNodeIndex = nodeIndex; subNodeIndex--;) {
+ if (!block->at(subNodeIndex)->isSemanticallySkippable())
+ return false;
+ }
+ return true;
+ }
+
+ HashSet<const NaturalLoop*> findLoopsContainingLoopHintWithoutOSREnter(const NaturalLoops& naturalLoops, FTL::CapabilityLevel level)
+ {
+ HashSet<const NaturalLoop*> loopsContainingLoopHintWithoutOSREnter;
+ for (BasicBlock* block : m_graph.blocksInNaturalOrder()) {
+ for (unsigned nodeIndex = 0; nodeIndex < block->size(); ++nodeIndex) {
+ Node* node = block->at(nodeIndex);
+ if (node->op() != LoopHint)
+ continue;
+
+ if (!canOSREnterAtLoopHint(level, block, nodeIndex)) {
+ const NaturalLoop* loop = naturalLoops.innerMostLoopOf(block);
+ while (loop) {
+ loopsContainingLoopHintWithoutOSREnter.add(loop);
+ loop = naturalLoops.innerMostOuterLoop(*loop);
+ }
+ }
+ }
+ }
+ return loopsContainingLoopHintWithoutOSREnter;
+ }
+#endif
};
bool performTierUpCheckInjection(Graph& graph)