The DFG should be able to tier-up and OSR enter into the FTL
authorfpizlo@apple.com <fpizlo@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 4 Sep 2013 06:26:04 +0000 (06:26 +0000)
committerfpizlo@apple.com <fpizlo@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 4 Sep 2013 06:26:04 +0000 (06:26 +0000)
https://bugs.webkit.org/show_bug.cgi?id=112838

Source/JavaScriptCore:

Reviewed by Mark Hahnenberg.

This adds the ability for the DFG to tier-up into the FTL. This works in both
of the expected tier-up modes:

Replacement: frequently called functions eventually have their entrypoint
replaced with one that goes into FTL-compiled code. Note, this will be a
slow-down for now since we don't yet have LLVM calling convention integration.

OSR entry: code stuck in hot loops gets OSR'd into the FTL from the DFG.

This means that if the DFG detects that a function is an FTL candidate, it
inserts execution counting code similar to the kind that the baseline JIT
would use. If you trip on a loop count in a loop header that is an OSR
candidate (it's not an inlined loop), we do OSR; otherwise we do replacement.
OSR almost always also implies future replacement.

OSR entry into the FTL is really cool. It uses a specialized FTL compile of
the code, where early in the DFG pipeline we replace the original root block
with an OSR entrypoint block that jumps to the pre-header of the hot loop.
The OSR entrypoint loads all live state at the loop pre-header using loads
from a scratch buffer, which gets populated by the runtime's OSR entry
preparation code (FTL::prepareOSREntry()). This approach appears to work well
with all of our subsequent optimizations, including prediction propagation,
CFA, and LICM. LLVM seems happy with it, too. Best of all, it works naturally
with concurrent compilation: when we hit the tier-up trigger we spawn a
compilation plan at the bytecode index from which we triggered; once the
compilation finishes the next trigger will try to enter, at that bytecode
index. If it can't - for example because the code has moved on to another
loop - then we just try again. Loops that get hot enough for OSR entry (about
25,000 iterations) will probably still be running when a concurrent compile
finishes, so this doesn't appear to be a big problem.

This immediately gives us a 70% speed-up on imaging-gaussian-blur. We could
get a bigger speed-up by adding some more intelligence and tweaking LLVM to
compile code faster. Those things will happen eventually but this is a good
start. Probably this code will see more tuning as we get more coverage in the
FTL JIT, but I'll worry about that in future patches.

* CMakeLists.txt:
* GNUmakefile.list.am:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Target.pri:
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::CodeBlock):
(JSC::CodeBlock::hasOptimizedReplacement):
(JSC::CodeBlock::setOptimizationThresholdBasedOnCompilationResult):
* bytecode/CodeBlock.h:
* dfg/DFGAbstractInterpreterInlines.h:
(JSC::DFG::::executeEffects):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::parseBlock):
(JSC::DFG::ByteCodeParser::parse):
* dfg/DFGCFGSimplificationPhase.cpp:
(JSC::DFG::CFGSimplificationPhase::run):
* dfg/DFGClobberize.h:
(JSC::DFG::clobberize):
* dfg/DFGDriver.cpp:
(JSC::DFG::compileImpl):
(JSC::DFG::compile):
* dfg/DFGDriver.h:
* dfg/DFGFixupPhase.cpp:
(JSC::DFG::FixupPhase::fixupNode):
* dfg/DFGGraph.cpp:
(JSC::DFG::Graph::dump):
(JSC::DFG::Graph::killBlockAndItsContents):
(JSC::DFG::Graph::killUnreachableBlocks):
* dfg/DFGGraph.h:
* dfg/DFGInPlaceAbstractState.cpp:
(JSC::DFG::InPlaceAbstractState::initialize):
* dfg/DFGJITCode.cpp:
(JSC::DFG::JITCode::reconstruct):
(JSC::DFG::JITCode::checkIfOptimizationThresholdReached):
(JSC::DFG::JITCode::optimizeNextInvocation):
(JSC::DFG::JITCode::dontOptimizeAnytimeSoon):
(JSC::DFG::JITCode::optimizeAfterWarmUp):
(JSC::DFG::JITCode::optimizeSoon):
(JSC::DFG::JITCode::forceOptimizationSlowPathConcurrently):
(JSC::DFG::JITCode::setOptimizationThresholdBasedOnCompilationResult):
* dfg/DFGJITCode.h:
* dfg/DFGJITFinalizer.cpp:
(JSC::DFG::JITFinalizer::finalize):
(JSC::DFG::JITFinalizer::finalizeFunction):
(JSC::DFG::JITFinalizer::finalizeCommon):
* dfg/DFGLoopPreHeaderCreationPhase.cpp:
(JSC::DFG::createPreHeader):
(JSC::DFG::LoopPreHeaderCreationPhase::run):
* dfg/DFGLoopPreHeaderCreationPhase.h:
* dfg/DFGNode.h:
(JSC::DFG::Node::hasUnlinkedLocal):
(JSC::DFG::Node::unlinkedLocal):
* dfg/DFGNodeType.h:
* dfg/DFGOSREntry.cpp:
(JSC::DFG::prepareOSREntry):
* dfg/DFGOSREntrypointCreationPhase.cpp: Added.
(JSC::DFG::OSREntrypointCreationPhase::OSREntrypointCreationPhase):
(JSC::DFG::OSREntrypointCreationPhase::run):
(JSC::DFG::performOSREntrypointCreation):
* dfg/DFGOSREntrypointCreationPhase.h: Added.
* dfg/DFGOperations.cpp:
* dfg/DFGOperations.h:
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::Plan):
(JSC::DFG::Plan::compileInThread):
(JSC::DFG::Plan::compileInThreadImpl):
* dfg/DFGPlan.h:
* dfg/DFGPredictionInjectionPhase.cpp:
(JSC::DFG::PredictionInjectionPhase::run):
* dfg/DFGPredictionPropagationPhase.cpp:
(JSC::DFG::PredictionPropagationPhase::propagate):
* dfg/DFGSafeToExecute.h:
(JSC::DFG::safeToExecute):
* dfg/DFGSpeculativeJIT32_64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGSpeculativeJIT64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGTierUpCheckInjectionPhase.cpp: Added.
(JSC::DFG::TierUpCheckInjectionPhase::TierUpCheckInjectionPhase):
(JSC::DFG::TierUpCheckInjectionPhase::run):
(JSC::DFG::performTierUpCheckInjection):
* dfg/DFGTierUpCheckInjectionPhase.h: Added.
* dfg/DFGToFTLDeferredCompilationCallback.cpp: Added.
(JSC::DFG::ToFTLDeferredCompilationCallback::ToFTLDeferredCompilationCallback):
(JSC::DFG::ToFTLDeferredCompilationCallback::~ToFTLDeferredCompilationCallback):
(JSC::DFG::ToFTLDeferredCompilationCallback::create):
(JSC::DFG::ToFTLDeferredCompilationCallback::compilationDidBecomeReadyAsynchronously):
(JSC::DFG::ToFTLDeferredCompilationCallback::compilationDidComplete):
* dfg/DFGToFTLDeferredCompilationCallback.h: Added.
* dfg/DFGToFTLForOSREntryDeferredCompilationCallback.cpp: Added.
(JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::ToFTLForOSREntryDeferredCompilationCallback):
(JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::~ToFTLForOSREntryDeferredCompilationCallback):
(JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::create):
(JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::compilationDidBecomeReadyAsynchronously):
(JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::compilationDidComplete):
* dfg/DFGToFTLForOSREntryDeferredCompilationCallback.h: Added.
* dfg/DFGWorklist.cpp:
(JSC::DFG::globalWorklist):
* dfg/DFGWorklist.h:
* ftl/FTLCapabilities.cpp:
(JSC::FTL::canCompile):
* ftl/FTLCapabilities.h:
* ftl/FTLForOSREntryJITCode.cpp: Added.
(JSC::FTL::ForOSREntryJITCode::ForOSREntryJITCode):
(JSC::FTL::ForOSREntryJITCode::~ForOSREntryJITCode):
(JSC::FTL::ForOSREntryJITCode::ftlForOSREntry):
(JSC::FTL::ForOSREntryJITCode::initializeEntryBuffer):
* ftl/FTLForOSREntryJITCode.h: Added.
(JSC::FTL::ForOSREntryJITCode::entryBuffer):
(JSC::FTL::ForOSREntryJITCode::setBytecodeIndex):
(JSC::FTL::ForOSREntryJITCode::bytecodeIndex):
(JSC::FTL::ForOSREntryJITCode::countEntryFailure):
(JSC::FTL::ForOSREntryJITCode::entryFailureCount):
* ftl/FTLJITFinalizer.cpp:
(JSC::FTL::JITFinalizer::finalizeFunction):
* ftl/FTLLink.cpp:
(JSC::FTL::link):
* ftl/FTLLowerDFGToLLVM.cpp:
(JSC::FTL::LowerDFGToLLVM::compileBlock):
(JSC::FTL::LowerDFGToLLVM::compileNode):
(JSC::FTL::LowerDFGToLLVM::compileExtractOSREntryLocal):
(JSC::FTL::LowerDFGToLLVM::compileGetLocal):
(JSC::FTL::LowerDFGToLLVM::addWeakReference):
* ftl/FTLOSREntry.cpp: Added.
(JSC::FTL::prepareOSREntry):
* ftl/FTLOSREntry.h: Added.
* ftl/FTLOutput.h:
(JSC::FTL::Output::crashNonTerminal):
(JSC::FTL::Output::crash):
* ftl/FTLState.cpp:
(JSC::FTL::State::State):
* interpreter/Register.h:
(JSC::Register::unboxedDouble):
* jit/JIT.cpp:
(JSC::JIT::emitEnterOptimizationCheck):
* jit/JITCode.cpp:
(JSC::JITCode::ftlForOSREntry):
* jit/JITCode.h:
* jit/JITStubs.cpp:
(JSC::DEFINE_STUB_FUNCTION):
* runtime/Executable.cpp:
(JSC::ScriptExecutable::newReplacementCodeBlockFor):
* runtime/Options.h:
* runtime/VM.cpp:
(JSC::VM::ensureWorklist):
* runtime/VM.h:

LayoutTests:

Reviewed by Mark Hahnenberg.

Fix marsaglia to check the result instead of printing, and add a second
version that relies on OSR entry.

* fast/js/regress/marsaglia-osr-entry-expected.txt: Added.
* fast/js/regress/marsaglia-osr-entry.html: Added.
* fast/js/regress/script-tests/marsaglia-osr-entry.js: Added.
(marsaglia):
* fast/js/regress/script-tests/marsaglia.js:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@155023 268f45cc-cd09-0410-ab3c-d52691b4dbfc

69 files changed:
LayoutTests/ChangeLog
LayoutTests/fast/js/regress/marsaglia-osr-entry-expected.txt [new file with mode: 0644]
LayoutTests/fast/js/regress/marsaglia-osr-entry.html [new file with mode: 0644]
LayoutTests/fast/js/regress/script-tests/marsaglia-osr-entry.js [new file with mode: 0644]
LayoutTests/fast/js/regress/script-tests/marsaglia.js
Source/JavaScriptCore/CMakeLists.txt
Source/JavaScriptCore/ChangeLog
Source/JavaScriptCore/GNUmakefile.list.am
Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj
Source/JavaScriptCore/Target.pri
Source/JavaScriptCore/bytecode/CodeBlock.cpp
Source/JavaScriptCore/bytecode/CodeBlock.h
Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h
Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp
Source/JavaScriptCore/dfg/DFGCFGSimplificationPhase.cpp
Source/JavaScriptCore/dfg/DFGClobberize.h
Source/JavaScriptCore/dfg/DFGDriver.cpp
Source/JavaScriptCore/dfg/DFGDriver.h
Source/JavaScriptCore/dfg/DFGFixupPhase.cpp
Source/JavaScriptCore/dfg/DFGGraph.cpp
Source/JavaScriptCore/dfg/DFGGraph.h
Source/JavaScriptCore/dfg/DFGInPlaceAbstractState.cpp
Source/JavaScriptCore/dfg/DFGJITCode.cpp
Source/JavaScriptCore/dfg/DFGJITCode.h
Source/JavaScriptCore/dfg/DFGJITFinalizer.cpp
Source/JavaScriptCore/dfg/DFGLoopPreHeaderCreationPhase.cpp
Source/JavaScriptCore/dfg/DFGLoopPreHeaderCreationPhase.h
Source/JavaScriptCore/dfg/DFGNode.h
Source/JavaScriptCore/dfg/DFGNodeType.h
Source/JavaScriptCore/dfg/DFGOSREntry.cpp
Source/JavaScriptCore/dfg/DFGOSREntrypointCreationPhase.cpp [new file with mode: 0644]
Source/JavaScriptCore/dfg/DFGOSREntrypointCreationPhase.h [new file with mode: 0644]
Source/JavaScriptCore/dfg/DFGOperations.cpp
Source/JavaScriptCore/dfg/DFGOperations.h
Source/JavaScriptCore/dfg/DFGPlan.cpp
Source/JavaScriptCore/dfg/DFGPlan.h
Source/JavaScriptCore/dfg/DFGPredictionInjectionPhase.cpp
Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp
Source/JavaScriptCore/dfg/DFGSafeToExecute.h
Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp
Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp
Source/JavaScriptCore/dfg/DFGTierUpCheckInjectionPhase.cpp [new file with mode: 0644]
Source/JavaScriptCore/dfg/DFGTierUpCheckInjectionPhase.h [new file with mode: 0644]
Source/JavaScriptCore/dfg/DFGToFTLDeferredCompilationCallback.cpp [new file with mode: 0644]
Source/JavaScriptCore/dfg/DFGToFTLDeferredCompilationCallback.h [new file with mode: 0644]
Source/JavaScriptCore/dfg/DFGToFTLForOSREntryDeferredCompilationCallback.cpp [new file with mode: 0644]
Source/JavaScriptCore/dfg/DFGToFTLForOSREntryDeferredCompilationCallback.h [new file with mode: 0644]
Source/JavaScriptCore/dfg/DFGWorklist.cpp
Source/JavaScriptCore/dfg/DFGWorklist.h
Source/JavaScriptCore/ftl/FTLCapabilities.cpp
Source/JavaScriptCore/ftl/FTLCapabilities.h
Source/JavaScriptCore/ftl/FTLForOSREntryJITCode.cpp [new file with mode: 0644]
Source/JavaScriptCore/ftl/FTLForOSREntryJITCode.h [new file with mode: 0644]
Source/JavaScriptCore/ftl/FTLJITFinalizer.cpp
Source/JavaScriptCore/ftl/FTLLink.cpp
Source/JavaScriptCore/ftl/FTLLowerDFGToLLVM.cpp
Source/JavaScriptCore/ftl/FTLOSREntry.cpp [new file with mode: 0644]
Source/JavaScriptCore/ftl/FTLOSREntry.h [new file with mode: 0644]
Source/JavaScriptCore/ftl/FTLOutput.h
Source/JavaScriptCore/ftl/FTLState.cpp
Source/JavaScriptCore/interpreter/Register.h
Source/JavaScriptCore/jit/JIT.cpp
Source/JavaScriptCore/jit/JITCode.cpp
Source/JavaScriptCore/jit/JITCode.h
Source/JavaScriptCore/jit/JITStubs.cpp
Source/JavaScriptCore/runtime/Executable.cpp
Source/JavaScriptCore/runtime/Options.h
Source/JavaScriptCore/runtime/VM.cpp
Source/JavaScriptCore/runtime/VM.h

index 136731e..961ff26 100644 (file)
@@ -1,3 +1,19 @@
+2013-09-03  Filip Pizlo  <fpizlo@apple.com>
+
+        The DFG should be able to tier-up and OSR enter into the FTL
+        https://bugs.webkit.org/show_bug.cgi?id=112838
+
+        Reviewed by Mark Hahnenberg.
+        
+        Fix marsaglia to check the result instead of printing, and add a second
+        version that relies on OSR entry.
+
+        * fast/js/regress/marsaglia-osr-entry-expected.txt: Added.
+        * fast/js/regress/marsaglia-osr-entry.html: Added.
+        * fast/js/regress/script-tests/marsaglia-osr-entry.js: Added.
+        (marsaglia):
+        * fast/js/regress/script-tests/marsaglia.js:
+
 2013-09-03  Chris Fleizach  <cfleizach@apple.com>
 
         AX: REGRESSION: @title is exposed as AXDescription when label label from contents already exists.
diff --git a/LayoutTests/fast/js/regress/marsaglia-osr-entry-expected.txt b/LayoutTests/fast/js/regress/marsaglia-osr-entry-expected.txt
new file mode 100644 (file)
index 0000000..dd1284e
--- /dev/null
@@ -0,0 +1,10 @@
+JSRegress/marsaglia-osr-entry
+
+On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
+
+
+PASS no exception thrown
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
diff --git a/LayoutTests/fast/js/regress/marsaglia-osr-entry.html b/LayoutTests/fast/js/regress/marsaglia-osr-entry.html
new file mode 100644 (file)
index 0000000..34f5443
--- /dev/null
@@ -0,0 +1,12 @@
+<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
+<html>
+<head>
+<script src="../resources/js-test-pre.js"></script>
+</head>
+<body>
+<script src="resources/regress-pre.js"></script>
+<script src="script-tests/marsaglia-osr-entry.js"></script>
+<script src="resources/regress-post.js"></script>
+<script src="../resources/js-test-post.js"></script>
+</body>
+</html>
diff --git a/LayoutTests/fast/js/regress/script-tests/marsaglia-osr-entry.js b/LayoutTests/fast/js/regress/script-tests/marsaglia-osr-entry.js
new file mode 100644 (file)
index 0000000..4cd98c5
--- /dev/null
@@ -0,0 +1,15 @@
+function marsaglia(m_z, m_w, n) {
+    var result;
+    for (var i = 0; i < n; ++i) {
+        m_z = (36969 * (m_z & 65535) + (m_z >> 16)) | 0;
+        m_w = (18000 * (m_w & 65535) + (m_w >> 16)) | 0;
+        result = ((m_z << 16) + m_w) | 0;
+    }
+    return result;
+}
+
+var result = marsaglia(5, 7, 10000000);
+
+if (result != -1047364056)
+    throw "Error: bad result: " + result;
+
index 5f2ce01..85c2a2d 100644 (file)
@@ -12,4 +12,6 @@ var result = 0;
 for (var i = 0; i < 100; ++i)
     result += marsaglia(i, i + 1, 1000000);
 
-print(result);
+if (result != 8216386243)
+    throw "Error: bad result: " + result;
+
index bb3fc5b..72621b8 100644 (file)
@@ -132,6 +132,7 @@ set(JavaScriptCore_SOURCES
     dfg/DFGNodeFlags.cpp
     dfg/DFGOSRAvailabilityAnalysisPhase.cpp
     dfg/DFGOSREntry.cpp
+    dfg/DFGOSREntrypointCreationPhase.cpp
     dfg/DFGOSRExit.cpp
     dfg/DFGOSRExitBase.cpp
     dfg/DFGOSRExitCompiler.cpp
@@ -151,6 +152,7 @@ set(JavaScriptCore_SOURCES
     dfg/DFGSpeculativeJIT32_64.cpp
     dfg/DFGSpeculativeJIT64.cpp
     dfg/DFGThunks.cpp
+    dfg/DFGTierUpCheckInjectionPhase.cpp
     dfg/DFGTypeCheckHoistingPhase.cpp
     dfg/DFGUnificationPhase.cpp
     dfg/DFGUseKind.cpp
index d61a610..f5806e0 100644 (file)
@@ -1,5 +1,196 @@
 2013-09-03  Filip Pizlo  <fpizlo@apple.com>
 
+        The DFG should be able to tier-up and OSR enter into the FTL
+        https://bugs.webkit.org/show_bug.cgi?id=112838
+
+        Reviewed by Mark Hahnenberg.
+        
+        This adds the ability for the DFG to tier-up into the FTL. This works in both
+        of the expected tier-up modes:
+        
+        Replacement: frequently called functions eventually have their entrypoint
+        replaced with one that goes into FTL-compiled code. Note, this will be a
+        slow-down for now since we don't yet have LLVM calling convention integration.
+        
+        OSR entry: code stuck in hot loops gets OSR'd into the FTL from the DFG.
+        
+        This means that if the DFG detects that a function is an FTL candidate, it
+        inserts execution counting code similar to the kind that the baseline JIT
+        would use. If you trip on a loop count in a loop header that is an OSR
+        candidate (it's not an inlined loop), we do OSR; otherwise we do replacement.
+        OSR almost always also implies future replacement.
+        
+        OSR entry into the FTL is really cool. It uses a specialized FTL compile of
+        the code, where early in the DFG pipeline we replace the original root block
+        with an OSR entrypoint block that jumps to the pre-header of the hot loop.
+        The OSR entrypoint loads all live state at the loop pre-header using loads
+        from a scratch buffer, which gets populated by the runtime's OSR entry
+        preparation code (FTL::prepareOSREntry()). This approach appears to work well
+        with all of our subsequent optimizations, including prediction propagation,
+        CFA, and LICM. LLVM seems happy with it, too. Best of all, it works naturally
+        with concurrent compilation: when we hit the tier-up trigger we spawn a
+        compilation plan at the bytecode index from which we triggered; once the
+        compilation finishes the next trigger will try to enter, at that bytecode
+        index. If it can't - for example because the code has moved on to another
+        loop - then we just try again. Loops that get hot enough for OSR entry (about
+        25,000 iterations) will probably still be running when a concurrent compile
+        finishes, so this doesn't appear to be a big problem.
+        
+        This immediately gives us a 70% speed-up on imaging-gaussian-blur. We could
+        get a bigger speed-up by adding some more intelligence and tweaking LLVM to
+        compile code faster. Those things will happen eventually but this is a good
+        start. Probably this code will see more tuning as we get more coverage in the
+        FTL JIT, but I'll worry about that in future patches.
+
+        * CMakeLists.txt:
+        * GNUmakefile.list.am:
+        * JavaScriptCore.xcodeproj/project.pbxproj:
+        * Target.pri:
+        * bytecode/CodeBlock.cpp:
+        (JSC::CodeBlock::CodeBlock):
+        (JSC::CodeBlock::hasOptimizedReplacement):
+        (JSC::CodeBlock::setOptimizationThresholdBasedOnCompilationResult):
+        * bytecode/CodeBlock.h:
+        * dfg/DFGAbstractInterpreterInlines.h:
+        (JSC::DFG::::executeEffects):
+        * dfg/DFGByteCodeParser.cpp:
+        (JSC::DFG::ByteCodeParser::parseBlock):
+        (JSC::DFG::ByteCodeParser::parse):
+        * dfg/DFGCFGSimplificationPhase.cpp:
+        (JSC::DFG::CFGSimplificationPhase::run):
+        * dfg/DFGClobberize.h:
+        (JSC::DFG::clobberize):
+        * dfg/DFGDriver.cpp:
+        (JSC::DFG::compileImpl):
+        (JSC::DFG::compile):
+        * dfg/DFGDriver.h:
+        * dfg/DFGFixupPhase.cpp:
+        (JSC::DFG::FixupPhase::fixupNode):
+        * dfg/DFGGraph.cpp:
+        (JSC::DFG::Graph::dump):
+        (JSC::DFG::Graph::killBlockAndItsContents):
+        (JSC::DFG::Graph::killUnreachableBlocks):
+        * dfg/DFGGraph.h:
+        * dfg/DFGInPlaceAbstractState.cpp:
+        (JSC::DFG::InPlaceAbstractState::initialize):
+        * dfg/DFGJITCode.cpp:
+        (JSC::DFG::JITCode::reconstruct):
+        (JSC::DFG::JITCode::checkIfOptimizationThresholdReached):
+        (JSC::DFG::JITCode::optimizeNextInvocation):
+        (JSC::DFG::JITCode::dontOptimizeAnytimeSoon):
+        (JSC::DFG::JITCode::optimizeAfterWarmUp):
+        (JSC::DFG::JITCode::optimizeSoon):
+        (JSC::DFG::JITCode::forceOptimizationSlowPathConcurrently):
+        (JSC::DFG::JITCode::setOptimizationThresholdBasedOnCompilationResult):
+        * dfg/DFGJITCode.h:
+        * dfg/DFGJITFinalizer.cpp:
+        (JSC::DFG::JITFinalizer::finalize):
+        (JSC::DFG::JITFinalizer::finalizeFunction):
+        (JSC::DFG::JITFinalizer::finalizeCommon):
+        * dfg/DFGLoopPreHeaderCreationPhase.cpp:
+        (JSC::DFG::createPreHeader):
+        (JSC::DFG::LoopPreHeaderCreationPhase::run):
+        * dfg/DFGLoopPreHeaderCreationPhase.h:
+        * dfg/DFGNode.h:
+        (JSC::DFG::Node::hasUnlinkedLocal):
+        (JSC::DFG::Node::unlinkedLocal):
+        * dfg/DFGNodeType.h:
+        * dfg/DFGOSREntry.cpp:
+        (JSC::DFG::prepareOSREntry):
+        * dfg/DFGOSREntrypointCreationPhase.cpp: Added.
+        (JSC::DFG::OSREntrypointCreationPhase::OSREntrypointCreationPhase):
+        (JSC::DFG::OSREntrypointCreationPhase::run):
+        (JSC::DFG::performOSREntrypointCreation):
+        * dfg/DFGOSREntrypointCreationPhase.h: Added.
+        * dfg/DFGOperations.cpp:
+        * dfg/DFGOperations.h:
+        * dfg/DFGPlan.cpp:
+        (JSC::DFG::Plan::Plan):
+        (JSC::DFG::Plan::compileInThread):
+        (JSC::DFG::Plan::compileInThreadImpl):
+        * dfg/DFGPlan.h:
+        * dfg/DFGPredictionInjectionPhase.cpp:
+        (JSC::DFG::PredictionInjectionPhase::run):
+        * dfg/DFGPredictionPropagationPhase.cpp:
+        (JSC::DFG::PredictionPropagationPhase::propagate):
+        * dfg/DFGSafeToExecute.h:
+        (JSC::DFG::safeToExecute):
+        * dfg/DFGSpeculativeJIT32_64.cpp:
+        (JSC::DFG::SpeculativeJIT::compile):
+        * dfg/DFGSpeculativeJIT64.cpp:
+        (JSC::DFG::SpeculativeJIT::compile):
+        * dfg/DFGTierUpCheckInjectionPhase.cpp: Added.
+        (JSC::DFG::TierUpCheckInjectionPhase::TierUpCheckInjectionPhase):
+        (JSC::DFG::TierUpCheckInjectionPhase::run):
+        (JSC::DFG::performTierUpCheckInjection):
+        * dfg/DFGTierUpCheckInjectionPhase.h: Added.
+        * dfg/DFGToFTLDeferredCompilationCallback.cpp: Added.
+        (JSC::DFG::ToFTLDeferredCompilationCallback::ToFTLDeferredCompilationCallback):
+        (JSC::DFG::ToFTLDeferredCompilationCallback::~ToFTLDeferredCompilationCallback):
+        (JSC::DFG::ToFTLDeferredCompilationCallback::create):
+        (JSC::DFG::ToFTLDeferredCompilationCallback::compilationDidBecomeReadyAsynchronously):
+        (JSC::DFG::ToFTLDeferredCompilationCallback::compilationDidComplete):
+        * dfg/DFGToFTLDeferredCompilationCallback.h: Added.
+        * dfg/DFGToFTLForOSREntryDeferredCompilationCallback.cpp: Added.
+        (JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::ToFTLForOSREntryDeferredCompilationCallback):
+        (JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::~ToFTLForOSREntryDeferredCompilationCallback):
+        (JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::create):
+        (JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::compilationDidBecomeReadyAsynchronously):
+        (JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::compilationDidComplete):
+        * dfg/DFGToFTLForOSREntryDeferredCompilationCallback.h: Added.
+        * dfg/DFGWorklist.cpp:
+        (JSC::DFG::globalWorklist):
+        * dfg/DFGWorklist.h:
+        * ftl/FTLCapabilities.cpp:
+        (JSC::FTL::canCompile):
+        * ftl/FTLCapabilities.h:
+        * ftl/FTLForOSREntryJITCode.cpp: Added.
+        (JSC::FTL::ForOSREntryJITCode::ForOSREntryJITCode):
+        (JSC::FTL::ForOSREntryJITCode::~ForOSREntryJITCode):
+        (JSC::FTL::ForOSREntryJITCode::ftlForOSREntry):
+        (JSC::FTL::ForOSREntryJITCode::initializeEntryBuffer):
+        * ftl/FTLForOSREntryJITCode.h: Added.
+        (JSC::FTL::ForOSREntryJITCode::entryBuffer):
+        (JSC::FTL::ForOSREntryJITCode::setBytecodeIndex):
+        (JSC::FTL::ForOSREntryJITCode::bytecodeIndex):
+        (JSC::FTL::ForOSREntryJITCode::countEntryFailure):
+        (JSC::FTL::ForOSREntryJITCode::entryFailureCount):
+        * ftl/FTLJITFinalizer.cpp:
+        (JSC::FTL::JITFinalizer::finalizeFunction):
+        * ftl/FTLLink.cpp:
+        (JSC::FTL::link):
+        * ftl/FTLLowerDFGToLLVM.cpp:
+        (JSC::FTL::LowerDFGToLLVM::compileBlock):
+        (JSC::FTL::LowerDFGToLLVM::compileNode):
+        (JSC::FTL::LowerDFGToLLVM::compileExtractOSREntryLocal):
+        (JSC::FTL::LowerDFGToLLVM::compileGetLocal):
+        (JSC::FTL::LowerDFGToLLVM::addWeakReference):
+        * ftl/FTLOSREntry.cpp: Added.
+        (JSC::FTL::prepareOSREntry):
+        * ftl/FTLOSREntry.h: Added.
+        * ftl/FTLOutput.h:
+        (JSC::FTL::Output::crashNonTerminal):
+        (JSC::FTL::Output::crash):
+        * ftl/FTLState.cpp:
+        (JSC::FTL::State::State):
+        * interpreter/Register.h:
+        (JSC::Register::unboxedDouble):
+        * jit/JIT.cpp:
+        (JSC::JIT::emitEnterOptimizationCheck):
+        * jit/JITCode.cpp:
+        (JSC::JITCode::ftlForOSREntry):
+        * jit/JITCode.h:
+        * jit/JITStubs.cpp:
+        (JSC::DEFINE_STUB_FUNCTION):
+        * runtime/Executable.cpp:
+        (JSC::ScriptExecutable::newReplacementCodeBlockFor):
+        * runtime/Options.h:
+        * runtime/VM.cpp:
+        (JSC::VM::ensureWorklist):
+        * runtime/VM.h:
+
+2013-09-03  Filip Pizlo  <fpizlo@apple.com>
+
         CodeBlock memory cost reporting should be rationalized
         https://bugs.webkit.org/show_bug.cgi?id=120615
 
index 68aff1f..a2dfe77 100644 (file)
@@ -316,6 +316,8 @@ javascriptcore_sources += \
        Source/JavaScriptCore/dfg/DFGOSRAvailabilityAnalysisPhase.h \
        Source/JavaScriptCore/dfg/DFGOSREntry.cpp \
        Source/JavaScriptCore/dfg/DFGOSREntry.h \
+       Source/JavaScriptCore/dfg/DFGOSREntrypointCreationPhase.cpp \
+       Source/JavaScriptCore/dfg/DFGOSREntrypointCreationPhase.h \
        Source/JavaScriptCore/dfg/DFGOSRExitCompilationInfo.h \
        Source/JavaScriptCore/dfg/DFGOSRExitCompiler32_64.cpp \
        Source/JavaScriptCore/dfg/DFGOSRExitCompiler64.cpp \
@@ -356,6 +358,8 @@ javascriptcore_sources += \
        Source/JavaScriptCore/dfg/DFGSSAConversionPhase.cpp \
        Source/JavaScriptCore/dfg/DFGSSAConversionPhase.h \
        Source/JavaScriptCore/dfg/DFGStructureAbstractValue.h \
+       Source/JavaScriptCore/dfg/DFGTierUpCheckInjectionPhase.cpp \
+       Source/JavaScriptCore/dfg/DFGTierUpCheckInjectionPhase.h \
        Source/JavaScriptCore/dfg/DFGThunks.cpp \
        Source/JavaScriptCore/dfg/DFGThunks.h \
        Source/JavaScriptCore/dfg/DFGTypeCheckHoistingPhase.cpp \
index f8341b8..2e6744d 100644 (file)
                0FD82E86141F3FF100179C94 /* SpeculatedType.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FD82E84141F3FDA00179C94 /* SpeculatedType.cpp */; };
                0FD8A31317D4326C00CA2C40 /* CodeBlockSet.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FD8A31117D4326C00CA2C40 /* CodeBlockSet.cpp */; };
                0FD8A31417D4326C00CA2C40 /* CodeBlockSet.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FD8A31217D4326C00CA2C40 /* CodeBlockSet.h */; settings = {ATTRIBUTES = (Private, ); }; };
+               0FD8A31917D51F2200CA2C40 /* FTLForOSREntryJITCode.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FD8A31517D51F2200CA2C40 /* FTLForOSREntryJITCode.cpp */; };
+               0FD8A31A17D51F2200CA2C40 /* FTLForOSREntryJITCode.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FD8A31617D51F2200CA2C40 /* FTLForOSREntryJITCode.h */; settings = {ATTRIBUTES = (Private, ); }; };
+               0FD8A31B17D51F2200CA2C40 /* FTLOSREntry.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FD8A31717D51F2200CA2C40 /* FTLOSREntry.cpp */; };
+               0FD8A31C17D51F2200CA2C40 /* FTLOSREntry.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FD8A31817D51F2200CA2C40 /* FTLOSREntry.h */; settings = {ATTRIBUTES = (Private, ); }; };
+               0FD8A32517D51F5700CA2C40 /* DFGOSREntrypointCreationPhase.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FD8A31D17D51F5700CA2C40 /* DFGOSREntrypointCreationPhase.cpp */; };
+               0FD8A32617D51F5700CA2C40 /* DFGOSREntrypointCreationPhase.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FD8A31E17D51F5700CA2C40 /* DFGOSREntrypointCreationPhase.h */; settings = {ATTRIBUTES = (Private, ); }; };
+               0FD8A32717D51F5700CA2C40 /* DFGTierUpCheckInjectionPhase.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FD8A31F17D51F5700CA2C40 /* DFGTierUpCheckInjectionPhase.cpp */; };
+               0FD8A32817D51F5700CA2C40 /* DFGTierUpCheckInjectionPhase.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FD8A32017D51F5700CA2C40 /* DFGTierUpCheckInjectionPhase.h */; settings = {ATTRIBUTES = (Private, ); }; };
+               0FD8A32917D51F5700CA2C40 /* DFGToFTLDeferredCompilationCallback.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FD8A32117D51F5700CA2C40 /* DFGToFTLDeferredCompilationCallback.cpp */; };
+               0FD8A32A17D51F5700CA2C40 /* DFGToFTLDeferredCompilationCallback.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FD8A32217D51F5700CA2C40 /* DFGToFTLDeferredCompilationCallback.h */; settings = {ATTRIBUTES = (Private, ); }; };
+               0FD8A32B17D51F5700CA2C40 /* DFGToFTLForOSREntryDeferredCompilationCallback.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FD8A32317D51F5700CA2C40 /* DFGToFTLForOSREntryDeferredCompilationCallback.cpp */; };
+               0FD8A32C17D51F5700CA2C40 /* DFGToFTLForOSREntryDeferredCompilationCallback.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FD8A32417D51F5700CA2C40 /* DFGToFTLForOSREntryDeferredCompilationCallback.h */; settings = {ATTRIBUTES = (Private, ); }; };
                0FDB2CC9173DA520007B3C1B /* FTLAbbreviatedTypes.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FDB2CC7173DA51E007B3C1B /* FTLAbbreviatedTypes.h */; settings = {ATTRIBUTES = (Private, ); }; };
                0FDB2CCA173DA523007B3C1B /* FTLValueFromBlock.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FDB2CC8173DA51E007B3C1B /* FTLValueFromBlock.h */; settings = {ATTRIBUTES = (Private, ); }; };
                0FDB2CE7174830A2007B3C1B /* DFGWorklist.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FDB2CE5174830A2007B3C1B /* DFGWorklist.cpp */; };
                0FD82E84141F3FDA00179C94 /* SpeculatedType.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = SpeculatedType.cpp; sourceTree = "<group>"; };
                0FD8A31117D4326C00CA2C40 /* CodeBlockSet.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = CodeBlockSet.cpp; sourceTree = "<group>"; };
                0FD8A31217D4326C00CA2C40 /* CodeBlockSet.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = CodeBlockSet.h; sourceTree = "<group>"; };
+               0FD8A31517D51F2200CA2C40 /* FTLForOSREntryJITCode.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = FTLForOSREntryJITCode.cpp; path = ftl/FTLForOSREntryJITCode.cpp; sourceTree = "<group>"; };
+               0FD8A31617D51F2200CA2C40 /* FTLForOSREntryJITCode.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = FTLForOSREntryJITCode.h; path = ftl/FTLForOSREntryJITCode.h; sourceTree = "<group>"; };
+               0FD8A31717D51F2200CA2C40 /* FTLOSREntry.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = FTLOSREntry.cpp; path = ftl/FTLOSREntry.cpp; sourceTree = "<group>"; };
+               0FD8A31817D51F2200CA2C40 /* FTLOSREntry.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = FTLOSREntry.h; path = ftl/FTLOSREntry.h; sourceTree = "<group>"; };
+               0FD8A31D17D51F5700CA2C40 /* DFGOSREntrypointCreationPhase.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGOSREntrypointCreationPhase.cpp; path = dfg/DFGOSREntrypointCreationPhase.cpp; sourceTree = "<group>"; };
+               0FD8A31E17D51F5700CA2C40 /* DFGOSREntrypointCreationPhase.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGOSREntrypointCreationPhase.h; path = dfg/DFGOSREntrypointCreationPhase.h; sourceTree = "<group>"; };
+               0FD8A31F17D51F5700CA2C40 /* DFGTierUpCheckInjectionPhase.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGTierUpCheckInjectionPhase.cpp; path = dfg/DFGTierUpCheckInjectionPhase.cpp; sourceTree = "<group>"; };
+               0FD8A32017D51F5700CA2C40 /* DFGTierUpCheckInjectionPhase.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGTierUpCheckInjectionPhase.h; path = dfg/DFGTierUpCheckInjectionPhase.h; sourceTree = "<group>"; };
+               0FD8A32117D51F5700CA2C40 /* DFGToFTLDeferredCompilationCallback.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGToFTLDeferredCompilationCallback.cpp; path = dfg/DFGToFTLDeferredCompilationCallback.cpp; sourceTree = "<group>"; };
+               0FD8A32217D51F5700CA2C40 /* DFGToFTLDeferredCompilationCallback.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGToFTLDeferredCompilationCallback.h; path = dfg/DFGToFTLDeferredCompilationCallback.h; sourceTree = "<group>"; };
+               0FD8A32317D51F5700CA2C40 /* DFGToFTLForOSREntryDeferredCompilationCallback.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGToFTLForOSREntryDeferredCompilationCallback.cpp; path = dfg/DFGToFTLForOSREntryDeferredCompilationCallback.cpp; sourceTree = "<group>"; };
+               0FD8A32417D51F5700CA2C40 /* DFGToFTLForOSREntryDeferredCompilationCallback.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGToFTLForOSREntryDeferredCompilationCallback.h; path = dfg/DFGToFTLForOSREntryDeferredCompilationCallback.h; sourceTree = "<group>"; };
                0FDB2CC7173DA51E007B3C1B /* FTLAbbreviatedTypes.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; name = FTLAbbreviatedTypes.h; path = ftl/FTLAbbreviatedTypes.h; sourceTree = "<group>"; };
                0FDB2CC8173DA51E007B3C1B /* FTLValueFromBlock.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; name = FTLValueFromBlock.h; path = ftl/FTLValueFromBlock.h; sourceTree = "<group>"; };
                0FDB2CE5174830A2007B3C1B /* DFGWorklist.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGWorklist.cpp; path = dfg/DFGWorklist.cpp; sourceTree = "<group>"; };
                                A7F2996917A0BB670010417A /* FTLFail.cpp */,
                                A7F2996A17A0BB670010417A /* FTLFail.h */,
                                0FEA0A2B170B661900BB722C /* FTLFormattedValue.h */,
+                               0FD8A31517D51F2200CA2C40 /* FTLForOSREntryJITCode.cpp */,
+                               0FD8A31617D51F2200CA2C40 /* FTLForOSREntryJITCode.h */,
                                A78A977C179738D5009DF744 /* FTLGeneratedFunction.h */,
                                0FEA0A261709623B00BB722C /* FTLIntrinsicRepository.cpp */,
                                0FEA0A221709606900BB722C /* FTLIntrinsicRepository.h */,
                                0FEA0A04170513DB00BB722C /* FTLLowerDFGToLLVM.cpp */,
                                0FEA0A05170513DB00BB722C /* FTLLowerDFGToLLVM.h */,
                                A7D89D0117A0B90400773AD8 /* FTLLoweredNodeValue.h */,
+                               0FD8A31717D51F2200CA2C40 /* FTLOSREntry.cpp */,
+                               0FD8A31817D51F2200CA2C40 /* FTLOSREntry.h */,
                                0F235BC617178E1C00690C7F /* FTLOSRExit.cpp */,
                                0F235BC717178E1C00690C7F /* FTLOSRExit.h */,
                                0F235BC817178E1C00690C7F /* FTLOSRExitCompilationInfo.h */,
                                A7D89CEF17A0B8CC00773AD8 /* DFGOSRAvailabilityAnalysisPhase.h */,
                                0FD82E52141DAEDE00179C94 /* DFGOSREntry.cpp */,
                                0FD82E53141DAEDE00179C94 /* DFGOSREntry.h */,
+                               0FD8A31D17D51F5700CA2C40 /* DFGOSREntrypointCreationPhase.cpp */,
+                               0FD8A31E17D51F5700CA2C40 /* DFGOSREntrypointCreationPhase.h */,
                                0FC0978E146A6F6300CF2442 /* DFGOSRExit.cpp */,
                                0FC097681468A6EF00CF2442 /* DFGOSRExit.h */,
                                0F235BE717178E7300690C7F /* DFGOSRExitBase.cpp */,
                                0F63947615DCE347006A597C /* DFGStructureAbstractValue.h */,
                                0FC0979F146B28C700CF2442 /* DFGThunks.cpp */,
                                0FC097A0146B28C700CF2442 /* DFGThunks.h */,
+                               0FD8A31F17D51F5700CA2C40 /* DFGTierUpCheckInjectionPhase.cpp */,
+                               0FD8A32017D51F5700CA2C40 /* DFGTierUpCheckInjectionPhase.h */,
+                               0FD8A32117D51F5700CA2C40 /* DFGToFTLDeferredCompilationCallback.cpp */,
+                               0FD8A32217D51F5700CA2C40 /* DFGToFTLDeferredCompilationCallback.h */,
+                               0FD8A32317D51F5700CA2C40 /* DFGToFTLForOSREntryDeferredCompilationCallback.cpp */,
+                               0FD8A32417D51F5700CA2C40 /* DFGToFTLForOSREntryDeferredCompilationCallback.h */,
                                0F63943C15C75F14006A597C /* DFGTypeCheckHoistingPhase.cpp */,
                                0F63943D15C75F14006A597C /* DFGTypeCheckHoistingPhase.h */,
                                0FBE0F6F16C1DB010082C5E8 /* DFGUnificationPhase.cpp */,
                                0F3B3A2C15475002003ED0FF /* DFGValidate.h in Headers */,
                                0F2BDC471522802500CD8910 /* DFGValueRecoveryOverride.h in Headers */,
                                0F2BDC481522802900CD8910 /* DFGValueSource.h in Headers */,
+                               0FD8A32C17D51F5700CA2C40 /* DFGToFTLForOSREntryDeferredCompilationCallback.h in Headers */,
                                0F620174143FCD330068B77C /* DFGVariableAccessData.h in Headers */,
                                0FDDBFB61666EEDA00C55FEF /* DFGVariableAccessDataDump.h in Headers */,
                                0F2BDC491522809600CD8910 /* DFGVariableEvent.h in Headers */,
                                14BA7A9813AADFF8005B7C2C /* Heap.h in Headers */,
                                C2C8D03114A3CEFC00578E65 /* HeapBlock.h in Headers */,
                                14F97447138C853E00DA1C67 /* HeapRootVisitor.h in Headers */,
+                               0FD8A32817D51F5700CA2C40 /* DFGTierUpCheckInjectionPhase.h in Headers */,
                                C24D31E3161CD695002AA4DB /* HeapStatistics.h in Headers */,
                                C2E526BE1590EF000054E48D /* HeapTimer.h in Headers */,
                                0F4680D514BBD24B00BFE272 /* HostCallReturnValue.h in Headers */,
                                0F13912A16771C36009CCB07 /* ProfilerBytecodeSequence.h in Headers */,
                                0FF729BA166AD360000F5BA3 /* ProfilerCompilation.h in Headers */,
                                0FF729BB166AD360000F5BA3 /* ProfilerCompilationKind.h in Headers */,
+                               0FD8A32617D51F5700CA2C40 /* DFGOSREntrypointCreationPhase.h in Headers */,
                                0FF729BC166AD360000F5BA3 /* ProfilerCompiledBytecode.h in Headers */,
                                0FF729BD166AD360000F5BA3 /* ProfilerDatabase.h in Headers */,
                                0FF729BE166AD360000F5BA3 /* ProfilerExecutionCounter.h in Headers */,
                                0FF60AC216740F8300029779 /* ReduceWhitespace.h in Headers */,
                                BC18C45A0E16F5CD00B34460 /* RegExp.h in Headers */,
                                A1712B3F11C7B228007A5315 /* RegExpCache.h in Headers */,
+                               0FD8A31A17D51F2200CA2C40 /* FTLForOSREntryJITCode.h in Headers */,
                                BCD202C20E1706A7002C7E82 /* RegExpConstructor.h in Headers */,
                                BCD202D60E170708002C7E82 /* RegExpConstructor.lut.h in Headers */,
                                A1712B4111C7B235007A5315 /* RegExpKey.h in Headers */,
                                0FB7F39E15ED8E4600F167B2 /* SparseArrayValueMap.h in Headers */,
                                A7386554118697B400540279 /* SpecializedThunkJIT.h in Headers */,
                                0F5541B21613C1FB00CE3E25 /* SpecialPointer.h in Headers */,
+                               0FD8A31C17D51F2200CA2C40 /* FTLOSREntry.h in Headers */,
                                0FD82E54141DAEEE00179C94 /* SpeculatedType.h in Headers */,
                                A7C1EAF217987AB600299DB2 /* StackIterator.h in Headers */,
                                14DF04DA16B3996D0016A513 /* StaticPropertyAnalysis.h in Headers */,
                                BC18C46B0E16F5CD00B34460 /* SymbolTable.h in Headers */,
                                A784A26411D16622005776AC /* SyntaxChecker.h in Headers */,
                                0F572D4F16879FDD00E57FBD /* ThunkGenerator.h in Headers */,
+                               0FD8A32A17D51F5700CA2C40 /* DFGToFTLDeferredCompilationCallback.h in Headers */,
                                A7386556118697B400540279 /* ThunkGenerators.h in Headers */,
                                141448CD13A1783700F5BA1A /* TinyBloomFilter.h in Headers */,
                                0F55989817C86C5800A1E543 /* ToNativeFromValue.h in Headers */,
                                0F8F9446166764F100D61971 /* CodeOrigin.cpp in Sources */,
                                86B5826714D2796C00A9C306 /* CodeProfile.cpp in Sources */,
                                86B5826914D2797000A9C306 /* CodeProfiling.cpp in Sources */,
+                               0FD8A32717D51F5700CA2C40 /* DFGTierUpCheckInjectionPhase.cpp in Sources */,
                                0F8F943C1667631300D61971 /* CodeSpecializationKind.cpp in Sources */,
                                0F8F94421667633500D61971 /* CodeType.cpp in Sources */,
                                147F39C1107EC37600427A48 /* CommonIdentifiers.cpp in Sources */,
                                0F2B66DE17B6B5AB00A7AE3F /* DataView.cpp in Sources */,
                                147F39C3107EC37600427A48 /* DateConstructor.cpp in Sources */,
                                147F39C4107EC37600427A48 /* DateConversion.cpp in Sources */,
+                               0FD8A31B17D51F2200CA2C40 /* FTLOSREntry.cpp in Sources */,
                                147F39C5107EC37600427A48 /* DateInstance.cpp in Sources */,
                                147F39C6107EC37600427A48 /* DatePrototype.cpp in Sources */,
                                14280823107EC02C0013E7B2 /* Debugger.cpp in Sources */,
                                86880F4D14353B2100B08D42 /* DFGSpeculativeJIT64.cpp in Sources */,
                                A7D89CFF17A0B8CC00773AD8 /* DFGSSAConversionPhase.cpp in Sources */,
                                0FC097A1146B28CA00CF2442 /* DFGThunks.cpp in Sources */,
+                               0FD8A32917D51F5700CA2C40 /* DFGToFTLDeferredCompilationCallback.cpp in Sources */,
                                0F63944015C75F1D006A597C /* DFGTypeCheckHoistingPhase.cpp in Sources */,
                                0FBE0F7616C1DB0F0082C5E8 /* DFGUnificationPhase.cpp in Sources */,
                                0F34B14916D42010001CDA5A /* DFGUseKind.cpp in Sources */,
                                A78507D617CBC6FD0011F6E7 /* MapData.cpp in Sources */,
                                A700873D17CBE8D300C3E643 /* MapPrototype.cpp in Sources */,
                                C2B916C514DA040C00CBAC86 /* MarkedAllocator.cpp in Sources */,
+                               0FD8A31917D51F2200CA2C40 /* FTLForOSREntryJITCode.cpp in Sources */,
                                142D6F0813539A2800B02E86 /* MarkedBlock.cpp in Sources */,
                                14D2F3DA139F4BE200491031 /* MarkedSpace.cpp in Sources */,
                                142D6F1113539A4100B02E86 /* MarkStack.cpp in Sources */,
                                14280841107EC0930013E7B2 /* RegExp.cpp in Sources */,
                                A1712B3B11C7B212007A5315 /* RegExpCache.cpp in Sources */,
                                8642C510151C06A90046D4EF /* RegExpCachedResult.cpp in Sources */,
+                               0FD8A32B17D51F5700CA2C40 /* DFGToFTLForOSREntryDeferredCompilationCallback.cpp in Sources */,
                                14280842107EC0930013E7B2 /* RegExpConstructor.cpp in Sources */,
                                8642C512151C083D0046D4EF /* RegExpMatchesArray.cpp in Sources */,
                                14280843107EC0930013E7B2 /* RegExpObject.cpp in Sources */,
                                BCCF0D0C0EF0B8A500413C8F /* StructureStubInfo.cpp in Sources */,
                                C2DF442F1707AC0100A5CA96 /* SuperRegion.cpp in Sources */,
                                0F919D2815856773004A4E7D /* SymbolTable.cpp in Sources */,
+                               0FD8A32517D51F5700CA2C40 /* DFGOSREntrypointCreationPhase.cpp in Sources */,
                                A7386555118697B400540279 /* ThunkGenerators.cpp in Sources */,
                                0F2B670717B6B5AB00A7AE3F /* TypedArrayController.cpp in Sources */,
                                0F2B670A17B6B5AB00A7AE3F /* TypedArrayType.cpp in Sources */,
index eba5dc7..df7ae10 100644 (file)
@@ -170,6 +170,7 @@ SOURCES += \
     dfg/DFGOperations.cpp \
     dfg/DFGOSRAvailabilityAnalysisPhase.cpp \
     dfg/DFGOSREntry.cpp \
+    dfg/DFGOSREntrypointCreationPhase.cpp \
     dfg/DFGOSRExit.cpp \
     dfg/DFGOSRExitBase.cpp \
     dfg/DFGOSRExitCompiler.cpp \
@@ -189,6 +190,7 @@ SOURCES += \
     dfg/DFGSpeculativeJIT64.cpp \
     dfg/DFGTypeCheckHoistingPhase.cpp \
     dfg/DFGThunks.cpp \
+    dfg/DFGTierUpCheckInjectionPhase.cpp \
     dfg/DFGUnificationPhase.cpp \
     dfg/DFGUseKind.cpp \
     dfg/DFGValueSource.cpp \
index 1f94066..68aa517 100644 (file)
@@ -1479,6 +1479,7 @@ CodeBlock::CodeBlock(CopyParsedBlockTag, CodeBlock& other)
     , m_numVars(other.m_numVars)
     , m_isConstructor(other.m_isConstructor)
     , m_shouldAlwaysBeInlined(true)
+    , m_didFailFTLCompilation(false)
     , m_unlinkedCode(*other.m_vm, other.m_ownerExecutable.get(), other.m_unlinkedCode.get())
     , m_ownerExecutable(*other.m_vm, other.m_ownerExecutable.get(), other.m_ownerExecutable.get())
     , m_vm(other.m_vm)
@@ -1529,6 +1530,7 @@ CodeBlock::CodeBlock(ScriptExecutable* ownerExecutable, UnlinkedCodeBlock* unlin
     , m_numVars(unlinkedCodeBlock->m_numVars)
     , m_isConstructor(unlinkedCodeBlock->isConstructor())
     , m_shouldAlwaysBeInlined(true)
+    , m_didFailFTLCompilation(false)
     , m_unlinkedCode(m_globalObject->vm(), ownerExecutable, unlinkedCodeBlock)
     , m_ownerExecutable(m_globalObject->vm(), ownerExecutable, ownerExecutable)
     , m_vm(unlinkedCodeBlock->vm())
@@ -1902,6 +1904,18 @@ void EvalCodeCache::visitAggregate(SlotVisitor& visitor)
         visitor.append(&ptr->value);
 }
 
+CodeBlock* CodeBlock::specialOSREntryBlockOrNull()
+{
+#if ENABLE(FTL_JIT)
+    if (jitType() != JITCode::DFGJIT)
+        return 0;
+    DFG::JITCode* jitCode = m_jitCode->dfg();
+    return jitCode->osrEntryBlock.get();
+#else // ENABLE(FTL_JIT)
+    return 0;
+#endif // ENABLE(FTL_JIT)
+}
+
 void CodeBlock::visitAggregate(SlotVisitor& visitor)
 {
 #if ENABLE(PARALLEL_GC)
@@ -1930,6 +1944,9 @@ void CodeBlock::visitAggregate(SlotVisitor& visitor)
     
     if (!!m_alternative)
         m_alternative->visitAggregate(visitor);
+    
+    if (CodeBlock* otherBlock = specialOSREntryBlockOrNull())
+        otherBlock->visitAggregate(visitor);
 
     visitor.reportExtraMemoryUsage(sizeof(CodeBlock));
     if (m_jitCode)
@@ -2385,17 +2402,14 @@ CodeBlock* CodeBlock::baselineVersion()
 }
 
 #if ENABLE(JIT)
+bool CodeBlock::hasOptimizedReplacement(JITCode::JITType typeToReplace)
+{
+    return JITCode::isHigherTier(replacement()->jitType(), typeToReplace);
+}
+
 bool CodeBlock::hasOptimizedReplacement()
 {
-    ASSERT(JITCode::isBaselineCode(jitType()));
-    bool result = JITCode::isHigherTier(replacement()->jitType(), jitType());
-    if (result)
-        ASSERT(JITCode::isOptimizingJIT(replacement()->jitType()));
-    else {
-        ASSERT(JITCode::isBaselineCode(replacement()->jitType()));
-        ASSERT(replacement() == this);
-    }
-    return result;
+    return hasOptimizedReplacement(jitType());
 }
 #endif
 
@@ -2661,6 +2675,8 @@ void CodeBlock::clearEvalCache()
 {
     if (!!m_alternative)
         m_alternative->clearEvalCache();
+    if (CodeBlock* otherBlock = specialOSREntryBlockOrNull())
+        otherBlock->clearEvalCache();
     if (!m_rareData)
         return;
     m_rareData->m_evalCodeCache.clear();
@@ -3030,10 +3046,10 @@ void CodeBlock::setOptimizationThresholdBasedOnCompilationResult(CompilationResu
     case CompilationSuccessful:
         RELEASE_ASSERT(JITCode::isOptimizingJIT(replacement()->jitType()));
         optimizeNextInvocation();
-        break;
+        return;
     case CompilationFailed:
         dontOptimizeAnytimeSoon();
-        break;
+        return;
     case CompilationDeferred:
         // We'd like to do dontOptimizeAnytimeSoon() but we cannot because
         // forceOptimizationSlowPathConcurrently() is inherently racy. It won't
@@ -3041,16 +3057,14 @@ void CodeBlock::setOptimizationThresholdBasedOnCompilationResult(CompilationResu
         // function ends up being a no-op, we still eventually retry and realize
         // that we have optimized code ready.
         optimizeAfterWarmUp();
-        break;
+        return;
     case CompilationInvalidated:
         // Retry with exponential backoff.
         countReoptimization();
         optimizeAfterWarmUp();
-        break;
-    default:
-        RELEASE_ASSERT_NOT_REACHED();
-        break;
+        return;
     }
+    RELEASE_ASSERT_NOT_REACHED();
 }
 
 #endif
index 50cae29..c723d3c 100644 (file)
@@ -308,7 +308,8 @@ public:
     }
     DFG::CapabilityLevel capabilityLevelState() { return m_capabilityLevelState; }
 
-    bool hasOptimizedReplacement();
+    bool hasOptimizedReplacement(JITCode::JITType typeToReplace);
+    bool hasOptimizedReplacement(); // the typeToReplace is my JITType
 #endif
 
     ScriptExecutable* ownerExecutable() const { return m_ownerExecutable.get(); }
@@ -957,6 +958,8 @@ public:
     bool m_shouldAlwaysBeInlined;
     bool m_allTransitionsHaveBeenMarked; // Initialized and used on every GC.
     
+    bool m_didFailFTLCompilation;
+    
 protected:
     virtual void visitWeakReferences(SlotVisitor&);
     virtual void finalizeUnconditionally();
@@ -970,6 +973,8 @@ protected:
 private:
     friend class CodeBlockSet;
     
+    CodeBlock* specialOSREntryBlockOrNull();
+    
     void noticeIncomingCall(ExecState* callerFrame);
     
     double optimizationThresholdScalingFactor();
index c721d36..e7ba132 100644 (file)
@@ -150,6 +150,20 @@ bool AbstractInterpreter<AbstractStateType>::executeEffects(unsigned clobberLimi
         forNode(node) = value;
         break;
     }
+        
+    case ExtractOSREntryLocal: {
+        forNode(node).makeTop();
+        if (!operandIsArgument(node->unlinkedLocal())
+            && m_graph.m_lazyVars.get(node->unlinkedLocal())) {
+            // This is kind of pessimistic - we could know in some cases that the
+            // DFG code at the point of the OSR had already initialized the lazy
+            // variable. But maybe this is fine, since we're inserting OSR
+            // entrypoints very early in the pipeline - so any lazy initializations
+            // ought to be hoisted out anyway.
+            forNode(node).merge(SpecEmpty);
+        }
+        break;
+    }
             
     case GetLocal: {
         VariableAccessData* variableAccessData = node->variableAccessData();
@@ -1515,6 +1529,14 @@ bool AbstractInterpreter<AbstractStateType>::executeEffects(unsigned clobberLimi
     case Phantom:
     case InlineStart:
     case CountExecution:
+    case CheckTierUpInLoop:
+    case CheckTierUpAtReturn:
+        break;
+
+    case CheckTierUpAndOSREnter:
+    case LoopHint:
+        // We pretend that it can exit because it may want to get all state.
+        node->setCanExit(true);
         break;
 
     case Unreachable:
index 7a0a56d..833092f 100644 (file)
@@ -3140,19 +3140,18 @@ bool ByteCodeParser::parseBlock(unsigned limit)
             if (!m_inlineStackTop->m_caller)
                 m_currentBlock->isOSRTarget = true;
 
+            addToGraph(LoopHint);
+            
             if (m_vm->watchdog.isEnabled())
                 addToGraph(CheckWatchdogTimer);
-            else {
-                // Emit a phantom node to ensure that there is a placeholder
-                // node for this bytecode op.
-                addToGraph(Phantom);
-            }
             
             NEXT_OPCODE(op_loop_hint);
         }
             
         case op_init_lazy_reg: {
             set(currentInstruction[1].u.operand, getJSConstantForValue(JSValue()));
+            ASSERT(currentInstruction[1].u.operand >= 0);
+            m_graph.m_lazyVars.set(currentInstruction[1].u.operand);
             NEXT_OPCODE(op_init_lazy_reg);
         }
             
@@ -3634,7 +3633,7 @@ bool ByteCodeParser::parse()
         BasicBlock* block = m_graph.block(blockIndex);
         ASSERT(block);
         if (!block->isReachable) {
-            m_graph.killBlock(block);
+            m_graph.killBlockAndItsContents(block);
             continue;
         }
         
index 63fff9b..ef69fed 100644 (file)
@@ -270,16 +270,7 @@ public:
                 
                 m_graph.invalidateCFG();
                 m_graph.resetReachability();
-
-                for (BlockIndex blockIndex = 0; blockIndex < m_graph.numBlocks(); ++blockIndex) {
-                    BasicBlock* block = m_graph.block(blockIndex);
-                    if (!block)
-                        continue;
-                    if (block->isReachable)
-                        continue;
-                    
-                    killUnreachable(block);
-                }
+                m_graph.killUnreachableBlocks();
             }
             
             if (Options::validateGraphAtEachPhase())
@@ -320,19 +311,6 @@ private:
         }
     }
 
-    void killUnreachable(BasicBlock* block)
-    {
-        ASSERT(block);
-        ASSERT(!block->isReachable);
-        
-        for (unsigned phiIndex = block->phis.size(); phiIndex--;)
-            m_graph.m_allocator.free(block->phis[phiIndex]);
-        for (unsigned nodeIndex = block->size(); nodeIndex--;)
-            m_graph.m_allocator.free(block->at(nodeIndex));
-        
-        m_graph.killBlock(block);
-    }
-    
     void keepOperandAlive(BasicBlock* block, BasicBlock* jettisonedBlock, CodeOrigin codeOrigin, int operand)
     {
         Node* livenessNode = jettisonedBlock->variablesAtHead.operand(operand);
index 0ed6490..e82fd71 100644 (file)
@@ -111,6 +111,7 @@ void clobberize(Graph& graph, Node* node, ReadFunctor& read, WriteFunctor& write
     case IsString:
     case LogicalNot:
     case Int32ToDouble:
+    case ExtractOSREntryLocal:
         return;
         
     case MovHintAndCheck:
@@ -133,6 +134,10 @@ void clobberize(Graph& graph, Node* node, ReadFunctor& read, WriteFunctor& write
     case ForceOSRExit:
     case Return:
     case Unreachable:
+    case CheckTierUpInLoop:
+    case CheckTierUpAtReturn:
+    case CheckTierUpAndOSREnter:
+    case LoopHint:
         write(SideState);
         return;
 
index dc147ed..dadc2b9 100644 (file)
@@ -55,9 +55,9 @@ unsigned getNumCompilations()
 
 #if ENABLE(DFG_JIT)
 static CompilationResult compileImpl(
-    ExecState* exec, CodeBlock* codeBlock, CompilationMode mode,
-    unsigned osrEntryBytecodeIndex, PassRefPtr<DeferredCompilationCallback> callback,
-    Worklist* worklist)
+    VM& vm, CodeBlock* codeBlock, CompilationMode mode, unsigned osrEntryBytecodeIndex,
+    const Operands<JSValue>& mustHandleValues,
+    PassRefPtr<DeferredCompilationCallback> callback, Worklist* worklist)
 {
     SamplingRegion samplingRegion("DFG Compilation (Driver)");
     
@@ -67,8 +67,6 @@ static CompilationResult compileImpl(
     ASSERT(codeBlock->alternative());
     ASSERT(codeBlock->alternative()->jitType() == JITCode::BaselineJIT);
     
-    ASSERT(osrEntryBytecodeIndex != UINT_MAX);
-
     if (!Options::useDFGJIT() || !MacroAssembler::supportsFloatingPoint())
         return CompilationFailed;
 
@@ -76,9 +74,7 @@ static CompilationResult compileImpl(
         return CompilationFailed;
     
     if (logCompilationChanges())
-        dataLog("DFG(Driver) compiling ", *codeBlock, ", number of instructions = ", codeBlock->instructionCount(), "\n");
-    
-    VM& vm = exec->vm();
+        dataLog("DFG(Driver) compiling ", *codeBlock, " with ", mode, ", number of instructions = ", codeBlock->instructionCount(), "\n");
     
     // Make sure that any stubs that the DFG is going to use are initialized. We want to
     // make sure that al JIT code generation does finalization on the main thread.
@@ -93,29 +89,8 @@ static CompilationResult compileImpl(
     vm.getCTIStub(FTL::osrExitGenerationThunkGenerator);
 #endif
     
-    // Derive our set of must-handle values. The compilation must be at least conservative
-    // enough to allow for OSR entry with these values.
-    unsigned numVarsWithValues;
-    if (osrEntryBytecodeIndex)
-        numVarsWithValues = codeBlock->m_numVars;
-    else
-        numVarsWithValues = 0;
     RefPtr<Plan> plan = adoptRef(
-        new Plan(codeBlock, mode, osrEntryBytecodeIndex, numVarsWithValues));
-    for (size_t i = 0; i < plan->mustHandleValues.size(); ++i) {
-        int operand = plan->mustHandleValues.operandForIndex(i);
-        if (operandIsArgument(operand)
-            && !operandToArgument(operand)
-            && codeBlock->codeType() == FunctionCode
-            && codeBlock->specializationKind() == CodeForConstruct) {
-            // Ugh. If we're in a constructor, the 'this' argument may hold garbage. It will
-            // also never be used. It doesn't matter what we put into the value for this,
-            // but it has to be an actual value that can be grokked by subsequent DFG passes,
-            // so we sanitize it here by turning it into Undefined.
-            plan->mustHandleValues[i] = jsUndefined();
-        } else
-            plan->mustHandleValues[i] = exec->uncheckedR(operand).jsValue();
-    }
+        new Plan(codeBlock, mode, osrEntryBytecodeIndex, mustHandleValues));
     
     if (worklist) {
         plan->callback = callback;
@@ -130,7 +105,7 @@ static CompilationResult compileImpl(
 }
 #else // ENABLE(DFG_JIT)
 static CompilationResult compileImpl(
-    ExecState*, CodeBlock*, CompilationMode, unsigned,
+    VM&, CodeBlock*, CompilationMode, unsigned, const Operands<JSValue>&,
     PassRefPtr<DeferredCompilationCallback>, Worklist*)
 {
     return CompilationFailed;
@@ -138,13 +113,13 @@ static CompilationResult compileImpl(
 #endif // ENABLE(DFG_JIT)
 
 CompilationResult compile(
-    ExecState* exec, CodeBlock* codeBlock, CompilationMode mode,
-    unsigned osrEntryBytecodeIndex, PassRefPtr<DeferredCompilationCallback> passedCallback,
-    Worklist* worklist)
+    VM& vm, CodeBlock* codeBlock, CompilationMode mode, unsigned osrEntryBytecodeIndex,
+    const Operands<JSValue>& mustHandleValues,
+    PassRefPtr<DeferredCompilationCallback> passedCallback, Worklist* worklist)
 {
     RefPtr<DeferredCompilationCallback> callback = passedCallback;
     CompilationResult result = compileImpl(
-        exec, codeBlock, mode, osrEntryBytecodeIndex, callback, worklist);
+        vm, codeBlock, mode, osrEntryBytecodeIndex, mustHandleValues, callback, worklist);
     if (result != CompilationDeferred)
         callback->compilationDidComplete(codeBlock, result);
     return result;
index 03204c8..9d43638 100644 (file)
@@ -46,7 +46,10 @@ JS_EXPORT_PRIVATE unsigned getNumCompilations();
 
 // If the worklist is non-null, we do a concurrent compile. Otherwise we do a synchronous
 // compile. Even if we do a synchronous compile, we call the callback with the result.
-CompilationResult compile(ExecState*, CodeBlock*, CompilationMode, unsigned osrEntryBytecodeIndex, PassRefPtr<DeferredCompilationCallback>, Worklist*);
+CompilationResult compile(
+    VM&, CodeBlock*, CompilationMode, unsigned osrEntryBytecodeIndex,
+    const Operands<JSValue>& mustHandleValues,
+    PassRefPtr<DeferredCompilationCallback>, Worklist*);
 
 } } // namespace JSC::DFG
 
index ddaa642..feb27c0 100644 (file)
@@ -844,6 +844,9 @@ private:
         case MovHint:
         case MovHintAndCheck:
         case ZombieHint:
+        case CheckTierUpInLoop:
+        case CheckTierUpAtReturn:
+        case CheckTierUpAndOSREnter:
             RELEASE_ASSERT_NOT_REACHED();
             break;
         
@@ -896,6 +899,8 @@ private:
         case ForceOSRExit:
         case CheckWatchdogTimer:
         case Unreachable:
+        case ExtractOSREntryLocal:
+        case LoopHint:
             break;
 #else
         default:
index 8baa03c..e2c0561 100644 (file)
@@ -236,6 +236,13 @@ void Graph::dump(PrintStream& out, const char* prefix, Node* node, DumpContext*
         else
             out.print(comma, "r", operand, "(", VariableAccessDataDump(*this, variableAccessData), ")");
     }
+    if (node->hasUnlinkedLocal()) {
+        int operand = node->unlinkedLocal();
+        if (operandIsArgument(operand))
+            out.print(comma, "arg", operandToArgument(operand));
+        else
+            out.print(comma, "r", operand);
+    }
     if (node->hasConstantBuffer()) {
         out.print(comma);
         out.print(node->startConstant(), ":[");
@@ -487,6 +494,29 @@ void Graph::resetReachability()
     determineReachability();
 }
 
+void Graph::killBlockAndItsContents(BasicBlock* block)
+{
+    for (unsigned phiIndex = block->phis.size(); phiIndex--;)
+        m_allocator.free(block->phis[phiIndex]);
+    for (unsigned nodeIndex = block->size(); nodeIndex--;)
+        m_allocator.free(block->at(nodeIndex));
+    
+    killBlock(block);
+}
+
+void Graph::killUnreachableBlocks()
+{
+    for (BlockIndex blockIndex = 0; blockIndex < numBlocks(); ++blockIndex) {
+        BasicBlock* block = this->block(blockIndex);
+        if (!block)
+            continue;
+        if (block->isReachable)
+            continue;
+        
+        killBlockAndItsContents(block);
+    }
+}
+
 void Graph::resetExitStates()
 {
     for (BlockIndex blockIndex = 0; blockIndex < m_blocks.size(); ++blockIndex) {
index 026cffe..d25fa08 100644 (file)
@@ -489,6 +489,10 @@ public:
         killBlock(basicBlock->index);
     }
     
+    void killBlockAndItsContents(BasicBlock*);
+    
+    void killUnreachableBlocks();
+    
     bool isPredictedNumerical(Node* node)
     {
         return isNumerical(node->child1().useKind()) && isNumerical(node->child2().useKind());
@@ -698,6 +702,7 @@ public:
     bool m_hasArguments;
     HashSet<ExecutableBase*> m_executablesWhoseArgumentsEscaped;
     BitVector m_preservedVars;
+    BitVector m_lazyVars;
     Dominators m_dominators;
     NaturalLoops m_naturalLoops;
     unsigned m_localVars;
index f4c4a9f..68530a7 100644 (file)
@@ -107,6 +107,7 @@ void InPlaceAbstractState::initialize()
     root->cfaHasVisited = false;
     root->cfaFoundConstants = false;
     for (size_t i = 0; i < root->valuesAtHead.numberOfArguments(); ++i) {
+        root->valuesAtTail.argument(i).clear();
         if (m_graph.m_form == SSA) {
             root->valuesAtHead.argument(i).makeTop();
             continue;
@@ -129,8 +130,6 @@ void InPlaceAbstractState::initialize()
             root->valuesAtHead.argument(i).setType(SpecCell);
         else
             root->valuesAtHead.argument(i).makeTop();
-        
-        root->valuesAtTail.argument(i).clear();
     }
     for (size_t i = 0; i < root->valuesAtHead.numberOfLocals(); ++i) {
         Node* node = root->variablesAtHead.local(i);
index ba80553..a3f1f37 100644 (file)
@@ -28,6 +28,8 @@
 
 #if ENABLE(DFG_JIT)
 
+#include "CodeBlock.h"
+
 namespace JSC { namespace DFG {
 
 JITCode::JITCode()
@@ -59,6 +61,145 @@ void JITCode::shrinkToFit()
     variableEventStream.shrinkToFit();
 }
 
+void JITCode::reconstruct(
+    CodeBlock* codeBlock, CodeOrigin codeOrigin, unsigned streamIndex,
+    Operands<ValueRecovery>& result)
+{
+    variableEventStream.reconstruct(
+        codeBlock, codeOrigin, minifiedDFG, streamIndex, result);
+}
+
+void JITCode::reconstruct(
+    ExecState* exec, CodeBlock* codeBlock, CodeOrigin codeOrigin, unsigned streamIndex,
+    Operands<JSValue>& result)
+{
+    Operands<ValueRecovery> recoveries;
+    reconstruct(codeBlock, codeOrigin, streamIndex, recoveries);
+    
+    result = Operands<JSValue>(OperandsLike, recoveries);
+    for (size_t i = result.size(); i--;) {
+        int operand = result.operandForIndex(i);
+        
+        if (operandIsArgument(operand)
+            && !operandToArgument(operand)
+            && codeBlock->codeType() == FunctionCode
+            && codeBlock->specializationKind() == CodeForConstruct) {
+            // Ugh. If we're in a constructor, the 'this' argument may hold garbage. It will
+            // also never be used. It doesn't matter what we put into the value for this,
+            // but it has to be an actual value that can be grokked by subsequent DFG passes,
+            // so we sanitize it here by turning it into Undefined.
+            result[i] = jsUndefined();
+            continue;
+        }
+        
+        ValueRecovery recovery = recoveries[i];
+        JSValue value;
+        switch (recovery.technique()) {
+        case AlreadyInJSStack:
+        case AlreadyInJSStackAsUnboxedCell:
+        case AlreadyInJSStackAsUnboxedBoolean:
+            value = exec->r(operand).jsValue();
+            break;
+        case AlreadyInJSStackAsUnboxedInt32:
+            value = jsNumber(exec->r(operand).unboxedInt32());
+            break;
+        case AlreadyInJSStackAsUnboxedDouble:
+            value = jsDoubleNumber(exec->r(operand).unboxedDouble());
+            break;
+        case Constant:
+            value = recovery.constant();
+            break;
+        default:
+            RELEASE_ASSERT_NOT_REACHED();
+            break;
+        }
+        result[i] = value;
+    }
+}
+
+#if ENABLE(FTL_JIT)
+bool JITCode::checkIfOptimizationThresholdReached(CodeBlock* codeBlock)
+{
+    ASSERT(codeBlock->jitType() == JITCode::DFGJIT);
+    return tierUpCounter.checkIfThresholdCrossedAndSet(codeBlock->baselineVersion());
+}
+
+void JITCode::optimizeNextInvocation(CodeBlock* codeBlock)
+{
+    ASSERT(codeBlock->jitType() == JITCode::DFGJIT);
+    if (Options::verboseOSR())
+        dataLog(*codeBlock, ": FTL-optimizing next invocation.\n");
+    tierUpCounter.setNewThreshold(0, codeBlock->baselineVersion());
+}
+
+void JITCode::dontOptimizeAnytimeSoon(CodeBlock* codeBlock)
+{
+    ASSERT(codeBlock->jitType() == JITCode::DFGJIT);
+    if (Options::verboseOSR())
+        dataLog(*codeBlock, ": Not FTL-optimizing anytime soon.\n");
+    tierUpCounter.deferIndefinitely();
+}
+
+void JITCode::optimizeAfterWarmUp(CodeBlock* codeBlock)
+{
+    ASSERT(codeBlock->jitType() == JITCode::DFGJIT);
+    if (Options::verboseOSR())
+        dataLog(*codeBlock, ": FTL-optimizing after warm-up.\n");
+    CodeBlock* baseline = codeBlock->baselineVersion();
+    tierUpCounter.setNewThreshold(
+        baseline->adjustedCounterValue(Options::thresholdForFTLOptimizeAfterWarmUp()),
+        baseline);
+}
+
+void JITCode::optimizeSoon(CodeBlock* codeBlock)
+{
+    ASSERT(codeBlock->jitType() == JITCode::DFGJIT);
+    if (Options::verboseOSR())
+        dataLog(*codeBlock, ": FTL-optimizing soon.\n");
+    CodeBlock* baseline = codeBlock->baselineVersion();
+    tierUpCounter.setNewThreshold(
+        baseline->adjustedCounterValue(Options::thresholdForFTLOptimizeSoon()),
+        baseline);
+}
+
+void JITCode::forceOptimizationSlowPathConcurrently(CodeBlock* codeBlock)
+{
+    ASSERT(codeBlock->jitType() == JITCode::DFGJIT);
+    if (Options::verboseOSR())
+        dataLog(*codeBlock, ": Forcing slow path concurrently for FTL entry.\n");
+    tierUpCounter.forceSlowPathConcurrently();
+}
+
+void JITCode::setOptimizationThresholdBasedOnCompilationResult(
+    CodeBlock* codeBlock, CompilationResult result)
+{
+    ASSERT(codeBlock->jitType() == JITCode::DFGJIT);
+    switch (result) {
+    case CompilationSuccessful:
+        optimizeNextInvocation(codeBlock);
+        return;
+    case CompilationFailed:
+        dontOptimizeAnytimeSoon(codeBlock);
+        codeBlock->baselineVersion()->m_didFailFTLCompilation = true;
+        return;
+    case CompilationDeferred:
+        optimizeAfterWarmUp(codeBlock);
+        return;
+    case CompilationInvalidated:
+        // This is weird - it will only happen in cases when the DFG code block (i.e.
+        // the code block that this JITCode belongs to) is also invalidated. So it
+        // doesn't really matter what we do. But, we do the right thing anyway. Note
+        // that us counting the reoptimization actually means that we might count it
+        // twice. But that's generally OK. It's better to overcount reoptimizations
+        // than it is to undercount them.
+        codeBlock->baselineVersion()->countReoptimization();
+        optimizeAfterWarmUp(codeBlock);
+        return;
+    }
+    RELEASE_ASSERT_NOT_REACHED();
+}
+#endif // ENABLE(FTL_JIT)
+
 } } // namespace JSC::DFG
 
 #endif // ENABLE(DFG_JIT)
index fb2f2b6..864cca8 100644 (file)
 
 #if ENABLE(DFG_JIT)
 
+#include "CompilationResult.h"
 #include "DFGCommonData.h"
 #include "DFGMinifiedGraph.h"
 #include "DFGOSREntry.h"
 #include "DFGOSRExit.h"
 #include "DFGVariableEventStream.h"
+#include "ExecutionCounter.h"
 #include "JITCode.h"
 #include "JumpReplacementWatchpoint.h"
 #include <wtf/SegmentedVector.h>
@@ -93,8 +95,30 @@ public:
         return result;
     }
     
+    void reconstruct(
+        CodeBlock*, CodeOrigin, unsigned streamIndex, Operands<ValueRecovery>& result);
+    
+    // This is only applicable if we're at a point where all values are spilled to the
+    // stack. Currently, it also has the restriction that the values must be in their
+    // bytecode-designated stack slots.
+    void reconstruct(
+        ExecState*, CodeBlock*, CodeOrigin, unsigned streamIndex, Operands<JSValue>& result);
+
+#if ENABLE(FTL_JIT)
+    // NB. All of these methods take CodeBlock* because they may want to use
+    // CodeBlock's logic about scaling thresholds. It should be a DFG CodeBlock.
+    
+    bool checkIfOptimizationThresholdReached(CodeBlock*);
+    void optimizeNextInvocation(CodeBlock*);
+    void dontOptimizeAnytimeSoon(CodeBlock*);
+    void optimizeAfterWarmUp(CodeBlock*);
+    void optimizeSoon(CodeBlock*);
+    void forceOptimizationSlowPathConcurrently(CodeBlock*);
+    void setOptimizationThresholdBasedOnCompilationResult(CodeBlock*, CompilationResult);
+#endif // ENABLE(FTL_JIT)
+    
     void shrinkToFit();
-        
+    
 private:
     friend class JITCompiler; // Allow JITCompiler to call setCodeRef().
 
@@ -106,6 +130,10 @@ public:
     SegmentedVector<JumpReplacementWatchpoint, 1, 0> watchpoints;
     DFG::VariableEventStream variableEventStream;
     DFG::MinifiedGraph minifiedDFG;
+#if ENABLE(FTL_JIT)
+    ExecutionCounter tierUpCounter;
+    RefPtr<CodeBlock> osrEntryBlock;
+#endif // ENABLE(FTL_JIT)
 };
 
 } } // namespace JSC::DFG
index 150eea6..c2d0702 100644 (file)
@@ -48,27 +48,31 @@ JITFinalizer::~JITFinalizer()
 
 bool JITFinalizer::finalize()
 {
-    finalizeCommon();
-    
     m_jitCode->initializeCodeRef(m_linkBuffer->finalizeCodeWithoutDisassembly());
     m_plan.codeBlock->setJITCode(m_jitCode, MacroAssemblerCodePtr());
     
+    finalizeCommon();
+    
     return true;
 }
 
 bool JITFinalizer::finalizeFunction()
 {
-    finalizeCommon();
-    
     MacroAssemblerCodePtr withArityCheck = m_linkBuffer->locationOf(m_arityCheck);
     m_jitCode->initializeCodeRef(m_linkBuffer->finalizeCodeWithoutDisassembly());
     m_plan.codeBlock->setJITCode(m_jitCode, withArityCheck);
     
+    finalizeCommon();
+    
     return true;
 }
 
 void JITFinalizer::finalizeCommon()
 {
+#if ENABLE(FTL_JIT)
+    m_jitCode->optimizeAfterWarmUp(m_plan.codeBlock.get());
+#endif // ENABLE(FTL_JIT)
+    
     if (m_plan.compilation)
         m_plan.vm.m_perBytecodeProfiler->addCompilation(m_plan.compilation);
 }
index 14d8b19..507e00f 100644 (file)
 
 namespace JSC { namespace DFG {
 
+BasicBlock* createPreHeader(Graph& graph, BlockInsertionSet& insertionSet, BasicBlock* block)
+{
+    BasicBlock* preHeader = insertionSet.insertBefore(block);
+    preHeader->appendNode(
+        graph, SpecNone, Jump, block->at(0)->codeOrigin, OpInfo(block));
+    
+    for (unsigned predecessorIndex = 0; predecessorIndex < block->predecessors.size(); predecessorIndex++) {
+        BasicBlock* predecessor = block->predecessors[predecessorIndex];
+        if (graph.m_dominators.dominates(block, predecessor))
+            continue;
+        block->predecessors[predecessorIndex--] = block->predecessors.last();
+        block->predecessors.removeLast();
+        for (unsigned successorIndex = predecessor->numSuccessors(); successorIndex--;) {
+            BasicBlock*& successor = predecessor->successor(successorIndex);
+            if (successor != block)
+                continue;
+            successor = preHeader;
+            preHeader->predecessors.append(predecessor);
+        }
+    }
+    
+    block->predecessors.append(preHeader);
+    return preHeader;
+}
+
 class LoopPreHeaderCreationPhase : public Phase {
 public:
     LoopPreHeaderCreationPhase(Graph& graph)
@@ -70,26 +95,7 @@ public:
             if (!needsNewPreHeader)
                 continue;
             
-            BasicBlock* preHeader = m_insertionSet.insertBefore(loop.header());
-            preHeader->appendNode(
-                m_graph, SpecNone, Jump, loop.header()->at(0)->codeOrigin, OpInfo(loop.header()));
-            
-            for (unsigned predecessorIndex = 0; predecessorIndex < loop.header()->predecessors.size(); predecessorIndex++) {
-                BasicBlock* predecessor = loop.header()->predecessors[predecessorIndex];
-                if (m_graph.m_dominators.dominates(loop.header(), predecessor))
-                    continue;
-                loop.header()->predecessors[predecessorIndex--] = loop.header()->predecessors.last();
-                loop.header()->predecessors.takeLast();
-                for (unsigned successorIndex = predecessor->numSuccessors(); successorIndex--;) {
-                    BasicBlock*& successor = predecessor->successor(successorIndex);
-                    if (successor != loop.header())
-                        continue;
-                    successor = preHeader;
-                    preHeader->predecessors.append(predecessor);
-                }
-            }
-            
-            loop.header()->predecessors.append(preHeader);
+            createPreHeader(m_graph, m_insertionSet, loop.header());
         }
         
         return m_insertionSet.execute();
index cc362d7..a229875 100644 (file)
 
 namespace JSC { namespace DFG {
 
+class BlockInsertionSet;
 class Graph;
+struct BasicBlock;
 
 // Inserts dummy basic blocks before any loop headers that don't already have
 // a single non-loop predecessor.
 
 bool performLoopPreHeaderCreation(Graph&);
 
+// Creates a new basic block (a pre-header) that jumps to the given block. All
+// predecessors of the given block that aren't dominated by it are rerouted to
+// the pre-header.
+//
+// This function is used internally and it's used in a surgical fashion by
+// OSREntrypointCreationPhase.
+//
+// Note that executing this function requires having an intact dominators
+// analysis. You should run that analysis before doing damage to the CFG, even
+// if said damage is done before you call this.
+BasicBlock* createPreHeader(Graph&, BlockInsertionSet&, BasicBlock*);
+
 } } // namespace JSC::DFG
 
 #endif // ENABLE(DFG_JIT)
index d898b70..e55d4be 100644 (file)
@@ -527,9 +527,20 @@ struct Node {
         return variableAccessData()->local();
     }
     
+    bool hasUnlinkedLocal()
+    {
+        switch (op()) {
+        case GetLocalUnlinked:
+        case ExtractOSREntryLocal:
+            return true;
+        default:
+            return false;
+        }
+    }
+    
     VirtualRegister unlinkedLocal()
     {
-        ASSERT(op() == GetLocalUnlinked);
+        ASSERT(hasUnlinkedLocal());
         return static_cast<VirtualRegister>(m_opInfo);
     }
     
index 6f6a462..02f4bb5 100644 (file)
@@ -69,6 +69,21 @@ namespace JSC { namespace DFG {
     macro(Flush, NodeMustGenerate | NodeDoesNotExit) \
     macro(PhantomLocal, NodeMustGenerate | NodeDoesNotExit) \
     \
+    /* Hint that this is where bytecode thinks is a good place to OSR. Note that this */\
+    /* will exist even in inlined loops. This has no execution semantics but it must */\
+    /* survive all DCE. We treat this as being a can-exit because tier-up to FTL may */\
+    /* want all state. */\
+    macro(LoopHint, NodeMustGenerate) \
+    \
+    /* Special node for OSR entry into the FTL. Indicates that we're loading a local */\
+    /* variable from the scratch buffer. */\
+    macro(ExtractOSREntryLocal, NodeResultJS) \
+    \
+    /* Tier-up checks from the DFG to the FTL. */\
+    macro(CheckTierUpInLoop, NodeMustGenerate) \
+    macro(CheckTierUpAndOSREnter, NodeMustGenerate) \
+    macro(CheckTierUpAtReturn, NodeMustGenerate) \
+    \
     /* Get the value of a local variable, without linking into the VariableAccessData */\
     /* network. This is only valid for variable accesses whose predictions originated */\
     /* as something other than a local access, and thus had their own profiling. */\
index 46b65e7..0b70cab 100644 (file)
@@ -40,18 +40,43 @@ namespace JSC { namespace DFG {
 void* prepareOSREntry(ExecState* exec, CodeBlock* codeBlock, unsigned bytecodeIndex)
 {
 #if DFG_ENABLE(OSR_ENTRY)
-    ASSERT(codeBlock->jitType() == JITCode::DFGJIT);
+    ASSERT(JITCode::isOptimizingJIT(codeBlock->jitType()));
     ASSERT(codeBlock->alternative());
     ASSERT(codeBlock->alternative()->jitType() == JITCode::BaselineJIT);
     ASSERT(!codeBlock->jitCodeMap());
 
     if (Options::verboseOSR()) {
         dataLog(
-            "OSR in ", *codeBlock->alternative(), " -> ", *codeBlock,
+            "DFG OSR in ", *codeBlock->alternative(), " -> ", *codeBlock,
             " from bc#", bytecodeIndex, "\n");
     }
     
     VM* vm = &exec->vm();
+    if (codeBlock->jitType() != JITCode::DFGJIT) {
+        RELEASE_ASSERT(codeBlock->jitType() == JITCode::FTLJIT);
+        
+        // When will this happen? We could have:
+        //
+        // - An exit from the FTL JIT into the baseline JIT followed by an attempt
+        //   to reenter. We're fine with allowing this to fail. If it happens
+        //   enough we'll just reoptimize. It basically means that the OSR exit cost
+        //   us dearly and so reoptimizing is the right thing to do.
+        //
+        // - We have recursive code with hot loops. Consider that foo has a hot loop
+        //   that calls itself. We have two foo's on the stack, lets call them foo1
+        //   and foo2, with foo1 having called foo2 from foo's hot loop. foo2 gets
+        //   optimized all the way into the FTL. Then it returns into foo1, and then
+        //   foo1 wants to get optimized. It might reach this conclusion from its
+        //   hot loop and attempt to OSR enter. And we'll tell it that it can't. It
+        //   might be worth addressing this case, but I just think this case will
+        //   be super rare. For now, if it does happen, it'll cause some compilation
+        //   thrashing.
+        
+        if (Options::verboseOSR())
+            dataLog("    OSR failed because the target code block is not DFG.\n");
+        return 0;
+    }
+    
     OSREntryData* entry = codeBlock->jitCode()->dfg()->osrEntryDataForBytecodeIndex(bytecodeIndex);
     
     if (!entry) {
diff --git a/Source/JavaScriptCore/dfg/DFGOSREntrypointCreationPhase.cpp b/Source/JavaScriptCore/dfg/DFGOSREntrypointCreationPhase.cpp
new file mode 100644 (file)
index 0000000..ef4f4bc
--- /dev/null
@@ -0,0 +1,130 @@
+/*
+ * Copyright (C) 2013 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#include "config.h"
+#include "DFGOSREntrypointCreationPhase.h"
+
+#if ENABLE(DFG_JIT)
+
+#include "DFGBasicBlockInlines.h"
+#include "DFGBlockInsertionSet.h"
+#include "DFGGraph.h"
+#include "DFGLoopPreHeaderCreationPhase.h"
+#include "DFGPhase.h"
+#include "Operations.h"
+
+namespace JSC { namespace DFG {
+
+class OSREntrypointCreationPhase : public Phase {
+public:
+    OSREntrypointCreationPhase(Graph& graph)
+        : Phase(graph, "OSR entrypoint creation")
+    {
+    }
+    
+    bool run()
+    {
+        RELEASE_ASSERT(m_graph.m_plan.mode == FTLForOSREntryMode);
+        RELEASE_ASSERT(m_graph.m_form == ThreadedCPS);
+        
+        unsigned bytecodeIndex = m_graph.m_plan.osrEntryBytecodeIndex;
+        RELEASE_ASSERT(bytecodeIndex);
+        RELEASE_ASSERT(bytecodeIndex != UINT_MAX);
+        
+        // Needed by createPreHeader().
+        m_graph.m_dominators.computeIfNecessary(m_graph);
+        
+        CodeBlock* baseline = m_graph.m_profiledBlock;
+        
+        BasicBlock* target = 0;
+        for (unsigned blockIndex = m_graph.numBlocks(); blockIndex--;) {
+            BasicBlock* block = m_graph.block(blockIndex);
+            if (!block)
+                continue;
+            Node* firstNode = block->at(0);
+            if (firstNode->op() == LoopHint
+                && firstNode->codeOrigin == CodeOrigin(bytecodeIndex)) {
+                target = block;
+                break;
+            }
+        }
+
+        if (!target) {
+            // This is a terrible outcome. It shouldn't often happen but it might
+            // happen and so we should defend against it. If it happens, then this
+            // compilation is a failure.
+            return false;
+        }
+        
+        BlockInsertionSet insertionSet(m_graph);
+        
+        BasicBlock* newRoot = insertionSet.insert(0);
+        CodeOrigin codeOrigin = target->at(0)->codeOrigin;
+        
+        for (int argument = 0; argument < baseline->numParameters(); ++argument) {
+            Node* oldNode = target->variablesAtHead.argument(argument);
+            if (!oldNode) {
+                // Just for sanity, always have a SetArgument even if it's not needed.
+                oldNode = m_graph.m_arguments[argument];
+            }
+            Node* node = newRoot->appendNode(
+                m_graph, SpecNone, SetArgument, codeOrigin,
+                OpInfo(oldNode->variableAccessData()));
+            m_graph.m_arguments[argument] = node;
+        }
+        for (int local = 0; local < baseline->m_numCalleeRegisters; ++local) {
+            Node* previousHead = target->variablesAtHead.local(local);
+            if (!previousHead)
+                continue;
+            VariableAccessData* variable = previousHead->variableAccessData();
+            Node* node = newRoot->appendNode(
+                m_graph, variable->prediction(), ExtractOSREntryLocal, codeOrigin,
+                OpInfo(variable->local()));
+            newRoot->appendNode(
+                m_graph, SpecNone, SetLocal, codeOrigin, OpInfo(variable), Edge(node));
+        }
+        
+        newRoot->appendNode(
+            m_graph, SpecNone, Jump, codeOrigin,
+            OpInfo(createPreHeader(m_graph, insertionSet, target)));
+        
+        insertionSet.execute();
+        m_graph.resetReachability();
+        m_graph.killUnreachableBlocks();
+        return true;
+    }
+};
+
+bool performOSREntrypointCreation(Graph& graph)
+{
+    SamplingRegion samplingRegion("DFG OSR Entrypoint Creation");
+    return runPhase<OSREntrypointCreationPhase>(graph);
+}
+
+} } // namespace JSC::DFG
+
+#endif // ENABLE(DFG_JIT)
+
+
diff --git a/Source/JavaScriptCore/dfg/DFGOSREntrypointCreationPhase.h b/Source/JavaScriptCore/dfg/DFGOSREntrypointCreationPhase.h
new file mode 100644 (file)
index 0000000..a763721
--- /dev/null
@@ -0,0 +1,54 @@
+/*
+ * Copyright (C) 2013 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#ifndef DFGOSREntrypointCreationPhase_h
+#define DFGOSREntrypointCreationPhase_h
+
+#include <wtf/Platform.h>
+
+#if ENABLE(DFG_JIT)
+
+namespace JSC { namespace DFG {
+
+class Graph;
+
+// This phase is only for compilation pipelines that go through SSA, and that
+// do a specialized OSR entry compile. Currently that means FTL.
+//
+// Creates an OSR entrypoint that establishes all argument and local variable
+// values. This entrypoint only works for the Plan::osrEntryBytecodeIndex.
+// Unlike ordinary OSR entry into the DFG, this forces all variables slurped
+// in through OSR entry to appear TOP to the CFA. Hence this phase must be run
+// early - before doing any CFA. But this also does some additional hacks that
+// require this to run before unification.
+
+bool performOSREntrypointCreation(Graph&);
+
+} } // namespace JSC::DFG
+
+#endif // ENABLE(DFG_JIT)
+
+#endif // DFGOSREntrypointCreationPhase_h
+
index 802c7e6..7a3b0bd 100644 (file)
 #include "CodeBlock.h"
 #include "CommonSlowPaths.h"
 #include "CopiedSpaceInlines.h"
+#include "DFGDriver.h"
 #include "DFGOSRExit.h"
 #include "DFGRepatch.h"
 #include "DFGThunks.h"
+#include "DFGToFTLDeferredCompilationCallback.h"
+#include "DFGToFTLForOSREntryDeferredCompilationCallback.h"
+#include "DFGWorklist.h"
+#include "FTLForOSREntryJITCode.h"
+#include "FTLOSREntry.h"
 #include "HostCallReturnValue.h"
 #include "GetterSetter.h"
 #include "Interpreter.h"
@@ -2015,6 +2021,187 @@ extern "C" void DFG_OPERATION triggerReoptimizationNow(CodeBlock* codeBlock)
     codeBlock->reoptimize();
 }
 
+#if ENABLE(FTL_JIT)
+void DFG_OPERATION triggerTierUpNow(ExecState* exec)
+{
+    VM* vm = &exec->vm();
+    NativeCallFrameTracer tracer(vm, exec);
+    DeferGC deferGC(vm->heap);
+    CodeBlock* codeBlock = exec->codeBlock();
+    
+    JITCode* jitCode = codeBlock->jitCode()->dfg();
+    
+    if (Options::verboseOSR()) {
+        dataLog(
+            *codeBlock, ": Entered triggerTierUpNow with executeCounter = ",
+            jitCode->tierUpCounter, "\n");
+    }
+    
+    if (codeBlock->baselineVersion()->m_didFailFTLCompilation) {
+        if (Options::verboseOSR())
+            dataLog("Deferring FTL-optimization of ", *codeBlock, " indefinitely because there was an FTL failure.\n");
+        jitCode->dontOptimizeAnytimeSoon(codeBlock);
+        return;
+    }
+    
+    if (!jitCode->checkIfOptimizationThresholdReached(codeBlock)) {
+        if (Options::verboseOSR())
+            dataLog("Choosing not to FTL-optimize ", *codeBlock, " yet.\n");
+        return;
+    }
+    
+    Worklist::State worklistState;
+    if (Worklist* worklist = vm->worklist.get()) {
+        worklistState = worklist->completeAllReadyPlansForVM(
+            *vm, CompilationKey(codeBlock->baselineVersion(), FTLMode));
+    } else
+        worklistState = Worklist::NotKnown;
+    
+    if (worklistState == Worklist::Compiling) {
+        jitCode->setOptimizationThresholdBasedOnCompilationResult(
+            codeBlock, CompilationDeferred);
+        return;
+    }
+    
+    if (codeBlock->hasOptimizedReplacement()) {
+        // That's great, we've compiled the code - next time we call this function,
+        // we'll enter that replacement.
+        jitCode->optimizeSoon(codeBlock);
+        return;
+    }
+    
+    if (worklistState == Worklist::Compiled) {
+        // This means that we finished compiling, but failed somehow; in that case the
+        // thresholds will be set appropriately.
+        if (Options::verboseOSR())
+            dataLog("Code block ", *codeBlock, " was compiled but it doesn't have an optimized replacement.\n");
+        return;
+    }
+
+    // We need to compile the code.
+    compile(
+        *vm, codeBlock->newReplacement().get(), FTLMode, UINT_MAX, Operands<JSValue>(),
+        ToFTLDeferredCompilationCallback::create(codeBlock), vm->ensureWorklist());
+}
+
+char* DFG_OPERATION triggerOSREntryNow(
+    ExecState* exec, int32_t bytecodeIndex, int32_t streamIndex)
+{
+    VM* vm = &exec->vm();
+    NativeCallFrameTracer tracer(vm, exec);
+    DeferGC deferGC(vm->heap);
+    CodeBlock* codeBlock = exec->codeBlock();
+    
+    JITCode* jitCode = codeBlock->jitCode()->dfg();
+    
+    if (Options::verboseOSR()) {
+        dataLog(
+            *codeBlock, ": Entered triggerTierUpNow with executeCounter = ",
+            jitCode->tierUpCounter, "\n");
+    }
+    
+    if (codeBlock->baselineVersion()->m_didFailFTLCompilation) {
+        if (Options::verboseOSR())
+            dataLog("Deferring FTL-optimization of ", *codeBlock, " indefinitely because there was an FTL failure.\n");
+        jitCode->dontOptimizeAnytimeSoon(codeBlock);
+        return 0;
+    }
+    
+    if (!jitCode->checkIfOptimizationThresholdReached(codeBlock)) {
+        if (Options::verboseOSR())
+            dataLog("Choosing not to FTL-optimize ", *codeBlock, " yet.\n");
+        return 0;
+    }
+    
+    Worklist::State worklistState;
+    if (Worklist* worklist = vm->worklist.get()) {
+        worklistState = worklist->completeAllReadyPlansForVM(
+            *vm, CompilationKey(codeBlock->baselineVersion(), FTLForOSREntryMode));
+    } else
+        worklistState = Worklist::NotKnown;
+    
+    if (worklistState == Worklist::Compiling) {
+        ASSERT(!jitCode->osrEntryBlock);
+        jitCode->setOptimizationThresholdBasedOnCompilationResult(
+            codeBlock, CompilationDeferred);
+        return 0;
+    }
+    
+    if (CodeBlock* entryBlock = jitCode->osrEntryBlock.get()) {
+        void* address = FTL::prepareOSREntry(
+            exec, codeBlock, entryBlock, bytecodeIndex, streamIndex);
+        if (address) {
+            jitCode->optimizeSoon(codeBlock);
+            return static_cast<char*>(address);
+        }
+        
+        FTL::ForOSREntryJITCode* entryCode = entryBlock->jitCode()->ftlForOSREntry();
+        entryCode->countEntryFailure();
+        if (entryCode->entryFailureCount() <
+            Options::ftlOSREntryFailureCountForReoptimization()) {
+            
+            jitCode->optimizeSoon(codeBlock);
+            return 0;
+        }
+        
+        // OSR entry failed. Oh no! This implies that we need to retry. We retry
+        // without exponential backoff and we only do this for the entry code block.
+        jitCode->osrEntryBlock.clear();
+        
+        jitCode->optimizeAfterWarmUp(codeBlock);
+        return 0;
+    }
+    
+    if (worklistState == Worklist::Compiled) {
+        // This means that compilation failed and we already set the thresholds.
+        if (Options::verboseOSR())
+            dataLog("Code block ", *codeBlock, " was compiled but it doesn't have an optimized replacement.\n");
+        return 0;
+    }
+
+    // The first order of business is to trigger a for-entry compile.
+    Operands<JSValue> mustHandleValues;
+    jitCode->reconstruct(
+        exec, codeBlock, CodeOrigin(bytecodeIndex), streamIndex, mustHandleValues);
+    CompilationResult forEntryResult = DFG::compile(
+        *vm, codeBlock->newReplacement().get(), FTLForOSREntryMode, bytecodeIndex,
+        mustHandleValues, ToFTLForOSREntryDeferredCompilationCallback::create(codeBlock),
+        vm->ensureWorklist());
+    
+    // But we also want to trigger a replacement compile. Of course, we don't want to
+    // trigger it if we don't need to. Note that this is kind of weird because we might
+    // have just finished an FTL compile and that compile failed or was invalidated.
+    // But this seems uncommon enough that we sort of don't care. It's certainly sound
+    // to fire off another compile right now so long as we're not already compiling and
+    // we don't already have an optimized replacement. Note, we don't do this for
+    // obviously bad cases like global code, where we know that there is a slim chance
+    // of this code being invoked ever again.
+    CompilationKey keyForReplacement(codeBlock->baselineVersion(), FTLMode);
+    if (codeBlock->codeType() != GlobalCode
+        && !codeBlock->hasOptimizedReplacement()
+        && (!vm->worklist.get()
+            || vm->worklist->compilationState(keyForReplacement) == Worklist::NotKnown)) {
+        compile(
+            *vm, codeBlock->newReplacement().get(), FTLMode, UINT_MAX, Operands<JSValue>(),
+            ToFTLDeferredCompilationCallback::create(codeBlock), vm->ensureWorklist());
+    }
+    
+    if (forEntryResult != CompilationSuccessful)
+        return 0;
+    
+    // It's possible that the for-entry compile already succeeded. In that case OSR
+    // entry will succeed unless we ran out of stack. It's not clear what we should do.
+    // We signal to try again after a while if that happens.
+    void* address = FTL::prepareOSREntry(
+        exec, codeBlock, jitCode->osrEntryBlock.get(), bytecodeIndex, streamIndex);
+    if (address)
+        jitCode->optimizeSoon(codeBlock);
+    else
+        jitCode->optimizeAfterWarmUp(codeBlock);
+    return static_cast<char*>(address);
+}
+#endif // ENABLE(FTL_JIT)
+
 } // extern "C"
 } } // namespace JSC::DFG
 
index c9d35a3..6c0171b 100644 (file)
@@ -99,6 +99,7 @@ typedef size_t DFG_OPERATION (*S_DFGOperation_ECC)(ExecState*, JSCell*, JSCell*)
 typedef size_t DFG_OPERATION (*S_DFGOperation_EJ)(ExecState*, EncodedJSValue);
 typedef size_t DFG_OPERATION (*S_DFGOperation_EJJ)(ExecState*, EncodedJSValue, EncodedJSValue);
 typedef size_t DFG_OPERATION (*S_DFGOperation_J)(EncodedJSValue);
+typedef void DFG_OPERATION (*V_DFGOperation_E)(ExecState*);
 typedef void DFG_OPERATION (*V_DFGOperation_EOZD)(ExecState*, JSObject*, int32_t, double);
 typedef void DFG_OPERATION (*V_DFGOperation_EOZJ)(ExecState*, JSObject*, int32_t, EncodedJSValue);
 typedef void DFG_OPERATION (*V_DFGOperation_EC)(ExecState*, JSCell*);
@@ -126,6 +127,7 @@ typedef char* DFG_OPERATION (*P_DFGOperation_EStJ)(ExecState*, Structure*, Encod
 typedef char* DFG_OPERATION (*P_DFGOperation_EStPS)(ExecState*, Structure*, void*, size_t);
 typedef char* DFG_OPERATION (*P_DFGOperation_EStSS)(ExecState*, Structure*, size_t, size_t);
 typedef char* DFG_OPERATION (*P_DFGOperation_EStZ)(ExecState*, Structure*, int32_t);
+typedef char* DFG_OPERATION (*P_DFGOperation_EZZ)(ExecState*, int32_t, int32_t);
 typedef StringImpl* DFG_OPERATION (*I_DFGOperation_EJss)(ExecState*, JSString*);
 typedef JSString* DFG_OPERATION (*Jss_DFGOperation_EZ)(ExecState*, int32_t);
 JSCell* DFG_OPERATION operationStringFromCharCode(ExecState*, int32_t)  WTF_INTERNAL; 
@@ -302,6 +304,11 @@ void DFG_OPERATION debugOperationPrintSpeculationFailure(ExecState*, void*, void
 
 void DFG_OPERATION triggerReoptimizationNow(CodeBlock*) WTF_INTERNAL;
 
+#if ENABLE(FTL_JIT)
+void DFG_OPERATION triggerTierUpNow(ExecState*) WTF_INTERNAL;
+char* DFG_OPERATION triggerOSREntryNow(ExecState*, int32_t bytecodeIndex, int32_t streamIndex) WTF_INTERNAL;
+#endif // ENABLE(FTL_JIT)
+
 } // extern "C"
 
 inline P_DFGOperation_EStZ operationNewTypedArrayWithSizeForType(TypedArrayType type)
index 696dc0e..05e2174 100644 (file)
 #include "DFGLivenessAnalysisPhase.h"
 #include "DFGLoopPreHeaderCreationPhase.h"
 #include "DFGOSRAvailabilityAnalysisPhase.h"
+#include "DFGOSREntrypointCreationPhase.h"
 #include "DFGPredictionInjectionPhase.h"
 #include "DFGPredictionPropagationPhase.h"
 #include "DFGSSAConversionPhase.h"
+#include "DFGTierUpCheckInjectionPhase.h"
 #include "DFGTypeCheckHoistingPhase.h"
 #include "DFGUnificationPhase.h"
 #include "DFGValidate.h"
 #include "DFGVirtualRegisterAllocationPhase.h"
+#include "OperandsInlines.h"
 #include "Operations.h"
 #include <wtf/CurrentTime.h>
 
@@ -81,13 +84,12 @@ static void dumpAndVerifyGraph(Graph& graph, const char* text)
 
 Plan::Plan(
     PassRefPtr<CodeBlock> passedCodeBlock, CompilationMode mode,
-    unsigned osrEntryBytecodeIndex, unsigned numVarsWithValues)
+    unsigned osrEntryBytecodeIndex, const Operands<JSValue>& mustHandleValues)
     : vm(*passedCodeBlock->vm())
     , codeBlock(passedCodeBlock)
     , mode(mode)
     , osrEntryBytecodeIndex(osrEntryBytecodeIndex)
-    , numVarsWithValues(numVarsWithValues)
-    , mustHandleValues(codeBlock->numParameters(), numVarsWithValues)
+    , mustHandleValues(mustHandleValues)
     , compilation(codeBlock->vm()->m_perBytecodeProfiler ? adoptRef(new Profiler::Compilation(codeBlock->vm()->m_perBytecodeProfiler->ensureBytecodesFor(codeBlock.get()), Profiler::DFG)) : 0)
     , identifiers(codeBlock.get())
     , weakReferences(codeBlock.get())
@@ -109,7 +111,7 @@ void Plan::compileInThread(LongLivedState& longLivedState)
     CompilationScope compilationScope;
 
     if (logCompilationChanges())
-        dataLog("DFG(Plan) compiling ", *codeBlock, ", number of instructions = ", codeBlock->instructionCount(), "\n");
+        dataLog("DFG(Plan) compiling ", *codeBlock, " with ", mode, ", number of instructions = ", codeBlock->instructionCount(), "\n");
 
     CompilationPath path = compileInThreadImpl(longLivedState);
 
@@ -133,7 +135,7 @@ void Plan::compileInThread(LongLivedState& longLivedState)
             break;
         }
         double now = currentTimeMS();
-        dataLog("Optimized ", *codeBlock->alternative(), " with ", pathName, " in ", now - before, " ms");
+        dataLog("Optimized ", *codeBlock->alternative(), " using ", mode, " with ", pathName, " in ", now - before, " ms");
         if (path == FTLPath)
             dataLog(" (DFG: ", beforeFTL - before, ", LLVM: ", now - beforeFTL, ")");
         dataLog(".\n");
@@ -142,6 +144,12 @@ void Plan::compileInThread(LongLivedState& longLivedState)
 
 Plan::CompilationPath Plan::compileInThreadImpl(LongLivedState& longLivedState)
 {
+    if (verboseCompilationEnabled() && osrEntryBytecodeIndex != UINT_MAX) {
+        dataLog("\n");
+        dataLog("Compiler must handle OSR entry from bc#", osrEntryBytecodeIndex, " with values: ", mustHandleValues, "\n");
+        dataLog("\n");
+    }
+    
     Graph dfg(vm, *this, longLivedState);
     
     if (!parse(dfg)) {
@@ -162,6 +170,15 @@ Plan::CompilationPath Plan::compileInThreadImpl(LongLivedState& longLivedState)
     performUnification(dfg);
     performPredictionInjection(dfg);
     
+    if (mode == FTLForOSREntryMode) {
+        bool result = performOSREntrypointCreation(dfg);
+        if (!result) {
+            finalizer = adoptPtr(new FailedFinalizer(*this));
+            return FailPath;
+        }
+        performCPSRethreading(dfg);
+    }
+    
     if (validationEnabled())
         validate(dfg);
     
@@ -206,10 +223,19 @@ Plan::CompilationPath Plan::compileInThreadImpl(LongLivedState& longLivedState)
         dfg.m_naturalLoops.computeIfNecessary(dfg);
     }
 
+    switch (mode) {
+    case DFGMode: {
+        performTierUpCheckInjection(dfg);
+        break;
+    }
+    
+    case FTLMode:
+    case FTLForOSREntryMode: {
 #if ENABLE(FTL_JIT)
-    if (Options::useExperimentalFTL()
-        && codeBlock->codeType() == FunctionCode
-        && FTL::canCompile(dfg)) {
+        if (FTL::canCompile(dfg) == FTL::CannotCompile) {
+            finalizer = adoptPtr(new FailedFinalizer(*this));
+            return FailPath;
+        }
         
         performCriticalEdgeBreaking(dfg);
         performLoopPreHeaderCreation(dfg);
@@ -244,10 +270,16 @@ Plan::CompilationPath Plan::compileInThreadImpl(LongLivedState& longLivedState)
         FTL::compile(state);
         FTL::link(state);
         return FTLPath;
-    }
 #else
-    RELEASE_ASSERT(!Options::useExperimentalFTL());
+        RELEASE_ASSERT_NOT_REACHED();
+        break;
 #endif // ENABLE(FTL_JIT)
+    }
+        
+    default:
+        RELEASE_ASSERT_NOT_REACHED();
+        break;
+    }
     
     performCPSRethreading(dfg);
     performDCE(dfg);
index 01d7c3c..a602697 100644 (file)
@@ -56,7 +56,7 @@ class LongLivedState;
 struct Plan : public ThreadSafeRefCounted<Plan> {
     Plan(
         PassRefPtr<CodeBlock>, CompilationMode, unsigned osrEntryBytecodeIndex,
-        unsigned numVarsWithValues);
+        const Operands<JSValue>& mustHandleValues);
     ~Plan();
     
     void compileInThread(LongLivedState&);
@@ -72,7 +72,6 @@ struct Plan : public ThreadSafeRefCounted<Plan> {
     RefPtr<CodeBlock> codeBlock;
     CompilationMode mode;
     const unsigned osrEntryBytecodeIndex;
-    const unsigned numVarsWithValues;
     Operands<JSValue> mustHandleValues;
 
     RefPtr<Profiler::Compilation> compilation;
index c116ef9..a8e403b 100644 (file)
@@ -77,8 +77,8 @@ public:
             if (block->bytecodeBegin != m_graph.m_plan.osrEntryBytecodeIndex)
                 continue;
             for (size_t i = 0; i < m_graph.m_plan.mustHandleValues.size(); ++i) {
-                Node* node = block->variablesAtHead.operand(
-                    m_graph.m_plan.mustHandleValues.operandForIndex(i));
+                int operand = m_graph.m_plan.mustHandleValues.operandForIndex(i);
+                Node* node = block->variablesAtHead.operand(operand);
                 if (!node)
                     continue;
                 ASSERT(node->hasLocal(m_graph));
index cad4c7a..f682510 100644 (file)
@@ -484,7 +484,10 @@ private:
         case ArrayifyToStructure:
         case MovHint:
         case MovHintAndCheck:
-        case ZombieHint: {
+        case ZombieHint:
+        case CheckTierUpInLoop:
+        case CheckTierUpAtReturn:
+        case CheckTierUpAndOSREnter: {
             // This node should never be visible at this stage of compilation. It is
             // inserted by fixup(), which follows this phase.
             RELEASE_ASSERT_NOT_REACHED();
@@ -549,6 +552,11 @@ private:
         case PutGlobalVar:
         case CheckWatchdogTimer:
         case Unreachable:
+        case LoopHint:
+            break;
+            
+        // This gets ignored because it already has a prediction.
+        case ExtractOSREntryLocal:
             break;
             
         // These gets ignored because it doesn't do anything.
index bf99b12..15a6c18 100644 (file)
@@ -233,6 +233,11 @@ bool safeToExecute(AbstractStateType& state, Graph& graph, Node* node)
     case StringFromCharCode:
     case NewTypedArray:
     case Unreachable:
+    case ExtractOSREntryLocal:
+    case CheckTierUpInLoop:
+    case CheckTierUpAtReturn:
+    case CheckTierUpAndOSREnter:
+    case LoopHint:
         return true;
         
     case GetByVal:
index 8e47021..48057e5 100644 (file)
@@ -4823,6 +4823,7 @@ void SpeculativeJIT::compile(Node* node)
         break;
 
     case PhantomLocal:
+    case LoopHint:
         // This is a no-op.
         noResult(node);
         break;
@@ -4835,6 +4836,10 @@ void SpeculativeJIT::compile(Node* node)
     case Phi:
     case Upsilon:
     case GetArgument:
+    case ExtractOSREntryLocal:
+    case CheckTierUpInLoop:
+    case CheckTierUpAtReturn:
+    case CheckTierUpAndOSREnter:
         RELEASE_ASSERT_NOT_REACHED();
         break;
     }
index d30a7bc..d2c51b7 100644 (file)
@@ -4673,6 +4673,7 @@ void SpeculativeJIT::compile(Node* node)
         break;
         
     case PhantomLocal:
+    case LoopHint:
         // This is a no-op.
         noResult(node);
         break;
@@ -4680,11 +4681,75 @@ void SpeculativeJIT::compile(Node* node)
     case Unreachable:
         RELEASE_ASSERT_NOT_REACHED();
         break;
+
+#if ENABLE(FTL_JIT)        
+    case CheckTierUpInLoop: {
+        MacroAssembler::Jump done = m_jit.branchAdd32(
+            MacroAssembler::Signed,
+            TrustedImm32(Options::ftlTierUpCounterIncrementForLoop()),
+            MacroAssembler::AbsoluteAddress(&m_jit.jitCode()->tierUpCounter.m_counter));
+        
+        silentSpillAllRegisters(InvalidGPRReg);
+        m_jit.setupArgumentsExecState();
+        appendCall(triggerTierUpNow);
+        silentFillAllRegisters(InvalidGPRReg);
+        
+        done.link(&m_jit);
+        break;
+    }
+        
+    case CheckTierUpAtReturn: {
+        MacroAssembler::Jump done = m_jit.branchAdd32(
+            MacroAssembler::Signed,
+            TrustedImm32(Options::ftlTierUpCounterIncrementForReturn()),
+            MacroAssembler::AbsoluteAddress(&m_jit.jitCode()->tierUpCounter.m_counter));
+        
+        silentSpillAllRegisters(InvalidGPRReg);
+        m_jit.setupArgumentsExecState();
+        appendCall(triggerTierUpNow);
+        silentFillAllRegisters(InvalidGPRReg);
+        
+        done.link(&m_jit);
+        break;
+    }
+        
+    case CheckTierUpAndOSREnter: {
+        ASSERT(!node->codeOrigin.inlineCallFrame);
+        
+        GPRTemporary temp(this);
+        GPRReg tempGPR = temp.gpr();
+        
+        MacroAssembler::Jump done = m_jit.branchAdd32(
+            MacroAssembler::Signed,
+            TrustedImm32(Options::ftlTierUpCounterIncrementForLoop()),
+            MacroAssembler::AbsoluteAddress(&m_jit.jitCode()->tierUpCounter.m_counter));
+        
+        silentSpillAllRegisters(tempGPR);
+        m_jit.setupArgumentsWithExecState(
+            TrustedImm32(node->codeOrigin.bytecodeIndex),
+            TrustedImm32(m_stream->size()));
+        appendCallSetResult(triggerOSREntryNow, tempGPR);
+        MacroAssembler::Jump dontEnter = m_jit.branchTestPtr(MacroAssembler::Zero, tempGPR);
+        m_jit.jump(tempGPR);
+        dontEnter.link(&m_jit);
+        silentFillAllRegisters(tempGPR);
+        
+        done.link(&m_jit);
+        break;
+    }
+#else // ENABLE(FTL_JIT)
+    case CheckTierUpInLoop:
+    case CheckTierUpAtReturn:
+    case CheckTierUpAndOSREnter:
+        RELEASE_ASSERT_NOT_REACHED();
+        break;
+#endif // ENABLE(FTL_JIT)
         
     case LastNodeType:
     case Phi:
     case Upsilon:
     case GetArgument:
+    case ExtractOSREntryLocal:
         RELEASE_ASSERT_NOT_REACHED();
         break;
     }
diff --git a/Source/JavaScriptCore/dfg/DFGTierUpCheckInjectionPhase.cpp b/Source/JavaScriptCore/dfg/DFGTierUpCheckInjectionPhase.cpp
new file mode 100644 (file)
index 0000000..d51a1f0
--- /dev/null
@@ -0,0 +1,101 @@
+/*
+ * Copyright (C) 2013 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#include "config.h"
+#include "DFGTierUpCheckInjectionPhase.h"
+
+#if ENABLE(DFG_JIT)
+
+#include "DFGGraph.h"
+#include "DFGInsertionSet.h"
+#include "DFGPhase.h"
+#include "FTLCapabilities.h"
+#include "Operations.h"
+
+namespace JSC { namespace DFG {
+
+class TierUpCheckInjectionPhase : public Phase {
+public:
+    TierUpCheckInjectionPhase(Graph& graph)
+        : Phase(graph, "tier-up check injection")
+    {
+    }
+    
+    bool run()
+    {
+        RELEASE_ASSERT(m_graph.m_plan.mode == DFGMode);
+        
+        if (!Options::useExperimentalFTL())
+            return false;
+
+#if ENABLE(FTL_JIT)
+        FTL::CapabilityLevel level = FTL::canCompile(m_graph);
+        if (level == FTL::CannotCompile)
+            return false;
+        
+        InsertionSet insertionSet(m_graph);
+        for (BlockIndex blockIndex = m_graph.numBlocks(); blockIndex--;) {
+            BasicBlock* block = m_graph.block(blockIndex);
+            if (!block)
+                continue;
+            
+            if (block->at(0)->op() == LoopHint) {
+                CodeOrigin codeOrigin = block->at(0)->codeOrigin;
+                NodeType nodeType;
+                if (level == FTL::CanCompileAndOSREnter && !codeOrigin.inlineCallFrame) {
+                    nodeType = CheckTierUpAndOSREnter;
+                    RELEASE_ASSERT(block->bytecodeBegin == codeOrigin.bytecodeIndex);
+                } else
+                    nodeType = CheckTierUpInLoop;
+                insertionSet.insertNode(1, SpecNone, nodeType, codeOrigin);
+            }
+            
+            if (block->last()->op() == Return) {
+                insertionSet.insertNode(
+                    block->size() - 1, SpecNone, CheckTierUpAtReturn, block->last()->codeOrigin);
+            }
+            
+            insertionSet.execute(block);
+        }
+        
+        return true;
+#else // ENABLE(FTL_JIT)
+        RELEASE_ASSERT_NOT_REACHED();
+        return false;
+#endif // ENABLE(FTL_JIT)
+    }
+};
+
+bool performTierUpCheckInjection(Graph& graph)
+{
+    SamplingRegion samplingRegion("DFG Tier-up Check Injection");
+    return runPhase<TierUpCheckInjectionPhase>(graph);
+}
+
+} } // namespace JSC::DFG
+
+#endif // ENABLE(DFG_JIT)
+
+
diff --git a/Source/JavaScriptCore/dfg/DFGTierUpCheckInjectionPhase.h b/Source/JavaScriptCore/dfg/DFGTierUpCheckInjectionPhase.h
new file mode 100644 (file)
index 0000000..f6e799a
--- /dev/null
@@ -0,0 +1,47 @@
+/*
+ * Copyright (C) 2013 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#ifndef DFGTierUpCheckInjectionPhase_h
+#define DFGTierUpCheckInjectionPhase_h
+
+#include <wtf/Platform.h>
+
+#if ENABLE(DFG_JIT)
+
+namespace JSC { namespace DFG {
+
+class Graph;
+
+// This phase checks if the this code block could be recompiled with the FTL,
+// and if so, it injects tier-up checks.
+
+bool performTierUpCheckInjection(Graph&);
+
+} } // namespace JSC::DFG
+
+#endif // ENABLE(DFG_JIT)
+
+#endif // DFGTierUpCheckInjectionPhase_h
+
diff --git a/Source/JavaScriptCore/dfg/DFGToFTLDeferredCompilationCallback.cpp b/Source/JavaScriptCore/dfg/DFGToFTLDeferredCompilationCallback.cpp
new file mode 100644 (file)
index 0000000..1cb32d7
--- /dev/null
@@ -0,0 +1,82 @@
+/*
+ * Copyright (C) 2013 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#include "config.h"
+#include "DFGToFTLDeferredCompilationCallback.h"
+
+#if ENABLE(FTL_JIT)
+
+#include "CodeBlock.h"
+#include "DFGJITCode.h"
+#include "Executable.h"
+
+namespace JSC { namespace DFG {
+
+ToFTLDeferredCompilationCallback::ToFTLDeferredCompilationCallback(
+    PassRefPtr<CodeBlock> dfgCodeBlock)
+    : m_dfgCodeBlock(dfgCodeBlock)
+{
+}
+
+ToFTLDeferredCompilationCallback::~ToFTLDeferredCompilationCallback() { }
+
+PassRefPtr<ToFTLDeferredCompilationCallback> ToFTLDeferredCompilationCallback::create(
+    PassRefPtr<CodeBlock> dfgCodeBlock)
+{
+    return adoptRef(new ToFTLDeferredCompilationCallback(dfgCodeBlock));
+}
+
+void ToFTLDeferredCompilationCallback::compilationDidBecomeReadyAsynchronously(
+    CodeBlock* codeBlock)
+{
+    if (Options::verboseOSR()) {
+        dataLog(
+            "Optimizing compilation of ", *codeBlock, " (for ", *m_dfgCodeBlock,
+            ") did become ready.\n");
+    }
+    
+    m_dfgCodeBlock->jitCode()->dfg()->forceOptimizationSlowPathConcurrently(
+        m_dfgCodeBlock.get());
+}
+
+void ToFTLDeferredCompilationCallback::compilationDidComplete(
+    CodeBlock* codeBlock, CompilationResult result)
+{
+    if (Options::verboseOSR()) {
+        dataLog(
+            "Optimizing compilation of ", *codeBlock, " (for ", *m_dfgCodeBlock,
+            ") result: ", result, "\n");
+    }
+    
+    if (result == CompilationSuccessful)
+        codeBlock->install();
+    
+    m_dfgCodeBlock->jitCode()->dfg()->setOptimizationThresholdBasedOnCompilationResult(
+        m_dfgCodeBlock.get(), result);
+}
+
+} } // JSC::DFG
+
+#endif // ENABLE(FTL_JIT)
diff --git a/Source/JavaScriptCore/dfg/DFGToFTLDeferredCompilationCallback.h b/Source/JavaScriptCore/dfg/DFGToFTLDeferredCompilationCallback.h
new file mode 100644 (file)
index 0000000..a4d840b
--- /dev/null
@@ -0,0 +1,65 @@
+/*
+ * Copyright (C) 2013 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#ifndef DFGToFTLDeferredCompilationCallback_h
+#define DFGToFTLDeferredCompilationCallback_h
+
+#include <wtf/Platform.h>
+
+#if ENABLE(FTL_JIT)
+
+#include "DeferredCompilationCallback.h"
+#include <wtf/PassRefPtr.h>
+#include <wtf/RefPtr.h>
+
+namespace JSC {
+
+class ScriptExecutable;
+
+namespace DFG {
+
+class ToFTLDeferredCompilationCallback : public DeferredCompilationCallback {
+protected:
+    ToFTLDeferredCompilationCallback(PassRefPtr<CodeBlock> dfgCodeBlock);
+
+public:
+    virtual ~ToFTLDeferredCompilationCallback();
+
+    static PassRefPtr<ToFTLDeferredCompilationCallback> create(
+        PassRefPtr<CodeBlock> dfgCodeBlock);
+    
+    virtual void compilationDidBecomeReadyAsynchronously(CodeBlock*);
+    virtual void compilationDidComplete(CodeBlock*, CompilationResult);
+
+private:
+    RefPtr<CodeBlock> m_dfgCodeBlock;
+};
+
+} } // namespace JSC::DFG
+
+#endif // ENABLE(FTL_JIT)
+
+#endif // DFGToFTLDeferredCompilationCallback_h
+
diff --git a/Source/JavaScriptCore/dfg/DFGToFTLForOSREntryDeferredCompilationCallback.cpp b/Source/JavaScriptCore/dfg/DFGToFTLForOSREntryDeferredCompilationCallback.cpp
new file mode 100644 (file)
index 0000000..17b45a3
--- /dev/null
@@ -0,0 +1,87 @@
+/*
+ * Copyright (C) 2013 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#include "config.h"
+#include "DFGToFTLForOSREntryDeferredCompilationCallback.h"
+
+#if ENABLE(FTL_JIT)
+
+#include "CodeBlock.h"
+#include "DFGJITCode.h"
+#include "Executable.h"
+
+namespace JSC { namespace DFG {
+
+ToFTLForOSREntryDeferredCompilationCallback::ToFTLForOSREntryDeferredCompilationCallback(
+    PassRefPtr<CodeBlock> dfgCodeBlock)
+    : m_dfgCodeBlock(dfgCodeBlock)
+{
+}
+
+ToFTLForOSREntryDeferredCompilationCallback::~ToFTLForOSREntryDeferredCompilationCallback()
+{
+}
+
+PassRefPtr<ToFTLForOSREntryDeferredCompilationCallback>
+ToFTLForOSREntryDeferredCompilationCallback::create(
+    PassRefPtr<CodeBlock> dfgCodeBlock)
+{
+    return adoptRef(new ToFTLForOSREntryDeferredCompilationCallback(dfgCodeBlock));
+}
+
+void ToFTLForOSREntryDeferredCompilationCallback::compilationDidBecomeReadyAsynchronously(
+    CodeBlock* codeBlock)
+{
+    if (Options::verboseOSR()) {
+        dataLog(
+            "Optimizing compilation of ", *codeBlock, " (for ", *m_dfgCodeBlock,
+            ") did become ready.\n");
+    }
+    
+    m_dfgCodeBlock->jitCode()->dfg()->forceOptimizationSlowPathConcurrently(
+        m_dfgCodeBlock.get());
+}
+
+void ToFTLForOSREntryDeferredCompilationCallback::compilationDidComplete(
+    CodeBlock* codeBlock, CompilationResult result)
+{
+    if (Options::verboseOSR()) {
+        dataLog(
+            "Optimizing compilation of ", *codeBlock, " (for ", *m_dfgCodeBlock,
+            ") result: ", result, "\n");
+    }
+    
+    if (result == CompilationSuccessful)
+        m_dfgCodeBlock->jitCode()->dfg()->osrEntryBlock = codeBlock;
+    
+    // FIXME: if we failed, we might want to just turn off OSR entry rather than
+    // totally turning off tier-up.
+    m_dfgCodeBlock->jitCode()->dfg()->setOptimizationThresholdBasedOnCompilationResult(
+        m_dfgCodeBlock.get(), result);
+}
+
+} } // JSC::DFG
+
+#endif // ENABLE(FTL_JIT)
diff --git a/Source/JavaScriptCore/dfg/DFGToFTLForOSREntryDeferredCompilationCallback.h b/Source/JavaScriptCore/dfg/DFGToFTLForOSREntryDeferredCompilationCallback.h
new file mode 100644 (file)
index 0000000..af6b97b
--- /dev/null
@@ -0,0 +1,65 @@
+/*
+ * Copyright (C) 2013 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#ifndef DFGToFTLForOSREntryDeferredCompilationCallback_h
+#define DFGToFTLForOSREntryDeferredCompilationCallback_h
+
+#include <wtf/Platform.h>
+
+#if ENABLE(FTL_JIT)
+
+#include "DeferredCompilationCallback.h"
+#include <wtf/PassRefPtr.h>
+#include <wtf/RefPtr.h>
+
+namespace JSC {
+
+class ScriptExecutable;
+
+namespace DFG {
+
+class ToFTLForOSREntryDeferredCompilationCallback : public DeferredCompilationCallback {
+protected:
+    ToFTLForOSREntryDeferredCompilationCallback(PassRefPtr<CodeBlock> dfgCodeBlock);
+
+public:
+    virtual ~ToFTLForOSREntryDeferredCompilationCallback();
+
+    static PassRefPtr<ToFTLForOSREntryDeferredCompilationCallback> create(
+        PassRefPtr<CodeBlock> dfgCodeBlock);
+    
+    virtual void compilationDidBecomeReadyAsynchronously(CodeBlock*);
+    virtual void compilationDidComplete(CodeBlock*, CompilationResult);
+
+private:
+    RefPtr<CodeBlock> m_dfgCodeBlock;
+};
+
+} } // namespace JSC::DFG
+
+#endif // ENABLE(FTL_JIT)
+
+#endif // DFGToFTLForOSREntryDeferredCompilationCallback_h
+
index 3f434be..98de6fa 100644 (file)
@@ -277,8 +277,6 @@ static void initializeGlobalWorklistOnce()
 
 Worklist* globalWorklist()
 {
-    if (!enableConcurrentJIT())
-        return 0;
     pthread_once(&initializeGlobalWorklistKeyOnce, initializeGlobalWorklistOnce);
     return theGlobalWorklist;
 }
index f8f586d..208f246 100644 (file)
@@ -98,9 +98,6 @@ private:
 // For now we use a single global worklist. It's not clear that this
 // is the right thing to do, but it is what we do, for now. This function
 // will lazily create one when it's needed.
-//
-// This returns null if for any reason we shouldn't be doing concurrent
-// compilation.
 Worklist* globalWorklist();
 
 } } // namespace JSC::DFG
index 2ff3038..b81f370 100644 (file)
@@ -32,8 +32,11 @@ namespace JSC { namespace FTL {
 
 using namespace DFG;
 
-inline bool canCompile(Node* node)
+inline CapabilityLevel canCompile(Node* node)
 {
+    // NOTE: If we ever have phantom arguments, we can compile them but we cannot
+    // OSR enter.
+    
     switch (node->op()) {
     case JSConstant:
     case WeakJSConstant:
@@ -81,6 +84,8 @@ inline bool canCompile(Node* node)
     case ForceOSRExit:
     case Phi:
     case Upsilon:
+    case ExtractOSREntryLocal:
+    case LoopHint:
         // These are OK.
         break;
     case GetArrayLength:
@@ -90,39 +95,39 @@ inline bool canCompile(Node* node)
         case Array::Contiguous:
             break;
         default:
-            return false;
+            return CannotCompile;
         }
         break;
     case GetByVal:
         switch (node->arrayMode().type()) {
         case Array::ForceExit:
-            return true;
+            return CanCompileAndOSREnter;
         case Array::Int32:
         case Array::Double:
         case Array::Contiguous:
             break;
         default:
-            return false;
+            return CannotCompile;
         }
         switch (node->arrayMode().speculation()) {
         case Array::SaneChain:
         case Array::InBounds:
             break;
         default:
-            return false;
+            return CannotCompile;
         }
         break;
     case PutByVal:
     case PutByValAlias:
         switch (node->arrayMode().type()) {
         case Array::ForceExit:
-            return true;
+            return CanCompileAndOSREnter;
         case Array::Int32:
         case Array::Double:
         case Array::Contiguous:
             break;
         default:
-            return false;
+            return CannotCompile;
         }
         break;
     case CompareEq:
@@ -133,7 +138,7 @@ inline bool canCompile(Node* node)
             break;
         if (node->isBinaryUseKind(ObjectUse))
             break;
-        return false;
+        return CannotCompile;
     case CompareLess:
     case CompareLessEq:
     case CompareGreater:
@@ -142,7 +147,7 @@ inline bool canCompile(Node* node)
             break;
         if (node->isBinaryUseKind(NumberUse))
             break;
-        return false;
+        return CannotCompile;
     case Branch:
     case LogicalNot:
         switch (node->child1().useKind()) {
@@ -152,7 +157,7 @@ inline bool canCompile(Node* node)
         case ObjectOrOtherUse:
             break;
         default:
-            return false;
+            return CannotCompile;
         }
         break;
     case Switch:
@@ -161,18 +166,26 @@ inline bool canCompile(Node* node)
         case SwitchChar:
             break;
         default:
-            return false;
+            return CannotCompile;
         }
         break;
     default:
         // Don't know how to handle anything else.
-        return false;
+        return CannotCompile;
     }
-    return true;
+    return CanCompileAndOSREnter;
 }
 
-bool canCompile(Graph& graph)
+CapabilityLevel canCompile(Graph& graph)
 {
+    if (graph.m_codeBlock->codeType() != FunctionCode) {
+        if (verboseCompilationEnabled())
+            dataLog("FTL rejecting code block that doesn't belong to a function.\n");
+        return CannotCompile;
+    }
+    
+    CapabilityLevel result = CanCompileAndOSREnter;
+    
     for (BlockIndex blockIndex = graph.numBlocks(); blockIndex--;) {
         BasicBlock* block = graph.block(blockIndex);
         if (!block)
@@ -210,25 +223,36 @@ bool canCompile(Graph& graph)
                         dataLog("FTL rejecting node because of bad use kind: ", edge.useKind(), " in node:\n");
                         graph.dump(WTF::dataFile(), "    ", node);
                     }
-                    return false;
+                    return CannotCompile;
                 }
             }
             
-            if (!canCompile(node)) {
+            switch (canCompile(node)) {
+            case CannotCompile: 
                 if (verboseCompilationEnabled()) {
                     dataLog("FTL rejecting node:\n");
                     graph.dump(WTF::dataFile(), "    ", node);
                 }
-                return false;
+                return CannotCompile;
+                
+            case CanCompile:
+                if (result == CanCompileAndOSREnter && verboseCompilationEnabled()) {
+                    dataLog("FTL disabling OSR entry because of node:\n");
+                    graph.dump(WTF::dataFile(), "    ", node);
+                }
+                result = CanCompile;
+                break;
+                
+            case CanCompileAndOSREnter:
+                break;
             }
             
-            // We don't care if we can compile anything after a force-exit.
             if (node->op() == ForceOSRExit)
                 break;
         }
     }
     
-    return true;
+    return result;
 }
 
 } } // namespace JSC::FTL
index d999917..75a426d 100644 (file)
 
 namespace JSC { namespace FTL {
 
-bool canCompile(DFG::Graph&);
+enum CapabilityLevel {
+    CannotCompile,
+    CanCompile,
+    CanCompileAndOSREnter
+};
+
+CapabilityLevel canCompile(DFG::Graph&);
 
 } } // namespace JSC::FTL
 
diff --git a/Source/JavaScriptCore/ftl/FTLForOSREntryJITCode.cpp b/Source/JavaScriptCore/ftl/FTLForOSREntryJITCode.cpp
new file mode 100644 (file)
index 0000000..708bedc
--- /dev/null
@@ -0,0 +1,54 @@
+/*
+ * Copyright (C) 2013 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#include "config.h"
+#include "FTLForOSREntryJITCode.h"
+
+#if ENABLE(FTL_JIT)
+
+namespace JSC { namespace FTL {
+
+ForOSREntryJITCode::ForOSREntryJITCode()
+{
+}
+
+ForOSREntryJITCode::~ForOSREntryJITCode()
+{
+}
+
+ForOSREntryJITCode* ForOSREntryJITCode::ftlForOSREntry()
+{
+    return this;
+}
+
+void ForOSREntryJITCode::initializeEntryBuffer(VM& vm, unsigned numCalleeRegisters)
+{
+    m_entryBuffer = vm.scratchBufferForSize(numCalleeRegisters * sizeof(EncodedJSValue));
+}
+
+} } // namespace JSC::FTL
+
+#endif // ENABLE(FTL_JIT)
+
diff --git a/Source/JavaScriptCore/ftl/FTLForOSREntryJITCode.h b/Source/JavaScriptCore/ftl/FTLForOSREntryJITCode.h
new file mode 100644 (file)
index 0000000..8d863a1
--- /dev/null
@@ -0,0 +1,73 @@
+/*
+ * Copyright (C) 2013 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#ifndef FTLForOSREntryJITCode_h
+#define FTLForOSREntryJITCode_h
+
+#include <wtf/Platform.h>
+
+#if ENABLE(FTL_JIT)
+
+#include "FTLJITCode.h"
+
+namespace JSC { namespace FTL {
+
+// OSR entry into the FTL has a number of quirks:
+//
+// - OSR entry only happens through special OSR entry compilations. They have their
+//   own CodeBlock and their own JITCode.
+//
+// - We only OSR enter in loop headers that have a null inline call frame.
+//
+// - Each OSR entry compilation allows entry through only one bytecode index.
+
+class ForOSREntryJITCode : public FTL::JITCode {
+public:
+    ForOSREntryJITCode();
+    ~ForOSREntryJITCode();
+    
+    void initializeEntryBuffer(VM&, unsigned numCalleeRegisters);
+    ScratchBuffer* entryBuffer() const { return m_entryBuffer; }
+    
+    void setBytecodeIndex(unsigned value) { m_bytecodeIndex = value; }
+    unsigned bytecodeIndex() const { return m_bytecodeIndex; }
+    
+    void countEntryFailure() { m_entryFailureCount++; }
+    unsigned entryFailureCount() const { return m_entryFailureCount; }
+    
+    ForOSREntryJITCode* ftlForOSREntry();
+    
+private:
+    ScratchBuffer* m_entryBuffer; // Only for OSR entry code blocks.
+    unsigned m_bytecodeIndex;
+    unsigned m_entryFailureCount;
+};
+
+} } // namespace JSC::FTL
+
+#endif // ENABLE(FLT_JIT)
+
+#endif // FTLForOSREntryJITCode_h
+
index 077e08a..b23c5b2 100644 (file)
@@ -64,7 +64,9 @@ bool JITFinalizer::finalizeFunction()
                 ("FTL exit thunks for %s", toCString(CodeBlockWithJITType(m_plan.codeBlock.get(), JITCode::FTLJIT)).data())));
     } // else this function had no OSR exits, so no exit thunks.
     
-    MacroAssemblerCodePtr withArityCheck = m_entrypointLinkBuffer->locationOf(m_arityCheck);
+    MacroAssemblerCodePtr withArityCheck;
+    if (m_arityCheck.isSet())
+        withArityCheck = m_entrypointLinkBuffer->locationOf(m_arityCheck);
     m_jitCode->initializeCode(
         FINALIZE_DFG_CODE(
             *m_entrypointLinkBuffer,
index 5b8d33a..1ea5ddf 100644 (file)
@@ -55,79 +55,114 @@ void link(State& state)
     // LLVM will create its own jump tables as needed.
     codeBlock->clearSwitchJumpTables();
     
-    // Create the entrypoint.
-    // FIXME: This is a total kludge - LLVM should just use our calling convention.
+    // Create the entrypoint. Note that we use this entrypoint totally differently
+    // depending on whether we're doing OSR entry or not.
+    // FIXME: Except for OSR entry, this is a total kludge - LLVM should just use our
+    // calling convention.
     // https://bugs.webkit.org/show_bug.cgi?id=113621
     CCallHelpers jit(&state.graph.m_vm, codeBlock);
     
-    compileEntry(jit);
+    OwnPtr<LinkBuffer> linkBuffer;
+    CCallHelpers::Label arityCheck;
+    switch (state.graph.m_plan.mode) {
+    case FTLMode: {
+        compileEntry(jit);
     
-    // This part is only necessary for functions. We currently only compile functions.
+        // This part is only necessary for functions. We currently only compile functions.
         
-    CCallHelpers::Label fromArityCheck = jit.label();
+        CCallHelpers::Label fromArityCheck = jit.label();
         
-    // Plant a check that sufficient space is available in the JSStack.
-    // FIXME: https://bugs.webkit.org/show_bug.cgi?id=56291
-    jit.addPtr(
-        CCallHelpers::TrustedImm32(codeBlock->m_numCalleeRegisters * sizeof(Register)),
-        GPRInfo::callFrameRegister, GPRInfo::regT1);
-    CCallHelpers::Jump stackCheck = jit.branchPtr(
-        CCallHelpers::Below,
-        CCallHelpers::AbsoluteAddress(state.graph.m_vm.interpreter->stack().addressOfEnd()),
-        GPRInfo::regT1);
-    CCallHelpers::Label fromStackCheck = jit.label();
+        // Plant a check that sufficient space is available in the JSStack.
+        // FIXME: https://bugs.webkit.org/show_bug.cgi?id=56291
+        jit.addPtr(
+            CCallHelpers::TrustedImm32(codeBlock->m_numCalleeRegisters * sizeof(Register)),
+            GPRInfo::callFrameRegister, GPRInfo::regT1);
+        CCallHelpers::Jump stackCheck = jit.branchPtr(
+            CCallHelpers::Below,
+            CCallHelpers::AbsoluteAddress(state.graph.m_vm.interpreter->stack().addressOfEnd()),
+            GPRInfo::regT1);
+        CCallHelpers::Label fromStackCheck = jit.label();
         
-    jit.setupArgumentsExecState();
-    jit.move(CCallHelpers::TrustedImmPtr(reinterpret_cast<void*>(state.generatedFunction)), GPRInfo::nonArgGPR0);
-    jit.call(GPRInfo::nonArgGPR0);
-    jit.emitGetFromCallFrameHeaderPtr(JSStack::ReturnPC, GPRInfo::regT1);
-    jit.emitGetFromCallFrameHeaderPtr(JSStack::CallerFrame, GPRInfo::callFrameRegister);
-    jit.restoreReturnAddressBeforeReturn(GPRInfo::regT1);
-    jit.ret();
+        jit.setupArgumentsExecState();
+        jit.move(
+            CCallHelpers::TrustedImmPtr(reinterpret_cast<void*>(state.generatedFunction)),
+            GPRInfo::nonArgGPR0);
+        jit.call(GPRInfo::nonArgGPR0);
+        jit.emitGetFromCallFrameHeaderPtr(JSStack::ReturnPC, GPRInfo::regT1);
+        jit.emitGetFromCallFrameHeaderPtr(JSStack::CallerFrame, GPRInfo::callFrameRegister);
+        jit.restoreReturnAddressBeforeReturn(GPRInfo::regT1);
+        jit.ret();
         
-    stackCheck.link(&jit);
-    jit.move(CCallHelpers::stackPointerRegister, GPRInfo::argumentGPR0);
-    jit.poke(
-        GPRInfo::callFrameRegister,
-        OBJECT_OFFSETOF(struct JITStackFrame, callFrame) / sizeof(void*));
+        stackCheck.link(&jit);
+        jit.move(CCallHelpers::stackPointerRegister, GPRInfo::argumentGPR0);
+        jit.poke(
+            GPRInfo::callFrameRegister,
+            OBJECT_OFFSETOF(struct JITStackFrame, callFrame) / sizeof(void*));
         
-    jit.store32(
-        CCallHelpers::TrustedImm32(CallFrame::Location::encodeAsBytecodeOffset(0)),
-        CCallHelpers::tagFor(static_cast<VirtualRegister>(JSStack::ArgumentCount)));
-    CCallHelpers::Call callStackCheck = jit.call();
-    // FIXME: need to make this call register with exception handling somehow. This is
-    // part of a bigger problem: FTL should be able to handle exceptions.
-    // https://bugs.webkit.org/show_bug.cgi?id=113622
-    jit.jump(fromStackCheck);
+        jit.store32(
+            CCallHelpers::TrustedImm32(CallFrame::Location::encodeAsBytecodeOffset(0)),
+            CCallHelpers::tagFor(static_cast<VirtualRegister>(JSStack::ArgumentCount)));
+        CCallHelpers::Call callStackCheck = jit.call();
+        // FIXME: need to make this call register with exception handling somehow. This is
+        // part of a bigger problem: FTL should be able to handle exceptions.
+        // https://bugs.webkit.org/show_bug.cgi?id=113622
+        jit.jump(fromStackCheck);
         
-    CCallHelpers::Label arityCheck = jit.label();
-    compileEntry(jit);
-    jit.load32(
-        CCallHelpers::payloadFor(static_cast<VirtualRegister>(JSStack::ArgumentCount)),
-        GPRInfo::regT1);
-    jit.branch32(
-        CCallHelpers::AboveOrEqual, GPRInfo::regT1,
-        CCallHelpers::TrustedImm32(codeBlock->numParameters()))
-        .linkTo(fromArityCheck, &jit);
-    jit.move(CCallHelpers::stackPointerRegister, GPRInfo::argumentGPR0);
-    jit.poke(
-        GPRInfo::callFrameRegister,
-        OBJECT_OFFSETOF(struct JITStackFrame, callFrame) / sizeof(void*));
-    jit.store32(
-        CCallHelpers::TrustedImm32(CallFrame::Location::encodeAsBytecodeOffset(0)),
-        CCallHelpers::tagFor(static_cast<VirtualRegister>(JSStack::ArgumentCount)));
-    CCallHelpers::Call callArityCheck = jit.call();
-    // FIXME: need to make this call register with exception handling somehow. This is
-    // part of a bigger problem: FTL should be able to handle exceptions.
-    // https://bugs.webkit.org/show_bug.cgi?id=113622
-    jit.branchTest32(CCallHelpers::Zero, GPRInfo::regT0).linkTo(fromArityCheck, &jit);
-    CCallHelpers::Call callArityFixup = jit.call();
-    jit.jump(fromArityCheck);
+        arityCheck = jit.label();
+        compileEntry(jit);
+        jit.load32(
+            CCallHelpers::payloadFor(static_cast<VirtualRegister>(JSStack::ArgumentCount)),
+            GPRInfo::regT1);
+        jit.branch32(
+            CCallHelpers::AboveOrEqual, GPRInfo::regT1,
+            CCallHelpers::TrustedImm32(codeBlock->numParameters()))
+            .linkTo(fromArityCheck, &jit);
+        jit.move(CCallHelpers::stackPointerRegister, GPRInfo::argumentGPR0);
+        jit.poke(
+            GPRInfo::callFrameRegister,
+            OBJECT_OFFSETOF(struct JITStackFrame, callFrame) / sizeof(void*));
+        jit.store32(
+            CCallHelpers::TrustedImm32(CallFrame::Location::encodeAsBytecodeOffset(0)),
+            CCallHelpers::tagFor(static_cast<VirtualRegister>(JSStack::ArgumentCount)));
+        CCallHelpers::Call callArityCheck = jit.call();
+        // FIXME: need to make this call register with exception handling somehow. This is
+        // part of a bigger problem: FTL should be able to handle exceptions.
+        // https://bugs.webkit.org/show_bug.cgi?id=113622
+        jit.branchTest32(CCallHelpers::Zero, GPRInfo::regT0).linkTo(fromArityCheck, &jit);
+        CCallHelpers::Call callArityFixup = jit.call();
+        jit.jump(fromArityCheck);
         
-    OwnPtr<LinkBuffer> linkBuffer = adoptPtr(new LinkBuffer(state.graph.m_vm, &jit, codeBlock, JITCompilationMustSucceed));
-    linkBuffer->link(callStackCheck, cti_stack_check);
-    linkBuffer->link(callArityCheck, codeBlock->m_isConstructor ? cti_op_construct_arityCheck : cti_op_call_arityCheck);
-    linkBuffer->link(callArityFixup, FunctionPtr((state.graph.m_vm.getCTIStub(arityFixup)).code().executableAddress()));
+        linkBuffer = adoptPtr(new LinkBuffer(state.graph.m_vm, &jit, codeBlock, JITCompilationMustSucceed));
+        linkBuffer->link(callStackCheck, cti_stack_check);
+        linkBuffer->link(callArityCheck, codeBlock->m_isConstructor ? cti_op_construct_arityCheck : cti_op_call_arityCheck);
+        linkBuffer->link(callArityFixup, FunctionPtr((state.graph.m_vm.getCTIStub(arityFixup)).code().executableAddress()));
+        break;
+    }
+        
+    case FTLForOSREntryMode: {
+        // We jump to here straight from DFG code, after having boxed up all of the
+        // values into the scratch buffer. Everything should be good to go - at this
+        // point we've even done the stack check. Basically we just have to make the
+        // call to the LLVM-generated code.
+        jit.setupArgumentsExecState();
+        jit.move(
+            CCallHelpers::TrustedImmPtr(reinterpret_cast<void*>(state.generatedFunction)),
+            GPRInfo::nonArgGPR0);
+        jit.call(GPRInfo::nonArgGPR0);
+        jit.emitGetFromCallFrameHeaderPtr(JSStack::ReturnPC, GPRInfo::regT1);
+        jit.emitGetFromCallFrameHeaderPtr(JSStack::CallerFrame, GPRInfo::callFrameRegister);
+        jit.restoreReturnAddressBeforeReturn(GPRInfo::regT1);
+        jit.ret();
+        
+        linkBuffer = adoptPtr(new LinkBuffer(
+            state.graph.m_vm, &jit, codeBlock, JITCompilationMustSucceed));
+        break;
+    }
+        
+    default:
+        RELEASE_ASSERT_NOT_REACHED();
+        break;
+    }
     
     state.finalizer->initializeEntrypointLinkBuffer(linkBuffer.release());
     state.finalizer->initializeFunction(state.generatedFunction);
index 68c5efb..fa73419 100644 (file)
@@ -33,6 +33,7 @@
 #include "DFGInPlaceAbstractState.h"
 #include "FTLAbstractHeapRepository.h"
 #include "FTLExitThunkGenerator.h"
+#include "FTLForOSREntryJITCode.h"
 #include "FTLFormattedValue.h"
 #include "FTLLoweredNodeValue.h"
 #include "FTLOutput.h"
@@ -191,6 +192,9 @@ private:
         // make IR dumps easier to read.
         m_out.appendTo(lowBlock, m_nextLowBlock);
         
+        if (Options::ftlCrashes())
+            m_out.crashNonTerminal();
+        
         if (!m_highBlock->cfaHasVisited) {
             m_out.crash();
             return;
@@ -243,6 +247,9 @@ private:
         case GetArgument:
             compileGetArgument();
             break;
+        case ExtractOSREntryLocal:
+            compileExtractOSREntryLocal();
+            break;
         case GetLocal:
             compileGetLocal();
             break;
@@ -264,6 +271,7 @@ private:
         case Flush:
         case PhantomLocal:
         case SetArgument:
+        case LoopHint:
             break;
         case ArithAdd:
         case ValueAdd:
@@ -506,6 +514,13 @@ private:
         }
     }
     
+    void compileExtractOSREntryLocal()
+    {
+        EncodedJSValue* buffer = static_cast<EncodedJSValue*>(
+            m_ftlState.jitCode->ftlForOSREntry()->entryBuffer()->dataBuffer());
+        setJSValue(m_out.load64(m_out.absolute(buffer + m_node->unlinkedLocal())));
+    }
+    
     void compileGetLocal()
     {
         // GetLocals arise only for captured variables.
@@ -515,7 +530,7 @@ private:
         
         RELEASE_ASSERT(variable->isCaptured());
         
-        if (isInt32Speculation(value.m_value))
+        if (isInt32Speculation(value.m_type))
             setInt32(m_out.load32(payloadFor(variable->local())));
         else
             setJSValue(m_out.load64(addressFor(variable->local())));
@@ -2817,8 +2832,7 @@ private:
     
     void addWeakReference(JSCell* target)
     {
-        m_ftlState.jitCode->common.weakReferences.append(
-            WriteBarrier<JSCell>(vm(), codeBlock()->ownerExecutable(), target));
+        m_graph.m_plan.weakReferences.addLazily(target);
     }
     
     LValue weakPointer(JSCell* pointer)
diff --git a/Source/JavaScriptCore/ftl/FTLOSREntry.cpp b/Source/JavaScriptCore/ftl/FTLOSREntry.cpp
new file mode 100644 (file)
index 0000000..affdec7
--- /dev/null
@@ -0,0 +1,100 @@
+/*
+ * Copyright (C) 2013 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#include "config.h"
+#include "FTLOSREntry.h"
+
+#include "CallFrame.h"
+#include "CodeBlock.h"
+#include "DFGJITCode.h"
+#include "FTLForOSREntryJITCode.h"
+
+#if ENABLE(FTL_JIT)
+
+namespace JSC { namespace FTL {
+
+void* prepareOSREntry(
+    ExecState* exec, CodeBlock* dfgCodeBlock, CodeBlock* entryCodeBlock,
+    unsigned bytecodeIndex, unsigned streamIndex)
+{
+    VM& vm = exec->vm();
+    CodeBlock* baseline = dfgCodeBlock->baselineVersion();
+    DFG::JITCode* dfgCode = dfgCodeBlock->jitCode()->dfg();
+    ForOSREntryJITCode* entryCode = entryCodeBlock->jitCode()->ftlForOSREntry();
+    
+    if (Options::verboseOSR()) {
+        dataLog(
+            "FTL OSR from ", *dfgCodeBlock, " to ", *entryCodeBlock, " at bc#",
+            bytecodeIndex, ".\n");
+    }
+    
+    if (bytecodeIndex != entryCode->bytecodeIndex()) {
+        if (Options::verboseOSR())
+            dataLog("    OSR failed because we don't have an entrypoint for bc#", bytecodeIndex, "; ours is for bc#", entryCode->bytecodeIndex());
+        return 0;
+    }
+    
+    Operands<JSValue> values;
+    dfgCode->reconstruct(
+        exec, dfgCodeBlock, CodeOrigin(bytecodeIndex), streamIndex, values);
+    
+    if (Options::verboseOSR())
+        dataLog("    Values at entry: ", values, "\n");
+    
+    for (int argument = values.numberOfArguments(); argument--;) {
+        RELEASE_ASSERT(
+            exec->r(argumentToOperand(argument)).jsValue() == values.argument(argument));
+    }
+    
+    RELEASE_ASSERT(
+        static_cast<int>(values.numberOfLocals()) == baseline->m_numCalleeRegisters);
+    
+    EncodedJSValue* scratch = static_cast<EncodedJSValue*>(
+        entryCode->entryBuffer()->dataBuffer());
+    
+    for (int local = values.numberOfLocals(); local--;)
+        scratch[local] = JSValue::encode(values.local(local));
+    
+    unsigned stackFrameSize = entryCodeBlock->m_numCalleeRegisters;
+    if (!vm.interpreter->stack().grow(&exec->registers()[stackFrameSize])) {
+        if (Options::verboseOSR())
+            dataLog("    OSR failed bcause stack growth failed.\n");
+        return 0;
+    }
+    
+    exec->setCodeBlock(entryCodeBlock);
+    
+    void* result = entryCode->addressForCall().executableAddress();
+    if (Options::verboseOSR())
+        dataLog("    Entry will succeed, going to address", RawPointer(result), "\n");
+    
+    return result;
+}
+
+} } // namespace JSC::FTL
+
+#endif // ENABLE(FTL_JIT)
+
+
diff --git a/Source/JavaScriptCore/ftl/FTLOSREntry.h b/Source/JavaScriptCore/ftl/FTLOSREntry.h
new file mode 100644 (file)
index 0000000..d19f10e
--- /dev/null
@@ -0,0 +1,49 @@
+/*
+ * Copyright (C) 2013 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#ifndef FTLOSREntry_h
+#define FTLOSREntry_h
+
+#include <wtf/Platform.h>
+
+#if ENABLE(FTL_JIT)
+
+namespace JSC {
+
+class CodeBlock;
+class ExecState;
+
+namespace FTL {
+
+void* prepareOSREntry(
+    ExecState*, CodeBlock* dfgCodeBlock, CodeBlock* entryCodeBlock, unsigned bytecodeIndex,
+    unsigned streamIndex);
+
+} } // namespace JSC::FTL
+
+#endif // ENABLE(FTL_JIT)
+
+#endif // FTLOSREntry_h
+
index 5855e43..d9e3da6 100644 (file)
@@ -351,9 +351,14 @@ public:
     {
         call(trapIntrinsic());
     }
-    void crash()
+    
+    void crashNonTerminal()
     {
         call(intToPtr(constIntPtr(abort), pointerType(functionType(voidType))));
+    }
+    void crash()
+    {
+        crashNonTerminal();
         unreachable();
     }
     
index c3174a9..8739bc6 100644 (file)
@@ -29,6 +29,8 @@
 #if ENABLE(FTL_JIT)
 
 #include "CodeBlockWithJITType.h"
+#include "FTLForOSREntryJITCode.h"
+#include "FTLJITCode.h"
 #include "FTLJITFinalizer.h"
 
 namespace JSC { namespace FTL {
@@ -40,9 +42,25 @@ State::State(Graph& graph)
     , context(LLVMContextCreate())
     , module(0)
     , function(0)
-    , jitCode(adoptRef(new JITCode()))
     , generatedFunction(0)
 {
+    switch (graph.m_plan.mode) {
+    case FTLMode: {
+        jitCode = adoptRef(new JITCode());
+        break;
+    }
+    case FTLForOSREntryMode: {
+        RefPtr<ForOSREntryJITCode> code = adoptRef(new ForOSREntryJITCode());
+        code->initializeEntryBuffer(graph.m_vm, graph.m_profiledBlock->m_numCalleeRegisters);
+        code->setBytecodeIndex(graph.m_plan.osrEntryBytecodeIndex);
+        jitCode = code;
+        break;
+    }
+    default:
+        RELEASE_ASSERT_NOT_REACHED();
+        break;
+    }
+    
     finalizer = new JITFinalizer(graph.m_plan);
     graph.m_plan.finalizer = adoptPtr(finalizer);
 }
index bc23356..52d6a62 100644 (file)
@@ -74,6 +74,7 @@ namespace JSC {
         InlineCallFrame* asInlineCallFrame() const;
         int32_t unboxedInt32() const;
         bool unboxedBoolean() const;
+        double unboxedDouble() const;
         JSCell* unboxedCell() const;
         int32_t payload() const;
         int32_t tag() const;
@@ -96,6 +97,7 @@ namespace JSC {
             Instruction* vPC;
             InlineCallFrame* inlineCallFrame;
             EncodedValueDescriptor encodedValue;
+            double number;
         } u;
     };
 
@@ -188,6 +190,11 @@ namespace JSC {
         return !!payload();
     }
 
+    ALWAYS_INLINE double Register::unboxedDouble() const
+    {
+        return u.number;
+    }
+
     ALWAYS_INLINE JSCell* Register::unboxedCell() const
     {
 #if USE(JSVALUE64)
index ea88ad1..089dd44 100644 (file)
@@ -104,7 +104,7 @@ void JIT::emitEnterOptimizationCheck()
     if (!canBeOptimized())
         return;
 
-    Jump skipOptimize = branchAdd32(Signed, TrustedImm32(Options::executionCounterIncrementForReturn()), AbsoluteAddress(m_codeBlock->addressOfJITExecuteCounter()));
+    Jump skipOptimize = branchAdd32(Signed, TrustedImm32(Options::executionCounterIncrementForEntry()), AbsoluteAddress(m_codeBlock->addressOfJITExecuteCounter()));
     JITStubCall stubCall(this, cti_optimize);
     stubCall.addArgument(TrustedImm32(m_bytecodeOffset));
     ASSERT(!m_bytecodeOffset);
index d111110..a9d1610 100644 (file)
@@ -66,6 +66,12 @@ FTL::JITCode* JITCode::ftl()
     return 0;
 }
 
+FTL::ForOSREntryJITCode* JITCode::ftlForOSREntry()
+{
+    RELEASE_ASSERT_NOT_REACHED();
+    return 0;
+}
+
 PassRefPtr<JITCode> JITCode::hostFunction(JITCode::CodeRef code)
 {
     return adoptRef(new DirectJITCode(code, HostCallThunk));
index 3b4a631..26eaea8 100644 (file)
@@ -42,6 +42,7 @@ class CommonData;
 class JITCode;
 }
 namespace FTL {
+class ForOSREntryJITCode;
 class JITCode;
 }
 
@@ -174,6 +175,7 @@ public:
     virtual DFG::CommonData* dfgCommon();
     virtual DFG::JITCode* dfg();
     virtual FTL::JITCode* ftl();
+    virtual FTL::ForOSREntryJITCode* ftlForOSREntry();
     
     JSValue execute(JSStack*, CallFrame*, VM*);
     
index 559dda4..62f17c7 100644 (file)
@@ -1036,16 +1036,32 @@ DEFINE_STUB_FUNCTION(void, optimize)
         if (Options::verboseOSR())
             dataLog("Triggering optimized compilation of ", *codeBlock, "\n");
         
-        RefPtr<DeferredCompilationCallback> callback =
-            JITToDFGDeferredCompilationCallback::create();
-        RefPtr<CodeBlock> newCodeBlock = codeBlock->newReplacement();
-        
-        if (!vm.worklist)
-            vm.worklist = DFG::globalWorklist();
+        unsigned numVarsWithValues;
+        if (bytecodeIndex)
+            numVarsWithValues = codeBlock->m_numVars;
+        else
+            numVarsWithValues = 0;
+        Operands<JSValue> mustHandleValues(
+            codeBlock->numParameters(), numVarsWithValues);
+        for (size_t i = 0; i < mustHandleValues.size(); ++i) {
+            int operand = mustHandleValues.operandForIndex(i);
+            if (operandIsArgument(operand)
+                && !operandToArgument(operand)
+                && codeBlock->codeType() == FunctionCode
+                && codeBlock->specializationKind() == CodeForConstruct) {
+                // Ugh. If we're in a constructor, the 'this' argument may hold garbage. It will
+                // also never be used. It doesn't matter what we put into the value for this,
+                // but it has to be an actual value that can be grokked by subsequent DFG passes,
+                // so we sanitize it here by turning it into Undefined.
+                mustHandleValues[i] = jsUndefined();
+            } else
+                mustHandleValues[i] = callFrame->uncheckedR(operand).jsValue();
+        }
         
         CompilationResult result = DFG::compile(
-            callFrame, newCodeBlock.get(), DFG::DFGMode, bytecodeIndex, callback,
-            vm.worklist.get());
+            vm, codeBlock->newReplacement().get(), DFG::DFGMode, bytecodeIndex,
+            mustHandleValues, JITToDFGDeferredCompilationCallback::create(),
+            vm.ensureWorklist());
         
         if (result != CompilationSuccessful)
             return;
@@ -1054,15 +1070,6 @@ DEFINE_STUB_FUNCTION(void, optimize)
     CodeBlock* optimizedCodeBlock = codeBlock->replacement();
     ASSERT(JITCode::isOptimizingJIT(optimizedCodeBlock->jitType()));
     
-    if (optimizedCodeBlock->jitType() == JITCode::FTLJIT) {
-        // FTL JIT doesn't support OSR entry yet.
-        // https://bugs.webkit.org/show_bug.cgi?id=113625
-        
-        // Don't attempt OSR entry again.
-        codeBlock->dontOptimizeAnytimeSoon();
-        return;
-    }
-    
     if (void* address = DFG::prepareOSREntry(callFrame, optimizedCodeBlock, bytecodeIndex)) {
         if (Options::verboseOSR()) {
             dataLog(
index ca22a85..8bd282f 100644 (file)
@@ -230,26 +230,32 @@ PassRefPtr<CodeBlock> ScriptExecutable::newReplacementCodeBlockFor(
     if (classInfo() == EvalExecutable::info()) {
         RELEASE_ASSERT(kind == CodeForCall);
         EvalExecutable* executable = jsCast<EvalExecutable*>(this);
+        EvalCodeBlock* baseline = static_cast<EvalCodeBlock*>(
+            executable->m_evalCodeBlock->baselineVersion());
         RefPtr<EvalCodeBlock> result = adoptRef(new EvalCodeBlock(
-            CodeBlock::CopyParsedBlock, *executable->m_evalCodeBlock));
-        result->setAlternative(executable->m_evalCodeBlock);
+            CodeBlock::CopyParsedBlock, *baseline));
+        result->setAlternative(baseline);
         return result;
     }
     
     if (classInfo() == ProgramExecutable::info()) {
         RELEASE_ASSERT(kind == CodeForCall);
         ProgramExecutable* executable = jsCast<ProgramExecutable*>(this);
+        ProgramCodeBlock* baseline = static_cast<ProgramCodeBlock*>(
+            executable->m_programCodeBlock->baselineVersion());
         RefPtr<ProgramCodeBlock> result = adoptRef(new ProgramCodeBlock(
-            CodeBlock::CopyParsedBlock, *executable->m_programCodeBlock));
-        result->setAlternative(executable->m_programCodeBlock);
+            CodeBlock::CopyParsedBlock, *baseline));
+        result->setAlternative(baseline);
         return result;
     }
 
     RELEASE_ASSERT(classInfo() == FunctionExecutable::info());
     FunctionExecutable* executable = jsCast<FunctionExecutable*>(this);
+    FunctionCodeBlock* baseline = static_cast<FunctionCodeBlock*>(
+        executable->codeBlockFor(kind)->baselineVersion());
     RefPtr<FunctionCodeBlock> result = adoptRef(new FunctionCodeBlock(
-        CodeBlock::CopyParsedBlock, *executable->codeBlockFor(kind)));
-    result->setAlternative(executable->codeBlockFor(kind));
+        CodeBlock::CopyParsedBlock, *baseline));
+    result->setAlternative(baseline);
     return result;
 }
 
index 3fd865e..85d4135 100644 (file)
@@ -129,6 +129,7 @@ typedef OptionRange optionRange;
     v(unsigned, llvmBackendOptimizationLevel, 2) \
     v(unsigned, llvmOptimizationLevel, 2) \
     v(unsigned, llvmSizeLevel, 0) \
+    v(bool, ftlCrashes, false) /* fool-proof way of checking that you ended up in the FTL. ;-) */\
     \
     v(bool, enableConcurrentJIT, true) \
     v(unsigned, numberOfCompilerThreads, computeNumberOfWorkerThreads(2) - 1) \
@@ -156,9 +157,14 @@ typedef OptionRange optionRange;
     v(int32, thresholdForOptimizeAfterWarmUp, 1000) \
     v(int32, thresholdForOptimizeAfterLongWarmUp, 1000) \
     v(int32, thresholdForOptimizeSoon, 1000) \
-    \
     v(int32, executionCounterIncrementForLoop, 1) \
-    v(int32, executionCounterIncrementForReturn, 15) \
+    v(int32, executionCounterIncrementForEntry, 15) \
+    \
+    v(int32, thresholdForFTLOptimizeAfterWarmUp, 25000) \
+    v(int32, thresholdForFTLOptimizeSoon, 1000) \
+    v(int32, ftlTierUpCounterIncrementForLoop, 1) \
+    v(int32, ftlTierUpCounterIncrementForReturn, 15) \
+    v(unsigned, ftlOSREntryFailureCountForReoptimization, 15) \
     \
     v(int32, evalThresholdMultiplier, 10) \
     \
index 430d734..9beeb2f 100644 (file)
@@ -710,6 +710,15 @@ void VM::gatherConservativeRoots(ConservativeRoots& conservativeRoots)
         }
     }
 }
+
+DFG::Worklist* VM::ensureWorklist()
+{
+    if (!DFG::enableConcurrentJIT())
+        return 0;
+    if (!worklist)
+        worklist = DFG::globalWorklist();
+    return worklist.get();
+}
 #endif
 
 #if ENABLE(REGEXP_TRACING)
index cf49a66..ac6f8ed 100644 (file)
@@ -193,6 +193,10 @@ namespace JSC {
         JS_EXPORT_PRIVATE ~VM();
 
         void makeUsableFromMultipleThreads() { heap.machineThreads().makeUsableFromMultipleThreads(); }
+        
+#if ENABLE(DFG_JIT)
+        DFG::Worklist* ensureWorklist();
+#endif // ENABLE(DFG_JIT)
 
     private:
         RefPtr<JSLock> m_apiLock;