DFG JIT does not optimize booleans
authorfpizlo@apple.com <fpizlo@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 7 Sep 2011 02:47:51 +0000 (02:47 +0000)
committerfpizlo@apple.com <fpizlo@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 7 Sep 2011 02:47:51 +0000 (02:47 +0000)
https://bugs.webkit.org/show_bug.cgi?id=67670

Reviewed by Gavin Barraclough.

This adds boolean value profiling, boolean prediction in the DFG,
boolean forward flow propagation in the DFGPropagator, boolean
data format in DFG generation info, and comprehensive optimizations
based on both boolean prediction and boolean generation info.
This is brings the speed-up on v8-richards to 12%, and gives slight
speed-ups elsewhere as well.

Making this work right required navigating some subtleties in
value profiling.  Some functions get compiled with insufficient
information because some important path of the function never
executed.  In these cases, we wish to fall back on static
speculation.  But to do so, we need to ensure that predictions that
are inherent in the code (like that GetById almost certainly takes
a cell operand) are reflected in predictions that we make in
DFGPropagator.  Thus, DFGPropagator now does both backward and
forward flow, using a both forward and backward fixpoint.

The backward flow in DFGPropagator is a separate static analysis,
and needs to keep a set of backward flow abstract values for
variables, arguments, and globals.  To make this easy, this patch
factors out DFGGraph's prediction tracking capability into
DFGPredictionTracker, which now gets used by both DFGGraph (for
forward flow predictions) and DFGPropagator (for backward flow
predictions).  Backward flow predictions eventually get merged
into forward flow ones, but the two are not equivalent: a forward
flow prediction is a superset of the backward flow prediction.

Debugging these prediction issues required a better understanding
of where we fail speculation, and what our value predictions look
like.  This patch also adds optional verbose speculation failure
(so an informative printf fires whenever speculation failure occurs)
and slight improvements to the verbosity in other places.

* bytecode/ValueProfile.h:
(JSC::ValueProfile::numberOfBooleans):
(JSC::ValueProfile::probabilityOfBoolean):
(JSC::ValueProfile::dump):
(JSC::ValueProfile::computeStatistics):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::stronglyPredict):
(JSC::DFG::ByteCodeParser::parseBlock):
* dfg/DFGGenerationInfo.h:
(JSC::DFG::dataFormatToString):
(JSC::DFG::needDataFormatConversion):
* dfg/DFGGraph.cpp:
(JSC::DFG::Graph::dump):
(JSC::DFG::Graph::predictArgumentTypes):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::Graph):
(JSC::DFG::Graph::predictions):
(JSC::DFG::Graph::predict):
(JSC::DFG::Graph::predictGlobalVar):
(JSC::DFG::Graph::getPrediction):
(JSC::DFG::Graph::getGlobalVarPrediction):
(JSC::DFG::Graph::isBooleanConstant):
(JSC::DFG::Graph::valueOfBooleanConstant):
* dfg/DFGJITCodeGenerator.cpp:
(JSC::DFG::JITCodeGenerator::fillInteger):
(JSC::DFG::JITCodeGenerator::fillDouble):
(JSC::DFG::JITCodeGenerator::fillJSValue):
(JSC::DFG::JITCodeGenerator::isKnownNotInteger):
(JSC::DFG::JITCodeGenerator::isKnownBoolean):
(JSC::DFG::JITCodeGenerator::nonSpeculativeNonPeepholeCompareNull):
(JSC::DFG::JITCodeGenerator::nonSpeculativeNonPeepholeCompare):
(JSC::DFG::JITCodeGenerator::nonSpeculativeNonPeepholeStrictEq):
(JSC::DFG::JITCodeGenerator::emitBranch):
(JSC::DFG::JITCodeGenerator::speculationCheck):
(JSC::DFG::GPRTemporary::GPRTemporary):
* dfg/DFGJITCodeGenerator.h:
(JSC::DFG::JITCodeGenerator::isBooleanConstant):
(JSC::DFG::JITCodeGenerator::valueOfBooleanConstant):
* dfg/DFGJITCompiler.cpp:
(JSC::DFG::JITCompiler::jumpFromSpeculativeToNonSpeculative):
(JSC::DFG::JITCompiler::link):
* dfg/DFGJITCompiler.h:
(JSC::DFG::JITCompiler::debugCall):
(JSC::DFG::JITCompiler::isBooleanConstant):
(JSC::DFG::JITCompiler::valueOfBooleanConstant):
* dfg/DFGNode.h:
(JSC::DFG::isBooleanPrediction):
(JSC::DFG::predictionToString):
(JSC::DFG::mergePredictions):
(JSC::DFG::makePrediction):
(JSC::DFG::Node::isBooleanConstant):
(JSC::DFG::Node::valueOfBooleanConstant):
(JSC::DFG::Node::hasBooleanResult):
(JSC::DFG::Node::hasNumericResult):
(JSC::DFG::Node::predict):
* dfg/DFGOperations.cpp:
* dfg/DFGOperations.h:
* dfg/DFGPredictionTracker.h: Added.
(JSC::DFG::operandIsArgument):
(JSC::DFG::PredictionSlot::PredictionSlot):
(JSC::DFG::PredictionTracker::PredictionTracker):
(JSC::DFG::PredictionTracker::initializeSimilarTo):
(JSC::DFG::PredictionTracker::numberOfArguments):
(JSC::DFG::PredictionTracker::numberOfVariables):
(JSC::DFG::PredictionTracker::argumentIndexForOperand):
(JSC::DFG::PredictionTracker::predictArgument):
(JSC::DFG::PredictionTracker::predict):
(JSC::DFG::PredictionTracker::predictGlobalVar):
(JSC::DFG::PredictionTracker::getArgumentPrediction):
(JSC::DFG::PredictionTracker::getPrediction):
(JSC::DFG::PredictionTracker::getGlobalVarPrediction):
* dfg/DFGPropagator.cpp:
(JSC::DFG::Propagator::Propagator):
(JSC::DFG::Propagator::fixpoint):
(JSC::DFG::Propagator::setPrediction):
(JSC::DFG::Propagator::mergeUse):
(JSC::DFG::Propagator::mergePrediction):
(JSC::DFG::Propagator::propagateNode):
* dfg/DFGSpeculativeJIT.cpp:
(JSC::DFG::SpeculativeJIT::fillSpeculateIntInternal):
(JSC::DFG::SpeculativeJIT::fillSpeculateDouble):
(JSC::DFG::SpeculativeJIT::fillSpeculateCell):
(JSC::DFG::SpeculativeJIT::fillSpeculateBoolean):
(JSC::DFG::SpeculativeJIT::compare):
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGSpeculativeJIT.h:
(JSC::DFG::SpeculateBooleanOperand::SpeculateBooleanOperand):
(JSC::DFG::SpeculateBooleanOperand::~SpeculateBooleanOperand):
(JSC::DFG::SpeculateBooleanOperand::index):
(JSC::DFG::SpeculateBooleanOperand::gpr):
(JSC::DFG::SpeculateBooleanOperand::use):
* runtime/JSGlobalData.h:
* runtime/JSValue.cpp:
(JSC::JSValue::description):

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@94629 268f45cc-cd09-0410-ab3c-d52691b4dbfc

19 files changed:
Source/JavaScriptCore/ChangeLog
Source/JavaScriptCore/bytecode/ValueProfile.h
Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp
Source/JavaScriptCore/dfg/DFGGenerationInfo.h
Source/JavaScriptCore/dfg/DFGGraph.cpp
Source/JavaScriptCore/dfg/DFGGraph.h
Source/JavaScriptCore/dfg/DFGJITCodeGenerator.cpp
Source/JavaScriptCore/dfg/DFGJITCodeGenerator.h
Source/JavaScriptCore/dfg/DFGJITCompiler.cpp
Source/JavaScriptCore/dfg/DFGJITCompiler.h
Source/JavaScriptCore/dfg/DFGNode.h
Source/JavaScriptCore/dfg/DFGOperations.cpp
Source/JavaScriptCore/dfg/DFGOperations.h
Source/JavaScriptCore/dfg/DFGPredictionTracker.h [new file with mode: 0644]
Source/JavaScriptCore/dfg/DFGPropagator.cpp
Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp
Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h
Source/JavaScriptCore/runtime/JSGlobalData.h
Source/JavaScriptCore/runtime/JSValue.cpp

index 5283a0d..769287d 100644 (file)
@@ -1,3 +1,138 @@
+2011-09-06  Filip Pizlo  <fpizlo@apple.com>
+
+        DFG JIT does not optimize booleans
+        https://bugs.webkit.org/show_bug.cgi?id=67670
+
+        Reviewed by Gavin Barraclough.
+        
+        This adds boolean value profiling, boolean prediction in the DFG,
+        boolean forward flow propagation in the DFGPropagator, boolean
+        data format in DFG generation info, and comprehensive optimizations
+        based on both boolean prediction and boolean generation info.
+        This is brings the speed-up on v8-richards to 12%, and gives slight
+        speed-ups elsewhere as well.
+        
+        Making this work right required navigating some subtleties in
+        value profiling.  Some functions get compiled with insufficient
+        information because some important path of the function never
+        executed.  In these cases, we wish to fall back on static
+        speculation.  But to do so, we need to ensure that predictions that
+        are inherent in the code (like that GetById almost certainly takes
+        a cell operand) are reflected in predictions that we make in
+        DFGPropagator.  Thus, DFGPropagator now does both backward and
+        forward flow, using a both forward and backward fixpoint.
+        
+        The backward flow in DFGPropagator is a separate static analysis,
+        and needs to keep a set of backward flow abstract values for
+        variables, arguments, and globals.  To make this easy, this patch
+        factors out DFGGraph's prediction tracking capability into
+        DFGPredictionTracker, which now gets used by both DFGGraph (for
+        forward flow predictions) and DFGPropagator (for backward flow
+        predictions).  Backward flow predictions eventually get merged
+        into forward flow ones, but the two are not equivalent: a forward
+        flow prediction is a superset of the backward flow prediction.
+        
+        Debugging these prediction issues required a better understanding
+        of where we fail speculation, and what our value predictions look
+        like.  This patch also adds optional verbose speculation failure
+        (so an informative printf fires whenever speculation failure occurs)
+        and slight improvements to the verbosity in other places.
+
+        * bytecode/ValueProfile.h:
+        (JSC::ValueProfile::numberOfBooleans):
+        (JSC::ValueProfile::probabilityOfBoolean):
+        (JSC::ValueProfile::dump):
+        (JSC::ValueProfile::computeStatistics):
+        * dfg/DFGByteCodeParser.cpp:
+        (JSC::DFG::ByteCodeParser::stronglyPredict):
+        (JSC::DFG::ByteCodeParser::parseBlock):
+        * dfg/DFGGenerationInfo.h:
+        (JSC::DFG::dataFormatToString):
+        (JSC::DFG::needDataFormatConversion):
+        * dfg/DFGGraph.cpp:
+        (JSC::DFG::Graph::dump):
+        (JSC::DFG::Graph::predictArgumentTypes):
+        * dfg/DFGGraph.h:
+        (JSC::DFG::Graph::Graph):
+        (JSC::DFG::Graph::predictions):
+        (JSC::DFG::Graph::predict):
+        (JSC::DFG::Graph::predictGlobalVar):
+        (JSC::DFG::Graph::getPrediction):
+        (JSC::DFG::Graph::getGlobalVarPrediction):
+        (JSC::DFG::Graph::isBooleanConstant):
+        (JSC::DFG::Graph::valueOfBooleanConstant):
+        * dfg/DFGJITCodeGenerator.cpp:
+        (JSC::DFG::JITCodeGenerator::fillInteger):
+        (JSC::DFG::JITCodeGenerator::fillDouble):
+        (JSC::DFG::JITCodeGenerator::fillJSValue):
+        (JSC::DFG::JITCodeGenerator::isKnownNotInteger):
+        (JSC::DFG::JITCodeGenerator::isKnownBoolean):
+        (JSC::DFG::JITCodeGenerator::nonSpeculativeNonPeepholeCompareNull):
+        (JSC::DFG::JITCodeGenerator::nonSpeculativeNonPeepholeCompare):
+        (JSC::DFG::JITCodeGenerator::nonSpeculativeNonPeepholeStrictEq):
+        (JSC::DFG::JITCodeGenerator::emitBranch):
+        (JSC::DFG::JITCodeGenerator::speculationCheck):
+        (JSC::DFG::GPRTemporary::GPRTemporary):
+        * dfg/DFGJITCodeGenerator.h:
+        (JSC::DFG::JITCodeGenerator::isBooleanConstant):
+        (JSC::DFG::JITCodeGenerator::valueOfBooleanConstant):
+        * dfg/DFGJITCompiler.cpp:
+        (JSC::DFG::JITCompiler::jumpFromSpeculativeToNonSpeculative):
+        (JSC::DFG::JITCompiler::link):
+        * dfg/DFGJITCompiler.h:
+        (JSC::DFG::JITCompiler::debugCall):
+        (JSC::DFG::JITCompiler::isBooleanConstant):
+        (JSC::DFG::JITCompiler::valueOfBooleanConstant):
+        * dfg/DFGNode.h:
+        (JSC::DFG::isBooleanPrediction):
+        (JSC::DFG::predictionToString):
+        (JSC::DFG::mergePredictions):
+        (JSC::DFG::makePrediction):
+        (JSC::DFG::Node::isBooleanConstant):
+        (JSC::DFG::Node::valueOfBooleanConstant):
+        (JSC::DFG::Node::hasBooleanResult):
+        (JSC::DFG::Node::hasNumericResult):
+        (JSC::DFG::Node::predict):
+        * dfg/DFGOperations.cpp:
+        * dfg/DFGOperations.h:
+        * dfg/DFGPredictionTracker.h: Added.
+        (JSC::DFG::operandIsArgument):
+        (JSC::DFG::PredictionSlot::PredictionSlot):
+        (JSC::DFG::PredictionTracker::PredictionTracker):
+        (JSC::DFG::PredictionTracker::initializeSimilarTo):
+        (JSC::DFG::PredictionTracker::numberOfArguments):
+        (JSC::DFG::PredictionTracker::numberOfVariables):
+        (JSC::DFG::PredictionTracker::argumentIndexForOperand):
+        (JSC::DFG::PredictionTracker::predictArgument):
+        (JSC::DFG::PredictionTracker::predict):
+        (JSC::DFG::PredictionTracker::predictGlobalVar):
+        (JSC::DFG::PredictionTracker::getArgumentPrediction):
+        (JSC::DFG::PredictionTracker::getPrediction):
+        (JSC::DFG::PredictionTracker::getGlobalVarPrediction):
+        * dfg/DFGPropagator.cpp:
+        (JSC::DFG::Propagator::Propagator):
+        (JSC::DFG::Propagator::fixpoint):
+        (JSC::DFG::Propagator::setPrediction):
+        (JSC::DFG::Propagator::mergeUse):
+        (JSC::DFG::Propagator::mergePrediction):
+        (JSC::DFG::Propagator::propagateNode):
+        * dfg/DFGSpeculativeJIT.cpp:
+        (JSC::DFG::SpeculativeJIT::fillSpeculateIntInternal):
+        (JSC::DFG::SpeculativeJIT::fillSpeculateDouble):
+        (JSC::DFG::SpeculativeJIT::fillSpeculateCell):
+        (JSC::DFG::SpeculativeJIT::fillSpeculateBoolean):
+        (JSC::DFG::SpeculativeJIT::compare):
+        (JSC::DFG::SpeculativeJIT::compile):
+        * dfg/DFGSpeculativeJIT.h:
+        (JSC::DFG::SpeculateBooleanOperand::SpeculateBooleanOperand):
+        (JSC::DFG::SpeculateBooleanOperand::~SpeculateBooleanOperand):
+        (JSC::DFG::SpeculateBooleanOperand::index):
+        (JSC::DFG::SpeculateBooleanOperand::gpr):
+        (JSC::DFG::SpeculateBooleanOperand::use):
+        * runtime/JSGlobalData.h:
+        * runtime/JSValue.cpp:
+        (JSC::JSValue::description):
+
 2011-09-06  Mark Hahnenberg  <mhahnenberg@apple.com>
 
         Unzip initialization lists and constructors in JSCell hierarchy (5/7)
index 3ff4b4c..71a8380 100644 (file)
@@ -117,6 +117,16 @@ struct ValueProfile {
         }
         return result;
     }
+    
+    unsigned numberOfBooleans() const
+    {
+        unsigned result = 0;
+        for (unsigned i = 0; i < numberOfBuckets; ++i) {
+            if (!!buckets[i] && JSValue::decode(buckets[i]).isBoolean())
+                result++;
+        }
+        return result;
+    }
         
     // These methods are not particularly optimized, in that they will each
     // perform two passes over the buckets array. However, they are
@@ -142,17 +152,23 @@ struct ValueProfile {
     {
         return computeProbability(numberOfArrays(), numberOfSamples());
     }
+    
+    unsigned probabilityOfBoolean() const
+    {
+        return computeProbability(numberOfBooleans(), numberOfSamples());
+    }
 
 #ifndef NDEBUG
     void dump(FILE* out)
     {
         fprintf(out,
-                "samples = %u, int32 = %u, double = %u, cell = %u, array = %u",
+                "samples = %u, int32 = %u (%u), double = %u (%u), cell = %u (%u), array = %u (%u), boolean = %u (%u)",
                 numberOfSamples(),
-                numberOfInt32s(),
-                numberOfDoubles(),
-                numberOfCells(),
-                numberOfArrays());
+                probabilityOfInt32(), numberOfInt32s(),
+                probabilityOfDouble(), numberOfDoubles(),
+                probabilityOfCell(), numberOfCells(),
+                probabilityOfArray(), numberOfArrays(),
+                probabilityOfBoolean(), numberOfBooleans());
         bool first = true;
         for (unsigned i = 0; i < numberOfBuckets; ++i) {
             if (!!buckets[i] || !!weakBuckets[i]) {
@@ -178,16 +194,18 @@ struct ValueProfile {
         unsigned doubles;
         unsigned cells;
         unsigned arrays;
+        unsigned booleans;
     };
 
     // Optimized method for getting all counts at once.
     void computeStatistics(JSGlobalData& globalData, Statistics& statistics) const
     {
-        unsigned samples = 0;
-        unsigned int32s  = 0;
-        unsigned doubles = 0;
-        unsigned cells   = 0;
-        unsigned arrays  = 0;
+        unsigned samples  = 0;
+        unsigned int32s   = 0;
+        unsigned doubles  = 0;
+        unsigned cells    = 0;
+        unsigned arrays   = 0;
+        unsigned booleans = 0;
         
         for (unsigned i = 0; i < numberOfBuckets; ++i) {
             if (!buckets[i]) {
@@ -213,14 +231,16 @@ struct ValueProfile {
                 cells++;
                 if (isJSArray(&globalData, value.asCell()))
                     arrays++;
-            }
+            } else if (value.isBoolean())
+                booleans++;
         }
         
-        statistics.samples = samples;
-        statistics.int32s  = int32s;
-        statistics.doubles = doubles;
-        statistics.cells   = cells;
-        statistics.arrays  = arrays;
+        statistics.samples  = samples;
+        statistics.int32s   = int32s;
+        statistics.doubles  = doubles;
+        statistics.cells    = cells;
+        statistics.arrays   = arrays;
+        statistics.booleans = booleans;
     }
     
     int bytecodeOffset; // -1 for prologue
index deff4c2..0cbb8b0 100644 (file)
@@ -475,11 +475,14 @@ private:
         ValueProfile* profile = m_profiledBlock->valueProfileForBytecodeOffset(bytecodeIndex);
         ASSERT(profile);
 #if DFG_DEBUG_VERBOSE
-        printf("Dynamic prediction [%u, %u]: ", nodeIndex, bytecodeIndex);
+        printf("Dynamic profile [%u, %u]: ", nodeIndex, bytecodeIndex);
         profile->dump(stdout);
         printf("\n");
 #endif
         m_graph[nodeIndex].predict(makePrediction(*m_globalData, *profile), StrongPrediction);
+#if DFG_DEBUG_VERBOSE
+        printf("    Prediction: %s\n", predictionToString(m_graph[nodeIndex].getPrediction()));
+#endif
 #else
         UNUSED_PARAM(nodeIndex);
         UNUSED_PARAM(bytecodeIndex);
index 9b1e4d8..74b03c1 100644 (file)
@@ -42,11 +42,13 @@ enum DataFormat {
     DataFormatNone = 0,
     DataFormatInteger = 1,
     DataFormatDouble = 2,
-    DataFormatCell = 3,
+    DataFormatBoolean = 3,
+    DataFormatCell = 4,
     DataFormatJS = 8,
     DataFormatJSInteger = DataFormatJS | DataFormatInteger,
     DataFormatJSDouble = DataFormatJS | DataFormatDouble,
     DataFormatJSCell = DataFormatJS | DataFormatCell,
+    DataFormatJSBoolean = DataFormatJS | DataFormatBoolean
 };
 
 #ifndef NDEBUG
@@ -61,6 +63,8 @@ inline const char* dataFormatToString(DataFormat dataFormat)
         return "Double";
     case DataFormatCell:
         return "Cell";
+    case DataFormatBoolean:
+        return "Boolean";
     case DataFormatJS:
         return "JS";
     case DataFormatJSInteger:
@@ -69,6 +73,8 @@ inline const char* dataFormatToString(DataFormat dataFormat)
         return "JSDouble";
     case DataFormatJSCell:
         return "JSCell";
+    case DataFormatJSBoolean:
+        return "JSBoolean";
     default:
         return "Unknown";
     }
@@ -88,6 +94,7 @@ inline bool needDataFormatConversion(DataFormat from, DataFormat to)
     case DataFormatJSInteger:
     case DataFormatJSDouble:
     case DataFormatJSCell:
+    case DataFormatJSBoolean:
         switch (to) {
         case DataFormatInteger:
         case DataFormatDouble:
@@ -97,11 +104,14 @@ inline bool needDataFormatConversion(DataFormat from, DataFormat to)
         case DataFormatJSInteger:
         case DataFormatJSDouble:
         case DataFormatJSCell:
+        case DataFormatJSBoolean:
             return false;
         default:
+            // This captures DataFormatBoolean, which is currently unused.
             ASSERT_NOT_REACHED();
         }
     default:
+        // This captures DataFormatBoolean, which is currently unused.
         ASSERT_NOT_REACHED();
     }
     return true;
index e5a7d2c..f11f732 100644 (file)
@@ -52,8 +52,10 @@ void Graph::dump(NodeIndex nodeIndex, CodeBlock* codeBlock)
     NodeType op = node.op;
 
     unsigned refCount = node.refCount();
-    if (!refCount)
+    if (!refCount) {
+        printf("% 4d:\tskipped %s\n", (int)nodeIndex, opName(op));
         return;
+    }
     bool mustGenerate = node.mustGenerate();
     if (mustGenerate)
         --refCount;
@@ -193,14 +195,14 @@ void Graph::refChildren(NodeIndex op)
 void Graph::predictArgumentTypes(ExecState* exec, CodeBlock* codeBlock)
 {
     if (exec) {
-        size_t numberOfArguments = std::min(exec->argumentCountIncludingThis(), m_argumentPredictions.size());
+        size_t numberOfArguments = std::min(exec->argumentCountIncludingThis(), m_predictions.numberOfArguments());
         
         for (size_t arg = 1; arg < numberOfArguments; ++arg) {
             JSValue argumentValue = exec->argument(arg - 1);
             if (argumentValue.isInt32())
-                m_argumentPredictions[arg].m_value |= PredictInt32;
+                m_predictions.predictArgument(arg, PredictInt32, WeakPrediction);
             else if (argumentValue.isDouble())
-                m_argumentPredictions[arg].m_value |= PredictDouble;
+                m_predictions.predictArgument(arg, PredictDouble, WeakPrediction);
         }
     }
     
@@ -222,10 +224,10 @@ void Graph::predictArgumentTypes(ExecState* exec, CodeBlock* codeBlock)
         printf("\n");
 #endif
         
-        mergePrediction(m_argumentPredictions[arg].m_value, makePrediction(globalData, *profile));
+        m_predictions.predictArgument(arg, makePrediction(globalData, *profile) & ~PredictionTagMask, StrongPrediction);
         
 #if DFG_DEBUG_VERBOSE
-        printf("    Prediction: %s\n", predictionToString(m_argumentPredictions[arg].m_value));
+        printf("    Prediction: %s\n", predictionToString(m_predictions.getArgumentPrediction(arg)));
 #endif
     }
 #else
index d641b11..4ab1812 100644 (file)
@@ -30,6 +30,7 @@
 
 #include "CodeBlock.h"
 #include "DFGNode.h"
+#include "DFGPredictionTracker.h"
 #include "RegisterFile.h"
 #include <wtf/HashMap.h>
 #include <wtf/Vector.h>
@@ -42,18 +43,6 @@ class ExecState;
 
 namespace DFG {
 
-// helper function to distinguish vars & temporaries from arguments.
-inline bool operandIsArgument(int operand) { return operand < 0; }
-
-struct PredictionSlot {
-public:
-    PredictionSlot()
-        : m_value(PredictNone)
-    {
-    }
-    PredictedType m_value;
-};
-
 typedef uint32_t BlockIndex;
 
 // For every local variable we track any existing get or set of the value.
@@ -103,8 +92,7 @@ struct BasicBlock {
 class Graph : public Vector<Node, 64> {
 public:
     Graph(unsigned numArguments, unsigned numVariables)
-        : m_argumentPredictions(numArguments)
-        , m_variablePredictions(numVariables)
+        : m_predictions(numArguments, numVariables)
     {
     }
 
@@ -135,70 +123,35 @@ public:
     {
         return *m_blocks[blockIndexForBytecodeOffset(bytecodeBegin)];
     }
-
+    
+    PredictionTracker& predictions()
+    {
+        return m_predictions;
+    }
+    
     bool predict(int operand, PredictedType prediction, PredictionSource source)
     {
-        if (operandIsArgument(operand)) {
-            unsigned argument = operand + m_argumentPredictions.size() + RegisterFile::CallFrameHeaderSize;
-            return mergePrediction(m_argumentPredictions[argument].m_value, makePrediction(prediction, source));
-        }
-        
-        if ((unsigned)operand >= m_variablePredictions.size()) {
-            ASSERT(operand >= 0);
-            m_variablePredictions.resize(operand + 1);
-        }
-        
-        return mergePrediction(m_variablePredictions[operand].m_value, makePrediction(prediction, source));
+        return m_predictions.predict(operand, prediction, source);
     }
     
     bool predictGlobalVar(unsigned varNumber, PredictedType prediction, PredictionSource source)
     {
-        HashMap<unsigned, PredictionSlot>::iterator iter = m_globalVarPredictions.find(varNumber + 1);
-        if (iter == m_globalVarPredictions.end()) {
-            PredictionSlot predictionSlot;
-            bool result = mergePrediction(predictionSlot.m_value, makePrediction(prediction, source));
-            m_globalVarPredictions.add(varNumber + 1, predictionSlot);
-            return result;
-        } else
-            return mergePrediction(iter->second.m_value, makePrediction(prediction, source));
+        return m_predictions.predictGlobalVar(varNumber, prediction, source);
     }
     
     bool predict(Node& node, PredictedType prediction, PredictionSource source)
     {
-        switch (node.op) {
-        case GetLocal:
-            return predict(node.local(), prediction, source);
-            break;
-        case GetGlobalVar:
-            return predictGlobalVar(node.varNumber(), prediction, source);
-        case GetById:
-        case GetMethod:
-        case GetByVal:
-        case Call:
-        case Construct:
-            return node.predict(prediction, source);
-        default:
-            return false;
-        }
+        return m_predictions.predict(node, prediction, source);
     }
 
     PredictedType getPrediction(int operand)
     {
-        if (operandIsArgument(operand)) {
-            unsigned argument = operand + m_argumentPredictions.size() + RegisterFile::CallFrameHeaderSize;
-            return m_argumentPredictions[argument].m_value;
-        }
-        if ((unsigned)operand < m_variablePredictions.size())
-            return m_variablePredictions[operand].m_value;
-        return PredictNone;
+        return m_predictions.getPrediction(operand);
     }
     
     PredictedType getGlobalVarPrediction(unsigned varNumber)
     {
-        HashMap<unsigned, PredictionSlot>::iterator iter = m_globalVarPredictions.find(varNumber + 1);
-        if (iter == m_globalVarPredictions.end())
-            return PredictNone;
-        return iter->second.m_value;
+        return m_predictions.getGlobalVarPrediction(varNumber);
     }
     
     PredictedType getPrediction(Node& node)
@@ -211,20 +164,7 @@ public:
         if (nodePtr->op == ValueToInt32)
             nodePtr = &(*this)[nodePtr->child1()];
         
-        switch (nodePtr->op) {
-        case GetLocal:
-            return getPrediction(nodePtr->local());
-        case GetGlobalVar:
-            return getGlobalVarPrediction(nodePtr->varNumber());
-        case GetById:
-        case GetMethod:
-        case GetByVal:
-        case Call:
-        case Construct:
-            return nodePtr->getPrediction();
-        default:
-            return PredictNone;
-        }
+        return m_predictions.getPrediction(*nodePtr);
     }
     
     // Helper methods to check nodes for constants.
@@ -244,6 +184,10 @@ public:
     {
         return at(nodeIndex).isDoubleConstant(codeBlock);
     }
+    bool isBooleanConstant(CodeBlock* codeBlock, NodeIndex nodeIndex)
+    {
+        return at(nodeIndex).isBooleanConstant(codeBlock);
+    }
     // Helper methods get constant values from nodes.
     JSValue valueOfJSConstant(CodeBlock* codeBlock, NodeIndex nodeIndex)
     {
@@ -257,6 +201,10 @@ public:
     {
         return at(nodeIndex).valueOfDoubleConstant(codeBlock);
     }
+    bool valueOfBooleanConstant(CodeBlock* codeBlock, NodeIndex nodeIndex)
+    {
+        return at(nodeIndex).valueOfBooleanConstant(codeBlock);
+    }
 
 #ifndef NDEBUG
     static const char *opName(NodeType);
@@ -271,9 +219,7 @@ private:
     // When a node's refCount goes from 0 to 1, it must (logically) recursively ref all of its children, and vice versa.
     void refChildren(NodeIndex);
 
-    Vector<PredictionSlot, 16> m_argumentPredictions;
-    Vector<PredictionSlot, 16> m_variablePredictions;
-    HashMap<unsigned, PredictionSlot> m_globalVarPredictions;
+    PredictionTracker m_predictions;
 };
 
 } } // namespace JSC::DFG
index eb15f27..1fdc1e1 100644 (file)
@@ -80,6 +80,8 @@ GPRReg JITCodeGenerator::fillInteger(NodeIndex nodeIndex, DataFormat& returnForm
     case DataFormatJS:
     case DataFormatCell:
     case DataFormatJSCell:
+    case DataFormatBoolean:
+    case DataFormatJSBoolean:
         // Should only be calling this function if we know this operand to be integer.
         ASSERT_NOT_REACHED();
 
@@ -153,6 +155,8 @@ FPRReg JITCodeGenerator::fillDouble(NodeIndex nodeIndex)
         // Should have filled, above.
     case DataFormatCell:
     case DataFormatJSCell:
+    case DataFormatBoolean:
+    case DataFormatJSBoolean:
         // Should only be calling this function if we know this operand to be numeric.
         ASSERT_NOT_REACHED();
 
@@ -295,11 +299,16 @@ GPRReg JITCodeGenerator::fillJSValue(NodeIndex nodeIndex)
     case DataFormatJS:
     case DataFormatJSInteger:
     case DataFormatJSDouble:
-    case DataFormatJSCell: {
+    case DataFormatJSCell:
+    case DataFormatJSBoolean: {
         GPRReg gpr = info.gpr();
         m_gprs.lock(gpr);
         return gpr;
     }
+        
+    case DataFormatBoolean:
+        // this type currently never occurs
+        ASSERT_NOT_REACHED();
     }
 
     ASSERT_NOT_REACHED();
@@ -406,9 +415,26 @@ bool JITCodeGenerator::isKnownNotInteger(NodeIndex nodeIndex)
     
     return (info.registerFormat() | DataFormatJS) == DataFormatJSDouble
         || (info.registerFormat() | DataFormatJS) == DataFormatJSCell
+        || (info.registerFormat() | DataFormatJS) == DataFormatJSBoolean
         || (node.isConstant() && !valueOfJSConstant(nodeIndex).isInt32());
 }
 
+bool JITCodeGenerator::isKnownBoolean(NodeIndex nodeIndex)
+{
+    Node& node = m_jit.graph()[nodeIndex];
+    if (node.hasBooleanResult())
+        return true;
+    
+    if (isBooleanConstant(nodeIndex))
+        return true;
+    
+    VirtualRegister virtualRegister = node.virtualRegister();
+    GenerationInfo& info = m_generationInfo[virtualRegister];
+    
+    return (info.registerFormat() | DataFormatJS) == DataFormatJSBoolean
+        || (info.spillFormat() | DataFormatJS) == DataFormatJSBoolean;
+}
+
 template<typename To, typename From>
 inline To safeCast(From value)
 {
@@ -588,7 +614,7 @@ void JITCodeGenerator::nonSpeculativeNonPeepholeCompareNull(NodeIndex operand, b
     }
     
     m_jit.or32(TrustedImm32(ValueFalse), resultGPR);
-    jsValueResult(resultGPR, m_compileIndex);
+    jsValueResult(resultGPR, m_compileIndex, DataFormatJSBoolean);
 }
 
 void JITCodeGenerator::nonSpeculativePeepholeBranchNull(NodeIndex operand, NodeIndex branchNodeIndex, bool invert)
@@ -743,7 +769,7 @@ void JITCodeGenerator::nonSpeculativeNonPeepholeCompare(Node& node, MacroAssembl
         callOperation(helperFunction, resultGPR, arg1GPR, arg2GPR);
         
         m_jit.or32(TrustedImm32(ValueFalse), resultGPR);
-        jsValueResult(resultGPR, m_compileIndex, UseChildrenCalledExplicitly);
+        jsValueResult(resultGPR, m_compileIndex, DataFormatJSBoolean, UseChildrenCalledExplicitly);
     } else {
         GPRTemporary result(this, arg2);
         GPRReg resultGPR = result.gpr();
@@ -777,7 +803,7 @@ void JITCodeGenerator::nonSpeculativeNonPeepholeCompare(Node& node, MacroAssembl
         
         m_jit.or32(TrustedImm32(ValueFalse), resultGPR);
         
-        jsValueResult(resultGPR, m_compileIndex, UseChildrenCalledExplicitly);
+        jsValueResult(resultGPR, m_compileIndex, DataFormatJSBoolean, UseChildrenCalledExplicitly);
     }
 }
 
@@ -939,7 +965,7 @@ void JITCodeGenerator::nonSpeculativeNonPeepholeStrictEq(Node& node, bool invert
         done2.link(&m_jit);
     }
     
-    jsValueResult(resultGPR, m_compileIndex, UseChildrenCalledExplicitly);
+    jsValueResult(resultGPR, m_compileIndex, DataFormatJSBoolean, UseChildrenCalledExplicitly);
 }
 
 bool JITCodeGenerator::nonSpeculativeStrictEq(Node& node, bool invert)
@@ -968,31 +994,63 @@ void JITCodeGenerator::emitBranch(Node& node)
     JSValueOperand value(this, node.child1());
     GPRReg valueGPR = value.gpr();
     
-    GPRTemporary result(this);
-    GPRReg resultGPR = result.gpr();
-    
-    value.use();
-    
     BlockIndex taken = m_jit.graph().blockIndexForBytecodeOffset(node.takenBytecodeOffset());
     BlockIndex notTaken = m_jit.graph().blockIndexForBytecodeOffset(node.notTakenBytecodeOffset());
-
-    addBranch(m_jit.branchPtr(MacroAssembler::Equal, valueGPR, MacroAssembler::ImmPtr(JSValue::encode(jsNumber(0)))), notTaken);
-    addBranch(m_jit.branchPtr(MacroAssembler::AboveOrEqual, valueGPR, GPRInfo::tagTypeNumberRegister), taken);
-    addBranch(m_jit.branchPtr(MacroAssembler::Equal, valueGPR, MacroAssembler::ImmPtr(JSValue::encode(jsBoolean(false)))), notTaken);
-    addBranch(m_jit.branchPtr(MacroAssembler::Equal, valueGPR, MacroAssembler::ImmPtr(JSValue::encode(jsBoolean(true)))), taken);
     
-    silentSpillAllRegisters(resultGPR);
-    m_jit.move(valueGPR, GPRInfo::argumentGPR1);
-    m_jit.move(GPRInfo::callFrameRegister, GPRInfo::argumentGPR0);
-    appendCallWithExceptionCheck(dfgConvertJSValueToBoolean);
-    m_jit.move(GPRInfo::returnValueGPR, resultGPR);
-    silentFillAllRegisters(resultGPR);
+    if (isKnownBoolean(node.child1())) {
+        MacroAssembler::ResultCondition condition = MacroAssembler::NonZero;
+        
+        if (taken == (m_block + 1)) {
+            condition = MacroAssembler::Zero;
+            BlockIndex tmp = taken;
+            taken = notTaken;
+            notTaken = tmp;
+        }
+        
+        addBranch(m_jit.branchTest32(condition, valueGPR, TrustedImm32(true)), taken);
+        if (notTaken != (m_block + 1))
+            addBranch(m_jit.jump(), notTaken);
+        
+        noResult(m_compileIndex);
+    } else {
+        GPRTemporary result(this);
+        GPRReg resultGPR = result.gpr();
+        
+        bool predictBoolean = isBooleanPrediction(m_jit.graph().getPrediction(m_jit.graph()[node.child1()]));
     
-    addBranch(m_jit.branchTest8(MacroAssembler::NonZero, resultGPR), taken);
-    if (notTaken != (m_block + 1))
-        addBranch(m_jit.jump(), notTaken);
+        if (predictBoolean) {
+            addBranch(m_jit.branchPtr(MacroAssembler::Equal, valueGPR, MacroAssembler::ImmPtr(JSValue::encode(jsBoolean(false)))), notTaken);
+            addBranch(m_jit.branchPtr(MacroAssembler::Equal, valueGPR, MacroAssembler::ImmPtr(JSValue::encode(jsBoolean(true)))), taken);
+        }
+        
+        if (m_isSpeculative && predictBoolean) {
+            speculationCheck(m_jit.jump());
+            value.use();
+        } else {
+            addBranch(m_jit.branchPtr(MacroAssembler::Equal, valueGPR, MacroAssembler::ImmPtr(JSValue::encode(jsNumber(0)))), notTaken);
+            addBranch(m_jit.branchPtr(MacroAssembler::AboveOrEqual, valueGPR, GPRInfo::tagTypeNumberRegister), taken);
+    
+            if (!predictBoolean) {
+                addBranch(m_jit.branchPtr(MacroAssembler::Equal, valueGPR, MacroAssembler::ImmPtr(JSValue::encode(jsBoolean(false)))), notTaken);
+                addBranch(m_jit.branchPtr(MacroAssembler::Equal, valueGPR, MacroAssembler::ImmPtr(JSValue::encode(jsBoolean(true)))), taken);
+            }
     
-    noResult(m_compileIndex, UseChildrenCalledExplicitly);
+            value.use();
+    
+            silentSpillAllRegisters(resultGPR);
+            m_jit.move(valueGPR, GPRInfo::argumentGPR1);
+            m_jit.move(GPRInfo::callFrameRegister, GPRInfo::argumentGPR0);
+            appendCallWithExceptionCheck(dfgConvertJSValueToBoolean);
+            m_jit.move(GPRInfo::returnValueGPR, resultGPR);
+            silentFillAllRegisters(resultGPR);
+    
+            addBranch(m_jit.branchTest8(MacroAssembler::NonZero, resultGPR), taken);
+            if (notTaken != (m_block + 1))
+                addBranch(m_jit.jump(), notTaken);
+        }
+        
+        noResult(m_compileIndex, UseChildrenCalledExplicitly);
+    }
 }
 
 void JITCodeGenerator::emitCall(Node& node)
@@ -1081,6 +1139,12 @@ void JITCodeGenerator::emitCall(Node& node)
     m_jit.addJSCall(fastCall, slowCall, targetToCheck, isCall, m_jit.graph()[m_compileIndex].exceptionInfo);
 }
 
+void JITCodeGenerator::speculationCheck(MacroAssembler::Jump jumpToFail)
+{
+    ASSERT(m_isSpeculative);
+    static_cast<SpeculativeJIT*>(this)->speculationCheck(jumpToFail);
+}
+
 #ifndef NDEBUG
 static const char* dataFormatString(DataFormat format)
 {
@@ -1287,6 +1351,16 @@ GPRTemporary::GPRTemporary(JITCodeGenerator* jit, SpeculateCellOperand& op1)
         m_gpr = m_jit->allocate();
 }
 
+GPRTemporary::GPRTemporary(JITCodeGenerator* jit, SpeculateBooleanOperand& op1)
+    : m_jit(jit)
+    , m_gpr(InvalidGPRReg)
+{
+    if (m_jit->canReuse(op1.index()))
+        m_gpr = m_jit->reuse(op1.gpr());
+    else
+        m_gpr = m_jit->allocate();
+}
+
 GPRTemporary::GPRTemporary(JITCodeGenerator* jit, JSValueOperand& op1)
     : m_jit(jit)
     , m_gpr(InvalidGPRReg)
index 2c728ec..ebcf892 100644 (file)
@@ -41,6 +41,7 @@ class SpeculateIntegerOperand;
 class SpeculateStrictInt32Operand;
 class SpeculateDoubleOperand;
 class SpeculateCellOperand;
+class SpeculateBooleanOperand;
 
 
 // === JITCodeGenerator ===
@@ -410,14 +411,18 @@ protected:
     
     bool isKnownNotInteger(NodeIndex);
 
+    bool isKnownBoolean(NodeIndex);
+    
     // Checks/accessors for constant values.
     bool isConstant(NodeIndex nodeIndex) { return m_jit.isConstant(nodeIndex); }
     bool isJSConstant(NodeIndex nodeIndex) { return m_jit.isJSConstant(nodeIndex); }
     bool isInt32Constant(NodeIndex nodeIndex) { return m_jit.isInt32Constant(nodeIndex); }
     bool isDoubleConstant(NodeIndex nodeIndex) { return m_jit.isDoubleConstant(nodeIndex); }
+    bool isBooleanConstant(NodeIndex nodeIndex) { return m_jit.isBooleanConstant(nodeIndex); }
     int32_t valueOfInt32Constant(NodeIndex nodeIndex) { return m_jit.valueOfInt32Constant(nodeIndex); }
     double valueOfDoubleConstant(NodeIndex nodeIndex) { return m_jit.valueOfDoubleConstant(nodeIndex); }
     JSValue valueOfJSConstant(NodeIndex nodeIndex) { return m_jit.valueOfJSConstant(nodeIndex); }
+    bool valueOfBooleanConstant(NodeIndex nodeIndex) { return m_jit.valueOfBooleanConstant(nodeIndex); }
     bool isNullConstant(NodeIndex nodeIndex)
     {
         if (!isConstant(nodeIndex))
@@ -575,6 +580,8 @@ protected:
     }
     
     void emitCall(Node&);
+    
+    void speculationCheck(MacroAssembler::Jump jumpToFail);
 
     // Called once a node has completed code generation but prior to setting
     // its result, to free up its children. (This must happen prior to setting
@@ -1121,6 +1128,7 @@ public:
     GPRTemporary(JITCodeGenerator*, IntegerOperand&);
     GPRTemporary(JITCodeGenerator*, IntegerOperand&, IntegerOperand&);
     GPRTemporary(JITCodeGenerator*, SpeculateCellOperand&);
+    GPRTemporary(JITCodeGenerator*, SpeculateBooleanOperand&);
     GPRTemporary(JITCodeGenerator*, JSValueOperand&);
 
     ~GPRTemporary()
index 0343876..678eab9 100644 (file)
@@ -488,23 +488,45 @@ void JITCompiler::jumpFromSpeculativeToNonSpeculative(const SpeculationCheck& ch
 #if DFG_JIT_BREAK_ON_SPECULATION_FAILURE
     breakpoint();
 #endif
+    
+#if DFG_VERBOSE_SPECULATION_FAILURE
+    SpeculationFailureDebugInfo* debugInfo = new SpeculationFailureDebugInfo;
+    debugInfo->codeBlock = m_codeBlock;
+    debugInfo->debugOffset = debugOffset();
+    
+    debugCall(debugOperationPrintSpeculationFailure, debugInfo);
+#endif
 
     // Does this speculation check require any additional recovery to be performed,
     // to restore any state that has been overwritten before we enter back in to the
     // non-speculative path.
     if (recovery) {
-        // The only additional recovery we currently support is for integer add operation
-        ASSERT(recovery->type() == SpeculativeAdd);
-        ASSERT(check.m_gprInfo[GPRInfo::toIndex(recovery->dest())].nodeIndex != NoNode);
-        // Revert the add.
-        sub32(recovery->src(), recovery->dest());
-        
-        // If recovery->dest() should have been boxed prior to the addition, then rebox
-        // it.
-        DataFormat format = check.m_gprInfo[GPRInfo::toIndex(recovery->dest())].format;
-        ASSERT(format == DataFormatInteger || format == DataFormatJSInteger || format == DataFormatJS);
-        if (format != DataFormatInteger)
-            orPtr(GPRInfo::tagTypeNumberRegister, recovery->dest());
+        switch (recovery->type()) {
+        case SpeculativeAdd: {
+            ASSERT(check.m_gprInfo[GPRInfo::toIndex(recovery->dest())].nodeIndex != NoNode);
+            // Revert the add.
+            sub32(recovery->src(), recovery->dest());
+            
+            // If recovery->dest() should have been boxed prior to the addition, then rebox
+            // it.
+            DataFormat format = check.m_gprInfo[GPRInfo::toIndex(recovery->dest())].format;
+            ASSERT(format == DataFormatInteger || format == DataFormatJSInteger || format == DataFormatJS);
+            if (format != DataFormatInteger)
+                orPtr(GPRInfo::tagTypeNumberRegister, recovery->dest());
+            break;
+        }
+            
+        case BooleanSpeculationCheck: {
+            ASSERT(check.m_gprInfo[GPRInfo::toIndex(recovery->dest())].nodeIndex != NoNode);
+            // Rebox the (non-)boolean
+            xorPtr(TrustedImm32(static_cast<int32_t>(ValueFalse)), recovery->dest());
+            break;
+        }
+            
+        default:
+            ASSERT_NOT_REACHED();
+            break;
+        }
     }
     
     // First, we need a reverse mapping that tells us, for a NodeIndex, which register
@@ -875,7 +897,7 @@ void JITCompiler::link(LinkBuffer& linkBuffer)
 {
     // Link the code, populate data in CodeBlock data structures.
 #if DFG_DEBUG_VERBOSE
-    fprintf(stderr, "JIT code start at [%p, %p)\n", linkBuffer.debugAddress(), static_cast<char*>(linkBuffer.debugAddress()) + linkBuffer.debugSize());
+    fprintf(stderr, "JIT code for %p start at [%p, %p)\n", m_codeBlock, linkBuffer.debugAddress(), static_cast<char*>(linkBuffer.debugAddress()) + linkBuffer.debugSize());
 #endif
 
     // Link all calls out from the JIT code to their respective functions.
index e648623..91ce37a 100644 (file)
@@ -54,6 +54,17 @@ class SpeculationRecovery;
 struct EntryLocation;
 struct SpeculationCheck;
 
+#ifndef NDEBUG
+typedef void (*V_DFGDebugOperation_EP)(ExecState*, void*);
+#endif
+
+#if DFG_VERBOSE_SPECULATION_FAILURE
+struct SpeculationFailureDebugInfo {
+    CodeBlock* codeBlock;
+    unsigned debugOffset;
+};
+#endif
+
 // === CallRecord ===
 //
 // A record of a call out from JIT code to a helper function.
@@ -205,16 +216,41 @@ public:
         m_calls.append(CallRecord(functionCall, function, exceptionCheck, exceptionInfo));
         return functionCall;
     }
+    
+#ifndef NDEBUG
+    // Add a debug call. This call has no effect on JIT code execution state.
+    void debugCall(V_DFGDebugOperation_EP function, void* argument)
+    {
+        for (unsigned i = 0; i < GPRInfo::numberOfRegisters; ++i)
+            storePtr(GPRInfo::toRegister(i), m_globalData->debugDataBuffer + i);
+        for (unsigned i = 0; i < FPRInfo::numberOfRegisters; ++i) {
+            move(TrustedImmPtr(m_globalData->debugDataBuffer + GPRInfo::numberOfRegisters + i), GPRInfo::regT0);
+            storeDouble(FPRInfo::toRegister(i), GPRInfo::regT0);
+        }
+        move(TrustedImmPtr(argument), GPRInfo::argumentGPR1);
+        move(GPRInfo::callFrameRegister, GPRInfo::argumentGPR0);
+        move(TrustedImmPtr(reinterpret_cast<void*>(function)), GPRInfo::regT0);
+        call(GPRInfo::regT0);
+        for (unsigned i = 0; i < FPRInfo::numberOfRegisters; ++i) {
+            move(TrustedImmPtr(m_globalData->debugDataBuffer + GPRInfo::numberOfRegisters + i), GPRInfo::regT0);
+            loadDouble(GPRInfo::regT0, FPRInfo::toRegister(i));
+        }
+        for (unsigned i = 0; i < GPRInfo::numberOfRegisters; ++i)
+            loadPtr(m_globalData->debugDataBuffer + i, GPRInfo::toRegister(i));
+    }
+#endif
 
     // Helper methods to check nodes for constants.
     bool isConstant(NodeIndex nodeIndex) { return graph().isConstant(nodeIndex); }
     bool isJSConstant(NodeIndex nodeIndex) { return graph().isJSConstant(nodeIndex); }
     bool isInt32Constant(NodeIndex nodeIndex) { return graph().isInt32Constant(codeBlock(), nodeIndex); }
     bool isDoubleConstant(NodeIndex nodeIndex) { return graph().isDoubleConstant(codeBlock(), nodeIndex); }
+    bool isBooleanConstant(NodeIndex nodeIndex) { return graph().isBooleanConstant(codeBlock(), nodeIndex); }
     // Helper methods get constant values from nodes.
     JSValue valueOfJSConstant(NodeIndex nodeIndex) { return graph().valueOfJSConstant(codeBlock(), nodeIndex); }
     int32_t valueOfInt32Constant(NodeIndex nodeIndex) { return graph().valueOfInt32Constant(codeBlock(), nodeIndex); }
     double valueOfDoubleConstant(NodeIndex nodeIndex) { return graph().valueOfDoubleConstant(codeBlock(), nodeIndex); }
+    bool valueOfBooleanConstant(NodeIndex nodeIndex) { return graph().valueOfBooleanConstant(codeBlock(), nodeIndex); }
 
     // These methods JIT generate dynamic, debug-only checks - akin to ASSERTs.
 #if DFG_JIT_ASSERT
index 56ca7cf..1bccbd5 100644 (file)
@@ -44,6 +44,8 @@
 #define DFG_JIT_BREAK_ON_EVERY_NODE 0
 // Emit a breakpoint into the speculation failure code.
 #define DFG_JIT_BREAK_ON_SPECULATION_FAILURE 0
+// Log every speculation failure.
+#define DFG_VERBOSE_SPECULATION_FAILURE 0
 // Disable the DFG JIT without having to touch Platform.h!
 #define DFG_DEBUG_LOCAL_DISBALE 0
 // Disable the SpeculativeJIT without having to touch Platform.h!
@@ -89,6 +91,7 @@ typedef uint32_t ExceptionInfo;
 #define NodeResultJS      0x1000
 #define NodeResultDouble  0x2000
 #define NodeResultInt32   0x3000
+#define NodeResultBoolean 0x4000
 
 // This macro defines a set of information about all known node types, used to populate NodeId, NodeType below.
 #define FOR_EACH_DFG_OP(macro) \
@@ -140,12 +143,12 @@ typedef uint32_t ExceptionInfo;
     macro(PutGlobalVar, NodeMustGenerate) \
     \
     /* Nodes for comparison operations. */\
-    macro(CompareLess, NodeResultJS | NodeMustGenerate) \
-    macro(CompareLessEq, NodeResultJS | NodeMustGenerate) \
-    macro(CompareGreater, NodeResultJS | NodeMustGenerate) \
-    macro(CompareGreaterEq, NodeResultJS | NodeMustGenerate) \
-    macro(CompareEq, NodeResultJS | NodeMustGenerate) \
-    macro(CompareStrictEq, NodeResultJS) \
+    macro(CompareLess, NodeResultBoolean | NodeMustGenerate) \
+    macro(CompareLessEq, NodeResultBoolean | NodeMustGenerate) \
+    macro(CompareGreater, NodeResultBoolean | NodeMustGenerate) \
+    macro(CompareGreaterEq, NodeResultBoolean | NodeMustGenerate) \
+    macro(CompareEq, NodeResultBoolean | NodeMustGenerate) \
+    macro(CompareStrictEq, NodeResultBoolean) \
     \
     /* Calls. */\
     macro(Call, NodeResultJS | NodeMustGenerate | NodeHasVarArgs) \
@@ -159,8 +162,8 @@ typedef uint32_t ExceptionInfo;
     /* Nodes for misc operations. */\
     macro(Breakpoint, NodeMustGenerate) \
     macro(CheckHasInstance, NodeMustGenerate) \
-    macro(InstanceOf, NodeResultJS) \
-    macro(LogicalNot, NodeResultJS) \
+    macro(InstanceOf, NodeResultBoolean) \
+    macro(LogicalNot, NodeResultBoolean) \
     \
     /* Block terminals. */\
     macro(Jump, NodeMustGenerate | NodeIsTerminal | NodeIsJump) \
@@ -198,7 +201,8 @@ static const PredictedType PredictArray  = 0x03;
 static const PredictedType PredictInt32  = 0x04;
 static const PredictedType PredictDouble = 0x08;
 static const PredictedType PredictNumber = 0x0c;
-static const PredictedType PredictTop    = 0x0f;
+static const PredictedType PredictBoolean = 0x10;
+static const PredictedType PredictTop    = 0x1f;
 static const PredictedType StrongPredictionTag = 0x80;
 static const PredictedType PredictionTagMask    = 0x80;
 
@@ -229,6 +233,11 @@ inline bool isNumberPrediction(PredictedType value)
     return !!(value & PredictNumber) && !(value & ~(PredictNumber | PredictionTagMask));
 }
 
+inline bool isBooleanPrediction(PredictedType value)
+{
+    return (value & ~PredictionTagMask) == PredictBoolean;
+}
+
 inline bool isStrongPrediction(PredictedType value)
 {
     ASSERT(value != (PredictNone | StrongPredictionTag));
@@ -248,8 +257,12 @@ inline const char* predictionToString(PredictedType value)
             return "p-strong-array";
         case PredictInt32:
             return "p-strong-int32";
+        case PredictDouble:
+            return "p-strong-double";
         case PredictNumber:
             return "p-strong-number";
+        case PredictBoolean:
+            return "p-strong-boolean";
         default:
             return "p-strong-top";
         }
@@ -263,8 +276,12 @@ inline const char* predictionToString(PredictedType value)
         return "p-weak-array";
     case PredictInt32:
         return "p-weak-int32";
+    case PredictDouble:
+        return "p-weak-double";
     case PredictNumber:
         return "p-weak-number";
+    case PredictBoolean:
+        return "p-weak-boolean";
     default:
         return "p-weak-top";
     }
@@ -326,6 +343,9 @@ inline PredictedType makePrediction(JSGlobalData& globalData, const ValueProfile
     if (statistics.cells == statistics.samples)
         return StrongPredictionTag | PredictCell;
     
+    if (statistics.booleans == statistics.samples)
+        return StrongPredictionTag | PredictBoolean;
+    
     return StrongPredictionTag | PredictTop;
 }
 #endif
@@ -423,6 +443,11 @@ struct Node {
         return isConstant() && valueOfJSConstant(codeBlock).isNumber();
     }
     
+    bool isBooleanConstant(CodeBlock* codeBlock)
+    {
+        return isConstant() && valueOfJSConstant(codeBlock).isBoolean();
+    }
+    
     int32_t valueOfInt32Constant(CodeBlock* codeBlock)
     {
         ASSERT(isInt32Constant(codeBlock));
@@ -434,6 +459,12 @@ struct Node {
         ASSERT(isDoubleConstant(codeBlock));
         return valueOfJSConstant(codeBlock).uncheckedGetNumber();
     }
+    
+    bool valueOfBooleanConstant(CodeBlock* codeBlock)
+    {
+        ASSERT(isBooleanConstant(codeBlock));
+        return valueOfJSConstant(codeBlock).getBoolean();
+    }
 
     bool hasLocal()
     {
@@ -492,13 +523,16 @@ struct Node {
     {
         return (op & NodeResultMask) == NodeResultJS;
     }
+    
+    bool hasBooleanResult()
+    {
+        return (op & NodeResultMask) == NodeResultBoolean;
+    }
 
     // Check for integers or doubles.
     bool hasNumericResult()
     {
-        // This check will need updating if more result types are added.
-        ASSERT((hasInt32Result() || hasDoubleResult()) == !hasJSResult());
-        return !hasJSResult();
+        return hasInt32Result() || hasDoubleResult();
     }
 
     bool isJump()
@@ -563,9 +597,9 @@ struct Node {
         
         ASSERT(source == StrongPrediction);
         
-        PredictedType newPrediction = StrongPredictionTag | prediction;
+        PredictedType newPrediction = StrongPredictionTag | prediction | m_opInfo2;
         bool result = m_opInfo2 != newPrediction;
-        m_opInfo2 |= newPrediction;
+        m_opInfo2 = newPrediction;
         return result;
     }
     
index 6ae20eb..847af5e 100644 (file)
@@ -678,6 +678,14 @@ RegisterSizedBoolean dfgConvertJSValueToBoolean(ExecState* exec, EncodedJSValue
     return JSValue::decode(encodedOp).toBoolean(exec);
 }
 
+#if DFG_VERBOSE_SPECULATION_FAILURE
+void debugOperationPrintSpeculationFailure(ExecState*, void* debugInfoRaw)
+{
+    SpeculationFailureDebugInfo* debugInfo = static_cast<SpeculationFailureDebugInfo*>(debugInfoRaw);
+    printf("Speculation failure in %p at 0x%x!\n", debugInfo->codeBlock, debugInfo->debugOffset);
+}
+#endif
+
 } // extern "C"
 } } // namespace JSC::DFG
 
index 767e7f6..9d36ffd 100644 (file)
@@ -115,6 +115,10 @@ double dfgConvertJSValueToNumber(ExecState*, EncodedJSValue);
 int32_t dfgConvertJSValueToInt32(ExecState*, EncodedJSValue);
 RegisterSizedBoolean dfgConvertJSValueToBoolean(ExecState*, EncodedJSValue);
 
+#if DFG_VERBOSE_SPECULATION_FAILURE
+void debugOperationPrintSpeculationFailure(ExecState*, void*);
+#endif
+
 } // extern "C"
 } } // namespace JSC::DFG
 
diff --git a/Source/JavaScriptCore/dfg/DFGPredictionTracker.h b/Source/JavaScriptCore/dfg/DFGPredictionTracker.h
new file mode 100644 (file)
index 0000000..3b7057d
--- /dev/null
@@ -0,0 +1,170 @@
+/*
+ * Copyright (C) 2011 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#ifndef DFGPredictionTracker_h
+#define DFGPredictionTracker_h
+
+#if ENABLE(DFG_JIT)
+
+#include "DFGNode.h"
+
+namespace JSC { namespace DFG {
+
+// helper function to distinguish vars & temporaries from arguments.
+inline bool operandIsArgument(int operand) { return operand < 0; }
+
+struct PredictionSlot {
+public:
+    PredictionSlot()
+        : m_value(PredictNone)
+    {
+    }
+    PredictedType m_value;
+};
+
+class PredictionTracker {
+public:
+    PredictionTracker()
+    {
+    }
+    
+    PredictionTracker(unsigned numArguments, unsigned numVariables)
+        : m_arguments(numArguments)
+        , m_variables(numVariables)
+    {
+    }
+    
+    void initializeSimilarTo(const PredictionTracker& other)
+    {
+        m_arguments.resize(other.numberOfArguments());
+        m_variables.resize(other.numberOfVariables());
+    }
+    
+    size_t numberOfArguments() const { return m_arguments.size(); }
+    size_t numberOfVariables() const { return m_variables.size(); }
+    
+    unsigned argumentIndexForOperand(int operand) const { return operand + m_arguments.size() + RegisterFile::CallFrameHeaderSize; }
+
+    bool predictArgument(unsigned argument, PredictedType prediction, PredictionSource source)
+    {
+        return mergePrediction(m_arguments[argument].m_value, makePrediction(prediction, source));
+    }
+    
+    bool predict(int operand, PredictedType prediction, PredictionSource source)
+    {
+        if (operandIsArgument(operand))
+            return predictArgument(argumentIndexForOperand(operand), prediction, source);
+        if ((unsigned)operand >= m_variables.size()) {
+            ASSERT(operand >= 0);
+            m_variables.resize(operand + 1);
+        }
+        
+        return mergePrediction(m_variables[operand].m_value, makePrediction(prediction, source));
+    }
+    
+    bool predictGlobalVar(unsigned varNumber, PredictedType prediction, PredictionSource source)
+    {
+        HashMap<unsigned, PredictionSlot>::iterator iter = m_globalVars.find(varNumber + 1);
+        if (iter == m_globalVars.end()) {
+            PredictionSlot predictionSlot;
+            bool result = mergePrediction(predictionSlot.m_value, makePrediction(prediction, source));
+            m_globalVars.add(varNumber + 1, predictionSlot);
+            return result;
+        }
+        return mergePrediction(iter->second.m_value, makePrediction(prediction, source));
+    }
+    
+    bool predict(Node& node, PredictedType prediction, PredictionSource source)
+    {
+        switch (node.op) {
+        case GetLocal:
+            return predict(node.local(), prediction, source);
+            break;
+        case GetGlobalVar:
+            return predictGlobalVar(node.varNumber(), prediction, source);
+            break;
+        case GetById:
+        case GetMethod:
+        case GetByVal:
+        case Call:
+        case Construct:
+            return node.predict(prediction, source);
+        default:
+            return false;
+        }
+    }
+    
+    PredictedType getArgumentPrediction(unsigned argument)
+    {
+        return m_arguments[argument].m_value;
+    }
+
+    PredictedType getPrediction(int operand)
+    {
+        if (operandIsArgument(operand))
+            return getArgumentPrediction(argumentIndexForOperand(operand));
+        if ((unsigned)operand < m_variables.size())
+            return m_variables[operand].m_value;
+        return PredictNone;
+    }
+    
+    PredictedType getGlobalVarPrediction(unsigned varNumber)
+    {
+        HashMap<unsigned, PredictionSlot>::iterator iter = m_globalVars.find(varNumber + 1);
+        if (iter == m_globalVars.end())
+            return PredictNone;
+        return iter->second.m_value;
+    }
+    
+    PredictedType getPrediction(Node& node)
+    {
+        switch (node.op) {
+        case GetLocal:
+            return getPrediction(node.local());
+        case GetGlobalVar:
+            return getGlobalVarPrediction(node.varNumber());
+        case GetById:
+        case GetMethod:
+        case GetByVal:
+        case Call:
+        case Construct:
+            return node.getPrediction();
+        default:
+            return PredictNone;
+        }
+    }
+    
+private:
+    Vector<PredictionSlot, 16> m_arguments;
+    Vector<PredictionSlot, 16> m_variables;
+    HashMap<unsigned, PredictionSlot> m_globalVars;
+};
+
+} } // namespace JSC::DFG
+
+#endif // ENABLE(DFG_JIT)
+
+#endif // DFGPredictionTracker_h
+
index 82681dd..c616fb6 100644 (file)
@@ -40,9 +40,28 @@ public:
         , m_codeBlock(codeBlock)
         , m_profiledBlock(profiledBlock)
     {
+        // Predictions is a forward flow property that propagates the values seen at
+        // a particular value source to their various uses, ensuring that uses perform
+        // speculation that does not contravene the expected values.
         m_predictions.resize(m_graph.size());
-        for (NodeIndex i = 0; i < m_graph.size(); ++i)
+        
+        // Uses is a backward flow property that propagates the hard expectations at
+        // certain uses to their value sources, ensuring that predictions about
+        // values do not contravene the code itself. This comes up only in the
+        // cases of obvious cell uses, like GetById and friends as well as Call.
+        // We're essentially statically speculating that if the value profile indicates
+        // that only booleans (for example) flow into a GetById, then the value
+        // profile is simply wrong due to insufficient coverage and needs to be
+        // adjusted accordingly. The alternatives would be to assume either
+        // that the GetById never executes, or always executes on a boolean leading
+        // to whatever bizarre behavior that's supposed to cause.
+        m_uses.resize(m_graph.size());
+        m_variableUses.initializeSimilarTo(m_graph.predictions());
+        
+        for (unsigned i = 0; i < m_graph.size(); ++i) {
             m_predictions[i] = PredictNone;
+            m_uses[i] = PredictNone;
+        }
     }
     
     void fixpoint()
@@ -51,9 +70,10 @@ public:
         m_count = 0;
 #endif
         do {
+            m_changed = false;
+            
             // Forward propagation is near-optimal for both topologically-sorted and
             // DFS-sorted code.
-            m_changed = false;
             propagateForward();
             if (!m_changed)
                 break;
@@ -64,30 +84,33 @@ public:
             // found a sound solution and (2) short-circuits backward flow.
             m_changed = false;
             propagateBackward();
-            if (!m_changed)
-                break;
-            
-            // Do another forward pass because forward passes are on average more
-            // profitable.
-            m_changed = false;
-            propagateForward();
         } while (m_changed);
     }
     
 private:
-    void setPrediction(PredictedType prediction)
+    bool setPrediction(PredictedType prediction)
     {
+        ASSERT(m_graph[m_compileIndex].hasResult());
+        
         if (m_predictions[m_compileIndex] == prediction)
-            return;
+            return false;
         
         m_predictions[m_compileIndex] = prediction;
-        m_changed = true;
+        return true;
     }
     
-    void mergePrediction(PredictedType prediction)
+    bool mergeUse(NodeIndex nodeIndex, PredictedType prediction)
     {
-        if (DFG::mergePrediction(m_predictions[m_compileIndex], prediction))
-            m_changed = true;
+        ASSERT(m_graph[nodeIndex].hasResult());
+        
+        return DFG::mergePrediction(m_uses[nodeIndex], prediction);
+    }
+    
+    bool mergePrediction(PredictedType prediction)
+    {
+        ASSERT(m_graph[m_compileIndex].hasResult());
+        
+        return DFG::mergePrediction(m_predictions[m_compileIndex], prediction);
     }
     
     void propagateNode(Node& node)
@@ -96,33 +119,46 @@ private:
             return;
         
         NodeType op = node.op;
+
+#if DFG_DEBUG_VERBOSE
+        printf("   %s[%u]: ", Graph::opName(op), m_compileIndex);
+#endif
+        
+        bool changed = false;
+        
         switch (op) {
         case JSConstant: {
             JSValue value = node.valueOfJSConstant(m_codeBlock);
             if (value.isInt32())
-                setPrediction(makePrediction(PredictInt32, StrongPrediction));
+                changed |= setPrediction(makePrediction(PredictInt32, StrongPrediction));
             else if (value.isDouble())
-                setPrediction(makePrediction(PredictDouble, StrongPrediction));
+                changed |= setPrediction(makePrediction(PredictDouble, StrongPrediction));
             else if (value.isCell()) {
                 JSCell* cell = value.asCell();
                 if (isJSArray(&m_globalData, cell))
-                    setPrediction(makePrediction(PredictArray, StrongPrediction));
+                    changed |= setPrediction(makePrediction(PredictArray, StrongPrediction));
                 else
-                    setPrediction(makePrediction(PredictCell, StrongPrediction));
-            } else
-                setPrediction(makePrediction(PredictTop, StrongPrediction));
+                    changed |= setPrediction(makePrediction(PredictCell, StrongPrediction));
+            } else if (value.isBoolean())
+                changed |= setPrediction(makePrediction(PredictBoolean, StrongPrediction));
+            else
+                changed |= setPrediction(makePrediction(PredictTop, StrongPrediction));
             break;
         }
             
         case GetLocal: {
+            changed |= m_graph.predict(node.local(), m_uses[m_compileIndex] & ~PredictionTagMask, StrongPrediction);
+            changed |= m_variableUses.predict(node.local(), m_uses[m_compileIndex] & ~PredictionTagMask, StrongPrediction);
+
             PredictedType prediction = m_graph.getPrediction(node.local());
             if (isStrongPrediction(prediction))
-                mergePrediction(prediction);
+                changed |= mergePrediction(prediction);
             break;
         }
             
         case SetLocal: {
-            m_changed |= m_graph.predict(node.local(), m_predictions[node.child1()] & ~PredictionTagMask, StrongPrediction);
+            changed |= m_graph.predict(node.local(), m_predictions[node.child1()] & ~PredictionTagMask, StrongPrediction);
+            changed |= mergeUse(node.child1(), m_variableUses.getPrediction(node.local()));
             break;
         }
             
@@ -135,7 +171,7 @@ private:
         case UInt32ToNumber:
         case ValueToInt32:
         case ArithMod: {
-            setPrediction(makePrediction(PredictInt32, StrongPrediction));
+            changed |= setPrediction(makePrediction(PredictInt32, StrongPrediction));
             break;
         }
             
@@ -144,10 +180,11 @@ private:
             
             if (isStrongPrediction(prediction)) {
                 if (isNumberPrediction(prediction))
-                    mergePrediction(prediction);
+                    changed |= mergePrediction(prediction);
                 else
-                    mergePrediction(makePrediction(PredictNumber, StrongPrediction));
+                    changed |= mergePrediction(makePrediction(PredictNumber, StrongPrediction));
             }
+            
             break;
         }
             
@@ -158,11 +195,11 @@ private:
             if (isStrongPrediction(left) && isStrongPrediction(right)) {
                 if (isNumberPrediction(left) && isNumberPrediction(right)) {
                     if (isInt32Prediction(mergePredictions(left, right)))
-                        mergePrediction(makePrediction(PredictInt32, StrongPrediction));
+                        changed |= mergePrediction(makePrediction(PredictInt32, StrongPrediction));
                     else
-                        mergePrediction(makePrediction(PredictDouble, StrongPrediction));
+                        changed |= mergePrediction(makePrediction(PredictDouble, StrongPrediction));
                 } else
-                    mergePrediction(makePrediction(PredictTop, StrongPrediction));
+                    changed |= mergePrediction(makePrediction(PredictTop, StrongPrediction));
             }
             break;
         }
@@ -175,15 +212,15 @@ private:
             
             if (isStrongPrediction(left) && isStrongPrediction(right)) {
                 if (isInt32Prediction(mergePredictions(left, right)))
-                    mergePrediction(makePrediction(PredictInt32, StrongPrediction));
+                    changed |= mergePrediction(makePrediction(PredictInt32, StrongPrediction));
                 else
-                    mergePrediction(makePrediction(PredictDouble, StrongPrediction));
+                    changed |= mergePrediction(makePrediction(PredictDouble, StrongPrediction));
             }
             break;
         }
             
         case ArithDiv: {
-            setPrediction(makePrediction(PredictDouble, StrongPrediction));
+            changed |= setPrediction(makePrediction(PredictDouble, StrongPrediction));
             break;
         }
             
@@ -195,48 +232,60 @@ private:
         case CompareEq:
         case CompareStrictEq:
         case InstanceOf: {
-            // This is sad, as we don't have a boolean prediction, even though we easily
-            // could.
-            setPrediction(makePrediction(PredictTop, StrongPrediction));
+            changed |= setPrediction(makePrediction(PredictBoolean, StrongPrediction));
             break;
         }
             
         case GetById:
         case GetMethod:
-        case GetByVal:
+        case GetByVal: {
+            changed |= mergeUse(node.child1(), PredictCell | StrongPredictionTag);
+            changed |= node.predict(m_uses[m_compileIndex] & ~PredictionTagMask, StrongPrediction);
+            if (isStrongPrediction(node.getPrediction()))
+                changed |= setPrediction(node.getPrediction());
+            break;
+        }
+            
         case Call:
         case Construct: {
+            changed |= mergeUse(m_graph.m_varArgChildren[node.firstChild()], PredictCell | StrongPredictionTag);
+            changed |= node.predict(m_uses[m_compileIndex] & ~PredictionTagMask, StrongPrediction);
             if (isStrongPrediction(node.getPrediction()))
-                setPrediction(node.getPrediction());
+                changed |= setPrediction(node.getPrediction());
             break;
         }
             
         case ConvertThis: {
-            setPrediction(makePrediction(PredictCell, StrongPrediction));
+            changed |= setPrediction(makePrediction(PredictCell, StrongPrediction));
             break;
         }
             
         case GetGlobalVar: {
+            changed |= m_variableUses.predictGlobalVar(node.varNumber(), m_uses[m_compileIndex] & ~PredictionTagMask, StrongPrediction);
             PredictedType prediction = m_graph.getGlobalVarPrediction(node.varNumber());
             if (isStrongPrediction(prediction))
-                mergePrediction(prediction);
+                changed |= mergePrediction(prediction);
             break;
         }
             
         case PutGlobalVar: {
-            m_changed |= m_graph.predictGlobalVar(node.varNumber(), m_predictions[node.child1()] & ~PredictionTagMask, StrongPrediction);
+            changed |= m_graph.predictGlobalVar(node.varNumber(), m_predictions[node.child1()] & ~PredictionTagMask, StrongPrediction);
+            changed |= mergeUse(node.child1(), m_variableUses.getGlobalVarPrediction(node.varNumber()));
             break;
         }
             
-#ifndef NDEBUG
-        // These get ignored because they don't return anything.
         case PutByVal:
         case PutByValAlias:
+        case PutById:
+        case PutByIdDirect:
+            changed |= mergeUse(node.child1(), PredictCell | StrongPredictionTag);
+            break;
+
+#ifndef NDEBUG
+        // These get ignored because they don't return anything.
         case DFG::Jump:
         case Branch:
         case Return:
-        case PutById:
-        case PutByIdDirect:
         case CheckHasInstance:
         case Phi:
             break;
@@ -255,6 +304,12 @@ private:
             break;
 #endif
         }
+
+#if DFG_DEBUG_VERBOSE
+        printf("expect(%s) use(%s) %s\n", predictionToString(m_predictions[m_compileIndex]), predictionToString(m_uses[m_compileIndex]), changed ? "CHANGED" : "");
+#endif
+        
+        m_changed |= changed;
     }
     
     void propagateForward()
@@ -283,6 +338,9 @@ private:
     NodeIndex m_compileIndex;
     
     Vector<PredictedType, 16> m_predictions;
+    Vector<PredictedType, 16> m_uses;
+    
+    PredictionTracker m_variableUses;
 
 #if DFG_DEBUG_VERBOSE
     unsigned m_count;
index 0799aba..167340f 100644 (file)
@@ -128,8 +128,10 @@ GPRReg SpeculativeJIT::fillSpeculateIntInternal(NodeIndex nodeIndex, DataFormat&
 
     case DataFormatDouble:
     case DataFormatCell:
+    case DataFormatBoolean:
     case DataFormatJSDouble:
-    case DataFormatJSCell: {
+    case DataFormatJSCell:
+    case DataFormatJSBoolean: {
         terminateSpeculativeExecution();
         returnFormat = DataFormatInteger;
         return allocate();
@@ -223,13 +225,14 @@ FPRReg SpeculativeJIT::fillSpeculateDouble(NodeIndex nodeIndex)
     }
 
     switch (info.registerFormat()) {
-    case DataFormatNone:
-        // Should have filled, above.
+    case DataFormatNone: // Should have filled, above.
+    case DataFormatBoolean: // This type never occurs.
         ASSERT_NOT_REACHED();
         
     case DataFormatCell:
     case DataFormatJSCell:
-    case DataFormatJS: {
+    case DataFormatJS:
+    case DataFormatJSBoolean: {
         GPRReg jsValueGpr = info.gpr();
         m_gprs.lock(jsValueGpr);
         FPRReg fpr = fprAllocate();
@@ -349,7 +352,76 @@ GPRReg SpeculativeJIT::fillSpeculateCell(NodeIndex nodeIndex)
     case DataFormatJSInteger:
     case DataFormatInteger:
     case DataFormatJSDouble:
-    case DataFormatDouble: {
+    case DataFormatDouble:
+    case DataFormatJSBoolean:
+    case DataFormatBoolean: {
+        terminateSpeculativeExecution();
+        return allocate();
+    }
+    }
+
+    ASSERT_NOT_REACHED();
+    return InvalidGPRReg;
+}
+
+GPRReg SpeculativeJIT::fillSpeculateBoolean(NodeIndex nodeIndex)
+{
+    Node& node = m_jit.graph()[nodeIndex];
+    VirtualRegister virtualRegister = node.virtualRegister();
+    GenerationInfo& info = m_generationInfo[virtualRegister];
+
+    switch (info.registerFormat()) {
+    case DataFormatNone: {
+        GPRReg gpr = allocate();
+
+        if (node.isConstant()) {
+            JSValue jsValue = valueOfJSConstant(nodeIndex);
+            if (jsValue.isBoolean()) {
+                m_gprs.retain(gpr, virtualRegister, SpillOrderConstant);
+                m_jit.move(MacroAssembler::TrustedImmPtr(JSValue::encode(jsValue)), gpr);
+                info.fillJSValue(gpr, DataFormatJSBoolean);
+                return gpr;
+            }
+            terminateSpeculativeExecution();
+            return gpr;
+        }
+        ASSERT(info.spillFormat() & DataFormatJS);
+        m_gprs.retain(gpr, virtualRegister, SpillOrderSpilled);
+        m_jit.loadPtr(JITCompiler::addressFor(virtualRegister), gpr);
+
+        info.fillJSValue(gpr, DataFormatJS);
+        if (info.spillFormat() != DataFormatJSBoolean) {
+            m_jit.xorPtr(TrustedImm32(static_cast<int32_t>(ValueFalse)), gpr);
+            speculationCheck(m_jit.branchTestPtr(MacroAssembler::NonZero, gpr, TrustedImm32(static_cast<int32_t>(~1))), SpeculationRecovery(BooleanSpeculationCheck, gpr, InvalidGPRReg));
+            m_jit.xorPtr(TrustedImm32(static_cast<int32_t>(ValueFalse)), gpr);
+        }
+        info.fillJSValue(gpr, DataFormatJSBoolean);
+        return gpr;
+    }
+
+    case DataFormatBoolean:
+    case DataFormatJSBoolean: {
+        GPRReg gpr = info.gpr();
+        m_gprs.lock(gpr);
+        return gpr;
+    }
+
+    case DataFormatJS: {
+        GPRReg gpr = info.gpr();
+        m_gprs.lock(gpr);
+        m_jit.xorPtr(TrustedImm32(static_cast<int32_t>(ValueFalse)), gpr);
+        speculationCheck(m_jit.branchTestPtr(MacroAssembler::NonZero, gpr, TrustedImm32(static_cast<int32_t>(~1))), SpeculationRecovery(BooleanSpeculationCheck, gpr, InvalidGPRReg));
+        m_jit.xorPtr(TrustedImm32(static_cast<int32_t>(ValueFalse)), gpr);
+        info.fillJSValue(gpr, DataFormatJSBoolean);
+        return gpr;
+    }
+
+    case DataFormatJSInteger:
+    case DataFormatInteger:
+    case DataFormatJSDouble:
+    case DataFormatDouble:
+    case DataFormatJSCell:
+    case DataFormatCell: {
         terminateSpeculativeExecution();
         return allocate();
     }
@@ -528,7 +600,7 @@ bool SpeculativeJIT::compare(Node& node, MacroAssembler::RelationalCondition con
         
         // If we add a DataFormatBool, we should use it here.
         m_jit.or32(TrustedImm32(ValueFalse), result.gpr());
-        jsValueResult(result.gpr(), m_compileIndex);
+        jsValueResult(result.gpr(), m_compileIndex, DataFormatJSBoolean);
     }
     
     return false;
@@ -561,7 +633,16 @@ void SpeculativeJIT::compile(Node& node)
             // and don't represent values within this dataflow with virtual registers.
             VirtualRegister virtualRegister = node.virtualRegister();
             m_gprs.retain(result.gpr(), virtualRegister, SpillOrderJS);
-            m_generationInfo[virtualRegister].initJSValue(m_compileIndex, node.refCount(), result.gpr(), isArrayPrediction(prediction) ? DataFormatJSCell : DataFormatJS);
+            
+            DataFormat format;
+            if (isArrayPrediction(prediction))
+                format = DataFormatJSCell;
+            else if (isBooleanPrediction(prediction))
+                format = DataFormatJSBoolean;
+            else
+                format = DataFormatJS;
+            
+            m_generationInfo[virtualRegister].initJSValue(m_compileIndex, node.refCount(), result.gpr(), format);
         }
         break;
     }
@@ -578,6 +659,10 @@ void SpeculativeJIT::compile(Node& node)
             speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(cellGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsArrayVPtr)));
             m_jit.storePtr(cellGPR, JITCompiler::addressFor(node.local()));
             noResult(m_compileIndex);
+        } else if (isBooleanPrediction(predictedType)) {
+            SpeculateBooleanOperand boolean(this, node.child1());
+            m_jit.storePtr(boolean.gpr(), JITCompiler::addressFor(node.local()));
+            noResult(m_compileIndex);
         } else {
             JSValueOperand value(this, node.child1());
             m_jit.storePtr(value.gpr(), JITCompiler::addressFor(node.local()));
@@ -841,6 +926,17 @@ void SpeculativeJIT::compile(Node& node)
     }
 
     case LogicalNot: {
+        if (isKnownBoolean(node.child1())) {
+            SpeculateBooleanOperand value(this, node.child1());
+            GPRTemporary result(this, value);
+            
+            m_jit.move(value.gpr(), result.gpr());
+            m_jit.xorPtr(TrustedImm32(true), result.gpr());
+            
+            jsValueResult(result.gpr(), m_compileIndex, DataFormatJSBoolean);
+            break;
+        }
+        
         JSValueOperand value(this, node.child1());
         GPRTemporary result(this); // FIXME: We could reuse, but on speculation fail would need recovery to restore tag (akin to add).
 
@@ -850,7 +946,7 @@ void SpeculativeJIT::compile(Node& node)
         m_jit.xorPtr(TrustedImm32(static_cast<int32_t>(ValueTrue)), result.gpr());
 
         // If we add a DataFormatBool, we should use it here.
-        jsValueResult(result.gpr(), m_compileIndex);
+        jsValueResult(result.gpr(), m_compileIndex, DataFormatJSBoolean);
         break;
     }
 
@@ -1227,7 +1323,7 @@ void SpeculativeJIT::compile(Node& node)
         m_jit.move(MacroAssembler::TrustedImmPtr(JSValue::encode(jsBoolean(true))), scratchReg);
 
         putResult.link(&m_jit);
-        jsValueResult(scratchReg, m_compileIndex);
+        jsValueResult(scratchReg, m_compileIndex, DataFormatJSBoolean);
         break;
     }
 
index cc46f20..ebaf619 100644 (file)
@@ -37,7 +37,8 @@ class SpeculativeJIT;
 // This enum describes the types of additional recovery that
 // may need be performed should a speculation check fail.
 enum SpeculationRecoveryType {
-    SpeculativeAdd
+    SpeculativeAdd,
+    BooleanSpeculationCheck
 };
 
 // === SpeculationRecovery ===
@@ -132,8 +133,11 @@ public:
     GPRReg fillSpeculateIntStrict(NodeIndex);
     FPRReg fillSpeculateDouble(NodeIndex);
     GPRReg fillSpeculateCell(NodeIndex);
+    GPRReg fillSpeculateBoolean(NodeIndex);
 
 private:
+    friend class JITCodeGenerator;
+    
     void compile(Node&);
     void compile(BasicBlock&);
 
@@ -408,6 +412,47 @@ private:
     GPRReg m_gprOrInvalid;
 };
 
+class SpeculateBooleanOperand {
+public:
+    explicit SpeculateBooleanOperand(SpeculativeJIT* jit, NodeIndex index)
+        : m_jit(jit)
+        , m_index(index)
+        , m_gprOrInvalid(InvalidGPRReg)
+    {
+        ASSERT(m_jit);
+        if (jit->isFilled(index))
+            gpr();
+    }
+    
+    ~SpeculateBooleanOperand()
+    {
+        ASSERT(m_gprOrInvalid != InvalidGPRReg);
+        m_jit->unlock(m_gprOrInvalid);
+    }
+    
+    NodeIndex index() const
+    {
+        return m_index;
+    }
+    
+    GPRReg gpr()
+    {
+        if (m_gprOrInvalid == InvalidGPRReg)
+            m_gprOrInvalid = m_jit->fillSpeculateBoolean(index());
+        return m_gprOrInvalid;
+    }
+    
+    void use()
+    {
+        m_jit->use(m_index);
+    }
+
+private:
+    SpeculativeJIT* m_jit;
+    NodeIndex m_index;
+    GPRReg m_gprOrInvalid;
+};
+
 
 // === SpeculationCheckIndexIterator ===
 //
index 262dadb..445b970 100644 (file)
@@ -232,6 +232,9 @@ namespace JSC {
 #if ENABLE(JIT)
         ReturnAddressPtr exceptionLocation;
         JSValue hostCallReturnValue;
+#ifndef NDEBUG
+        int64_t debugDataBuffer[64];
+#endif
 #endif
 
         HashMap<OpaqueJSClass*, OpaqueJSClassContextData*> opaqueJSClassData;
index 10d3bf1..581b92b 100644 (file)
@@ -126,9 +126,18 @@ char* JSValue::description()
         snprintf(description, size, "<JSValue()>");
     else if (isInt32())
         snprintf(description, size, "Int32: %d", asInt32());
-    else if (isDouble())
-        snprintf(description, size, "Double: %lf", asDouble());
-    else if (isCell())
+    else if (isDouble()) {
+#if USE(JSVALUE64)
+        snprintf(description, size, "Double: %lf, %lx", asDouble(), reinterpretDoubleToIntptr(asDouble()));
+#else
+        union {
+            double asDouble;
+            uint32_t asTwoInt32s[2];
+        } u;
+        u.asDouble = asDouble();
+        snprintf(description, size, "Double: %lf, %08x:%08x", asDouble(), u.asTwoInt32s[1], u.asTwoInt32s[0]);
+#endif
+    } else if (isCell())
         snprintf(description, size, "Cell: %p", asCell());
     else if (isTrue())
         snprintf(description, size, "True");