FTL should be able to convey branch weights to LLVM
authorfpizlo@apple.com <fpizlo@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 19 Feb 2014 22:29:43 +0000 (22:29 +0000)
committerfpizlo@apple.com <fpizlo@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 19 Feb 2014 22:29:43 +0000 (22:29 +0000)
https://bugs.webkit.org/show_bug.cgi?id=129054

Reviewed by Michael Saboff.

This introduces a really nice way to convey branch weights to LLVM. The basic class
is Weight, which just wraps a float; NaN is used when you are not sure. You can
pass this alongside a LBasicBlock to branching instructions like condbr and switch.
But for simplicity, you can just pass a WeightedTarget, which is a tuple of the
two. And for even greater simplicity, you can create WeightedTargets from
LBasicBlocks by doing:

    usually(b)   => WeightedTarget(b, Weight(1))
    rarely(b)    => WeightedTarget(b, Weight(0))
    unsure(b)    => WeightedTarget(b, Weight()) or WeightedTarget(b, Weight(NaN))

This allows for constructs like:

    m_out.branch(isCell(value), usually(isCellCase), rarely(slowCase));

This was intended to be perf-neutral for now, but it did end up creating a ~1%
speed-up on V8v7 and Octane2.

* JavaScriptCore.xcodeproj/project.pbxproj:
* ftl/FTLAbbreviations.h:
(JSC::FTL::mdNode):
* ftl/FTLCommonValues.cpp:
(JSC::FTL::CommonValues::CommonValues):
* ftl/FTLCommonValues.h:
* ftl/FTLLowerDFGToLLVM.cpp:
(JSC::FTL::LowerDFGToLLVM::lower):
(JSC::FTL::LowerDFGToLLVM::compileValueToInt32):
(JSC::FTL::LowerDFGToLLVM::compileStoreBarrierWithNullCheck):
(JSC::FTL::LowerDFGToLLVM::compileToThis):
(JSC::FTL::LowerDFGToLLVM::compileArithMul):
(JSC::FTL::LowerDFGToLLVM::compileArithDiv):
(JSC::FTL::LowerDFGToLLVM::compileArithMod):
(JSC::FTL::LowerDFGToLLVM::compileArithMinOrMax):
(JSC::FTL::LowerDFGToLLVM::compileCheckStructure):
(JSC::FTL::LowerDFGToLLVM::compileArrayifyToStructure):
(JSC::FTL::LowerDFGToLLVM::compileGetById):
(JSC::FTL::LowerDFGToLLVM::compileGetIndexedPropertyStorage):
(JSC::FTL::LowerDFGToLLVM::compileGetTypedArrayByteOffset):
(JSC::FTL::LowerDFGToLLVM::compileGetByVal):
(JSC::FTL::LowerDFGToLLVM::compilePutByVal):
(JSC::FTL::LowerDFGToLLVM::compileArrayPush):
(JSC::FTL::LowerDFGToLLVM::compileArrayPop):
(JSC::FTL::LowerDFGToLLVM::compileNewArrayWithSize):
(JSC::FTL::LowerDFGToLLVM::compileToString):
(JSC::FTL::LowerDFGToLLVM::compileToPrimitive):
(JSC::FTL::LowerDFGToLLVM::compileStringCharAt):
(JSC::FTL::LowerDFGToLLVM::compileStringCharCodeAt):
(JSC::FTL::LowerDFGToLLVM::compileMultiGetByOffset):
(JSC::FTL::LowerDFGToLLVM::compileNotifyWrite):
(JSC::FTL::LowerDFGToLLVM::compileBranch):
(JSC::FTL::LowerDFGToLLVM::compileSwitch):
(JSC::FTL::LowerDFGToLLVM::compareEqObjectOrOtherToObject):
(JSC::FTL::LowerDFGToLLVM::nonSpeculativeCompare):
(JSC::FTL::LowerDFGToLLVM::allocateCell):
(JSC::FTL::LowerDFGToLLVM::allocateBasicStorageAndGetEnd):
(JSC::FTL::LowerDFGToLLVM::boolify):
(JSC::FTL::LowerDFGToLLVM::equalNullOrUndefined):
(JSC::FTL::LowerDFGToLLVM::contiguousPutByValOutOfBounds):
(JSC::FTL::LowerDFGToLLVM::buildSwitch):
(JSC::FTL::LowerDFGToLLVM::doubleToInt32):
(JSC::FTL::LowerDFGToLLVM::sensibleDoubleToInt32):
(JSC::FTL::LowerDFGToLLVM::lowDouble):
(JSC::FTL::LowerDFGToLLVM::strictInt52ToJSValue):
(JSC::FTL::LowerDFGToLLVM::speculateObjectOrOther):
(JSC::FTL::LowerDFGToLLVM::speculateStringOrStringObject):
(JSC::FTL::LowerDFGToLLVM::emitStoreBarrier):
(JSC::FTL::LowerDFGToLLVM::callCheck):
(JSC::FTL::LowerDFGToLLVM::appendOSRExit):
* ftl/FTLOutput.cpp:
(JSC::FTL::Output::initialize):
(JSC::FTL::Output::appendTo):
(JSC::FTL::Output::newBlock):
(JSC::FTL::Output::sensibleDoubleToInt):
(JSC::FTL::Output::load):
(JSC::FTL::Output::store):
(JSC::FTL::Output::baseIndex):
(JSC::FTL::Output::branch):
(JSC::FTL::Output::crashNonTerminal):
* ftl/FTLOutput.h:
(JSC::FTL::Output::branch):
(JSC::FTL::Output::switchInstruction):
* ftl/FTLSwitchCase.h:
(JSC::FTL::SwitchCase::SwitchCase):
(JSC::FTL::SwitchCase::weight):
* ftl/FTLWeight.h: Added.
(JSC::FTL::Weight::Weight):
(JSC::FTL::Weight::isSet):
(JSC::FTL::Weight::operator!):
(JSC::FTL::Weight::value):
(JSC::FTL::Weight::scaleToTotal):
* ftl/FTLWeightedTarget.h: Added.
(JSC::FTL::WeightedTarget::WeightedTarget):
(JSC::FTL::WeightedTarget::target):
(JSC::FTL::WeightedTarget::weight):
(JSC::FTL::usually):
(JSC::FTL::rarely):
(JSC::FTL::unsure):

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@164389 268f45cc-cd09-0410-ab3c-d52691b4dbfc

Source/JavaScriptCore/ChangeLog
Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj
Source/JavaScriptCore/ftl/FTLAbbreviations.h
Source/JavaScriptCore/ftl/FTLCommonValues.cpp
Source/JavaScriptCore/ftl/FTLCommonValues.h
Source/JavaScriptCore/ftl/FTLLowerDFGToLLVM.cpp
Source/JavaScriptCore/ftl/FTLOutput.cpp
Source/JavaScriptCore/ftl/FTLOutput.h
Source/JavaScriptCore/ftl/FTLSwitchCase.h
Source/JavaScriptCore/ftl/FTLWeight.h [new file with mode: 0644]
Source/JavaScriptCore/ftl/FTLWeightedTarget.h [new file with mode: 0644]

index 57039d6..ff80e23 100644 (file)
@@ -1,3 +1,108 @@
+2014-02-19  Filip Pizlo  <fpizlo@apple.com>
+
+        FTL should be able to convey branch weights to LLVM
+        https://bugs.webkit.org/show_bug.cgi?id=129054
+
+        Reviewed by Michael Saboff.
+        
+        This introduces a really nice way to convey branch weights to LLVM. The basic class
+        is Weight, which just wraps a float; NaN is used when you are not sure. You can
+        pass this alongside a LBasicBlock to branching instructions like condbr and switch.
+        But for simplicity, you can just pass a WeightedTarget, which is a tuple of the
+        two. And for even greater simplicity, you can create WeightedTargets from
+        LBasicBlocks by doing:
+        
+            usually(b)   => WeightedTarget(b, Weight(1))
+            rarely(b)    => WeightedTarget(b, Weight(0))
+            unsure(b)    => WeightedTarget(b, Weight()) or WeightedTarget(b, Weight(NaN))
+        
+        This allows for constructs like:
+        
+            m_out.branch(isCell(value), usually(isCellCase), rarely(slowCase));
+        
+        This was intended to be perf-neutral for now, but it did end up creating a ~1%
+        speed-up on V8v7 and Octane2.
+
+        * JavaScriptCore.xcodeproj/project.pbxproj:
+        * ftl/FTLAbbreviations.h:
+        (JSC::FTL::mdNode):
+        * ftl/FTLCommonValues.cpp:
+        (JSC::FTL::CommonValues::CommonValues):
+        * ftl/FTLCommonValues.h:
+        * ftl/FTLLowerDFGToLLVM.cpp:
+        (JSC::FTL::LowerDFGToLLVM::lower):
+        (JSC::FTL::LowerDFGToLLVM::compileValueToInt32):
+        (JSC::FTL::LowerDFGToLLVM::compileStoreBarrierWithNullCheck):
+        (JSC::FTL::LowerDFGToLLVM::compileToThis):
+        (JSC::FTL::LowerDFGToLLVM::compileArithMul):
+        (JSC::FTL::LowerDFGToLLVM::compileArithDiv):
+        (JSC::FTL::LowerDFGToLLVM::compileArithMod):
+        (JSC::FTL::LowerDFGToLLVM::compileArithMinOrMax):
+        (JSC::FTL::LowerDFGToLLVM::compileCheckStructure):
+        (JSC::FTL::LowerDFGToLLVM::compileArrayifyToStructure):
+        (JSC::FTL::LowerDFGToLLVM::compileGetById):
+        (JSC::FTL::LowerDFGToLLVM::compileGetIndexedPropertyStorage):
+        (JSC::FTL::LowerDFGToLLVM::compileGetTypedArrayByteOffset):
+        (JSC::FTL::LowerDFGToLLVM::compileGetByVal):
+        (JSC::FTL::LowerDFGToLLVM::compilePutByVal):
+        (JSC::FTL::LowerDFGToLLVM::compileArrayPush):
+        (JSC::FTL::LowerDFGToLLVM::compileArrayPop):
+        (JSC::FTL::LowerDFGToLLVM::compileNewArrayWithSize):
+        (JSC::FTL::LowerDFGToLLVM::compileToString):
+        (JSC::FTL::LowerDFGToLLVM::compileToPrimitive):
+        (JSC::FTL::LowerDFGToLLVM::compileStringCharAt):
+        (JSC::FTL::LowerDFGToLLVM::compileStringCharCodeAt):
+        (JSC::FTL::LowerDFGToLLVM::compileMultiGetByOffset):
+        (JSC::FTL::LowerDFGToLLVM::compileNotifyWrite):
+        (JSC::FTL::LowerDFGToLLVM::compileBranch):
+        (JSC::FTL::LowerDFGToLLVM::compileSwitch):
+        (JSC::FTL::LowerDFGToLLVM::compareEqObjectOrOtherToObject):
+        (JSC::FTL::LowerDFGToLLVM::nonSpeculativeCompare):
+        (JSC::FTL::LowerDFGToLLVM::allocateCell):
+        (JSC::FTL::LowerDFGToLLVM::allocateBasicStorageAndGetEnd):
+        (JSC::FTL::LowerDFGToLLVM::boolify):
+        (JSC::FTL::LowerDFGToLLVM::equalNullOrUndefined):
+        (JSC::FTL::LowerDFGToLLVM::contiguousPutByValOutOfBounds):
+        (JSC::FTL::LowerDFGToLLVM::buildSwitch):
+        (JSC::FTL::LowerDFGToLLVM::doubleToInt32):
+        (JSC::FTL::LowerDFGToLLVM::sensibleDoubleToInt32):
+        (JSC::FTL::LowerDFGToLLVM::lowDouble):
+        (JSC::FTL::LowerDFGToLLVM::strictInt52ToJSValue):
+        (JSC::FTL::LowerDFGToLLVM::speculateObjectOrOther):
+        (JSC::FTL::LowerDFGToLLVM::speculateStringOrStringObject):
+        (JSC::FTL::LowerDFGToLLVM::emitStoreBarrier):
+        (JSC::FTL::LowerDFGToLLVM::callCheck):
+        (JSC::FTL::LowerDFGToLLVM::appendOSRExit):
+        * ftl/FTLOutput.cpp:
+        (JSC::FTL::Output::initialize):
+        (JSC::FTL::Output::appendTo):
+        (JSC::FTL::Output::newBlock):
+        (JSC::FTL::Output::sensibleDoubleToInt):
+        (JSC::FTL::Output::load):
+        (JSC::FTL::Output::store):
+        (JSC::FTL::Output::baseIndex):
+        (JSC::FTL::Output::branch):
+        (JSC::FTL::Output::crashNonTerminal):
+        * ftl/FTLOutput.h:
+        (JSC::FTL::Output::branch):
+        (JSC::FTL::Output::switchInstruction):
+        * ftl/FTLSwitchCase.h:
+        (JSC::FTL::SwitchCase::SwitchCase):
+        (JSC::FTL::SwitchCase::weight):
+        * ftl/FTLWeight.h: Added.
+        (JSC::FTL::Weight::Weight):
+        (JSC::FTL::Weight::isSet):
+        (JSC::FTL::Weight::operator!):
+        (JSC::FTL::Weight::value):
+        (JSC::FTL::Weight::scaleToTotal):
+        * ftl/FTLWeightedTarget.h: Added.
+        (JSC::FTL::WeightedTarget::WeightedTarget):
+        (JSC::FTL::WeightedTarget::target):
+        (JSC::FTL::WeightedTarget::weight):
+        (JSC::FTL::usually):
+        (JSC::FTL::rarely):
+        (JSC::FTL::unsure):
+
 2014-02-19  peavo@outlook.com  <peavo@outlook.com>
 
         [Win][LLINT] Incorrect stack alignment.
index ef46ee8..20f72de 100644 (file)
@@ -62,6 +62,8 @@
                0F0332C018ADFAE1005F979A /* ExitingJITType.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F0332BF18ADFAE1005F979A /* ExitingJITType.cpp */; };
                0F0332C318B01763005F979A /* GetByIdVariant.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F0332C118B01763005F979A /* GetByIdVariant.cpp */; };
                0F0332C418B01763005F979A /* GetByIdVariant.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F0332C218B01763005F979A /* GetByIdVariant.h */; settings = {ATTRIBUTES = (Private, ); }; };
+               0F0332C618B53FA9005F979A /* FTLWeight.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F0332C518B53FA9005F979A /* FTLWeight.h */; settings = {ATTRIBUTES = (Private, ); }; };
+               0F0332C818B546EC005F979A /* FTLWeightedTarget.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F0332C718B546EC005F979A /* FTLWeightedTarget.h */; settings = {ATTRIBUTES = (Private, ); }; };
                0F05C3B41683CF9200BAF45B /* DFGArrayifySlowPathGenerator.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F05C3B21683CF8F00BAF45B /* DFGArrayifySlowPathGenerator.h */; settings = {ATTRIBUTES = (Private, ); }; };
                0F0776BF14FF002B00102332 /* JITCompilationEffort.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F0776BD14FF002800102332 /* JITCompilationEffort.h */; settings = {ATTRIBUTES = (Private, ); }; };
                0F0B839C14BCF46300885B4F /* LLIntThunks.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F0B839714BCF45A00885B4F /* LLIntThunks.cpp */; };
                0F0332BF18ADFAE1005F979A /* ExitingJITType.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ExitingJITType.cpp; sourceTree = "<group>"; };
                0F0332C118B01763005F979A /* GetByIdVariant.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = GetByIdVariant.cpp; sourceTree = "<group>"; };
                0F0332C218B01763005F979A /* GetByIdVariant.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = GetByIdVariant.h; sourceTree = "<group>"; };
+               0F0332C518B53FA9005F979A /* FTLWeight.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = FTLWeight.h; path = ftl/FTLWeight.h; sourceTree = "<group>"; };
+               0F0332C718B546EC005F979A /* FTLWeightedTarget.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = FTLWeightedTarget.h; path = ftl/FTLWeightedTarget.h; sourceTree = "<group>"; };
                0F05C3B21683CF8F00BAF45B /* DFGArrayifySlowPathGenerator.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGArrayifySlowPathGenerator.h; path = dfg/DFGArrayifySlowPathGenerator.h; sourceTree = "<group>"; };
                0F0776BD14FF002800102332 /* JITCompilationEffort.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JITCompilationEffort.h; sourceTree = "<group>"; };
                0F0B839714BCF45A00885B4F /* LLIntThunks.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = LLIntThunks.cpp; path = llint/LLIntThunks.cpp; sourceTree = "<group>"; };
                0FEA09FC1705137F00BB722C /* ftl */ = {
                        isa = PBXGroup;
                        children = (
-                               2AC922B918A16182003CE0FB /* FTLDWARFDebugLineInfo.cpp */,
-                               2AC922BA18A16182003CE0FB /* FTLDWARFDebugLineInfo.h */,
                                0FDB2CC7173DA51E007B3C1B /* FTLAbbreviatedTypes.h */,
                                0FEA09FD170513DB00BB722C /* FTLAbbreviations.h */,
                                0FEA0A171708B00700BB722C /* FTLAbstractHeap.cpp */,
                                0FEA0A211709606900BB722C /* FTLCommonValues.h */,
                                0FEA0A00170513DB00BB722C /* FTLCompile.cpp */,
                                0FEA0A01170513DB00BB722C /* FTLCompile.h */,
+                               2AC922B918A16182003CE0FB /* FTLDWARFDebugLineInfo.cpp */,
+                               2AC922BA18A16182003CE0FB /* FTLDWARFDebugLineInfo.h */,
                                0F235BBD17178E1C00690C7F /* FTLExitArgument.cpp */,
                                0F235BBE17178E1C00690C7F /* FTLExitArgument.h */,
                                0F235BBF17178E1C00690C7F /* FTLExitArgumentForOperand.cpp */,
                                0FDB2CC8173DA51E007B3C1B /* FTLValueFromBlock.h */,
                                0F5A6281188C98D40072C9DF /* FTLValueRange.cpp */,
                                0F5A6282188C98D40072C9DF /* FTLValueRange.h */,
+                               0F0332C518B53FA9005F979A /* FTLWeight.h */,
+                               0F0332C718B546EC005F979A /* FTLWeightedTarget.h */,
                        );
                        name = ftl;
                        sourceTree = "<group>";
                                A7D89D0017A0B8CC00773AD8 /* DFGSSAConversionPhase.h in Headers */,
                                0F9FB4F517FCB91700CB67F8 /* DFGStackLayoutPhase.h in Headers */,
                                A593CF831840377100BFCE27 /* InspectorValues.h in Headers */,
+                               0F0332C818B546EC005F979A /* FTLWeightedTarget.h in Headers */,
                                0F63947815DCE34B006A597C /* DFGStructureAbstractValue.h in Headers */,
                                0FC097A2146B28CC00CF2442 /* DFGThunks.h in Headers */,
                                0FD8A32817D51F5700CA2C40 /* DFGTierUpCheckInjectionPhase.h in Headers */,
                                A7A8AF4217ADB5F3005AB174 /* Uint32Array.h in Headers */,
                                A5FD0086189B1B7E00633231 /* JSGlobalObjectConsoleAgent.h in Headers */,
                                A7A8AF3F17ADB5F3005AB174 /* Uint8Array.h in Headers */,
+                               0F0332C618B53FA9005F979A /* FTLWeight.h in Headers */,
                                A7A8AF4017ADB5F3005AB174 /* Uint8ClampedArray.h in Headers */,
                                0F5F08CF146C7633000472A9 /* UnconditionalFinalizer.h in Headers */,
                                0F666EC71835672B00D017F1 /* DFGAvailability.h in Headers */,
index 0f9fa4c..6185873 100644 (file)
@@ -31,7 +31,6 @@
 #if ENABLE(FTL_JIT)
 
 #include "FTLAbbreviatedTypes.h"
-#include "FTLSwitchCase.h"
 #include "FTLValueFromBlock.h"
 #include "LLVMAPI.h"
 #include <cstring>
@@ -120,6 +119,8 @@ static inline unsigned mdKindID(LContext context, const char* string) { return l
 static inline LValue mdString(LContext context, const char* string, unsigned length) { return llvm->MDStringInContext(context, string, length); }
 static inline LValue mdString(LContext context, const char* string) { return mdString(context, string, std::strlen(string)); }
 static inline LValue mdNode(LContext context, LValue* args, unsigned numArgs) { return llvm->MDNodeInContext(context, args, numArgs); }
+template<typename VectorType>
+static inline LValue mdNode(LContext context, const VectorType& vector) { return mdNode(context, const_cast<LValue*>(vector.begin()), vector.size()); }
 static inline LValue mdNode(LContext context) { return mdNode(context, 0, 0); }
 static inline LValue mdNode(LContext context, LValue arg1) { return mdNode(context, &arg1, 1); }
 static inline LValue mdNode(LContext context, LValue arg1, LValue arg2)
@@ -127,6 +128,11 @@ static inline LValue mdNode(LContext context, LValue arg1, LValue arg2)
     LValue args[] = { arg1, arg2 };
     return mdNode(context, args, 2);
 }
+static inline LValue mdNode(LContext context, LValue arg1, LValue arg2, LValue arg3)
+{
+    LValue args[] = { arg1, arg2, arg3 };
+    return mdNode(context, args, 3);
+}
 
 static inline void setMetadata(LValue instruction, unsigned kind, LValue metadata) { llvm->SetMetadata(instruction, kind, metadata); }
 
index cca90d3..f25bbe2 100644 (file)
@@ -62,6 +62,8 @@ CommonValues::CommonValues(LContext context)
     , intPtrPtr(constInt(intPtr, sizeof(void*), SignExtend))
     , doubleZero(constReal(doubleType, 0))
     , rangeKind(mdKindID(context, "range"))
+    , profKind(mdKindID(context, "prof"))
+    , branchWeights(mdString(context, "branch_weights"))
     , nonNegativeInt32(constInt(int32, 0, SignExtend), constInt(int32, 1ll << 31, SignExtend))
     , m_context(context)
     , m_module(0)
index 9f331cf..89d46ec 100644 (file)
@@ -76,9 +76,11 @@ public:
     const LValue doubleZero;
     
     const unsigned rangeKind;
+    const unsigned profKind;
+    const LValue branchWeights;
     
     const ValueRange nonNegativeInt32;
-    
+
     LContext const m_context;
     LModule m_module;
 };
index 9b69fe0..228e4c2 100644 (file)
@@ -39,6 +39,7 @@
 #include "FTLLoweredNodeValue.h"
 #include "FTLOutput.h"
 #include "FTLThunks.h"
+#include "FTLWeightedTarget.h"
 #include "LinkBuffer.h"
 #include "OperandsInlines.h"
 #include "JSCInlines.h"
@@ -137,7 +138,7 @@ public:
         m_out.storePtr(m_out.constIntPtr(codeBlock()), addressFor(JSStack::CodeBlock));
         m_out.branch(
             m_out.below(m_callFrame, m_out.loadPtr(m_out.absolute(vm().addressOfFTLStackLimit()))),
-            stackOverflow, lowBlock(m_graph.block(0)));
+            rarely(stackOverflow), usually(lowBlock(m_graph.block(0))));
         
         m_out.appendTo(stackOverflow, m_handleExceptions);
         vmCall(m_out.operation(operationThrowStackOverflowError), m_callFrame, m_out.constIntPtr(codeBlock()), NoExceptions);
@@ -626,7 +627,8 @@ private:
                 
                 Vector<ValueFromBlock> results;
                 
-                m_out.branch(isNotInt32(value.value()), notIntCase, intCase);
+                m_out.branch(
+                    isNotInt32(value.value()), unsure(notIntCase), unsure(intCase));
                 
                 LBasicBlock lastNext = m_out.appendTo(intCase, notIntCase);
                 results.append(m_out.anchor(unboxInt32(value.value())));
@@ -641,7 +643,9 @@ private:
                     m_out.jump(continuation);
                 } else {
                     m_out.appendTo(notIntCase, doubleCase);
-                    m_out.branch(isCellOrMisc(value.value()), notNumberCase, doubleCase);
+                    m_out.branch(
+                        isCellOrMisc(value.value()),
+                        unsure(notNumberCase), unsure(doubleCase));
                     
                     m_out.appendTo(doubleCase, notNumberCase);
                     results.append(m_out.anchor(doubleToInt32(unboxDouble(value.value()))));
@@ -711,7 +715,7 @@ private:
         LBasicBlock continuation = FTL_NEW_BLOCK(m_out, ("Store barrier continuation"));
 
         LValue base = lowJSValue(m_node->child1());
-        m_out.branch(m_out.isZero64(base), continuation, isNotNull);
+        m_out.branch(m_out.isZero64(base), unsure(continuation), unsure(isNotNull));
         LBasicBlock lastNext = m_out.appendTo(isNotNull, continuation);
         emitStoreBarrier(base);
         m_out.appendTo(continuation, lastNext);
@@ -909,11 +913,11 @@ private:
         LBasicBlock slowCase = FTL_NEW_BLOCK(m_out, ("ToThis slow case"));
         LBasicBlock continuation = FTL_NEW_BLOCK(m_out, ("ToThis continuation"));
         
-        m_out.branch(isCell(value), isCellCase, slowCase);
+        m_out.branch(isCell(value), usually(isCellCase), rarely(slowCase));
         
         LBasicBlock lastNext = m_out.appendTo(isCellCase, slowCase);
         ValueFromBlock fastResult = m_out.anchor(value);
-        m_out.branch(isType(value, FinalObjectType), continuation, slowCase);
+        m_out.branch(isType(value, FinalObjectType), usually(continuation), rarely(slowCase));
         
         m_out.appendTo(slowCase, continuation);
         J_JITOperation_EJ function;
@@ -1077,7 +1081,8 @@ private:
                 LBasicBlock slowCase = FTL_NEW_BLOCK(m_out, ("ArithMul slow case"));
                 LBasicBlock continuation = FTL_NEW_BLOCK(m_out, ("ArithMul continuation"));
                 
-                m_out.branch(m_out.notZero32(result), continuation, slowCase);
+                m_out.branch(
+                    m_out.notZero32(result), usually(continuation), rarely(slowCase));
                 
                 LBasicBlock lastNext = m_out.appendTo(slowCase, continuation);
                 LValue cond = m_out.bitOr(m_out.lessThan(left, m_out.int32Zero), m_out.lessThan(right, m_out.int32Zero));
@@ -1103,7 +1108,8 @@ private:
                 LBasicBlock slowCase = FTL_NEW_BLOCK(m_out, ("ArithMul slow case"));
                 LBasicBlock continuation = FTL_NEW_BLOCK(m_out, ("ArithMul continuation"));
                 
-                m_out.branch(m_out.notZero64(result), continuation, slowCase);
+                m_out.branch(
+                    m_out.notZero64(result), usually(continuation), rarely(slowCase));
                 
                 LBasicBlock lastNext = m_out.appendTo(slowCase, continuation);
                 LValue cond = m_out.bitOr(m_out.lessThan(left, m_out.int64Zero), m_out.lessThan(right, m_out.int64Zero));
@@ -1143,7 +1149,9 @@ private:
             
             LValue adjustedDenominator = m_out.add(denominator, m_out.int32One);
             
-            m_out.branch(m_out.above(adjustedDenominator, m_out.int32One), continuation, unsafeDenominator);
+            m_out.branch(
+                m_out.above(adjustedDenominator, m_out.int32One),
+                usually(continuation), rarely(unsafeDenominator));
             
             LBasicBlock lastNext = m_out.appendTo(unsafeDenominator, continuation);
             
@@ -1163,14 +1171,17 @@ private:
                 LBasicBlock notDivByZero = FTL_NEW_BLOCK(m_out, ("ArithDiv not divide by zero"));
                 LBasicBlock neg2ToThe31ByNeg1 = FTL_NEW_BLOCK(m_out, ("ArithDiv -2^31/-1"));
                 
-                m_out.branch(m_out.isZero32(denominator), divByZero, notDivByZero);
+                m_out.branch(
+                    m_out.isZero32(denominator), rarely(divByZero), usually(notDivByZero));
                 
                 m_out.appendTo(divByZero, notDivByZero);
                 results.append(m_out.anchor(m_out.int32Zero));
                 m_out.jump(done);
                 
                 m_out.appendTo(notDivByZero, neg2ToThe31ByNeg1);
-                m_out.branch(m_out.equal(numerator, neg2ToThe31), neg2ToThe31ByNeg1, continuation);
+                m_out.branch(
+                    m_out.equal(numerator, neg2ToThe31),
+                    rarely(neg2ToThe31ByNeg1), usually(continuation));
                 
                 m_out.appendTo(neg2ToThe31ByNeg1, continuation);
                 results.append(m_out.anchor(neg2ToThe31));
@@ -1183,7 +1194,9 @@ private:
                 LBasicBlock zeroNumerator = FTL_NEW_BLOCK(m_out, ("ArithDiv zero numerator"));
                 LBasicBlock numeratorContinuation = FTL_NEW_BLOCK(m_out, ("ArithDiv numerator continuation"));
                 
-                m_out.branch(m_out.isZero32(numerator), zeroNumerator, numeratorContinuation);
+                m_out.branch(
+                    m_out.isZero32(numerator),
+                    rarely(zeroNumerator), usually(numeratorContinuation));
                 
                 LBasicBlock innerLastNext = m_out.appendTo(zeroNumerator, numeratorContinuation);
                 
@@ -1239,7 +1252,9 @@ private:
             
             LValue adjustedDenominator = m_out.add(denominator, m_out.int32One);
             
-            m_out.branch(m_out.above(adjustedDenominator, m_out.int32One), continuation, unsafeDenominator);
+            m_out.branch(
+                m_out.above(adjustedDenominator, m_out.int32One),
+                usually(continuation), rarely(unsafeDenominator));
             
             LBasicBlock lastNext = m_out.appendTo(unsafeDenominator, continuation);
             
@@ -1261,14 +1276,17 @@ private:
                 LBasicBlock notModByZero = FTL_NEW_BLOCK(m_out, ("ArithMod not modulo by zero"));
                 LBasicBlock neg2ToThe31ByNeg1 = FTL_NEW_BLOCK(m_out, ("ArithMod -2^31/-1"));
                 
-                m_out.branch(m_out.isZero32(denominator), modByZero, notModByZero);
+                m_out.branch(
+                    m_out.isZero32(denominator), rarely(modByZero), usually(notModByZero));
                 
                 m_out.appendTo(modByZero, notModByZero);
                 results.append(m_out.anchor(m_out.int32Zero));
                 m_out.jump(done);
                 
                 m_out.appendTo(notModByZero, neg2ToThe31ByNeg1);
-                m_out.branch(m_out.equal(numerator, neg2ToThe31), neg2ToThe31ByNeg1, continuation);
+                m_out.branch(
+                    m_out.equal(numerator, neg2ToThe31),
+                    rarely(neg2ToThe31ByNeg1), usually(continuation));
                 
                 m_out.appendTo(neg2ToThe31ByNeg1, continuation);
                 results.append(m_out.anchor(m_out.int32Zero));
@@ -1285,7 +1303,7 @@ private:
                 
                 m_out.branch(
                     m_out.lessThan(numerator, m_out.int32Zero),
-                    negativeNumerator, numeratorContinuation);
+                    unsure(negativeNumerator), unsure(numeratorContinuation));
                 
                 LBasicBlock innerLastNext = m_out.appendTo(negativeNumerator, numeratorContinuation);
                 
@@ -1347,7 +1365,7 @@ private:
                 m_node->op() == ArithMin
                     ? m_out.doubleLessThan(left, right)
                     : m_out.doubleGreaterThan(left, right),
-                continuation, notLessThan);
+                unsure(continuation), unsure(notLessThan));
             
             LBasicBlock lastNext = m_out.appendTo(notLessThan, continuation);
             results.append(m_out.anchor(m_out.select(
@@ -1535,7 +1553,7 @@ private:
             LBasicBlock nextStructure = FTL_NEW_BLOCK(m_out, ("CheckStructure nextStructure"));
             m_out.branch(
                 m_out.equal(structure, weakPointer(m_node->structureSet()[i])),
-                continuation, nextStructure);
+                unsure(continuation), unsure(nextStructure));
             m_out.appendTo(nextStructure);
         }
         
@@ -1585,7 +1603,7 @@ private:
         
         m_out.branch(
             m_out.notEqual(structure, weakPointer(m_node->structure())),
-            unexpectedStructure, continuation);
+            rarely(unexpectedStructure), usually(continuation));
         
         LBasicBlock lastNext = m_out.appendTo(unexpectedStructure, continuation);
         
@@ -1669,7 +1687,7 @@ private:
             LBasicBlock notCellCase = FTL_NEW_BLOCK(m_out, ("GetById untyped not cell case"));
             LBasicBlock continuation = FTL_NEW_BLOCK(m_out, ("GetById untyped continuation"));
             
-            m_out.branch(isCell(value), cellCase, notCellCase);
+            m_out.branch(isCell(value), unsure(cellCase), unsure(notCellCase));
             
             LBasicBlock lastNext = m_out.appendTo(cellCase, notCellCase);
             ValueFromBlock cellResult = m_out.anchor(getById(value));
@@ -1741,7 +1759,8 @@ private:
             ValueFromBlock fastResult = m_out.anchor(
                 m_out.loadPtr(cell, m_heaps.JSString_value));
             
-            m_out.branch(m_out.notNull(fastResult.value()), continuation, slowPath);
+            m_out.branch(
+                m_out.notNull(fastResult.value()), usually(continuation), rarely(slowPath));
             
             LBasicBlock lastNext = m_out.appendTo(slowPath, continuation);
             
@@ -1781,7 +1800,9 @@ private:
         LBasicBlock continuation = FTL_NEW_BLOCK(m_out, ("continuation branch"));
         
         LValue baseAddress = m_out.addPtr(basePtr, JSArrayBufferView::offsetOfMode());
-        m_out.branch(m_out.notEqual(baseAddress , m_out.constIntPtr(WastefulTypedArray)), simpleCase, wastefulCase);
+        m_out.branch(
+            m_out.notEqual(baseAddress , m_out.constIntPtr(WastefulTypedArray)),
+            unsure(simpleCase), unsure(wastefulCase));
 
         // begin simple case        
         LBasicBlock lastNext = m_out.appendTo(simpleCase, wastefulCase);
@@ -1875,13 +1896,14 @@ private:
             m_out.branch(
                 m_out.aboveOrEqual(
                     index, m_out.load32NonNegative(storage, m_heaps.Butterfly_publicLength)),
-                slowCase, fastCase);
+                rarely(slowCase), usually(fastCase));
             
             LBasicBlock lastNext = m_out.appendTo(fastCase, slowCase);
             
             ValueFromBlock fastResult = m_out.anchor(
                 m_out.load64(baseIndex(heap, storage, index, m_node->child2())));
-            m_out.branch(m_out.isZero64(fastResult.value()), slowCase, continuation);
+            m_out.branch(
+                m_out.isZero64(fastResult.value()), rarely(slowCase), usually(continuation));
             
             m_out.appendTo(slowCase, continuation);
             ValueFromBlock slowResult = m_out.anchor(
@@ -1922,13 +1944,14 @@ private:
             m_out.branch(
                 m_out.aboveOrEqual(
                     index, m_out.load32NonNegative(storage, m_heaps.Butterfly_publicLength)),
-                slowCase, inBounds);
+                rarely(slowCase), usually(inBounds));
             
             LBasicBlock lastNext = m_out.appendTo(inBounds, boxPath);
             LValue doubleValue = m_out.loadDouble(
                 baseIndex(heap, storage, index, m_node->child2()));
             m_out.branch(
-                m_out.doubleNotEqualOrUnordered(doubleValue, doubleValue), slowCase, boxPath);
+                m_out.doubleNotEqualOrUnordered(doubleValue, doubleValue),
+                rarely(slowCase), usually(boxPath));
             
             m_out.appendTo(boxPath, slowCase);
             ValueFromBlock fastResult = m_out.anchor(boxDouble(doubleValue));
@@ -2181,7 +2204,7 @@ private:
                             intValues.append(m_out.anchor(m_out.int32Zero));
                             m_out.branch(
                                 m_out.lessThan(intValue, m_out.int32Zero),
-                                continuation, atLeastZero);
+                                unsure(continuation), unsure(atLeastZero));
                             
                             LBasicBlock lastNext = m_out.appendTo(atLeastZero, continuation);
                             
@@ -2211,13 +2234,13 @@ private:
                             intValues.append(m_out.anchor(m_out.int32Zero));
                             m_out.branch(
                                 m_out.doubleLessThanOrUnordered(doubleValue, m_out.doubleZero),
-                                continuation, atLeastZero);
+                                unsure(continuation), unsure(atLeastZero));
                             
                             LBasicBlock lastNext = m_out.appendTo(atLeastZero, withinRange);
                             intValues.append(m_out.anchor(m_out.constInt32(255)));
                             m_out.branch(
                                 m_out.doubleGreaterThan(doubleValue, m_out.constDouble(255)),
-                                continuation, withinRange);
+                                unsure(continuation), unsure(withinRange));
                             
                             m_out.appendTo(withinRange, continuation);
                             intValues.append(m_out.anchor(m_out.fpToInt32(doubleValue)));
@@ -2274,7 +2297,7 @@ private:
                     
                     m_out.branch(
                         m_out.aboveOrEqual(index, lowInt32(child5)),
-                        continuation, isInBounds);
+                        unsure(continuation), unsure(isInBounds));
                     
                     LBasicBlock lastNext = m_out.appendTo(isInBounds, continuation);
                     m_out.store(valueToStore, pointer, refType);
@@ -2329,7 +2352,7 @@ private:
             m_out.branch(
                 m_out.aboveOrEqual(
                     prevLength, m_out.load32(storage, m_heaps.Butterfly_vectorLength)),
-                slowPath, fastPath);
+                rarely(slowPath), usually(fastPath));
             
             LBasicBlock lastNext = m_out.appendTo(fastPath, slowPath);
             m_out.store(
@@ -2382,7 +2405,8 @@ private:
             
             Vector<ValueFromBlock, 3> results;
             results.append(m_out.anchor(m_out.constInt64(JSValue::encode(jsUndefined()))));
-            m_out.branch(m_out.isZero32(prevLength), continuation, fastCase);
+            m_out.branch(
+                m_out.isZero32(prevLength), rarely(continuation), usually(fastCase));
             
             LBasicBlock lastNext = m_out.appendTo(fastCase, slowCase);
             LValue newLength = m_out.sub(prevLength, m_out.int32One);
@@ -2393,12 +2417,15 @@ private:
                 LValue result = m_out.load64(pointer);
                 m_out.store64(m_out.int64Zero, pointer);
                 results.append(m_out.anchor(result));
-                m_out.branch(m_out.notZero64(result), continuation, slowCase);
+                m_out.branch(
+                    m_out.notZero64(result), usually(continuation), rarely(slowCase));
             } else {
                 LValue result = m_out.loadDouble(pointer);
                 m_out.store64(m_out.constInt64(bitwise_cast<int64_t>(QNaN)), pointer);
                 results.append(m_out.anchor(boxDouble(result)));
-                m_out.branch(m_out.doubleEqual(result, result), continuation, slowCase);
+                m_out.branch(
+                    m_out.doubleEqual(result, result),
+                    usually(continuation), rarely(slowCase));
             }
             
             m_out.appendTo(slowCase, continuation);
@@ -2588,7 +2615,7 @@ private:
             
             m_out.branch(
                 m_out.aboveOrEqual(publicLength, m_out.constInt32(MIN_SPARSE_ARRAY_INDEX)),
-                largeCase, fastCase);
+                rarely(largeCase), usually(fastCase));
 
             LBasicBlock lastNext = m_out.appendTo(fastCase, largeCase);
             
@@ -2617,7 +2644,8 @@ private:
                 
                 ValueFromBlock originalIndex = m_out.anchor(vectorLength);
                 ValueFromBlock originalPointer = m_out.anchor(butterfly);
-                m_out.branch(m_out.notZero32(vectorLength), initLoop, initDone);
+                m_out.branch(
+                    m_out.notZero32(vectorLength), unsure(initLoop), unsure(initDone));
                 
                 LBasicBlock initLastNext = m_out.appendTo(initLoop, initDone);
                 LValue index = m_out.phi(m_out.int32, originalIndex);
@@ -2630,7 +2658,8 @@ private:
                 LValue nextIndex = m_out.sub(index, m_out.int32One);
                 addIncoming(index, m_out.anchor(nextIndex));
                 addIncoming(pointer, m_out.anchor(m_out.add(pointer, m_out.intPtrEight)));
-                m_out.branch(m_out.notZero32(nextIndex), initLoop, initDone);
+                m_out.branch(
+                    m_out.notZero32(nextIndex), unsure(initLoop), unsure(initDone));
                 
                 m_out.appendTo(initDone, initLastNext);
             }
@@ -2792,7 +2821,7 @@ private:
             ValueFromBlock simpleResult = m_out.anchor(cell);
             m_out.branch(
                 m_out.equal(structure, m_out.constIntPtr(vm().stringStructure.get())),
-                continuation, notString);
+                unsure(continuation), unsure(notString));
             
             LBasicBlock lastNext = m_out.appendTo(notString, continuation);
             speculateStringObjectForStructure(m_node->child1(), structure);
@@ -2824,7 +2853,7 @@ private:
                 isCellPredicate = m_out.booleanTrue;
             else
                 isCellPredicate = this->isCell(value);
-            m_out.branch(isCellPredicate, isCell, notString);
+            m_out.branch(isCellPredicate, unsure(isCell), unsure(notString));
             
             LBasicBlock lastNext = m_out.appendTo(isCell, notString);
             ValueFromBlock simpleResult = m_out.anchor(value);
@@ -2835,7 +2864,7 @@ private:
                     m_out.constIntPtr(vm().stringStructure.get()));
             } else
                 isStringPredicate = m_out.booleanFalse;
-            m_out.branch(isStringPredicate, continuation, notString);
+            m_out.branch(isStringPredicate, unsure(continuation), unsure(notString));
             
             m_out.appendTo(notString, continuation);
             LValue operation;
@@ -2868,11 +2897,11 @@ private:
         Vector<ValueFromBlock, 3> results;
         
         results.append(m_out.anchor(value));
-        m_out.branch(isCell(value), isCellCase, continuation);
+        m_out.branch(isCell(value), unsure(isCellCase), unsure(continuation));
         
         LBasicBlock lastNext = m_out.appendTo(isCellCase, isObjectCase);
         results.append(m_out.anchor(value));
-        m_out.branch(isObject(value), isObjectCase, continuation);
+        m_out.branch(isObject(value), unsure(isObjectCase), unsure(continuation));
         
         m_out.appendTo(isObjectCase, continuation);
         results.append(m_out.anchor(vmCall(
@@ -2963,7 +2992,7 @@ private:
         m_out.branch(
             m_out.aboveOrEqual(
                 index, m_out.load32NonNegative(base, m_heaps.JSString_length)),
-            slowPath, fastPath);
+            rarely(slowPath), usually(fastPath));
             
         LBasicBlock lastNext = m_out.appendTo(fastPath, slowPath);
             
@@ -2978,7 +3007,7 @@ private:
             m_out.testIsZero32(
                 m_out.load32(stringImpl, m_heaps.StringImpl_hashAndFlags),
                 m_out.constInt32(StringImpl::flagIs8Bit())),
-            is16Bit, is8Bit);
+            unsure(is16Bit), unsure(is8Bit));
             
         m_out.appendTo(is8Bit, is16Bit);
             
@@ -2998,7 +3027,9 @@ private:
                 storage, m_out.zeroExt(index, m_out.intPtr),
                 m_state.forNode(m_node->child2()).m_value)),
             m_out.int32));
-        m_out.branch(m_out.aboveOrEqual(char16Bit.value(), m_out.constInt32(0x100)), bigCharacter, bitsContinuation);
+        m_out.branch(
+            m_out.aboveOrEqual(char16Bit.value(), m_out.constInt32(0x100)),
+            rarely(bigCharacter), usually(bitsContinuation));
             
         m_out.appendTo(bigCharacter, bitsContinuation);
             
@@ -3031,7 +3062,9 @@ private:
                 LBasicBlock negativeIndex = FTL_NEW_BLOCK(m_out, ("GetByVal String negative index"));
                     
                 results.append(m_out.anchor(m_out.constInt64(JSValue::encode(jsUndefined()))));
-                m_out.branch(m_out.lessThan(index, m_out.int32Zero), negativeIndex, continuation);
+                m_out.branch(
+                    m_out.lessThan(index, m_out.int32Zero),
+                    rarely(negativeIndex), usually(continuation));
                     
                 m_out.appendTo(negativeIndex, continuation);
             }
@@ -3067,7 +3100,7 @@ private:
             m_out.testIsZero32(
                 m_out.load32(stringImpl, m_heaps.StringImpl_hashAndFlags),
                 m_out.constInt32(StringImpl::flagIs8Bit())),
-            is16Bit, is8Bit);
+            unsure(is16Bit), unsure(is8Bit));
             
         LBasicBlock lastNext = m_out.appendTo(is8Bit, is16Bit);
             
@@ -3122,10 +3155,13 @@ private:
         Vector<SwitchCase, 2> cases;
         for (unsigned i = data.variants.size(); i--;) {
             GetByIdVariant variant = data.variants[i];
-            for (unsigned j = variant.structureSet().size(); j--;)
-                cases.append(SwitchCase(weakPointer(variant.structureSet()[j]), blocks[i]));
+            for (unsigned j = variant.structureSet().size(); j--;) {
+                cases.append(SwitchCase(
+                    weakPointer(variant.structureSet()[j]), blocks[i], Weight(1)));
+            }
         }
-        m_out.switchInstruction(m_out.loadPtr(base, m_heaps.JSCell_structure), cases, exit);
+        m_out.switchInstruction(
+            m_out.loadPtr(base, m_heaps.JSCell_structure), cases, exit, Weight(0));
         
         LBasicBlock lastNext = m_out.m_nextBlock;
         
@@ -3206,7 +3242,7 @@ private:
         
         m_out.branch(
             m_out.equal(state, m_out.constInt8(IsInvalidated)),
-            continuation, isNotInvalidated);
+            usually(continuation), rarely(isNotInvalidated));
         
         LBasicBlock lastNext = m_out.appendTo(isNotInvalidated, isClear);
 
@@ -3215,7 +3251,7 @@ private:
             isClearValue = m_out.equal(state, m_out.constInt8(ClearWatchpoint));
         else
             isClearValue = m_out.booleanFalse;
-        m_out.branch(isClearValue, isClear, isWatched);
+        m_out.branch(isClearValue, unsure(isClear), unsure(isWatched));
         
         m_out.appendTo(isClear, isWatched);
         
@@ -3227,13 +3263,13 @@ private:
         
         m_out.branch(
             m_out.equal(value, m_out.load64(m_out.absolute(set->addressOfInferredValue()))),
-            continuation, invalidate);
+            unsure(continuation), unsure(invalidate));
         
         m_out.appendTo(invalidate, invalidateFast);
         
         m_out.branch(
             m_out.notZero8(m_out.load8(m_out.absolute(set->addressOfSetIsNotEmpty()))),
-            invalidateSlow, invalidateFast);
+            rarely(invalidateSlow), usually(invalidateFast));
         
         m_out.appendTo(invalidateFast, invalidateSlow);
         
@@ -3456,14 +3492,20 @@ private:
     
     void compileBranch()
     {
+        // FIXME: DFG should be able to tell us branch weights here.
+        // https://bugs.webkit.org/show_bug.cgi?id=129055
+        
         m_out.branch(
             boolify(m_node->child1()),
-            lowBlock(m_node->takenBlock()),
-            lowBlock(m_node->notTakenBlock()));
+            unsure(lowBlock(m_node->takenBlock())),
+            unsure(lowBlock(m_node->notTakenBlock())));
     }
     
     void compileSwitch()
     {
+        // FIXME: DFG should be able to tell us branch weights here.
+        // https://bugs.webkit.org/show_bug.cgi?id=129055
+        
         SwitchData* data = m_node->switchData();
         switch (data->kind) {
         case SwitchImm: {
@@ -3485,7 +3527,7 @@ private:
                 LBasicBlock isDouble = FTL_NEW_BLOCK(m_out, ("Switch/SwitchImm is double"));
                 
                 LValue boxedValue = lowJSValue(m_node->child1());
-                m_out.branch(isNotInt32(boxedValue), isNotInt, isInt);
+                m_out.branch(isNotInt32(boxedValue), unsure(isNotInt), unsure(isInt));
                 
                 LBasicBlock innerLastNext = m_out.appendTo(isInt, isNotInt);
                 
@@ -3494,7 +3536,8 @@ private:
                 
                 m_out.appendTo(isNotInt, isDouble);
                 m_out.branch(
-                    isCellOrMisc(boxedValue), lowBlock(data->fallThrough), isDouble);
+                    isCellOrMisc(boxedValue),
+                    usually(lowBlock(data->fallThrough)), rarely(isDouble));
                 
                 m_out.appendTo(isDouble, innerLastNext);
                 LValue doubleValue = unboxDouble(boxedValue);
@@ -3502,7 +3545,7 @@ private:
                 intValues.append(m_out.anchor(intInDouble));
                 m_out.branch(
                     m_out.doubleEqual(m_out.intToDouble(intInDouble), doubleValue),
-                    switchOnInts, lowBlock(data->fallThrough));
+                    unsure(switchOnInts), unsure(lowBlock(data->fallThrough)));
                 break;
             }
                 
@@ -3532,11 +3575,14 @@ private:
                 LBasicBlock isStringCase = FTL_NEW_BLOCK(m_out, ("Switch/SwitchChar is string"));
                 
                 m_out.branch(
-                    isNotCell(unboxedValue), lowBlock(data->fallThrough), isCellCase);
+                    isNotCell(unboxedValue),
+                    unsure(lowBlock(data->fallThrough)), unsure(isCellCase));
                 
                 LBasicBlock lastNext = m_out.appendTo(isCellCase, isStringCase);
                 LValue cellValue = unboxedValue;
-                m_out.branch(isNotString(cellValue), lowBlock(data->fallThrough), isStringCase);
+                m_out.branch(
+                    isNotString(cellValue),
+                    unsure(lowBlock(data->fallThrough)), unsure(isStringCase));
                 
                 m_out.appendTo(isStringCase, lastNext);
                 stringValue = cellValue;
@@ -3559,13 +3605,13 @@ private:
                 m_out.notEqual(
                     m_out.load32NonNegative(stringValue, m_heaps.JSString_length),
                     m_out.int32One),
-                lowBlock(data->fallThrough), lengthIs1);
+                unsure(lowBlock(data->fallThrough)), unsure(lengthIs1));
             
             LBasicBlock lastNext = m_out.appendTo(lengthIs1, needResolution);
             Vector<ValueFromBlock, 2> values;
             LValue fastValue = m_out.loadPtr(stringValue, m_heaps.JSString_value);
             values.append(m_out.anchor(fastValue));
-            m_out.branch(m_out.isNull(fastValue), needResolution, resolved);
+            m_out.branch(m_out.isNull(fastValue), rarely(needResolution), usually(resolved));
             
             m_out.appendTo(needResolution, resolved);
             values.append(m_out.anchor(
@@ -3579,7 +3625,7 @@ private:
                 m_out.testNonZero32(
                     m_out.load32(value, m_heaps.StringImpl_hashAndFlags),
                     m_out.constInt32(StringImpl::flagIs8Bit())),
-                is8Bit, is16Bit);
+                unsure(is8Bit), unsure(is16Bit));
             
             Vector<ValueFromBlock, 2> characters;
             m_out.appendTo(is8Bit, is16Bit);
@@ -3727,7 +3773,7 @@ private:
         LBasicBlock leftNotCellCase = FTL_NEW_BLOCK(m_out, ("CompareEqObjectOrOtherToObject left not cell case"));
         LBasicBlock continuation = FTL_NEW_BLOCK(m_out, ("CompareEqObjectOrOtherToObject continuation"));
         
-        m_out.branch(isCell(leftValue), leftCellCase, leftNotCellCase);
+        m_out.branch(isCell(leftValue), unsure(leftCellCase), unsure(leftNotCellCase));
         
         LBasicBlock lastNext = m_out.appendTo(leftCellCase, leftNotCellCase);
         speculateTruthyObject(leftChild, leftValue, SpecObject | (~SpecCell));
@@ -3772,10 +3818,10 @@ private:
         LBasicBlock slowPath = FTL_NEW_BLOCK(m_out, ("CompareEq untyped slow path"));
         LBasicBlock continuation = FTL_NEW_BLOCK(m_out, ("CompareEq untyped continuation"));
         
-        m_out.branch(isNotInt32(left), slowPath, leftIsInt);
+        m_out.branch(isNotInt32(left), rarely(slowPath), usually(leftIsInt));
         
         LBasicBlock lastNext = m_out.appendTo(leftIsInt, fastPath);
-        m_out.branch(isNotInt32(right), slowPath, fastPath);
+        m_out.branch(isNotInt32(right), rarely(slowPath), usually(fastPath));
         
         m_out.appendTo(fastPath, slowPath);
         ValueFromBlock fastResult = m_out.anchor(
@@ -3798,7 +3844,7 @@ private:
         LValue result = m_out.loadPtr(
             allocator, m_heaps.MarkedAllocator_freeListHead);
         
-        m_out.branch(m_out.notNull(result), success, slowPath);
+        m_out.branch(m_out.notNull(result), usually(success), rarely(slowPath));
         
         m_out.appendTo(success);
         
@@ -3843,7 +3889,9 @@ private:
         LValue remaining = m_out.loadPtr(m_out.absolute(&allocator.m_currentRemaining));
         LValue newRemaining = m_out.sub(remaining, size);
         
-        m_out.branch(m_out.lessThan(newRemaining, m_out.intPtrZero), slowPath, success);
+        m_out.branch(
+            m_out.lessThan(newRemaining, m_out.intPtrZero),
+            rarely(slowPath), usually(success));
         
         m_out.appendTo(success);
         
@@ -3971,7 +4019,7 @@ private:
             LBasicBlock fastCase = FTL_NEW_BLOCK(m_out, ("Boolify untyped fast case"));
             LBasicBlock continuation = FTL_NEW_BLOCK(m_out, ("Boolify untyped continuation"));
             
-            m_out.branch(isNotBoolean(value), slowCase, fastCase);
+            m_out.branch(isNotBoolean(value), rarely(slowCase), usually(fastCase));
             
             LBasicBlock lastNext = m_out.appendTo(fastCase, slowCase);
             ValueFromBlock fastResult = m_out.anchor(unboxBoolean(value));
@@ -4013,7 +4061,7 @@ private:
         LBasicBlock primitiveCase = FTL_NEW_BLOCK(m_out, ("EqualNullOrUndefined primitive case"));
         LBasicBlock continuation = FTL_NEW_BLOCK(m_out, ("EqualNullOrUndefined continuation"));
         
-        m_out.branch(isNotCell(value), primitiveCase, cellCase);
+        m_out.branch(isNotCell(value), unsure(primitiveCase), unsure(cellCase));
         
         LBasicBlock lastNext = m_out.appendTo(cellCase, primitiveCase);
         
@@ -4046,7 +4094,7 @@ private:
                 m_out.testNonZero8(
                     m_out.load8(structure, m_heaps.Structure_typeInfoFlags),
                     m_out.constInt8(MasqueradesAsUndefined)),
-                masqueradesCase, continuation);
+                rarely(masqueradesCase), usually(continuation));
             
             m_out.appendTo(masqueradesCase, primitiveCase);
             
@@ -4097,7 +4145,7 @@ private:
             LBasicBlock performStore =
                 FTL_NEW_BLOCK(m_out, ("PutByVal perform store"));
                 
-            m_out.branch(isNotInBounds, notInBoundsCase, performStore);
+            m_out.branch(isNotInBounds, unsure(notInBoundsCase), unsure(performStore));
                 
             LBasicBlock lastNext = m_out.appendTo(notInBoundsCase, performStore);
                 
@@ -4112,7 +4160,7 @@ private:
                 LBasicBlock holeCase =
                     FTL_NEW_BLOCK(m_out, ("PutByVal hole case"));
                     
-                m_out.branch(isOutOfBounds, outOfBoundsCase, holeCase);
+                m_out.branch(isOutOfBounds, unsure(outOfBoundsCase), unsure(holeCase));
                     
                 LBasicBlock innerLastNext = m_out.appendTo(outOfBoundsCase, holeCase);
                     
@@ -4140,10 +4188,10 @@ private:
         for (unsigned i = 0; i < data->cases.size(); ++i) {
             cases.append(SwitchCase(
                 constInt(type, data->cases[i].value.switchLookupValue()),
-                lowBlock(data->cases[i].target)));
+                lowBlock(data->cases[i].target), Weight()));
         }
         
-        m_out.switchInstruction(switchValue, cases, lowBlock(data->fallThrough));
+        m_out.switchInstruction(switchValue, cases, lowBlock(data->fallThrough), Weight());
     }
     
     LValue doubleToInt32(LValue doubleValue, double low, double high, bool isSigned = true)
@@ -4160,12 +4208,12 @@ private:
         
         m_out.branch(
             m_out.doubleGreaterThanOrEqual(doubleValue, m_out.constDouble(low)),
-            greatEnough, slowPath);
+            unsure(greatEnough), unsure(slowPath));
         
         LBasicBlock lastNext = m_out.appendTo(greatEnough, withinRange);
         m_out.branch(
             m_out.doubleLessThanOrEqual(doubleValue, m_out.constDouble(high)),
-            withinRange, slowPath);
+            unsure(withinRange), unsure(slowPath));
         
         m_out.appendTo(withinRange, slowPath);
         LValue fastResult;
@@ -4202,7 +4250,7 @@ private:
             m_out.sensibleDoubleToInt(doubleValue));
         m_out.branch(
             m_out.equal(fastResult.value(), m_out.constInt32(0x80000000)),
-            slowPath, continuation);
+            rarely(slowPath), usually(continuation));
         
         LBasicBlock lastNext = m_out.appendTo(slowPath, continuation);
         ValueFromBlock slowResult = m_out.anchor(
@@ -4510,7 +4558,7 @@ private:
             LBasicBlock doubleCase = FTL_NEW_BLOCK(m_out, ("Double unboxing double case"));
             LBasicBlock continuation = FTL_NEW_BLOCK(m_out, ("Double unboxing continuation"));
             
-            m_out.branch(isNotInt32(boxedResult), doubleCase, intCase);
+            m_out.branch(isNotInt32(boxedResult), unsure(doubleCase), unsure(intCase));
             
             LBasicBlock lastNext = m_out.appendTo(intCase, doubleCase);
             
@@ -4622,7 +4670,7 @@ private:
         LValue int32Value = m_out.castToInt32(value);
         m_out.branch(
             m_out.equal(m_out.signExt(int32Value, m_out.int64), value),
-            isInt32, isDouble);
+            unsure(isInt32), unsure(isDouble));
         
         LBasicBlock lastNext = m_out.appendTo(isInt32, isDouble);
         
@@ -4913,7 +4961,7 @@ private:
         LBasicBlock primitiveCase = FTL_NEW_BLOCK(m_out, ("speculateObjectOrOther primitive case"));
         LBasicBlock continuation = FTL_NEW_BLOCK(m_out, ("speculateObjectOrOther continuation"));
         
-        m_out.branch(isNotCell(value), primitiveCase, cellCase);
+        m_out.branch(isNotCell(value), unsure(primitiveCase), unsure(cellCase));
         
         LBasicBlock lastNext = m_out.appendTo(cellCase, primitiveCase);
         
@@ -4973,7 +5021,7 @@ private:
         LValue structure = m_out.loadPtr(lowCell(edge), m_heaps.JSCell_structure);
         m_out.branch(
             m_out.equal(structure, m_out.constIntPtr(vm().stringStructure.get())),
-            continuation, notString);
+            unsure(continuation), unsure(notString));
         
         LBasicBlock lastNext = m_out.appendTo(notString, continuation);
         speculateStringObjectForStructure(edge, structure);
@@ -5082,7 +5130,7 @@ private:
         LBasicBlock isCell = FTL_NEW_BLOCK(m_out, ("Store barrier is cell block"));
 
         if (m_state.forNode(valueEdge.node()).couldBeType(SpecCell))
-            m_out.branch(isNotCell(value), continuation, isCell);
+            m_out.branch(isNotCell(value), unsure(continuation), unsure(isCell));
         else
             m_out.jump(isCell);
 
@@ -5107,13 +5155,16 @@ private:
         LBasicBlock bufferIsFull = FTL_NEW_BLOCK(m_out, ("Store barrier buffer is full"));
 
         // Check the mark byte. 
-        m_out.branch(m_out.isZero8(loadMarkByte(base)), continuation, isMarked);
+        m_out.branch(
+            m_out.isZero8(loadMarkByte(base)), usually(continuation), rarely(isMarked));
 
         // Append to the write barrier buffer.
         LBasicBlock lastNext = m_out.appendTo(isMarked, bufferHasSpace);
         LValue currentBufferIndex = m_out.load32(m_out.absolute(&vm().heap.writeBarrierBuffer().m_currentIndex));
         LValue bufferCapacity = m_out.load32(m_out.absolute(&vm().heap.writeBarrierBuffer().m_capacity));
-        m_out.branch(m_out.lessThan(currentBufferIndex, bufferCapacity), bufferHasSpace, bufferIsFull);
+        m_out.branch(
+            m_out.lessThan(currentBufferIndex, bufferCapacity),
+            usually(bufferHasSpace), rarely(bufferIsFull));
 
         // Buffer has space, store to it.
         m_out.appendTo(bufferHasSpace, bufferIsFull);
@@ -5193,7 +5244,7 @@ private:
         
         m_out.branch(
             m_out.notZero64(m_out.load64(m_out.absolute(vm().addressOfException()))),
-            m_handleExceptions, continuation);
+            rarely(m_handleExceptions), usually(continuation));
         
         m_out.appendTo(continuation);
     }
@@ -5233,7 +5284,7 @@ private:
         LBasicBlock failCase = FTL_NEW_BLOCK(m_out, ("OSR exit failCase for ", m_node));
         continuation = FTL_NEW_BLOCK(m_out, ("OSR exit continuation for ", m_node));
         
-        m_out.branch(failCondition, failCase, continuation);
+        m_out.branch(failCondition, rarely(failCase), usually(continuation));
         
         lastNext = m_out.appendTo(failCase, continuation);
         
index fb2fc93..986d374 100644 (file)
@@ -45,6 +45,104 @@ Output::~Output()
     llvm->DisposeBuilder(m_builder);
 }
 
+void Output::initialize(LModule module, LValue function, AbstractHeapRepository& heaps)
+{
+    IntrinsicRepository::initialize(module);
+    m_function = function;
+    m_heaps = &heaps;
+}
+
+LBasicBlock Output::appendTo(LBasicBlock block, LBasicBlock nextBlock)
+{
+    appendTo(block);
+    return insertNewBlocksBefore(nextBlock);
+}
+
+void Output::appendTo(LBasicBlock block)
+{
+    m_block = block;
+    
+    llvm->PositionBuilderAtEnd(m_builder, block);
+}
+
+LBasicBlock Output::newBlock(const char* name)
+{
+    if (!m_nextBlock)
+        return appendBasicBlock(m_context, m_function, name);
+    return insertBasicBlock(m_context, m_nextBlock, name);
+}
+
+LValue Output::sensibleDoubleToInt(LValue value)
+{
+    RELEASE_ASSERT(isX86());
+    return call(
+        x86SSE2CvtTSD2SIIntrinsic(),
+        insertElement(
+            insertElement(getUndef(vectorType(doubleType, 2)), value, int32Zero),
+            doubleZero, int32One));
+}
+
+LValue Output::load(TypedPointer pointer, LType refType)
+{
+    LValue result = get(intToPtr(pointer.value(), refType));
+    pointer.heap().decorateInstruction(result, *m_heaps);
+    return result;
+}
+
+void Output::store(LValue value, TypedPointer pointer, LType refType)
+{
+    LValue result = set(value, intToPtr(pointer.value(), refType));
+    pointer.heap().decorateInstruction(result, *m_heaps);
+}
+
+LValue Output::baseIndex(LValue base, LValue index, Scale scale, ptrdiff_t offset)
+{
+    LValue accumulatedOffset;
+        
+    switch (scale) {
+    case ScaleOne:
+        accumulatedOffset = index;
+        break;
+    case ScaleTwo:
+        accumulatedOffset = shl(index, intPtrOne);
+        break;
+    case ScaleFour:
+        accumulatedOffset = shl(index, intPtrTwo);
+        break;
+    case ScaleEight:
+    case ScalePtr:
+        accumulatedOffset = shl(index, intPtrThree);
+        break;
+    }
+        
+    if (offset)
+        accumulatedOffset = add(accumulatedOffset, constIntPtr(offset));
+        
+    return add(base, accumulatedOffset);
+}
+
+void Output::branch(LValue condition, LBasicBlock taken, Weight takenWeight, LBasicBlock notTaken, Weight notTakenWeight)
+{
+    LValue branch = buildCondBr(m_builder, condition, taken, notTaken);
+    
+    if (!takenWeight || !notTakenWeight)
+        return;
+    
+    double total = takenWeight.value() + notTakenWeight.value();
+    
+    setMetadata(
+        branch, profKind,
+        mdNode(
+            m_context, branchWeights,
+            constInt32(takenWeight.scaleToTotal(total)),
+            constInt32(notTakenWeight.scaleToTotal(total))));
+}
+
+void Output::crashNonTerminal()
+{
+    call(intToPtr(constIntPtr(abort), pointerType(functionType(voidType))));
+}
+
 } } // namespace JSC::FTL
 
 #endif // ENABLE(FTL_JIT)
index be40d11..fc19ad1 100644 (file)
 #include "FTLAbstractHeapRepository.h"
 #include "FTLCommonValues.h"
 #include "FTLIntrinsicRepository.h"
+#include "FTLSwitchCase.h"
 #include "FTLTypedPointer.h"
+#include "FTLWeight.h"
+#include "FTLWeightedTarget.h"
 #include <wtf/StringPrintStream.h>
 
 namespace JSC { namespace FTL {
@@ -70,12 +73,7 @@ public:
     Output(LContext);
     ~Output();
     
-    void initialize(LModule module, LValue function, AbstractHeapRepository& heaps)
-    {
-        IntrinsicRepository::initialize(module);
-        m_function = function;
-        m_heaps = &heaps;
-    }
+    void initialize(LModule, LValue, AbstractHeapRepository&);
     
     LBasicBlock insertNewBlocksBefore(LBasicBlock nextBlock)
     {
@@ -84,24 +82,11 @@ public:
         return lastNextBlock;
     }
     
-    LBasicBlock appendTo(LBasicBlock block, LBasicBlock nextBlock)
-    {
-        appendTo(block);
-        return insertNewBlocksBefore(nextBlock);
-    }
+    LBasicBlock appendTo(LBasicBlock, LBasicBlock nextBlock);
     
-    void appendTo(LBasicBlock block)
-    {
-        m_block = block;
-        
-        llvm->PositionBuilderAtEnd(m_builder, block);
-    }
-    LBasicBlock newBlock(const char* name = "")
-    {
-        if (!m_nextBlock)
-            return appendBasicBlock(m_context, m_function, name);
-        return insertBasicBlock(m_context, m_nextBlock, name);
-    }
+    void appendTo(LBasicBlock);
+    
+    LBasicBlock newBlock(const char* name = "");
     
     LValue param(unsigned index) { return getParam(m_function, index); }
     LValue constBool(bool value) { return constInt(boolean, value); }
@@ -189,7 +174,6 @@ public:
         return call(doubleSinIntrinsic(), value);
     }
 
-
     LValue doubleCos(LValue value)
     {
         return call(doubleCosIntrinsic(), value);
@@ -200,17 +184,8 @@ public:
         return call(doubleSqrtIntrinsic(), value);
     }
 
-
     static bool hasSensibleDoubleToInt() { return isX86(); }
-    LValue sensibleDoubleToInt(LValue value)
-    {
-        RELEASE_ASSERT(isX86());
-        return call(
-            x86SSE2CvtTSD2SIIntrinsic(),
-            insertElement(
-                insertElement(getUndef(vectorType(doubleType, 2)), value, int32Zero),
-                doubleZero, int32One));
-    }
+    LValue sensibleDoubleToInt(LValue);
     
     LValue signExt(LValue value, LType type) { return buildSExt(m_builder, value, type); }
     LValue zeroExt(LValue value, LType type) { return buildZExt(m_builder, value, type); }
@@ -233,17 +208,8 @@ public:
     LValue get(LValue reference) { return buildLoad(m_builder, reference); }
     LValue set(LValue value, LValue reference) { return buildStore(m_builder, value, reference); }
     
-    LValue load(TypedPointer pointer, LType refType)
-    {
-        LValue result = get(intToPtr(pointer.value(), refType));
-        pointer.heap().decorateInstruction(result, *m_heaps);
-        return result;
-    }
-    void store(LValue value, TypedPointer pointer, LType refType)
-    {
-        LValue result = set(value, intToPtr(pointer.value(), refType));
-        pointer.heap().decorateInstruction(result, *m_heaps);
-    }
+    LValue load(TypedPointer, LType refType);
+    void store(LValue, TypedPointer, LType refType);
     
     LValue load8(TypedPointer pointer) { return load(pointer, ref8); }
     LValue load16(TypedPointer pointer) { return load(pointer, ref16); }
@@ -281,31 +247,8 @@ public:
         return address(field, base, offset + field.offset());
     }
     
-    LValue baseIndex(LValue base, LValue index, Scale scale, ptrdiff_t offset = 0)
-    {
-        LValue accumulatedOffset;
-        
-        switch (scale) {
-        case ScaleOne:
-            accumulatedOffset = index;
-            break;
-        case ScaleTwo:
-            accumulatedOffset = shl(index, intPtrOne);
-            break;
-        case ScaleFour:
-            accumulatedOffset = shl(index, intPtrTwo);
-            break;
-        case ScaleEight:
-        case ScalePtr:
-            accumulatedOffset = shl(index, intPtrThree);
-            break;
-        }
-        
-        if (offset)
-            accumulatedOffset = add(accumulatedOffset, constIntPtr(offset));
-        
-        return add(base, accumulatedOffset);
-    }
+    LValue baseIndex(LValue base, LValue index, Scale, ptrdiff_t offset = 0);
+
     TypedPointer baseIndex(const AbstractHeap& heap, LValue base, LValue index, Scale scale, ptrdiff_t offset = 0)
     {
         return TypedPointer(heap, baseIndex(base, index, scale, offset));
@@ -412,9 +355,36 @@ public:
     }
     
     void jump(LBasicBlock destination) { buildBr(m_builder, destination); }
-    void branch(LValue condition, LBasicBlock taken, LBasicBlock notTaken) { buildCondBr(m_builder, condition, taken, notTaken); }
+    void branch(LValue condition, LBasicBlock taken, Weight takenWeight, LBasicBlock notTaken, Weight notTakenWeight);
+    void branch(LValue condition, WeightedTarget taken, WeightedTarget notTaken)
+    {
+        branch(condition, taken.target(), taken.weight(), notTaken.target(), notTaken.weight());
+    }
+    
     template<typename VectorType>
-    void switchInstruction(LValue value, const VectorType& cases, LBasicBlock fallThrough) { buildSwitch(m_builder, value, cases, fallThrough); }
+    void switchInstruction(LValue value, const VectorType& cases, LBasicBlock fallThrough, Weight fallThroughWeight)
+    {
+        LValue inst = buildSwitch(m_builder, value, cases, fallThrough);
+        
+        double total = 0;
+        if (!fallThroughWeight)
+            return;
+        total += fallThroughWeight.value();
+        for (unsigned i = cases.size(); i--;) {
+            if (!cases[i].weight())
+                return;
+            total += cases[i].weight().value();
+        }
+        
+        Vector<LValue> mdArgs;
+        mdArgs.append(branchWeights);
+        mdArgs.append(constInt32(fallThroughWeight.scaleToTotal(total)));
+        for (unsigned i = 0; i < cases.size(); ++i)
+            mdArgs.append(constInt32(cases[i].weight().scaleToTotal(total)));
+        
+        setMetadata(inst, profKind, mdNode(m_context, mdArgs));
+    }
+    
     void ret(LValue value) { buildRet(m_builder, value); }
     
     void unreachable() { buildUnreachable(m_builder); }
@@ -424,10 +394,8 @@ public:
         call(trapIntrinsic());
     }
     
-    void crashNonTerminal()
-    {
-        call(intToPtr(constIntPtr(abort), pointerType(functionType(voidType))));
-    }
+    void crashNonTerminal();
+
     void crash()
     {
         crashNonTerminal();
index e9f17d0..7816700 100644 (file)
 #if ENABLE(FTL_JIT)
 
 #include "FTLAbbreviatedTypes.h"
+#include "FTLWeight.h"
 
 namespace JSC { namespace FTL {
 
 class SwitchCase {
 public:
-    SwitchCase(LValue value, LBasicBlock target)
+    SwitchCase(LValue value, LBasicBlock target, Weight weight)
         : m_value(value)
         , m_target(target)
+        , m_weight(weight)
     {
     }
 
     LValue value() const { return m_value; }
     LBasicBlock target() const { return m_target; }
+    Weight weight() const { return m_weight; }
 
 private:
     LValue m_value;
     LBasicBlock m_target;
+    Weight m_weight;
 };
 
 } } // namespace JSC::FTL
diff --git a/Source/JavaScriptCore/ftl/FTLWeight.h b/Source/JavaScriptCore/ftl/FTLWeight.h
new file mode 100644 (file)
index 0000000..6394e4e
--- /dev/null
@@ -0,0 +1,75 @@
+/*
+ * Copyright (C) 2014 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#ifndef FTLWeight_h
+#define FTLWeight_h
+
+#if ENABLE(FTL_JIT)
+
+#include <wtf/MathExtras.h>
+#include <wtf/StdLibExtras.h>
+
+namespace JSC { namespace FTL {
+
+class Weight {
+public:
+    Weight()
+        : m_value(std::numeric_limits<float>::quiet_NaN())
+    {
+    }
+    
+    explicit Weight(float value)
+        : m_value(value)
+    {
+    }
+    
+    bool isSet() const { return m_value == m_value; }
+    bool operator!() const { return !isSet(); }
+    
+    float value() const { return m_value; }
+    
+    unsigned scaleToTotal(double total) const
+    {
+        // LLVM accepts branch 32-bit unsigned branch weights but in dumps it might
+        // display them as signed values. We don't need all 32 bits, so we just use the
+        // 31 bits.
+        double result = static_cast<double>(m_value) * INT_MAX / total;
+        if (result < 0)
+            return 0;
+        if (result > INT_MAX)
+            return INT_MAX;
+        return static_cast<unsigned>(result);
+    }
+    
+private:
+    float m_value;
+};
+
+} } // namespace JSC::FTL
+
+#endif // ENABLE(FTL_JIT)
+
+#endif // FTLWeight_h
+
diff --git a/Source/JavaScriptCore/ftl/FTLWeightedTarget.h b/Source/JavaScriptCore/ftl/FTLWeightedTarget.h
new file mode 100644 (file)
index 0000000..b4a8e5e
--- /dev/null
@@ -0,0 +1,89 @@
+/*
+ * Copyright (C) 2014 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#ifndef FTLWeightedTarget_h
+#define FTLWeightedTarget_h
+
+#if ENABLE(FTL_JIT)
+
+#include "FTLAbbreviatedTypes.h"
+#include "FTLWeight.h"
+
+namespace JSC { namespace FTL {
+
+class WeightedTarget {
+public:
+    WeightedTarget()
+        : m_target(nullptr)
+    {
+    }
+    
+    WeightedTarget(LBasicBlock target, Weight weight)
+        : m_target(target)
+        , m_weight(weight)
+    {
+    }
+    
+    WeightedTarget(LBasicBlock target, float weight)
+        : m_target(target)
+        , m_weight(weight)
+    {
+    }
+    
+    LBasicBlock target() const { return m_target; }
+    Weight weight() const { return m_weight; }
+    
+private:
+    LBasicBlock m_target;
+    Weight m_weight;
+};
+
+// Helpers for creating weighted targets for statically known (or unknown) branch
+// profiles.
+
+inline WeightedTarget usually(LBasicBlock block)
+{
+    return WeightedTarget(block, 1);
+}
+
+inline WeightedTarget rarely(LBasicBlock block)
+{
+    return WeightedTarget(block, 0);
+}
+
+// This means we let LLVM figure it out basic on its static estimates. LLVM's static
+// estimates are usually pretty darn good, so there's generally nothing wrong with
+// using this.
+inline WeightedTarget unsure(LBasicBlock block)
+{
+    return WeightedTarget(block, Weight());
+}
+
+} } // namespace JSC::FTL
+
+#endif // ENABLE(FTL_JIT)
+
+#endif // FTLWeightedTarget_h
+