OSR exit using llvm.webkit.stackmap should pass more tests
authorfpizlo@apple.com <fpizlo@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 10 Oct 2013 22:57:10 +0000 (22:57 +0000)
committerfpizlo@apple.com <fpizlo@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 10 Oct 2013 22:57:10 +0000 (22:57 +0000)
https://bugs.webkit.org/show_bug.cgi?id=122518

Reviewed by Mark Hahnenberg.

- Make the X86Assembler capable of dealing with all XMM registers.

- Make the StackMaps code on WebKit's side capable of dealing with XMM registers.

- Factor out most of the smarts of StackMaps::Location into a self-contained object.
  Previously you needed both StackMaps::Location and a StackMaps reference to do most
  things since the Location might have referred to a constant. Now you can just get a
  self-contained Location object.

- Fix a bug where OSR exit generation thunk generator was assuming that the call frame
  register is already in argumentGPR0. In the future, the call frame will just be the
  machine FP and we won't have to do anything special. But for now the "call frame" is
  just a normal value in LLVM IR and may end up in any register. Make the OSR exit
  generation thunk generator polymorphic over the call frame argument's Location.

- Move the stuff that depends on the polymorphic OSR exit generation thunk generator
  into the finalizer, since generating and linking one of those thunks requires a cache
  flush and we need to do that on the main thread.

* JavaScriptCore.xcodeproj/project.pbxproj:
* assembler/ARMv7Assembler.h:
(JSC::ARMv7Assembler::firstRegister):
(JSC::ARMv7Assembler::lastRegister):
(JSC::ARMv7Assembler::firstFPRegister):
(JSC::ARMv7Assembler::lastFPRegister):
* assembler/AbstractMacroAssembler.h:
(JSC::AbstractMacroAssembler::firstFPRegister):
(JSC::AbstractMacroAssembler::lastFPRegister):
* assembler/MacroAssembler.h:
(JSC::MacroAssembler::nextFPRegister):
* assembler/MacroAssemblerARMv7.h:
* assembler/MacroAssemblerX86Common.h:
* assembler/X86Assembler.h:
(JSC::X86Assembler::firstFPRegister):
(JSC::X86Assembler::lastFPRegister):
* dfg/DFGDriver.cpp:
(JSC::DFG::compileImpl):
* ftl/FTLCompile.cpp:
(JSC::FTL::fixFunctionBasedOnStackMaps):
* ftl/FTLExitThunkGenerator.cpp:
(JSC::FTL::ExitThunkGenerator::emitThunk):
(JSC::FTL::ExitThunkGenerator::emitThunks):
* ftl/FTLJITFinalizer.cpp:
(JSC::FTL::JITFinalizer::finalizeFunction):
* ftl/FTLJITFinalizer.h:
* ftl/FTLLink.cpp:
(JSC::FTL::link):
* ftl/FTLLocation.cpp: Added.
(JSC::FTL::Location::forStackmaps):
(JSC::FTL::Location::dump):
(JSC::FTL::Location::involvesGPR):
(JSC::FTL::Location::isGPR):
(JSC::FTL::Location::gpr):
(JSC::FTL::Location::isFPR):
(JSC::FTL::Location::fpr):
(JSC::FTL::Location::restoreInto):
(WTF::printInternal):
* ftl/FTLLocation.h: Added.
(JSC::FTL::Location::Location):
(JSC::FTL::Location::forRegister):
(JSC::FTL::Location::forIndirect):
(JSC::FTL::Location::forConstant):
(JSC::FTL::Location::kind):
(JSC::FTL::Location::hasDwarfRegNum):
(JSC::FTL::Location::dwarfRegNum):
(JSC::FTL::Location::hasOffset):
(JSC::FTL::Location::offset):
(JSC::FTL::Location::hasConstant):
(JSC::FTL::Location::constant):
(JSC::FTL::Location::operator!):
(JSC::FTL::Location::isHashTableDeletedValue):
(JSC::FTL::Location::operator==):
(JSC::FTL::Location::hash):
(JSC::FTL::LocationHash::hash):
(JSC::FTL::LocationHash::equal):
* ftl/FTLLowerDFGToLLVM.cpp:
(JSC::FTL::LowerDFGToLLVM::appendOSRExit):
(JSC::FTL::LowerDFGToLLVM::linkOSRExitsAndCompleteInitializationBlocks):
* ftl/FTLSaveRestore.cpp:
(JSC::FTL::bytesForFPRs):
(JSC::FTL::requiredScratchMemorySizeInBytes):
(JSC::FTL::offsetOfFPR):
(JSC::FTL::saveAllRegisters):
(JSC::FTL::restoreAllRegisters):
* ftl/FTLSaveRestore.h:
* ftl/FTLStackMaps.cpp:
(JSC::FTL::StackMaps::Location::restoreInto):
* ftl/FTLStackMaps.h:
* ftl/FTLState.h:
* ftl/FTLThunks.cpp:
(JSC::FTL::osrExitGenerationWithoutStackMapThunkGenerator):
(JSC::FTL::osrExitGenerationWithStackMapThunkGenerator):
* ftl/FTLThunks.h:
(JSC::FTL::generateIfNecessary):
(JSC::FTL::Thunks::getOSRExitGenerationThunk):
* runtime/VM.cpp:
(JSC::VM::VM):
* runtime/VM.h:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@157264 268f45cc-cd09-0410-ab3c-d52691b4dbfc

26 files changed:
Source/JavaScriptCore/ChangeLog
Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj
Source/JavaScriptCore/assembler/ARMv7Assembler.h
Source/JavaScriptCore/assembler/AbstractMacroAssembler.h
Source/JavaScriptCore/assembler/MacroAssembler.h
Source/JavaScriptCore/assembler/MacroAssemblerARMv7.h
Source/JavaScriptCore/assembler/MacroAssemblerX86Common.h
Source/JavaScriptCore/assembler/X86Assembler.h
Source/JavaScriptCore/dfg/DFGDriver.cpp
Source/JavaScriptCore/ftl/FTLCompile.cpp
Source/JavaScriptCore/ftl/FTLExitThunkGenerator.cpp
Source/JavaScriptCore/ftl/FTLJITFinalizer.cpp
Source/JavaScriptCore/ftl/FTLJITFinalizer.h
Source/JavaScriptCore/ftl/FTLLink.cpp
Source/JavaScriptCore/ftl/FTLLocation.cpp [new file with mode: 0644]
Source/JavaScriptCore/ftl/FTLLocation.h [new file with mode: 0644]
Source/JavaScriptCore/ftl/FTLLowerDFGToLLVM.cpp
Source/JavaScriptCore/ftl/FTLSaveRestore.cpp
Source/JavaScriptCore/ftl/FTLSaveRestore.h
Source/JavaScriptCore/ftl/FTLStackMaps.cpp
Source/JavaScriptCore/ftl/FTLStackMaps.h
Source/JavaScriptCore/ftl/FTLState.h
Source/JavaScriptCore/ftl/FTLThunks.cpp
Source/JavaScriptCore/ftl/FTLThunks.h
Source/JavaScriptCore/runtime/VM.cpp
Source/JavaScriptCore/runtime/VM.h

index 5f3d839..08c52eb 100644 (file)
@@ -1,3 +1,109 @@
+2013-10-10  Filip Pizlo  <fpizlo@apple.com>
+
+        OSR exit using llvm.webkit.stackmap should pass more tests
+        https://bugs.webkit.org/show_bug.cgi?id=122518
+
+        Reviewed by Mark Hahnenberg.
+        
+        - Make the X86Assembler capable of dealing with all XMM registers.
+        
+        - Make the StackMaps code on WebKit's side capable of dealing with XMM registers.
+        
+        - Factor out most of the smarts of StackMaps::Location into a self-contained object.
+          Previously you needed both StackMaps::Location and a StackMaps reference to do most
+          things since the Location might have referred to a constant. Now you can just get a
+          self-contained Location object.
+        
+        - Fix a bug where OSR exit generation thunk generator was assuming that the call frame
+          register is already in argumentGPR0. In the future, the call frame will just be the
+          machine FP and we won't have to do anything special. But for now the "call frame" is
+          just a normal value in LLVM IR and may end up in any register. Make the OSR exit
+          generation thunk generator polymorphic over the call frame argument's Location.
+        
+        - Move the stuff that depends on the polymorphic OSR exit generation thunk generator
+          into the finalizer, since generating and linking one of those thunks requires a cache
+          flush and we need to do that on the main thread.
+
+        * JavaScriptCore.xcodeproj/project.pbxproj:
+        * assembler/ARMv7Assembler.h:
+        (JSC::ARMv7Assembler::firstRegister):
+        (JSC::ARMv7Assembler::lastRegister):
+        (JSC::ARMv7Assembler::firstFPRegister):
+        (JSC::ARMv7Assembler::lastFPRegister):
+        * assembler/AbstractMacroAssembler.h:
+        (JSC::AbstractMacroAssembler::firstFPRegister):
+        (JSC::AbstractMacroAssembler::lastFPRegister):
+        * assembler/MacroAssembler.h:
+        (JSC::MacroAssembler::nextFPRegister):
+        * assembler/MacroAssemblerARMv7.h:
+        * assembler/MacroAssemblerX86Common.h:
+        * assembler/X86Assembler.h:
+        (JSC::X86Assembler::firstFPRegister):
+        (JSC::X86Assembler::lastFPRegister):
+        * dfg/DFGDriver.cpp:
+        (JSC::DFG::compileImpl):
+        * ftl/FTLCompile.cpp:
+        (JSC::FTL::fixFunctionBasedOnStackMaps):
+        * ftl/FTLExitThunkGenerator.cpp:
+        (JSC::FTL::ExitThunkGenerator::emitThunk):
+        (JSC::FTL::ExitThunkGenerator::emitThunks):
+        * ftl/FTLJITFinalizer.cpp:
+        (JSC::FTL::JITFinalizer::finalizeFunction):
+        * ftl/FTLJITFinalizer.h:
+        * ftl/FTLLink.cpp:
+        (JSC::FTL::link):
+        * ftl/FTLLocation.cpp: Added.
+        (JSC::FTL::Location::forStackmaps):
+        (JSC::FTL::Location::dump):
+        (JSC::FTL::Location::involvesGPR):
+        (JSC::FTL::Location::isGPR):
+        (JSC::FTL::Location::gpr):
+        (JSC::FTL::Location::isFPR):
+        (JSC::FTL::Location::fpr):
+        (JSC::FTL::Location::restoreInto):
+        (WTF::printInternal):
+        * ftl/FTLLocation.h: Added.
+        (JSC::FTL::Location::Location):
+        (JSC::FTL::Location::forRegister):
+        (JSC::FTL::Location::forIndirect):
+        (JSC::FTL::Location::forConstant):
+        (JSC::FTL::Location::kind):
+        (JSC::FTL::Location::hasDwarfRegNum):
+        (JSC::FTL::Location::dwarfRegNum):
+        (JSC::FTL::Location::hasOffset):
+        (JSC::FTL::Location::offset):
+        (JSC::FTL::Location::hasConstant):
+        (JSC::FTL::Location::constant):
+        (JSC::FTL::Location::operator!):
+        (JSC::FTL::Location::isHashTableDeletedValue):
+        (JSC::FTL::Location::operator==):
+        (JSC::FTL::Location::hash):
+        (JSC::FTL::LocationHash::hash):
+        (JSC::FTL::LocationHash::equal):
+        * ftl/FTLLowerDFGToLLVM.cpp:
+        (JSC::FTL::LowerDFGToLLVM::appendOSRExit):
+        (JSC::FTL::LowerDFGToLLVM::linkOSRExitsAndCompleteInitializationBlocks):
+        * ftl/FTLSaveRestore.cpp:
+        (JSC::FTL::bytesForFPRs):
+        (JSC::FTL::requiredScratchMemorySizeInBytes):
+        (JSC::FTL::offsetOfFPR):
+        (JSC::FTL::saveAllRegisters):
+        (JSC::FTL::restoreAllRegisters):
+        * ftl/FTLSaveRestore.h:
+        * ftl/FTLStackMaps.cpp:
+        (JSC::FTL::StackMaps::Location::restoreInto):
+        * ftl/FTLStackMaps.h:
+        * ftl/FTLState.h:
+        * ftl/FTLThunks.cpp:
+        (JSC::FTL::osrExitGenerationWithoutStackMapThunkGenerator):
+        (JSC::FTL::osrExitGenerationWithStackMapThunkGenerator):
+        * ftl/FTLThunks.h:
+        (JSC::FTL::generateIfNecessary):
+        (JSC::FTL::Thunks::getOSRExitGenerationThunk):
+        * runtime/VM.cpp:
+        (JSC::VM::VM):
+        * runtime/VM.h:
+
 2013-10-09  Filip Pizlo  <fpizlo@apple.com>
 
         FTL: Soft-link LLVM as a workaround for LLVM's static initializers and exit-time destructors
index 168242d..0ad0536 100644 (file)
                0FCEFADC18064A1400472CE4 /* config_llvm.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FCEFADB18064A1400472CE4 /* config_llvm.h */; };
                0FCEFAAB1804C13E00472CE4 /* FTLSaveRestore.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FCEFAA91804C13E00472CE4 /* FTLSaveRestore.cpp */; };
                0FCEFAAC1804C13E00472CE4 /* FTLSaveRestore.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FCEFAAA1804C13E00472CE4 /* FTLSaveRestore.h */; settings = {ATTRIBUTES = (Private, ); }; };
+               0FCEFADF180738C000472CE4 /* FTLLocation.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FCEFADD180738C000472CE4 /* FTLLocation.cpp */; };
+               0FCEFAE0180738C000472CE4 /* FTLLocation.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FCEFADE180738C000472CE4 /* FTLLocation.h */; settings = {ATTRIBUTES = (Private, ); }; };
                0FD2C92416D01EE900C7803F /* StructureInlines.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FD2C92316D01EE900C7803F /* StructureInlines.h */; settings = {ATTRIBUTES = (Private, ); }; };
                0FD3C82614115D4000FD81CB /* DFGDriver.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FD3C82014115CF800FD81CB /* DFGDriver.cpp */; };
                0FD3C82814115D4F00FD81CB /* DFGDriver.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FD3C82214115D0E00FD81CB /* DFGDriver.h */; };
                0FCEFADB18064A1400472CE4 /* config_llvm.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = config_llvm.h; sourceTree = "<group>"; };
                0FCEFAA91804C13E00472CE4 /* FTLSaveRestore.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = FTLSaveRestore.cpp; path = ftl/FTLSaveRestore.cpp; sourceTree = "<group>"; };
                0FCEFAAA1804C13E00472CE4 /* FTLSaveRestore.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = FTLSaveRestore.h; path = ftl/FTLSaveRestore.h; sourceTree = "<group>"; };
+               0FCEFADD180738C000472CE4 /* FTLLocation.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = FTLLocation.cpp; path = ftl/FTLLocation.cpp; sourceTree = "<group>"; };
+               0FCEFADE180738C000472CE4 /* FTLLocation.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = FTLLocation.h; path = ftl/FTLLocation.h; sourceTree = "<group>"; };
                0FD2C92316D01EE900C7803F /* StructureInlines.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = StructureInlines.h; sourceTree = "<group>"; };
                0FD3C82014115CF800FD81CB /* DFGDriver.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGDriver.cpp; path = dfg/DFGDriver.cpp; sourceTree = "<group>"; };
                0FD3C82214115D0E00FD81CB /* DFGDriver.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGDriver.h; path = dfg/DFGDriver.h; sourceTree = "<group>"; };
                                A78A977E179738D5009DF744 /* FTLJITFinalizer.h */,
                                0F8F2B93172E049E007DBDA5 /* FTLLink.cpp */,
                                0F8F2B94172E049E007DBDA5 /* FTLLink.h */,
+                               0FCEFADD180738C000472CE4 /* FTLLocation.cpp */,
+                               0FCEFADE180738C000472CE4 /* FTLLocation.h */,
                                0FEA0A04170513DB00BB722C /* FTLLowerDFGToLLVM.cpp */,
                                0FEA0A05170513DB00BB722C /* FTLLowerDFGToLLVM.h */,
                                A7D89D0117A0B90400773AD8 /* FTLLoweredNodeValue.h */,
                                0F2B66E917B6B5AB00A7AE3F /* JSArrayBufferView.h in Headers */,
                                0F2B66EA17B6B5AB00A7AE3F /* JSArrayBufferViewInlines.h in Headers */,
                                BC18C4180E16F5CD00B34460 /* JSBase.h in Headers */,
+                               0FCEFAE0180738C000472CE4 /* FTLLocation.h in Headers */,
                                140D17D70E8AD4A9000CD17D /* JSBasePrivate.h in Headers */,
                                86FA9E92142BBB2E001773B7 /* JSBoundFunction.h in Headers */,
                                BC18C4190E16F5CD00B34460 /* JSCallbackConstructor.h in Headers */,
                                C2239D1A16262BDD005AC5FD /* GCThread.cpp in Sources */,
                                C21122E115DD9AB300790E3A /* GCThreadSharedData.cpp in Sources */,
                                0F93329F14CA7DCA0085F3C6 /* GetByIdStatus.cpp in Sources */,
+                               0FCEFADF180738C000472CE4 /* FTLLocation.cpp in Sources */,
                                14280855107EC0E70013E7B2 /* GetterSetter.cpp in Sources */,
                                142E3135134FF0A600AFADB5 /* HandleSet.cpp in Sources */,
                                142E3137134FF0A600AFADB5 /* HandleStack.cpp in Sources */,
index cc556b6..d855c2f 100644 (file)
@@ -489,6 +489,13 @@ public:
     typedef ARMRegisters::FPSingleRegisterID FPSingleRegisterID;
     typedef ARMRegisters::FPDoubleRegisterID FPDoubleRegisterID;
     typedef ARMRegisters::FPQuadRegisterID FPQuadRegisterID;
+    typedef FPDoubleRegisterID FPRegisterID;
+    
+    static RegisterID firstRegister() { return ARMRegisters::r0; }
+    static RegisterID lastRegister() { return ARMRegisters::r13; }
+    
+    static FPRegisterID firstFPRegister() { return ARMRegisters::d0; }
+    static FPRegisterID lastFPRegister() { return ARMRegisters::d31; }
 
     // (HS, LO, HI, LS) -> (AE, B, A, BE)
     // (VS, VC) -> (O, NO)
index ff1f3dd..18e0a34 100644 (file)
@@ -74,10 +74,14 @@ public:
     class Jump;
 
     typedef typename AssemblerType::RegisterID RegisterID;
+    typedef typename AssemblerType::FPRegisterID FPRegisterID;
     
     static RegisterID firstRegister() { return AssemblerType::firstRegister(); }
     static RegisterID lastRegister() { return AssemblerType::lastRegister(); }
 
+    static FPRegisterID firstFPRegister() { return AssemblerType::firstFPRegister(); }
+    static FPRegisterID lastFPRegister() { return AssemblerType::lastFPRegister(); }
+
     // Section 1: MacroAssembler operand types
     //
     // The following types are used as operands to MacroAssembler operations,
index 98058e7..0a188e9 100644 (file)
@@ -92,6 +92,11 @@ public:
     {
         return nextRegister(firstRealRegister());
     }
+    
+    static FPRegisterID nextFPRegister(FPRegisterID reg)
+    {
+        return static_cast<FPRegisterID>(reg + 1);
+    }
 
     using MacroAssemblerBase::pop;
     using MacroAssemblerBase::jump;
index 35d5732..b64c1c5 100644 (file)
@@ -101,8 +101,6 @@ public:
     };
     
 public:
-    typedef ARMRegisters::FPDoubleRegisterID FPRegisterID;
-
     static const Scale ScalePtr = TimesFour;
 
     enum RelationalCondition {
index 64c77cc..bbd124c 100644 (file)
@@ -46,7 +46,6 @@ protected:
     static const int DoubleConditionBits = DoubleConditionBitInvert | DoubleConditionBitSpecial;
 
 public:
-    typedef X86Assembler::FPRegisterID FPRegisterID;
     typedef X86Assembler::XMMRegisterID XMMRegisterID;
     
     static bool isCompactPtrAlignedAddressOffset(ptrdiff_t value)
index a1d2f25..f7738a1 100644 (file)
@@ -74,6 +74,17 @@ namespace X86Registers {
         xmm5,
         xmm6,
         xmm7,
+
+#if CPU(X86_64)
+        xmm8,
+        xmm9,
+        xmm10,
+        xmm11,
+        xmm12,
+        xmm13,
+        xmm14,
+        xmm15,
+#endif
     } XMMRegisterID;
 
 #if USE(MASM_PROBE)
@@ -126,6 +137,7 @@ namespace X86Registers {
 class X86Assembler {
 public:
     typedef X86Registers::RegisterID RegisterID;
+    
     static RegisterID firstRegister() { return X86Registers::eax; }
     static RegisterID lastRegister()
     {
@@ -138,6 +150,16 @@ public:
     
     typedef X86Registers::XMMRegisterID XMMRegisterID;
     typedef XMMRegisterID FPRegisterID;
+    
+    static FPRegisterID firstFPRegister() { return X86Registers::xmm0; }
+    static FPRegisterID lastFPRegister()
+    {
+#if CPU(X86_64)
+        return X86Registers::xmm15;
+#else
+        return X86Registers::xmm7;
+#endif
+    }
 
     typedef enum {
         ConditionO,
index dadc2b9..6eeb197 100644 (file)
@@ -77,7 +77,7 @@ static CompilationResult compileImpl(
         dataLog("DFG(Driver) compiling ", *codeBlock, " with ", mode, ", number of instructions = ", codeBlock->instructionCount(), "\n");
     
     // Make sure that any stubs that the DFG is going to use are initialized. We want to
-    // make sure that al JIT code generation does finalization on the main thread.
+    // make sure that all JIT code generation does finalization on the main thread.
     vm.getCTIStub(osrExitGenerationThunkGenerator);
     vm.getCTIStub(throwExceptionFromCallSlowPathGenerator);
     vm.getCTIStub(linkCallThunkGenerator);
@@ -86,7 +86,7 @@ static CompilationResult compileImpl(
     vm.getCTIStub(virtualCallThunkGenerator);
     vm.getCTIStub(virtualConstructThunkGenerator);
 #if ENABLE(FTL_JIT)
-    vm.getCTIStub(FTL::osrExitGenerationThunkGenerator);
+    vm.getCTIStub(FTL::osrExitGenerationWithoutStackMapThunkGenerator);
 #endif
     
     RefPtr<Plan> plan = adoptRef(
index 039be55..b11a3f1 100644 (file)
@@ -110,9 +110,6 @@ static void fixFunctionBasedOnStackMaps(
     StackMaps::RecordMap& recordMap)
 {
     VM& vm = state.graph.m_vm;
-    MacroAssemblerCodeRef osrExitThunk =
-        vm.getCTIStub(osrExitGenerationThunkGenerator);
-    CodeLocationLabel target = CodeLocationLabel(osrExitThunk.code());
 
     ExitThunkGenerator exitThunkGenerator(state);
     exitThunkGenerator.emitThunks();
@@ -120,28 +117,37 @@ static void fixFunctionBasedOnStackMaps(
         OwnPtr<LinkBuffer> linkBuffer = adoptPtr(new LinkBuffer(
             vm, &exitThunkGenerator, codeBlock, JITCompilationMustSucceed));
         
-        ASSERT(state.osrExit.size() == state.jitCode->osrExit.size());
+        ASSERT(state.finalizer->osrExit.size() == state.jitCode->osrExit.size());
         
-        for (unsigned i = 0; i < state.osrExit.size(); ++i) {
-            OSRExitCompilationInfo& info = state.osrExit[i];
+        for (unsigned i = 0; i < state.jitCode->osrExit.size(); ++i) {
+            OSRExitCompilationInfo& info = state.finalizer->osrExit[i];
             OSRExit& exit = jitCode->osrExit[i];
-            
-            linkBuffer->link(info.m_thunkJump, target);
+            StackMaps::RecordMap::iterator iter = recordMap.find(exit.m_stackmapID);
+            if (iter == recordMap.end()) {
+                // It was optimized out.
+                continue;
+            }
             
             info.m_thunkAddress = linkBuffer->locationOf(info.m_thunkLabel);
             
             exit.m_patchableCodeOffset = linkBuffer->offsetOf(info.m_thunkJump);
         }
         
-        state.finalizer->initializeExitThunksLinkBuffer(linkBuffer.release());
+        state.finalizer->exitThunksLinkBuffer = linkBuffer.release();
     }
 
     RepatchBuffer repatchBuffer(codeBlock);
     
     for (unsigned exitIndex = jitCode->osrExit.size(); exitIndex--;) {
-        OSRExitCompilationInfo& info = state.osrExit[exitIndex];
+        OSRExitCompilationInfo& info = state.finalizer->osrExit[exitIndex];
         OSRExit& exit = jitCode->osrExit[exitIndex];
-        StackMaps::Record& record = recordMap.find(exit.m_stackmapID)->value;
+        StackMaps::RecordMap::iterator iter = recordMap.find(exit.m_stackmapID);
+        if (iter == recordMap.end()) {
+            // This could happen if LLVM optimizes out an OSR exit.
+            continue;
+        }
+        
+        StackMaps::Record& record = iter->value;
         
         repatchBuffer.replaceWithJump(
             CodeLocationLabel(
index 115013e..01eb4bc 100644 (file)
@@ -48,7 +48,7 @@ ExitThunkGenerator::~ExitThunkGenerator()
 
 void ExitThunkGenerator::emitThunk(unsigned index)
 {
-    OSRExitCompilationInfo& info = m_state.osrExit[index];
+    OSRExitCompilationInfo& info = m_state.finalizer->osrExit[index];
     
     info.m_thunkLabel = label();
     if (Options::ftlOSRExitUsesStackmap())
@@ -62,7 +62,7 @@ void ExitThunkGenerator::emitThunk(unsigned index)
 
 void ExitThunkGenerator::emitThunks()
 {
-    for (unsigned i = 0; i < m_state.osrExit.size(); ++i)
+    for (unsigned i = 0; i < m_state.finalizer->osrExit.size(); ++i)
         emitThunk(i);
 }
 
index b23c5b2..583faa1 100644 (file)
@@ -30,6 +30,7 @@
 
 #include "CodeBlockWithJITType.h"
 #include "DFGPlan.h"
+#include "FTLThunks.h"
 
 namespace JSC { namespace FTL {
 
@@ -52,27 +53,46 @@ bool JITFinalizer::finalize()
 
 bool JITFinalizer::finalizeFunction()
 {
-    for (unsigned i = m_jitCode->handles().size(); i--;) {
+    for (unsigned i = jitCode->handles().size(); i--;) {
         MacroAssembler::cacheFlush(
-            m_jitCode->handles()[i]->start(), m_jitCode->handles()[i]->sizeInBytes());
+            jitCode->handles()[i]->start(), jitCode->handles()[i]->sizeInBytes());
     }
     
-    if (m_exitThunksLinkBuffer) {
-        m_jitCode->initializeExitThunks(
+    if (exitThunksLinkBuffer) {
+        StackMaps::RecordMap recordMap = jitCode->stackmaps.getRecordMap();
+        
+        for (unsigned i = 0; i < osrExit.size(); ++i) {
+            OSRExitCompilationInfo& info = osrExit[i];
+            OSRExit& exit = jitCode->osrExit[i];
+            StackMaps::RecordMap::iterator iter = recordMap.find(exit.m_stackmapID);
+            if (iter == recordMap.end()) {
+                // It's OK, it was optimized out.
+                continue;
+            }
+            
+            exitThunksLinkBuffer->link(
+                info.m_thunkJump,
+                CodeLocationLabel(
+                    m_plan.vm.ftlThunks->getOSRExitGenerationThunk(
+                        m_plan.vm, Location::forStackmaps(
+                            jitCode->stackmaps, iter->value.locations[0])).code()));
+        }
+        
+        jitCode->initializeExitThunks(
             FINALIZE_DFG_CODE(
-                *m_exitThunksLinkBuffer,
+                *exitThunksLinkBuffer,
                 ("FTL exit thunks for %s", toCString(CodeBlockWithJITType(m_plan.codeBlock.get(), JITCode::FTLJIT)).data())));
     } // else this function had no OSR exits, so no exit thunks.
     
     MacroAssemblerCodePtr withArityCheck;
-    if (m_arityCheck.isSet())
-        withArityCheck = m_entrypointLinkBuffer->locationOf(m_arityCheck);
-    m_jitCode->initializeCode(
+    if (arityCheck.isSet())
+        withArityCheck = entrypointLinkBuffer->locationOf(arityCheck);
+    jitCode->initializeCode(
         FINALIZE_DFG_CODE(
-            *m_entrypointLinkBuffer,
-            ("FTL entrypoint thunk for %s with LLVM generated code at %p", toCString(CodeBlockWithJITType(m_plan.codeBlock.get(), JITCode::FTLJIT)).data(), m_function)));
+            *entrypointLinkBuffer,
+            ("FTL entrypoint thunk for %s with LLVM generated code at %p", toCString(CodeBlockWithJITType(m_plan.codeBlock.get(), JITCode::FTLJIT)).data(), function)));
     
-    m_plan.codeBlock->setJITCode(m_jitCode, withArityCheck);
+    m_plan.codeBlock->setJITCode(jitCode, withArityCheck);
     
     return true;
 }
index d9bcb01..535c410 100644 (file)
@@ -33,6 +33,7 @@
 #include "DFGFinalizer.h"
 #include "FTLGeneratedFunction.h"
 #include "FTLJITCode.h"
+#include "FTLOSRExitCompilationInfo.h"
 #include "LLVMAPI.h"
 #include "LinkBuffer.h"
 #include "MacroAssembler.h"
@@ -44,36 +45,15 @@ public:
     JITFinalizer(DFG::Plan&);
     virtual ~JITFinalizer();
     
-    void initializeExitThunksLinkBuffer(PassOwnPtr<LinkBuffer> buffer)
-    {
-        m_exitThunksLinkBuffer = buffer;
-    }
-    void initializeEntrypointLinkBuffer(PassOwnPtr<LinkBuffer> buffer)
-    {
-        m_entrypointLinkBuffer = buffer;
-    }
-    void initializeFunction(GeneratedFunction function)
-    {
-        m_function = function;
-    }
-    void initializeArityCheck(MacroAssembler::Label label)
-    {
-        m_arityCheck = label;
-    }
-    void initializeJITCode(PassRefPtr<JITCode> jitCode)
-    {
-        m_jitCode = jitCode;
-    }
-    
     bool finalize();
     bool finalizeFunction();
 
-private:
-    OwnPtr<LinkBuffer> m_exitThunksLinkBuffer;
-    OwnPtr<LinkBuffer> m_entrypointLinkBuffer;
-    MacroAssembler::Label m_arityCheck;
-    GeneratedFunction m_function;
-    RefPtr<JITCode> m_jitCode;
+    OwnPtr<LinkBuffer> exitThunksLinkBuffer;
+    OwnPtr<LinkBuffer> entrypointLinkBuffer;
+    Vector<OSRExitCompilationInfo> osrExit;
+    MacroAssembler::Label arityCheck;
+    GeneratedFunction function;
+    RefPtr<JITCode> jitCode;
 };
 
 } } // namespace JSC::FTL
index c453da9..6af6f06 100644 (file)
@@ -174,10 +174,10 @@ void link(State& state)
         break;
     }
     
-    state.finalizer->initializeEntrypointLinkBuffer(linkBuffer.release());
-    state.finalizer->initializeFunction(state.generatedFunction);
-    state.finalizer->initializeArityCheck(arityCheck);
-    state.finalizer->initializeJITCode(state.jitCode);
+    state.finalizer->entrypointLinkBuffer = linkBuffer.release();
+    state.finalizer->function = state.generatedFunction;
+    state.finalizer->arityCheck = arityCheck;
+    state.finalizer->jitCode = state.jitCode;
 }
 
 } } // namespace JSC::FTL
diff --git a/Source/JavaScriptCore/ftl/FTLLocation.cpp b/Source/JavaScriptCore/ftl/FTLLocation.cpp
new file mode 100644 (file)
index 0000000..fbabf57
--- /dev/null
@@ -0,0 +1,195 @@
+/*
+ * Copyright (C) 2013 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#include "config.h"
+#include "FTLLocation.h"
+
+#if ENABLE(FTL_JIT)
+
+#include "FTLSaveRestore.h"
+#include <wtf/CommaPrinter.h>
+#include <wtf/DataLog.h>
+#include <wtf/ListDump.h>
+
+namespace JSC { namespace FTL {
+
+Location Location::forStackmaps(const StackMaps& stackmaps, const StackMaps::Location& location)
+{
+    switch (location.kind) {
+    case StackMaps::Location::Unprocessed:
+        return Location();
+        
+    case StackMaps::Location::Register:
+        return forRegister(location.dwarfRegNum);
+        
+    case StackMaps::Location::Indirect:
+        return forIndirect(location.dwarfRegNum, location.offset);
+        
+    case StackMaps::Location::Constant:
+        return forConstant(location.offset);
+        
+    case StackMaps::Location::ConstantIndex:
+        return forConstant(stackmaps.constants[location.offset].integer);
+    }
+    
+    RELEASE_ASSERT_NOT_REACHED();
+}
+
+void Location::dump(PrintStream& out) const
+{
+    out.print("(", kind());
+    if (hasDwarfRegNum())
+        out.print(", reg", dwarfRegNum());
+    if (hasOffset())
+        out.print(", ", offset());
+    if (hasConstant())
+        out.print(", ", constant());
+    out.print(")");
+}
+
+bool Location::involvesGPR() const
+{
+    return isGPR() || kind() == Indirect;
+}
+
+#if CPU(X86_64) // CPU cases for Location methods
+// This decodes Dwarf flavour 0 for x86-64.
+bool Location::isGPR() const
+{
+    return kind() == Register && dwarfRegNum() < 16;
+}
+
+GPRReg Location::gpr() const
+{
+    // Stupidly, Dwarf doesn't number the registers in the same way as the architecture;
+    // for example, the architecture encodes CX as 1 and DX as 2 while Dwarf does the
+    // opposite. Hence we need the switch.
+    
+    ASSERT(isGPR());
+    
+    switch (dwarfRegNum()) {
+    case 0:
+        return X86Registers::eax;
+    case 1:
+        return X86Registers::edx;
+    case 2:
+        return X86Registers::ecx;
+    case 3:
+        return X86Registers::ebx;
+    case 4:
+        return X86Registers::esi;
+    case 5:
+        return X86Registers::edi;
+    case 6:
+        return X86Registers::ebp;
+    case 7:
+        return X86Registers::esp;
+    default:
+        RELEASE_ASSERT(dwarfRegNum() < 16);
+        // Registers r8..r15 are numbered sensibly.
+        return static_cast<GPRReg>(dwarfRegNum());
+    }
+}
+
+bool Location::isFPR() const
+{
+    return kind() == Register && dwarfRegNum() >= 17 && dwarfRegNum() <= 32;
+}
+
+FPRReg Location::fpr() const
+{
+    ASSERT(isFPR());
+    return static_cast<FPRReg>(dwarfRegNum() - 17);
+}
+
+void Location::restoreInto(MacroAssembler& jit, char* savedRegisters, GPRReg result) const
+{
+    if (isGPR()) {
+        jit.load64(savedRegisters + offsetOfGPR(gpr()), result);
+        return;
+    }
+    
+    if (isFPR()) {
+        jit.load64(savedRegisters + offsetOfFPR(fpr()), result);
+        return;
+    }
+    
+    switch (kind()) {
+    case Register:
+        // LLVM used some register that we don't know about!
+        RELEASE_ASSERT_NOT_REACHED();
+        return;
+        
+    case Indirect:
+        jit.load64(savedRegisters + offsetOfGPR(gpr()), result);
+        jit.load64(MacroAssembler::Address(result, offset()), result);
+        return;
+        
+    case Constant:
+        jit.move(MacroAssembler::TrustedImm64(constant()), result);
+        return;
+        
+    case Unprocessed:
+        // Should never see this - it's an enumeration entry on LLVM's side that means that
+        // it hasn't processed this location.
+        RELEASE_ASSERT_NOT_REACHED();
+        return;
+    }
+    
+    RELEASE_ASSERT_NOT_REACHED();
+}
+#else // CPU cases for Location methods
+#error "CPU architecture not supported."
+#endif // CPU cases for Location methods
+
+} } // namespace JSC::FTL
+
+namespace WTF {
+
+using namespace JSC::FTL;
+
+void printInternal(PrintStream& out, JSC::FTL::Location::Kind kind)
+{
+    switch (kind) {
+    case Location::Unprocessed:
+        out.print("Unprocessed");
+        return;
+    case Location::Register:
+        out.print("Register");
+        return;
+    case Location::Indirect:
+        out.print("Indirect");
+        return;
+    case Location::Constant:
+        out.print("Constant");
+        return;
+    }
+    RELEASE_ASSERT_NOT_REACHED();
+}
+
+} // namespace WTF
+
+#endif // ENABLE(FTL_JIT)
+
diff --git a/Source/JavaScriptCore/ftl/FTLLocation.h b/Source/JavaScriptCore/ftl/FTLLocation.h
new file mode 100644 (file)
index 0000000..74fcf16
--- /dev/null
@@ -0,0 +1,197 @@
+/*
+ * Copyright (C) 2013 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#ifndef FTLLocation_h
+#define FTLLocation_h
+
+#include <wtf/Platform.h>
+
+#if ENABLE(FTL_JIT)
+
+#include "FPRInfo.h"
+#include "FTLStackMaps.h"
+#include "GPRInfo.h"
+#include <wtf/HashMap.h>
+
+namespace JSC { namespace FTL {
+
+class Location {
+public:
+    enum Kind {
+        Unprocessed,
+        Register,
+        Indirect,
+        Constant
+    };
+    
+    Location()
+        : m_kind(Unprocessed)
+    {
+        u.constant = 0;
+    }
+    
+    Location(WTF::HashTableDeletedValueType)
+        : m_kind(Unprocessed)
+    {
+        u.constant = 1;
+    }
+    
+    static Location forRegister(int16_t dwarfRegNum)
+    {
+        Location result;
+        result.m_kind = Register;
+        result.u.variable.dwarfRegNum = dwarfRegNum;
+        return result;
+    }
+    
+    static Location forIndirect(int16_t dwarfRegNum, int32_t offset)
+    {
+        Location result;
+        result.m_kind = Indirect;
+        result.u.variable.dwarfRegNum = dwarfRegNum;
+        result.u.variable.offset = offset;
+        return result;
+    }
+    
+    static Location forConstant(int64_t constant)
+    {
+        Location result;
+        result.m_kind = Constant;
+        result.u.constant = constant;
+        return result;
+    }
+
+    static Location forStackmaps(const StackMaps&, const StackMaps::Location&);
+    
+    Kind kind() const { return m_kind; }
+    
+    bool hasDwarfRegNum() const { return kind() == Register || kind() == Indirect; }
+    int16_t dwarfRegNum() const
+    {
+        ASSERT(hasDwarfRegNum());
+        return u.variable.dwarfRegNum;
+    }
+    
+    bool hasOffset() const { return kind() == Indirect; }
+    int32_t offset() const
+    {
+        ASSERT(hasOffset());
+        return u.variable.offset;
+    }
+    
+    bool hasConstant() const { return kind() == Constant; }
+    int64_t constant() const
+    {
+        ASSERT(hasConstant());
+        return u.constant;
+    }
+    
+    bool operator!() const { return kind() == Unprocessed && !u.variable.offset; }
+    
+    bool isHashTableDeletedValue() const { return kind() == Unprocessed && u.variable.offset; }
+    
+    bool operator==(const Location& other) const
+    {
+        return m_kind == other.m_kind
+            && u.constant == other.u.constant;
+    }
+    
+    unsigned hash() const
+    {
+        unsigned result = m_kind;
+        
+        switch (kind()) {
+        case Unprocessed:
+            result ^= u.variable.offset;
+            break;
+
+        case Register:
+            result ^= u.variable.dwarfRegNum;
+            break;
+            
+        case Indirect:
+            result ^= u.variable.dwarfRegNum;
+            result ^= u.variable.offset;
+            break;
+            
+        case Constant:
+            result ^= WTF::IntHash<int64_t>::hash(u.constant);
+            break;
+        }
+        
+        return WTF::IntHash<unsigned>::hash(result);
+    }
+    
+    void dump(PrintStream&) const;
+    
+    bool isGPR() const;
+    bool involvesGPR() const;
+    GPRReg gpr() const;
+    
+    bool isFPR() const;
+    FPRReg fpr() const;
+    
+    // Assuming that all registers are saved to the savedRegisters buffer according
+    // to FTLSaveRestore convention, this loads the value into the given register.
+    // The code that this generates isn't exactly super fast.
+    void restoreInto(MacroAssembler&, char* savedRegisters, GPRReg result) const;
+    
+private:
+    Kind m_kind;
+    union {
+        int64_t constant;
+        struct {
+            int16_t dwarfRegNum;
+            int32_t offset;
+        } variable;
+    } u;
+};
+
+struct LocationHash {
+    static unsigned hash(const Location& key) { return key.hash(); }
+    static bool equal(const Location& a, const Location& b) { return a == b; }
+    static const bool safeToCompareToEmptyOrDeleted = true;
+};
+
+} } // namespace JSC::FTL
+
+namespace WTF {
+
+void printInternal(PrintStream&, JSC::FTL::Location::Kind);
+
+template<typename T> struct DefaultHash;
+template<> struct DefaultHash<JSC::FTL::Location> {
+    typedef JSC::FTL::LocationHash Hash;
+};
+
+template<typename T> struct HashTraits;
+template<> struct HashTraits<JSC::FTL::Location> : SimpleClassHashTraits<JSC::FTL::Location> { };
+
+} // namespace WTF
+
+#endif // ENABLE(FTL_JIT)
+
+#endif // FTLLocation_h
+
index dbeb29d..834456f 100644 (file)
@@ -3241,17 +3241,17 @@ private:
         if (verboseCompilationEnabled())
             dataLog("    OSR exit with value sources: ", m_valueSources, "\n");
         
-        ASSERT(m_ftlState.jitCode->osrExit.size() == m_ftlState.osrExit.size());
-        unsigned index = m_ftlState.osrExit.size();
+        ASSERT(m_ftlState.jitCode->osrExit.size() == m_ftlState.finalizer->osrExit.size());
+        unsigned index = m_ftlState.finalizer->osrExit.size();
         
         m_ftlState.jitCode->osrExit.append(OSRExit(
             kind, lowValue.format(), m_graph.methodOfGettingAValueProfileFor(highValue),
             m_codeOriginForExitTarget, m_codeOriginForExitProfile, m_lastSetOperand.offset(),
             m_valueSources.numberOfArguments(), m_valueSources.numberOfLocals()));
-        m_ftlState.osrExit.append(OSRExitCompilationInfo());
+        m_ftlState.finalizer->osrExit.append(OSRExitCompilationInfo());
         
         OSRExit& exit = m_ftlState.jitCode->osrExit.last();
-        OSRExitCompilationInfo& info = m_ftlState.osrExit.last();
+        OSRExitCompilationInfo& info = m_ftlState.finalizer->osrExit.last();
 
         LBasicBlock lastNext = 0;
         LBasicBlock continuation = 0;
@@ -3493,7 +3493,7 @@ private:
     void linkOSRExitsAndCompleteInitializationBlocks()
     {
         MacroAssemblerCodeRef osrExitThunk =
-            vm().getCTIStub(osrExitGenerationThunkGenerator);
+            vm().getCTIStub(osrExitGenerationWithoutStackMapThunkGenerator);
         CodeLocationLabel target = CodeLocationLabel(osrExitThunk.code());
         
         m_out.appendTo(m_prologue);
@@ -3506,10 +3506,10 @@ private:
                 vm(), &m_exitThunkGenerator, m_ftlState.graph.m_codeBlock,
                 JITCompilationMustSucceed));
         
-            ASSERT(m_ftlState.osrExit.size() == m_ftlState.jitCode->osrExit.size());
+            ASSERT(m_ftlState.finalizer->osrExit.size() == m_ftlState.jitCode->osrExit.size());
         
-            for (unsigned i = 0; i < m_ftlState.osrExit.size(); ++i) {
-                OSRExitCompilationInfo& info = m_ftlState.osrExit[i];
+            for (unsigned i = 0; i < m_ftlState.finalizer->osrExit.size(); ++i) {
+                OSRExitCompilationInfo& info = m_ftlState.finalizer->osrExit[i];
                 OSRExit& exit = m_ftlState.jitCode->osrExit[i];
             
                 linkBuffer->link(info.m_thunkJump, target);
@@ -3522,7 +3522,7 @@ private:
                 exit.m_patchableCodeOffset = linkBuffer->offsetOf(info.m_thunkJump);
             }
         
-            m_ftlState.finalizer->initializeExitThunksLinkBuffer(linkBuffer.release());
+            m_ftlState.finalizer->exitThunksLinkBuffer = linkBuffer.release();
         }
 
         m_out.jump(lowBlock(m_graph.block(0)));
index df45394..227d39e 100644 (file)
@@ -39,9 +39,17 @@ static size_t bytesForGPRs()
     return (MacroAssembler::lastRegister() - MacroAssembler::firstRegister() + 1) * sizeof(int64_t);
 }
 
+static size_t bytesForFPRs()
+{
+    // FIXME: It might be worthwhile saving the full state of the FP registers, at some point.
+    // Right now we don't need this since we only do the save/restore just prior to OSR exit, and
+    // OSR exit will be guaranteed to only need the double portion of the FP registers.
+    return (MacroAssembler::lastFPRegister() - MacroAssembler::firstFPRegister() + 1) * sizeof(double);
+}
+
 size_t requiredScratchMemorySizeInBytes()
 {
-    return bytesForGPRs() + FPRInfo::numberOfArgumentRegisters * sizeof(double);
+    return bytesForGPRs() + bytesForFPRs();
 }
 
 size_t offsetOfGPR(GPRReg reg)
@@ -49,6 +57,11 @@ size_t offsetOfGPR(GPRReg reg)
     return (reg - MacroAssembler::firstRegister()) * sizeof(int64_t);
 }
 
+size_t offsetOfFPR(FPRReg reg)
+{
+    return bytesForGPRs() + (reg - MacroAssembler::firstFPRegister()) * sizeof(double);
+}
+
 void saveAllRegisters(MacroAssembler& jit, char* scratchMemory)
 {
     // Get the first register out of the way, so that we can use it as a pointer.
@@ -63,28 +76,20 @@ void saveAllRegisters(MacroAssembler& jit, char* scratchMemory)
     jit.peek64(MacroAssembler::secondRealRegister(), 0);
     jit.store64(MacroAssembler::secondRealRegister(), MacroAssembler::Address(MacroAssembler::firstRealRegister(), offsetOfGPR(MacroAssembler::firstRealRegister())));
     
-    // Save all FP argument registers.
-    // FIXME: We should actually be saving all FP registers.
-    // https://bugs.webkit.org/show_bug.cgi?id=122518
-    for (unsigned i = 0; i < FPRInfo::numberOfArgumentRegisters; ++i) {
-        jit.move(MacroAssembler::TrustedImmPtr(scratchMemory + bytesForGPRs() + sizeof(double) * i), GPRInfo::regT0);
-        jit.storeDouble(FPRInfo::toArgumentRegister(i), GPRInfo::regT0);
-    }
+    // Finally save all FPR's.
+    for (MacroAssembler::FPRegisterID reg = MacroAssembler::firstFPRegister(); reg <= MacroAssembler::lastFPRegister(); reg = MacroAssembler::nextFPRegister(reg))
+        jit.storeDouble(reg, MacroAssembler::Address(MacroAssembler::firstRealRegister(), offsetOfFPR(reg)));
 }
 
 void restoreAllRegisters(MacroAssembler& jit, char* scratchMemory)
 {
-    // Restore all FP argument registers.
-    // FIXME: We should actually be restoring all FP registers.
-    // https://bugs.webkit.org/show_bug.cgi?id=122518
-    for (unsigned i = 0; i < FPRInfo::numberOfArgumentRegisters; ++i) {
-        jit.move(MacroAssembler::TrustedImmPtr(scratchMemory + bytesForGPRs() + sizeof(double) * i), GPRInfo::regT0);
-        jit.loadDouble(GPRInfo::regT0, FPRInfo::toArgumentRegister(i));
-    }
-    
-    // Now, restore all GPRs.
+    // Give ourselves a pointer to the scratch memory.
     jit.move(MacroAssembler::TrustedImmPtr(scratchMemory), MacroAssembler::firstRealRegister());
     
+    // Restore all FPR's.
+    for (MacroAssembler::FPRegisterID reg = MacroAssembler::firstFPRegister(); reg <= MacroAssembler::lastFPRegister(); reg = MacroAssembler::nextFPRegister(reg))
+        jit.loadDouble(MacroAssembler::Address(MacroAssembler::firstRealRegister(), offsetOfFPR(reg)), reg);
+    
     for (MacroAssembler::RegisterID reg = MacroAssembler::secondRealRegister(); reg <= MacroAssembler::lastRegister(); reg = MacroAssembler::nextRegister(reg))
         jit.load64(MacroAssembler::Address(MacroAssembler::firstRealRegister(), offsetOfGPR(reg)), reg);
     
index 1cd0b8b..1ef3f02 100644 (file)
@@ -30,6 +30,7 @@
 
 #if ENABLE(FTL_JIT)
 
+#include "FPRInfo.h"
 #include "GPRInfo.h"
 
 namespace JSC {
@@ -41,6 +42,7 @@ namespace FTL {
 size_t requiredScratchMemorySizeInBytes();
 
 size_t offsetOfGPR(GPRReg);
+size_t offsetOfFPR(FPRReg);
 
 // Assumes that top-of-stack can be used as a pointer-sized scratchpad. Saves all of
 // the registers into the scratch buffer such that RegisterID * sizeof(int64_t) is the
index 32a8b30..7e93767 100644 (file)
@@ -28,7 +28,7 @@
 
 #if ENABLE(FTL_JIT)
 
-#include "FTLSaveRestore.h"
+#include "FTLLocation.h"
 #include <wtf/CommaPrinter.h>
 #include <wtf/DataLog.h>
 #include <wtf/ListDump.h>
@@ -73,88 +73,11 @@ void StackMaps::Location::dump(PrintStream& out) const
     out.print("(", kind, ", reg", dwarfRegNum, ", ", offset, ")");
 }
 
-bool StackMaps::Location::involvesGPR() const
-{
-    return isGPR() || kind == Indirect;
-}
-
-#if CPU(X86_64) // CPU cases for StackMaps::Location
-// This decodes Dwarf flavour 0 for x86-64.
-bool StackMaps::Location::isGPR() const
-{
-    return kind == Register && dwarfRegNum < 16;
-}
-
-GPRReg StackMaps::Location::gpr() const
-{
-    // Stupidly, Dwarf doesn't number the registers in the same way as the architecture;
-    // for example, the architecture encodes CX as 1 and DX as 2 while Dwarf does the
-    // opposite. Hence we need the switch.
-    
-    switch (dwarfRegNum) {
-    case 0:
-        return X86Registers::eax;
-    case 1:
-        return X86Registers::edx;
-    case 2:
-        return X86Registers::ecx;
-    case 3:
-        return X86Registers::ebx;
-    case 4:
-        return X86Registers::esi;
-    case 5:
-        return X86Registers::edi;
-    case 6:
-        return X86Registers::ebp;
-    case 7:
-        return X86Registers::esp;
-    default:
-        RELEASE_ASSERT(dwarfRegNum < 16);
-        // Registers r8..r15 are numbered sensibly.
-        return static_cast<GPRReg>(dwarfRegNum);
-    }
-}
-
 void StackMaps::Location::restoreInto(
-    MacroAssembler& jit, const StackMaps& stackmaps, char* savedRegisters, GPRReg result)
+    MacroAssembler& jit, StackMaps& stackmaps, char* savedRegisters, GPRReg result) const
 {
-    if (isGPR()) {
-        jit.load64(savedRegisters + offsetOfGPR(gpr()), result);
-        return;
-    }
-    
-    switch (kind) {
-    case Register:
-        // FIXME: We need to handle the other registers.
-        // https://bugs.webkit.org/show_bug.cgi?id=122518
-        RELEASE_ASSERT_NOT_REACHED();
-        return;
-        
-    case Indirect:
-        jit.load64(savedRegisters + offsetOfGPR(gpr()), result);
-        jit.load64(MacroAssembler::Address(result, offset), result);
-        return;
-        
-    case Constant:
-        jit.move(MacroAssembler::TrustedImm32(offset), result);
-        return;
-        
-    case ConstantIndex:
-        jit.move(MacroAssembler::TrustedImm64(stackmaps.constants[offset].integer), result);
-        return;
-        
-    case Unprocessed:
-        // Should never see this - it's an enumeration entry on LLVM's side that means that
-        // it hasn't processed this location.
-        RELEASE_ASSERT_NOT_REACHED();
-        return;
-    }
-    
-    RELEASE_ASSERT_NOT_REACHED();
+    FTL::Location::forStackmaps(stackmaps, *this).restoreInto(jit, savedRegisters, result);
 }
-#else // CPU cases for StackMaps::Location
-#error "CPU architecture not supported."
-#endif // CPU cases for StackMaps::Location
 
 bool StackMaps::Record::parse(DataView* view, unsigned& offset)
 {
index 4e4c427..1a8a67b 100644 (file)
@@ -32,7 +32,6 @@
 
 #include "DataView.h"
 #include "GPRInfo.h"
-#include "JSCJSValue.h"
 #include <wtf/HashMap.h>
 
 namespace JSC {
@@ -42,8 +41,7 @@ class MacroAssembler;
 namespace FTL {
 
 struct StackMaps {
-    union Constant {
-        EncodedJSValue value;
+    struct Constant {
         int64_t integer;
         
         void parse(DataView*, unsigned& offset);
@@ -66,16 +64,7 @@ struct StackMaps {
         void parse(DataView*, unsigned& offset);
         void dump(PrintStream& out) const;
         
-        bool isGPR() const;
-        bool involvesGPR() const;
-        GPRReg gpr() const;
-        
-        // Assuming that all registers are saved to the savedRegisters buffer according
-        // to FTLSaveRestore convention, this loads the value into the given register.
-        // The code that this generates isn't exactly super fast.
-        void restoreInto(
-            MacroAssembler&, const StackMaps& stackmaps, char* savedRegisters,
-            GPRReg result);
+        void restoreInto(MacroAssembler&, StackMaps&, char* savedRegisters, GPRReg result) const;
     };
     
     struct Record {
index db21a58..1ff8a63 100644 (file)
@@ -35,7 +35,6 @@
 #include "FTLGeneratedFunction.h"
 #include "FTLJITCode.h"
 #include "FTLJITFinalizer.h"
-#include "FTLOSRExitCompilationInfo.h"
 #include "FTLStackMaps.h"
 #include <wtf/Noncopyable.h>
 
@@ -55,7 +54,6 @@ public:
     LModule module;
     LValue function;
     RefPtr<JITCode> jitCode;
-    Vector<OSRExitCompilationInfo> osrExit;
     GeneratedFunction generatedFunction;
     JITFinalizer* finalizer;
     Vector<CString> codeSectionNames;
index 594332e..a33aa8c 100644 (file)
@@ -39,21 +39,17 @@ namespace JSC { namespace FTL {
 
 using namespace DFG;
 
-MacroAssemblerCodeRef osrExitGenerationThunkGenerator(VM* vm)
+MacroAssemblerCodeRef osrExitGenerationWithoutStackMapThunkGenerator(VM* vm)
 {
     AssemblyHelpers jit(vm, 0);
     
-    // Note that in the ftlOSRExitUsesStackmap() case, the "return address" will be the
-    // OSR exit ID.
-    
     // Pretend that we're a C call frame.
     jit.push(MacroAssembler::framePointerRegister);
     jit.move(MacroAssembler::stackPointerRegister, MacroAssembler::framePointerRegister);
     jit.push(GPRInfo::regT0);
     jit.push(GPRInfo::regT0);
     
-    if (!Options::ftlOSRExitUsesStackmap())
-        jit.poke(GPRInfo::nonArgGPR0, 1);
+    jit.poke(GPRInfo::nonArgGPR0, 1);
     
     ScratchBuffer* scratchBuffer = vm->scratchBufferForSize(requiredScratchMemorySizeInBytes());
     char* buffer = static_cast<char*>(scratchBuffer->dataBuffer());
@@ -65,10 +61,7 @@ MacroAssemblerCodeRef osrExitGenerationThunkGenerator(VM* vm)
     jit.storePtr(MacroAssembler::TrustedImmPtr(requiredScratchMemorySizeInBytes()), GPRInfo::nonArgGPR1);
 
     // argument 0 is already the call frame.
-    if (Options::ftlOSRExitUsesStackmap())
-        jit.peek(GPRInfo::argumentGPR1, 3);
-    else
-        jit.peek(GPRInfo::argumentGPR1, 1);
+    jit.peek(GPRInfo::argumentGPR1, 1);
     MacroAssembler::Call functionCall = jit.call();
     
     // At this point we want to make a tail call to what was returned to us in the
@@ -98,6 +91,59 @@ MacroAssemblerCodeRef osrExitGenerationThunkGenerator(VM* vm)
     return FINALIZE_CODE(patchBuffer, ("FTL OSR exit generation thunk"));
 }
 
+MacroAssemblerCodeRef osrExitGenerationWithStackMapThunkGenerator(
+    VM& vm, const Location& location)
+{
+    AssemblyHelpers jit(&vm, 0);
+    
+    // Note that the "return address" will be the OSR exit ID.
+    
+    // Pretend that we're a C call frame.
+    jit.push(MacroAssembler::framePointerRegister);
+    jit.move(MacroAssembler::stackPointerRegister, MacroAssembler::framePointerRegister);
+    jit.push(GPRInfo::regT0);
+    jit.push(GPRInfo::regT0);
+    
+    ScratchBuffer* scratchBuffer = vm.scratchBufferForSize(requiredScratchMemorySizeInBytes());
+    char* buffer = static_cast<char*>(scratchBuffer->dataBuffer());
+    
+    saveAllRegisters(jit, buffer);
+    
+    // Tell GC mark phase how much of the scratch buffer is active during call.
+    jit.move(MacroAssembler::TrustedImmPtr(scratchBuffer->activeLengthPtr()), GPRInfo::nonArgGPR1);
+    jit.storePtr(MacroAssembler::TrustedImmPtr(requiredScratchMemorySizeInBytes()), GPRInfo::nonArgGPR1);
+
+    location.restoreInto(jit, buffer, GPRInfo::argumentGPR0);
+    jit.peek(GPRInfo::argumentGPR1, 3);
+    MacroAssembler::Call functionCall = jit.call();
+    
+    // At this point we want to make a tail call to what was returned to us in the
+    // returnValueGPR. But at the same time as we do this, we must restore all registers.
+    // The way we will accomplish this is by arranging to have the tail call target in the
+    // return address "slot" (be it a register or the stack).
+    
+    jit.move(GPRInfo::returnValueGPR, GPRInfo::regT0);
+    
+    // Prepare for tail call.
+    jit.pop(GPRInfo::regT1);
+    jit.pop(GPRInfo::regT1);
+    jit.pop(MacroAssembler::framePointerRegister);
+    
+    // At this point we're sitting on the return address - so if we did a jump right now, the
+    // tail-callee would be happy. Instead we'll stash the callee in the return address and then
+    // restore all registers.
+    
+    jit.restoreReturnAddressBeforeReturn(GPRInfo::regT0);
+    
+    restoreAllRegisters(jit, buffer);
+
+    jit.ret();
+    
+    LinkBuffer patchBuffer(vm, &jit, GLOBAL_THUNK_ID);
+    patchBuffer.link(functionCall, compileFTLOSRExit);
+    return FINALIZE_CODE(patchBuffer, ("FTL OSR exit generation thunk for callFrame at %s", toCString(location).data()));
+}
+
 } } // namespace JSC::FTL
 
 #endif // ENABLE(FTL_JIT)
index b83deed..0a3f29e 100644 (file)
@@ -30,7 +30,9 @@
 
 #if ENABLE(FTL_JIT)
 
+#include "FTLLocation.h"
 #include "MacroAssemblerCodeRef.h"
+#include <wtf/HashMap.h>
 
 namespace JSC {
 
@@ -38,7 +40,34 @@ class VM;
 
 namespace FTL {
 
-MacroAssemblerCodeRef osrExitGenerationThunkGenerator(VM*);
+MacroAssemblerCodeRef osrExitGenerationWithoutStackMapThunkGenerator(VM*);
+
+MacroAssemblerCodeRef osrExitGenerationWithStackMapThunkGenerator(VM&, const Location&);
+
+template<typename MapType, typename KeyType, typename GeneratorType>
+MacroAssemblerCodeRef generateIfNecessary(
+    VM& vm, MapType& map, const KeyType& key, GeneratorType generator)
+{
+    typename MapType::iterator iter = map.find(key);
+    if (iter != map.end())
+        return iter->value;
+    
+    MacroAssemblerCodeRef result = generator(vm, key);
+    map.add(key, result);
+    return result;
+}
+
+class Thunks {
+public:
+    MacroAssemblerCodeRef getOSRExitGenerationThunk(VM& vm, const Location& location)
+    {
+        return generateIfNecessary(
+            vm, m_osrExitThunks, location, osrExitGenerationWithStackMapThunkGenerator);
+    }
+    
+private:
+    HashMap<Location, MacroAssemblerCodeRef> m_osrExitThunks;
+};
 
 } } // namespace JSC::FTL
 
index da4b755..433d944 100644 (file)
@@ -38,6 +38,7 @@
 #include "DFGWorklist.h"
 #include "DebuggerActivation.h"
 #include "ErrorInstance.h"
+#include "FTLThunks.h"
 #include "FunctionConstructor.h"
 #include "GCActivityCallback.h"
 #include "GetterSetter.h"
@@ -254,6 +255,10 @@ VM::VM(VMType vmType, HeapType heapType)
     jitStubs = adoptPtr(new JITThunks());
     performPlatformSpecificJITAssertions(this);
 #endif
+
+#if ENABLE(FTL_JIT)
+    ftlThunks = std::make_unique<FTL::Thunks>();
+#endif // ENABLE(FTL_JIT)
     
     interpreter->initialize(this->canUseJIT());
     
index 32f473c..460a640 100644 (file)
@@ -35,7 +35,6 @@
 #include "Heap.h"
 #include "Intrinsic.h"
 #include "JITThunks.h"
-#include "JITThunks.h"
 #include "JSCJSValue.h"
 #include "JSLock.h"
 #include "LLIntData.h"
@@ -99,6 +98,11 @@ namespace JSC {
     class Worklist;
     }
 #endif // ENABLE(DFG_JIT)
+#if ENABLE(FTL_JIT)
+    namespace FTL {
+    class Thunks;
+    }
+#endif // ENABLE(FTL_JIT)
 
     struct HashTable;
     struct Instruction;
@@ -327,6 +331,9 @@ namespace JSC {
         }
         NativeExecutable* getHostFunction(NativeFunction, Intrinsic);
 #endif
+#if ENABLE(FTL_JIT)
+        std::unique_ptr<FTL::Thunks> ftlThunks;
+#endif
         NativeExecutable* getHostFunction(NativeFunction, NativeFunction constructor);
 
         static ptrdiff_t exceptionOffset()