Reenable Gigacage on ARM64.
authorkeith_miller@apple.com <keith_miller@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 6 Jun 2019 13:20:56 +0000 (13:20 +0000)
committerkeith_miller@apple.com <keith_miller@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 6 Jun 2019 13:20:56 +0000 (13:20 +0000)
https://bugs.webkit.org/show_bug.cgi?id=198453

Reviewed by Michael Saboff.

Source/bmalloc:

* bmalloc/Gigacage.h:

Source/JavaScriptCore:

This patch adds back Gigacaging on Apple's ARM64 ports. Unlike the
old Gigacage however, arm64e uses both Gigacaging and PAC. In
order to ensure the PAC bits are not stripped in the caging
process we use the bit field insert instruction to take the low
bits from caging and the high bits from the PAC authentication.

* assembler/MacroAssemblerARM64.h:
(JSC::MacroAssemblerARM64::bitFieldInsert64):
* assembler/MacroAssemblerARM64E.h:
* assembler/testmasm.cpp:
(JSC::testCagePreservesPACFailureBit):
(JSC::run):
* dfg/DFGSpeculativeJIT.cpp:
(JSC::DFG::SpeculativeJIT::jumpForTypedArrayIsNeuteredIfOutOfBounds):
(JSC::DFG::SpeculativeJIT::cageTypedArrayStorage):
(JSC::DFG::SpeculativeJIT::compileGetTypedArrayByteOffset):
(JSC::DFG::SpeculativeJIT::compileNewTypedArrayWithSize):
* ftl/FTLLowerDFGToB3.cpp:
(JSC::FTL::DFG::LowerDFGToB3::compileNewTypedArray):
(JSC::FTL::DFG::LowerDFGToB3::caged):
* jit/AssemblyHelpers.h:
(JSC::AssemblyHelpers::cageWithoutUntaging):
(JSC::AssemblyHelpers::cageConditionally):
(JSC::AssemblyHelpers::cage): Deleted.
* jit/JITPropertyAccess.cpp:
(JSC::JIT::emitIntTypedArrayGetByVal):
(JSC::JIT::emitFloatTypedArrayGetByVal):
(JSC::JIT::emitIntTypedArrayPutByVal):
(JSC::JIT::emitFloatTypedArrayPutByVal):
* llint/LowLevelInterpreter.asm:
* llint/LowLevelInterpreter64.asm:
* offlineasm/arm64.rb:
* offlineasm/instructions.rb:
* offlineasm/registers.rb:
* wasm/WasmAirIRGenerator.cpp:
(JSC::Wasm::AirIRGenerator::restoreWebAssemblyGlobalState):
(JSC::Wasm::AirIRGenerator::addCallIndirect):
* wasm/WasmB3IRGenerator.cpp:
(JSC::Wasm::B3IRGenerator::restoreWebAssemblyGlobalState):
(JSC::Wasm::B3IRGenerator::addCallIndirect):
* wasm/WasmBinding.cpp:
(JSC::Wasm::wasmToWasm):
* wasm/js/JSToWasm.cpp:
(JSC::Wasm::createJSToWasmWrapper):
* wasm/js/WebAssemblyFunction.cpp:
(JSC::WebAssemblyFunction::jsCallEntrypointSlow):

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@246150 268f45cc-cd09-0410-ab3c-d52691b4dbfc

20 files changed:
Source/JavaScriptCore/ChangeLog
Source/JavaScriptCore/assembler/MacroAssemblerARM64.h
Source/JavaScriptCore/assembler/MacroAssemblerARM64E.h
Source/JavaScriptCore/assembler/testmasm.cpp
Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp
Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp
Source/JavaScriptCore/jit/AssemblyHelpers.h
Source/JavaScriptCore/jit/JITPropertyAccess.cpp
Source/JavaScriptCore/llint/LowLevelInterpreter.asm
Source/JavaScriptCore/llint/LowLevelInterpreter64.asm
Source/JavaScriptCore/offlineasm/arm64.rb
Source/JavaScriptCore/offlineasm/instructions.rb
Source/JavaScriptCore/offlineasm/registers.rb
Source/JavaScriptCore/wasm/WasmAirIRGenerator.cpp
Source/JavaScriptCore/wasm/WasmB3IRGenerator.cpp
Source/JavaScriptCore/wasm/WasmBinding.cpp
Source/JavaScriptCore/wasm/js/JSToWasm.cpp
Source/JavaScriptCore/wasm/js/WebAssemblyFunction.cpp
Source/bmalloc/ChangeLog
Source/bmalloc/bmalloc/Gigacage.h

index a3b9ec5..38e08e3 100644 (file)
@@ -1,3 +1,57 @@
+2019-06-06  Keith Miller  <keith_miller@apple.com>
+
+        Reenable Gigacage on ARM64.
+        https://bugs.webkit.org/show_bug.cgi?id=198453
+
+        Reviewed by Michael Saboff.
+
+        This patch adds back Gigacaging on Apple's ARM64 ports. Unlike the
+        old Gigacage however, arm64e uses both Gigacaging and PAC. In
+        order to ensure the PAC bits are not stripped in the caging
+        process we use the bit field insert instruction to take the low
+        bits from caging and the high bits from the PAC authentication.
+
+        * assembler/MacroAssemblerARM64.h:
+        (JSC::MacroAssemblerARM64::bitFieldInsert64):
+        * assembler/MacroAssemblerARM64E.h:
+        * assembler/testmasm.cpp:
+        (JSC::testCagePreservesPACFailureBit):
+        (JSC::run):
+        * dfg/DFGSpeculativeJIT.cpp:
+        (JSC::DFG::SpeculativeJIT::jumpForTypedArrayIsNeuteredIfOutOfBounds):
+        (JSC::DFG::SpeculativeJIT::cageTypedArrayStorage):
+        (JSC::DFG::SpeculativeJIT::compileGetTypedArrayByteOffset):
+        (JSC::DFG::SpeculativeJIT::compileNewTypedArrayWithSize):
+        * ftl/FTLLowerDFGToB3.cpp:
+        (JSC::FTL::DFG::LowerDFGToB3::compileNewTypedArray):
+        (JSC::FTL::DFG::LowerDFGToB3::caged):
+        * jit/AssemblyHelpers.h:
+        (JSC::AssemblyHelpers::cageWithoutUntaging):
+        (JSC::AssemblyHelpers::cageConditionally):
+        (JSC::AssemblyHelpers::cage): Deleted.
+        * jit/JITPropertyAccess.cpp:
+        (JSC::JIT::emitIntTypedArrayGetByVal):
+        (JSC::JIT::emitFloatTypedArrayGetByVal):
+        (JSC::JIT::emitIntTypedArrayPutByVal):
+        (JSC::JIT::emitFloatTypedArrayPutByVal):
+        * llint/LowLevelInterpreter.asm:
+        * llint/LowLevelInterpreter64.asm:
+        * offlineasm/arm64.rb:
+        * offlineasm/instructions.rb:
+        * offlineasm/registers.rb:
+        * wasm/WasmAirIRGenerator.cpp:
+        (JSC::Wasm::AirIRGenerator::restoreWebAssemblyGlobalState):
+        (JSC::Wasm::AirIRGenerator::addCallIndirect):
+        * wasm/WasmB3IRGenerator.cpp:
+        (JSC::Wasm::B3IRGenerator::restoreWebAssemblyGlobalState):
+        (JSC::Wasm::B3IRGenerator::addCallIndirect):
+        * wasm/WasmBinding.cpp:
+        (JSC::Wasm::wasmToWasm):
+        * wasm/js/JSToWasm.cpp:
+        (JSC::Wasm::createJSToWasmWrapper):
+        * wasm/js/WebAssemblyFunction.cpp:
+        (JSC::WebAssemblyFunction::jsCallEntrypointSlow):
+
 2019-06-06  Michael Saboff  <msaboff@apple.com>
 
         [ARM64E]: Add disassembler support for authenticated instructions
index 4efd6f9..3f4c16d 100644 (file)
@@ -2526,6 +2526,15 @@ public:
         m_assembler.fcsel<64>(dest, thenCase, elseCase, ARM64Condition(cond));
     }
 
+    // Bit field operations:
+
+    // destBitOffset is the top bit of the destination where the bits should be copied to. Zero is the lowest order bit.
+    void bitFieldInsert64(RegisterID source, unsigned destBitOffset, unsigned width, RegisterID dest)
+    {
+        ASSERT(width <= 64 - destBitOffset && destBitOffset < 64);
+        m_assembler.bfi<64>(dest, source, destBitOffset, width);
+    }
+
     // Forwards / external control flow operations:
     //
     // This set of jump and conditional branch operations return a Jump
index 0b968f6..fd7eec8 100644 (file)
@@ -39,6 +39,8 @@ using Assembler = TARGET_ASSEMBLER;
 
 class MacroAssemblerARM64E : public MacroAssemblerARM64 {
 public:
+    static constexpr unsigned numberOfPACBits = 25;
+
     ALWAYS_INLINE void tagReturnAddress()
     {
         tagPtr(ARM64Registers::sp, ARM64Registers::lr);
index cc25c94..21b347a 100644 (file)
@@ -39,6 +39,7 @@
 #include <wtf/Function.h>
 #include <wtf/Lock.h>
 #include <wtf/NumberOfCores.h>
+#include <wtf/PtrTag.h>
 #include <wtf/Threading.h>
 #include <wtf/text/StringCommon.h>
 
@@ -109,6 +110,15 @@ template<typename T> T nextID(T id) { return static_cast<T>(id + 1); }
         CRASH();                                                        \
     } while (false)
 
+#define CHECK_NOT_EQ(_actual, _expected) do {                               \
+        if ((_actual) != (_expected))                                   \
+            break;                                                      \
+        crashLock.lock();                                               \
+        dataLog("FAILED while testing " #_actual ": expected not: ", _expected, ", actual: ", _actual, "\n"); \
+        WTFReportAssertionFailure(__FILE__, __LINE__, WTF_PRETTY_FUNCTION, "CHECK_NOT_EQ("#_actual ", " #_expected ")"); \
+        CRASH();                                                        \
+    } while (false)
+
 #if ENABLE(MASM_PROBE)
 bool isPC(MacroAssembler::RegisterID id)
 {
@@ -1003,6 +1013,47 @@ void testMoveDoubleConditionally64()
 #endif
 }
 
+static void testCagePreservesPACFailureBit()
+{
+    auto cage = compile([] (CCallHelpers& jit) {
+        jit.emitFunctionPrologue();
+        jit.cageConditionally(Gigacage::Primitive, GPRInfo::argumentGPR0, GPRInfo::argumentGPR1, GPRInfo::argumentGPR2);
+        jit.move(GPRInfo::argumentGPR0, GPRInfo::returnValueGPR);
+        jit.emitFunctionEpilogue();
+        jit.ret();
+    });
+
+    void* ptr = Gigacage::tryMalloc(Gigacage::Primitive, 1);
+    void* taggedPtr = tagArrayPtr(ptr, 1);
+    dataLogLn("starting test");
+    if (isARM64E()) {
+        // FIXME: This won't work if authentication failures trap but I don't know how to test for that right now.
+        CHECK_NOT_EQ(invoke<void*>(cage, taggedPtr, 2), ptr);
+    } else
+        CHECK_EQ(invoke<void*>(cage, taggedPtr, 2), ptr);
+
+    CHECK_EQ(invoke<void*>(cage, taggedPtr, 1), ptr);
+
+    auto cageWithoutAuthentication = compile([] (CCallHelpers& jit) {
+        jit.emitFunctionPrologue();
+        jit.cageWithoutUntaging(Gigacage::Primitive, GPRInfo::argumentGPR0);
+        jit.move(GPRInfo::argumentGPR0, GPRInfo::returnValueGPR);
+        jit.emitFunctionEpilogue();
+        jit.ret();
+    });
+
+    if (isARM64E()) {
+        // FIXME: This won't work if authentication failures trap but I don't know how to test for that right now.
+        CHECK_NOT_EQ(invoke<void*>(cageWithoutAuthentication, untagArrayPtr(taggedPtr, 2)), ptr);
+    } else
+        CHECK_EQ(invoke<void*>(cageWithoutAuthentication, untagArrayPtr(taggedPtr, 2)), ptr);
+
+    CHECK_EQ(untagArrayPtr(taggedPtr, 1), ptr);
+    CHECK_EQ(invoke<void*>(cageWithoutAuthentication, untagArrayPtr(taggedPtr, 1)), ptr);
+
+    Gigacage::free(Gigacage::Primitive, ptr);
+}
+
 #define RUN(test) do {                          \
         if (!shouldRun(#test))                  \
             break;                              \
@@ -1088,6 +1139,8 @@ void run(const char* filter)
     RUN(testMoveDoubleConditionally32());
     RUN(testMoveDoubleConditionally64());
 
+    RUN(testCagePreservesPACFailureBit());
+
     if (tasks.isEmpty())
         usage();
 
index 7b96ec0..9558e55 100644 (file)
@@ -2873,7 +2873,7 @@ JITCompiler::Jump SpeculativeJIT::jumpForTypedArrayIsNeuteredIfOutOfBounds(Node*
                 TrustedImm32(WastefulTypedArray));
 
             JITCompiler::Jump hasNullVector;
-#if !GIGACAGE_ENABLED && CPU(ARM64E)
+#if CPU(ARM64E)
             {
                 GPRReg scratch = m_jit.scratchRegister();
                 DisallowMacroScratchRegisterUsage disallowScratch(m_jit);
@@ -2882,7 +2882,7 @@ JITCompiler::Jump SpeculativeJIT::jumpForTypedArrayIsNeuteredIfOutOfBounds(Node*
                 m_jit.removeArrayPtrTag(scratch);
                 hasNullVector = m_jit.branchTestPtr(MacroAssembler::Zero, scratch);
             }
-#else // !GIGACAGE_ENABLED && CPU(ARM64E)
+#else // CPU(ARM64E)
             hasNullVector = m_jit.branchTestPtr(
                 MacroAssembler::Zero,
                 MacroAssembler::Address(base, JSArrayBufferView::offsetOfVector()));
@@ -6760,6 +6760,13 @@ void SpeculativeJIT::compileConstantStoragePointer(Node* node)
 
 void SpeculativeJIT::cageTypedArrayStorage(GPRReg baseReg, GPRReg storageReg)
 {
+#if CPU(ARM64E)
+    m_jit.untagArrayPtr(MacroAssembler::Address(baseReg, JSArrayBufferView::offsetOfLength()), storageReg);
+#else
+    UNUSED_PARAM(baseReg);
+    UNUSED_PARAM(storageReg);
+#endif
+
 #if GIGACAGE_ENABLED
     UNUSED_PARAM(baseReg);
     if (!Gigacage::shouldBeEnabled())
@@ -6772,12 +6779,7 @@ void SpeculativeJIT::cageTypedArrayStorage(GPRReg baseReg, GPRReg storageReg)
             return;
     }
     
-    m_jit.cage(Gigacage::Primitive, storageReg);
-#elif CPU(ARM64E)
-    m_jit.untagArrayPtr(MacroAssembler::Address(baseReg, JSArrayBufferView::offsetOfLength()), storageReg);
-#else
-    UNUSED_PARAM(baseReg);
-    UNUSED_PARAM(storageReg);
+    m_jit.cageWithoutUntaging(Gigacage::Primitive, storageReg);
 #endif
 }
 
@@ -6841,7 +6843,7 @@ void SpeculativeJIT::compileGetTypedArrayByteOffset(Node* node)
     JITCompiler::Jump nullVector = m_jit.branchTestPtr(JITCompiler::Zero, vectorGPR);
 
     m_jit.loadPtr(MacroAssembler::Address(baseGPR, JSObject::butterflyOffset()), dataGPR);
-    m_jit.cage(Gigacage::JSValue, dataGPR);
+    m_jit.cageWithoutUntaging(Gigacage::JSValue, dataGPR);
 
     cageTypedArrayStorage(baseGPR, vectorGPR);
 
@@ -9845,7 +9847,7 @@ void SpeculativeJIT::compileNewTypedArrayWithSize(Node* node)
         MacroAssembler::BaseIndex(storageGPR, scratchGPR, MacroAssembler::TimesFour));
     m_jit.branchTest32(MacroAssembler::NonZero, scratchGPR).linkTo(loop, &m_jit);
     done.link(&m_jit);
-#if !GIGACAGE_ENABLED && CPU(ARM64E)
+#if CPU(ARM64E)
     // sizeGPR is still boxed as a number and there is no 32-bit variant of the PAC instructions.
     m_jit.zeroExtend32ToPtr(sizeGPR, scratchGPR);
     m_jit.tagArrayPtr(scratchGPR, storageGPR);
index d72af41..92197d8 100644 (file)
@@ -6485,7 +6485,7 @@ private:
                 m_out.int64Zero,
                 m_heaps.typedArrayProperties);
 
-#if !GIGACAGE_ENABLED && CPU(ARM64E)
+#if CPU(ARM64E)
             {
                 LValue sizePtr = m_out.zeroExtPtr(size);
                 PatchpointValue* authenticate = m_out.patchpoint(pointerType());
@@ -14157,6 +14157,16 @@ private:
 
     LValue caged(Gigacage::Kind kind, LValue ptr, LValue base)
     {
+#if CPU(ARM64E)
+        if (kind == Gigacage::Primitive) {
+            LValue size = m_out.load32(base, m_heaps.JSArrayBufferView_length);
+            ptr = untagArrayPtr(ptr, size);
+        }
+#else
+        UNUSED_PARAM(kind);
+        UNUSED_PARAM(base);
+#endif
+
 #if GIGACAGE_ENABLED
         UNUSED_PARAM(base);
         if (!Gigacage::isEnabled(kind))
@@ -14175,6 +14185,18 @@ private:
         LValue masked = m_out.bitAnd(ptr, mask);
         LValue result = m_out.add(masked, basePtr);
 
+#if CPU(ARM64E)
+        {
+            PatchpointValue* merge = m_out.patchpoint(pointerType());
+            merge->append(result, B3::ValueRep(B3::ValueRep::SomeLateRegister));
+            merge->appendSomeRegister(ptr);
+            merge->setGenerator([=] (CCallHelpers& jit, const StackmapGenerationParams& params) {
+                jit.move(params[2].gpr(), params[0].gpr());
+                jit.bitFieldInsert64(params[1].gpr(), 0, 64 - MacroAssembler::numberOfPACBits, params[0].gpr());
+            });
+            result = merge;
+        }
+#endif
         // Make sure that B3 doesn't try to do smart reassociation of these pointer bits.
         // FIXME: In an ideal world, B3 would not do harmful reassociations, and if it did, it would be able
         // to undo them during constant hoisting and regalloc. As it stands, if you remove this then Octane
@@ -14187,17 +14209,6 @@ private:
         // and possibly other smart things if we want to be able to remove this opaque.
         // https://bugs.webkit.org/show_bug.cgi?id=175493
         return m_out.opaque(result);
-#elif CPU(ARM64E)
-        if (kind == Gigacage::Primitive) {
-            LValue size = m_out.load32(base, m_heaps.JSArrayBufferView_length);
-            return untagArrayPtr(ptr, size);
-        }
-
-        return ptr;
-#else
-        UNUSED_PARAM(kind);
-        UNUSED_PARAM(base);
-        return ptr;
 #endif
     }
     
index d9268e8..5d3fcc1 100644 (file)
@@ -1554,43 +1554,74 @@ public:
         storeFence();
         ok.link(this);
     }
-    
-    void cage(Gigacage::Kind kind, GPRReg storage)
+
+    void cageWithoutUntaging(Gigacage::Kind kind, GPRReg storage)
     {
 #if GIGACAGE_ENABLED
         if (!Gigacage::isEnabled(kind))
             return;
-        
+
+#if CPU(ARM64E)
+        RegisterID tempReg = InvalidGPRReg;
+        if (kind == Gigacage::Primitive) {
+            tempReg = getCachedMemoryTempRegisterIDAndInvalidate();
+            move(storage, tempReg);
+            // Flip the registers since bitFieldInsert only inserts into the low bits.
+            std::swap(storage, tempReg);
+        }
+#endif
         andPtr(TrustedImmPtr(Gigacage::mask(kind)), storage);
         addPtr(TrustedImmPtr(Gigacage::basePtr(kind)), storage);
+#if CPU(ARM64E)
+        if (kind == Gigacage::Primitive)
+            bitFieldInsert64(storage, 0, 64 - numberOfPACBits, tempReg);
+#endif
+
 #else
         UNUSED_PARAM(kind);
         UNUSED_PARAM(storage);
 #endif
     }
-    
-    void cageConditionally(Gigacage::Kind kind, GPRReg storage, GPRReg scratchOrLength)
+
+    // length may be the same register as scratch.
+    void cageConditionally(Gigacage::Kind kind, GPRReg storage, GPRReg length, GPRReg scratch)
     {
+#if CPU(ARM64E)
+        if (kind == Gigacage::Primitive)
+            untagArrayPtr(length, storage);
+#else
+        UNUSED_PARAM(kind);
+        UNUSED_PARAM(storage);
+        UNUSED_PARAM(length);
+#endif
+
 #if GIGACAGE_ENABLED
         if (!Gigacage::isEnabled(kind))
             return;
         
         if (kind != Gigacage::Primitive || Gigacage::isDisablingPrimitiveGigacageDisabled())
-            return cage(kind, storage);
-        
-        loadPtr(&Gigacage::basePtr(kind), scratchOrLength);
-        Jump done = branchTestPtr(Zero, scratchOrLength);
-        andPtr(TrustedImmPtr(Gigacage::mask(kind)), storage);
-        addPtr(scratchOrLength, storage);
-        done.link(this);
-#elif CPU(ARM64E)
-        if (kind == Gigacage::Primitive)
-            untagArrayPtr(scratchOrLength, storage);
+            cageWithoutUntaging(kind, storage);
+        else {
+            loadPtr(&Gigacage::basePtr(kind), scratch);
+            Jump done = branchTestPtr(Zero, scratch);
+#if CPU(ARM64E)
+            auto tempReg = getCachedMemoryTempRegisterIDAndInvalidate();
+            move(storage, tempReg);
+            andPtr(TrustedImmPtr(Gigacage::mask(kind)), tempReg);
+            addPtr(scratch, tempReg);
+            bitFieldInsert64(tempReg, 0, 64 - numberOfPACBits, storage);
 #else
-        UNUSED_PARAM(kind);
-        UNUSED_PARAM(storage);
-        UNUSED_PARAM(scratchOrLength);
+            andPtr(TrustedImmPtr(Gigacage::mask(kind)), storage);
+            addPtr(scratch, storage);
 #endif
+            done.link(this);
+
+
+        }
+#else
+        UNUSED_PARAM(scratch);
+#endif
+
     }
 
     void emitComputeButterflyIndexingMask(GPRReg vectorLengthGPR, GPRReg scratchGPR, GPRReg resultGPR)
index 9d62148..cbb78d5 100644 (file)
@@ -1672,7 +1672,7 @@ JIT::JumpList JIT::emitIntTypedArrayGetByVal(const Instruction*, PatchableJump&
     load32(Address(base, JSArrayBufferView::offsetOfLength()), scratch2);
     slowCases.append(branch32(AboveOrEqual, property, scratch2));
     loadPtr(Address(base, JSArrayBufferView::offsetOfVector()), scratch);
-    cageConditionally(Gigacage::Primitive, scratch, scratch2);
+    cageConditionally(Gigacage::Primitive, scratch, scratch2, scratch2);
 
     switch (elementSize(type)) {
     case 1:
@@ -1736,7 +1736,7 @@ JIT::JumpList JIT::emitFloatTypedArrayGetByVal(const Instruction*, PatchableJump
     load32(Address(base, JSArrayBufferView::offsetOfLength()), scratch2);
     slowCases.append(branch32(AboveOrEqual, property, scratch2));
     loadPtr(Address(base, JSArrayBufferView::offsetOfVector()), scratch);
-    cageConditionally(Gigacage::Primitive, scratch, scratch2);
+    cageConditionally(Gigacage::Primitive, scratch, scratch2, scratch2);
     
     switch (elementSize(type)) {
     case 4:
@@ -1801,7 +1801,7 @@ JIT::JumpList JIT::emitIntTypedArrayPutByVal(Op bytecode, PatchableJump& badType
     // We would be loading this into base as in get_by_val, except that the slow
     // path expects the base to be unclobbered.
     loadPtr(Address(base, JSArrayBufferView::offsetOfVector()), lateScratch);
-    cageConditionally(Gigacage::Primitive, lateScratch, lateScratch2);
+    cageConditionally(Gigacage::Primitive, lateScratch, lateScratch2, lateScratch2);
     
     if (isClamped(type)) {
         ASSERT(elementSize(type) == 1);
@@ -1890,7 +1890,7 @@ JIT::JumpList JIT::emitFloatTypedArrayPutByVal(Op bytecode, PatchableJump& badTy
     // We would be loading this into base as in get_by_val, except that the slow
     // path expects the base to be unclobbered.
     loadPtr(Address(base, JSArrayBufferView::offsetOfVector()), lateScratch);
-    cageConditionally(Gigacage::Primitive, lateScratch, lateScratch2);
+    cageConditionally(Gigacage::Primitive, lateScratch, lateScratch2, lateScratch2);
     
     switch (elementSize(type)) {
     case 4:
index 4032e7e..c0ad83c 100644 (file)
@@ -76,7 +76,7 @@
 #
 #  - pc holds the (native) program counter on 32-bits ARM architectures (ARMv7)
 #
-#  - t0, t1, t2, t3, t4 and optionally t5 are temporary registers that can get trashed on
+#  - t0, t1, t2, t3, t4, and optionally t5, t6, and t7 are temporary registers that can get trashed on
 #  calls, and are pairwise distinct registers. t4 holds the JS program counter, so use
 #  with caution in opcodes (actually, don't use it in opcodes at all, except as PC).
 #
index 6aaf0dd..903635a 100644 (file)
@@ -434,10 +434,19 @@ end
 
 macro loadCagedPrimitive(source, dest, scratchOrLength)
     loadp source, dest
-    if GIGACAGE_ENABLED
-        uncage(_g_gigacageBasePtrs + Gigacage::BasePtrs::primitive, constexpr Gigacage::primitiveGigacageMask, dest, scratchOrLength)
-    elsif ARM64E
+    if ARM64E
+        const result = t7
         untagArrayPtr scratchOrLength, dest
+        move dest, result
+    else
+        const result = dest
+    end
+    if GIGACAGE_ENABLED
+        uncage(_g_gigacageBasePtrs + Gigacage::BasePtrs::primitive, constexpr Gigacage::primitiveGigacageMask, result, scratchOrLength)
+        if ARM64E
+            const numberOfPACBits = constexpr MacroAssembler::numberOfPACBits
+            bfiq result, 0, 64 - numberOfPACBits, dest
+        end
     end
 end
 
index 58bbc2a..23d948f 100644 (file)
@@ -126,6 +126,8 @@ class RegisterID
           arm64GPRName('x5', kind)
         when 't6'
           arm64GPRName('x6', kind)
+        when 't7'
+          arm64GPRName('x7', kind)
         when 'cfr'
             arm64GPRName('x29', kind)
         when 'csr0'
@@ -1019,6 +1021,8 @@ class Instruction
             $asm.puts "smaddl #{operands[2].arm64Operand(:quad)}, #{operands[0].arm64Operand(:word)}, #{operands[1].arm64Operand(:word)}, xzr"
         when "memfence"
             $asm.puts "dmb sy"
+        when "bfiq"
+            $asm.puts "bfi #{operands[3].arm64Operand(:quad)}, #{operands[0].arm64Operand(:quad)}, #{operands[1].value}, #{operands[2].value}"
         when "pcrtoaddr"
             $asm.puts "adr #{operands[1].arm64Operand(:quad)}, #{operands[0].value}"
         when "nopCortexA53Fix835769"
index 2ad5944..71788b1 100644 (file)
@@ -273,6 +273,7 @@ ARM_INSTRUCTIONS =
 
 ARM64_INSTRUCTIONS =
     [
+     "bfiq", # Bit field insert <source reg> <last bit written> <width immediate> <dest reg>
      "pcrtoaddr",   # Address from PC relative offset - adr instruction
      "nopFixCortexA53Err835769", # nop on Cortex-A53 (nothing otherwise)
      "globaladdr"
index aa8a40f..158d141 100644 (file)
@@ -32,6 +32,7 @@ GPRS =
      "t4",
      "t5",
      "t6",
+     "t7",
      "cfr",
      "a0",
      "a1",
index 57b3df8..adb33f5 100644 (file)
@@ -848,15 +848,14 @@ void AirIRGenerator::restoreWebAssemblyGlobalState(RestoreCachedStackLimit resto
         patchpoint->numGPScratchRegisters = Gigacage::isEnabled(Gigacage::Primitive) ? 1 : 0;
 
         patchpoint->setGenerator([pinnedRegs] (CCallHelpers& jit, const B3::StackmapGenerationParams& params) {
-            RELEASE_ASSERT(!Gigacage::isEnabled(Gigacage::Primitive) || !isARM64());
-            AllowMacroScratchRegisterUsageIf allowScratch(jit, !isARM64());
+            AllowMacroScratchRegisterUsage allowScratch(jit);
             GPRReg baseMemory = pinnedRegs->baseMemoryPointer;
             GPRReg scratchOrSize = Gigacage::isEnabled(Gigacage::Primitive) ? params.gpScratch(0) : pinnedRegs->sizeRegister;
 
             jit.loadPtr(CCallHelpers::Address(params[0].gpr(), Instance::offsetOfCachedMemorySize()), pinnedRegs->sizeRegister);
             jit.loadPtr(CCallHelpers::Address(params[0].gpr(), Instance::offsetOfCachedMemory()), baseMemory);
 
-            jit.cageConditionally(Gigacage::Primitive, baseMemory, scratchOrSize);
+            jit.cageConditionally(Gigacage::Primitive, baseMemory, pinnedRegs->sizeRegister, scratchOrSize);
         });
 
         emitPatchpoint(block, patchpoint, Tmp(), instance);
@@ -1988,7 +1987,7 @@ auto AirIRGenerator::addCallIndirect(const Signature& signature, Vector<Expressi
             jit.loadPtr(CCallHelpers::Address(newContextInstance, Instance::offsetOfCachedMemorySize()), pinnedRegs.sizeRegister); // Memory size.
             jit.loadPtr(CCallHelpers::Address(newContextInstance, Instance::offsetOfCachedMemory()), baseMemory); // Memory::void*.
 
-            jit.cageConditionally(Gigacage::Primitive, baseMemory, scratchOrSize);
+            jit.cageConditionally(Gigacage::Primitive, baseMemory, pinnedRegs.sizeRegister, scratchOrSize);
         });
 
         emitPatchpoint(doContextSwitch, patchpoint, Tmp(), newContextInstance, instanceValue());
index 2226ffc..55bd2d8 100644 (file)
@@ -491,15 +491,14 @@ void B3IRGenerator::restoreWebAssemblyGlobalState(RestoreCachedStackLimit restor
 
         patchpoint->append(instance, ValueRep::SomeRegister);
         patchpoint->setGenerator([pinnedRegs] (CCallHelpers& jit, const B3::StackmapGenerationParams& params) {
-            RELEASE_ASSERT(!Gigacage::isEnabled(Gigacage::Primitive) || !isARM64());
-            AllowMacroScratchRegisterUsageIf allowScratch(jit, !isARM64());
+            AllowMacroScratchRegisterUsage allowScratch(jit);
             GPRReg baseMemory = pinnedRegs->baseMemoryPointer;
             GPRReg scratchOrSize = Gigacage::isEnabled(Gigacage::Primitive) ? params.gpScratch(0) : pinnedRegs->sizeRegister;
 
             jit.loadPtr(CCallHelpers::Address(params[0].gpr(), Instance::offsetOfCachedMemorySize()), pinnedRegs->sizeRegister);
             jit.loadPtr(CCallHelpers::Address(params[0].gpr(), Instance::offsetOfCachedMemory()), baseMemory);
 
-            jit.cageConditionally(Gigacage::Primitive, baseMemory, scratchOrSize);
+            jit.cageConditionally(Gigacage::Primitive, baseMemory, pinnedRegs->sizeRegister, scratchOrSize);
         });
     }
 }
@@ -1401,7 +1400,7 @@ auto B3IRGenerator::addCallIndirect(const Signature& signature, Vector<Expressio
             jit.loadPtr(CCallHelpers::Address(newContextInstance, Instance::offsetOfCachedMemorySize()), pinnedRegs.sizeRegister); // Memory size.
             jit.loadPtr(CCallHelpers::Address(newContextInstance, Instance::offsetOfCachedMemory()), baseMemory); // Memory::void*.
 
-            jit.cageConditionally(Gigacage::Primitive, baseMemory, scratchOrSize);
+            jit.cageConditionally(Gigacage::Primitive, baseMemory, pinnedRegs.sizeRegister, scratchOrSize);
         });
         doContextSwitch->appendNewControlValue(m_proc, Jump, origin(), continuation);
 
index fed45b1..23c9647 100644 (file)
@@ -67,11 +67,11 @@ Expected<MacroAssemblerCodeRef<WasmEntryPtrTag>, BindingFailure> wasmToWasm(unsi
     // FIXME the following code assumes that all Wasm::Instance have the same pinned registers. https://bugs.webkit.org/show_bug.cgi?id=162952
     // Set up the callee's baseMemory register as well as the memory size registers.
     {
-        GPRReg scratchOrSize = isARM64E() ? pinnedRegs.sizeRegister : wasmCallingConventionAir().prologueScratch(1);
+        GPRReg scratchOrSize = !Gigacage::isEnabled(Gigacage::Primitive) ? pinnedRegs.sizeRegister : wasmCallingConventionAir().prologueScratch(1);
 
         jit.loadPtr(JIT::Address(baseMemory, Wasm::Instance::offsetOfCachedMemorySize()), pinnedRegs.sizeRegister); // Memory size.
         jit.loadPtr(JIT::Address(baseMemory, Wasm::Instance::offsetOfCachedMemory()), baseMemory); // Wasm::Memory::TaggedArrayStoragePtr<void> (void*).
-        jit.cageConditionally(Gigacage::Primitive, baseMemory, scratchOrSize);
+        jit.cageConditionally(Gigacage::Primitive, baseMemory, pinnedRegs.sizeRegister, scratchOrSize);
     }
 
     // Tail call into the callee WebAssembly function.
index 657da57..26313e8 100644 (file)
@@ -228,7 +228,7 @@ std::unique_ptr<InternalFunction> createJSToWasmWrapper(CompilationContext& comp
         }
 
         jit.loadPtr(CCallHelpers::Address(currentInstanceGPR, Wasm::Instance::offsetOfCachedMemory()), baseMemory);
-        jit.cageConditionally(Gigacage::Primitive, baseMemory, scratchOrSize);
+        jit.cageConditionally(Gigacage::Primitive, baseMemory, scratchOrSize, scratchOrSize);
     }
 
     CCallHelpers::Call call = jit.threadSafePatchableNearCall();
index 2ff0e78..91b5b83 100644 (file)
@@ -423,7 +423,7 @@ MacroAssemblerCodePtr<JSEntryPtrTag> WebAssemblyFunction::jsCallEntrypointSlow()
         }
 
         jit.loadPtr(CCallHelpers::Address(scratchGPR, Wasm::Instance::offsetOfCachedMemory()), baseMemory);
-        jit.cageConditionally(Gigacage::Primitive, baseMemory, scratchOrSize);
+        jit.cageConditionally(Gigacage::Primitive, baseMemory, scratchOrSize, scratchOrSize);
     }
 
     // We use this callee to indicate how to unwind past these types of frames:
index a7e8000..9f0491f 100644 (file)
@@ -1,3 +1,12 @@
+2019-06-06  Keith Miller  <keith_miller@apple.com>
+
+        Reenable Gigacage on ARM64.
+        https://bugs.webkit.org/show_bug.cgi?id=198453
+
+        Reviewed by Michael Saboff.
+
+        * bmalloc/Gigacage.h:
+
 2019-06-03  Commit Queue  <commit-queue@webkit.org>
 
         Unreviewed, rolling out r246022.
index 76d72df..a37d0ba 100644 (file)
@@ -34,7 +34,8 @@
 #include <cstddef>
 #include <inttypes.h>
 
-#if ((BOS(DARWIN) || BOS(LINUX)) && BCPU(X86_64))
+#if ((BOS(DARWIN) || BOS(LINUX)) && \
+    (BCPU(X86_64) || (BCPU(ARM64) && !defined(__ILP32__) && (!BPLATFORM(IOS_FAMILY) || BPLATFORM(IOS)))))
 #define GIGACAGE_ENABLED 1
 #else
 #define GIGACAGE_ENABLED 0