Enable gigacage on iOS
authorfpizlo@apple.com <fpizlo@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Sat, 7 Oct 2017 02:29:19 +0000 (02:29 +0000)
committerfpizlo@apple.com <fpizlo@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Sat, 7 Oct 2017 02:29:19 +0000 (02:29 +0000)
https://bugs.webkit.org/show_bug.cgi?id=177586

Reviewed by JF Bastien.
JSTests:

Add tests for when Gigacage gets runtime disabled.

* stress/disable-gigacage-arrays.js: Added.
(foo):
* stress/disable-gigacage-strings.js: Added.
(foo):
* stress/disable-gigacage-typed-arrays.js: Added.
(foo):

Source/bmalloc:

Introduce the ability to disable gigacage at runtime if allocation fails. If any step of gigacage
allocation fails, we free all of the gigacages and turn off gigacage support.

* CMakeLists.txt:
* bmalloc.xcodeproj/project.pbxproj:
* bmalloc/Cache.cpp:
(bmalloc::Cache::scavenge):
* bmalloc/Cache.h:
(bmalloc::Cache::tryAllocate):
(bmalloc::Cache::allocate):
(bmalloc::Cache::deallocate):
(bmalloc::Cache::reallocate):
* bmalloc/Gigacage.cpp:
(Gigacage::ensureGigacage):
(Gigacage::runway):
(Gigacage::totalSize):
(Gigacage::shouldBeEnabled):
(): Deleted.
(Gigacage::Callback::Callback): Deleted.
(Gigacage::Callback::function): Deleted.
(Gigacage::PrimitiveDisableCallbacks::PrimitiveDisableCallbacks): Deleted.
* bmalloc/Gigacage.h:
(Gigacage::wasEnabled):
(Gigacage::isEnabled):
(Gigacage::runway): Deleted.
(Gigacage::totalSize): Deleted.
* bmalloc/HeapKind.cpp: Added.
(bmalloc::isActiveHeapKind):
(bmalloc::mapToActiveHeapKind):
* bmalloc/HeapKind.h:
(bmalloc::isActiveHeapKindAfterEnsuringGigacage):
(bmalloc::mapToActiveHeapKindAfterEnsuringGigacage):
* bmalloc/Scavenger.cpp:
(bmalloc::Scavenger::scavenge):
* bmalloc/bmalloc.h:
(bmalloc::api::tryLargeMemalignVirtual):
(bmalloc::api::freeLargeVirtual):
(bmalloc::api::isEnabled):

Source/JavaScriptCore:

The hardest part of enabling Gigacage on iOS is that it requires loading global variables while
executing JS, so the LLInt needs to know how to load from global variables on all platforms that
have Gigacage. So, this teaches ARM64 how to load from global variables.

Also, this makes the code handle disabling the gigacage a bit better.

* ftl/FTLLowerDFGToB3.cpp:
(JSC::FTL::DFG::LowerDFGToB3::caged):
* jit/AssemblyHelpers.h:
(JSC::AssemblyHelpers::cage):
(JSC::AssemblyHelpers::cageConditionally):
* offlineasm/arm64.rb:
* offlineasm/asm.rb:
* offlineasm/instructions.rb:

Tools:

Add a mode to test disabling Gigacage.

* Scripts/run-jsc-stress-tests:
* Scripts/webkitruby/jsc-stress-test-writer-default.rb:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@223015 268f45cc-cd09-0410-ab3c-d52691b4dbfc

25 files changed:
JSTests/ChangeLog
JSTests/stress/disable-gigacage-arrays.js [new file with mode: 0644]
JSTests/stress/disable-gigacage-strings.js [new file with mode: 0644]
JSTests/stress/disable-gigacage-typed-arrays.js [new file with mode: 0644]
Source/JavaScriptCore/ChangeLog
Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp
Source/JavaScriptCore/jit/AssemblyHelpers.h
Source/JavaScriptCore/offlineasm/arm64.rb
Source/JavaScriptCore/offlineasm/asm.rb
Source/JavaScriptCore/offlineasm/instructions.rb
Source/bmalloc/CMakeLists.txt
Source/bmalloc/ChangeLog
Source/bmalloc/bmalloc.xcodeproj/project.pbxproj
Source/bmalloc/bmalloc/Cache.cpp
Source/bmalloc/bmalloc/Cache.h
Source/bmalloc/bmalloc/Gigacage.cpp
Source/bmalloc/bmalloc/Gigacage.h
Source/bmalloc/bmalloc/Heap.cpp
Source/bmalloc/bmalloc/HeapKind.cpp [new file with mode: 0644]
Source/bmalloc/bmalloc/HeapKind.h
Source/bmalloc/bmalloc/Scavenger.cpp
Source/bmalloc/bmalloc/bmalloc.h
Tools/ChangeLog
Tools/Scripts/run-jsc-stress-tests
Tools/Scripts/webkitruby/jsc-stress-test-writer-default.rb

index 435cdf6..89f10a5 100644 (file)
@@ -1,3 +1,19 @@
+2017-09-29  Filip Pizlo  <fpizlo@apple.com>
+
+        Enable gigacage on iOS
+        https://bugs.webkit.org/show_bug.cgi?id=177586
+
+        Reviewed by JF Bastien.
+        
+        Add tests for when Gigacage gets runtime disabled.
+
+        * stress/disable-gigacage-arrays.js: Added.
+        (foo):
+        * stress/disable-gigacage-strings.js: Added.
+        (foo):
+        * stress/disable-gigacage-typed-arrays.js: Added.
+        (foo):
+
 2017-10-06  Commit Queue  <commit-queue@webkit.org>
 
         Unreviewed, rolling out r222791 and r222873.
diff --git a/JSTests/stress/disable-gigacage-arrays.js b/JSTests/stress/disable-gigacage-arrays.js
new file mode 100644 (file)
index 0000000..212fb1e
--- /dev/null
@@ -0,0 +1,24 @@
+//@ runNoisyTestWithEnv "disable-gigacage", "GIGACAGE_ENABLED=0"
+
+(function() {
+    function foo(array, i)
+    {
+        return array[i];
+    }
+    
+    noInline(foo);
+    
+    var array = new Array(1000);
+    for (var i = 0; i < array.length; ++i)
+        array[i] = 5 - i;
+    for (var i = 0; i < 1000; ++i) {
+        var result = 0;
+        var expectedResult = 0;
+        for (var j = 0; j < array.length; ++j) {
+            result += foo(array, j);
+            expectedResult += 5 - j;
+        }
+        if (result != expectedResult)
+            throw new Error("Bad result: " + result);
+    }
+})();
diff --git a/JSTests/stress/disable-gigacage-strings.js b/JSTests/stress/disable-gigacage-strings.js
new file mode 100644 (file)
index 0000000..761d067
--- /dev/null
@@ -0,0 +1,25 @@
+//@ runNoisyTestWithEnv "disable-gigacage", "GIGACAGE_ENABLED=0"
+
+(function() {
+    function foo(array, i)
+    {
+        return array.charCodeAt(i);
+    }
+    
+    noInline(foo);
+    
+    var array = "";
+    for (var i = 0; i < array.length; ++i)
+        array += String.fromCharCode(5 - i);
+    for (var i = 0; i < 1000; ++i) {
+        var result = 0;
+        var expectedResult = 0;
+        for (var j = 0; j < array.length; ++j) {
+            result += foo(array, j);
+            expectedResult += 5 - j;
+        }
+        if (result != expectedResult)
+            throw new Error("Bad result: " + result);
+    }
+})();
+
diff --git a/JSTests/stress/disable-gigacage-typed-arrays.js b/JSTests/stress/disable-gigacage-typed-arrays.js
new file mode 100644 (file)
index 0000000..3663c1e
--- /dev/null
@@ -0,0 +1,25 @@
+//@ runNoisyTestWithEnv "disable-gigacage", "GIGACAGE_ENABLED=0"
+
+(function() {
+    function foo(array, i)
+    {
+        return array[i];
+    }
+    
+    noInline(foo);
+    
+    var array = new Int32Array(1000);
+    for (var i = 0; i < array.length; ++i)
+        array[i] = 5 - i;
+    for (var i = 0; i < 1000; ++i) {
+        var result = 0;
+        var expectedResult = 0;
+        for (var j = 0; j < array.length; ++j) {
+            result += foo(array, j);
+            expectedResult += 5 - j;
+        }
+        if (result != expectedResult)
+            throw new Error("Bad result: " + result);
+    }
+})();
+
index 6f00e14..4b2f902 100644 (file)
@@ -1,3 +1,25 @@
+2017-09-29  Filip Pizlo  <fpizlo@apple.com>
+
+        Enable gigacage on iOS
+        https://bugs.webkit.org/show_bug.cgi?id=177586
+
+        Reviewed by JF Bastien.
+
+        The hardest part of enabling Gigacage on iOS is that it requires loading global variables while
+        executing JS, so the LLInt needs to know how to load from global variables on all platforms that
+        have Gigacage. So, this teaches ARM64 how to load from global variables.
+        
+        Also, this makes the code handle disabling the gigacage a bit better.
+
+        * ftl/FTLLowerDFGToB3.cpp:
+        (JSC::FTL::DFG::LowerDFGToB3::caged):
+        * jit/AssemblyHelpers.h:
+        (JSC::AssemblyHelpers::cage):
+        (JSC::AssemblyHelpers::cageConditionally):
+        * offlineasm/arm64.rb:
+        * offlineasm/asm.rb:
+        * offlineasm/instructions.rb:
+
 2017-10-06  Michael Saboff  <msaboff@apple.com>
 
         Enable RegExp JIT for match only Unicode RegExp's
         * runtime/DatePrototype.cpp:
         (JSC::fillStructuresUsingDateArgs):
 
-2017-09-28  Filip Pizlo  <fpizlo@apple.com>
-
-        Enable gigacage on iOS
-        https://bugs.webkit.org/show_bug.cgi?id=177586
-
-        Reviewed by Michael Saboff.
-        
-        The hardest part of enabling Gigacage on iOS is that it requires loading global variables whil
-        executing JS, so the LLInt needs to know how to load from global variables on all platforms that
-        have Gigacage. So, this teaches ARM64 how to load from global variables.
-
-        * offlineasm/arm64.rb:
-        * offlineasm/asm.rb:
-        * offlineasm/instructions.rb:
-
 2017-09-28  Mark Lam  <mark.lam@apple.com>
 
         Add missing exception checks and book-keeping for exception check validation.
index 4a03280..0c187ad 100644 (file)
@@ -11994,7 +11994,7 @@ private:
     
     LValue caged(Gigacage::Kind kind, LValue ptr)
     {
-        if (!Gigacage::shouldBeEnabled())
+        if (!Gigacage::isEnabled(kind))
             return ptr;
         
         if (kind == Gigacage::Primitive && Gigacage::canPrimitiveGigacageBeDisabled()) {
index a68c369..5a7f6b0 100644 (file)
@@ -1314,7 +1314,7 @@ public:
     void cage(Gigacage::Kind kind, GPRReg storage)
     {
 #if GIGACAGE_ENABLED
-        if (!Gigacage::shouldBeEnabled())
+        if (!Gigacage::isEnabled(kind))
             return;
         
         andPtr(TrustedImmPtr(Gigacage::mask(kind)), storage);
@@ -1328,7 +1328,7 @@ public:
     void cageConditionally(Gigacage::Kind kind, GPRReg storage, GPRReg scratch)
     {
 #if GIGACAGE_ENABLED
-        if (!Gigacage::shouldBeEnabled())
+        if (!Gigacage::isEnabled(kind))
             return;
         
         if (kind != Gigacage::Primitive || Gigacage::isDisablingPrimitiveGigacageDisabled())
index ead4891..e806f99 100644 (file)
@@ -260,6 +260,31 @@ def arm64LowerMalformedLoadStoreAddresses(list)
     newList
 end
 
+def arm64LowerLabelReferences(list)
+    newList = []
+    list.each {
+        | node |
+        if node.is_a? Instruction
+            case node.opcode
+            when "loadi", "loadis", "loadp", "loadq", "loadb", "loadbs", "loadh", "loadhs"
+                labelRef = node.operands[0]
+                if labelRef.is_a? LabelReference
+                    tmp = Tmp.new(node.codeOrigin, :gpr)
+                    newList << Instruction.new(codeOrigin, "globaladdr", [LabelReference.new(node.codeOrigin, labelRef.label), tmp])
+                    newList << Instruction.new(codeOrigin, node.opcode, [Address.new(node.codeOrigin, tmp, Immediate.new(node.codeOrigin, labelRef.offset)), node.operands[1]])
+                else
+                    newList << node
+                end
+            else
+                newList << node
+            end
+        else
+            newList << node
+        end
+    }
+    newList
+end
+
 # Workaround for Cortex-A53 erratum (835769)
 def arm64CortexA53Fix835769(list)
     newList = []
@@ -296,6 +321,7 @@ class Sequence
         result = riscLowerHardBranchOps64(result)
         result = riscLowerShiftOps(result)
         result = arm64LowerMalformedLoadStoreAddresses(result)
+        result = arm64LowerLabelReferences(result)
         result = riscLowerMalformedAddresses(result) {
             | node, address |
             case node.opcode
@@ -904,6 +930,15 @@ class Instruction
             $asm.putStr("#if CPU(ARM64_CORTEXA53)")
             $asm.puts "nop"
             $asm.putStr("#endif")
+        when "globaladdr"
+            uid = $asm.newUID
+            $asm.puts "L_offlineasm_loh_adrp_#{uid}:"
+            $asm.puts "adrp #{operands[1].arm64Operand(:ptr)}, #{operands[0].asmLabel}@GOTPAGE"
+            $asm.puts "L_offlineasm_loh_ldr_#{uid}:"
+            $asm.puts "ldr #{operands[1].arm64Operand(:ptr)}, [#{operands[1].arm64Operand(:ptr)}, #{operands[0].asmLabel}@GOTPAGEOFF]"
+            $asm.deferAction {
+                $asm.puts ".loh AdrpLdrGot L_offlineasm_loh_adrp_#{uid}, L_offlineasm_loh_ldr_#{uid}"
+            }
         else
             lowerDefault
         end
index 8d76565..024049b 100644 (file)
@@ -46,6 +46,8 @@ class Assembler
         @codeOrigin = nil
         @numLocalLabels = 0
         @numGlobalLabels = 0
+        @deferredActions = []
+        @count = 0
 
         @newlineSpacerState = :none
         @lastlabel = ""
@@ -73,10 +75,23 @@ class Assembler
             putsProcEndIfNeeded
         end
         putsLastComment
+        @deferredActions.each {
+            | action |
+            action.call()
+        }
         @outp.puts "OFFLINE_ASM_END" if !$emitWinAsm
         @state = :cpp
     end
     
+    def deferAction(&proc)
+        @deferredActions << proc
+    end
+    
+    def newUID
+        @count += 1
+        @count
+    end
+    
     def inAsm
         enterAsm
         yield
index bbfce71..9dd4dc7 100644 (file)
@@ -267,7 +267,8 @@ ARM_INSTRUCTIONS =
 ARM64_INSTRUCTIONS =
     [
      "pcrtoaddr",   # Address from PC relative offset - adr instruction
-     "nopFixCortexA53Err835769" # nop on Cortex-A53 (nothing otherwise)
+     "nopFixCortexA53Err835769", # nop on Cortex-A53 (nothing otherwise)
+     "globaladdr"
     ]
 
 RISC_INSTRUCTIONS =
index 3fab69f..3b849fd 100644 (file)
@@ -13,6 +13,7 @@ set(bmalloc_SOURCES
     bmalloc/Environment.cpp
     bmalloc/Gigacage.cpp
     bmalloc/Heap.cpp
+    bmalloc/HeapKind.cpp
     bmalloc/LargeMap.cpp
     bmalloc/Logging.cpp
     bmalloc/ObjectType.cpp
index 3fe2cd0..b57fa01 100644 (file)
@@ -1,3 +1,49 @@
+2017-09-29  Filip Pizlo  <fpizlo@apple.com>
+
+        Enable gigacage on iOS
+        https://bugs.webkit.org/show_bug.cgi?id=177586
+
+        Reviewed by JF Bastien.
+        
+        Introduce the ability to disable gigacage at runtime if allocation fails. If any step of gigacage
+        allocation fails, we free all of the gigacages and turn off gigacage support.
+
+        * CMakeLists.txt:
+        * bmalloc.xcodeproj/project.pbxproj:
+        * bmalloc/Cache.cpp:
+        (bmalloc::Cache::scavenge):
+        * bmalloc/Cache.h:
+        (bmalloc::Cache::tryAllocate):
+        (bmalloc::Cache::allocate):
+        (bmalloc::Cache::deallocate):
+        (bmalloc::Cache::reallocate):
+        * bmalloc/Gigacage.cpp:
+        (Gigacage::ensureGigacage):
+        (Gigacage::runway):
+        (Gigacage::totalSize):
+        (Gigacage::shouldBeEnabled):
+        (): Deleted.
+        (Gigacage::Callback::Callback): Deleted.
+        (Gigacage::Callback::function): Deleted.
+        (Gigacage::PrimitiveDisableCallbacks::PrimitiveDisableCallbacks): Deleted.
+        * bmalloc/Gigacage.h:
+        (Gigacage::wasEnabled):
+        (Gigacage::isEnabled):
+        (Gigacage::runway): Deleted.
+        (Gigacage::totalSize): Deleted.
+        * bmalloc/HeapKind.cpp: Added.
+        (bmalloc::isActiveHeapKind):
+        (bmalloc::mapToActiveHeapKind):
+        * bmalloc/HeapKind.h:
+        (bmalloc::isActiveHeapKindAfterEnsuringGigacage):
+        (bmalloc::mapToActiveHeapKindAfterEnsuringGigacage):
+        * bmalloc/Scavenger.cpp:
+        (bmalloc::Scavenger::scavenge):
+        * bmalloc/bmalloc.h:
+        (bmalloc::api::tryLargeMemalignVirtual):
+        (bmalloc::api::freeLargeVirtual):
+        (bmalloc::api::isEnabled):
+
 2017-10-05  Filip Pizlo  <fpizlo@apple.com>
 
         Use one Scavenger thread for all Heaps
index c90760a..ff45305 100644 (file)
@@ -15,6 +15,7 @@
                0F5BF1521F22E1570029D91D /* Scavenger.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F5BF1501F22E1570029D91D /* Scavenger.cpp */; };
                0F5BF1531F22E1570029D91D /* Scavenger.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F5BF1511F22E1570029D91D /* Scavenger.h */; settings = {ATTRIBUTES = (Private, ); }; };
                0F5BF1731F23C5710029D91D /* BExport.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F5BF1721F23C5710029D91D /* BExport.h */; settings = {ATTRIBUTES = (Private, ); }; };
+               0FD557331F7EDB7B00B1F0A3 /* HeapKind.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FD557321F7EDB7B00B1F0A3 /* HeapKind.cpp */; };
                1400274918F89C1300115C97 /* Heap.h in Headers */ = {isa = PBXBuildFile; fileRef = 14DA320C18875B09007269E0 /* Heap.h */; settings = {ATTRIBUTES = (Private, ); }; };
                1400274A18F89C2300115C97 /* VMHeap.h in Headers */ = {isa = PBXBuildFile; fileRef = 144F7BFC18BFC517003537F3 /* VMHeap.h */; settings = {ATTRIBUTES = (Private, ); }; };
                140FA00319CE429C00FFD3C8 /* BumpRange.h in Headers */ = {isa = PBXBuildFile; fileRef = 140FA00219CE429C00FFD3C8 /* BumpRange.h */; settings = {ATTRIBUTES = (Private, ); }; };
@@ -90,6 +91,7 @@
                0F5BF1501F22E1570029D91D /* Scavenger.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; name = Scavenger.cpp; path = bmalloc/Scavenger.cpp; sourceTree = "<group>"; };
                0F5BF1511F22E1570029D91D /* Scavenger.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; name = Scavenger.h; path = bmalloc/Scavenger.h; sourceTree = "<group>"; };
                0F5BF1721F23C5710029D91D /* BExport.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; name = BExport.h; path = bmalloc/BExport.h; sourceTree = "<group>"; };
+               0FD557321F7EDB7B00B1F0A3 /* HeapKind.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = HeapKind.cpp; path = bmalloc/HeapKind.cpp; sourceTree = "<group>"; };
                140FA00219CE429C00FFD3C8 /* BumpRange.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = BumpRange.h; path = bmalloc/BumpRange.h; sourceTree = "<group>"; };
                140FA00419CE4B6800FFD3C8 /* LineMetadata.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = LineMetadata.h; path = bmalloc/LineMetadata.h; sourceTree = "<group>"; };
                14105E8318E14374003A106E /* ObjectType.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = ObjectType.cpp; path = bmalloc/ObjectType.cpp; sourceTree = "<group>"; };
                                1413E460189DCE1E00546D68 /* BInline.h */,
                                14C919C818FCC59F0028DB43 /* BPlatform.h */,
                                14D9DB4517F2447100EAAB79 /* FixedVector.h */,
+                               0FD557321F7EDB7B00B1F0A3 /* HeapKind.cpp */,
                                0F5BF1461F22A8B10029D91D /* HeapKind.h */,
                                141D9AFF1C8E51C0000ABBA0 /* List.h */,
                                4426E27E1C838EE0008EB042 /* Logging.cpp */,
                        buildActionMask = 2147483647;
                        files = (
                                0F5BF1521F22E1570029D91D /* Scavenger.cpp in Sources */,
+                               0FD557331F7EDB7B00B1F0A3 /* HeapKind.cpp in Sources */,
                                14F271C318EA3978008C152F /* Allocator.cpp in Sources */,
                                6599C5CC1EC3F15900A2F7BB /* AvailableMemory.cpp in Sources */,
                                14F271C418EA397B008C152F /* Cache.cpp in Sources */,
index 26f4202..c97b909 100644 (file)
@@ -35,6 +35,8 @@ void Cache::scavenge(HeapKind heapKind)
     PerHeapKind<Cache>* caches = PerThread<PerHeapKind<Cache>>::getFastCase();
     if (!caches)
         return;
+    if (!isActiveHeapKind(heapKind))
+        return;
 
     caches->at(heapKind).allocator().scavenge();
     caches->at(heapKind).deallocator().scavenge();
@@ -48,27 +50,27 @@ Cache::Cache(HeapKind heapKind)
 
 BNO_INLINE void* Cache::tryAllocateSlowCaseNullCache(HeapKind heapKind, size_t size)
 {
-    return PerThread<PerHeapKind<Cache>>::getSlowCase()->at(heapKind).allocator().tryAllocate(size);
+    return PerThread<PerHeapKind<Cache>>::getSlowCase()->at(mapToActiveHeapKind(heapKind)).allocator().tryAllocate(size);
 }
 
 BNO_INLINE void* Cache::allocateSlowCaseNullCache(HeapKind heapKind, size_t size)
 {
-    return PerThread<PerHeapKind<Cache>>::getSlowCase()->at(heapKind).allocator().allocate(size);
+    return PerThread<PerHeapKind<Cache>>::getSlowCase()->at(mapToActiveHeapKind(heapKind)).allocator().allocate(size);
 }
 
 BNO_INLINE void* Cache::allocateSlowCaseNullCache(HeapKind heapKind, size_t alignment, size_t size)
 {
-    return PerThread<PerHeapKind<Cache>>::getSlowCase()->at(heapKind).allocator().allocate(alignment, size);
+    return PerThread<PerHeapKind<Cache>>::getSlowCase()->at(mapToActiveHeapKind(heapKind)).allocator().allocate(alignment, size);
 }
 
 BNO_INLINE void Cache::deallocateSlowCaseNullCache(HeapKind heapKind, void* object)
 {
-    PerThread<PerHeapKind<Cache>>::getSlowCase()->at(heapKind).deallocator().deallocate(object);
+    PerThread<PerHeapKind<Cache>>::getSlowCase()->at(mapToActiveHeapKind(heapKind)).deallocator().deallocate(object);
 }
 
 BNO_INLINE void* Cache::reallocateSlowCaseNullCache(HeapKind heapKind, void* object, size_t newSize)
 {
-    return PerThread<PerHeapKind<Cache>>::getSlowCase()->at(heapKind).allocator().reallocate(object, newSize);
+    return PerThread<PerHeapKind<Cache>>::getSlowCase()->at(mapToActiveHeapKind(heapKind)).allocator().reallocate(object, newSize);
 }
 
 } // namespace bmalloc
index f27c04d..c414ec8 100644 (file)
@@ -68,7 +68,7 @@ inline void* Cache::tryAllocate(HeapKind heapKind, size_t size)
     PerHeapKind<Cache>* caches = PerThread<PerHeapKind<Cache>>::getFastCase();
     if (!caches)
         return tryAllocateSlowCaseNullCache(heapKind, size);
-    return caches->at(heapKind).allocator().tryAllocate(size);
+    return caches->at(mapToActiveHeapKindAfterEnsuringGigacage(heapKind)).allocator().tryAllocate(size);
 }
 
 inline void* Cache::allocate(HeapKind heapKind, size_t size)
@@ -76,7 +76,7 @@ inline void* Cache::allocate(HeapKind heapKind, size_t size)
     PerHeapKind<Cache>* caches = PerThread<PerHeapKind<Cache>>::getFastCase();
     if (!caches)
         return allocateSlowCaseNullCache(heapKind, size);
-    return caches->at(heapKind).allocator().allocate(size);
+    return caches->at(mapToActiveHeapKindAfterEnsuringGigacage(heapKind)).allocator().allocate(size);
 }
 
 inline void* Cache::tryAllocate(HeapKind heapKind, size_t alignment, size_t size)
@@ -84,7 +84,7 @@ inline void* Cache::tryAllocate(HeapKind heapKind, size_t alignment, size_t size
     PerHeapKind<Cache>* caches = PerThread<PerHeapKind<Cache>>::getFastCase();
     if (!caches)
         return allocateSlowCaseNullCache(heapKind, alignment, size);
-    return caches->at(heapKind).allocator().tryAllocate(alignment, size);
+    return caches->at(mapToActiveHeapKindAfterEnsuringGigacage(heapKind)).allocator().tryAllocate(alignment, size);
 }
 
 inline void* Cache::allocate(HeapKind heapKind, size_t alignment, size_t size)
@@ -92,7 +92,7 @@ inline void* Cache::allocate(HeapKind heapKind, size_t alignment, size_t size)
     PerHeapKind<Cache>* caches = PerThread<PerHeapKind<Cache>>::getFastCase();
     if (!caches)
         return allocateSlowCaseNullCache(heapKind, alignment, size);
-    return caches->at(heapKind).allocator().allocate(alignment, size);
+    return caches->at(mapToActiveHeapKindAfterEnsuringGigacage(heapKind)).allocator().allocate(alignment, size);
 }
 
 inline void Cache::deallocate(HeapKind heapKind, void* object)
@@ -100,7 +100,7 @@ inline void Cache::deallocate(HeapKind heapKind, void* object)
     PerHeapKind<Cache>* caches = PerThread<PerHeapKind<Cache>>::getFastCase();
     if (!caches)
         return deallocateSlowCaseNullCache(heapKind, object);
-    return caches->at(heapKind).deallocator().deallocate(object);
+    return caches->at(mapToActiveHeapKindAfterEnsuringGigacage(heapKind)).deallocator().deallocate(object);
 }
 
 inline void* Cache::reallocate(HeapKind heapKind, void* object, size_t newSize)
@@ -108,7 +108,7 @@ inline void* Cache::reallocate(HeapKind heapKind, void* object, size_t newSize)
     PerHeapKind<Cache>* caches = PerThread<PerHeapKind<Cache>>::getFastCase();
     if (!caches)
         return reallocateSlowCaseNullCache(heapKind, object, newSize);
-    return caches->at(heapKind).allocator().reallocate(object, newSize);
+    return caches->at(mapToActiveHeapKindAfterEnsuringGigacage(heapKind)).allocator().reallocate(object, newSize);
 }
 
 } // namespace bmalloc
index 463ef16..4c4bd19 100644 (file)
 #include <cstdio>
 #include <mutex>
 
+#if BCPU(ARM64)
+// FIXME: There is no good reason for ARM64 to be special.
+// https://bugs.webkit.org/show_bug.cgi?id=177605
+#define PRIMITIVE_GIGACAGE_RUNWAY 0
+#else
+// FIXME: Consider making this 32GB, in case unsigned 32-bit indices find their way into indexed accesses.
+// https://bugs.webkit.org/show_bug.cgi?id=175062
+#define PRIMITIVE_GIGACAGE_RUNWAY (16llu * 1024 * 1024 * 1024)
+#endif
+
+// FIXME: Reconsider this.
+// https://bugs.webkit.org/show_bug.cgi?id=175921
+#define JSVALUE_GIGACAGE_RUNWAY 0
+#define STRING_GIGACAGE_RUNWAY 0
+
 char g_gigacageBasePtrs[GIGACAGE_BASE_PTRS_SIZE] __attribute__((aligned(GIGACAGE_BASE_PTRS_SIZE)));
 
 using namespace bmalloc;
 
 namespace Gigacage {
 
+bool g_wasEnabled;
+
 namespace {
 
 bool s_isDisablingPrimitiveGigacageDisabled;
@@ -69,8 +86,6 @@ public:
     }
 };
 
-} // anonymous namespce
-
 struct Callback {
     Callback() { }
     
@@ -90,6 +105,8 @@ struct PrimitiveDisableCallbacks {
     Vector<Callback> callbacks;
 };
 
+} // anonymous namespace
+
 void ensureGigacage()
 {
 #if GIGACAGE_ENABLED
@@ -100,12 +117,20 @@ void ensureGigacage()
             if (!shouldBeEnabled())
                 return;
             
+            bool ok = true;
+            
             forEachKind(
                 [&] (Kind kind) {
+                    if (!ok)
+                        return;
                     // FIXME: Randomize where this goes.
                     // https://bugs.webkit.org/show_bug.cgi?id=175245
                     basePtr(kind) = tryVMAllocate(alignment(kind), totalSize(kind));
                     if (!basePtr(kind)) {
+                        if (GIGACAGE_ALLOCATION_CAN_FAIL) {
+                            ok = false;
+                            return;
+                        }
                         fprintf(stderr, "FATAL: Could not allocate %s gigacage.\n", name(kind));
                         BCRASH();
                     }
@@ -113,11 +138,44 @@ void ensureGigacage()
                     vmDeallocatePhysicalPages(basePtr(kind), totalSize(kind));
                 });
             
+            if (!ok) {
+                forEachKind(
+                    [&] (Kind kind) {
+                        if (!basePtr(kind))
+                            return;
+                        
+                        vmDeallocate(basePtr(kind), totalSize(kind));
+                        
+                        basePtr(kind) = nullptr;
+                    });
+                return;
+            }
+            
             protectGigacageBasePtrs();
+            g_wasEnabled = true;
         });
 #endif // GIGACAGE_ENABLED
 }
 
+size_t runway(Kind kind)
+{
+    switch (kind) {
+    case Primitive:
+        return static_cast<size_t>(PRIMITIVE_GIGACAGE_RUNWAY);
+    case JSValue:
+        return static_cast<size_t>(JSVALUE_GIGACAGE_RUNWAY);
+    case String:
+        return static_cast<size_t>(STRING_GIGACAGE_RUNWAY);
+    }
+    BCRASH();
+    return 0;
+}
+
+size_t totalSize(Kind kind)
+{
+    return size(kind) + runway(kind);
+}
+
 void disablePrimitiveGigacage()
 {
     ensureGigacage();
@@ -187,7 +245,27 @@ bool isDisablingPrimitiveGigacageDisabled()
 
 bool shouldBeEnabled()
 {
-    return GIGACAGE_ENABLED && !PerProcess<Environment>::get()->isDebugHeapEnabled();
+    static std::once_flag onceFlag;
+    static bool cached;
+    std::call_once(
+        onceFlag,
+        [] {
+            bool result = GIGACAGE_ENABLED && !PerProcess<Environment>::get()->isDebugHeapEnabled();
+            if (!result)
+                return;
+            
+            if (char* gigacageEnabled = getenv("GIGACAGE_ENABLED")) {
+                if (!strcasecmp(gigacageEnabled, "no") || !strcasecmp(gigacageEnabled, "false") || !strcasecmp(gigacageEnabled, "0")) {
+                    fprintf(stderr, "Warning: disabling gigacage because GIGACAGE_ENABLED=%s!\n", gigacageEnabled);
+                    return;
+                } else if (strcasecmp(gigacageEnabled, "yes") && strcasecmp(gigacageEnabled, "true") && strcasecmp(gigacageEnabled, "1"))
+                    fprintf(stderr, "Warning: invalid argument to GIGACAGE_ENABLED: %s\n", gigacageEnabled);
+            }
+            
+            cached = true;
+        });
+    
+    return cached;
 }
 
 } // namespace Gigacage
index 3b7579b..aa6a11c 100644 (file)
 #include <cstddef>
 #include <inttypes.h>
 
+#if BCPU(ARM64)
+// FIXME: This can probably be a lot bigger on iOS. I just haven't tried to make it bigger yet.
+// https://bugs.webkit.org/show_bug.cgi?id=177605
+#define PRIMITIVE_GIGACAGE_SIZE 0x40000000llu
+#define JSVALUE_GIGACAGE_SIZE 0x40000000llu
+#define STRING_GIGACAGE_SIZE 0x40000000llu
+#define GIGACAGE_ALLOCATION_CAN_FAIL 1
+#else
 #define PRIMITIVE_GIGACAGE_SIZE 0x800000000llu
 #define JSVALUE_GIGACAGE_SIZE 0x400000000llu
 #define STRING_GIGACAGE_SIZE 0x400000000llu
+#define GIGACAGE_ALLOCATION_CAN_FAIL 0
+#endif
 
 #define GIGACAGE_SIZE_TO_MASK(size) ((size) - 1)
 
 #define JSVALUE_GIGACAGE_MASK GIGACAGE_SIZE_TO_MASK(JSVALUE_GIGACAGE_SIZE)
 #define STRING_GIGACAGE_MASK GIGACAGE_SIZE_TO_MASK(STRING_GIGACAGE_SIZE)
 
-// FIXME: Consider making this 32GB, in case unsigned 32-bit indices find their way into indexed accesses.
-// https://bugs.webkit.org/show_bug.cgi?id=175062
-#define PRIMITIVE_GIGACAGE_RUNWAY (16llu * 1024 * 1024 * 1024)
-
-// FIXME: Reconsider this.
-// https://bugs.webkit.org/show_bug.cgi?id=175921
-#define JSVALUE_GIGACAGE_RUNWAY 0
-#define STRING_GIGACAGE_RUNWAY 0
-
-#if (BOS(DARWIN) || BOS(LINUX)) && BCPU(X86_64)
+#if (BOS(DARWIN) || BOS(LINUX)) && (BCPU(ARM64) || BCPU(X86_64))
 #define GIGACAGE_ENABLED 1
 #else
 #define GIGACAGE_ENABLED 0
 #endif
 
-#define GIGACAGE_BASE_PTRS_SIZE 8192
+#if BCPU(ARM64)
+#define GIGACAGE_BASE_PTRS_SIZE 16384
+#else
+#define GIGACAGE_BASE_PTRS_SIZE 4096
+#endif
 
 extern "C" BEXPORT char g_gigacageBasePtrs[GIGACAGE_BASE_PTRS_SIZE] __attribute__((aligned(GIGACAGE_BASE_PTRS_SIZE)));
 
 namespace Gigacage {
 
+extern BEXPORT bool g_wasEnabled;
+BINLINE bool wasEnabled() { return g_wasEnabled; }
+
 struct BasePtrs {
     void* primitive;
     void* jsValue;
@@ -127,6 +135,11 @@ BINLINE void*& basePtr(Kind kind)
     return basePtr(basePtrs(), kind);
 }
 
+BINLINE bool isEnabled(Kind kind)
+{
+    return !!basePtr(kind);
+}
+
 BINLINE size_t size(Kind kind)
 {
     switch (kind) {
@@ -151,24 +164,8 @@ BINLINE size_t mask(Kind kind)
     return GIGACAGE_SIZE_TO_MASK(size(kind));
 }
 
-BINLINE size_t runway(Kind kind)
-{
-    switch (kind) {
-    case Primitive:
-        return static_cast<size_t>(PRIMITIVE_GIGACAGE_RUNWAY);
-    case JSValue:
-        return static_cast<size_t>(JSVALUE_GIGACAGE_RUNWAY);
-    case String:
-        return static_cast<size_t>(STRING_GIGACAGE_RUNWAY);
-    }
-    BCRASH();
-    return 0;
-}
-
-BINLINE size_t totalSize(Kind kind)
-{
-    return size(kind) + runway(kind);
-}
+size_t runway(Kind kind);
+size_t totalSize(Kind kind);
 
 template<typename Func>
 void forEachKind(const Func& func)
index 1c1710c..4aceb26 100644 (file)
@@ -177,6 +177,8 @@ void Heap::deallocateLineCache(std::lock_guard<StaticMutex>&, LineCache& lineCac
 
 void Heap::allocateSmallChunk(std::lock_guard<StaticMutex>& lock, size_t pageClass)
 {
+    RELEASE_BASSERT(isActiveHeapKind(m_kind));
+    
     size_t pageSize = bmalloc::pageSize(pageClass);
 
     Chunk* chunk = [&]() {
@@ -221,6 +223,8 @@ void Heap::deallocateSmallChunk(Chunk* chunk, size_t pageClass)
 
 SmallPage* Heap::allocateSmallPage(std::lock_guard<StaticMutex>& lock, size_t sizeClass, LineCache& lineCache)
 {
+    RELEASE_BASSERT(isActiveHeapKind(m_kind));
+
     if (!lineCache[sizeClass].isEmpty())
         return lineCache[sizeClass].popFront();
 
@@ -300,6 +304,8 @@ void Heap::allocateSmallBumpRangesByMetadata(
     BumpAllocator& allocator, BumpRangeCache& rangeCache,
     LineCache& lineCache)
 {
+    RELEASE_BASSERT(isActiveHeapKind(m_kind));
+
     SmallPage* page = allocateSmallPage(lock, sizeClass, lineCache);
     SmallLine* lines = page->begin();
     BASSERT(page->hasFreeLines(lock));
@@ -362,6 +368,8 @@ void Heap::allocateSmallBumpRangesByObject(
     BumpAllocator& allocator, BumpRangeCache& rangeCache,
     LineCache& lineCache)
 {
+    RELEASE_BASSERT(isActiveHeapKind(m_kind));
+
     size_t size = allocator.size();
     SmallPage* page = allocateSmallPage(lock, sizeClass, lineCache);
     BASSERT(page->hasFreeLines(lock));
@@ -414,6 +422,8 @@ void Heap::allocateSmallBumpRangesByObject(
 
 LargeRange Heap::splitAndAllocate(LargeRange& range, size_t alignment, size_t size, AllocationKind allocationKind)
 {
+    RELEASE_BASSERT(isActiveHeapKind(m_kind));
+
     LargeRange prev;
     LargeRange next;
 
@@ -461,6 +471,8 @@ LargeRange Heap::splitAndAllocate(LargeRange& range, size_t alignment, size_t si
 
 void* Heap::tryAllocateLarge(std::lock_guard<StaticMutex>&, size_t alignment, size_t size, AllocationKind allocationKind)
 {
+    RELEASE_BASSERT(isActiveHeapKind(m_kind));
+
     BASSERT(isPowerOfTwo(alignment));
     
     if (m_debugHeap)
diff --git a/Source/bmalloc/bmalloc/HeapKind.cpp b/Source/bmalloc/bmalloc/HeapKind.cpp
new file mode 100644 (file)
index 0000000..2343b95
--- /dev/null
@@ -0,0 +1,42 @@
+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+ */
+
+#include "HeapKind.h"
+
+namespace bmalloc {
+
+bool isActiveHeapKind(HeapKind kind)
+{
+    Gigacage::ensureGigacage();
+    return isActiveHeapKindAfterEnsuringGigacage(kind);
+}
+
+HeapKind mapToActiveHeapKind(HeapKind kind)
+{
+    Gigacage::ensureGigacage();
+    return mapToActiveHeapKindAfterEnsuringGigacage(kind);
+}
+
+} // namespace bmalloc
index fe8762d..95afce3 100644 (file)
@@ -85,5 +85,37 @@ BINLINE HeapKind heapKind(Gigacage::Kind kind)
     return HeapKind::Primary;
 }
 
+BINLINE bool isActiveHeapKindAfterEnsuringGigacage(HeapKind kind)
+{
+    switch (kind) {
+    case HeapKind::PrimitiveGigacage:
+    case HeapKind::JSValueGigacage:
+    case HeapKind::StringGigacage:
+        if (Gigacage::wasEnabled())
+            return true;
+        return false;
+    default:
+        return true;
+    }
+}
+
+BEXPORT bool isActiveHeapKind(HeapKind);
+
+BINLINE HeapKind mapToActiveHeapKindAfterEnsuringGigacage(HeapKind kind)
+{
+    switch (kind) {
+    case HeapKind::PrimitiveGigacage:
+    case HeapKind::JSValueGigacage:
+    case HeapKind::StringGigacage:
+        if (Gigacage::wasEnabled())
+            return kind;
+        return HeapKind::Primary;
+    default:
+        return kind;
+    }
+}
+
+BEXPORT HeapKind mapToActiveHeapKind(HeapKind);
+
 } // namespace bmalloc
 
index ac2a668..b11a06c 100644 (file)
@@ -117,8 +117,11 @@ void Scavenger::schedule(size_t bytes)
 void Scavenger::scavenge()
 {
     std::lock_guard<StaticMutex> lock(Heap::mutex());
-    for (unsigned i = numHeaps; i--;)
+    for (unsigned i = numHeaps; i--;) {
+        if (!isActiveHeapKind(static_cast<HeapKind>(i)))
+            continue;
         PerProcess<PerHeapKind<Heap>>::get()->at(i).scavenge(lock);
+    }
 }
 
 void Scavenger::threadEntryPoint(Scavenger* scavenger)
index 5d6dd21..d131734 100644 (file)
@@ -68,6 +68,7 @@ inline void* realloc(void* object, size_t newSize, HeapKind kind = HeapKind::Pri
 // Returns null for failure
 inline void* tryLargeMemalignVirtual(size_t alignment, size_t size, HeapKind kind = HeapKind::Primary)
 {
+    kind = mapToActiveHeapKind(kind);
     Heap& heap = PerProcess<PerHeapKind<Heap>>::get()->at(kind);
     std::lock_guard<StaticMutex> lock(Heap::mutex());
     return heap.allocateLarge(lock, alignment, size, AllocationKind::Virtual);
@@ -80,6 +81,7 @@ inline void free(void* object, HeapKind kind = HeapKind::Primary)
 
 inline void freeLargeVirtual(void* object, HeapKind kind = HeapKind::Primary)
 {
+    kind = mapToActiveHeapKind(kind);
     Heap& heap = PerProcess<PerHeapKind<Heap>>::get()->at(kind);
     std::lock_guard<StaticMutex> lock(Heap::mutex());
     heap.deallocateLarge(lock, object, AllocationKind::Virtual);
@@ -100,6 +102,7 @@ inline void scavenge()
 
 inline bool isEnabled(HeapKind kind = HeapKind::Primary)
 {
+    kind = mapToActiveHeapKind(kind);
     std::unique_lock<StaticMutex> lock(Heap::mutex());
     return !PerProcess<PerHeapKind<Heap>>::getFastCase()->at(kind).debugHeap();
 }
index 568ac19..8a81b31 100644 (file)
@@ -1,3 +1,15 @@
+2017-09-29  Filip Pizlo  <fpizlo@apple.com>
+
+        Enable gigacage on iOS
+        https://bugs.webkit.org/show_bug.cgi?id=177586
+
+        Reviewed by JF Bastien.
+        
+        Add a mode to test disabling Gigacage.
+
+        * Scripts/run-jsc-stress-tests:
+        * Scripts/webkitruby/jsc-stress-test-writer-default.rb:
+
 2017-10-06  Aishwarya Nirmal  <anirmal@apple.com>
 
         [iOS] Respect the "caret-color" CSS property when editing
index 4210964..4a707c7 100755 (executable)
@@ -513,7 +513,7 @@ def baseOutputName(kind)
     "#{$collectionName}/#{$benchmark}.#{kind}"
 end
 
-def addRunCommand(kind, command, outputHandler, errorHandler)
+def addRunCommand(kind, command, outputHandler, errorHandler, *additionalEnv)
     $didAddRunCommand = true
     name = baseOutputName(kind)
     if $filter and name !~ $filter
@@ -522,6 +522,7 @@ def addRunCommand(kind, command, outputHandler, errorHandler)
     plan = Plan.new(
         $benchmarkDirectory, command, "#{$collectionName}/#{$benchmark}", name, outputHandler,
         errorHandler)
+    plan.additionalEnv.push(*additionalEnv)
     if $numChildProcesses > 1 and $runCommandOptions[:isSlow]
         $runlist.unshift plan
     else
@@ -1307,8 +1308,16 @@ def defaultRunMozillaTest(mode, *extraFiles)
     end
 end
 
+def runNoisyTestImpl(kind, options, additionalEnv)
+    addRunCommand(kind, [pathToVM.to_s] + BASE_OPTIONS + options + [$benchmark.to_s], noisyOutputHandler, noisyErrorHandler, *additionalEnv)
+end
+
 def runNoisyTest(kind, *options)
-    addRunCommand(kind, [pathToVM.to_s] + BASE_OPTIONS + options + [$benchmark.to_s], noisyOutputHandler, noisyErrorHandler)
+    runNoisyTestImpl(kind, options, [])
+end
+
+def runNoisyTestWithEnv(kind, *additionalEnv)
+    runNoisyTestImpl(kind, [], additionalEnv)
 end
 
 def runNoisyTestDefault
index aed5736..0553f2e 100644 (file)
@@ -215,7 +215,7 @@ def chakraPassFailErrorHandler
 end
 
 class Plan
-    attr_reader :directory, :arguments, :family, :name, :outputHandler, :errorHandler
+    attr_reader :directory, :arguments, :family, :name, :outputHandler, :errorHandler, :additionalEnv
     attr_accessor :index
     
     def initialize(directory, arguments, family, name, outputHandler, errorHandler)
@@ -226,6 +226,7 @@ class Plan
         @outputHandler = outputHandler
         @errorHandler = errorHandler
         @isSlow = !!$runCommandOptions[:isSlow]
+        @additionalEnv = []
     end
     
     def shellCommand
@@ -233,7 +234,7 @@ class Plan
         # in the subshell when we return we will be in our original directory. This is nice because we don't
         # have to bend over backwards to do things relative to the root.
         script = "(cd ../#{Shellwords.shellescape(@directory.to_s)} && ("
-        $envVars.each { |var| script += "export " << var << "; " }
+        ($envVars + additionalEnv).each { |var| script += "export " << var << "; " }
         script += "\"$@\" " + escapeAll(@arguments) + "))"
         return script
     end