We should be able to eliminate rest parameter allocations
authorsbarati@apple.com <sbarati@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 1 Nov 2016 20:03:03 +0000 (20:03 +0000)
committersbarati@apple.com <sbarati@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 1 Nov 2016 20:03:03 +0000 (20:03 +0000)
https://bugs.webkit.org/show_bug.cgi?id=163925

Reviewed by Filip Pizlo.

JSTests:

* microbenchmarks/rest-parameter-allocation-elimination.js: Added.
(assert):
(test1.bar):
(test1):
(test2.jaz):
(test2.jaz2.kaz):
(test2.jaz2):
(test2):
(test3.foo):
(test3.baz):
(test3.jaz):
(test3):
(test4.baz):
(test4.jaz):
(test4):
(test5.baz):
(test5.jaz):
(test5):
(test6.baz):
(test6.jaz):
(test6):
(test7.baz):
(test7.jaz):
(test7.check):
(test7):
(test8.baz):
(test8.jaz):
(test8.check):
(test8):
(test9.baz):
(test9.jaz):
(test9.check):
(test9):
(test10.baz):
(test10.jaz):
(test10):
(test11.bar):
(test11.foo):
(test11.makeArguments):
(test11.):
(test12):
(test12.bar):
(test12.foo):
(test12.makeArguments):
(test12.):
(test13.bar):
(test13.top):
(test13.foo):
(test13.makeArguments):
(test13.):
(test13):
(test14.bar):
(test14.top):
(test14.foo):
(test14.makeArguments):
(test14.):
(test14):
(test15.bar):
(test15.top):
(test15.foo):
(test15.makeArguments):
(test15.):
(test15):

Source/JavaScriptCore:

This is the first step towards eliminating rest parameter
allocations when they're spread to other function calls:
`function foo(...args) { bar(...args); }`

This patch simply removes the allocation for rest parameter
allocations using the same escape analysis that is performed
in the argument elimination phase. I've added a new rule to
the phase to make sure that CheckStructure doesn't count as
an escape for an allocation since this often shows up in code
like this:

```
function foo(...args) {
    let r = [];
    for (let i = 0; i < args.length; i++)
        r.push(args[i]);
    return r;
}
```

The above program now entirely eliminates the allocation for args
compiled in the FTL. Programs like this also eliminate the allocation
for args:

```
function foo(...args) { return [args.length, args[0]]; }

function bar(...args) { return someOtherFunction.apply(null, args); }
```

This patch extends the arguments elimination phase to understand
the concept that we may want to forward arguments, or get from
the arguments region, starting at some offset. The offset is the
number of names parameter before the rest parameter. For example:

```
function foo(a, b, ...args) { return bar.apply(null, args); }
```

Will forward arguments starting at the *third* argument.
Our arguments forwarding code already had the notion of starting
from some offset, however, I found bugs in that code. I extended
it to work properly for rest parameters with arbitrary skip offsets.

And this program:
```
function foo(...args) {
    let r = [];
    for (let i = 0; i < args.length; i++)
        r.push(args[i]);
    return r;
}
```

Knows to perform the GetMyArgumentByVal* with an offset of 3
inside the loop. To make this work, I taught GetMyArgumentByVal
and GetMyArgumentByValOutOfBounds to take an offset representing
the number of arguments to skip.

This patch is a ~20% speedup on microbenchmarks.

* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::finishCreation):
* dfg/DFGAbstractInterpreterInlines.h:
(JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects):
* dfg/DFGArgumentsEliminationPhase.cpp:
* dfg/DFGArgumentsUtilities.cpp:
(JSC::DFG::emitCodeToGetArgumentsArrayLength):
* dfg/DFGClobberize.h:
(JSC::DFG::clobberize):
* dfg/DFGConstantFoldingPhase.cpp:
(JSC::DFG::ConstantFoldingPhase::foldConstants):
* dfg/DFGDoesGC.cpp:
(JSC::DFG::doesGC):
* dfg/DFGFixupPhase.cpp:
(JSC::DFG::FixupPhase::fixupNode):
* dfg/DFGNode.h:
(JSC::DFG::Node::hasConstant):
(JSC::DFG::Node::constant):
(JSC::DFG::Node::isPhantomAllocation):
(JSC::DFG::Node::numberOfArgumentsToSkip):
* dfg/DFGNodeType.h:
* dfg/DFGOSRAvailabilityAnalysisPhase.cpp:
(JSC::DFG::LocalOSRAvailabilityCalculator::executeNode):
* dfg/DFGOperations.cpp:
* dfg/DFGPreciseLocalClobberize.h:
(JSC::DFG::PreciseLocalClobberizeAdaptor::readTop):
* dfg/DFGPredictionPropagationPhase.cpp:
* dfg/DFGSafeToExecute.h:
(JSC::DFG::safeToExecute):
* dfg/DFGSpeculativeJIT.cpp:
(JSC::DFG::SpeculativeJIT::compileCreateRest):
* dfg/DFGSpeculativeJIT32_64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGSpeculativeJIT64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGStructureRegistrationPhase.cpp:
(JSC::DFG::StructureRegistrationPhase::run):
* dfg/DFGValidate.cpp:
* ftl/FTLCapabilities.cpp:
(JSC::FTL::canCompile):
* ftl/FTLLowerDFGToB3.cpp:
(JSC::FTL::DFG::LowerDFGToB3::compileNode):
(JSC::FTL::DFG::LowerDFGToB3::compileGetMyArgumentByVal):
(JSC::FTL::DFG::LowerDFGToB3::compileCreateRest):
(JSC::FTL::DFG::LowerDFGToB3::compileForwardVarargs):
(JSC::FTL::DFG::LowerDFGToB3::getArgumentsStart):
* ftl/FTLOperations.cpp:
(JSC::FTL::operationPopulateObjectInOSR):
(JSC::FTL::operationMaterializeObjectInOSR):
* jit/SetupVarargsFrame.cpp:
(JSC::emitSetVarargsFrame):
* runtime/CommonSlowPaths.cpp:
(JSC::SLOW_PATH_DECL):
* runtime/JSGlobalObject.h:
(JSC::JSGlobalObject::restParameterStructure):

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@208235 268f45cc-cd09-0410-ab3c-d52691b4dbfc

29 files changed:
JSTests/ChangeLog
JSTests/microbenchmarks/rest-parameter-allocation-elimination.js [new file with mode: 0644]
Source/JavaScriptCore/ChangeLog
Source/JavaScriptCore/bytecode/CodeBlock.cpp
Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h
Source/JavaScriptCore/dfg/DFGArgumentsEliminationPhase.cpp
Source/JavaScriptCore/dfg/DFGArgumentsUtilities.cpp
Source/JavaScriptCore/dfg/DFGClobberize.h
Source/JavaScriptCore/dfg/DFGConstantFoldingPhase.cpp
Source/JavaScriptCore/dfg/DFGDoesGC.cpp
Source/JavaScriptCore/dfg/DFGFixupPhase.cpp
Source/JavaScriptCore/dfg/DFGNode.h
Source/JavaScriptCore/dfg/DFGNodeType.h
Source/JavaScriptCore/dfg/DFGOSRAvailabilityAnalysisPhase.cpp
Source/JavaScriptCore/dfg/DFGOperations.cpp
Source/JavaScriptCore/dfg/DFGPreciseLocalClobberize.h
Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp
Source/JavaScriptCore/dfg/DFGSafeToExecute.h
Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp
Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp
Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp
Source/JavaScriptCore/dfg/DFGStructureRegistrationPhase.cpp
Source/JavaScriptCore/dfg/DFGValidate.cpp
Source/JavaScriptCore/ftl/FTLCapabilities.cpp
Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp
Source/JavaScriptCore/ftl/FTLOperations.cpp
Source/JavaScriptCore/jit/SetupVarargsFrame.cpp
Source/JavaScriptCore/runtime/CommonSlowPaths.cpp
Source/JavaScriptCore/runtime/JSGlobalObject.h

index 2f02685..0f5dc71 100644 (file)
@@ -1,3 +1,74 @@
+2016-11-01  Saam Barati  <sbarati@apple.com>
+
+        We should be able to eliminate rest parameter allocations
+        https://bugs.webkit.org/show_bug.cgi?id=163925
+
+        Reviewed by Filip Pizlo.
+
+        * microbenchmarks/rest-parameter-allocation-elimination.js: Added.
+        (assert):
+        (test1.bar):
+        (test1):
+        (test2.jaz):
+        (test2.jaz2.kaz):
+        (test2.jaz2):
+        (test2):
+        (test3.foo):
+        (test3.baz):
+        (test3.jaz):
+        (test3):
+        (test4.baz):
+        (test4.jaz):
+        (test4):
+        (test5.baz):
+        (test5.jaz):
+        (test5):
+        (test6.baz):
+        (test6.jaz):
+        (test6):
+        (test7.baz):
+        (test7.jaz):
+        (test7.check):
+        (test7):
+        (test8.baz):
+        (test8.jaz):
+        (test8.check):
+        (test8):
+        (test9.baz):
+        (test9.jaz):
+        (test9.check):
+        (test9):
+        (test10.baz):
+        (test10.jaz):
+        (test10):
+        (test11.bar):
+        (test11.foo):
+        (test11.makeArguments):
+        (test11.):
+        (test12):
+        (test12.bar):
+        (test12.foo):
+        (test12.makeArguments):
+        (test12.):
+        (test13.bar):
+        (test13.top):
+        (test13.foo):
+        (test13.makeArguments):
+        (test13.):
+        (test13):
+        (test14.bar):
+        (test14.top):
+        (test14.foo):
+        (test14.makeArguments):
+        (test14.):
+        (test14):
+        (test15.bar):
+        (test15.top):
+        (test15.foo):
+        (test15.makeArguments):
+        (test15.):
+        (test15):
+
 2016-11-01  Commit Queue  <commit-queue@webkit.org>
 
         Unreviewed, rolling out r208208 and r208210.
diff --git a/JSTests/microbenchmarks/rest-parameter-allocation-elimination.js b/JSTests/microbenchmarks/rest-parameter-allocation-elimination.js
new file mode 100644 (file)
index 0000000..d369e33
--- /dev/null
@@ -0,0 +1,613 @@
+function assert(b, m = "") {
+    if (!b)
+        throw new Error("Bad assertion: " + m);
+}
+noInline(assert);
+
+const iterations = 20000;
+
+function test1() {
+    function bar(a, b, ...args) {
+        return args.length;
+    }
+    noInline(bar);
+    for (let i = 0; i < iterations; i++) {
+        assert(bar() === 0, bar());
+        assert(bar(i) === 0);
+        assert(bar(i, i) === 0);
+        assert(bar(i, i, i) === 1);
+        assert(bar(i, i, i, i, i) === 3);
+    }
+}
+
+function shallowEq(a, b) {
+    if (a.length !== b.length)
+        return false;
+    for (let i = 0; i < a.length; i++) {
+        if (a.length !== b.length)
+            return false;
+    }
+    return true;
+}
+noInline(shallowEq);
+
+function test2() {
+    function jaz(a, b, ...args) {
+        let result = [];
+        for (let i = 0; i < args.length; i++) {
+            result.push(args[i]);
+        }
+        return result;
+    }
+    noInline(jaz);
+
+    function jaz2(...args) {
+        function kaz(a, b, ...args) {
+            let result = [];
+            for (let i = 0; i < args.length; i++) {
+                result.push(args[i]);
+            }
+            return result;
+        }
+        return kaz.apply(null, args);
+    }
+    noInline(jaz2);
+
+    for (let f of [jaz, jaz2]) {
+        for (let i = 0; i < iterations; i++) {
+            let r = f();
+            assert(!r.length);
+
+            r = f(i);
+            assert(!r.length);
+
+            r = f(i, i)
+            assert(!r.length);
+
+            r = f(i, i, i)
+            assert(r.length === 1);
+            assert(shallowEq(r, [i]));
+
+            r = f(i, i, i)
+            assert(r.length === 1);
+            assert(shallowEq(r, [i]));
+
+            r = f(i, i, i, i*2, i*4)
+            assert(r.length === 3);
+            assert(shallowEq(r, [i, i*2, i*4]));
+        }
+    }
+}
+
+function test3() {
+    function foo(...args) {
+        return args;
+    }
+    function baz(a, b, c, ...args) {
+        return foo.apply(null, args);
+    }
+    function jaz(a, b, c, d, e, f) {
+        return baz(a, b, c, d, e, f);
+    }
+    noInline(jaz);
+
+    for (let i = 0; i < iterations; i++) {
+        let r = jaz();
+        assert(r.length === 3);
+        assert(shallowEq(r, [undefined, undefined, undefined]));
+
+        r = jaz(i, i);
+        assert(r.length === 3);
+        assert(shallowEq(r, [undefined, undefined, undefined]));
+
+        r = jaz(i, i, i);
+        assert(r.length === 3);
+        assert(shallowEq(r, [undefined, undefined, undefined]));
+
+        r = jaz(i, i, i, i);
+        assert(r.length === 3);
+        assert(shallowEq(r, [i, undefined, undefined]));
+
+        r = jaz(i, i, i, i, i, i);
+        assert(r.length === 3);
+        assert(shallowEq(r, [i, i, i]));
+    }
+}
+
+function test4() {
+    function baz(...args) {
+        return args;
+    }
+    function jaz(a, b, ...args) {
+        return baz.apply(null, args);
+    }
+    noInline(jaz);
+
+    for (let i = 0; i < iterations; i++) {
+        let r = jaz();
+        assert(r.length === 0);
+
+        r = jaz(i);
+        assert(r.length === 0);
+
+        r = jaz(i, i);
+        assert(r.length === 0);
+
+        r = jaz(i, i, i);
+        assert(r.length === 1);
+        assert(shallowEq(r, [i]));
+
+        r = jaz(i, i, i, i*10);
+        assert(r.length === 2);
+        assert(shallowEq(r, [i, i*10]));
+
+        let o = {};
+        r = jaz(i, i, i, i*10, o);
+        assert(r.length === 3);
+        assert(shallowEq(r, [i, i*10, o]));
+    }
+}
+
+function test5() {
+    function baz(...args) {
+        return args;
+    }
+    noInline(baz);
+    function jaz(a, b, ...args) {
+        return baz.apply(null, args);
+    }
+    noInline(jaz);
+
+    for (let i = 0; i < iterations; i++) {
+        let r = jaz();
+        assert(r.length === 0);
+
+        r = jaz(i);
+        assert(r.length === 0);
+
+        r = jaz(i, i);
+        assert(r.length === 0);
+
+        r = jaz(i, i, i);
+        assert(r.length === 1);
+        assert(shallowEq(r, [i]));
+
+        r = jaz(i, i, i, i*10);
+        assert(r.length === 2);
+        assert(shallowEq(r, [i, i*10]));
+
+        let o = {};
+        r = jaz(i, i, i, i*10, o);
+        assert(r.length === 3);
+        assert(shallowEq(r, [i, i*10, o]));
+    }
+}
+
+function test6() {
+    "use strict";
+    function baz(...args) {
+        return args;
+    }
+    noInline(baz);
+    function jaz(a, b, ...args) {
+        return baz.apply(null, args);
+    }
+    noInline(jaz);
+
+    for (let i = 0; i < iterations; i++) {
+        let r = jaz();
+        assert(r.length === 0);
+
+        r = jaz(i);
+        assert(r.length === 0);
+
+        r = jaz(i, i);
+        assert(r.length === 0);
+
+        r = jaz(i, i, i);
+        assert(r.length === 1);
+        assert(shallowEq(r, [i]));
+
+        r = jaz(i, i, i, i*10);
+        assert(r.length === 2);
+        assert(shallowEq(r, [i, i*10]));
+
+        let o = {};
+        r = jaz(i, i, i, i*10, o);
+        assert(r.length === 3);
+        assert(shallowEq(r, [i, i*10, o]));
+    }
+}
+
+function test7() {
+    let shouldExit = false;
+    function baz(...args) {
+        if (shouldExit) {
+            OSRExit();
+            return [args.length, args[0], args[1], args[2]];
+        }
+        return [args.length, args[0], args[1], args[2]];
+    }
+    function jaz(a, b, ...args) {
+        return baz.apply(null, args);
+    }
+    noInline(jaz);
+
+    function check(i) {
+        let [length, a, b, c] = jaz();
+        assert(length === 0);
+        assert(a === undefined);
+        assert(b === undefined);
+        assert(c === undefined);
+
+        [length, a, b, c] = jaz(i);
+        assert(length === 0);
+        assert(a === undefined);
+        assert(b === undefined);
+        assert(c === undefined);
+
+        [length, a, b, c] = jaz(i, i);
+        assert(length === 0);
+        assert(a === undefined);
+        assert(b === undefined);
+        assert(c === undefined);
+
+        [length, a, b, c] = jaz(i, i, i);
+        assert(length === 1);
+        assert(a === i, JSON.stringify(a));
+        assert(b === undefined);
+        assert(c === undefined);
+
+        [length, a, b, c] = jaz(i, i, i, i*10);
+        assert(length === 2);
+        assert(a === i);
+        assert(b === i*10);
+        assert(c === undefined);
+
+        let o = {oooo:20};
+        [length, a, b, c] = jaz(i, i, i, i*10, o);
+        assert(length === 3);
+        assert(a === i);
+        assert(b === i*10);
+        assert(c === o);
+    }
+
+    shouldExit = true;
+    for (let i = 0; i < 400; i++) {
+        check(i);
+    }
+
+    shouldExit = false;
+    for (let i = 0; i < iterations; i++) {
+        check(i);
+    }
+
+    shouldExit = false;
+    check(10);
+}
+
+function test8() {
+    let shouldExit = false;
+    function baz(...args) {
+        if (shouldExit) {
+            OSRExit();
+            return args;
+        }
+        return args;
+    }
+    function jaz(a, b, ...args) {
+        return baz.apply(null, args);
+    }
+    noInline(jaz);
+
+    function check(i) {
+        let args = jaz();
+        assert(args.length === 0);
+
+        args = jaz(i);
+        assert(args.length === 0);
+
+        args = jaz(i, i);
+        assert(args.length === 0);
+
+        args = jaz(i, i, i);
+        assert(args.length === 1);
+        assert(args[0] === i);
+
+        args = jaz(i, i, i, i*10);
+        assert(args.length === 2);
+        assert(args[0] === i);
+        assert(args[1] === i*10);
+
+        let o = {oooo:20};
+        args = jaz(i, i, i, i*10, o);
+        assert(args.length === 3);
+        assert(args[0] === i);
+        assert(args[1] === i*10);
+        assert(args[2] === o);
+    }
+
+    shouldExit = true;
+    for (let i = 0; i < 400; i++) {
+        check(i);
+    }
+
+    shouldExit = false;
+    for (let i = 0; i < iterations; i++) {
+        check(i);
+    }
+
+    shouldExit = false;
+    check(10);
+}
+
+function test9() {
+    let shouldExit = false;
+    function baz(a, ...args) {
+        if (shouldExit) {
+            OSRExit();
+            return [args.length, args[0], args[1]];
+        }
+        return [args.length, args[0], args[1]];
+    }
+    function jaz(...args) {
+        return baz.apply(null, args);
+    }
+    noInline(jaz);
+
+    function check(i) {
+        let [length, a, b] = jaz();
+        assert(length === 0);
+
+        [length, a, b] = jaz(i);
+        assert(length === 0);
+        assert(a === undefined);
+        assert(b === undefined);
+
+        [length, a, b] = jaz(i, i + 1);
+        assert(length === 1);
+        assert(a === i+1);
+        assert(b === undefined);
+
+        [length, a, b] = jaz(i, i+1, i+2);
+        assert(length === 2);
+        assert(a === i+1);
+        assert(b === i+2);
+
+        let o = {oooo:20};
+        [length, a, b] = jaz(i, i+1, o);
+        assert(length === 2);
+        assert(a === i+1);
+        assert(b === o);
+    }
+
+    shouldExit = true;
+    for (let i = 0; i < 400; i++) {
+        check(i);
+    }
+
+    shouldExit = false;
+    for (let i = 0; i < iterations; i++) {
+        check(i);
+    }
+
+    shouldExit = false;
+    check(10);
+}
+
+function test10() {
+    function baz(a, b, c, ...args) {
+        return [args.length, args[0], args[1], args[2], args[3]];
+    }
+    function jaz(a, b, c, d, e, f) {
+        return baz(a, b, c, d, e, f);
+    }
+    noInline(jaz);
+
+    for (let i = 0; i < iterations; i++) {
+        let [length, a, b, c, d] = jaz(1, 2, 3, 4, 5, 6);
+        assert(length === 3);
+        assert(a === 4);
+        assert(b === 5);
+        assert(c === 6);
+        assert(d === undefined);
+    }
+}
+
+function test11() {
+    function bar(...args) {
+        return args;
+    }
+    noInline(bar);
+
+    function foo(a, b, c, d, ...args) {
+        return bar.apply(null, args);
+    }
+    noInline(foo);
+
+    function makeArguments(args) {
+        return [1,2,3,4, ...args];
+    }
+    for (let i = 0; i < iterations; i++) {
+        function test() { assert(shallowEq(a, foo.apply(null, makeArguments(a)))); }
+        let a = [{}, 25, 50];
+        test();
+
+        a = [];
+        test();
+
+        a = [{foo: 20}];
+        test();
+
+        a = [10, 20, 30, 40, 50, 60, 70, 80];
+        test();
+    }
+}
+
+function test12() {
+    "use strict";
+    let thisValue = {};
+    function getThisValue() { return thisValue; }
+    noInline(getThisValue);
+
+    function bar(...args) {
+        assert(this === thisValue);
+        return args;
+    }
+    noInline(bar);
+
+    function foo(a, b, c, d, ...args) {
+        return bar.apply(getThisValue(), args);
+    }
+    noInline(foo);
+
+    function makeArguments(args) {
+        return [1,2,3,4, ...args];
+    }
+    for (let i = 0; i < iterations; i++) {
+        function test() { assert(shallowEq(a, foo.apply(null, makeArguments(a)))); }
+        let a = [{}, 25, 50];
+        test();
+
+        a = [];
+        test();
+
+        a = [{foo: 20}];
+        test();
+
+        a = [10, 20, 30, 40, 50, 60, 70, 80];
+        test();
+    }
+}
+
+function test13() {
+    "use strict";
+    function bar(...args) {
+        return args;
+    }
+    noInline(bar);
+
+    function top(a, b, c, d, ...args) {
+        return bar.apply(null, args);
+    }
+    function foo(...args) {
+        let r = top.apply(null, args);
+        return r;
+    }
+    noInline(foo);
+
+    function makeArguments(args) {
+        return [1,2,3,4, ...args];
+    }
+    for (let i = 0; i < iterations; i++) {
+        function test() { assert(shallowEq(a, foo.apply(null, makeArguments(a)))); }
+        let a = [{}, 25, 50];
+        test();
+
+        a = [];
+        test();
+
+        a = [10, 20, 30, 40, 50, 60, 70, 80];
+        test();
+    }
+}
+
+function test14() {
+    "use strict";
+    function bar(...args) {
+        return args;
+    }
+    noInline(bar);
+
+    function top(a, b, c, d, ...args) {
+        return bar.apply(null, args);
+    }
+    function foo(a1, a2, a3, a4, a5, a6, a7, a8, a9, a10, a11, a12, a13, a14, a15, a16, a17) {
+        return top(a1, a2, a3, a4, a5, a6, a7, a8, a9, a10, a11, a12, a13, a14, a15, a16, a17);
+    }
+    noInline(foo);
+
+    function makeArguments(args) {
+        let r = [1,2,3,4, ...args];
+        while (r.length < foo.length)
+            r.push(undefined);
+        return r;
+    }
+    for (let i = 0; i < iterations; i++) {
+        function test()
+        {
+            let args = makeArguments(a);
+            assert(shallowEq(args.slice(4), foo.apply(null, args)));
+        }
+
+        let a = [{}, 25, 50];
+        test();
+
+        a = [];
+        test();
+
+        a = [10, 20, 30, 40, 50, 60, 70, 80];
+        test();
+    }
+}
+
+function test15() {
+    "use strict";
+    function bar(...args) {
+        return args;
+    }
+    noInline(bar);
+
+    function top(a, b, c, d, ...args) {
+        return bar.apply(null, args);
+    }
+    function foo(a1, a2, a3, a4, a5, a6, a7, a8, a9, a10, a11, a12, a13, a14, a15, a16, a17) {
+        let r = top(a1, a2, a3, a4, a5, a6, a7, a8, a9, a10, a11, a12, a13, a14, a15, a16, a17);
+        return r;
+    }
+    noInline(foo);
+
+    function makeArguments(args) {
+        let r = [1,2,3,4, ...args];
+        while (r.length < foo.length)
+            r.push(undefined);
+        return r;
+    }
+    for (let i = 0; i < iterations; i++) {
+        function test()
+        {
+            let args = makeArguments(a);
+            assert(shallowEq(args.slice(4), foo.apply(null, args)));
+        }
+
+        let a = [{}, 25, 50];
+        test();
+
+        a = [];
+        test();
+
+        a = [10, 20, 30, 40, 50, 60, 70, 80];
+        test();
+    }
+}
+
+let start = Date.now();
+test1();
+test2();
+test3();
+test4();
+test5();
+test6();
+test7();
+test8();
+test9();
+test10();
+test11();
+test12();
+test13();
+test14();
+test15();
+const verbose = false;
+if (verbose)
+    print(Date.now() - start);
+
index 645b47e..70f94e8 100644 (file)
@@ -1,3 +1,127 @@
+2016-11-01  Saam Barati  <sbarati@apple.com>
+
+        We should be able to eliminate rest parameter allocations
+        https://bugs.webkit.org/show_bug.cgi?id=163925
+
+        Reviewed by Filip Pizlo.
+
+        This is the first step towards eliminating rest parameter
+        allocations when they're spread to other function calls:
+        `function foo(...args) { bar(...args); }`
+        
+        This patch simply removes the allocation for rest parameter
+        allocations using the same escape analysis that is performed
+        in the argument elimination phase. I've added a new rule to
+        the phase to make sure that CheckStructure doesn't count as
+        an escape for an allocation since this often shows up in code
+        like this:
+        
+        ```
+        function foo(...args) {
+            let r = [];
+            for (let i = 0; i < args.length; i++)
+                r.push(args[i]);
+            return r;
+        }
+        ```
+        
+        The above program now entirely eliminates the allocation for args
+        compiled in the FTL. Programs like this also eliminate the allocation
+        for args:
+        
+        ```
+        function foo(...args) { return [args.length, args[0]]; }
+        
+        function bar(...args) { return someOtherFunction.apply(null, args); }
+        ```
+        
+        This patch extends the arguments elimination phase to understand
+        the concept that we may want to forward arguments, or get from
+        the arguments region, starting at some offset. The offset is the
+        number of names parameter before the rest parameter. For example:
+        
+        ```
+        function foo(a, b, ...args) { return bar.apply(null, args); }
+        ```
+        
+        Will forward arguments starting at the *third* argument.
+        Our arguments forwarding code already had the notion of starting
+        from some offset, however, I found bugs in that code. I extended
+        it to work properly for rest parameters with arbitrary skip offsets.
+        
+        And this program:
+        ```
+        function foo(...args) {
+            let r = [];
+            for (let i = 0; i < args.length; i++)
+                r.push(args[i]);
+            return r;
+        }
+        ```
+        
+        Knows to perform the GetMyArgumentByVal* with an offset of 3
+        inside the loop. To make this work, I taught GetMyArgumentByVal 
+        and GetMyArgumentByValOutOfBounds to take an offset representing
+        the number of arguments to skip.
+        
+        This patch is a ~20% speedup on microbenchmarks.
+
+        * bytecode/CodeBlock.cpp:
+        (JSC::CodeBlock::finishCreation):
+        * dfg/DFGAbstractInterpreterInlines.h:
+        (JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects):
+        * dfg/DFGArgumentsEliminationPhase.cpp:
+        * dfg/DFGArgumentsUtilities.cpp:
+        (JSC::DFG::emitCodeToGetArgumentsArrayLength):
+        * dfg/DFGClobberize.h:
+        (JSC::DFG::clobberize):
+        * dfg/DFGConstantFoldingPhase.cpp:
+        (JSC::DFG::ConstantFoldingPhase::foldConstants):
+        * dfg/DFGDoesGC.cpp:
+        (JSC::DFG::doesGC):
+        * dfg/DFGFixupPhase.cpp:
+        (JSC::DFG::FixupPhase::fixupNode):
+        * dfg/DFGNode.h:
+        (JSC::DFG::Node::hasConstant):
+        (JSC::DFG::Node::constant):
+        (JSC::DFG::Node::isPhantomAllocation):
+        (JSC::DFG::Node::numberOfArgumentsToSkip):
+        * dfg/DFGNodeType.h:
+        * dfg/DFGOSRAvailabilityAnalysisPhase.cpp:
+        (JSC::DFG::LocalOSRAvailabilityCalculator::executeNode):
+        * dfg/DFGOperations.cpp:
+        * dfg/DFGPreciseLocalClobberize.h:
+        (JSC::DFG::PreciseLocalClobberizeAdaptor::readTop):
+        * dfg/DFGPredictionPropagationPhase.cpp:
+        * dfg/DFGSafeToExecute.h:
+        (JSC::DFG::safeToExecute):
+        * dfg/DFGSpeculativeJIT.cpp:
+        (JSC::DFG::SpeculativeJIT::compileCreateRest):
+        * dfg/DFGSpeculativeJIT32_64.cpp:
+        (JSC::DFG::SpeculativeJIT::compile):
+        * dfg/DFGSpeculativeJIT64.cpp:
+        (JSC::DFG::SpeculativeJIT::compile):
+        * dfg/DFGStructureRegistrationPhase.cpp:
+        (JSC::DFG::StructureRegistrationPhase::run):
+        * dfg/DFGValidate.cpp:
+        * ftl/FTLCapabilities.cpp:
+        (JSC::FTL::canCompile):
+        * ftl/FTLLowerDFGToB3.cpp:
+        (JSC::FTL::DFG::LowerDFGToB3::compileNode):
+        (JSC::FTL::DFG::LowerDFGToB3::compileGetMyArgumentByVal):
+        (JSC::FTL::DFG::LowerDFGToB3::compileCreateRest):
+        (JSC::FTL::DFG::LowerDFGToB3::compileForwardVarargs):
+        (JSC::FTL::DFG::LowerDFGToB3::getArgumentsStart):
+        * ftl/FTLOperations.cpp:
+        (JSC::FTL::operationPopulateObjectInOSR):
+        (JSC::FTL::operationMaterializeObjectInOSR):
+        * jit/SetupVarargsFrame.cpp:
+        (JSC::emitSetVarargsFrame):
+        * runtime/CommonSlowPaths.cpp:
+        (JSC::SLOW_PATH_DECL):
+        * runtime/JSGlobalObject.h:
+        (JSC::JSGlobalObject::restParameterStructure):
+
 2016-11-01  Geoffrey Garen  <ggaren@apple.com>
 
         Lots of stderr logging in JSManagedValue
index 586ff8c..1067b22 100644 (file)
@@ -2360,6 +2360,13 @@ void CodeBlock::finishCreation(VM& vm, ScriptExecutable* ownerExecutable, Unlink
             break;
         }
 
+        case op_create_rest: {
+            int numberOfArgumentsToSkip = instructions[i + 3].u.operand;
+            ASSERT_UNUSED(numberOfArgumentsToSkip, numberOfArgumentsToSkip >= 0);
+            ASSERT_WITH_MESSAGE(numberOfArgumentsToSkip == numParameters() - 1, "We assume that this is true when rematerializing the rest parameter during OSR exit in the FTL JIT.");
+            break;
+        }
+
         default:
             break;
         }
index 1d1f308..43b51a1 100644 (file)
@@ -1665,16 +1665,17 @@ bool AbstractInterpreter<AbstractStateType>::executeEffects(unsigned clobberLimi
             // that's the conservative thing to do. Otherwise we'd need to write more code to mark such
             // paths as unreachable, or to return undefined. We could implement that eventually.
             
+            unsigned argumentIndex = index.asUInt32() + node->numberOfArgumentsToSkip();
             if (inlineCallFrame) {
-                if (index.asUInt32() < inlineCallFrame->arguments.size() - 1) {
+                if (argumentIndex < inlineCallFrame->arguments.size() - 1) {
                     forNode(node) = m_state.variables().operand(
-                        virtualRegisterForArgument(index.asInt32() + 1) + inlineCallFrame->stackOffset);
+                        virtualRegisterForArgument(argumentIndex + 1) + inlineCallFrame->stackOffset);
                     m_state.setFoundConstants(true);
                     break;
                 }
             } else {
-                if (index.asUInt32() < m_state.variables().numberOfArguments() - 1) {
-                    forNode(node) = m_state.variables().argument(index.asInt32() + 1);
+                if (argumentIndex < m_state.variables().numberOfArguments() - 1) {
+                    forNode(node) = m_state.variables().argument(argumentIndex + 1);
                     m_state.setFoundConstants(true);
                     break;
                 }
@@ -1685,7 +1686,7 @@ bool AbstractInterpreter<AbstractStateType>::executeEffects(unsigned clobberLimi
             // We have a bound on the types even though it's random access. Take advantage of this.
             
             AbstractValue result;
-            for (unsigned i = inlineCallFrame->arguments.size(); i-- > 1;) {
+            for (unsigned i = 1 + node->numberOfArgumentsToSkip(); i < inlineCallFrame->arguments.size(); ++i) {
                 result.merge(
                     m_state.variables().operand(
                         virtualRegisterForArgument(i) + inlineCallFrame->stackOffset));
@@ -1950,6 +1951,7 @@ bool AbstractInterpreter<AbstractStateType>::executeEffects(unsigned clobberLimi
     case PhantomCreateActivation:
     case PhantomDirectArguments:
     case PhantomClonedArguments:
+    case PhantomCreateRest:
     case BottomValue:
         m_state.setDidClobber(true); // Prevent constant folding.
         // This claims to return bottom.
@@ -2865,9 +2867,15 @@ bool AbstractInterpreter<AbstractStateType>::executeEffects(unsigned clobberLimi
         break;
 
     case CreateRest:
-        if (!m_graph.isWatchingHavingABadTimeWatchpoint(node)) // This means we're already having a bad time.
+        if (!m_graph.isWatchingHavingABadTimeWatchpoint(node)) {
+            // This means we're already having a bad time.
             clobberWorld(node->origin.semantic, clobberLimit);
-        forNode(node).setType(m_graph, SpecArray);
+            forNode(node).setType(m_graph, SpecArray);
+            break;
+        }
+        forNode(node).set(
+            m_graph,
+            m_graph.globalObjectFor(node->origin.semantic)->restParameterStructure());
         break;
             
     case Check: {
index bdf2741..97879bb 100644 (file)
@@ -97,6 +97,17 @@ private:
                 case CreateClonedArguments:
                     m_candidates.add(node);
                     break;
+
+                case CreateRest:
+                    if (m_graph.isWatchingHavingABadTimeWatchpoint(node)) {
+                        // If we're watching the HavingABadTime watchpoint it means that we will be invalidated
+                        // when it fires (it may or may not have actually fired yet). We don't try to eliminate
+                        // this allocation when we're not watching the watchpoint because it could entail calling
+                        // indexed accessors (and probably more crazy things) on out of bound accesses to the
+                        // rest parameter. It's also much easier to reason about this way.
+                        m_candidates.add(node);
+                    }
+                    break;
                     
                 case CreateScopedArguments:
                     // FIXME: We could handle this if it wasn't for the fact that scoped arguments are
@@ -134,7 +145,7 @@ private:
                 break;
             
             case Array::Contiguous: {
-                if (edge->op() != CreateClonedArguments) {
+                if (edge->op() != CreateClonedArguments && edge->op() != CreateRest) {
                     escape(edge, source);
                     return;
                 }
@@ -222,6 +233,32 @@ private:
                     escapeBasedOnArrayMode(node->arrayMode(), node->child1(), node);
                     break;
 
+                case CheckStructure: {
+                    if (!m_candidates.contains(node->child1().node()))
+                        break;
+
+                    Structure* structure = nullptr;
+                    JSGlobalObject* globalObject = m_graph.globalObjectFor(node->child1().node()->origin.semantic);
+                    switch (node->child1().node()->op()) {
+                    case CreateDirectArguments:
+                        structure = globalObject->directArgumentsStructure();
+                        break;
+                    case CreateClonedArguments:
+                        structure = globalObject->clonedArgumentsStructure();
+                        break;
+                    case CreateRest:
+                        ASSERT(m_graph.isWatchingHavingABadTimeWatchpoint(node));
+                        structure = globalObject->restParameterStructure();
+                        break;
+                    default:
+                        RELEASE_ASSERT_NOT_REACHED();
+                    }
+                    ASSERT(structure);
+
+                    if (!node->structureSet().contains(structure))
+                        escape(node->child1(), node);
+                    break;
+                }
                     
                 // FIXME: We should be able to handle GetById/GetByOffset on callee.
                 // https://bugs.webkit.org/show_bug.cgi?id=143075
@@ -395,7 +432,7 @@ private:
     {
         InsertionSet insertionSet(m_graph);
         
-        for (BasicBlock* block : m_graph.blocksInNaturalOrder()) {
+        for (BasicBlock* block : m_graph.blocksInPreOrder()) {
             for (unsigned nodeIndex = 0; nodeIndex < block->size(); ++nodeIndex) {
                 Node* node = block->at(nodeIndex);
                 
@@ -403,6 +440,16 @@ private:
                     return emitCodeToGetArgumentsArrayLength(
                         insertionSet, candidate, nodeIndex, node->origin);
                 };
+
+                auto isEliminatedAllocation = [&] (Node* candidate) -> bool {
+                    if (!m_candidates.contains(candidate))
+                        return false;
+                    // We traverse in such a way that we are guaranteed to see a def before a use.
+                    // Therefore, we should have already transformed the allocation before the use
+                    // of an allocation.
+                    ASSERT(candidate->op() == PhantomCreateRest || candidate->op() == PhantomDirectArguments || candidate->op() == PhantomClonedArguments);
+                    return true;
+                };
         
                 switch (node->op()) {
                 case CreateDirectArguments:
@@ -411,6 +458,16 @@ private:
                     
                     node->setOpAndDefaultFlags(PhantomDirectArguments);
                     break;
+
+                case CreateRest:
+                    if (!m_candidates.contains(node))
+                        break;
+
+                    node->setOpAndDefaultFlags(PhantomCreateRest);
+                    // We don't need this parameter for OSR exit, we can find out all the information
+                    // we need via the static parameter count and the dynamic argument count.
+                    node->child1() = Edge(); 
+                    break;
                     
                 case CreateClonedArguments:
                     if (!m_candidates.contains(node))
@@ -421,13 +478,11 @@ private:
                     
                 case GetFromArguments: {
                     Node* candidate = node->child1().node();
-                    if (!m_candidates.contains(candidate))
+                    if (!isEliminatedAllocation(candidate))
                         break;
                     
                     DFG_ASSERT(
-                        m_graph, node,
-                        node->child1()->op() == CreateDirectArguments
-                        || node->child1()->op() == PhantomDirectArguments);
+                        m_graph, node, node->child1()->op() == PhantomDirectArguments);
                     VirtualRegister reg =
                         virtualRegisterForArgument(node->capturedArgumentsOffset().offset() + 1) +
                         node->origin.semantic.stackOffset();
@@ -438,10 +493,10 @@ private:
 
                 case GetByOffset: {
                     Node* candidate = node->child2().node();
-                    if (!m_candidates.contains(candidate))
+                    if (!isEliminatedAllocation(candidate))
                         break;
 
-                    if (node->child2()->op() != PhantomClonedArguments && node->child2()->op() != CreateClonedArguments)
+                    if (node->child2()->op() != PhantomClonedArguments)
                         break;
 
                     ASSERT(node->storageAccessData().offset == clonedArgumentsLengthPropertyOffset);
@@ -454,7 +509,7 @@ private:
                     
                 case GetArrayLength: {
                     Node* candidate = node->child1().node();
-                    if (!m_candidates.contains(candidate))
+                    if (!isEliminatedAllocation(candidate))
                         break;
                     
                     // Meh, this is kind of hackish - we use an Identity so that we can reuse the
@@ -472,13 +527,18 @@ private:
                     // https://bugs.webkit.org/show_bug.cgi?id=143076
                     
                     Node* candidate = node->child1().node();
-                    if (!m_candidates.contains(candidate))
+                    if (!isEliminatedAllocation(candidate))
                         break;
+
+                    unsigned numberOfArgumentsToSkip = 0;
+                    if (candidate->op() == PhantomCreateRest)
+                        numberOfArgumentsToSkip = candidate->numberOfArgumentsToSkip();
                     
                     Node* result = nullptr;
                     if (node->child2()->isInt32Constant()) {
                         unsigned index = node->child2()->asUInt32();
                         InlineCallFrame* inlineCallFrame = candidate->origin.semantic.inlineCallFrame;
+                        index += numberOfArgumentsToSkip;
                         
                         bool safeToGetStack;
                         if (inlineCallFrame)
@@ -512,10 +572,10 @@ private:
                         else
                             op = GetMyArgumentByValOutOfBounds;
                         result = insertionSet.insertNode(
-                            nodeIndex, node->prediction(), op, node->origin,
+                            nodeIndex, node->prediction(), op, node->origin, OpInfo(numberOfArgumentsToSkip),
                             node->child1(), node->child2());
                     }
-                    
+
                     // Need to do this because we may have a data format conversion here.
                     node->convertToIdentityOn(result);
                     break;
@@ -523,83 +583,95 @@ private:
                     
                 case LoadVarargs: {
                     Node* candidate = node->child1().node();
-                    if (!m_candidates.contains(candidate))
+                    if (!isEliminatedAllocation(candidate))
                         break;
                     
                     LoadVarargsData* varargsData = node->loadVarargsData();
+                    unsigned numberOfArgumentsToSkip = 0;
+                    if (candidate->op() == PhantomCreateRest)
+                        numberOfArgumentsToSkip = candidate->numberOfArgumentsToSkip();
+                    varargsData->offset += numberOfArgumentsToSkip;
+
                     InlineCallFrame* inlineCallFrame = candidate->origin.semantic.inlineCallFrame;
+
                     if (inlineCallFrame
-                        && !inlineCallFrame->isVarargs()
-                        && inlineCallFrame->arguments.size() - varargsData->offset <= varargsData->limit) {
-                        
-                        // LoadVarargs can exit, so it better be exitOK.
-                        DFG_ASSERT(m_graph, node, node->origin.exitOK);
-                        bool canExit = true;
-                        
-                        Node* argumentCount = insertionSet.insertConstant(
-                            nodeIndex, node->origin.withExitOK(canExit),
-                            jsNumber(inlineCallFrame->arguments.size() - varargsData->offset));
-                        insertionSet.insertNode(
-                            nodeIndex, SpecNone, MovHint, node->origin.takeValidExit(canExit),
-                            OpInfo(varargsData->count.offset()), Edge(argumentCount));
-                        insertionSet.insertNode(
-                            nodeIndex, SpecNone, PutStack, node->origin.withExitOK(canExit),
-                            OpInfo(m_graph.m_stackAccessData.add(varargsData->count, FlushedInt32)),
-                            Edge(argumentCount, KnownInt32Use));
-                        
-                        DFG_ASSERT(m_graph, node, varargsData->limit - 1 >= varargsData->mandatoryMinimum);
-                        // Define our limit to not include "this", since that's a bit easier to reason about.
-                        unsigned limit = varargsData->limit - 1;
-                        Node* undefined = nullptr;
-                        for (unsigned storeIndex = 0; storeIndex < limit; ++storeIndex) {
-                            // First determine if we have an element we can load, and load it if
-                            // possible.
+                        && !inlineCallFrame->isVarargs()) {
+
+                        unsigned argumentCountIncludingThis = inlineCallFrame->arguments.size();
+                        if (argumentCountIncludingThis > varargsData->offset)
+                            argumentCountIncludingThis -= varargsData->offset;
+                        else
+                            argumentCountIncludingThis = 1;
+                        RELEASE_ASSERT(argumentCountIncludingThis >= 1);
+
+                        if (argumentCountIncludingThis <= varargsData->limit) {
+                            // LoadVarargs can exit, so it better be exitOK.
+                            DFG_ASSERT(m_graph, node, node->origin.exitOK);
+                            bool canExit = true;
                             
-                            unsigned loadIndex = storeIndex + varargsData->offset;
+                            Node* argumentCountIncludingThisNode = insertionSet.insertConstant(
+                                nodeIndex, node->origin.withExitOK(canExit),
+                                jsNumber(argumentCountIncludingThis));
+                            insertionSet.insertNode(
+                                nodeIndex, SpecNone, MovHint, node->origin.takeValidExit(canExit),
+                                OpInfo(varargsData->count.offset()), Edge(argumentCountIncludingThisNode));
+                            insertionSet.insertNode(
+                                nodeIndex, SpecNone, PutStack, node->origin.withExitOK(canExit),
+                                OpInfo(m_graph.m_stackAccessData.add(varargsData->count, FlushedInt32)),
+                                Edge(argumentCountIncludingThisNode, KnownInt32Use));
                             
-                            Node* value;
-                            if (loadIndex + 1 < inlineCallFrame->arguments.size()) {
-                                VirtualRegister reg =
-                                    virtualRegisterForArgument(loadIndex + 1) +
-                                    inlineCallFrame->stackOffset;
-                                StackAccessData* data = m_graph.m_stackAccessData.add(
-                                    reg, FlushedJSValue);
+                            DFG_ASSERT(m_graph, node, varargsData->limit - 1 >= varargsData->mandatoryMinimum);
+                            // Define our limit to exclude "this", since that's a bit easier to reason about.
+                            unsigned limit = varargsData->limit - 1;
+                            Node* undefined = nullptr;
+                            for (unsigned storeIndex = 0; storeIndex < limit; ++storeIndex) {
+                                // First determine if we have an element we can load, and load it if
+                                // possible.
                                 
-                                value = insertionSet.insertNode(
-                                    nodeIndex, SpecNone, GetStack, node->origin.withExitOK(canExit),
-                                    OpInfo(data));
-                            } else {
-                                // FIXME: We shouldn't have to store anything if
-                                // storeIndex >= varargsData->mandatoryMinimum, but we will still
-                                // have GetStacks in that range. So if we don't do the stores, we'll
-                                // have degenerate IR: we'll have GetStacks of something that didn't
-                                // have PutStacks.
-                                // https://bugs.webkit.org/show_bug.cgi?id=147434
+                                unsigned loadIndex = storeIndex + varargsData->offset;
                                 
-                                if (!undefined) {
-                                    undefined = insertionSet.insertConstant(
-                                        nodeIndex, node->origin.withExitOK(canExit), jsUndefined());
+                                Node* value;
+                                if (loadIndex + 1 < inlineCallFrame->arguments.size()) {
+                                    VirtualRegister reg = virtualRegisterForArgument(loadIndex + 1) + inlineCallFrame->stackOffset;
+                                    StackAccessData* data = m_graph.m_stackAccessData.add(
+                                        reg, FlushedJSValue);
+                                    
+                                    value = insertionSet.insertNode(
+                                        nodeIndex, SpecNone, GetStack, node->origin.withExitOK(canExit),
+                                        OpInfo(data));
+                                } else {
+                                    // FIXME: We shouldn't have to store anything if
+                                    // storeIndex >= varargsData->mandatoryMinimum, but we will still
+                                    // have GetStacks in that range. So if we don't do the stores, we'll
+                                    // have degenerate IR: we'll have GetStacks of something that didn't
+                                    // have PutStacks.
+                                    // https://bugs.webkit.org/show_bug.cgi?id=147434
+                                    
+                                    if (!undefined) {
+                                        undefined = insertionSet.insertConstant(
+                                            nodeIndex, node->origin.withExitOK(canExit), jsUndefined());
+                                    }
+                                    value = undefined;
                                 }
-                                value = undefined;
+                                
+                                // Now that we have a value, store it.
+                                
+                                VirtualRegister reg = varargsData->start + storeIndex;
+                                StackAccessData* data =
+                                    m_graph.m_stackAccessData.add(reg, FlushedJSValue);
+                                
+                                insertionSet.insertNode(
+                                    nodeIndex, SpecNone, MovHint, node->origin.takeValidExit(canExit),
+                                    OpInfo(reg.offset()), Edge(value));
+                                insertionSet.insertNode(
+                                    nodeIndex, SpecNone, PutStack, node->origin.withExitOK(canExit),
+                                    OpInfo(data), Edge(value));
                             }
                             
-                            // Now that we have a value, store it.
-                            
-                            VirtualRegister reg = varargsData->start + storeIndex;
-                            StackAccessData* data =
-                                m_graph.m_stackAccessData.add(reg, FlushedJSValue);
-                            
-                            insertionSet.insertNode(
-                                nodeIndex, SpecNone, MovHint, node->origin.takeValidExit(canExit),
-                                OpInfo(reg.offset()), Edge(value));
-                            insertionSet.insertNode(
-                                nodeIndex, SpecNone, PutStack, node->origin.withExitOK(canExit),
-                                OpInfo(data), Edge(value));
+                            node->remove();
+                            node->origin.exitOK = canExit;
+                            break;
                         }
-                        
-                        node->remove();
-                        node->origin.exitOK = canExit;
-                        break;
                     }
                     
                     node->setOpAndDefaultFlags(ForwardVarargs);
@@ -611,10 +683,15 @@ private:
                 case TailCallVarargs:
                 case TailCallVarargsInlinedCaller: {
                     Node* candidate = node->child3().node();
-                    if (!m_candidates.contains(candidate))
+                    if (!isEliminatedAllocation(candidate))
                         break;
                     
+                    unsigned numberOfArgumentsToSkip = 0;
+                    if (candidate->op() == PhantomCreateRest)
+                        numberOfArgumentsToSkip = candidate->numberOfArgumentsToSkip();
                     CallVarargsData* varargsData = node->callVarargsData();
+                    varargsData->firstVarArgOffset += numberOfArgumentsToSkip;
+
                     InlineCallFrame* inlineCallFrame = candidate->origin.semantic.inlineCallFrame;
                     if (inlineCallFrame && !inlineCallFrame->isVarargs()) {
                         Vector<Node*> arguments;
@@ -677,11 +754,18 @@ private:
                     
                 case CheckArray:
                 case GetButterfly: {
-                    if (!m_candidates.contains(node->child1().node()))
+                    if (!isEliminatedAllocation(node->child1().node()))
                         break;
                     node->remove();
                     break;
                 }
+
+                case CheckStructure:
+                    if (!isEliminatedAllocation(node->child1().node()))
+                        break;
+                    node->child1() = Edge(); // Remove the cell check since we've proven it's not needed and FTL lowering might botch this.
+                    node->remove();
+                    break;
                     
                 default:
                     break;
index 022e3b0..061a284 100644 (file)
@@ -64,14 +64,23 @@ Node* emitCodeToGetArgumentsArrayLength(
     DFG_ASSERT(
         graph, arguments,
         arguments->op() == CreateDirectArguments || arguments->op() == CreateScopedArguments
-        || arguments->op() == CreateClonedArguments || arguments->op() == PhantomDirectArguments
-        || arguments->op() == PhantomClonedArguments);
+        || arguments->op() == CreateClonedArguments || arguments->op() == CreateRest
+        || arguments->op() == PhantomDirectArguments || arguments->op() == PhantomClonedArguments || arguments->op() == PhantomCreateRest);
     
     InlineCallFrame* inlineCallFrame = arguments->origin.semantic.inlineCallFrame;
+
+    unsigned numberOfArgumentsToSkip = 0;
+    if (arguments->op() == CreateRest || arguments->op() == PhantomCreateRest)
+        numberOfArgumentsToSkip = arguments->numberOfArgumentsToSkip();
     
     if (inlineCallFrame && !inlineCallFrame->isVarargs()) {
+        unsigned argumentsSize = inlineCallFrame->arguments.size() - 1;
+        if (argumentsSize >= numberOfArgumentsToSkip)
+            argumentsSize -= numberOfArgumentsToSkip;
+        else
+            argumentsSize = 0;
         return insertionSet.insertConstant(
-            nodeIndex, origin, jsNumber(inlineCallFrame->arguments.size() - 1));
+            nodeIndex, origin, jsNumber(argumentsSize));
     }
     
     Node* argumentCount;
@@ -84,12 +93,23 @@ Node* emitCodeToGetArgumentsArrayLength(
             nodeIndex, SpecInt32Only, GetStack, origin,
             OpInfo(graph.m_stackAccessData.add(argumentCountRegister, FlushedInt32)));
     }
-    
-    return insertionSet.insertNode(
+
+    Node* result = insertionSet.insertNode(
         nodeIndex, SpecInt32Only, ArithSub, origin, OpInfo(Arith::Unchecked),
         Edge(argumentCount, Int32Use),
         insertionSet.insertConstantForUse(
-            nodeIndex, origin, jsNumber(1), Int32Use));
+            nodeIndex, origin, jsNumber(1 + numberOfArgumentsToSkip), Int32Use));
+
+    if (numberOfArgumentsToSkip) {
+        // The above subtraction may produce a negative number if this number is non-zero. We correct that here.
+        result = insertionSet.insertNode(
+            nodeIndex, SpecInt32Only, ArithMax, origin, 
+            Edge(result, Int32Use), 
+            insertionSet.insertConstantForUse(nodeIndex, origin, jsNumber(0), Int32Use));
+        result->setResult(NodeResultInt32);
+    }
+
+    return result;
 }
 
 } } // namespace JSC::DFG
index 5a3294d..b621ef0 100644 (file)
@@ -477,6 +477,12 @@ void clobberize(Graph& graph, Node* node, const ReadFunctor& read, const WriteFu
         write(HeapObjectCount);
         return;
 
+    case PhantomCreateRest:
+        // Even though it's phantom, it still has the property that one can't be replaced with another.
+        read(HeapObjectCount);
+        write(HeapObjectCount);
+        return;
+
     case CallObjectConstructor:
     case ToThis:
     case CreateThis:
index 38b6402..321dcf9 100644 (file)
@@ -314,9 +314,11 @@ private:
                 
             case GetMyArgumentByVal:
             case GetMyArgumentByValOutOfBounds: {
-                JSValue index = m_state.forNode(node->child2()).value();
-                if (!index || !index.isInt32())
+                JSValue indexValue = m_state.forNode(node->child2()).value();
+                if (!indexValue || !indexValue.isInt32())
                     break;
+
+                unsigned index = indexValue.asUInt32() + node->numberOfArgumentsToSkip();
                 
                 Node* arguments = node->child1().node();
                 InlineCallFrame* inlineCallFrame = arguments->origin.semantic.inlineCallFrame;
@@ -335,10 +337,10 @@ private:
                 // GetMyArgumentByVal in such statically-out-of-bounds accesses; we just lose CFA unless
                 // GCSE removes the access entirely.
                 if (inlineCallFrame) {
-                    if (index.asUInt32() >= inlineCallFrame->arguments.size() - 1)
+                    if (index >= inlineCallFrame->arguments.size() - 1)
                         break;
                 } else {
-                    if (index.asUInt32() >= m_state.variables().numberOfArguments() - 1)
+                    if (index >= m_state.variables().numberOfArguments() - 1)
                         break;
                 }
                 
@@ -349,15 +351,14 @@ private:
                     data = m_graph.m_stackAccessData.add(
                         VirtualRegister(
                             inlineCallFrame->stackOffset +
-                            CallFrame::argumentOffset(index.asInt32())),
+                            CallFrame::argumentOffset(index)),
                         FlushedJSValue);
                 } else {
                     data = m_graph.m_stackAccessData.add(
-                        virtualRegisterForArgument(index.asInt32() + 1), FlushedJSValue);
+                        virtualRegisterForArgument(index + 1), FlushedJSValue);
                 }
                 
-                if (inlineCallFrame && !inlineCallFrame->isVarargs()
-                    && index.asUInt32() < inlineCallFrame->arguments.size() - 1) {
+                if (inlineCallFrame && !inlineCallFrame->isVarargs() && index < inlineCallFrame->arguments.size() - 1) {
                     node->convertToGetStack(data);
                     eliminated = true;
                     break;
index cb48e5c..97cd848 100644 (file)
@@ -246,6 +246,7 @@ bool doesGC(Graph& graph, Node* node)
     case PhantomNewGeneratorFunction:
     case PhantomCreateActivation:
     case PhantomDirectArguments:
+    case PhantomCreateRest:
     case PhantomClonedArguments:
     case GetMyArgumentByVal:
     case GetMyArgumentByValOutOfBounds:
index 78fe0e9..1d4e907 100644 (file)
@@ -1399,6 +1399,7 @@ private:
         case PhantomNewGeneratorFunction:
         case PhantomCreateActivation:
         case PhantomDirectArguments:
+        case PhantomCreateRest:
         case PhantomClonedArguments:
         case GetMyArgumentByVal:
         case GetMyArgumentByValOutOfBounds:
index a67b293..88f8811 100644 (file)
@@ -458,6 +458,7 @@ public:
             
         case PhantomDirectArguments:
         case PhantomClonedArguments:
+        case PhantomCreateRest:
             // These pretend to be the empty value constant for the benefit of the DFG backend, which
             // otherwise wouldn't take kindly to a node that doesn't compute a value.
             return true;
@@ -471,7 +472,7 @@ public:
     {
         ASSERT(hasConstant());
         
-        if (op() == PhantomDirectArguments || op() == PhantomClonedArguments) {
+        if (op() == PhantomDirectArguments || op() == PhantomClonedArguments || op() == PhantomCreateRest) {
             // These pretend to be the empty value constant for the benefit of the DFG backend, which
             // otherwise wouldn't take kindly to a node that doesn't compute a value.
             return FrozenValue::emptySingleton();
@@ -1745,6 +1746,7 @@ public:
         switch (op()) {
         case PhantomNewObject:
         case PhantomDirectArguments:
+        case PhantomCreateRest:
         case PhantomClonedArguments:
         case PhantomNewFunction:
         case PhantomNewGeneratorFunction:
@@ -2388,7 +2390,7 @@ public:
 
     unsigned numberOfArgumentsToSkip()
     {
-        ASSERT(op() == CreateRest || op() == GetRestLength);
+        ASSERT(op() == CreateRest || op() == PhantomCreateRest || op() == GetRestLength || op() == GetMyArgumentByVal || op() == GetMyArgumentByValOutOfBounds);
         return m_opInfo.as<unsigned>();
     }
 
index cd24a44..d975b76 100644 (file)
@@ -342,6 +342,7 @@ namespace JSC { namespace DFG {
     \
     macro(CreateDirectArguments, NodeResultJS) \
     macro(PhantomDirectArguments, NodeResultJS | NodeMustGenerate) \
+    macro(PhantomCreateRest, NodeResultJS | NodeMustGenerate) \
     macro(CreateScopedArguments, NodeResultJS) \
     macro(CreateClonedArguments, NodeResultJS) \
     macro(PhantomClonedArguments, NodeResultJS | NodeMustGenerate) \
index eb217e6..c188775 100644 (file)
@@ -165,6 +165,7 @@ void LocalOSRAvailabilityCalculator::executeNode(Node* node)
         break;
     }
         
+    case PhantomCreateRest:
     case PhantomDirectArguments:
     case PhantomClonedArguments: {
         InlineCallFrame* inlineCallFrame = node->origin.semantic.inlineCallFrame;
@@ -173,6 +174,10 @@ void LocalOSRAvailabilityCalculator::executeNode(Node* node)
             // given that we can read them from the stack.
             break;
         }
+
+        unsigned numberOfArgumentsToSkip = 0;
+        if (node->op() == PhantomCreateRest)
+            numberOfArgumentsToSkip = node->numberOfArgumentsToSkip();
         
         if (inlineCallFrame->isVarargs()) {
             // Record how to read each argument and the argument count.
@@ -188,7 +193,7 @@ void LocalOSRAvailabilityCalculator::executeNode(Node* node)
             m_availability.m_heap.set(PromotedHeapLocation(ArgumentsCalleePLoc, node), callee);
         }
         
-        for (unsigned i = 0; i < inlineCallFrame->arguments.size() - 1; ++i) {
+        for (unsigned i = numberOfArgumentsToSkip; i < inlineCallFrame->arguments.size() - 1; ++i) {
             Availability argument = m_availability.m_locals.operand(
                 inlineCallFrame->stackOffset + CallFrame::argumentOffset(i));
             
index fdf689d..e204d3e 100644 (file)
@@ -1358,7 +1358,7 @@ JSCell* JIT_OPERATION operationCreateRest(ExecState* exec, Register* argumentSta
     NativeCallFrameTracer tracer(vm, exec);
 
     JSGlobalObject* globalObject = exec->lexicalGlobalObject();
-    Structure* structure = globalObject->arrayStructureForIndexingTypeDuringAllocation(ArrayWithContiguous);
+    Structure* structure = globalObject->restParameterStructure();
     static_assert(sizeof(Register) == sizeof(JSValue), "This is a strong assumption here.");
     JSValue* argumentsToCopyRegion = bitwise_cast<JSValue*>(argumentStart) + numberOfParamsToSkip;
     return constructArray(exec, structure, argumentsToCopyRegion, arraySize);
index 70eb75f..6fbd7bd 100644 (file)
@@ -120,15 +120,24 @@ private:
                 inlineCallFrame = m_node->argumentsChild()->origin.semantic.inlineCallFrame;
             else
                 inlineCallFrame = m_node->origin.semantic.inlineCallFrame;
+
+            unsigned numberOfArgumentsToSkip = 0;
+            if (m_node->op() == GetMyArgumentByVal || m_node->op() == GetMyArgumentByValOutOfBounds) {
+                // The value of numberOfArgumentsToSkip guarantees that GetMyArgumentByVal* will never
+                // read any arguments below the number of arguments to skip. For example, if numberOfArgumentsToSkip is 2,
+                // we will never read argument 0 or argument 1.
+                numberOfArgumentsToSkip = m_node->numberOfArgumentsToSkip();
+            }
+
             if (!inlineCallFrame) {
                 // Read the outermost arguments and argument count.
-                for (unsigned i = m_graph.m_codeBlock->numParameters(); i-- > 1;)
+                for (unsigned i = 1 + numberOfArgumentsToSkip; i < static_cast<unsigned>(m_graph.m_codeBlock->numParameters()); i++)
                     m_read(virtualRegisterForArgument(i));
                 m_read(VirtualRegister(CallFrameSlot::argumentCount));
                 break;
             }
             
-            for (unsigned i = inlineCallFrame->arguments.size(); i-- > 1;)
+            for (unsigned i = 1 + numberOfArgumentsToSkip; i < inlineCallFrame->arguments.size(); i++)
                 m_read(VirtualRegister(inlineCallFrame->stackOffset + virtualRegisterForArgument(i).offset()));
             if (inlineCallFrame->isVarargs())
                 m_read(VirtualRegister(inlineCallFrame->stackOffset + CallFrameSlot::argumentCount));
index 397e999..6ed729b 100644 (file)
@@ -1004,6 +1004,7 @@ private:
         case PhantomNewGeneratorFunction:
         case PhantomCreateActivation:
         case PhantomDirectArguments:
+        case PhantomCreateRest:
         case PhantomClonedArguments:
         case GetMyArgumentByVal:
         case GetMyArgumentByValOutOfBounds:
index 0a2ecd8..2c99802 100644 (file)
@@ -354,6 +354,7 @@ bool safeToExecute(AbstractStateType& state, Graph& graph, Node* node)
     case MaterializeNewObject:
     case MaterializeCreateActivation:
     case PhantomDirectArguments:
+    case PhantomCreateRest:
     case PhantomClonedArguments:
     case GetMyArgumentByVal:
     case GetMyArgumentByValOutOfBounds:
index 7b57b18..2732fe8 100644 (file)
@@ -6852,6 +6852,7 @@ void SpeculativeJIT::compileCreateRest(Node* node)
         GPRReg arrayResultGPR = arrayResult.gpr();
 
         bool shouldAllowForArrayStorageStructureForLargeArrays = false;
+        ASSERT(m_jit.graph().globalObjectFor(node->origin.semantic)->restParameterStructure()->indexingType() == ArrayWithContiguous);
         compileAllocateNewArrayWithSize(m_jit.graph().globalObjectFor(node->origin.semantic), arrayResultGPR, arrayLengthGPR, ArrayWithContiguous, shouldAllowForArrayStorageStructureForLargeArrays);
 
         GPRTemporary argumentsStart(this);
index 8f72b1e..cb04a5a 100644 (file)
@@ -5601,6 +5601,7 @@ void SpeculativeJIT::compile(Node* node)
     case GetStack:
     case GetMyArgumentByVal:
     case GetMyArgumentByValOutOfBounds:
+    case PhantomCreateRest:
         DFG_CRASH(m_jit.graph(), node, "unexpected node in DFG backend");
         break;
     }
index f89c1e6..b41c6f3 100644 (file)
@@ -5818,6 +5818,7 @@ void SpeculativeJIT::compile(Node* node)
     case PutStack:
     case KillStack:
     case GetStack:
+    case PhantomCreateRest:
         DFG_CRASH(m_jit.graph(), node, "Unexpected node");
         break;
     }
index 51cf489..86b50dd 100644 (file)
@@ -118,6 +118,14 @@ public:
                     registerStructure(globalObject->originalArrayStructureForIndexingType(ArrayWithSlowPutArrayStorage));
                     break;
                 }
+
+                case CreateRest: {
+                    if (m_graph.isWatchingHavingABadTimeWatchpoint(node)) {
+                        JSGlobalObject* globalObject = m_graph.globalObjectFor(node->origin.semantic);
+                        registerStructure(globalObject->restParameterStructure());
+                    }
+                    break;
+                }
                     
                 case NewTypedArray:
                     registerStructure(m_graph.globalObjectFor(node->origin.semantic)->typedArrayStructureConcurrently(node->typedArrayType()));
index 918d8ad..423301c 100644 (file)
@@ -670,6 +670,7 @@ private:
                 case PhantomNewGeneratorFunction:
                 case PhantomCreateActivation:
                 case PhantomDirectArguments:
+                case PhantomCreateRest:
                 case PhantomClonedArguments:
                 case MovHint:
                 case Upsilon:
index 3af1cb7..54f4e72 100644 (file)
@@ -230,6 +230,7 @@ inline CapabilityLevel canCompile(Node* node)
     case MaterializeNewObject:
     case MaterializeCreateActivation:
     case PhantomDirectArguments:
+    case PhantomCreateRest:
     case PhantomClonedArguments:
     case GetMyArgumentByVal:
     case GetMyArgumentByValOutOfBounds:
index 3f09dcf..ad16157 100644 (file)
@@ -1074,6 +1074,7 @@ private:
         case PhantomNewGeneratorFunction:
         case PhantomCreateActivation:
         case PhantomDirectArguments:
+        case PhantomCreateRest:
         case PhantomClonedArguments:
         case PutHint:
         case BottomValue:
@@ -3434,6 +3435,8 @@ private:
         InlineCallFrame* inlineCallFrame = m_node->child1()->origin.semantic.inlineCallFrame;
         
         LValue index = lowInt32(m_node->child2());
+        if (m_node->numberOfArgumentsToSkip())
+            index = m_out.add(index, m_out.constInt32(m_node->numberOfArgumentsToSkip()));
         
         LValue limit;
         if (inlineCallFrame && !inlineCallFrame->isVarargs())
@@ -4157,7 +4160,7 @@ private:
             LValue arrayLength = lowInt32(m_node->child1());
             LBasicBlock loopStart = m_out.newBlock();
             JSGlobalObject* globalObject = m_graph.globalObjectFor(m_node->origin.semantic);
-            Structure* structure = globalObject->arrayStructureForIndexingTypeDuringAllocation(ArrayWithContiguous);
+            Structure* structure = globalObject->restParameterStructure();
             ArrayValues arrayValues = allocateUninitializedContiguousJSArray(arrayLength, structure);
             LValue array = arrayValues.array;
             LValue butterfly = arrayValues.butterfly;
@@ -6285,17 +6288,47 @@ private:
             inlineCallFrame = m_node->child1()->origin.semantic.inlineCallFrame;
         else
             inlineCallFrame = m_node->origin.semantic.inlineCallFrame;
-        
-        LValue length = getArgumentsLength(inlineCallFrame).value;
-        LValue lengthIncludingThis = m_out.add(length, m_out.constInt32(1 - data->offset));
-        
+
+        LValue length = nullptr; 
+        LValue lengthIncludingThis = nullptr;
+        ArgumentsLength argumentsLength = getArgumentsLength(inlineCallFrame);
+        if (argumentsLength.isKnown) {
+            unsigned knownLength = argumentsLength.known;
+            if (knownLength >= data->offset)
+                knownLength = knownLength - data->offset;
+            else
+                knownLength = 0;
+            length = m_out.constInt32(knownLength);
+            lengthIncludingThis = m_out.constInt32(knownLength + 1);
+        } else {
+            // We need to perform the same logical operation as the code above, but through dynamic operations.
+            if (!data->offset)
+                length = argumentsLength.value;
+            else {
+                LBasicBlock isLarger = m_out.newBlock();
+                LBasicBlock continuation = m_out.newBlock();
+
+                ValueFromBlock smallerOrEqualLengthResult = m_out.anchor(m_out.constInt32(0));
+                m_out.branch(
+                    m_out.above(argumentsLength.value, m_out.constInt32(data->offset)), unsure(isLarger), unsure(continuation));
+                LBasicBlock lastNext = m_out.appendTo(isLarger, continuation);
+                ValueFromBlock largerLengthResult = m_out.anchor(m_out.sub(argumentsLength.value, m_out.constInt32(data->offset)));
+                m_out.jump(continuation);
+
+                m_out.appendTo(continuation, lastNext);
+                length = m_out.phi(Int32, smallerOrEqualLengthResult, largerLengthResult);
+            }
+            lengthIncludingThis = m_out.add(length, m_out.constInt32(1));
+        }
+
         speculate(
             VarargsOverflow, noValue(), nullptr,
             m_out.above(lengthIncludingThis, m_out.constInt32(data->limit)));
         
         m_out.store32(lengthIncludingThis, payloadFor(data->machineCount));
         
-        LValue sourceStart = getArgumentsStart(inlineCallFrame);
+        unsigned numberOfArgumentsToSkip = data->offset;
+        LValue sourceStart = getArgumentsStart(inlineCallFrame, numberOfArgumentsToSkip);
         LValue targetStart = addressFor(data->machineStart).value();
 
         LBasicBlock undefinedLoop = m_out.newBlock();
@@ -6328,9 +6361,7 @@ private:
         previousIndex = m_out.phi(pointerType(), loopBound);
         currentIndex = m_out.sub(previousIndex, m_out.intPtrOne);
         LValue value = m_out.load64(
-            m_out.baseIndex(
-                m_heaps.variables, sourceStart,
-                m_out.add(currentIndex, m_out.constIntPtr(data->offset))));
+            m_out.baseIndex(m_heaps.variables, sourceStart, currentIndex));
         m_out.store64(value, m_out.baseIndex(m_heaps.variables, targetStart, currentIndex));
         nextIndex = m_out.anchor(currentIndex);
         m_out.addIncomingToPhi(previousIndex, nextIndex);
@@ -8376,9 +8407,9 @@ private:
         return m_out.loadPtr(addressFor(CallFrameSlot::callee));
     }
     
-    LValue getArgumentsStart(InlineCallFrame* inlineCallFrame)
+    LValue getArgumentsStart(InlineCallFrame* inlineCallFrame, unsigned offset = 0)
     {
-        VirtualRegister start = AssemblyHelpers::argumentsStart(inlineCallFrame);
+        VirtualRegister start = AssemblyHelpers::argumentsStart(inlineCallFrame) + offset;
         return addressFor(start).value();
     }
     
index beeebc0..592b3ca 100644 (file)
@@ -83,6 +83,7 @@ extern "C" void JIT_OPERATION operationPopulateObjectInOSR(
     case PhantomNewGeneratorFunction:
     case PhantomDirectArguments:
     case PhantomClonedArguments:
+    case PhantomCreateRest:
         // Those are completely handled by operationMaterializeObjectInOSR
         break;
 
@@ -234,6 +235,7 @@ extern "C" JSCell* JIT_OPERATION operationMaterializeObjectInOSR(
         return result;
     }
 
+    case PhantomCreateRest:
     case PhantomDirectArguments:
     case PhantomClonedArguments: {
         if (!materialization->origin().inlineCallFrame) {
@@ -242,6 +244,17 @@ extern "C" JSCell* JIT_OPERATION operationMaterializeObjectInOSR(
                 return DirectArguments::createByCopying(exec);
             case PhantomClonedArguments:
                 return ClonedArguments::createWithMachineFrame(exec, exec, ArgumentsMode::Cloned);
+            case PhantomCreateRest: {
+                CodeBlock* codeBlock = baselineCodeBlockForOriginAndBaselineCodeBlock(
+                    materialization->origin(), exec->codeBlock());
+
+                unsigned numberOfArgumentsToSkip = codeBlock->numParameters() - 1;
+                JSGlobalObject* globalObject = codeBlock->globalObject();
+                Structure* structure = globalObject->restParameterStructure();
+                JSValue* argumentsToCopyRegion = exec->addressOfArgumentsStart() + numberOfArgumentsToSkip;
+                unsigned arraySize = exec->argumentCount() > numberOfArgumentsToSkip ? exec->argumentCount() - numberOfArgumentsToSkip : 0;
+                return constructArray(exec, structure, argumentsToCopyRegion, arraySize);
+            }
             default:
                 RELEASE_ASSERT_NOT_REACHED();
                 return nullptr;
@@ -255,14 +268,12 @@ extern "C" JSCell* JIT_OPERATION operationMaterializeObjectInOSR(
                 const ExitPropertyValue& property = materialization->properties()[i];
                 if (property.location() != PromotedLocationDescriptor(ArgumentCountPLoc))
                     continue;
-                
                 argumentCount = JSValue::decode(values[i]).asUInt32();
-                RELEASE_ASSERT(argumentCount);
                 break;
             }
-            RELEASE_ASSERT(argumentCount);
         } else
             argumentCount = materialization->origin().inlineCallFrame->arguments.size();
+        RELEASE_ASSERT(argumentCount);
         
         JSFunction* callee = nullptr;
         if (materialization->origin().inlineCallFrame->isClosureCall) {
@@ -330,6 +341,56 @@ extern "C" JSCell* JIT_OPERATION operationMaterializeObjectInOSR(
             
             return result;
         }
+        case PhantomCreateRest: {
+            unsigned numberOfArgumentsToSkip = codeBlock->numParameters() - 1;
+            JSGlobalObject* globalObject = codeBlock->globalObject();
+            Structure* structure = globalObject->restParameterStructure();
+            ASSERT(argumentCount > 0);
+            unsigned arraySize = (argumentCount - 1) > numberOfArgumentsToSkip ? argumentCount - 1 - numberOfArgumentsToSkip : 0;
+            JSArray* array = JSArray::tryCreateUninitialized(vm, structure, arraySize);
+            RELEASE_ASSERT(array);
+
+            for (unsigned i = materialization->properties().size(); i--;) {
+                const ExitPropertyValue& property = materialization->properties()[i];
+                if (property.location().kind() != ArgumentPLoc)
+                    continue;
+
+                unsigned argIndex = property.location().info();
+                if (numberOfArgumentsToSkip > argIndex)
+                    continue;
+                unsigned arrayIndex = argIndex - numberOfArgumentsToSkip;
+                if (arrayIndex >= arraySize)
+                    continue;
+                array->initializeIndex(vm, arrayIndex, JSValue::decode(values[i]));
+            }
+
+#if !ASSERT_DISABLED
+            // We avoid this O(n^2) loop when asserts are disabled, but the condition checked here
+            // must hold to ensure the correctness of the above loop because of how we allocate the array.
+            for (unsigned targetIndex = 0; targetIndex < arraySize; ++targetIndex) {
+                bool found = false;
+                for (unsigned i = materialization->properties().size(); i--;) {
+                    const ExitPropertyValue& property = materialization->properties()[i];
+                    if (property.location().kind() != ArgumentPLoc)
+                        continue;
+
+                    unsigned argIndex = property.location().info();
+                    if (numberOfArgumentsToSkip > argIndex)
+                        continue;
+                    unsigned arrayIndex = argIndex - numberOfArgumentsToSkip;
+                    if (arrayIndex >= arraySize)
+                        continue;
+                    if (arrayIndex == targetIndex) {
+                        found = true;
+                        break;
+                    }
+                }
+                ASSERT(found);
+            }
+#endif
+
+            return array;
+        }
         default:
             RELEASE_ASSERT_NOT_REACHED();
             return nullptr;
index 194e979..d2d4515 100644 (file)
@@ -47,7 +47,7 @@ void emitSetVarargsFrame(CCallHelpers& jit, GPRReg lengthGPR, bool lengthInclude
     jit.andPtr(CCallHelpers::TrustedImm32(~(stackAlignmentRegisters() - 1)), resultGPR);
 
     jit.addPtr(lengthGPR, resultGPR);
-    jit.addPtr(CCallHelpers::TrustedImm32(CallFrame::headerSizeInRegisters + (lengthIncludesThis? 0 : 1)), resultGPR);
+    jit.addPtr(CCallHelpers::TrustedImm32(CallFrame::headerSizeInRegisters + (lengthIncludesThis ? 0 : 1)), resultGPR);
     
     // resultGPR now has the required frame size in Register units
     // Round resultGPR to next multiple of stackAlignmentRegisters()
index 947ae86..9f64da0 100644 (file)
@@ -851,7 +851,7 @@ SLOW_PATH_DECL(slow_path_create_rest)
     BEGIN();
     unsigned arraySize = OP_C(2).jsValue().asUInt32();
     JSGlobalObject* globalObject = exec->lexicalGlobalObject();
-    Structure* structure = globalObject->arrayStructureForIndexingTypeDuringAllocation(ArrayWithContiguous);
+    Structure* structure = globalObject->restParameterStructure();
     unsigned numParamsToSkip = pc[3].u.unsignedValue;
     JSValue* argumentsToCopyRegion = exec->addressOfArgumentsStart() + numParamsToSkip;
     RETURN(constructArray(exec, structure, argumentsToCopyRegion, arraySize));
index 01ec341..ea57506 100644 (file)
@@ -623,6 +623,7 @@ public:
     Structure* callableProxyObjectStructure() const { return m_callableProxyObjectStructure.get(); }
     Structure* proxyRevokeStructure() const { return m_proxyRevokeStructure.get(); }
     Structure* moduleLoaderStructure() const { return m_moduleLoaderStructure.get(); }
+    Structure* restParameterStructure() const { return arrayStructureForIndexingTypeDuringAllocation(ArrayWithContiguous); }
 
     JS_EXPORT_PRIVATE void setRemoteDebuggingEnabled(bool);
     JS_EXPORT_PRIVATE bool remoteDebuggingEnabled() const;