Move slow-running microbenchmarks out of JSTests/microbenchmarks
authorfpizlo@apple.com <fpizlo@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 1 Feb 2017 01:48:18 +0000 (01:48 +0000)
committerfpizlo@apple.com <fpizlo@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 1 Feb 2017 01:48:18 +0000 (01:48 +0000)
https://bugs.webkit.org/show_bug.cgi?id=167676

Reviewed by Saam Barati.
JSTests:

The microbenchmarks directory is profitable when it's cheap to run. This change moves very slow
tests (>=200ms running time on my machine) and puts them in JSTests/slowMicrobenchmarks instead.
That directory is run only through `slow!; defaultQuickRun` and is not run by run-jsc-benchmarks.
This is a big progression in test running time. But because it frontloads more slow tests, the
tests will seem to run more slowly when you first start the test runner.

* microbenchmarks/default-derived-constructor.js: Removed.
* microbenchmarks/dense-set.js: Removed.
* microbenchmarks/ftl-polymorphic-div.js: Removed.
* microbenchmarks/ftl-polymorphic-mul.js: Removed.
* microbenchmarks/function-bind-inlining.js: Removed.
* microbenchmarks/function-bind-no-inlining.js: Removed.
* microbenchmarks/large-map-iteration-with-additions.js: Removed.
* microbenchmarks/large-map-iteration-with-mutation.js: Removed.
* microbenchmarks/map-get-get-cse.js: Removed.
* microbenchmarks/misc-bugs-847389-jpeg2000.js: Removed.
* microbenchmarks/spread-small-array.js: Removed.
* microbenchmarks/v8-raytrace-with-try-catch-high-frequency-throws.js: Removed.
* slowMicrobenchmarks: Added.
* slowMicrobenchmarks.yaml: Added.
* slowMicrobenchmarks/README: Added.
* slowMicrobenchmarks/default-derived-constructor.js: Copied from JSTests/microbenchmarks/default-derived-constructor.js.
* slowMicrobenchmarks/dense-set.js: Copied from JSTests/microbenchmarks/dense-set.js.
* slowMicrobenchmarks/ftl-polymorphic-div.js: Copied from JSTests/microbenchmarks/ftl-polymorphic-div.js.
* slowMicrobenchmarks/ftl-polymorphic-mul.js: Copied from JSTests/microbenchmarks/ftl-polymorphic-mul.js.
* slowMicrobenchmarks/function-bind-inlining.js: Copied from JSTests/microbenchmarks/function-bind-inlining.js.
* slowMicrobenchmarks/function-bind-no-inlining.js: Copied from JSTests/microbenchmarks/function-bind-no-inlining.js.
* slowMicrobenchmarks/large-map-iteration-with-additions.js: Copied from JSTests/microbenchmarks/large-map-iteration-with-additions.js.
* slowMicrobenchmarks/large-map-iteration-with-mutation.js: Copied from JSTests/microbenchmarks/large-map-iteration-with-mutation.js.
* slowMicrobenchmarks/map-get-get-cse.js: Copied from JSTests/microbenchmarks/map-get-get-cse.js.
* slowMicrobenchmarks/misc-bugs-847389-jpeg2000.js: Copied from JSTests/microbenchmarks/misc-bugs-847389-jpeg2000.js.
* slowMicrobenchmarks/spread-small-array.js: Copied from JSTests/microbenchmarks/spread-small-array.js.
* slowMicrobenchmarks/v8-raytrace-with-try-catch-high-frequency-throws.js: Copied from JSTests/microbenchmarks/v8-raytrace-with-try-catch-high-frequency-throws.js.

Tools:

* Scripts/run-javascriptcore-tests:
(runJSCStressTests): Add slowMicrobenchmarks.yaml.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@211464 268f45cc-cd09-0410-ab3c-d52691b4dbfc

17 files changed:
JSTests/ChangeLog
JSTests/slowMicrobenchmarks.yaml [new file with mode: 0644]
JSTests/slowMicrobenchmarks/README [new file with mode: 0644]
JSTests/slowMicrobenchmarks/default-derived-constructor.js [moved from JSTests/microbenchmarks/default-derived-constructor.js with 100% similarity]
JSTests/slowMicrobenchmarks/dense-set.js [moved from JSTests/microbenchmarks/dense-set.js with 100% similarity]
JSTests/slowMicrobenchmarks/ftl-polymorphic-div.js [moved from JSTests/microbenchmarks/ftl-polymorphic-div.js with 100% similarity]
JSTests/slowMicrobenchmarks/ftl-polymorphic-mul.js [moved from JSTests/microbenchmarks/ftl-polymorphic-mul.js with 100% similarity]
JSTests/slowMicrobenchmarks/function-bind-inlining.js [moved from JSTests/microbenchmarks/function-bind-inlining.js with 100% similarity]
JSTests/slowMicrobenchmarks/function-bind-no-inlining.js [moved from JSTests/microbenchmarks/function-bind-no-inlining.js with 100% similarity]
JSTests/slowMicrobenchmarks/large-map-iteration-with-additions.js [moved from JSTests/microbenchmarks/large-map-iteration-with-additions.js with 100% similarity]
JSTests/slowMicrobenchmarks/large-map-iteration-with-mutation.js [moved from JSTests/microbenchmarks/large-map-iteration-with-mutation.js with 100% similarity]
JSTests/slowMicrobenchmarks/map-get-get-cse.js [moved from JSTests/microbenchmarks/map-get-get-cse.js with 100% similarity]
JSTests/slowMicrobenchmarks/misc-bugs-847389-jpeg2000.js [moved from JSTests/microbenchmarks/misc-bugs-847389-jpeg2000.js with 100% similarity]
JSTests/slowMicrobenchmarks/spread-small-array.js [moved from JSTests/microbenchmarks/spread-small-array.js with 100% similarity]
JSTests/slowMicrobenchmarks/v8-raytrace-with-try-catch-high-frequency-throws.js [moved from JSTests/microbenchmarks/v8-raytrace-with-try-catch-high-frequency-throws.js with 100% similarity]
Tools/ChangeLog
Tools/Scripts/run-javascriptcore-tests

index 19bb1f2..9b2511d 100644 (file)
@@ -1,3 +1,44 @@
+2017-01-31  Filip Pizlo  <fpizlo@apple.com>
+
+        Move slow-running microbenchmarks out of JSTests/microbenchmarks
+        https://bugs.webkit.org/show_bug.cgi?id=167676
+
+        Reviewed by Saam Barati.
+        
+        The microbenchmarks directory is profitable when it's cheap to run. This change moves very slow
+        tests (>=200ms running time on my machine) and puts them in JSTests/slowMicrobenchmarks instead.
+        That directory is run only through `slow!; defaultQuickRun` and is not run by run-jsc-benchmarks.
+        This is a big progression in test running time. But because it frontloads more slow tests, the
+        tests will seem to run more slowly when you first start the test runner.
+
+        * microbenchmarks/default-derived-constructor.js: Removed.
+        * microbenchmarks/dense-set.js: Removed.
+        * microbenchmarks/ftl-polymorphic-div.js: Removed.
+        * microbenchmarks/ftl-polymorphic-mul.js: Removed.
+        * microbenchmarks/function-bind-inlining.js: Removed.
+        * microbenchmarks/function-bind-no-inlining.js: Removed.
+        * microbenchmarks/large-map-iteration-with-additions.js: Removed.
+        * microbenchmarks/large-map-iteration-with-mutation.js: Removed.
+        * microbenchmarks/map-get-get-cse.js: Removed.
+        * microbenchmarks/misc-bugs-847389-jpeg2000.js: Removed.
+        * microbenchmarks/spread-small-array.js: Removed.
+        * microbenchmarks/v8-raytrace-with-try-catch-high-frequency-throws.js: Removed.
+        * slowMicrobenchmarks: Added.
+        * slowMicrobenchmarks.yaml: Added.
+        * slowMicrobenchmarks/README: Added.
+        * slowMicrobenchmarks/default-derived-constructor.js: Copied from JSTests/microbenchmarks/default-derived-constructor.js.
+        * slowMicrobenchmarks/dense-set.js: Copied from JSTests/microbenchmarks/dense-set.js.
+        * slowMicrobenchmarks/ftl-polymorphic-div.js: Copied from JSTests/microbenchmarks/ftl-polymorphic-div.js.
+        * slowMicrobenchmarks/ftl-polymorphic-mul.js: Copied from JSTests/microbenchmarks/ftl-polymorphic-mul.js.
+        * slowMicrobenchmarks/function-bind-inlining.js: Copied from JSTests/microbenchmarks/function-bind-inlining.js.
+        * slowMicrobenchmarks/function-bind-no-inlining.js: Copied from JSTests/microbenchmarks/function-bind-no-inlining.js.
+        * slowMicrobenchmarks/large-map-iteration-with-additions.js: Copied from JSTests/microbenchmarks/large-map-iteration-with-additions.js.
+        * slowMicrobenchmarks/large-map-iteration-with-mutation.js: Copied from JSTests/microbenchmarks/large-map-iteration-with-mutation.js.
+        * slowMicrobenchmarks/map-get-get-cse.js: Copied from JSTests/microbenchmarks/map-get-get-cse.js.
+        * slowMicrobenchmarks/misc-bugs-847389-jpeg2000.js: Copied from JSTests/microbenchmarks/misc-bugs-847389-jpeg2000.js.
+        * slowMicrobenchmarks/spread-small-array.js: Copied from JSTests/microbenchmarks/spread-small-array.js.
+        * slowMicrobenchmarks/v8-raytrace-with-try-catch-high-frequency-throws.js: Copied from JSTests/microbenchmarks/v8-raytrace-with-try-catch-high-frequency-throws.js.
+
 2017-01-30  Yusuke Suzuki  <utatane.tea@gmail.com>
 
         [JSC] Do not reject WebAssembly.compile() with Exception
diff --git a/JSTests/slowMicrobenchmarks.yaml b/JSTests/slowMicrobenchmarks.yaml
new file mode 100644 (file)
index 0000000..3961973
--- /dev/null
@@ -0,0 +1,26 @@
+# Copyright (C) 2017 Apple Inc. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# 1.  Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer. 
+# 2.  Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in the
+#     documentation and/or other materials provided with the distribution. 
+#
+# THIS SOFTWARE IS PROVIDED BY APPLE AND ITS CONTRIBUTORS "AS IS" AND ANY
+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+# DISCLAIMED. IN NO EVENT SHALL APPLE OR ITS CONTRIBUTORS BE LIABLE FOR ANY
+# DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
+# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+- path: slowMicrobenchmarks
+  cmd: slow!; defaultQuickRun unless parseRunCommands
+
diff --git a/JSTests/slowMicrobenchmarks/README b/JSTests/slowMicrobenchmarks/README
new file mode 100644 (file)
index 0000000..8598613
--- /dev/null
@@ -0,0 +1,6 @@
+This directory contains tests that were once found in JSTests/microbenchmarks, but were pulled from there
+because they are too slow to run regularly. Now they run only for JSC testing, and only with
+defaultQuickRun.
+
+Only move benchmarks back into JSTests/microbenchmarks if you get them to run in under 100ms on a modern
+Mac.
index 08bb491..874e75e 100644 (file)
@@ -1,3 +1,13 @@
+2017-01-31  Filip Pizlo  <fpizlo@apple.com>
+
+        Move slow-running microbenchmarks out of JSTests/microbenchmarks
+        https://bugs.webkit.org/show_bug.cgi?id=167676
+
+        Reviewed by Saam Barati.
+
+        * Scripts/run-javascriptcore-tests:
+        (runJSCStressTests): Add slowMicrobenchmarks.yaml.
+
 2017-01-31  Megan Gardner  <megan_gardner@apple.com>
 
         Make Dump Render Tree not crash
index 48c2b8e..91ff1d5 100755 (executable)
@@ -279,6 +279,7 @@ sub runJSCStressTests
             "PerformanceTests/SunSpider/tests/v8-v6",
             "JSTests/stress",
             "JSTests/microbenchmarks",
+            "JSTests/slowMicrobenchmarks.yaml",
             "PerformanceTests/SunSpider/profiler-test.yaml",
             "LayoutTests/jsc-layout-tests.yaml",
             "JSTests/typeProfiler.yaml",