git://git.webkit.org / WebKit-https.git / commitdiff
? search:
summary | shortlog | log | commit | commitdiff | tree
raw | patch | inline | side by side (parent: aaac2c6)
DYEBench should run 20 iterations in browser
authorrniwa@webkit.org <rniwa@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 13 May 2014 21:08:10 +0000 (21:08 +0000)
committerrniwa@webkit.org <rniwa@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 13 May 2014 21:08:10 +0000 (21:08 +0000)
https://bugs.webkit.org/show_bug.cgi?id=132795

Reviewed by Gavin Barraclough.

Right now, run-perf-tests runs BYEBench 4 times with different instances of DRT/WTR to get
a more stable time and account for differences in the runtime environment, particularly,
ASLR (Address Space Layout Randomization).

While we can't account for the latter effect when the benchmark is ran inside a browser,
we can at least run 20 iterations to get a more stable (but perhaps biased by ALSR) number.

While the sample size of 20 is statistically unsound to compute the arithmetic mean from,
it's MUCH better than the current sample size of 5.

* DoYouEvenBench/Full.html:
(benchmarkClient.iterationCount): Set the default iteration count to 20.
(startTest): Use benchmarkClient.iterationCount as the iteration count.
* DoYouEvenBench/resources/benchmark-report.js:
(benchmarkClient.iterationCount): Continue to use the old iteration count of 5 when ran
inside run-perf-tests.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@168732 268f45cc-cd09-0410-ab3c-d52691b4dbfc

PerformanceTests/ChangeLog patch | blob | history
PerformanceTests/DoYouEvenBench/Full.html patch | blob | history
PerformanceTests/DoYouEvenBench/resources/benchmark-report.js patch | blob | history

diff --git a/PerformanceTests/ChangeLog b/PerformanceTests/ChangeLog
index 19a37d36c3fceec7ccdeee835bf82e741cbb3ec7..c37df7029411b6e370b00b317d87240e79b3cd90 100644 (file)
--- a/PerformanceTests/ChangeLog
+++ b/PerformanceTests/ChangeLog
@@ -1,3 +1,27 @@
+2014-05-13  Ryosuke Niwa  <rniwa@webkit.org>
+
+        DYEBench should run 20 iterations in browser
+        https://bugs.webkit.org/show_bug.cgi?id=132795
+
+        Reviewed by Gavin Barraclough.
+
+        Right now, run-perf-tests runs BYEBench 4 times with different instances of DRT/WTR to get
+        a more stable time and account for differences in the runtime environment, particularly,
+        ASLR (Address Space Layout Randomization).
+
+        While we can't account for the latter effect when the benchmark is ran inside a browser,
+        we can at least run 20 iterations to get a more stable (but perhaps biased by ALSR) number.
+
+        While the sample size of 20 is statistically unsound to compute the arithmetic mean from,
+        it's MUCH better than the current sample size of 5.
+
+        * DoYouEvenBench/Full.html:
+        (benchmarkClient.iterationCount): Set the default iteration count to 20.
+        (startTest): Use benchmarkClient.iterationCount as the iteration count.
+        * DoYouEvenBench/resources/benchmark-report.js:
+        (benchmarkClient.iterationCount): Continue to use the old iteration count of 5 when ran
+        inside run-perf-tests.
+
 2014-05-09  Ryosuke Niwa  <rniwa@webkit.org>
 
         DYEBench should show 95th percentile right next to the mean with ±
diff --git a/PerformanceTests/DoYouEvenBench/Full.html b/PerformanceTests/DoYouEvenBench/Full.html
index d4e576f77ffe8367b93db9a0034ae748dbfc89f0..16569a9eff8f761097c39aea1a9340eca183e1f2 100644 (file)
--- a/PerformanceTests/DoYouEvenBench/Full.html
+++ b/PerformanceTests/DoYouEvenBench/Full.html
@@ -1,7 +1,7 @@
 <!DOCTYPE html>
 <html>
 <head>
-<title>DoYouEvenBench v0.10</title>
+<title>DoYouEvenBench v0.11</title>
 <style type="text/css">
 caption { margin: 0; padding: 0; font-family: sans-serif; font-size: 1em; font-weight: bold; white-space: nowrap; }
 #progressContainer { padding: 605px 0 10px 0; width: 800px; }
@@ -43,6 +43,7 @@ td { text-align: left; }
     }
 
     window.benchmarkClient = {
+        iterationCount: 20,
         willRunTest: function () {
             if (!progress) {
                 // We don't use the real progress element as some implementations animate it.
@@ -79,10 +80,10 @@ td { text-align: left; }
 })();
 
 function startTest() {
-    var iterationCount = 5;
-    benchmarkClient.testsCount = iterationCount * Suites.reduce(function (testsCount, suite) { return testsCount + suite.tests.length; }, 0);
+    var totalSubtestCount = Suites.reduce(function (testsCount, suite) { return testsCount + suite.tests.length; }, 0);
+    benchmarkClient.testsCount = benchmarkClient.iterationCount * totalSubtestCount;
     var runner = new BenchmarkRunner(Suites, benchmarkClient);
-    runner.runMultipleIterations(iterationCount);
+    runner.runMultipleIterations(benchmarkClient.iterationCount);
 }
 
 </script>
diff --git a/PerformanceTests/DoYouEvenBench/resources/benchmark-report.js b/PerformanceTests/DoYouEvenBench/resources/benchmark-report.js
index 0ea96c17eda169fd82127cb2c5cc5fc28bdf8a6d..f7424e5b5436c5394209e6ab3bebf6a824b2ba65 100644 (file)
--- a/PerformanceTests/DoYouEvenBench/resources/benchmark-report.js
+++ b/PerformanceTests/DoYouEvenBench/resources/benchmark-report.js
@@ -19,6 +19,7 @@
     var valuesByIteration = new Array;
 
     window.benchmarkClient = {
+        iterationCount: 5, // Use 4 different instances of DRT/WTR to run 5 iterations.
         willStartFirstIteration: function (iterationCount) {
             createTest = function (name, aggregator, isLastTest) {
                 return {
Mirror of the WebKit open source project from svn.webkit.org.