Fix PerfTest standard deviation calculation.
authorpdr@google.com <pdr@google.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 2 Oct 2012 08:10:47 +0000 (08:10 +0000)
committerpdr@google.com <pdr@google.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 2 Oct 2012 08:10:47 +0000 (08:10 +0000)
commite0191fd3d99d09f25f6c4553e4e91a74004aed15
tree94c64afb7852e564b09f97a6b68b25a782cbad54
parent4a4c119fba52d435380f8ee09b0d4fc84d5c0242
Fix PerfTest standard deviation calculation.
https://bugs.webkit.org/show_bug.cgi?id=98115

Reviewed by Ryosuke Niwa.

Previously our standard deviation calculation was incorrect. This patch
updates perftest.py's algorithm to calculate the sample standard deviation
(with Bessel's correction) using Knuth's online algorithm:
http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Online_algorithm
An existing test has been modified to prove our new results.

This patch also updates runner.js to use Bessel's correction in
its sample standard deviation calculation, which is more accurate
for small sample sizes.

Additionally, runner.js has been modified to not calculate
the 'sum' statistic, which was not very useful.

PerformanceTests:

* resources/runner.js:
(PerfTestRunner.computeStatistics):

Tools:

* Scripts/webkitpy/performance_tests/perftest.py:

    The unused variable valueSum has also been removed.

(PageLoadingPerfTest.run):
* Scripts/webkitpy/performance_tests/perftest_unittest.py:

    This test calculates the stdev of {2000, 3000, ..., 20000} which
    was hand-calculated using a spreadsheet.

(TestPageLoadingPerfTest.test_run):

LayoutTests:

* fast/harness/perftests/perf-runner-compute-statistics-expected.txt:
* fast/harness/perftests/perf-runner-compute-statistics.html:
* fast/harness/perftests/runs-per-second-log-expected.txt:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@130135 268f45cc-cd09-0410-ab3c-d52691b4dbfc
LayoutTests/ChangeLog
LayoutTests/fast/harness/perftests/perf-runner-compute-statistics-expected.txt
LayoutTests/fast/harness/perftests/perf-runner-compute-statistics.html
LayoutTests/fast/harness/perftests/runs-per-second-log-expected.txt
PerformanceTests/ChangeLog
PerformanceTests/resources/runner.js
Tools/ChangeLog
Tools/Scripts/webkitpy/performance_tests/perftest.py
Tools/Scripts/webkitpy/performance_tests/perftest_unittest.py