Statistics used in perftest_unittest.py and perftest_integrationtest.py are bogus
authorrniwa@webkit.org <rniwa@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 4 Jan 2013 18:20:01 +0000 (18:20 +0000)
committerrniwa@webkit.org <rniwa@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 4 Jan 2013 18:20:01 +0000 (18:20 +0000)
commit21f59d060268cf03e4feb877fa2b5bcbbd581d10
tree30180582cfd5358ecb837c59afb8e81610485cae
parent6423d9f520f8c06b9834445ac3d9bf43c0e7558f
Statistics used in perftest_unittest.py and perftest_integrationtest.py are bogus
https://bugs.webkit.org/show_bug.cgi?id=106078

Reviewed by Tony Chang.

This patch updates values for individual iterations in dummy performance tests used in
perftest_unittest.py and perftest_integrationtest.py so that they match up with
statistics such as stdev and median reported in those tests. This change is required
to resolve the bug 97510.

This patch also refactors integration tests by extracting *TestData classes that
centralizes the definitions of each dummy performance test's text (DRT output),
runner output, and parsed results (JSON) so that we may share more data between
python tests.

* Scripts/webkitpy/performance_tests/perftest_unittest.py:
(MainTest._assert_results_are_correct): Extracted from test_parse_output.
Use self.assertAlmostEqual when comparing the standard deviation.
(MainTest.test_parse_output):
(test_parse_output_with_failing_line):
(test_parse_output_with_description): Use _assert_results_are_correct.
(test_parse_output_with_subtests): Ditto.

* Scripts/webkitpy/performance_tests/perftestsrunner_integrationtest.py:
(InspectorPassTestData): Extracted from tests below.
(EventTargetWrapperTestData): Ditto.
(SomeParserTestData): Ditto.
(MemoryTestData): Ditto.
(TestDriver.run_test): Moved test definitions out of the method.
(MainTest._normalize_output): Truncate standard deviations at 5th decimal point.
(MainTest._load_output_json): Extracted from create_runner.
(MainTest.create_runner):
(MainTest.test_run_test_set_for_parser_tests):
(MainTest.test_run_memory_test):
(MainTest._test_run_with_json_output):
(MainTest.test_run_with_json_output):
(MainTest.test_run_with_description):
(MainTest.test_run_generates_json_by_default):
(MainTest.test_run_merges_output_by_default):
(MainTest.test_run_respects_reset_results):
(MainTest.test_run_generates_and_show_results_page):
(MainTest.test_run_with_slave_config_json):
(MainTest.test_run_with_multiple_repositories):

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@138810 268f45cc-cd09-0410-ab3c-d52691b4dbfc
Tools/ChangeLog
Tools/Scripts/webkitpy/performance_tests/perftest_unittest.py
Tools/Scripts/webkitpy/performance_tests/perftestsrunner_integrationtest.py