CommitQueue and EWS should reject any patches that result in consistent test
authorcommit-queue@webkit.org <commit-queue@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 30 Oct 2014 05:02:33 +0000 (05:02 +0000)
committercommit-queue@webkit.org <commit-queue@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 30 Oct 2014 05:02:33 +0000 (05:02 +0000)
commite0eb500a821844c6dc19b9409043441659310f99
treec20c3d32482ff35ccc6ce2af037e0f530a61bca4
parenta61553b619ce138ffbcac8c2b72b3d45e490cd76
CommitQueue and EWS should reject any patches that result in consistent test
failures that aren't present on the tree.
https://bugs.webkit.org/show_bug.cgi?id=138184

Patch by Jake Nielsen <jacob_nielsen@apple.com> on 2014-10-29
Reviewed by Alexey Proskuryakov.

* Scripts/webkitpy/layout_tests/models/test_results.py:
Adds a simple hashing function to allow for set operations to handle
TestResult objects properly.
(TestResult.__hash__):
* Scripts/webkitpy/tool/bot/commitqueuetask_unittest.py:
Adds one unit test, and modifies others to agree with the notion that
patches that introduce new test failures (but also have flakyness)
should be rejected rather than spin.
(MockCommitQueue.report_flaky_tests):
(CommitQueueTaskTest._run_and_expect_patch_analysis_result):
(test_double_flaky_test_failure):
(test_two_flaky_tests):
(test_very_flaky_patch):
(test_very_flaky_patch_with_some_tree_redness):
(test_different_test_failures):
(test_different_test_failures_with_some_tree_redness):
(test_different_test_failures_with_some_tree_redness_and_some_fixes):
(test_mildly_flaky_patch):
(test_mildly_flaky_patch_with_some_tree_redness):
* Scripts/webkitpy/tool/bot/patchanalysistask.py:
Makes PatchAnalysisTask reject said patches.
(PatchAnalysisTask._test_patch):

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@175367 268f45cc-cd09-0410-ab3c-d52691b4dbfc
Tools/ChangeLog
Tools/Scripts/webkitpy/layout_tests/models/test_results.py
Tools/Scripts/webkitpy/tool/bot/commitqueuetask_unittest.py
Tools/Scripts/webkitpy/tool/bot/patchanalysistask.py