If NRWT gets killed halfway through a run, it incorrectly reports tests that weren...
authordpranke@chromium.org <dpranke@chromium.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 11 Apr 2012 02:38:12 +0000 (02:38 +0000)
committerdpranke@chromium.org <dpranke@chromium.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 11 Apr 2012 02:38:12 +0000 (02:38 +0000)
commit40af9d4cc918e2029752a68a53287f8295e5d893
treea2692c0bedbc06d7209657d3a6757c04ed4d5e66
parentf94e35d5ca363c046d9f2b78bc2fbc3a7ab5f142
If NRWT gets killed halfway through a run, it incorrectly reports tests that weren't run as passes
https://bugs.webkit.org/show_bug.cgi?id=82799

Reviewed by Ojan Vafai.

The tests we were skipping in _mark_interrupted_tests_as_skipped
had incorrectly-constructed TestResults, and so they were not
being treated as failures. The JSON generator doesn't record
tests that are passed or skiped in incremental_results, so you
couldn't tell these tests were being skipped at all.

This change changes _mark_interrupted_tests_as_skipped() to
treat the skipped tests as failures; this is arguably still
incorrect but is minimally invasive for now. We probably need to
revisit how we are uploading information about tests that are
skipped and passed altogether in a separate change.

This change also cleans up a couple of other nits that were
unnecessary or broken in this same code path.

* Scripts/webkitpy/layout_tests/controllers/manager.py:
(Manager._mark_interrupted_tests_as_skipped):
(Manager._upload_json_files): Deletes the check for
chromium-mac-leopard (as it is no longer necessary).
* Scripts/webkitpy/layout_tests/layout_package/json_layout_results_generator.py:
(JSONLayoutResultsGenerator.__init__): Here we were
recalculating the failure type unnecessarily.
* Scripts/webkitpy/layout_tests/models/test_failures.py:
(determine_result_type):
(FailureEarlyExit): Adds a new failure type so we can tell what
happened.
(FailureEarlyExit.message):
* Scripts/webkitpy/layout_tests/run_webkit_tests_integrationtest.py:
(MainTest.test_exit_after_n_failures_upload): Add more checks to
ensure that we are recording the SKIP correctly; this is less
than ideal, because we can't actually test the contents of
incremental_results.json. We should fix that at some point.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@113813 268f45cc-cd09-0410-ab3c-d52691b4dbfc
Tools/ChangeLog
Tools/Scripts/webkitpy/layout_tests/controllers/manager.py
Tools/Scripts/webkitpy/layout_tests/layout_package/json_layout_results_generator.py
Tools/Scripts/webkitpy/layout_tests/models/test_failures.py
Tools/Scripts/webkitpy/layout_tests/run_webkit_tests_integrationtest.py