REGRESSION (r231039): RunUnitTests step reports disabled tests as failures
authorryanhaddad@apple.com <ryanhaddad@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 27 Apr 2018 01:25:10 +0000 (01:25 +0000)
committerryanhaddad@apple.com <ryanhaddad@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 27 Apr 2018 01:25:10 +0000 (01:25 +0000)
https://bugs.webkit.org/show_bug.cgi?id=185061

Reviewed by Aakash Jain.

* BuildSlaveSupport/build.webkit.org-config/steps.py:
(RunUnitTests.countFailures): Return the difference between the total number of tests run and the number that passed.
* BuildSlaveSupport/build.webkit.org-config/steps_unittest.py: Add a test.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@231083 268f45cc-cd09-0410-ab3c-d52691b4dbfc

Tools/BuildSlaveSupport/build.webkit.org-config/steps.py
Tools/BuildSlaveSupport/build.webkit.org-config/steps_unittest.py
Tools/ChangeLog

index 7ea11f2..d1c6571 100644 (file)
@@ -498,12 +498,11 @@ class RunUnitTests(TestWithFailureCount):
 
     def countFailures(self, cmd):
         log_text = cmd.logs['stdio'].getText()
-        count = 0
 
         match = re.search(r'Ran (?P<ran>\d+) tests of (?P<total>\d+) with (?P<passed>\d+) successful', log_text)
         if not match:
             return -1
-        return int(match.group('total')) - int(match.group('passed'))
+        return int(match.group('ran')) - int(match.group('passed'))
 
 
 class RunPythonTests(TestWithFailureCount):
index e37972a..64d2746 100755 (executable)
@@ -241,6 +241,20 @@ Ran 1888 tests of 1888 with 1888 successful
 All tests successfully passed!
 """)
 
+    def test_no_failures_or_timeouts_with_disabled(self):
+        self.assertFailures(0, """...
+worker/0 TestWTF.WTF_Variant.OperatorAmpersand Passed
+worker/0 TestWTF.WTF_Variant.Ref Passed
+worker/0 TestWTF.WTF_Variant.RefPtr Passed
+worker/0 TestWTF.WTF_Variant.RetainPtr Passed
+worker/0 TestWTF.WTF_Variant.VisitorUsingMakeVisitor Passed
+worker/0 TestWTF.WTF_Variant.VisitorUsingSwitchOn Passed
+worker/0 exiting
+Ran 1881 tests of 1888 with 1881 successful
+------------------------------
+All tests successfully passed!
+""")
+
     def test_one_failure(self):
         self.assertFailures(1, """...
 worker/0 TestWTF.WTF_Variant.OperatorAmpersand Passed
index 386efdd..3018a41 100644 (file)
@@ -1,3 +1,14 @@
+2018-04-26  Ryan Haddad  <ryanhaddad@apple.com>
+
+        REGRESSION (r231039): RunUnitTests step reports disabled tests as failures
+        https://bugs.webkit.org/show_bug.cgi?id=185061
+
+        Reviewed by Aakash Jain.
+
+        * BuildSlaveSupport/build.webkit.org-config/steps.py:
+        (RunUnitTests.countFailures): Return the difference between the total number of tests run and the number that passed.
+        * BuildSlaveSupport/build.webkit.org-config/steps_unittest.py: Add a test.
+
 2018-04-26  Ross Kirsling  <ross.kirsling@sony.com>
 
         Add release bots for WinCairo.