test262: module test progressions need updated expectations (@@iterator changes)
authorcommit-queue@webkit.org <commit-queue@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 6 Apr 2017 04:42:33 +0000 (04:42 +0000)
committercommit-queue@webkit.org <commit-queue@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 6 Apr 2017 04:42:33 +0000 (04:42 +0000)
https://bugs.webkit.org/show_bug.cgi?id=170535

Patch by Joseph Pecoraro <pecoraro@apple.com> on 2017-04-05
Reviewed by Saam Barati.

JSTests:

* test262.yaml:
After r212430 the tests are out of date and are now expected to fail.

Tools:

* Scripts/run-jsc-stress-tests:
Add :failDueToOutdatedOrBadTest to distinguish between a test failure
in JavaScriptCore or an outdated or incorrect test262 test.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214985 268f45cc-cd09-0410-ab3c-d52691b4dbfc

JSTests/ChangeLog
JSTests/test262.yaml
Tools/ChangeLog
Tools/Scripts/run-jsc-stress-tests

index 1d5c394..29ed736 100644 (file)
@@ -1,3 +1,13 @@
+2017-04-05  Joseph Pecoraro  <pecoraro@apple.com>
+
+        test262: module test progressions need updated expectations (@@iterator changes)
+        https://bugs.webkit.org/show_bug.cgi?id=170535
+
+        Reviewed by Saam Barati.
+
+        * test262.yaml:
+        After r212430 the tests are out of date and are now expected to fail.
+
 2017-04-05  Michael Saboff  <msaboff@apple.com>
 
         Unreviewed rollout of r214642 as the test still intermittently fails.
index e344211..ea73d41 100644 (file)
 - path: test262/test/language/module-code/instn-uniq-env-rec.js
   cmd: runTest262 :normal, "NoException", ["../../../harness/assert.js", "../../../harness/sta.js"], [:module]
 - path: test262/test/language/module-code/namespace/Symbol.iterator/length.js
-  cmd: runTest262 :normal, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js", "../../../../../harness/propertyHelper.js"], [:module]
+  cmd: runTest262 :failDueToOutdatedOrBadTest, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js", "../../../../../harness/propertyHelper.js"], [:module]
 - path: test262/test/language/module-code/namespace/Symbol.iterator/list-iter-next-length.js
-  cmd: runTest262 :normal, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js", "../../../../../harness/propertyHelper.js"], [:module]
+  cmd: runTest262 :failDueToOutdatedOrBadTest, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js", "../../../../../harness/propertyHelper.js"], [:module]
 - path: test262/test/language/module-code/namespace/Symbol.iterator/list-iter-next-name.js
-  cmd: runTest262 :normal, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js", "../../../../../harness/propertyHelper.js"], [:module]
+  cmd: runTest262 :failDueToOutdatedOrBadTest, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js", "../../../../../harness/propertyHelper.js"], [:module]
 - path: test262/test/language/module-code/namespace/Symbol.iterator/list-iter-next-prop-desc.js
-  cmd: runTest262 :normal, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js", "../../../../../harness/propertyHelper.js"], [:module]
+  cmd: runTest262 :failDueToOutdatedOrBadTest, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js", "../../../../../harness/propertyHelper.js"], [:module]
 - path: test262/test/language/module-code/namespace/Symbol.iterator/name.js
-  cmd: runTest262 :normal, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js", "../../../../../harness/propertyHelper.js"], [:module]
+  cmd: runTest262 :failDueToOutdatedOrBadTest, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js", "../../../../../harness/propertyHelper.js"], [:module]
 - path: test262/test/language/module-code/namespace/Symbol.iterator/prop-desc.js
-  cmd: runTest262 :normal, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
+  cmd: runTest262 :failDueToOutdatedOrBadTest, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
 - path: test262/test/language/module-code/namespace/Symbol.iterator/this-val-not-ns.js
-  cmd: runTest262 :normal, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
+  cmd: runTest262 :failDueToOutdatedOrBadTest, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
 - path: test262/test/language/module-code/namespace/Symbol.iterator/values-binding-types.js
-  cmd: runTest262 :normal, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
+  cmd: runTest262 :failDueToOutdatedOrBadTest, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
 - path: test262/test/language/module-code/namespace/Symbol.iterator/values-binding-types_.js
   cmd: prepareTest262Fixture
 - path: test262/test/language/module-code/namespace/Symbol.iterator/values-order.js
-  cmd: runTest262 :normal, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
+  cmd: runTest262 :failDueToOutdatedOrBadTest, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
 - path: test262/test/language/module-code/namespace/Symbol.toStringTag.js
-  cmd: runTest262 :normal, "NoException", ["../../../../harness/assert.js", "../../../../harness/sta.js"], [:module]
+  cmd: runTest262 :failDueToOutdatedOrBadTest, "NoException", ["../../../../harness/assert.js", "../../../../harness/sta.js"], [:module]
 - path: test262/test/language/module-code/namespace/internals/define-own-property.js
   cmd: runTest262 :normal, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
 - path: test262/test/language/module-code/namespace/internals/delete-exported-init.js
 - path: test262/test/language/module-code/namespace/internals/delete-exported-uninit.js
   cmd: runTest262 :normal, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
 - path: test262/test/language/module-code/namespace/internals/delete-non-exported.js
-  cmd: runTest262 :normal, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
+  cmd: runTest262 :failDueToOutdatedOrBadTest, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
 - path: test262/test/language/module-code/namespace/internals/get-own-property-str-found-init.js
   cmd: runTest262 :normal, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
 - path: test262/test/language/module-code/namespace/internals/get-own-property-str-found-uninit.js
 - path: test262/test/language/module-code/namespace/internals/get-own-property-str-not-found.js
   cmd: runTest262 :normal, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
 - path: test262/test/language/module-code/namespace/internals/get-own-property-sym.js
-  cmd: runTest262 :normal, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
+  cmd: runTest262 :failDueToOutdatedOrBadTest, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
 - path: test262/test/language/module-code/namespace/internals/get-prototype-of.js
   cmd: runTest262 :normal, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
 - path: test262/test/language/module-code/namespace/internals/get-str-found-init.js
 - path: test262/test/language/module-code/namespace/internals/get-str-update.js
   cmd: runTest262 :normal, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
 - path: test262/test/language/module-code/namespace/internals/get-sym-found.js
-  cmd: runTest262 :normal, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
+  cmd: runTest262 :failDueToOutdatedOrBadTest, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
 - path: test262/test/language/module-code/namespace/internals/get-sym-not-found.js
   cmd: runTest262 :normal, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
 - path: test262/test/language/module-code/namespace/internals/has-property-str-found-init.js
 - path: test262/test/language/module-code/namespace/internals/has-property-str-not-found.js
   cmd: runTest262 :normal, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
 - path: test262/test/language/module-code/namespace/internals/has-property-sym-found.js
-  cmd: runTest262 :normal, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
+  cmd: runTest262 :failDueToOutdatedOrBadTest, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
 - path: test262/test/language/module-code/namespace/internals/has-property-sym-not-found.js
   cmd: runTest262 :normal, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
 - path: test262/test/language/module-code/namespace/internals/is-extensible.js
   cmd: runTest262 :normal, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
 - path: test262/test/language/module-code/namespace/internals/own-property-keys-binding-types.js
-  cmd: runTest262 :normal, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
+  cmd: runTest262 :failDueToOutdatedOrBadTest, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
 - path: test262/test/language/module-code/namespace/internals/own-property-keys-binding-types_FIXTURE.js
   cmd: prepareTest262Fixture
 - path: test262/test/language/module-code/namespace/internals/own-property-keys-sort.js
-  cmd: runTest262 :normal, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
+  cmd: runTest262 :failDueToOutdatedOrBadTest, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
 - path: test262/test/language/module-code/namespace/internals/prevent-extensions.js
   cmd: runTest262 :normal, "NoException", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], [:module]
 - path: test262/test/language/module-code/namespace/internals/set-prototype-of.js
index 080326c..1964ad3 100644 (file)
@@ -1,5 +1,16 @@
 2017-04-05  Joseph Pecoraro  <pecoraro@apple.com>
 
+        test262: module test progressions need updated expectations (@@iterator changes)
+        https://bugs.webkit.org/show_bug.cgi?id=170535
+
+        Reviewed by Saam Barati.
+
+        * Scripts/run-jsc-stress-tests:
+        Add :failDueToOutdatedOrBadTest to distinguish between a test failure
+        in JavaScriptCore or an outdated or incorrect test262 test.
+
+2017-04-05  Joseph Pecoraro  <pecoraro@apple.com>
+
         Remove run-jsc-stress-tests benign warning about otool '-S' switch
         https://bugs.webkit.org/show_bug.cgi?id=170527
 
index c659dc5..387f1ac 100755 (executable)
@@ -1128,6 +1128,9 @@ def runTest262(mode, exception, includeFiles, flags)
     when :fail
         errorHandler = expectedFailErrorHandler
         outputHandler = noisyOutputHandler
+    when :failDueToOutdatedOrBadTest
+        errorHandler = expectedFailErrorHandler
+        outputHandler = noisyOutputHandler
     else
         raise "Invalid mode: #{mode}"
     end