Web Inspector: TestSuite test cases should have their own timeout to ensure tests...
authordrousso@apple.com <drousso@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 5 Apr 2019 23:39:16 +0000 (23:39 +0000)
committerdrousso@apple.com <drousso@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 5 Apr 2019 23:39:16 +0000 (23:39 +0000)
commit3a03267f9fa6b2242f2899032b3bf631f29b7985
tree523fa73ba931ba030b5468b8e0da7ba8a84e490f
parent814c1be94443cc538be5d9b1b16f38d99119aa99
Web Inspector: TestSuite test cases should have their own timeout to ensure tests fail with output instead of timeout by test runner
https://bugs.webkit.org/show_bug.cgi?id=162814
<rdar://problem/28574102>

Reviewed by Brian Burg.

Source/WebInspectorUI:

A 10s timer is started for every test case added to an async suite. The timer is cleared
when the test finishes, but if the timer fires, the test is forcibly ended with an error.

This timer can be configured by setting a `timeout` value when adding the test case. Values
are expected to be in milliseconds. The value `-1` will prevent a timer from being set.

This change also relaxes the expectation that any individual test case failure will stop the
rest of the suite from running. Since timers are set per test case, it is possible to
recover from a "bad" test case to still run the remaining test cases.

NOTE: there may be unexpected behaviour if a test times out, as the timer doesn't actually
stop the execution of the test, so it may still run and log information, which may appear
"out of nowhere" in the middle of other tests.

* UserInterface/Test/TestSuite.js:
(TestSuite.prototype.get passCount):
(AsyncTestSuite.prototype.runTestCases):
(SyncTestSuite.prototype.runTestCases):

LayoutTests:

* inspector/unit-tests/async-test-suite.html:
* inspector/unit-tests/async-test-suite-expected.txt:
* inspector/unit-tests/sync-test-suite.html:
* inspector/unit-tests/sync-test-suite-expected.txt:

* http/tests/inspector/network/set-resource-caching-disabled-disk-cache-expected.txt:
* inspector/canvas/recording-2d.html:
* inspector/canvas/recording-webgl-snapshots.html:
* inspector/canvas/recording-webgl.html:
* inspector/canvas/resources/shaderProgram-utilities.js:
(TestPage.registerInitializer.whenProgramAdded): Added.
(TestPage.registerInitializer.whenProgramRemoved): Added.
(TestPage.registerInitializer.window.initializeTestSuite):
(TestPage.registerInitializer.window.addSimpleTestCase):
(TestPage.registerInitializer.window.addParentCanvasRemovedTestCase):
(TestPage.registerInitializer.awaitProgramAdded): Added.
(TestPage.registerInitializer.awaitProgramRemoved): Added.
* inspector/console/command-line-api-expected.txt:
* inspector/console/heap-snapshot.html:
* inspector/debugger/async-stack-trace-truncate-expected.txt:
* inspector/debugger/pause-for-internal-scripts-expected.txt:
* inspector/formatting/resources/utilities.js:
(TestPage.registerInitializer.window.addFormattingTests):

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@243953 268f45cc-cd09-0410-ab3c-d52691b4dbfc
17 files changed:
LayoutTests/ChangeLog
LayoutTests/http/tests/inspector/network/set-resource-caching-disabled-disk-cache-expected.txt
LayoutTests/inspector/canvas/recording-2d.html
LayoutTests/inspector/canvas/recording-webgl-snapshots.html
LayoutTests/inspector/canvas/recording-webgl.html
LayoutTests/inspector/canvas/resources/shaderProgram-utilities.js
LayoutTests/inspector/console/command-line-api-expected.txt
LayoutTests/inspector/console/heapSnapshot.html
LayoutTests/inspector/debugger/async-stack-trace-truncate-expected.txt
LayoutTests/inspector/debugger/pause-for-internal-scripts-expected.txt
LayoutTests/inspector/formatting/resources/utilities.js
LayoutTests/inspector/unit-tests/async-test-suite-expected.txt
LayoutTests/inspector/unit-tests/async-test-suite.html
LayoutTests/inspector/unit-tests/sync-test-suite-expected.txt
LayoutTests/inspector/unit-tests/sync-test-suite.html
Source/WebInspectorUI/ChangeLog
Source/WebInspectorUI/UserInterface/Test/TestSuite.js