WebDriver: add a common way to run tests with pytest
authorcarlosgc@webkit.org <carlosgc@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 14 Dec 2017 14:37:42 +0000 (14:37 +0000)
committercarlosgc@webkit.org <carlosgc@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 14 Dec 2017 14:37:42 +0000 (14:37 +0000)
commit2f6b0eb82e3506b9501c0434e97d27b8eab320b9
tree8fca220a7e15568573402209b3290afb2d7ddd9f
parent0fcfc77fe6e43207ebfd5557860afb1a687b131a
WebDriver: add a common way to run tests with pytest
https://bugs.webkit.org/show_bug.cgi?id=180800

Reviewed by Carlos Alberto Lopez Perez.

Tools:

We currently use pytestrunner from wpt for w3c tests and our own code for selenium tests. Using the same code
for both would simplify everything, but also allows us to have a custom results recorder to support other test
expectations like TIMEOUT. The code to run selenium tests with pytest has been moved to a new file
pytest_runner.py and made generic to be used also for w3c tests.

* Scripts/webkitpy/webdriver_tests/pytest_runner.py: Added.
(TemporaryDirectory):
(TemporaryDirectory.__enter__):
(TemporaryDirectory.__exit__):
(CollectRecorder):
(CollectRecorder.__init__):
(CollectRecorder.pytest_collectreport):
(HarnessResultRecorder):
(HarnessResultRecorder.__init__):
(HarnessResultRecorder.pytest_collectreport):
(SubtestResultRecorder):
(SubtestResultRecorder.__init__):
(SubtestResultRecorder.pytest_runtest_logreport):
(SubtestResultRecorder._was_timeout):
(SubtestResultRecorder.record_pass):
(SubtestResultRecorder.record_fail):
(SubtestResultRecorder.record_error):
(SubtestResultRecorder.record_skip):
(SubtestResultRecorder.record):
(collect):
(run):
* Scripts/webkitpy/webdriver_tests/webdriver_selenium_executor.py:
(do_delayed_imports): Import pytest_runner here to avoid cycles.
(WebDriverSeleniumExecutor.__init__): Save the driver parameter as args member and call do_delayed_imports() if
needed.
(WebDriverSeleniumExecutor.collect): Use pytest_runner.
(WebDriverSeleniumExecutor.run): Ditto.
* Scripts/webkitpy/webdriver_tests/webdriver_test_runner.py:
(WebDriverTestRunner.print_results): Handle all possible tests results.
(WebDriverTestRunner.print_results.report): Helper to dump test results.
* Scripts/webkitpy/webdriver_tests/webdriver_test_runner_selenium.py:
(WebDriverTestRunnerSelenium.run):
* Scripts/webkitpy/webdriver_tests/webdriver_test_runner_w3c.py:
(WebDriverTestRunnerW3C.__init__): Do not set PYTEST_TIMEOUT env var.
(WebDriverTestRunnerW3C._is_test): Fix check for support files.
(WebDriverTestRunnerW3C.run): Pass the timeout as parameter to WebDriverW3CExecutor.run().
* Scripts/webkitpy/webdriver_tests/webdriver_w3c_executor.py:
(do_delayed_imports): Import pytest_runner here to avoid cycles.
(WebDriverW3CExecutor.__init__): Call do_delayed_imports() if needed.
(WebDriverW3CExecutor.run): Use pytest_runner.

WebDriverTests:

Remove conftest.py since pytest_timeout plugin is now always loaded from the command line.

* imported/w3c/conftest.py: Removed.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@225902 268f45cc-cd09-0410-ab3c-d52691b4dbfc
Tools/ChangeLog
Tools/Scripts/webkitpy/webdriver_tests/pytest_runner.py [new file with mode: 0644]
Tools/Scripts/webkitpy/webdriver_tests/webdriver_selenium_executor.py
Tools/Scripts/webkitpy/webdriver_tests/webdriver_test_runner.py
Tools/Scripts/webkitpy/webdriver_tests/webdriver_test_runner_selenium.py
Tools/Scripts/webkitpy/webdriver_tests/webdriver_test_runner_w3c.py
Tools/Scripts/webkitpy/webdriver_tests/webdriver_w3c_executor.py
WebDriverTests/ChangeLog
WebDriverTests/imported/w3c/conftest.py [deleted file]