webkitpy: Implement device type specific expected results (Part 2)
authorjbedard@apple.com <jbedard@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 18 Jan 2019 16:37:59 +0000 (16:37 +0000)
committerjbedard@apple.com <jbedard@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 18 Jan 2019 16:37:59 +0000 (16:37 +0000)
commitdf32706e8ccd9dd359f9864104d1ca3ab01373f4
tree58706d4f02f7bd4d1655daacd4bbb9eae1dd074a
parentbcb1d7fb79b64cfdd95fb488ab985d35c61ee1c4
webkitpy: Implement device type specific expected results (Part 2)
https://bugs.webkit.org/show_bug.cgi?id=192162
<rdar://problem/46345449>

Rubber-stamped by Aakash Jain.

Tools:

This change uses device_type to select a different set of TestExpectation files based on the type of device running
tests. This requires multiple baseline search paths to be used in a single instantiation of run-webkit-tests. Note
that while multiple baseline search paths are used, any single test will only ever be run on a single device type.

* Scripts/webkitpy/layout_tests/controllers/layout_test_finder.py:
(LayoutTestFinder.find_tests): Pass device_type through.
* Scripts/webkitpy/layout_tests/controllers/layout_test_finder_unittest.py:
(LayoutTestFinderTests.test_touched_but_skipped_test): Allow device_type to be passed in.
* Scripts/webkitpy/layout_tests/controllers/manager.py:
(Manager.__init__): Make _expectations an OrderedDict indexed by device_type, add current_device_type.
(Manager._collect_tests): Pass device type to test finder.
(Manager._needs_web_platform_test):
(Manager._prepare_lists): Index _expectations by device type.
(Manager._test_input_for_file):
(Manager._test_is_slow): Ditto.
(Manager._test_should_dump_jsconsolelog_in_stderr):
(Manager._test_should_dump_jsconsolelog_in_stderr): Ditto.
(Manager._get_test_inputs):
(Manager._update_worker_count):
(Manager._set_up_run): Use current_device_type instead of passing device_type in.
(Manager.run): Re-write function to gather and parse expectations from multiple baseline search paths. In the event
that a test can be run on multiple device types, the first device in the device type list which is eligible will be
the one which runs the test.
(Manager._run_test_subset):
(Manager._run_tests): Index _expectations by current_device_type.
(Manager._print_expectation_line_for_test): Ditto.
(Manager._print_expectations_for_subset): Ditto.
(Manager.print_expectations): Re-write function to gather and parse expectations from multiple baseline search paths.
Unlike run, this function will behave the same regardless of what devices are available.
(Manager._custom_device_for_test): Deleted.
* Scripts/webkitpy/layout_tests/controllers/manager_unittest.py:
(ManagerTest.test_uses_custom_device): Deleted.
(ManagerTest.test_uses_custom_device.get_manager): Deleted.
* Scripts/webkitpy/layout_tests/layout_package/json_layout_results_generator.py:
(JSONLayoutResultsGenerator.__init__): Pass expectations as a dictionary indexed by device type.
(JSONLayoutResultsGenerator._insert_failure_summaries): The number of fixable tests must take into consideration
all of the expectations.
* Scripts/webkitpy/layout_tests/lint_test_expectations_unittest.py:
(FakePort.skipped_layout_tests): Allow device_type to be passed in.
* Scripts/webkitpy/layout_tests/models/test_expectations.py:
(TestExpectations.__init__): Allow device_type to be passed through.
(TestExpectations.parse_all_expectations): Pick expectations_dict by device type.
* Scripts/webkitpy/layout_tests/models/test_expectations_unittest.py: Allow device_type to be passed in.
* Scripts/webkitpy/layout_tests/models/test_run_results.py:
(summarize_results):
* Scripts/webkitpy/layout_tests/models/test_run_results_unittest.py:
(summarized_results): Expectations are stored in a dictionary indexed by device_type. When determining the expectation
a test ran with, one must check all expectations to find the first non-skip expectation.
* Scripts/webkitpy/layout_tests/run_webkit_tests_integrationtest.py:
(RunTest.test_device_type_test_division):
(RunTest):
(RunTest.test_device_type_specific_listing):
* Scripts/webkitpy/port/base.py:
(Port): Add default values for DEVICE_TYPE and DEFAULT_DEVICE_TYPES.
(Port.supported_device_types): The result of this function will change based on what devices are available.
* Scripts/webkitpy/port/device_port.py:
(DevicePort._device_type_with_version): Replace DEFAULT_DEVICE_TYPE with DEVICE_TYPE.
(DevicePort.default_child_processes): DEVICE_TYPE is a general type, so the built-in comparison of the DeviceType object
can be used.
(DevicePort.supported_device_types): Generates a list of device types determined by the available devices, if there are
any, or the DEFAULT_DEVICE_TYPES of the port.
* Scripts/webkitpy/port/ios.py:
(IOSPort): Change DEFAULT_DEVICE_TYPE to DEVICE_TYPE.
* Scripts/webkitpy/port/ios_simulator.py:
(IOSSimulatorPort): Set the DEFAULT_DEVICE_TYPES to the devices currently used to run layout tests.
* Scripts/webkitpy/port/watch.py:
(WatchPort): Change DEFAULT_DEVICE_TYPE to DEVICE_TYPE.
* Scripts/webkitpy/port/watch_simulator.py:
(WatchSimulatorPort): Set the DEFAULT_DEVICE_TYPES to the device currently used to run layout tests.
* Scripts/webkitpy/xcode/device_type.py:
(DeviceType):
(DeviceType.__hash__): Allow DeviceType to be used as a dictionary index.

LayoutTests:

Move iPad and iPhone 7 tests to their respective platform specific directories.

* TestExpectations:
* fast/events/touch/ios/iphone7: Removed.
* fast/forms/ios/ipad: Removed.
* fast/text-autosizing/ios/ipad: Removed.
* fast/viewport/ios/ipad: Removed.
* media/controls/ipad: Removed.
* media/modern-media-controls/media-documents/ipad: Removed.
* media/modern-media-controls/pip-support/ipad: Removed.
* platform/ios-wk1/TestExpectations:
* platform/ios/TestExpectations:
* platform/ipad: Added.
* platform/ipad/fast: Added.
* platform/ipad/fast/forms: Added.
* platform/ipad/fast/forms/choose-select-option-expected.txt: Added.
* platform/ipad/fast/forms/choose-select-option.html: Added.
* platform/ipad/fast/forms/focus-input-via-button-expected.txt: Added.
* platform/ipad/fast/forms/focus-input-via-button.html: Added.
* platform/ipad/fast/forms/multiple-select-updates-renderer-expected.txt: Added.
* platform/ipad/fast/forms/multiple-select-updates-renderer.html: Added.
* platform/ipad/fast/forms/select-form-run-twice-expected.txt: Added.
* platform/ipad/fast/forms/select-form-run-twice.html: Added.
* platform/ipad/fast/forms/select-with-title-expected.txt: Added.
* platform/ipad/fast/forms/select-with-title.html: Added.
* platform/ipad/fast/forms/unfocus-inside-fixed-hittest-expected.txt: Added.
* platform/ipad/fast/forms/unfocus-inside-fixed-hittest.html: Added.
* platform/ipad/fast/text-autosizing: Added.
* platform/ipad/fast/text-autosizing/programmatic-text-size-adjust-expected.txt: Added.
* platform/ipad/fast/text-autosizing/programmatic-text-size-adjust.html: Added.
* platform/ipad/fast/text-autosizing/text-size-adjust-inline-style-expected.html: Added.
* platform/ipad/fast/text-autosizing/text-size-adjust-inline-style.html: Added.
* platform/ipad/fast/viewport: Added.
* platform/ipad/fast/viewport/empty-meta-expected.txt: Added.
* platform/ipad/fast/viewport/empty-meta.html: Added.
* platform/ipad/fast/viewport/meta-viewport-ignored-expected.txt: Added.
* platform/ipad/fast/viewport/meta-viewport-ignored.html: Added.
* platform/ipad/fast/viewport/viewport-overriden-by-minimum-effective-width-if-ignore-meta-viewport-expected.txt: Added.
* platform/ipad/fast/viewport/viewport-overriden-by-minimum-effective-width-if-ignore-meta-viewport.html: Added.
* platform/ipad/fast/viewport/viewport-unchanged-by-minimum-effective-width-if-not-ignore-meta-viewport-expected.txt: Added.
* platform/ipad/fast/viewport/viewport-unchanged-by-minimum-effective-width-if-not-ignore-meta-viewport.html: Added.
* platform/ipad/fast/viewport/width-is-device-width-expected.txt: Added.
* platform/ipad/fast/viewport/width-is-device-width.html: Added.
* platform/ipad/media: Added.
* platform/ipad/media/controls: Added.
* platform/ipad/media/controls/close-page-with-picture-in-picture-video-assertion-failure-expected.txt: Added.
* platform/ipad/media/controls/close-page-with-picture-in-picture-video-assertion-failure.html: Added.
* platform/ipad/media/controls/resources: Added.
* platform/ipad/media/controls/resources/picture-in-picture.html: Added.
* platform/ipad/media/modern-media-controls: Added.
* platform/ipad/media/modern-media-controls/media-documents: Added.
* platform/ipad/media/modern-media-controls/media-documents/media-document-audio-ios-sizing-expected.txt: Added.
* platform/ipad/media/modern-media-controls/media-documents/media-document-audio-ios-sizing.html: Added.
* platform/ipad/media/modern-media-controls/media-documents/media-document-video-ios-sizing-expected.txt: Added.
* platform/ipad/media/modern-media-controls/media-documents/media-document-video-ios-sizing.html: Added.
* platform/ipad/media/modern-media-controls/pip-support: Added.
* platform/ipad/media/modern-media-controls/pip-support/pip-support-enabled-expected.txt: Added.
* platform/ipad/media/modern-media-controls/pip-support/pip-support-enabled.html: Added.
* platform/ipad/media/modern-media-controls/pip-support/pip-support-tap-expected.txt: Added.
* platform/ipad/media/modern-media-controls/pip-support/pip-support-tap.html: Added.
* platform/iphone-7: Added.
* platform/iphone-7/fast: Added.
* platform/iphone-7/fast/events: Added.
* platform/iphone-7/fast/events/touch: Added.
* platform/iphone-7/fast/events/touch/force-press-event-expected.txt: Added.
* platform/iphone-7/fast/events/touch/force-press-event.html: Added.
* platform/iphone-7/fast/events/touch/force-press-on-link-expected.txt: Added.
* platform/iphone-7/fast/events/touch/force-press-on-link.html: Added.
* platform/iphone-7/fast/events/touch/prevent-default-touchmove-prevents-scrolling-expected.txt: Added.
* platform/iphone-7/fast/events/touch/prevent-default-touchmove-prevents-scrolling.html: Added.
* platform/iphone-7/tiled-drawing: Added.
* platform/iphone-7/tiled-drawing/compositing-layers-deep-color-expected.txt: Added.
* platform/iphone-7/tiled-drawing/compositing-layers-deep-color.html: Added.
* platform/mac-wk2/TestExpectations:
* platform/mac/TestExpectations:
* tiled-drawing/ios: Removed.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@240150 268f45cc-cd09-0410-ab3c-d52691b4dbfc
70 files changed:
LayoutTests/ChangeLog
LayoutTests/TestExpectations
LayoutTests/platform/ios-wk1/TestExpectations
LayoutTests/platform/ios/TestExpectations
LayoutTests/platform/ipad/TestExpectations [new file with mode: 0644]
LayoutTests/platform/ipad/fast/forms/choose-select-option-expected.txt [moved from LayoutTests/fast/forms/ios/ipad/choose-select-option-expected.txt with 100% similarity]
LayoutTests/platform/ipad/fast/forms/choose-select-option.html [moved from LayoutTests/fast/forms/ios/ipad/choose-select-option.html with 96% similarity]
LayoutTests/platform/ipad/fast/forms/focus-input-via-button-expected.txt [moved from LayoutTests/fast/forms/ios/ipad/focus-input-via-button-expected.txt with 100% similarity]
LayoutTests/platform/ipad/fast/forms/focus-input-via-button.html [moved from LayoutTests/fast/forms/ios/ipad/focus-input-via-button.html with 90% similarity]
LayoutTests/platform/ipad/fast/forms/multiple-select-updates-renderer-expected.txt [moved from LayoutTests/fast/forms/ios/ipad/multiple-select-updates-renderer-expected.txt with 100% similarity]
LayoutTests/platform/ipad/fast/forms/multiple-select-updates-renderer.html [moved from LayoutTests/fast/forms/ios/ipad/multiple-select-updates-renderer.html with 100% similarity]
LayoutTests/platform/ipad/fast/forms/select-form-run-twice-expected.txt [moved from LayoutTests/fast/forms/ios/ipad/select-form-run-twice-expected.txt with 100% similarity]
LayoutTests/platform/ipad/fast/forms/select-form-run-twice.html [moved from LayoutTests/fast/forms/ios/ipad/select-form-run-twice.html with 98% similarity]
LayoutTests/platform/ipad/fast/forms/select-with-title-expected.txt [moved from LayoutTests/fast/forms/ios/ipad/select-with-title-expected.txt with 100% similarity]
LayoutTests/platform/ipad/fast/forms/select-with-title.html [moved from LayoutTests/fast/forms/ios/ipad/select-with-title.html with 100% similarity]
LayoutTests/platform/ipad/fast/forms/unfocus-inside-fixed-hittest-expected.txt [moved from LayoutTests/fast/forms/ios/ipad/unfocus-inside-fixed-hittest-expected.txt with 100% similarity]
LayoutTests/platform/ipad/fast/forms/unfocus-inside-fixed-hittest.html [moved from LayoutTests/fast/forms/ios/ipad/unfocus-inside-fixed-hittest.html with 97% similarity]
LayoutTests/platform/ipad/fast/text-autosizing/programmatic-text-size-adjust-expected.txt [moved from LayoutTests/fast/text-autosizing/ios/ipad/programmatic-text-size-adjust-expected.txt with 100% similarity]
LayoutTests/platform/ipad/fast/text-autosizing/programmatic-text-size-adjust.html [moved from LayoutTests/fast/text-autosizing/ios/ipad/programmatic-text-size-adjust.html with 100% similarity]
LayoutTests/platform/ipad/fast/text-autosizing/text-size-adjust-inline-style-expected.html [moved from LayoutTests/fast/text-autosizing/ios/ipad/text-size-adjust-inline-style-expected.html with 100% similarity]
LayoutTests/platform/ipad/fast/text-autosizing/text-size-adjust-inline-style.html [moved from LayoutTests/fast/text-autosizing/ios/ipad/text-size-adjust-inline-style.html with 100% similarity]
LayoutTests/platform/ipad/fast/viewport/empty-meta-expected.txt [moved from LayoutTests/fast/viewport/ios/ipad/empty-meta-expected.txt with 100% similarity]
LayoutTests/platform/ipad/fast/viewport/empty-meta.html [moved from LayoutTests/fast/viewport/ios/ipad/empty-meta.html with 62% similarity]
LayoutTests/platform/ipad/fast/viewport/meta-viewport-ignored-expected.txt [moved from LayoutTests/fast/viewport/ios/ipad/meta-viewport-ignored-expected.txt with 100% similarity]
LayoutTests/platform/ipad/fast/viewport/meta-viewport-ignored.html [moved from LayoutTests/fast/viewport/ios/ipad/meta-viewport-ignored.html with 70% similarity]
LayoutTests/platform/ipad/fast/viewport/viewport-overriden-by-minimum-effective-width-if-ignore-meta-viewport-expected.txt [moved from LayoutTests/fast/viewport/ios/ipad/viewport-overriden-by-minimum-effective-width-if-ignore-meta-viewport-expected.txt with 100% similarity]
LayoutTests/platform/ipad/fast/viewport/viewport-overriden-by-minimum-effective-width-if-ignore-meta-viewport.html [moved from LayoutTests/fast/viewport/ios/ipad/viewport-overriden-by-minimum-effective-width-if-ignore-meta-viewport.html with 100% similarity]
LayoutTests/platform/ipad/fast/viewport/viewport-unchanged-by-minimum-effective-width-if-not-ignore-meta-viewport-expected.txt [moved from LayoutTests/fast/viewport/ios/ipad/viewport-unchanged-by-minimum-effective-width-if-not-ignore-meta-viewport-expected.txt with 100% similarity]
LayoutTests/platform/ipad/fast/viewport/viewport-unchanged-by-minimum-effective-width-if-not-ignore-meta-viewport.html [moved from LayoutTests/fast/viewport/ios/ipad/viewport-unchanged-by-minimum-effective-width-if-not-ignore-meta-viewport.html with 100% similarity]
LayoutTests/platform/ipad/fast/viewport/width-is-device-width-expected.txt [moved from LayoutTests/fast/viewport/ios/ipad/width-is-device-width-expected.txt with 100% similarity]
LayoutTests/platform/ipad/fast/viewport/width-is-device-width.html [moved from LayoutTests/fast/viewport/ios/ipad/width-is-device-width.html with 69% similarity]
LayoutTests/platform/ipad/media/controls/close-page-with-picture-in-picture-video-assertion-failure-expected.txt [moved from LayoutTests/media/controls/ipad/close-page-with-picture-in-picture-video-assertion-failure-expected.txt with 100% similarity]
LayoutTests/platform/ipad/media/controls/close-page-with-picture-in-picture-video-assertion-failure.html [moved from LayoutTests/media/controls/ipad/close-page-with-picture-in-picture-video-assertion-failure.html with 93% similarity]
LayoutTests/platform/ipad/media/controls/resources/picture-in-picture.html [moved from LayoutTests/media/controls/ipad/resources/picture-in-picture.html with 83% similarity]
LayoutTests/platform/ipad/media/modern-media-controls/media-documents/media-document-audio-ios-sizing-expected.txt [moved from LayoutTests/media/modern-media-controls/media-documents/ipad/media-document-audio-ios-sizing-expected.txt with 100% similarity]
LayoutTests/platform/ipad/media/modern-media-controls/media-documents/media-document-audio-ios-sizing.html [moved from LayoutTests/media/modern-media-controls/media-documents/ipad/media-document-audio-ios-sizing.html with 72% similarity]
LayoutTests/platform/ipad/media/modern-media-controls/media-documents/media-document-video-ios-sizing-expected.txt [moved from LayoutTests/media/modern-media-controls/media-documents/ipad/media-document-video-ios-sizing-expected.txt with 100% similarity]
LayoutTests/platform/ipad/media/modern-media-controls/media-documents/media-document-video-ios-sizing.html [moved from LayoutTests/media/modern-media-controls/media-documents/ipad/media-document-video-ios-sizing.html with 73% similarity]
LayoutTests/platform/ipad/media/modern-media-controls/pip-support/pip-support-enabled-expected.txt [moved from LayoutTests/media/modern-media-controls/pip-support/ipad/pip-support-enabled-expected.txt with 100% similarity]
LayoutTests/platform/ipad/media/modern-media-controls/pip-support/pip-support-enabled.html [moved from LayoutTests/media/modern-media-controls/pip-support/ipad/pip-support-enabled.html with 72% similarity]
LayoutTests/platform/ipad/media/modern-media-controls/pip-support/pip-support-tap-expected.txt [moved from LayoutTests/media/modern-media-controls/pip-support/ipad/pip-support-tap-expected.txt with 100% similarity]
LayoutTests/platform/ipad/media/modern-media-controls/pip-support/pip-support-tap.html [moved from LayoutTests/media/modern-media-controls/pip-support/ipad/pip-support-tap.html with 68% similarity]
LayoutTests/platform/iphone-7/fast/events/touch/force-press-event-expected.txt [moved from LayoutTests/fast/events/touch/ios/iphone7/force-press-event-expected.txt with 100% similarity]
LayoutTests/platform/iphone-7/fast/events/touch/force-press-event.html [moved from LayoutTests/fast/events/touch/ios/iphone7/force-press-event.html with 100% similarity]
LayoutTests/platform/iphone-7/fast/events/touch/force-press-on-link-expected.txt [moved from LayoutTests/fast/events/touch/ios/iphone7/force-press-on-link-expected.txt with 100% similarity]
LayoutTests/platform/iphone-7/fast/events/touch/force-press-on-link.html [moved from LayoutTests/fast/events/touch/ios/iphone7/force-press-on-link.html with 100% similarity]
LayoutTests/platform/iphone-7/fast/events/touch/prevent-default-touchmove-prevents-scrolling-expected.txt [moved from LayoutTests/fast/events/touch/ios/iphone7/prevent-default-touchmove-prevents-scrolling-expected.txt with 100% similarity]
LayoutTests/platform/iphone-7/fast/events/touch/prevent-default-touchmove-prevents-scrolling.html [moved from LayoutTests/fast/events/touch/ios/iphone7/prevent-default-touchmove-prevents-scrolling.html with 100% similarity]
LayoutTests/platform/iphone-7/tiled-drawing/compositing-layers-deep-color-expected.txt [new file with mode: 0644]
LayoutTests/platform/iphone-7/tiled-drawing/compositing-layers-deep-color.html [new file with mode: 0644]
LayoutTests/platform/mac/TestExpectations
Tools/ChangeLog
Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_finder.py
Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_finder_unittest.py
Tools/Scripts/webkitpy/layout_tests/controllers/manager.py
Tools/Scripts/webkitpy/layout_tests/controllers/manager_unittest.py
Tools/Scripts/webkitpy/layout_tests/layout_package/json_layout_results_generator.py
Tools/Scripts/webkitpy/layout_tests/lint_test_expectations_unittest.py
Tools/Scripts/webkitpy/layout_tests/models/test_expectations.py
Tools/Scripts/webkitpy/layout_tests/models/test_expectations_unittest.py
Tools/Scripts/webkitpy/layout_tests/models/test_run_results.py
Tools/Scripts/webkitpy/layout_tests/models/test_run_results_unittest.py
Tools/Scripts/webkitpy/layout_tests/run_webkit_tests_integrationtest.py
Tools/Scripts/webkitpy/port/base.py
Tools/Scripts/webkitpy/port/device_port.py
Tools/Scripts/webkitpy/port/ios.py
Tools/Scripts/webkitpy/port/ios_simulator.py
Tools/Scripts/webkitpy/port/watch.py
Tools/Scripts/webkitpy/port/watch_simulator.py
Tools/Scripts/webkitpy/xcode/device_type.py