webkitpy: Implement device type specific expected results (Part 2)
authorjbedard@apple.com <jbedard@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 18 Jan 2019 16:37:59 +0000 (16:37 +0000)
committerjbedard@apple.com <jbedard@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 18 Jan 2019 16:37:59 +0000 (16:37 +0000)
https://bugs.webkit.org/show_bug.cgi?id=192162
<rdar://problem/46345449>

Rubber-stamped by Aakash Jain.

Tools:

This change uses device_type to select a different set of TestExpectation files based on the type of device running
tests. This requires multiple baseline search paths to be used in a single instantiation of run-webkit-tests. Note
that while multiple baseline search paths are used, any single test will only ever be run on a single device type.

* Scripts/webkitpy/layout_tests/controllers/layout_test_finder.py:
(LayoutTestFinder.find_tests): Pass device_type through.
* Scripts/webkitpy/layout_tests/controllers/layout_test_finder_unittest.py:
(LayoutTestFinderTests.test_touched_but_skipped_test): Allow device_type to be passed in.
* Scripts/webkitpy/layout_tests/controllers/manager.py:
(Manager.__init__): Make _expectations an OrderedDict indexed by device_type, add current_device_type.
(Manager._collect_tests): Pass device type to test finder.
(Manager._needs_web_platform_test):
(Manager._prepare_lists): Index _expectations by device type.
(Manager._test_input_for_file):
(Manager._test_is_slow): Ditto.
(Manager._test_should_dump_jsconsolelog_in_stderr):
(Manager._test_should_dump_jsconsolelog_in_stderr): Ditto.
(Manager._get_test_inputs):
(Manager._update_worker_count):
(Manager._set_up_run): Use current_device_type instead of passing device_type in.
(Manager.run): Re-write function to gather and parse expectations from multiple baseline search paths. In the event
that a test can be run on multiple device types, the first device in the device type list which is eligible will be
the one which runs the test.
(Manager._run_test_subset):
(Manager._run_tests): Index _expectations by current_device_type.
(Manager._print_expectation_line_for_test): Ditto.
(Manager._print_expectations_for_subset): Ditto.
(Manager.print_expectations): Re-write function to gather and parse expectations from multiple baseline search paths.
Unlike run, this function will behave the same regardless of what devices are available.
(Manager._custom_device_for_test): Deleted.
* Scripts/webkitpy/layout_tests/controllers/manager_unittest.py:
(ManagerTest.test_uses_custom_device): Deleted.
(ManagerTest.test_uses_custom_device.get_manager): Deleted.
* Scripts/webkitpy/layout_tests/layout_package/json_layout_results_generator.py:
(JSONLayoutResultsGenerator.__init__): Pass expectations as a dictionary indexed by device type.
(JSONLayoutResultsGenerator._insert_failure_summaries): The number of fixable tests must take into consideration
all of the expectations.
* Scripts/webkitpy/layout_tests/lint_test_expectations_unittest.py:
(FakePort.skipped_layout_tests): Allow device_type to be passed in.
* Scripts/webkitpy/layout_tests/models/test_expectations.py:
(TestExpectations.__init__): Allow device_type to be passed through.
(TestExpectations.parse_all_expectations): Pick expectations_dict by device type.
* Scripts/webkitpy/layout_tests/models/test_expectations_unittest.py: Allow device_type to be passed in.
* Scripts/webkitpy/layout_tests/models/test_run_results.py:
(summarize_results):
* Scripts/webkitpy/layout_tests/models/test_run_results_unittest.py:
(summarized_results): Expectations are stored in a dictionary indexed by device_type. When determining the expectation
a test ran with, one must check all expectations to find the first non-skip expectation.
* Scripts/webkitpy/layout_tests/run_webkit_tests_integrationtest.py:
(RunTest.test_device_type_test_division):
(RunTest):
(RunTest.test_device_type_specific_listing):
* Scripts/webkitpy/port/base.py:
(Port): Add default values for DEVICE_TYPE and DEFAULT_DEVICE_TYPES.
(Port.supported_device_types): The result of this function will change based on what devices are available.
* Scripts/webkitpy/port/device_port.py:
(DevicePort._device_type_with_version): Replace DEFAULT_DEVICE_TYPE with DEVICE_TYPE.
(DevicePort.default_child_processes): DEVICE_TYPE is a general type, so the built-in comparison of the DeviceType object
can be used.
(DevicePort.supported_device_types): Generates a list of device types determined by the available devices, if there are
any, or the DEFAULT_DEVICE_TYPES of the port.
* Scripts/webkitpy/port/ios.py:
(IOSPort): Change DEFAULT_DEVICE_TYPE to DEVICE_TYPE.
* Scripts/webkitpy/port/ios_simulator.py:
(IOSSimulatorPort): Set the DEFAULT_DEVICE_TYPES to the devices currently used to run layout tests.
* Scripts/webkitpy/port/watch.py:
(WatchPort): Change DEFAULT_DEVICE_TYPE to DEVICE_TYPE.
* Scripts/webkitpy/port/watch_simulator.py:
(WatchSimulatorPort): Set the DEFAULT_DEVICE_TYPES to the device currently used to run layout tests.
* Scripts/webkitpy/xcode/device_type.py:
(DeviceType):
(DeviceType.__hash__): Allow DeviceType to be used as a dictionary index.

LayoutTests:

Move iPad and iPhone 7 tests to their respective platform specific directories.

* TestExpectations:
* fast/events/touch/ios/iphone7: Removed.
* fast/forms/ios/ipad: Removed.
* fast/text-autosizing/ios/ipad: Removed.
* fast/viewport/ios/ipad: Removed.
* media/controls/ipad: Removed.
* media/modern-media-controls/media-documents/ipad: Removed.
* media/modern-media-controls/pip-support/ipad: Removed.
* platform/ios-wk1/TestExpectations:
* platform/ios/TestExpectations:
* platform/ipad: Added.
* platform/ipad/fast: Added.
* platform/ipad/fast/forms: Added.
* platform/ipad/fast/forms/choose-select-option-expected.txt: Added.
* platform/ipad/fast/forms/choose-select-option.html: Added.
* platform/ipad/fast/forms/focus-input-via-button-expected.txt: Added.
* platform/ipad/fast/forms/focus-input-via-button.html: Added.
* platform/ipad/fast/forms/multiple-select-updates-renderer-expected.txt: Added.
* platform/ipad/fast/forms/multiple-select-updates-renderer.html: Added.
* platform/ipad/fast/forms/select-form-run-twice-expected.txt: Added.
* platform/ipad/fast/forms/select-form-run-twice.html: Added.
* platform/ipad/fast/forms/select-with-title-expected.txt: Added.
* platform/ipad/fast/forms/select-with-title.html: Added.
* platform/ipad/fast/forms/unfocus-inside-fixed-hittest-expected.txt: Added.
* platform/ipad/fast/forms/unfocus-inside-fixed-hittest.html: Added.
* platform/ipad/fast/text-autosizing: Added.
* platform/ipad/fast/text-autosizing/programmatic-text-size-adjust-expected.txt: Added.
* platform/ipad/fast/text-autosizing/programmatic-text-size-adjust.html: Added.
* platform/ipad/fast/text-autosizing/text-size-adjust-inline-style-expected.html: Added.
* platform/ipad/fast/text-autosizing/text-size-adjust-inline-style.html: Added.
* platform/ipad/fast/viewport: Added.
* platform/ipad/fast/viewport/empty-meta-expected.txt: Added.
* platform/ipad/fast/viewport/empty-meta.html: Added.
* platform/ipad/fast/viewport/meta-viewport-ignored-expected.txt: Added.
* platform/ipad/fast/viewport/meta-viewport-ignored.html: Added.
* platform/ipad/fast/viewport/viewport-overriden-by-minimum-effective-width-if-ignore-meta-viewport-expected.txt: Added.
* platform/ipad/fast/viewport/viewport-overriden-by-minimum-effective-width-if-ignore-meta-viewport.html: Added.
* platform/ipad/fast/viewport/viewport-unchanged-by-minimum-effective-width-if-not-ignore-meta-viewport-expected.txt: Added.
* platform/ipad/fast/viewport/viewport-unchanged-by-minimum-effective-width-if-not-ignore-meta-viewport.html: Added.
* platform/ipad/fast/viewport/width-is-device-width-expected.txt: Added.
* platform/ipad/fast/viewport/width-is-device-width.html: Added.
* platform/ipad/media: Added.
* platform/ipad/media/controls: Added.
* platform/ipad/media/controls/close-page-with-picture-in-picture-video-assertion-failure-expected.txt: Added.
* platform/ipad/media/controls/close-page-with-picture-in-picture-video-assertion-failure.html: Added.
* platform/ipad/media/controls/resources: Added.
* platform/ipad/media/controls/resources/picture-in-picture.html: Added.
* platform/ipad/media/modern-media-controls: Added.
* platform/ipad/media/modern-media-controls/media-documents: Added.
* platform/ipad/media/modern-media-controls/media-documents/media-document-audio-ios-sizing-expected.txt: Added.
* platform/ipad/media/modern-media-controls/media-documents/media-document-audio-ios-sizing.html: Added.
* platform/ipad/media/modern-media-controls/media-documents/media-document-video-ios-sizing-expected.txt: Added.
* platform/ipad/media/modern-media-controls/media-documents/media-document-video-ios-sizing.html: Added.
* platform/ipad/media/modern-media-controls/pip-support: Added.
* platform/ipad/media/modern-media-controls/pip-support/pip-support-enabled-expected.txt: Added.
* platform/ipad/media/modern-media-controls/pip-support/pip-support-enabled.html: Added.
* platform/ipad/media/modern-media-controls/pip-support/pip-support-tap-expected.txt: Added.
* platform/ipad/media/modern-media-controls/pip-support/pip-support-tap.html: Added.
* platform/iphone-7: Added.
* platform/iphone-7/fast: Added.
* platform/iphone-7/fast/events: Added.
* platform/iphone-7/fast/events/touch: Added.
* platform/iphone-7/fast/events/touch/force-press-event-expected.txt: Added.
* platform/iphone-7/fast/events/touch/force-press-event.html: Added.
* platform/iphone-7/fast/events/touch/force-press-on-link-expected.txt: Added.
* platform/iphone-7/fast/events/touch/force-press-on-link.html: Added.
* platform/iphone-7/fast/events/touch/prevent-default-touchmove-prevents-scrolling-expected.txt: Added.
* platform/iphone-7/fast/events/touch/prevent-default-touchmove-prevents-scrolling.html: Added.
* platform/iphone-7/tiled-drawing: Added.
* platform/iphone-7/tiled-drawing/compositing-layers-deep-color-expected.txt: Added.
* platform/iphone-7/tiled-drawing/compositing-layers-deep-color.html: Added.
* platform/mac-wk2/TestExpectations:
* platform/mac/TestExpectations:
* tiled-drawing/ios: Removed.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@240150 268f45cc-cd09-0410-ab3c-d52691b4dbfc

70 files changed:
LayoutTests/ChangeLog
LayoutTests/TestExpectations
LayoutTests/platform/ios-wk1/TestExpectations
LayoutTests/platform/ios/TestExpectations
LayoutTests/platform/ipad/TestExpectations [new file with mode: 0644]
LayoutTests/platform/ipad/fast/forms/choose-select-option-expected.txt [moved from LayoutTests/fast/forms/ios/ipad/choose-select-option-expected.txt with 100% similarity]
LayoutTests/platform/ipad/fast/forms/choose-select-option.html [moved from LayoutTests/fast/forms/ios/ipad/choose-select-option.html with 96% similarity]
LayoutTests/platform/ipad/fast/forms/focus-input-via-button-expected.txt [moved from LayoutTests/fast/forms/ios/ipad/focus-input-via-button-expected.txt with 100% similarity]
LayoutTests/platform/ipad/fast/forms/focus-input-via-button.html [moved from LayoutTests/fast/forms/ios/ipad/focus-input-via-button.html with 90% similarity]
LayoutTests/platform/ipad/fast/forms/multiple-select-updates-renderer-expected.txt [moved from LayoutTests/fast/forms/ios/ipad/multiple-select-updates-renderer-expected.txt with 100% similarity]
LayoutTests/platform/ipad/fast/forms/multiple-select-updates-renderer.html [moved from LayoutTests/fast/forms/ios/ipad/multiple-select-updates-renderer.html with 100% similarity]
LayoutTests/platform/ipad/fast/forms/select-form-run-twice-expected.txt [moved from LayoutTests/fast/forms/ios/ipad/select-form-run-twice-expected.txt with 100% similarity]
LayoutTests/platform/ipad/fast/forms/select-form-run-twice.html [moved from LayoutTests/fast/forms/ios/ipad/select-form-run-twice.html with 98% similarity]
LayoutTests/platform/ipad/fast/forms/select-with-title-expected.txt [moved from LayoutTests/fast/forms/ios/ipad/select-with-title-expected.txt with 100% similarity]
LayoutTests/platform/ipad/fast/forms/select-with-title.html [moved from LayoutTests/fast/forms/ios/ipad/select-with-title.html with 100% similarity]
LayoutTests/platform/ipad/fast/forms/unfocus-inside-fixed-hittest-expected.txt [moved from LayoutTests/fast/forms/ios/ipad/unfocus-inside-fixed-hittest-expected.txt with 100% similarity]
LayoutTests/platform/ipad/fast/forms/unfocus-inside-fixed-hittest.html [moved from LayoutTests/fast/forms/ios/ipad/unfocus-inside-fixed-hittest.html with 97% similarity]
LayoutTests/platform/ipad/fast/text-autosizing/programmatic-text-size-adjust-expected.txt [moved from LayoutTests/fast/text-autosizing/ios/ipad/programmatic-text-size-adjust-expected.txt with 100% similarity]
LayoutTests/platform/ipad/fast/text-autosizing/programmatic-text-size-adjust.html [moved from LayoutTests/fast/text-autosizing/ios/ipad/programmatic-text-size-adjust.html with 100% similarity]
LayoutTests/platform/ipad/fast/text-autosizing/text-size-adjust-inline-style-expected.html [moved from LayoutTests/fast/text-autosizing/ios/ipad/text-size-adjust-inline-style-expected.html with 100% similarity]
LayoutTests/platform/ipad/fast/text-autosizing/text-size-adjust-inline-style.html [moved from LayoutTests/fast/text-autosizing/ios/ipad/text-size-adjust-inline-style.html with 100% similarity]
LayoutTests/platform/ipad/fast/viewport/empty-meta-expected.txt [moved from LayoutTests/fast/viewport/ios/ipad/empty-meta-expected.txt with 100% similarity]
LayoutTests/platform/ipad/fast/viewport/empty-meta.html [moved from LayoutTests/fast/viewport/ios/ipad/empty-meta.html with 62% similarity]
LayoutTests/platform/ipad/fast/viewport/meta-viewport-ignored-expected.txt [moved from LayoutTests/fast/viewport/ios/ipad/meta-viewport-ignored-expected.txt with 100% similarity]
LayoutTests/platform/ipad/fast/viewport/meta-viewport-ignored.html [moved from LayoutTests/fast/viewport/ios/ipad/meta-viewport-ignored.html with 70% similarity]
LayoutTests/platform/ipad/fast/viewport/viewport-overriden-by-minimum-effective-width-if-ignore-meta-viewport-expected.txt [moved from LayoutTests/fast/viewport/ios/ipad/viewport-overriden-by-minimum-effective-width-if-ignore-meta-viewport-expected.txt with 100% similarity]
LayoutTests/platform/ipad/fast/viewport/viewport-overriden-by-minimum-effective-width-if-ignore-meta-viewport.html [moved from LayoutTests/fast/viewport/ios/ipad/viewport-overriden-by-minimum-effective-width-if-ignore-meta-viewport.html with 100% similarity]
LayoutTests/platform/ipad/fast/viewport/viewport-unchanged-by-minimum-effective-width-if-not-ignore-meta-viewport-expected.txt [moved from LayoutTests/fast/viewport/ios/ipad/viewport-unchanged-by-minimum-effective-width-if-not-ignore-meta-viewport-expected.txt with 100% similarity]
LayoutTests/platform/ipad/fast/viewport/viewport-unchanged-by-minimum-effective-width-if-not-ignore-meta-viewport.html [moved from LayoutTests/fast/viewport/ios/ipad/viewport-unchanged-by-minimum-effective-width-if-not-ignore-meta-viewport.html with 100% similarity]
LayoutTests/platform/ipad/fast/viewport/width-is-device-width-expected.txt [moved from LayoutTests/fast/viewport/ios/ipad/width-is-device-width-expected.txt with 100% similarity]
LayoutTests/platform/ipad/fast/viewport/width-is-device-width.html [moved from LayoutTests/fast/viewport/ios/ipad/width-is-device-width.html with 69% similarity]
LayoutTests/platform/ipad/media/controls/close-page-with-picture-in-picture-video-assertion-failure-expected.txt [moved from LayoutTests/media/controls/ipad/close-page-with-picture-in-picture-video-assertion-failure-expected.txt with 100% similarity]
LayoutTests/platform/ipad/media/controls/close-page-with-picture-in-picture-video-assertion-failure.html [moved from LayoutTests/media/controls/ipad/close-page-with-picture-in-picture-video-assertion-failure.html with 93% similarity]
LayoutTests/platform/ipad/media/controls/resources/picture-in-picture.html [moved from LayoutTests/media/controls/ipad/resources/picture-in-picture.html with 83% similarity]
LayoutTests/platform/ipad/media/modern-media-controls/media-documents/media-document-audio-ios-sizing-expected.txt [moved from LayoutTests/media/modern-media-controls/media-documents/ipad/media-document-audio-ios-sizing-expected.txt with 100% similarity]
LayoutTests/platform/ipad/media/modern-media-controls/media-documents/media-document-audio-ios-sizing.html [moved from LayoutTests/media/modern-media-controls/media-documents/ipad/media-document-audio-ios-sizing.html with 72% similarity]
LayoutTests/platform/ipad/media/modern-media-controls/media-documents/media-document-video-ios-sizing-expected.txt [moved from LayoutTests/media/modern-media-controls/media-documents/ipad/media-document-video-ios-sizing-expected.txt with 100% similarity]
LayoutTests/platform/ipad/media/modern-media-controls/media-documents/media-document-video-ios-sizing.html [moved from LayoutTests/media/modern-media-controls/media-documents/ipad/media-document-video-ios-sizing.html with 73% similarity]
LayoutTests/platform/ipad/media/modern-media-controls/pip-support/pip-support-enabled-expected.txt [moved from LayoutTests/media/modern-media-controls/pip-support/ipad/pip-support-enabled-expected.txt with 100% similarity]
LayoutTests/platform/ipad/media/modern-media-controls/pip-support/pip-support-enabled.html [moved from LayoutTests/media/modern-media-controls/pip-support/ipad/pip-support-enabled.html with 72% similarity]
LayoutTests/platform/ipad/media/modern-media-controls/pip-support/pip-support-tap-expected.txt [moved from LayoutTests/media/modern-media-controls/pip-support/ipad/pip-support-tap-expected.txt with 100% similarity]
LayoutTests/platform/ipad/media/modern-media-controls/pip-support/pip-support-tap.html [moved from LayoutTests/media/modern-media-controls/pip-support/ipad/pip-support-tap.html with 68% similarity]
LayoutTests/platform/iphone-7/fast/events/touch/force-press-event-expected.txt [moved from LayoutTests/fast/events/touch/ios/iphone7/force-press-event-expected.txt with 100% similarity]
LayoutTests/platform/iphone-7/fast/events/touch/force-press-event.html [moved from LayoutTests/fast/events/touch/ios/iphone7/force-press-event.html with 100% similarity]
LayoutTests/platform/iphone-7/fast/events/touch/force-press-on-link-expected.txt [moved from LayoutTests/fast/events/touch/ios/iphone7/force-press-on-link-expected.txt with 100% similarity]
LayoutTests/platform/iphone-7/fast/events/touch/force-press-on-link.html [moved from LayoutTests/fast/events/touch/ios/iphone7/force-press-on-link.html with 100% similarity]
LayoutTests/platform/iphone-7/fast/events/touch/prevent-default-touchmove-prevents-scrolling-expected.txt [moved from LayoutTests/fast/events/touch/ios/iphone7/prevent-default-touchmove-prevents-scrolling-expected.txt with 100% similarity]
LayoutTests/platform/iphone-7/fast/events/touch/prevent-default-touchmove-prevents-scrolling.html [moved from LayoutTests/fast/events/touch/ios/iphone7/prevent-default-touchmove-prevents-scrolling.html with 100% similarity]
LayoutTests/platform/iphone-7/tiled-drawing/compositing-layers-deep-color-expected.txt [new file with mode: 0644]
LayoutTests/platform/iphone-7/tiled-drawing/compositing-layers-deep-color.html [new file with mode: 0644]
LayoutTests/platform/mac/TestExpectations
Tools/ChangeLog
Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_finder.py
Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_finder_unittest.py
Tools/Scripts/webkitpy/layout_tests/controllers/manager.py
Tools/Scripts/webkitpy/layout_tests/controllers/manager_unittest.py
Tools/Scripts/webkitpy/layout_tests/layout_package/json_layout_results_generator.py
Tools/Scripts/webkitpy/layout_tests/lint_test_expectations_unittest.py
Tools/Scripts/webkitpy/layout_tests/models/test_expectations.py
Tools/Scripts/webkitpy/layout_tests/models/test_expectations_unittest.py
Tools/Scripts/webkitpy/layout_tests/models/test_run_results.py
Tools/Scripts/webkitpy/layout_tests/models/test_run_results_unittest.py
Tools/Scripts/webkitpy/layout_tests/run_webkit_tests_integrationtest.py
Tools/Scripts/webkitpy/port/base.py
Tools/Scripts/webkitpy/port/device_port.py
Tools/Scripts/webkitpy/port/ios.py
Tools/Scripts/webkitpy/port/ios_simulator.py
Tools/Scripts/webkitpy/port/watch.py
Tools/Scripts/webkitpy/port/watch_simulator.py
Tools/Scripts/webkitpy/xcode/device_type.py

index cbcbd31..4bc8f13 100644 (file)
@@ -1,3 +1,88 @@
+2019-01-18  Jonathan Bedard  <jbedard@apple.com>
+
+        webkitpy: Implement device type specific expected results (Part 2)
+        https://bugs.webkit.org/show_bug.cgi?id=192162
+        <rdar://problem/46345449>
+
+        Rubber-stamped by Aakash Jain.
+
+        Move iPad and iPhone 7 tests to their respective platform specific directories.
+
+        * TestExpectations:
+        * fast/events/touch/ios/iphone7: Removed.
+        * fast/forms/ios/ipad: Removed.
+        * fast/text-autosizing/ios/ipad: Removed.
+        * fast/viewport/ios/ipad: Removed.
+        * media/controls/ipad: Removed.
+        * media/modern-media-controls/media-documents/ipad: Removed.
+        * media/modern-media-controls/pip-support/ipad: Removed.
+        * platform/ios-wk1/TestExpectations:
+        * platform/ios/TestExpectations:
+        * platform/ipad: Added.
+        * platform/ipad/fast: Added.
+        * platform/ipad/fast/forms: Added.
+        * platform/ipad/fast/forms/choose-select-option-expected.txt: Added.
+        * platform/ipad/fast/forms/choose-select-option.html: Added.
+        * platform/ipad/fast/forms/focus-input-via-button-expected.txt: Added.
+        * platform/ipad/fast/forms/focus-input-via-button.html: Added.
+        * platform/ipad/fast/forms/multiple-select-updates-renderer-expected.txt: Added.
+        * platform/ipad/fast/forms/multiple-select-updates-renderer.html: Added.
+        * platform/ipad/fast/forms/select-form-run-twice-expected.txt: Added.
+        * platform/ipad/fast/forms/select-form-run-twice.html: Added.
+        * platform/ipad/fast/forms/select-with-title-expected.txt: Added.
+        * platform/ipad/fast/forms/select-with-title.html: Added.
+        * platform/ipad/fast/forms/unfocus-inside-fixed-hittest-expected.txt: Added.
+        * platform/ipad/fast/forms/unfocus-inside-fixed-hittest.html: Added.
+        * platform/ipad/fast/text-autosizing: Added.
+        * platform/ipad/fast/text-autosizing/programmatic-text-size-adjust-expected.txt: Added.
+        * platform/ipad/fast/text-autosizing/programmatic-text-size-adjust.html: Added.
+        * platform/ipad/fast/text-autosizing/text-size-adjust-inline-style-expected.html: Added.
+        * platform/ipad/fast/text-autosizing/text-size-adjust-inline-style.html: Added.
+        * platform/ipad/fast/viewport: Added.
+        * platform/ipad/fast/viewport/empty-meta-expected.txt: Added.
+        * platform/ipad/fast/viewport/empty-meta.html: Added.
+        * platform/ipad/fast/viewport/meta-viewport-ignored-expected.txt: Added.
+        * platform/ipad/fast/viewport/meta-viewport-ignored.html: Added.
+        * platform/ipad/fast/viewport/viewport-overriden-by-minimum-effective-width-if-ignore-meta-viewport-expected.txt: Added.
+        * platform/ipad/fast/viewport/viewport-overriden-by-minimum-effective-width-if-ignore-meta-viewport.html: Added.
+        * platform/ipad/fast/viewport/viewport-unchanged-by-minimum-effective-width-if-not-ignore-meta-viewport-expected.txt: Added.
+        * platform/ipad/fast/viewport/viewport-unchanged-by-minimum-effective-width-if-not-ignore-meta-viewport.html: Added.
+        * platform/ipad/fast/viewport/width-is-device-width-expected.txt: Added.
+        * platform/ipad/fast/viewport/width-is-device-width.html: Added.
+        * platform/ipad/media: Added.
+        * platform/ipad/media/controls: Added.
+        * platform/ipad/media/controls/close-page-with-picture-in-picture-video-assertion-failure-expected.txt: Added.
+        * platform/ipad/media/controls/close-page-with-picture-in-picture-video-assertion-failure.html: Added.
+        * platform/ipad/media/controls/resources: Added.
+        * platform/ipad/media/controls/resources/picture-in-picture.html: Added.
+        * platform/ipad/media/modern-media-controls: Added.
+        * platform/ipad/media/modern-media-controls/media-documents: Added.
+        * platform/ipad/media/modern-media-controls/media-documents/media-document-audio-ios-sizing-expected.txt: Added.
+        * platform/ipad/media/modern-media-controls/media-documents/media-document-audio-ios-sizing.html: Added.
+        * platform/ipad/media/modern-media-controls/media-documents/media-document-video-ios-sizing-expected.txt: Added.
+        * platform/ipad/media/modern-media-controls/media-documents/media-document-video-ios-sizing.html: Added.
+        * platform/ipad/media/modern-media-controls/pip-support: Added.
+        * platform/ipad/media/modern-media-controls/pip-support/pip-support-enabled-expected.txt: Added.
+        * platform/ipad/media/modern-media-controls/pip-support/pip-support-enabled.html: Added.
+        * platform/ipad/media/modern-media-controls/pip-support/pip-support-tap-expected.txt: Added.
+        * platform/ipad/media/modern-media-controls/pip-support/pip-support-tap.html: Added.
+        * platform/iphone-7: Added.
+        * platform/iphone-7/fast: Added.
+        * platform/iphone-7/fast/events: Added.
+        * platform/iphone-7/fast/events/touch: Added.
+        * platform/iphone-7/fast/events/touch/force-press-event-expected.txt: Added.
+        * platform/iphone-7/fast/events/touch/force-press-event.html: Added.
+        * platform/iphone-7/fast/events/touch/force-press-on-link-expected.txt: Added.
+        * platform/iphone-7/fast/events/touch/force-press-on-link.html: Added.
+        * platform/iphone-7/fast/events/touch/prevent-default-touchmove-prevents-scrolling-expected.txt: Added.
+        * platform/iphone-7/fast/events/touch/prevent-default-touchmove-prevents-scrolling.html: Added.
+        * platform/iphone-7/tiled-drawing: Added.
+        * platform/iphone-7/tiled-drawing/compositing-layers-deep-color-expected.txt: Added.
+        * platform/iphone-7/tiled-drawing/compositing-layers-deep-color.html: Added.
+        * platform/mac-wk2/TestExpectations:
+        * platform/mac/TestExpectations:
+        * tiled-drawing/ios: Removed.
+
 2019-01-18  Zalan Bujtas  <zalan@apple.com>
 
         [LFC][BFC][MarginCollapsing] Collapsing through should not ignore floats.
 2019-01-18  Zalan Bujtas  <zalan@apple.com>
 
         [LFC][BFC][MarginCollapsing] Collapsing through should not ignore floats.
index 0f05fcb..45fb847 100644 (file)
@@ -43,7 +43,6 @@ scrollingcoordinator/ios [ Skip ]
 fast/content-observation [ Skip ]
 media/mac [ Skip ]
 media/ios [ Skip ]
 fast/content-observation [ Skip ]
 media/mac [ Skip ]
 media/ios [ Skip ]
-media/controls/ipad [ Skip ]
 fast/text-autosizing [ Skip ]
 fast/css/variables/env/ios [ Skip ]
 http/tests/events/touch/ios [ Skip ]
 fast/text-autosizing [ Skip ]
 fast/css/variables/env/ios [ Skip ]
 http/tests/events/touch/ios [ Skip ]
index f4f8266..edd696f 100644 (file)
@@ -2,9 +2,6 @@
 #
 # See http://trac.webkit.org/wiki/TestExpectations for more information on this file.
 
 #
 # See http://trac.webkit.org/wiki/TestExpectations for more information on this file.
 
-# Selectively re-enable viewport tests that work
-fast/viewport/ios/ipad/width-is-device-width.html [ Pass ]
-
 editing/input/focus-change-with-marked-text.html [ Pass ]
 
 # No service worker implementation for WK1
 editing/input/focus-change-with-marked-text.html [ Pass ]
 
 # No service worker implementation for WK1
@@ -1902,7 +1899,6 @@ fast/forms/selection-direction.html [ Timeout ]
 
 # Failure as of 09/21/2016
 css2.1/t1008-c44-ln-box-02-d-ag.html [ Failure ]
 
 # Failure as of 09/21/2016
 css2.1/t1008-c44-ln-box-02-d-ag.html [ Failure ]
-fast/text-autosizing/ios/ipad/text-size-adjust-inline-style.html [ ImageOnlyFailure ]
 fast/text-autosizing/ios/text-size-adjust-inline-style.html [ ImageOnlyFailure ]
 
 # Flaky as of 09/21/2016
 fast/text-autosizing/ios/text-size-adjust-inline-style.html [ ImageOnlyFailure ]
 
 # Flaky as of 09/21/2016
index 08b7d33..49dd3f6 100644 (file)
@@ -190,8 +190,11 @@ platform/ios/ios/fast/forms/range-input-touches.html [ Skip ]
 platform/ios/ios/fast/text/data-detectors/phone-disabled.html [ Skip ]
 platform/ios/ios/fast/text/data-detectors/phone.html [ Skip ]
 platform/ios/ios/touch [ Skip ]
 platform/ios/ios/fast/text/data-detectors/phone-disabled.html [ Skip ]
 platform/ios/ios/fast/text/data-detectors/phone.html [ Skip ]
 platform/ios/ios/touch [ Skip ]
+platform/iphone-7/fast/events/touch [ Skip ]
 media/modern-media-controls/css/pointer-events-none.html [ Skip ]
 
 media/modern-media-controls/css/pointer-events-none.html [ Skip ]
 
+platform/ipad/fast/forms [ Skip ]
+
 # Codecs not supported
 media/media-can-play-ogg.html [ WontFix ]
 media/media-can-play-flac-audio.html  [ WontFix ]
 # Codecs not supported
 media/media-can-play-ogg.html [ WontFix ]
 media/media-can-play-flac-audio.html  [ WontFix ]
@@ -2971,7 +2974,6 @@ media/modern-media-controls/fullscreen-support/fullscreen-support-press.html [ S
 media/modern-media-controls/media-controller/ios/media-controller-ios-only-enable-tap-gesture-recognizer-with-fades-when-idle.html [ Skip ]
 media/modern-media-controls/media-controller/ios/media-controller-stop-updates-in-fullscreen.html [ Skip ]
 media/modern-media-controls/mute-support/mute-support-press-on-button.html [ Skip ]
 media/modern-media-controls/media-controller/ios/media-controller-ios-only-enable-tap-gesture-recognizer-with-fades-when-idle.html [ Skip ]
 media/modern-media-controls/media-controller/ios/media-controller-stop-updates-in-fullscreen.html [ Skip ]
 media/modern-media-controls/mute-support/mute-support-press-on-button.html [ Skip ]
-media/modern-media-controls/pip-support/ipad/pip-support-tap.html [ Skip ]
 media/modern-media-controls/placard-support/ipad [ Skip ]
 media/modern-media-controls/scrubber-support/ipad/scrubber-support-drag.html [ Skip ]
 media/modern-media-controls/seek-backward-support [ Skip ]
 media/modern-media-controls/placard-support/ipad [ Skip ]
 media/modern-media-controls/scrubber-support/ipad/scrubber-support-drag.html [ Skip ]
 media/modern-media-controls/seek-backward-support [ Skip ]
@@ -3022,8 +3024,6 @@ media/modern-media-controls/tracks-panel [ Skip ]
 media/modern-media-controls/tracks-support [ Skip ]
 media/modern-media-controls/volume-support [ Skip ]
 
 media/modern-media-controls/tracks-support [ Skip ]
 media/modern-media-controls/volume-support [ Skip ]
 
-media/modern-media-controls/pip-support/ipad [ Pass ]
-
 webkit.org/b/178127 media/modern-media-controls/media-controller/media-controller-video-with-only-audio.html [ Skip ]
 
 # <rdar://problem/33731086>
 webkit.org/b/178127 media/modern-media-controls/media-controller/media-controller-video-with-only-audio.html [ Skip ]
 
 # <rdar://problem/33731086>
diff --git a/LayoutTests/platform/ipad/TestExpectations b/LayoutTests/platform/ipad/TestExpectations
new file mode 100644 (file)
index 0000000..a5cf137
--- /dev/null
@@ -0,0 +1 @@
+webkit.org/b/165311 platform/ipad/media/modern-media-controls/pip-support/pip-support-tap.html [ Skip ]
\ No newline at end of file
@@ -6,7 +6,7 @@
     <style>
     </style>
     
     <style>
     </style>
     
-    <script src="../resources/zooming-test-utils.js"></script>
+    <script src="../../../../fast/forms/ios/resources/zooming-test-utils.js"></script>
     <script>
     if (window.testRunner) {
         testRunner.dumpAsText();
     <script>
     if (window.testRunner) {
         testRunner.dumpAsText();
@@ -12,7 +12,7 @@
         }
     </style>
     
         }
     </style>
     
-    <script src="../resources/zooming-test-utils.js"></script>
+    <script src="../../../../fast/forms/ios/resources/zooming-test-utils.js"></script>
     <script>
     if (window.testRunner) {
         testRunner.dumpAsText();
     <script>
     if (window.testRunner) {
         testRunner.dumpAsText();
@@ -29,7 +29,7 @@
         }
         
     </style>
         }
         
     </style>
-    <script src="../resources/zooming-test-utils.js"></script>
+    <script src="../../../../fast/forms/ios/resources/zooming-test-utils.js"></script>
     <script src="../../../../resources/basic-gestures.js"></script>
     <script>
         if (window.testRunner)
     <script src="../../../../resources/basic-gestures.js"></script>
     <script>
         if (window.testRunner)
@@ -29,7 +29,7 @@
         }
         
     </style>
         }
         
     </style>
-    <script src="../resources/zooming-test-utils.js"></script>
+    <script src="../../../../fast/forms/ios/resources/zooming-test-utils.js"></script>
     <script src="../../../../resources/basic-gestures.js"></script>
     <script>
         if (window.testRunner)
     <script src="../../../../resources/basic-gestures.js"></script>
     <script>
         if (window.testRunner)
@@ -2,7 +2,7 @@
 
 <html>
 <head>
 
 <html>
 <head>
-    <script src="../resources/viewport-test-utils.js"></script>
+    <script src="../../../../fast/viewport/ios/resources/viewport-test-utils.js"></script>
 </head>
 <body onload="runTest()">
 
 </head>
 <body onload="runTest()">
 
@@ -2,7 +2,7 @@
 
 <html>
 <head>
 
 <html>
 <head>
-    <script src="../resources/viewport-test-utils.js"></script>
+    <script src="../../../../fast/viewport/ios/resources/viewport-test-utils.js"></script>
 </head>
 <body onload="runTest()">
 
 </head>
 <body onload="runTest()">
 
@@ -3,7 +3,7 @@
 <html>
 <head>
     <meta name="viewport" content="width=device-width">
 <html>
 <head>
     <meta name="viewport" content="width=device-width">
-    <script src="../resources/viewport-test-utils.js"></script>
+    <script src="../../../../fast/viewport/ios/resources/viewport-test-utils.js"></script>
 </head>
 <body onload="runTest()">
 
 </head>
 <body onload="runTest()">
 
@@ -1,7 +1,7 @@
 <!DOCTYPE html>
 <html>
 <head>
 <!DOCTYPE html>
 <html>
 <head>
-<script src="../../../resources/js-test-pre.js"></script>
+<script src="../../../../resources/js-test-pre.js"></script>
 <script>
 window.jsTestIsAsync = true;
 
 <script>
 window.jsTestIsAsync = true;
 
@@ -49,6 +49,6 @@ window.onload = function () {
 description("This tests that closing a window that is presenting a video in picture-in-picture does not cause an assertion failure in a debug build. To run this test by hand, click the Open new window button then click the picture-in-picture button (tap the video on iOS).");
 </script>
 <button id="openNewWindowButton">Open new window</button>
 description("This tests that closing a window that is presenting a video in picture-in-picture does not cause an assertion failure in a debug build. To run this test by hand, click the Open new window button then click the picture-in-picture button (tap the video on iOS).");
 </script>
 <button id="openNewWindowButton">Open new window</button>
-<script src="../../../resources/js-test-post.js"></script>
+<script src="../../../../resources/js-test-post.js"></script>
 </body>
 </html>
 </body>
 </html>
@@ -1,7 +1,7 @@
 <!DOCTYPE html>
 <html>
 <head>
 <!DOCTYPE html>
 <html>
 <head>
-<script src="../../../media-file.js"></script>
+<script src="../../../../../media/media-file.js"></script>
 </head>
 <body>
 <video controls></video>
 </head>
 <body>
 <video controls></video>
@@ -19,7 +19,7 @@ if (window.internals)
     internals.setMediaElementRestrictions(video, "NoRestrictions");
 video.addEventListener("webkitpresentationmodechanged", dispatchDidChangePresentationMode, false);
 video.addEventListener("canplaythrough", function () { video.webkitSetPresentationMode("picture-in-picture"); }, false);
     internals.setMediaElementRestrictions(video, "NoRestrictions");
 video.addEventListener("webkitpresentationmodechanged", dispatchDidChangePresentationMode, false);
 video.addEventListener("canplaythrough", function () { video.webkitSetPresentationMode("picture-in-picture"); }, false);
-video.src = findMediaFile("video", "../../../content/test");
+video.src = findMediaFile("video", "../../../../../media/content/test");
 </script>
 </body>
 </html>
 </script>
 </body>
 </html>
@@ -1,7 +1,7 @@
-<script src="../../../../resources/js-test-pre.js"></script>
+<script src="../../../../../resources/js-test-pre.js"></script>
 <meta name="viewport" content="width=device-width, initial-scale=1">
 <body>
 <meta name="viewport" content="width=device-width, initial-scale=1">
 <body>
-<iframe src="../../../content/silence.mp3" style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"></iframe>
+<iframe src="../../../../../media/content/silence.mp3" style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"></iframe>
 <script type="text/javascript">
 
 description("Testing the size of the media element in an audio media document on iOS.");
 <script type="text/javascript">
 
 description("Testing the size of the media element in an audio media document on iOS.");
@@ -30,5 +30,5 @@ let media;
 })();
 
 </script>
 })();
 
 </script>
-<script src="../../../../resources/js-test-post.js"></script>
+<script src="../../../../../resources/js-test-post.js"></script>
 </body>
 </body>
@@ -1,7 +1,7 @@
-<script src="../../../../resources/js-test-pre.js"></script>
+<script src="../../../../../resources/js-test-pre.js"></script>
 <meta name="viewport" content="width=device-width, initial-scale=1">
 <body>
 <meta name="viewport" content="width=device-width, initial-scale=1">
 <body>
-<iframe src="../../../content/test.mp4" style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"></iframe>
+<iframe src="../../../../../media/content/test.mp4" style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"></iframe>
 <script type="text/javascript">
 
 description("Testing the size of the media element in a video media document on iOS.");
 <script type="text/javascript">
 
 description("Testing the size of the media element in a video media document on iOS.");
@@ -30,5 +30,5 @@ let media;
 })();
 
 </script>
 })();
 
 </script>
-<script src="../../../../resources/js-test-post.js"></script>
+<script src="../../../../../resources/js-test-post.js"></script>
 </body>
 </body>
@@ -1,5 +1,5 @@
-<script src="../../../../resources/js-test-pre.js"></script>
-<script src="../../resources/media-controls-loader.js" type="text/javascript"></script>
+<script src="../../../../../resources/js-test-pre.js"></script>
+<script src="../../../../../media/modern-media-controls/resources/media-controls-loader.js" type="text/javascript"></script>
 <body>
 <style type="text/css" media="screen">
     
 <body>
 <style type="text/css" media="screen">
     
@@ -15,7 +15,7 @@
     }
     
 </style>
     }
     
 </style>
-<video src="../../../content/test.mp4" controls autoplay></video>
+<video src="../../../../../media/content/test.mp4" controls autoplay></video>
 <div id="host"></div>
 <script type="text/javascript">
 
 <div id="host"></div>
 <script type="text/javascript">
 
@@ -41,5 +41,5 @@ media.addEventListener("error", () => {
 });
 
 </script>
 });
 
 </script>
-<script src="../../../../resources/js-test-post.js"></script>
+<script src="../../../../../resources/js-test-post.js"></script>
 </body>
 </body>
@@ -1,6 +1,6 @@
-<script src="../../../../resources/js-test-pre.js"></script>
-<script src="../../resources/media-controls-loader.js" type="text/javascript"></script>
-<script src="../../resources/media-controls-utils.js" type="text/javascript"></script>
+<script src="../../../../../resources/js-test-pre.js"></script>
+<script src="../../../../../media/modern-media-controls/resources/media-controls-loader.js" type="text/javascript"></script>
+<script src="../../../../../media/modern-media-controls/resources/media-controls-utils.js" type="text/javascript"></script>
 <body>
 <style type="text/css" media="screen">
     
 <body>
 <style type="text/css" media="screen">
     
@@ -13,7 +13,7 @@
     }
     
 </style>
     }
     
 </style>
-<video src="../../../content/test.mp4" autoplay controls></video>
+<video src="../../../../../media/content/test.mp4" autoplay controls></video>
 <div id="host"></div>
 <script type="text/javascript">
 
 <div id="host"></div>
 <script type="text/javascript">
 
@@ -41,5 +41,5 @@ media.addEventListener("webkitpresentationmodechanged", () => {
 });
 
 </script>
 });
 
 </script>
-<script src="../../../../resources/js-test-post.js"></script>
+<script src="../../../../../resources/js-test-post.js"></script>
 </body>
 </body>
diff --git a/LayoutTests/platform/iphone-7/tiled-drawing/compositing-layers-deep-color-expected.txt b/LayoutTests/platform/iphone-7/tiled-drawing/compositing-layers-deep-color-expected.txt
new file mode 100644 (file)
index 0000000..474eb2a
--- /dev/null
@@ -0,0 +1,40 @@
+Box
+Box
+(GraphicsLayer
+  (anchor 0.00 0.00)
+  (bounds 5018.00 2018.00)
+  (deep color 1)
+  (children 1
+    (GraphicsLayer
+      (bounds 5018.00 2018.00)
+      (contentsOpaque 1)
+      (tile cache coverage 0, 0 1024 x 1024)
+      (tile size 512 x 512)
+      (top left tile 0, 0 tiles grid 2 x 2)
+      (in window 1)
+      (deep color 1)
+      (children 2
+        (GraphicsLayer
+          (position 18.00 10.00)
+          (bounds 100.00 100.00)
+          (contentsOpaque 1)
+          (drawsContent 1)
+          (deep color 1)
+        )
+        (GraphicsLayer
+          (position 18.00 120.00)
+          (bounds 5000.00 100.00)
+          (usingTiledLayer 1)
+          (contentsOpaque 1)
+          (drawsContent 1)
+          (tile cache coverage 0, 0 1024 x 100)
+          (tile size 512 x 512)
+          (top left tile 0, 0 tiles grid 2 x 1)
+          (in window 1)
+          (deep color 1)
+        )
+      )
+    )
+  )
+)
+
diff --git a/LayoutTests/platform/iphone-7/tiled-drawing/compositing-layers-deep-color.html b/LayoutTests/platform/iphone-7/tiled-drawing/compositing-layers-deep-color.html
new file mode 100644 (file)
index 0000000..9c8c310
--- /dev/null
@@ -0,0 +1,49 @@
+<!DOCTYPE html>
+
+<html>
+<head>
+    <style>
+        body {
+            height: 2000px;
+        }
+        
+        .box {
+            height: 100px;
+            width: 100px;
+            background-color: blue;
+            margin: 10px;
+        }
+        
+        .composited {
+            transform: translateZ(0);
+        }
+        
+        .tiled {
+            width: 5000px;
+        }
+    </style>
+    <script>
+        if (window.testRunner)
+            testRunner.dumpAsText();
+
+        function doTest()
+        {
+            if (window.internals)
+                document.getElementById('layers').innerText = internals.layerTreeAsText(document, internals.LAYER_TREE_INCLUDES_TILE_CACHES);
+        }
+        window.addEventListener('load', doTest, false);
+    </script>
+</head>
+<body>
+
+<div class="composited box">
+    Box
+</div>
+
+<div class="tiled composited box">
+    Box
+</div>
+
+<pre id="layers"></pre>
+</body>
+</html>
index 7fd69d2..bd345fe 100644 (file)
@@ -18,8 +18,6 @@ fast/forms/search/search-results-hidden-crash.html [ Pass ]
 fast/url/user-visible [ Pass ]
 fast/images/eps-as-image.html [ Pass ]
 
 fast/url/user-visible [ Pass ]
 fast/images/eps-as-image.html [ Pass ]
 
-fast/text-autosizing/ios/ipad/programmatic-text-size-adjust.html [ Skip ]
-fast/text-autosizing/ios/ipad/text-size-adjust-inline-style.html [ Skip ]
 fast/text-autosizing/ios/programmatic-text-size-adjust.html [ Skip ]
 fast/text-autosizing/ios/text-size-adjust-inline-style.html [ Skip ]
 fast/text-autosizing [ Pass ]
 fast/text-autosizing/ios/programmatic-text-size-adjust.html [ Skip ]
 fast/text-autosizing/ios/text-size-adjust-inline-style.html [ Skip ]
 fast/text-autosizing [ Pass ]
index b6d26c2..3126077 100644 (file)
@@ -1,3 +1,84 @@
+2019-01-18  Jonathan Bedard  <jbedard@apple.com>
+
+        webkitpy: Implement device type specific expected results (Part 2)
+        https://bugs.webkit.org/show_bug.cgi?id=192162
+        <rdar://problem/46345449>
+
+        Rubber-stamped by Aakash Jain.
+
+        This change uses device_type to select a different set of TestExpectation files based on the type of device running
+        tests. This requires multiple baseline search paths to be used in a single instantiation of run-webkit-tests. Note
+        that while multiple baseline search paths are used, any single test will only ever be run on a single device type.
+
+        * Scripts/webkitpy/layout_tests/controllers/layout_test_finder.py:
+        (LayoutTestFinder.find_tests): Pass device_type through.
+        * Scripts/webkitpy/layout_tests/controllers/layout_test_finder_unittest.py:
+        (LayoutTestFinderTests.test_touched_but_skipped_test): Allow device_type to be passed in.
+        * Scripts/webkitpy/layout_tests/controllers/manager.py:
+        (Manager.__init__): Make _expectations an OrderedDict indexed by device_type, add current_device_type.
+        (Manager._collect_tests): Pass device type to test finder.
+        (Manager._needs_web_platform_test):
+        (Manager._prepare_lists): Index _expectations by device type.
+        (Manager._test_input_for_file):
+        (Manager._test_is_slow): Ditto.
+        (Manager._test_should_dump_jsconsolelog_in_stderr):
+        (Manager._test_should_dump_jsconsolelog_in_stderr): Ditto.
+        (Manager._get_test_inputs):
+        (Manager._update_worker_count):
+        (Manager._set_up_run): Use current_device_type instead of passing device_type in.
+        (Manager.run): Re-write function to gather and parse expectations from multiple baseline search paths. In the event
+        that a test can be run on multiple device types, the first device in the device type list which is eligible will be
+        the one which runs the test.
+        (Manager._run_test_subset):
+        (Manager._run_tests): Index _expectations by current_device_type.
+        (Manager._print_expectation_line_for_test): Ditto.
+        (Manager._print_expectations_for_subset): Ditto.
+        (Manager.print_expectations): Re-write function to gather and parse expectations from multiple baseline search paths.
+        Unlike run, this function will behave the same regardless of what devices are available.
+        (Manager._custom_device_for_test): Deleted.
+        * Scripts/webkitpy/layout_tests/controllers/manager_unittest.py:
+        (ManagerTest.test_uses_custom_device): Deleted.
+        (ManagerTest.test_uses_custom_device.get_manager): Deleted.
+        * Scripts/webkitpy/layout_tests/layout_package/json_layout_results_generator.py:
+        (JSONLayoutResultsGenerator.__init__): Pass expectations as a dictionary indexed by device type.
+        (JSONLayoutResultsGenerator._insert_failure_summaries): The number of fixable tests must take into consideration
+        all of the expectations.
+        * Scripts/webkitpy/layout_tests/lint_test_expectations_unittest.py:
+        (FakePort.skipped_layout_tests): Allow device_type to be passed in.
+        * Scripts/webkitpy/layout_tests/models/test_expectations.py:
+        (TestExpectations.__init__): Allow device_type to be passed through.
+        (TestExpectations.parse_all_expectations): Pick expectations_dict by device type.
+        * Scripts/webkitpy/layout_tests/models/test_expectations_unittest.py: Allow device_type to be passed in.
+        * Scripts/webkitpy/layout_tests/models/test_run_results.py:
+        (summarize_results):
+        * Scripts/webkitpy/layout_tests/models/test_run_results_unittest.py:
+        (summarized_results): Expectations are stored in a dictionary indexed by device_type. When determining the expectation
+        a test ran with, one must check all expectations to find the first non-skip expectation.
+        * Scripts/webkitpy/layout_tests/run_webkit_tests_integrationtest.py:
+        (RunTest.test_device_type_test_division):
+        (RunTest):
+        (RunTest.test_device_type_specific_listing):
+        * Scripts/webkitpy/port/base.py:
+        (Port): Add default values for DEVICE_TYPE and DEFAULT_DEVICE_TYPES.
+        (Port.supported_device_types): The result of this function will change based on what devices are available.
+        * Scripts/webkitpy/port/device_port.py:
+        (DevicePort._device_type_with_version): Replace DEFAULT_DEVICE_TYPE with DEVICE_TYPE.
+        (DevicePort.default_child_processes): DEVICE_TYPE is a general type, so the built-in comparison of the DeviceType object
+        can be used.
+        (DevicePort.supported_device_types): Generates a list of device types determined by the available devices, if there are
+        any, or the DEFAULT_DEVICE_TYPES of the port.
+        * Scripts/webkitpy/port/ios.py:
+        (IOSPort): Change DEFAULT_DEVICE_TYPE to DEVICE_TYPE.
+        * Scripts/webkitpy/port/ios_simulator.py:
+        (IOSSimulatorPort): Set the DEFAULT_DEVICE_TYPES to the devices currently used to run layout tests.
+        * Scripts/webkitpy/port/watch.py:
+        (WatchPort): Change DEFAULT_DEVICE_TYPE to DEVICE_TYPE.
+        * Scripts/webkitpy/port/watch_simulator.py:
+        (WatchSimulatorPort): Set the DEFAULT_DEVICE_TYPES to the device currently used to run layout tests.
+        * Scripts/webkitpy/xcode/device_type.py:
+        (DeviceType):
+        (DeviceType.__hash__): Allow DeviceType to be used as a dictionary index.
+
 2019-01-18  Philippe Normand  <pnormand@igalia.com>
 
         [WPE] run-qt-wpe-minibrowser needlessly expects a --wpe argument
 2019-01-18  Philippe Normand  <pnormand@igalia.com>
 
         [WPE] run-qt-wpe-minibrowser needlessly expects a --wpe argument
index 8a6f9f7..c2e064a 100644 (file)
@@ -43,11 +43,11 @@ class LayoutTestFinder(object):
         self._filesystem = self._port.host.filesystem
         self.LAYOUT_TESTS_DIRECTORY = 'LayoutTests'
 
         self._filesystem = self._port.host.filesystem
         self.LAYOUT_TESTS_DIRECTORY = 'LayoutTests'
 
-    def find_tests(self, options, args):
+    def find_tests(self, options, args, device_type=None):
         paths = self._strip_test_dir_prefixes(args)
         if options and options.test_list:
             paths += self._strip_test_dir_prefixes(self._read_test_names_from_file(options.test_list, self._port.TEST_PATH_SEPARATOR))
         paths = self._strip_test_dir_prefixes(args)
         if options and options.test_list:
             paths += self._strip_test_dir_prefixes(self._read_test_names_from_file(options.test_list, self._port.TEST_PATH_SEPARATOR))
-        test_files = self._port.tests(paths)
+        test_files = self._port.tests(paths, device_type=device_type)
         return (paths, test_files)
 
     def find_touched_tests(self, new_or_modified_paths, apply_skip_expectations=True):
         return (paths, test_files)
 
     def find_touched_tests(self, new_or_modified_paths, apply_skip_expectations=True):
index 50ee9a1..ca1a20d 100644 (file)
@@ -84,7 +84,7 @@ class LayoutTestFinderTests(unittest.TestCase):
 
         expectations_dict = OrderedDict()
         expectations_dict['expectations'] = 'test1.html [ Skip ]\ntest3.html [ Skip ]\n'
 
         expectations_dict = OrderedDict()
         expectations_dict['expectations'] = 'test1.html [ Skip ]\ntest3.html [ Skip ]\n'
-        port.expectations_dict = lambda: expectations_dict
+        port.expectations_dict = lambda **kwargs: expectations_dict
         port.test_exists = lambda test: True
 
         paths = ['LayoutTests/test0.html', 'LayoutTests/test1.html', 'LayoutTests/test2-expected.txt', 'LayoutTests/test3-expected.txt']
         port.test_exists = lambda test: True
 
         paths = ['LayoutTests/test0.html', 'LayoutTests/test1.html', 'LayoutTests/test2-expected.txt', 'LayoutTests/test3-expected.txt']
index 96d1475..c9b029d 100644 (file)
@@ -39,7 +39,7 @@ import logging
 import random
 import sys
 import time
 import random
 import sys
 import time
-from collections import defaultdict
+from collections import defaultdict, OrderedDict
 
 from webkitpy.common.checkout.scm.detection import SCMDetector
 from webkitpy.common.net.file_uploader import FileUploader
 
 from webkitpy.common.checkout.scm.detection import SCMDetector
 from webkitpy.common.net.file_uploader import FileUploader
@@ -78,7 +78,7 @@ class Manager(object):
         self._filesystem = port.host.filesystem
         self._options = options
         self._printer = printer
         self._filesystem = port.host.filesystem
         self._options = options
         self._printer = printer
-        self._expectations = None
+        self._expectations = OrderedDict()
         self.HTTP_SUBDIR = 'http' + port.TEST_PATH_SEPARATOR + 'test'
         self.WEBSOCKET_SUBDIR = 'websocket' + port.TEST_PATH_SEPARATOR
         self.web_platform_test_subdir = self._port.web_platform_test_server_doc_root()
         self.HTTP_SUBDIR = 'http' + port.TEST_PATH_SEPARATOR + 'test'
         self.WEBSOCKET_SUBDIR = 'websocket' + port.TEST_PATH_SEPARATOR
         self.web_platform_test_subdir = self._port.web_platform_test_server_doc_root()
@@ -91,8 +91,8 @@ class Manager(object):
         test_options_json_path = self._port.path_from_webkit_base(self.LAYOUT_TESTS_DIRECTORY, "tests-options.json")
         self._tests_options = json.loads(self._filesystem.read_text_file(test_options_json_path)) if self._filesystem.exists(test_options_json_path) else {}
 
         test_options_json_path = self._port.path_from_webkit_base(self.LAYOUT_TESTS_DIRECTORY, "tests-options.json")
         self._tests_options = json.loads(self._filesystem.read_text_file(test_options_json_path)) if self._filesystem.exists(test_options_json_path) else {}
 
-    def _collect_tests(self, args):
-        return self._finder.find_tests(self._options, args)
+    def _collect_tests(self, args, device_type=None):
+        return self._finder.find_tests(self._options, args, device_type=device_type)
 
     def _is_http_test(self, test):
         return self.HTTP_SUBDIR in test or self._is_websocket_test(test) or self._needs_web_platform_test(test)
 
     def _is_http_test(self, test):
         return self.HTTP_SUBDIR in test or self._is_websocket_test(test) or self._needs_web_platform_test(test)
@@ -103,21 +103,11 @@ class Manager(object):
     def _needs_web_platform_test(self, test):
         return self.web_platform_test_subdir in test or self.webkit_specific_web_platform_test_subdir in test
 
     def _needs_web_platform_test(self, test):
         return self.web_platform_test_subdir in test or self.webkit_specific_web_platform_test_subdir in test
 
-    def _custom_device_for_test(self, test):
-        # FIXME: This is a terrible way to do device-specific expected results https://bugs.webkit.org/show_bug.cgi?id=192162
-        for device_type in self._port.CUSTOM_DEVICE_TYPES:
-            if device_type.hardware_family and device_type.hardware_family.lower() + self._port.TEST_PATH_SEPARATOR in test:
-                return device_type
-            if device_type.hardware_family and device_type.hardware_type and \
-                (device_type.hardware_family + device_type.hardware_type).lower().replace(' ', '') + self._port.TEST_PATH_SEPARATOR in test:
-                return device_type
-        return None
-
     def _http_tests(self, test_names):
         return set(test for test in test_names if self._is_http_test(test))
 
     def _http_tests(self, test_names):
         return set(test for test in test_names if self._is_http_test(test))
 
-    def _prepare_lists(self, paths, test_names):
-        tests_to_skip = self._finder.skip_tests(paths, test_names, self._expectations, self._http_tests(test_names))
+    def _prepare_lists(self, paths, test_names, device_type=None):
+        tests_to_skip = self._finder.skip_tests(paths, test_names, self._expectations[device_type], self._http_tests(test_names))
         tests_to_run = [test for test in test_names if test not in tests_to_skip]
 
         # Create a sorted list of test files so the subset chunk,
         tests_to_run = [test for test in test_names if test not in tests_to_skip]
 
         # Create a sorted list of test files so the subset chunk,
@@ -128,38 +118,38 @@ class Manager(object):
             random.shuffle(tests_to_run)
 
         tests_to_run, tests_in_other_chunks = self._finder.split_into_chunks(tests_to_run)
             random.shuffle(tests_to_run)
 
         tests_to_run, tests_in_other_chunks = self._finder.split_into_chunks(tests_to_run)
-        self._expectations.add_skipped_tests(tests_in_other_chunks)
+        self._expectations[device_type].add_skipped_tests(tests_in_other_chunks)
         tests_to_skip.update(tests_in_other_chunks)
 
         return tests_to_run, tests_to_skip
 
         tests_to_skip.update(tests_in_other_chunks)
 
         return tests_to_run, tests_to_skip
 
-    def _test_input_for_file(self, test_file):
+    def _test_input_for_file(self, test_file, device_type=None):
         return TestInput(test_file,
         return TestInput(test_file,
-            self._options.slow_time_out_ms if self._test_is_slow(test_file) else self._options.time_out_ms,
+            self._options.slow_time_out_ms if self._test_is_slow(test_file, device_type=device_type) else self._options.time_out_ms,
             self._is_http_test(test_file),
             self._is_http_test(test_file),
-            should_dump_jsconsolelog_in_stderr=self._test_should_dump_jsconsolelog_in_stderr(test_file))
+            should_dump_jsconsolelog_in_stderr=self._test_should_dump_jsconsolelog_in_stderr(test_file, device_type=device_type))
 
 
-    def _test_is_slow(self, test_file):
-        if self._expectations.model().has_modifier(test_file, test_expectations.SLOW):
+    def _test_is_slow(self, test_file, device_type=None):
+        if self._expectations[device_type].model().has_modifier(test_file, test_expectations.SLOW):
             return True
         return "slow" in self._tests_options.get(test_file, [])
 
             return True
         return "slow" in self._tests_options.get(test_file, [])
 
-    def _test_should_dump_jsconsolelog_in_stderr(self, test_file):
-        return self._expectations.model().has_modifier(test_file, test_expectations.DUMPJSCONSOLELOGINSTDERR)
+    def _test_should_dump_jsconsolelog_in_stderr(self, test_file, device_type=None):
+        return self._expectations[device_type].model().has_modifier(test_file, test_expectations.DUMPJSCONSOLELOGINSTDERR)
 
     def needs_servers(self, test_names):
         return any(self._is_http_test(test_name) for test_name in test_names) and self._options.http
 
 
     def needs_servers(self, test_names):
         return any(self._is_http_test(test_name) for test_name in test_names) and self._options.http
 
-    def _get_test_inputs(self, tests_to_run, repeat_each, iterations):
+    def _get_test_inputs(self, tests_to_run, repeat_each, iterations, device_type=None):
         test_inputs = []
         for _ in xrange(iterations):
             for test in tests_to_run:
                 for _ in xrange(repeat_each):
         test_inputs = []
         for _ in xrange(iterations):
             for test in tests_to_run:
                 for _ in xrange(repeat_each):
-                    test_inputs.append(self._test_input_for_file(test))
+                    test_inputs.append(self._test_input_for_file(test, device_type=device_type))
         return test_inputs
 
         return test_inputs
 
-    def _update_worker_count(self, test_names):
-        test_inputs = self._get_test_inputs(test_names, self._options.repeat_each, self._options.iterations)
+    def _update_worker_count(self, test_names, device_type=None):
+        test_inputs = self._get_test_inputs(test_names, self._options.repeat_each, self._options.iterations, device_type=device_type)
         worker_count = self._runner.get_worker_count(test_inputs, int(self._options.child_processes))
         self._options.child_processes = worker_count
 
         worker_count = self._runner.get_worker_count(test_inputs, int(self._options.child_processes))
         self._options.child_processes = worker_count
 
@@ -170,7 +160,7 @@ class Manager(object):
         if not self._port.start_helper(self._options.pixel_tests):
             return False
 
         if not self._port.start_helper(self._options.pixel_tests):
             return False
 
-        self._update_worker_count(test_names)
+        self._update_worker_count(test_names, device_type=device_type)
         self._port.reset_preferences()
 
         # Check that the system dependencies (themes, fonts, ...) are correct.
         self._port.reset_preferences()
 
         # Check that the system dependencies (themes, fonts, ...) are correct.
@@ -184,49 +174,47 @@ class Manager(object):
         return True
 
     def run(self, args):
         return True
 
     def run(self, args):
-        """Run the tests and return a RunDetails object with the results."""
-        self._printer.write_update("Collecting tests ...")
-        try:
-            paths, test_names = self._collect_tests(args)
-        except IOError:
-            # This is raised if --test-list doesn't exist
-            return test_run_results.RunDetails(exit_code=-1)
+        total_tests = set()
+        aggregate_test_names = set()
+        aggregate_tests = set()
+        tests_to_run_by_device = {}
+
+        device_type_list = self._port.supported_device_types()
+        for device_type in device_type_list:
+            """Run the tests and return a RunDetails object with the results."""
+            for_device_type = 'for {} '.format(device_type) if device_type else ''
+            self._printer.write_update('Collecting tests {}...'.format(for_device_type))
+            try:
+                paths, test_names = self._collect_tests(args, device_type=device_type)
+            except IOError:
+                # This is raised if --test-list doesn't exist
+                return test_run_results.RunDetails(exit_code=-1)
 
 
-        self._printer.write_update("Parsing expectations ...")
-        self._expectations = test_expectations.TestExpectations(self._port, test_names, force_expectations_pass=self._options.force)
-        self._expectations.parse_all_expectations()
+            self._printer.write_update('Parsing expectations {}...'.format(for_device_type))
+            self._expectations[device_type] = test_expectations.TestExpectations(self._port, test_names, force_expectations_pass=self._options.force, device_type=device_type)
+            self._expectations[device_type].parse_all_expectations()
 
 
-        tests_to_run, tests_to_skip = self._prepare_lists(paths, test_names)
-        self._printer.print_found(len(test_names), len(tests_to_run), self._options.repeat_each, self._options.iterations)
+            aggregate_test_names.update(test_names)
+            tests_to_run, tests_to_skip = self._prepare_lists(paths, test_names, device_type=device_type)
+
+            total_tests.update(tests_to_run)
+            total_tests.update(tests_to_skip)
+
+            tests_to_run_by_device[device_type] = [test for test in tests_to_run if test not in aggregate_tests]
+            aggregate_tests.update(tests_to_run)
+
+        tests_to_skip = total_tests - aggregate_tests
+        self._printer.print_found(len(aggregate_test_names), len(aggregate_tests), self._options.repeat_each, self._options.iterations)
         start_time = time.time()
 
         # Check to make sure we're not skipping every test.
         start_time = time.time()
 
         # Check to make sure we're not skipping every test.
-        if not tests_to_run:
+        if not sum([len(tests) for tests in tests_to_run_by_device.itervalues()]):
             _log.critical('No tests to run.')
             return test_run_results.RunDetails(exit_code=-1)
 
             _log.critical('No tests to run.')
             return test_run_results.RunDetails(exit_code=-1)
 
-        # Look for tests with custom device requirements.
-        test_device_mapping = defaultdict(list)
-        for test_file in tests_to_run:
-            test_device_mapping[self._custom_device_for_test(test_file) or self._port.DEFAULT_DEVICE_TYPE].append(test_file)
-
-        # Order device types from most specific to least specific in the hopes that some of the more specific device
-        # types will match the less specific device types.
-        device_type_order = []
-        types_with_family = []
-        remaining_types = []
-        for device_type in test_device_mapping.iterkeys():
-            if device_type and device_type.hardware_family and device_type.hardware_type:
-                device_type_order.append(device_type)
-            elif device_type and device_type.hardware_family:
-                types_with_family.append(device_type)
-            else:
-                remaining_types.append(device_type)
-        device_type_order.extend(types_with_family + remaining_types)
-
-        needs_http = any((self._is_http_test(test) and not self._needs_web_platform_test(test)) for test in tests_to_run)
-        needs_web_platform_test_server = any(self._needs_web_platform_test(test) for test in tests_to_run)
-        needs_websockets = any(self._is_websocket_test(test) for test in tests_to_run)
+        needs_http = any((self._is_http_test(test) and not self._needs_web_platform_test(test)) for tests in tests_to_run_by_device.itervalues() for test in tests)
+        needs_web_platform_test_server = any(self._needs_web_platform_test(test) for tests in tests_to_run_by_device.itervalues() for test in tests)
+        needs_websockets = any(self._is_websocket_test(test) for tests in tests_to_run_by_device.itervalues() for test in tests)
         self._runner = LayoutTestRunner(self._options, self._port, self._printer, self._results_directory, self._test_is_slow,
                                         needs_http=needs_http, needs_web_platform_test_server=needs_web_platform_test_server, needs_websockets=needs_websockets)
 
         self._runner = LayoutTestRunner(self._options, self._port, self._printer, self._results_directory, self._test_is_slow,
                                         needs_http=needs_http, needs_web_platform_test_server=needs_web_platform_test_server, needs_websockets=needs_websockets)
 
@@ -248,40 +236,26 @@ class Manager(object):
         max_child_processes_for_run = 1
         child_processes_option_value = self._options.child_processes
 
         max_child_processes_for_run = 1
         child_processes_option_value = self._options.child_processes
 
-        while device_type_order:
-            device_type = device_type_order[0]
-            tests = test_device_mapping[device_type]
-            del device_type_order[0]
-
+        for device_type in device_type_list:
+            self._runner._test_is_slow = lambda test_file: self._test_is_slow(test_file, device_type=device_type)
             self._options.child_processes = min(self._port.max_child_processes(device_type=device_type), int(child_processes_option_value or self._port.default_child_processes(device_type=device_type)))
 
             _log.info('')
             if not self._options.child_processes:
             self._options.child_processes = min(self._port.max_child_processes(device_type=device_type), int(child_processes_option_value or self._port.default_child_processes(device_type=device_type)))
 
             _log.info('')
             if not self._options.child_processes:
-                _log.info('Skipping {} because {} is not available'.format(pluralize(len(test_device_mapping[device_type]), 'test'), str(device_type)))
+                _log.info('Skipping {} because {} is not available'.format(pluralize(len(tests_to_run_by_device[device_type]), 'test'), str(device_type)))
                 _log.info('')
                 continue
 
             max_child_processes_for_run = max(self._options.child_processes, max_child_processes_for_run)
 
                 _log.info('')
                 continue
 
             max_child_processes_for_run = max(self._options.child_processes, max_child_processes_for_run)
 
-            # This loop looks for any less-specific device types which match the current device type
-            index = 0
-            while index < len(device_type_order):
-                if device_type_order[index] == device_type:
-                    tests.extend(test_device_mapping[device_type_order[index]])
-
-                    # Remove devices types from device_type_order once tests associated with that type have been claimed.
-                    del device_type_order[index]
-                else:
-                    index += 1
-
             self._printer.print_baseline_search_path(device_type=device_type)
 
             self._printer.print_baseline_search_path(device_type=device_type)
 
-            _log.info('Running {}{}'.format(pluralize(len(tests), 'test'), ' for {}'.format(str(device_type)) if device_type else ''))
+            _log.info('Running {}{}'.format(pluralize(len(tests_to_run_by_device[device_type]), 'test'), ' for {}'.format(str(device_type)) if device_type else ''))
             _log.info('')
             _log.info('')
-            if not self._set_up_run(testsdevice_type):
+            if not self._set_up_run(tests_to_run_by_device[device_type], device_type=device_type):
                 return test_run_results.RunDetails(exit_code=-1)
 
                 return test_run_results.RunDetails(exit_code=-1)
 
-            temp_initial_results, temp_retry_results, temp_enabled_pixel_tests_in_retry = self._run_test_subset(tests, tests_to_skip)
+            temp_initial_results, temp_retry_results, temp_enabled_pixel_tests_in_retry = self._run_test_subset(tests_to_run_by_device[device_type], tests_to_skip, device_type=device_type)
             initial_results = initial_results.merge(temp_initial_results) if initial_results else temp_initial_results
             retry_results = retry_results.merge(temp_retry_results) if retry_results else temp_retry_results
             enabled_pixel_tests_in_retry |= temp_enabled_pixel_tests_in_retry
             initial_results = initial_results.merge(temp_initial_results) if initial_results else temp_initial_results
             retry_results = retry_results.merge(temp_retry_results) if retry_results else temp_retry_results
             enabled_pixel_tests_in_retry |= temp_enabled_pixel_tests_in_retry
@@ -294,10 +268,10 @@ class Manager(object):
         end_time = time.time()
         return self._end_test_run(start_time, end_time, initial_results, retry_results, enabled_pixel_tests_in_retry)
 
         end_time = time.time()
         return self._end_test_run(start_time, end_time, initial_results, retry_results, enabled_pixel_tests_in_retry)
 
-    def _run_test_subset(self, tests_to_run, tests_to_skip):
+    def _run_test_subset(self, tests_to_run, tests_to_skip, device_type=None):
         try:
             enabled_pixel_tests_in_retry = False
         try:
             enabled_pixel_tests_in_retry = False
-            initial_results = self._run_tests(tests_to_run, tests_to_skip, self._options.repeat_each, self._options.iterations, int(self._options.child_processes), retrying=False)
+            initial_results = self._run_tests(tests_to_run, tests_to_skip, self._options.repeat_each, self._options.iterations, int(self._options.child_processes), retrying=False, device_type=device_type)
 
             tests_to_retry = self._tests_to_retry(initial_results, include_crashes=self._port.should_retry_crashes())
             # Don't retry failures when interrupted by user or failures limit exception.
 
             tests_to_retry = self._tests_to_retry(initial_results, include_crashes=self._port.should_retry_crashes())
             # Don't retry failures when interrupted by user or failures limit exception.
@@ -308,7 +282,7 @@ class Manager(object):
                 _log.info('')
                 _log.info("Retrying %s ..." % pluralize(len(tests_to_retry), "unexpected failure"))
                 _log.info('')
                 _log.info('')
                 _log.info("Retrying %s ..." % pluralize(len(tests_to_retry), "unexpected failure"))
                 _log.info('')
-                retry_results = self._run_tests(tests_to_retry, tests_to_skip=set(), repeat_each=1, iterations=1, num_workers=1, retrying=True)
+                retry_results = self._run_tests(tests_to_retry, tests_to_skip=set(), repeat_each=1, iterations=1, num_workers=1, retrying=True, device_type=device_type)
 
                 if enabled_pixel_tests_in_retry:
                     self._options.pixel_tests = False
 
                 if enabled_pixel_tests_in_retry:
                     self._options.pixel_tests = False
@@ -355,10 +329,10 @@ class Manager(object):
                 exit_code = self._port.exit_code_from_summarized_results(summarized_results)
         return test_run_results.RunDetails(exit_code, summarized_results, initial_results, retry_results, enabled_pixel_tests_in_retry)
 
                 exit_code = self._port.exit_code_from_summarized_results(summarized_results)
         return test_run_results.RunDetails(exit_code, summarized_results, initial_results, retry_results, enabled_pixel_tests_in_retry)
 
-    def _run_tests(self, tests_to_run, tests_to_skip, repeat_each, iterations, num_workers, retrying):
-        test_inputs = self._get_test_inputs(tests_to_run, repeat_each, iterations)
+    def _run_tests(self, tests_to_run, tests_to_skip, repeat_each, iterations, num_workers, retrying, device_type=None):
+        test_inputs = self._get_test_inputs(tests_to_run, repeat_each, iterations, device_type=device_type)
 
 
-        return self._runner.run_tests(self._expectations, test_inputs, tests_to_skip, num_workers, retrying)
+        return self._runner.run_tests(self._expectations[device_type], test_inputs, tests_to_skip, num_workers, retrying)
 
     def _clean_up_run(self):
         _log.debug("Flushing stdout")
 
     def _clean_up_run(self):
         _log.debug("Flushing stdout")
@@ -574,9 +548,9 @@ class Manager(object):
             json_results_generator.add_path_to_trie(name, value, stats_trie)
         return stats_trie
 
             json_results_generator.add_path_to_trie(name, value, stats_trie)
         return stats_trie
 
-    def _print_expectation_line_for_test(self, format_string, test):
-        line = self._expectations.model().get_expectation_line(test)
-        print(format_string.format(test, line.expected_behavior, self._expectations.readable_filename_and_line_number(line), line.original_string or ''))
+    def _print_expectation_line_for_test(self, format_string, test, device_type=None):
+        line = self._expectations[device_type].model().get_expectation_line(test)
+        print(format_string.format(test, line.expected_behavior, self._expectations[device_type].readable_filename_and_line_number(line), line.original_string or ''))
 
     def _print_expectations_for_subset(self, device_type, test_col_width, tests_to_run, tests_to_skip={}):
         format_string = '{{:{width}}} {{}} {{}} {{}}'.format(width=test_col_width)
 
     def _print_expectations_for_subset(self, device_type, test_col_width, tests_to_run, tests_to_skip={}):
         format_string = '{{:{width}}} {{}} {{}} {{}}'.format(width=test_col_width)
@@ -584,48 +558,49 @@ class Manager(object):
             print('')
             print('Tests to skip ({})'.format(len(tests_to_skip)))
             for test in sorted(tests_to_skip):
             print('')
             print('Tests to skip ({})'.format(len(tests_to_skip)))
             for test in sorted(tests_to_skip):
-                self._print_expectation_line_for_test(format_string, test)
+                self._print_expectation_line_for_test(format_string, test, device_type=device_type)
 
         print('')
         print('Tests to run{} ({})'.format(' for ' + str(device_type) if device_type else '', len(tests_to_run)))
         for test in sorted(tests_to_run):
 
         print('')
         print('Tests to run{} ({})'.format(' for ' + str(device_type) if device_type else '', len(tests_to_run)))
         for test in sorted(tests_to_run):
-            self._print_expectation_line_for_test(format_string, test)
+            self._print_expectation_line_for_test(format_string, test, device_type=device_type)
 
     def print_expectations(self, args):
 
     def print_expectations(self, args):
-        self._printer.write_update("Collecting tests ...")
-        try:
-            paths, test_names = self._collect_tests(args)
-        except IOError:
-            # This is raised if --test-list doesn't exist
-            return -1
-
-        self._printer.write_update("Parsing expectations ...")
-        self._expectations = test_expectations.TestExpectations(self._port, test_names, force_expectations_pass=self._options.force)
-        self._expectations.parse_all_expectations()
+        aggregate_test_names = set()
+        aggregate_tests_to_run = set()
+        aggregate_tests_to_skip = set()
+        tests_to_run_by_device = {}
+
+        device_type_list = self._port.DEFAULT_DEVICE_TYPES or [self._port.DEVICE_TYPE]
+        for device_type in device_type_list:
+            """Run the tests and return a RunDetails object with the results."""
+            for_device_type = 'for {} '.format(device_type) if device_type else ''
+            self._printer.write_update('Collecting tests {}...'.format(for_device_type))
+            try:
+                paths, test_names = self._collect_tests(args, device_type=device_type)
+            except IOError:
+                # This is raised if --test-list doesn't exist
+                return test_run_results.RunDetails(exit_code=-1)
 
 
-        tests_to_run, tests_to_skip = self._prepare_lists(paths, test_names)
-        self._printer.print_found(len(test_names), len(tests_to_run), self._options.repeat_each, self._options.iterations)
+            self._printer.write_update('Parsing expectations {}...'.format(for_device_type))
+            self._expectations[device_type] = test_expectations.TestExpectations(self._port, test_names, force_expectations_pass=self._options.force, device_type=device_type)
+            self._expectations[device_type].parse_all_expectations()
 
 
-        test_col_width = len(max(tests_to_run + list(tests_to_skip), key=len)) + 1
+            aggregate_test_names.update(test_names)
+            tests_to_run, tests_to_skip = self._prepare_lists(paths, test_names, device_type=device_type)
+            aggregate_tests_to_skip.update(tests_to_skip)
 
 
-        default_device_tests = []
+            tests_to_run_by_device[device_type] = [test for test in tests_to_run if test not in aggregate_tests_to_run]
+            aggregate_tests_to_run.update(tests_to_run)
 
 
-        # Look for tests with custom device requirements.
-        custom_device_tests = defaultdict(list)
-        for test_file in tests_to_run:
-            custom_device = self._custom_device_for_test(test_file)
-            if custom_device:
-                custom_device_tests[custom_device].append(test_file)
-            else:
-                default_device_tests.append(test_file)
+        aggregate_tests_to_skip = aggregate_tests_to_skip - aggregate_tests_to_run
 
 
-        if custom_device_tests:
-            for device_type, tests in custom_device_tests.iteritems():
-                _log.debug('{} tests use device {}'.format(len(tests), device_type))
+        self._printer.print_found(len(aggregate_test_names), len(aggregate_tests_to_run), self._options.repeat_each, self._options.iterations)
+        test_col_width = len(max(aggregate_tests_to_run.union(aggregate_tests_to_skip), key=len)) + 1
 
 
-        self._print_expectations_for_subset(None, test_col_width, tests_to_run, tests_to_skip)
+        self._print_expectations_for_subset(device_type_list[0], test_col_width, tests_to_run_by_device[device_type_list[0]], aggregate_tests_to_skip)
 
 
-        for device_type, tests in custom_device_tests.iteritems():
-            self._print_expectations_for_subset(device_type, test_col_width, tests)
+        for device_type in device_type_list[1:]:
+            self._print_expectations_for_subset(device_type, test_col_width, tests_to_run_by_device[device_type])
 
         return 0
 
         return 0
index a970b22..327d0d1 100644 (file)
@@ -105,19 +105,3 @@ class ManagerTest(unittest.TestCase):
         run_results = TestRunResults(expectations, len(tests))
         manager = get_manager()
         manager._look_for_new_crash_logs(run_results, time.time())
         run_results = TestRunResults(expectations, len(tests))
         manager = get_manager()
         manager._look_for_new_crash_logs(run_results, time.time())
-
-    def test_uses_custom_device(self):
-        class MockCustomDevicePort(TestPort):
-            CUSTOM_DEVICE_TYPES = [DeviceType(hardware_family='iPad')]
-
-            def __init__(self, host):
-                super(MockCustomDevicePort, self).__init__(host)
-
-        def get_manager():
-            host = MockHost()
-            port = MockCustomDevicePort(host)
-            manager = Manager(port, options=MockOptions(test_list=['fast/ipad/lasers.html'], http=True), printer=Mock())
-            return manager
-
-        manager = get_manager()
-        self.assertTrue(manager._custom_device_for_test('fast/ipad/lasers.html') == DeviceType(hardware_family='iPad'))
index d0c2201..a9a7e9e 100644 (file)
@@ -54,7 +54,7 @@ class JSONLayoutResultsGenerator(json_results_generator.JSONResultsGenerator):
 
     def __init__(self, port, builder_name, build_name, build_number,
         results_file_base_path,
 
     def __init__(self, port, builder_name, build_name, build_number,
         results_file_base_path,
-        expectations, run_results,
+        expectations_by_type, run_results,
         test_results_servers=[], test_type="", master_name=""):
         """Modifies the results.json file. Grabs it off the archive directory
         if it is not found locally.
         test_results_servers=[], test_type="", master_name=""):
         """Modifies the results.json file. Grabs it off the archive directory
         if it is not found locally.
@@ -67,7 +67,7 @@ class JSONLayoutResultsGenerator(json_results_generator.JSONResultsGenerator):
             {}, port.repository_paths(),
             test_results_servers, test_type, master_name)
 
             {}, port.repository_paths(),
             test_results_servers, test_type, master_name)
 
-        self._expectations = expectations
+        self._expectations = expectations_by_type
 
         self._run_results = run_results
         self._failures = dict((test_name, run_results.results_by_name[test_name].type) for test_name in run_results.failures_by_name)
 
         self._run_results = run_results
         self._failures = dict((test_name, run_results.results_by_name[test_name].type) for test_name in run_results.failures_by_name)
@@ -130,9 +130,12 @@ class JSONLayoutResultsGenerator(json_results_generator.JSONResultsGenerator):
         self._insert_item_into_raw_list(results_for_builder,
             self._get_failure_summary_entry(test_expectations.NOW),
             self.FIXABLE)
         self._insert_item_into_raw_list(results_for_builder,
             self._get_failure_summary_entry(test_expectations.NOW),
             self.FIXABLE)
-        self._insert_item_into_raw_list(results_for_builder,
-            len(self._expectations.model().get_tests_with_timeline(
-                test_expectations.NOW)), self.ALL_FIXABLE_COUNT)
+
+        num_fixable = 0
+        for expectation in self._expectations.itervalues():
+            num_fixable += len(expectation.model().get_tests_with_timeline(test_expectations.NOW))
+
+        self._insert_item_into_raw_list(results_for_builder, num_fixable, self.ALL_FIXABLE_COUNT)
         self._insert_item_into_raw_list(results_for_builder,
             self._get_failure_summary_entry(test_expectations.WONTFIX),
             self.WONTFIX)
         self._insert_item_into_raw_list(results_for_builder,
             self._get_failure_summary_entry(test_expectations.WONTFIX),
             self.WONTFIX)
index 5b8f7c8..dfb1a6f 100644 (file)
@@ -47,7 +47,7 @@ class FakePort(object):
         self.host.ports_parsed.append(self.name)
         return {self.path: ''}
 
         self.host.ports_parsed.append(self.name)
         return {self.path: ''}
 
-    def skipped_layout_tests(self, _):
+    def skipped_layout_tests(self, _, **kwargs):
         return set([])
 
     def all_test_configurations(self):
         return set([])
 
     def all_test_configurations(self):
index 46b14cb..7be6d48 100644 (file)
@@ -923,7 +923,7 @@ class TestExpectations(object):
             suffixes.add('wav')
         return set(suffixes)
 
             suffixes.add('wav')
         return set(suffixes)
 
-    def __init__(self, port, tests=None, include_generic=True, include_overrides=True, expectations_to_lint=None, force_expectations_pass=False):
+    def __init__(self, port, tests=None, include_generic=True, include_overrides=True, expectations_to_lint=None, force_expectations_pass=False, device_type=None):
         self._full_test_list = tests
         self._test_config = port.test_configuration()
         self._is_lint_mode = expectations_to_lint is not None
         self._full_test_list = tests
         self._test_config = port.test_configuration()
         self._is_lint_mode = expectations_to_lint is not None
@@ -933,6 +933,7 @@ class TestExpectations(object):
         self._skipped_tests_warnings = []
         self._expectations = []
         self._force_expectations_pass = force_expectations_pass
         self._skipped_tests_warnings = []
         self._expectations = []
         self._force_expectations_pass = force_expectations_pass
+        self._device_type = device_type
         self._include_generic = include_generic
         self._include_overrides = include_overrides
         self._expectations_to_lint = expectations_to_lint
         self._include_generic = include_generic
         self._include_overrides = include_overrides
         self._expectations_to_lint = expectations_to_lint
@@ -969,7 +970,7 @@ class TestExpectations(object):
             self._expectations_dict_index += 1
 
     def parse_all_expectations(self):
             self._expectations_dict_index += 1
 
     def parse_all_expectations(self):
-        self._expectations_dict = self._expectations_to_lint or self._port.expectations_dict()
+        self._expectations_dict = self._expectations_to_lint or self._port.expectations_dict(device_type=self._device_type)
         self._expectations_dict_index = 0
 
         self._has_warnings = False
         self._expectations_dict_index = 0
 
         self._has_warnings = False
@@ -979,7 +980,7 @@ class TestExpectations(object):
         self.parse_override_expectations()
 
         # FIXME: move ignore_tests into port.skipped_layout_tests()
         self.parse_override_expectations()
 
         # FIXME: move ignore_tests into port.skipped_layout_tests()
-        self.add_skipped_tests(self._port.skipped_layout_tests(self._full_test_list).union(set(self._port.get_option('ignore_tests', []))))
+        self.add_skipped_tests(self._port.skipped_layout_tests(self._full_test_list, device_type=self._device_type).union(set(self._port.get_option('ignore_tests', []))))
 
         self._report_warnings()
         self._process_tests_without_expectations()
 
         self._report_warnings()
         self._process_tests_without_expectations()
index 30adabc..2046446 100644 (file)
@@ -79,7 +79,7 @@ Bug(test) failures/expected/leaky-reftest.html [ ImageOnlyFailure Leak ]
         expectations_dict['expectations'] = expectations
         if overrides:
             expectations_dict['overrides'] = overrides
         expectations_dict['expectations'] = expectations
         if overrides:
             expectations_dict['overrides'] = overrides
-        self._port.expectations_dict = lambda: expectations_dict
+        self._port.expectations_dict = lambda **kwargs: expectations_dict
         expectations_to_lint = expectations_dict if is_lint_mode else None
         self._exp = TestExpectations(self._port, self.get_basic_tests(), expectations_to_lint=expectations_to_lint)
         self._exp.parse_all_expectations()
         expectations_to_lint = expectations_dict if is_lint_mode else None
         self._exp = TestExpectations(self._port, self.get_basic_tests(), expectations_to_lint=expectations_to_lint)
         self._exp.parse_all_expectations()
@@ -292,8 +292,8 @@ class SkippedTests(Base):
         expectations_dict['expectations'] = expectations
         if overrides:
             expectations_dict['overrides'] = overrides
         expectations_dict['expectations'] = expectations
         if overrides:
             expectations_dict['overrides'] = overrides
-        port.expectations_dict = lambda: expectations_dict
-        port.skipped_layout_tests = lambda tests: set(skips)
+        port.expectations_dict = lambda **kwargs: expectations_dict
+        port.skipped_layout_tests = lambda tests, **kwargs: set(skips)
         expectations_to_lint = expectations_dict if lint else None
         exp = TestExpectations(port, ['failures/expected/text.html'], expectations_to_lint=expectations_to_lint)
         exp.parse_all_expectations()
         expectations_to_lint = expectations_dict if lint else None
         exp = TestExpectations(port, ['failures/expected/text.html'], expectations_to_lint=expectations_to_lint)
         exp.parse_all_expectations()
@@ -329,8 +329,8 @@ class SkippedTests(Base):
         port = MockHost().port_factory.get('mac')
         expectations_dict = OrderedDict()
         expectations_dict['expectations'] = ''
         port = MockHost().port_factory.get('mac')
         expectations_dict = OrderedDict()
         expectations_dict['expectations'] = ''
-        port.expectations_dict = lambda: expectations_dict
-        port.skipped_layout_tests = lambda tests: set(['foo/bar/baz.html'])
+        port.expectations_dict = lambda **kwargs: expectations_dict
+        port.skipped_layout_tests = lambda tests, **kwargs: set(['foo/bar/baz.html'])
         capture = OutputCapture()
         capture.capture_output()
         exp = TestExpectations(port)
         capture = OutputCapture()
         capture.capture_output()
         exp = TestExpectations(port)
@@ -517,7 +517,7 @@ class RemoveConfigurationsTest(Base):
         test_port.test_isfile = lambda test: True
 
         test_config = test_port.test_configuration()
         test_port.test_isfile = lambda test: True
 
         test_config = test_port.test_configuration()
-        test_port.expectations_dict = lambda: {"expectations": """Bug(x) [ Linux Win Release ] failures/expected/foo.html [ Failure ]
+        test_port.expectations_dict = lambda **kwargs: {"expectations": """Bug(x) [ Linux Win Release ] failures/expected/foo.html [ Failure ]
 Bug(y) [ Win Mac Debug ] failures/expected/foo.html [ Crash ]
 """}
         expectations = TestExpectations(test_port, self.get_basic_tests())
 Bug(y) [ Win Mac Debug ] failures/expected/foo.html [ Crash ]
 """}
         expectations = TestExpectations(test_port, self.get_basic_tests())
@@ -536,7 +536,7 @@ Bug(y) [ Win Mac Debug ] failures/expected/foo.html [ Crash ]
         test_port.test_isfile = lambda test: True
 
         test_config = test_port.test_configuration()
         test_port.test_isfile = lambda test: True
 
         test_config = test_port.test_configuration()
-        test_port.expectations_dict = lambda: {'expectations': """Bug(x) [ Win Release ] failures/expected/foo.html [ Failure ]
+        test_port.expectations_dict = lambda **kwargs: {'expectations': """Bug(x) [ Win Release ] failures/expected/foo.html [ Failure ]
 Bug(y) [ Win Debug ] failures/expected/foo.html [ Crash ]
 """}
         expectations = TestExpectations(test_port)
 Bug(y) [ Win Debug ] failures/expected/foo.html [ Crash ]
 """}
         expectations = TestExpectations(test_port)
index 5683003..d0249ac 100644 (file)
@@ -195,7 +195,7 @@ def _interpret_test_failures(failures):
 
 
 # These results must match ones in print_unexpected_results() in views/buildbot_results.py.
 
 
 # These results must match ones in print_unexpected_results() in views/buildbot_results.py.
-def summarize_results(port_obj, expectations, initial_results, retry_results, enabled_pixel_tests_in_retry, include_passes=False, include_time_and_modifiers=False):
+def summarize_results(port_obj, expectations_by_type, initial_results, retry_results, enabled_pixel_tests_in_retry, include_passes=False, include_time_and_modifiers=False):
     """Returns a dictionary containing a summary of the test runs, with the following fields:
         'version': a version indicator
         'fixable': The number of fixable tests (NOW - PASS)
     """Returns a dictionary containing a summary of the test runs, with the following fields:
         'version': a version indicator
         'fixable': The number of fixable tests (NOW - PASS)
@@ -234,8 +234,17 @@ def summarize_results(port_obj, expectations, initial_results, retry_results, en
         # whether or not it crashed when we retried it (if we retried it),
         # and always consider the result not flaky.
         pixel_tests_enabled = enabled_pixel_tests_in_retry or port_obj._options.pixel_tests or bool(result.reftest_type)
         # whether or not it crashed when we retried it (if we retried it),
         # and always consider the result not flaky.
         pixel_tests_enabled = enabled_pixel_tests_in_retry or port_obj._options.pixel_tests or bool(result.reftest_type)
-        test_expectation = expectations.filtered_expectations_for_test(test_name, pixel_tests_enabled, port_obj._options.world_leaks)
-        expected = expectations.model().expectations_to_string(test_expectation)
+
+        # We're basically trying to find the first non-skip expectation, and use that expectation object for the remainder of the loop.
+        # This works because tests are run on the first device type which won't skip them, regardless of other expectations, and never re-run.
+        expected = 'SKIP'
+        expectations = expectations_by_type.values()[0]
+        for element in expectations_by_type.itervalues():
+            test_expectation = element.filtered_expectations_for_test(test_name, pixel_tests_enabled, port_obj._options.world_leaks)
+            expected = element.model().expectations_to_string(test_expectation)
+            if expected != 'SKIP':
+                expectations = element
+                continue
 
         result_type = result.type
         actual = [keywords[result_type]]
 
         result_type = result.type
         actual = [keywords[result_type]]
index 2838e29..2df45d9 100644 (file)
@@ -113,7 +113,7 @@ def summarized_results(port, expected, passing, flaky, include_passes=False):
     else:
         retry_results = None
 
     else:
         retry_results = None
 
-    return test_run_results.summarize_results(port, initial_results.expectations, initial_results, retry_results,
+    return test_run_results.summarize_results(port, {None: initial_results.expectations}, initial_results, retry_results,
         enabled_pixel_tests_in_retry=False, include_passes=include_passes)
 
 
         enabled_pixel_tests_in_retry=False, include_passes=include_passes)
 
 
index 4b21ba9..ba1a151 100644 (file)
@@ -54,6 +54,7 @@ from webkitpy.layout_tests.models.test_run_results import INTERRUPTED_EXIT_STATU
 from webkitpy.port import Port
 from webkitpy.port import test
 from webkitpy.test.skip import skip_if
 from webkitpy.port import Port
 from webkitpy.port import test
 from webkitpy.test.skip import skip_if
+from webkitpy.xcode.device_type import DeviceType
 
 
 def parse_args(extra_args=None, tests_included=False, new_results=False, print_nothing=True):
 
 
 def parse_args(extra_args=None, tests_included=False, new_results=False, print_nothing=True):
@@ -836,6 +837,63 @@ class RunTest(unittest.TestCase, StreamTestingMixin):
         self.assertTrue('text.html passed' in logging_stream.getvalue())
         self.assertTrue('image.html passed' in logging_stream.getvalue())
 
         self.assertTrue('text.html passed' in logging_stream.getvalue())
         self.assertTrue('image.html passed' in logging_stream.getvalue())
 
+    def test_device_type_test_division(self):
+        host = MockHost()
+        port = host.port_factory.get('ios-simulator')
+
+        host.filesystem.write_text_file('/mock-checkout/LayoutTests/test1.html', '')
+        host.filesystem.write_text_file('/mock-checkout/LayoutTests/platform/ios/test2.html', '')
+        host.filesystem.write_text_file('/mock-checkout/LayoutTests/platform/ipad/test3.html', '')
+        host.filesystem.write_text_file('/MOCK output of child process/ImageDiff', '')
+
+        oc = outputcapture.OutputCapture()
+        try:
+            oc.capture_output()
+            logging = StringIO.StringIO()
+            run_webkit_tests.run(port, run_webkit_tests.parse_args(['--debug-rwt-logging', '-n', '--no-build', '--root', '/build'])[0], [], logging_stream=logging)
+        finally:
+            output, err, _ = oc.restore_output()
+
+        for line in logging.getvalue():
+            if str(DeviceType.from_string('iPhone SE')) in line:
+                self.assertTrue('Skipping 2 tests' in line)
+            elif str(DeviceType.from_string('iPhone (5th generation)')) in line:
+                self.assertTrue('Skipping 1 test' in line)
+            elif str(DeviceType.from_string('iPhone 7')) in line:
+                self.assertTrue('Skipping 0 tests' in line)
+
+    def test_device_type_specific_listing(self):
+        host = MockHost()
+        port = host.port_factory.get('ios-simulator')
+
+        host.filesystem.write_text_file('/mock-checkout/LayoutTests/test1.html', '')
+        host.filesystem.write_text_file('/mock-checkout/LayoutTests/platform/ios/test2.html', '')
+        host.filesystem.write_text_file('/mock-checkout/LayoutTests/platform/ipad/test3.html', '')
+
+        oc = outputcapture.OutputCapture()
+        try:
+            oc.capture_output()
+            logging = StringIO.StringIO()
+            run_webkit_tests._print_expectations(port, run_webkit_tests.parse_args([])[0], [], logging_stream=logging)
+        finally:
+            output, _, _ = oc.restore_output()
+
+        current_type = None
+        by_type = {}
+        for line in output.splitlines():
+            if not line:
+                continue
+            if 'Tests to run' in line:
+                current_type = DeviceType.from_string(line.split('for ')[-1].split(' running')[0]) if 'for ' in line else None
+                by_type[current_type] = []
+                continue
+            by_type[current_type].append(line)
+
+        self.assertEqual(3, len(by_type.keys()))
+        self.assertEqual(2, len(by_type[DeviceType.from_string('iPhone SE')]))
+        self.assertEqual(1, len(by_type[DeviceType.from_string('iPad (5th generation)')]))
+        self.assertEqual(0, len(by_type[DeviceType.from_string('iPhone 7')]))
+
 
 class EndToEndTest(unittest.TestCase):
     def test_reftest_with_two_notrefs(self):
 
 class EndToEndTest(unittest.TestCase):
     def test_reftest_with_two_notrefs(self):
index a68f05b..3404915 100644 (file)
@@ -80,9 +80,8 @@ class Port(object):
     ALL_BUILD_TYPES = ('debug', 'release')
 
     DEFAULT_ARCHITECTURE = 'x86'
     ALL_BUILD_TYPES = ('debug', 'release')
 
     DEFAULT_ARCHITECTURE = 'x86'
-
-    DEFAULT_DEVICE_TYPE = None
-    CUSTOM_DEVICE_TYPES = []
+    DEVICE_TYPE = None
+    DEFAULT_DEVICE_TYPES = []
 
     @classmethod
     def determine_full_port_name(cls, host, options, port_name):
 
     @classmethod
     def determine_full_port_name(cls, host, options, port_name):
@@ -187,6 +186,10 @@ class Port(object):
             return 0
         return float('inf')
 
             return 0
         return float('inf')
 
+    def supported_device_types(self):
+        # An empty list would indicate a port was incapable of running tests.
+        return [None]
+
     def worker_startup_delay_secs(self):
         # FIXME: If we start workers up too quickly, DumpRenderTree appears
         # to thrash on something and time out its first few tests. Until
     def worker_startup_delay_secs(self):
         # FIXME: If we start workers up too quickly, DumpRenderTree appears
         # to thrash on something and time out its first few tests. Until
index 25d607e..3c4b314 100644 (file)
@@ -104,7 +104,7 @@ class DevicePort(DarwinPort):
                 raise RuntimeError('Failed to install dylibs at {} on device {}'.format(self._build_path(), device.udid))
 
     def _device_type_with_version(self, device_type=None):
                 raise RuntimeError('Failed to install dylibs at {} on device {}'.format(self._build_path(), device.udid))
 
     def _device_type_with_version(self, device_type=None):
-        device_type = device_type if device_type else self.DEFAULT_DEVICE_TYPE
+        device_type = device_type if device_type else self.DEVICE_TYPE
         return DeviceType(
             hardware_family=device_type.hardware_family,
             hardware_type=device_type.hardware_type,
         return DeviceType(
             hardware_family=device_type.hardware_family,
             hardware_type=device_type.hardware_type,
@@ -116,11 +116,8 @@ class DevicePort(DarwinPort):
         if not self.DEVICE_MANAGER:
             raise RuntimeError(self.NO_DEVICE_MANAGER)
 
         if not self.DEVICE_MANAGER:
             raise RuntimeError(self.NO_DEVICE_MANAGER)
 
-        # FIXME Checking software variant is important for simulators, otherwise an iOS port could boot a watchOS simulator.
-        # Really, the DEFAULT_DEVICE_TYPE for simulators should be a general instead of specific type, then this code would
-        # explicitly compare against device_type
         device_type = self._device_type_with_version(device_type)
         device_type = self._device_type_with_version(device_type)
-        if device_type.software_variant and self.DEFAULT_DEVICE_TYPE.software_variant != device_type.software_variant:
+        if device_type not in self.DEVICE_TYPE:
             return 0
 
         if self.get_option('force'):
             return 0
 
         if self.get_option('force'):
@@ -139,6 +136,17 @@ class DevicePort(DarwinPort):
             return super(DevicePort, self).max_child_processes(device_type=None)
         return result
 
             return super(DevicePort, self).max_child_processes(device_type=None)
         return result
 
+    def supported_device_types(self):
+        types = set()
+        for device in self.DEVICE_MANAGER.available_devices(host=self.host):
+            if self.DEVICE_MANAGER == SimulatedDeviceManager and not device.platform_device.is_booted_or_booting():
+                continue
+            if device.device_type in self.DEVICE_TYPE:
+                types.add(device.device_type)
+        if types:
+            return list(types)
+        return self.DEFAULT_DEVICE_TYPES or [self.DEVICE_TYPE]
+
     def setup_test_run(self, device_type=None):
         if not self.DEVICE_MANAGER:
             raise RuntimeError(self.NO_DEVICE_MANAGER)
     def setup_test_run(self, device_type=None):
         if not self.DEVICE_MANAGER:
             raise RuntimeError(self.NO_DEVICE_MANAGER)
index 54b8130..ba527cb 100644 (file)
@@ -39,8 +39,7 @@ class IOSPort(DevicePort):
     port_name = "ios"
 
     CURRENT_VERSION = Version(12)
     port_name = "ios"
 
     CURRENT_VERSION = Version(12)
-    # FIXME: This is not a clear way to do this (although it works) https://bugs.webkit.org/show_bug.cgi?id=192160
-    DEFAULT_DEVICE_TYPE = DeviceType(software_variant='iOS')
+    DEVICE_TYPE = DeviceType(software_variant='iOS')
 
     def __init__(self, host, port_name, **kwargs):
         super(IOSPort, self).__init__(host, port_name, **kwargs)
 
     def __init__(self, host, port_name, **kwargs):
         super(IOSPort, self).__init__(host, port_name, **kwargs)
index 6ceda0e..2a25d84 100644 (file)
@@ -42,8 +42,11 @@ class IOSSimulatorPort(IOSPort):
 
     DEVICE_MANAGER = SimulatedDeviceManager
 
 
     DEVICE_MANAGER = SimulatedDeviceManager
 
-    DEFAULT_DEVICE_TYPE = DeviceType(hardware_family='iPhone', hardware_type='SE')
-    CUSTOM_DEVICE_TYPES = [DeviceType(hardware_family='iPad'), DeviceType(hardware_family='iPhone', hardware_type='7')]
+    DEFAULT_DEVICE_TYPES = [
+        DeviceType(hardware_family='iPhone', hardware_type='SE'),
+        DeviceType(hardware_family='iPad', hardware_type='(5th generation)'),
+        DeviceType(hardware_family='iPhone', hardware_type='7'),
+    ]
     SDK = apple_additions().get_sdk('iphonesimulator') if apple_additions() else 'iphonesimulator'
 
     @staticmethod
     SDK = apple_additions().get_sdk('iphonesimulator') if apple_additions() else 'iphonesimulator'
 
     @staticmethod
index 9183e4a..0e2b614 100644 (file)
@@ -37,7 +37,7 @@ class WatchPort(DevicePort):
     port_name = 'watchos'
 
     CURRENT_VERSION = Version(5)
     port_name = 'watchos'
 
     CURRENT_VERSION = Version(5)
-    DEFAULT_DEVICE_TYPE = DeviceType(software_variant='watchOS')
+    DEVICE_TYPE = DeviceType(software_variant='watchOS')
 
     def __init__(self, *args, **kwargs):
         super(WatchPort, self).__init__(*args, **kwargs)
 
     def __init__(self, *args, **kwargs):
         super(WatchPort, self).__init__(*args, **kwargs)
index 19d20a2..3debee2 100644 (file)
@@ -40,7 +40,7 @@ class WatchSimulatorPort(WatchPort):
 
     DEVICE_MANAGER = SimulatedDeviceManager
 
 
     DEVICE_MANAGER = SimulatedDeviceManager
 
-    DEFAULT_DEVICE_TYPE = DeviceType(hardware_family='Apple Watch', hardware_type='Series 3 - 42mm')
+    DEFAULT_DEVICE_TYPES = [DeviceType(hardware_family='Apple Watch', hardware_type='Series 3 - 42mm')]
     SDK = apple_additions().get_sdk('watchsimulator') if apple_additions() else 'watchsimulator'
 
     def architecture(self):
     SDK = apple_additions().get_sdk('watchsimulator') if apple_additions() else 'watchsimulator'
 
     def architecture(self):
index 5c08ca6..0c04554 100644 (file)
@@ -138,3 +138,6 @@ class DeviceType(object):
         if self.software_version is not None and other.software_version is not None and not other.software_version in self.software_version:
             return False
         return True
         if self.software_version is not None and other.software_version is not None and not other.software_version in self.software_version:
             return False
         return True
+
+    def __hash__(self):
+        return hash((self.hardware_family, self.hardware_type, self.software_variant, self.software_version))