[EFL] Gardening of the WK2 Debug Bot failing tests
[WebKit-https.git] / LayoutTests / platform / efl-wk1 / TestExpectations
1 # We divide the EFL WK2 test expectations into several categories:
2 # 1. TESTS EXPECTED TO FAIL
3 #    Test cases related to features that we don't expect to be activated for EFL WK2 since
4 #    the feature implementation in WebKit2 is missing or not complete yet.
5 #    If test cases related to a feature cannot be activated since e.g.
6 #    * the feature implementation in WebCore is not complete yet
7 #    * the feature is not relevant for the EFL port
8 #    * the feature is meant to solve a backwards compatibility issue on other platforms
9 #    please put them into EFL TestExpectations file.
10 #    If in doubt, please discuss on webkit-efl mailing list whether a set of test cases
11 #    should be moved here.
12
13 # 2. CRASHES
14
15 # 3. FLAKY TESTS
16
17 # 4. FAILING TESTS
18 #    * Interim failures that are moved here for further investigation as a result of daily
19 #      gardening. They should be analysed quickly and either resolved or moved to 1).
20 #    * Remaining failures that are moved here until further investigation after
21 #      unskipping a directory or block of tests in the Skipped list.
22 #    * Tests that we expect to pass once the respective fix for DRT is available or
23 #      a relevant compile time feature can be enabled for EFL.
24
25 #////////////////////////////////////////////////////////////////////////////////////////
26 # TESTS EXPECTED TO FAIL
27 #////////////////////////////////////////////////////////////////////////////////////////
28
29 # CSS Sticky (-webkit-sticky) supported by EFL, these tests fail on WK1.
30 webkit.org/b/95182 fast/css/sticky/sticky-overflowing.html [ ImageOnlyFailure ]
31 webkit.org/b/95182 fast/css/sticky/inflow-sticky.html [ ImageOnlyFailure ]
32 webkit.org/b/95182 fast/css/sticky/replaced-sticky.html [ ImageOnlyFailure ]
33 webkit.org/b/95182 fast/css/sticky/sticky-as-positioning-container.html [ ImageOnlyFailure ]
34 webkit.org/b/95182 fast/css/sticky/sticky-both-sides.html [ ImageOnlyFailure ]
35 webkit.org/b/95182 fast/css/sticky/sticky-left-percentage.html [ ImageOnlyFailure ]
36 webkit.org/b/95182 fast/css/sticky/sticky-left.html [ ImageOnlyFailure ]
37 webkit.org/b/95182 fast/css/sticky/sticky-margins.html [ ImageOnlyFailure ]
38 webkit.org/b/95182 fast/css/sticky/sticky-side-margins.html [ ImageOnlyFailure ]
39 webkit.org/b/95182 fast/css/sticky/sticky-top-margins.html [ ImageOnlyFailure ]
40 webkit.org/b/95182 fast/css/sticky/sticky-top.html [ ImageOnlyFailure ]
41 webkit.org/b/95182 fast/css/sticky/sticky-writing-mode-horizontal-bt.html [ ImageOnlyFailure ]
42 webkit.org/b/95182 fast/css/sticky/sticky-writing-mode-vertical-lr.html [ ImageOnlyFailure ]
43 webkit.org/b/95182 fast/css/sticky/sticky-writing-mode-vertical-rl.html [ ImageOnlyFailure ]
44
45 # EFL's LayoutTestController does not implement setWindowIsKey
46 fast/dom/Window/window-onFocus.html
47 fast/events/blur-focus-window-should-blur-focus-element.html
48
49 # EFL's LayoutTestController does not implement authenticateSession
50 http/tests/xmlhttprequest/cross-origin-authorization-with-embedder.html
51
52 # EFL DRT doesn't fully support http tests yet. These tests make timeout error.
53 http/tests/cache/history-only-cached-subresource-loads-max-age-https.html
54 http/tests/cache/subresource-failover-to-network.html
55 http/tests/loading/authentication-after-redirect-stores-wrong-credentials/authentication-after-redirect-stores-wrong-credentials.html
56 http/tests/misc/401-alternative-content.php
57 http/tests/navigation/forward-and-cancel.html
58 http/tests/navigation/post-goback-same-url.html
59 http/tests/navigation/post-goback2.html
60 http/tests/navigation/postredirect-frames.html
61 http/tests/navigation/postredirect-goback2.html
62 http/tests/navigation/postredirect-reload.html
63 http/tests/security/401-logout/401-logout.php
64 http/tests/security/cookies/first-party-cookie-allow-xslt.xml
65 http/tests/security/cookies/third-party-cookie-blocking-main-frame.html
66 http/tests/security/no-referrer.html
67 http/tests/security/referrer-policy-redirect-link.html
68 http/tests/xmlhttprequest/basic-auth-nopassword.html
69 http/tests/xmlhttprequest/basic-auth-nouser.html
70 http/tests/xmlhttprequest/failed-auth.html
71 http/tests/xmlhttprequest/remember-bad-password.html
72
73 #////////////////////////////////////////////////////////////////////////////////////////
74 # CRASHES
75 #////////////////////////////////////////////////////////////////////////////////////////
76
77 # Crash shared with the GTK+ port
78 webkit.org/b/30724 [ Debug ] fast/loader/opaque-base-url.html [ Crash ]
79
80 #////////////////////////////////////////////////////////////////////////////////////////
81 # FLAKY TESTS
82 #////////////////////////////////////////////////////////////////////////////////////////
83
84 # Test passes, but editing delegates does not match expected ones
85 webkit.org/b/96641 media/video-seek-past-end-playing.html [ Failure Pass ]
86 webkit.org/b/96642 loader/go-back-to-different-window-size.html [ Failure Pass ]
87
88 webkit.org/b/96977 [ Release ] perf/nested-combined-selectors.html [ Failure Pass ]
89
90 #////////////////////////////////////////////////////////////////////////////////////////
91 # FAILING TESTS
92 #////////////////////////////////////////////////////////////////////////////////////////
93
94 # Frame flattening does not work properly for EFL port
95 Bug(EFL) fast/frames/flattening/frameset-flattening-advanced.html [ Failure ]
96 Bug(EFL) fast/frames/flattening/frameset-flattening-grid.html [ Failure ]
97 Bug(EFL) fast/frames/flattening/frameset-flattening-simple.html [ Failure ]
98 Bug(EFL) fast/frames/flattening/frameset-flattening-subframesets.html [ Failure ]
99 Bug(EFL) fast/frames/flattening/iframe-flattening-fixed-height.html [ Failure ]
100 Bug(EFL) fast/frames/flattening/iframe-flattening-fixed-width.html [ Failure ]
101 Bug(EFL) fast/frames/flattening/iframe-flattening-offscreen.html [ Failure ]
102 Bug(EFL) fast/frames/flattening/iframe-flattening-out-of-view.html [ Failure ]
103 Bug(EFL) fast/frames/flattening/iframe-flattening-out-of-view-and-scroll.html [ ImageOnlyFailure ]
104 Bug(EFL) fast/frames/flattening/iframe-flattening-out-of-view-scroll-and-relayout.html [ ImageOnlyFailure ]
105 Bug(EFL) fast/frames/flattening/iframe-flattening-simple.html [ Failure ]
106
107 # EFL's DRT explicitly disables X11 use
108 webkit.org/b/98231 fast/dom/Window/Plug-ins.html [ Failure ]
109 webkit.org/b/98231 fast/frames/sandboxed-iframe-about-blank.html [ Failure ]
110 webkit.org/b/98231 fast/frames/sandboxed-iframe-navigation-allowed.html [ Failure ]
111 webkit.org/b/98231 fast/replaced/invalid-object-with-fallback.html [ Failure ]
112 webkit.org/b/98231 http/tests/security/contentSecurityPolicy/object-src-none-allowed.html [ Failure ]
113 webkit.org/b/98231 http/tests/security/contentSecurityPolicy/object-src-url-blocked.html [ Failure ]
114 webkit.org/b/98231 http/tests/security/cross-origin-plugin-allowed.html [ Failure ]
115 webkit.org/b/98231 http/tests/security/cross-origin-plugin-private-browsing-toggled-allowed.html [ Failure ]
116 webkit.org/b/98231 http/tests/security/storage-blocking-loosened-plugin.html [ Failure ]
117 webkit.org/b/98231 http/tests/security/storage-blocking-loosened-private-browsing-plugin.html [ Failure ]
118
119 # EFL's DumpRenderTree should support LayoutTestController's dumpEditingCallbacks()
120 webkit.org/b/84835 editing/execCommand/indent-paragraphs.html [ Failure ]
121
122 # Incomplete policy delegate dumping (missing "originated from")
123 Bug(EFL) http/tests/misc/policy-delegate-called-twice.html [ Failure ]
124
125 # Custom font loading delaying text drawing on Canvas
126 webkit.org/b/87355 canvas/philip/tests/2d.text.draw.fontface.notinpage.html [ Failure ]
127
128 # EFL's TestRunner does not implement applicationCacheDiskUsageForOrigin
129 webkit.org/b/86460 http/tests/appcache/origin-usage.html [ Failure ]
130
131 # EFL's TestRunner does not implement originsWithApplicationCache
132 webkit.org/b/86498 http/tests/appcache/origins-with-appcache.html [ Failure ]
133
134 # Missing showModalDialog implementation
135 webkit.org/b/84589 fast/animation/request-animation-frame-during-modal.html [ Failure ]
136 webkit.org/b/84589 http/tests/security/cross-frame-access-call.html [ Failure ]
137
138 # Needs custom policy delegate enhancement to log DOM node info - EFL port is missing DOM node abstraction
139 webkit.org/b/87972 fast/forms/mailto/formenctype-attribute-button-html.html [ Failure ]
140 webkit.org/b/87972 fast/forms/mailto/formenctype-attribute-input-html.html [ Failure ]
141
142 # EFL's DRT needs CONFIRM NAVIGATION messages
143 webkit.org/b/87246 fast/loader/form-submission-after-beforeunload-cancel.html [ Failure ]
144
145 # New test introduced in r116694 fails
146 webkit.org/b/86182 fast/frames/seamless/seamless-inherited-document-style.html [ Failure ]
147
148 # Passes in some machines and fails on other (such as the bots).
149 webkit.org/b/97021 fast/events/dont-loose-last-event.html [ Failure ]
150
151 # Feature not mature yet and disabled by default on WebCore
152 webkit.org/b/87088 fast/js/names.html [ Failure ]
153
154 # Failing after 125592
155 webkit.org/b/94194 http/tests/security/frameNavigation/inactive-function-in-popup-navigate-child.html [ Failure ]
156
157 # EFL's LayoutTestController does not implement setTabKeyCyclesThroughElements.
158 webkit.org/b/82864 fast/events/keypress-insert-tab.html [ Failure ]
159
160 # scrolling with fixed position does not work correctly when using ewk_view_tiled. 
161 webkit.org/b/84933 compositing/geometry/fixed-position.html [ ImageOnlyFailure Pass ]
162
163 # Fail after r128393.
164 webkit.org/b/96620 http/tests/loading/pdf-commit-load-callbacks.html [ Failure ]
165 webkit.org/b/96620 http/tests/loading/text-content-type-with-binary-extension.html [ Failure ]
166
167 # Needs custom policy delegate enhancement to log downloads - EFL port is missing api for getting header fields from HTTP responses.
168 webkit.org/b/87974 http/tests/download [ Failure ]
169
170 webkit.org/b/98570 fast/replaced/object-with-embed-url-param.html [ Failure ]
171
172 webkit.org/b/53986 svg/text/caret-in-svg-text.xhtml [ Failure ]
173
174 # Relies on WebKit API [WebView _loadBackForwardListFromOtherView:]
175 fast/loader/crash-copying-backforwardlist.html
176
177 # BUG: No authentication challenge handling
178 http/tests/loading/basic-credentials-sent-automatically.html
179 http/tests/misc/authentication-redirect-1/authentication-sent-to-redirect-cross-origin.html
180 http/tests/misc/authentication-redirect-2/authentication-sent-to-redirect-same-origin.html
181 http/tests/misc/authentication-redirect-3/authentication-sent-to-redirect-same-origin-with-location-credentials.html
182
183 # The EFL port has no support for user gesture (dumpUserGestureInFrameLoadCallbacks in DRT and in the port itself as well)
184 fast/frames/location-redirect-user-gesture.html
185 fast/frames/meta-refresh-user-gesture.html
186
187 # Incomplete policy delegate dumping
188 http/tests/misc/redirect-to-external-url.html
189
190 # Probably related to https://bugs.webkit.org/show_bug.cgi?id=27637
191 http/tests/loading/basic-auth-resend-wrong-credentials.html
192
193 # BUG: wrong test results that need investigation
194 http/tests/navigation/go-back-to-error-page.html
195
196 # Probably related to https://bugs.webkit.org/show_bug.cgi?id=27637
197 http/tests/loading/basic-auth-resend-wrong-credentials.html
198
199 # Content Security Policy 1.1 (ENABLE_CSP_NEXT) is not enabled
200 # https://bugs.webkit.org/show_bug.cgi?id=85558
201 http/tests/security/contentSecurityPolicy/1.1
202
203 # Fails, seems like a rounding problem
204 svg/animations/animate-text-nested-transforms.html
205
206 # This should only be a layer change
207 # http://webkit.org/b/82129
208 fast/box-shadow/shadow-buffer-partial.html
209 fast/block/lineboxcontain/block-font.html
210 fast/block/lineboxcontain/block-glyphs.html
211 fast/block/lineboxcontain/font.html
212
213 # following tests started failing on EFL after r122952.
214 inspector/styles/override-screen-size.html
215 inspector/console/console-long-eval-crash.html
216 inspector/timeline/timeline-paint.html
217
218 # Incorrect rendering results after r115846
219 webkit.org/b/85599 http/tests/navigation/postredirect-basic.html [ Failure ]
220 webkit.org/b/85599 http/tests/navigation/postredirect-goback1.html [ Failure ]
221
222 # Consistently times out instead of being flaky.
223 webkit.org/b/56567 http/tests/cache/history-only-cached-subresource-loads.html [ Failure ]
224
225 # EFL WK1 does not support plugins yet.
226 webkit.org/b/96881 plugins [ Skip ]
227
228 # Needs custom policy delegate enhancement to log DOM node info - EFL port is missing DOM node abstraction
229 webkit.org/b/87972 fast/loader/policy-delegate-action-hit-test-zoomed.html [ Failure ]
230 webkit.org/b/87972 fast/encoding/mailto-always-utf-8.html [ Failure ]
231
232 # Frame does not get focus as expected
233 webkit.org/b/97077 inspector/extensions/extensions-panel.html [ Failure ]