Visual viewports: carets and selection UI are incorrectly positioned when editing...
authorwenson_hsieh@apple.com <wenson_hsieh@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 16 Dec 2016 20:24:37 +0000 (20:24 +0000)
committerwenson_hsieh@apple.com <wenson_hsieh@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 16 Dec 2016 20:24:37 +0000 (20:24 +0000)
https://bugs.webkit.org/show_bug.cgi?id=165767
<rdar://problem/29602382>

Reviewed by Simon Fraser.

Source/WebCore:

When changing the layout viewport override, mark viewport-constrained objects as needing layout. If only the
width and height of the old and new layout viewports are compared, EditorState info (namely selection and caret
rects) that depends on the document location of fixed elements may be stale immediately after the layout
viewport override changes and before layout occurs.

This caused one of the tests (fixed-caret-position-after-scroll.html) to occasionally fail.

Tests: editing/caret/ios/absolute-caret-position-after-scroll.html
       editing/caret/ios/fixed-caret-position-after-scroll.html
       editing/selection/ios/absolute-selection-after-scroll.html
       editing/selection/ios/fixed-selection-after-scroll.html

* page/FrameView.cpp:
(WebCore::FrameView::setLayoutViewportOverrideRect):

Source/WebKit2:

When focusing an input, the position of the caret on iOS is determined by the overridden layout viewport rect in
the web process. However, this rect is not updated until the end the scroll gesture. Whereas this is fine for
non-fixed inputs since their document location does not change, fixed inputs effectively change their position
in the document as the user scrolls. This causes the caret to be 'left behind' in the document position it was
in at the start of the scroll. To fix this, we deactivate the selection when exiting stable state if the
assisted node is in a fixed position container, and reenable it upon receiving the next stable state EditorState
update (as indicated by postLayoutData().isStableStateUpdate). Additionally, we apply similar treatment to the
web selection assistant -- this time, we need to force the selection view to hide (analagous to deactivating
the text selection assistant) and show it again upon receiving the next selection change update when the WebPage
(in the web process) is stable.

Furthermore, adds test support for querying text caret and selection rects, as well as perform a callback after
the WebPage has indicated that it is stable, both as SPI on the WKWebView.

Covered by 4 new layout tests in fast/editing/caret/ios and fast/editing/selection/ios.

* Platform/spi/ios/UIKitSPI.h:
* Shared/EditorState.cpp:
(WebKit::EditorState::PostLayoutData::encode):
(WebKit::EditorState::PostLayoutData::decode):
* Shared/EditorState.h:

Introduce isStableStateUpdate, which is true when the WebPage is known to be in stable state, as well as
insideFixedPosition, which is true when the current selection is inside a fixed position container.

* Shared/mac/RemoteLayerTreeTransaction.h:
(WebKit::RemoteLayerTreeTransaction::isInStableState):
(WebKit::RemoteLayerTreeTransaction::setIsInStableState):
* Shared/mac/RemoteLayerTreeTransaction.mm:
(WebKit::RemoteLayerTreeTransaction::encode):
(WebKit::RemoteLayerTreeTransaction::decode):
(WebKit::RemoteLayerTreeTransaction::description):
* UIProcess/API/Cocoa/WKWebView.mm:
(-[WKWebView _didCommitLayerTree:]):
(-[WKWebView _uiTextCaretRect]):

Introduced a new SPI method for fetching the current rect of the text assistant's caret view, at keyPath
"selectionView.selection.caretRect".

(-[WKWebView _uiTextSelectionRects]):

Renamed (and refactored) from _uiTextSelectionRectViews, which was previously fetching an array of UIViews. I
found this to cause character-granularity-rect.html to fail due to the array of UIViews here being empty, so I
refactored this to simply return an array of rects from the keyPath "selectionView.selection.selectionRects" for
the text selection assistant and @"selectionView.selectionRects" for the web selection assistant.

(-[WKWebView _doAfterNextStablePresentationUpdate:]):

Runs the given block after both the UI process and web processes agree that the visible content rect state is
stable. To do this, we fire presentation updates until the UI process (via RemoteLayerTreeTransactions)
discovers that the web page is in stable state. This is used solely for testing purposes.

(-[WKWebView _firePresentationUpdateForPendingStableStatePresentationCallbacks]):
(-[WKWebView _uiTextSelectionRectViews]): Deleted.
* UIProcess/API/Cocoa/WKWebViewPrivate.h:
* UIProcess/WebPageProxy.h:
(WebKit::WebPageProxy::inStableState):
* UIProcess/ios/WKContentView.mm:
(-[WKContentView _didExitStableState]):

Deactivate the text selection if the assisted node is inside a fixed container.

(-[WKContentView didUpdateVisibleRect:unobscuredRect:unobscuredRectInScrollViewCoordinates:obscuredInset:scale:minimumScale:inStableState:isChangingObscuredInsetsInteractively:enclosedInScrollableAncestorView:]):
* UIProcess/ios/WKContentViewInteraction.h:
* UIProcess/ios/WKContentViewInteraction.mm:
(-[WKContentView setupInteraction]):
(-[WKContentView cleanupInteraction]):
(-[WKContentView shouldHideSelectionWhenScrolling]):
(-[WKContentView _uiTextSelectionRects]):
(-[WKContentView _didEndScrollingOrZooming]):
(-[WKContentView _updateChangedSelection:]):

If the EditorState was created after a stable state update, reactivate the text selection assistant if it exists.
Additionally, if we are deferring the end scrolling selection update until after the first stable editor state
update arrives from the web process, we need to also call [_textSelectionAssistant didEndScrollingOverflow]
and [_webSelectionAssistant didEndScrollingOrZoomingPage] here instead of doing so immediately after scrolling
finishes. This ensures that selection UI (the callout and selection highlights) do not flicker from their old
position to the new position when scrolling finishes.

* WebProcess/WebPage/WebPage.cpp:
(WebKit::WebPage::willCommitLayerTree):
* WebProcess/WebPage/ios/WebPageIOS.mm:
(WebKit::WebPage::platformEditorState):
(WebKit::WebPage::updateVisibleContentRects):

When updating the layout viewport override rect, also recompute the caret if needed and send an updated
EditorState over to the UI process.

Tools:

Introduces two new UIScriptController methods: doAfterWebPageIsInStableState and textSelectionCaretRect. See
WebKit2 ChangeLog for more details.

* DumpRenderTree/ios/UIScriptControllerIOS.mm:
(WTR::UIScriptController::doAfterNextStablePresentationUpdate):
(WTR::UIScriptController::textSelectionCaretRect):
* DumpRenderTree/mac/UIScriptControllerMac.mm:
(WTR::UIScriptController::doAfterNextStablePresentationUpdate):
* TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl:
* TestRunnerShared/UIScriptContext/UIScriptController.cpp:
(WTR::UIScriptController::doAfterNextStablePresentationUpdate):
(WTR::UIScriptController::textSelectionCaretRect):
* TestRunnerShared/UIScriptContext/UIScriptController.h:
* WebKitTestRunner/cocoa/TestRunnerWKWebView.mm:
(-[TestRunnerWKWebView _setStableStateOverride:]):

Force the WKWebView to update its visible content rects when changing the stable state override.

* WebKitTestRunner/ios/UIScriptControllerIOS.mm:
(WTR::toNSDictionary):
(WTR::UIScriptController::doAfterNextStablePresentationUpdate):
(WTR::UIScriptController::selectionRangeViewRects):
(WTR::UIScriptController::textSelectionCaretRect):
* WebKitTestRunner/mac/UIScriptControllerMac.mm:
(WTR::UIScriptController::doAfterNextStablePresentationUpdate):

LayoutTests:

Adds new layout tests verifying that scrolling selected text (non-editable) and a text caret (in editable
content) results in the selection/caret rects having the correct location relative to the document, in both
cases where the selected/focused element has fixed position or absolute position. For fixed position elements,
this means that the rects must "move" down in the document as the document is scrolled, but for absolute
elements, these rects must remain in place.

* TestExpectations:
* editing/caret/ios/absolute-caret-position-after-scroll-expected.txt: Added.
* editing/caret/ios/absolute-caret-position-after-scroll.html: Added.
* editing/caret/ios/fixed-caret-position-after-scroll-expected.txt: Added.
* editing/caret/ios/fixed-caret-position-after-scroll.html: Added.
* editing/selection/ios/absolute-selection-after-scroll-expected.txt: Added.
* editing/selection/ios/absolute-selection-after-scroll.html: Added.
* editing/selection/ios/fixed-selection-after-scroll-expected.txt: Added.
* editing/selection/ios/fixed-selection-after-scroll.html: Added.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@209931 268f45cc-cd09-0410-ab3c-d52691b4dbfc

35 files changed:
LayoutTests/ChangeLog
LayoutTests/TestExpectations
LayoutTests/editing/caret/ios/absolute-caret-position-after-scroll-expected.txt [new file with mode: 0644]
LayoutTests/editing/caret/ios/absolute-caret-position-after-scroll.html [new file with mode: 0644]
LayoutTests/editing/caret/ios/fixed-caret-position-after-scroll-expected.txt [new file with mode: 0644]
LayoutTests/editing/caret/ios/fixed-caret-position-after-scroll.html [new file with mode: 0644]
LayoutTests/editing/selection/ios/absolute-selection-after-scroll-expected.txt [new file with mode: 0644]
LayoutTests/editing/selection/ios/absolute-selection-after-scroll.html [new file with mode: 0644]
LayoutTests/editing/selection/ios/fixed-selection-after-scroll-expected.txt [new file with mode: 0644]
LayoutTests/editing/selection/ios/fixed-selection-after-scroll.html [new file with mode: 0644]
Source/WebCore/ChangeLog
Source/WebCore/page/FrameView.cpp
Source/WebKit2/ChangeLog
Source/WebKit2/Platform/spi/ios/UIKitSPI.h
Source/WebKit2/Shared/EditorState.cpp
Source/WebKit2/Shared/EditorState.h
Source/WebKit2/Shared/mac/RemoteLayerTreeTransaction.h
Source/WebKit2/Shared/mac/RemoteLayerTreeTransaction.mm
Source/WebKit2/UIProcess/API/Cocoa/WKWebView.mm
Source/WebKit2/UIProcess/API/Cocoa/WKWebViewPrivate.h
Source/WebKit2/UIProcess/WebPageProxy.h
Source/WebKit2/UIProcess/ios/WKContentView.mm
Source/WebKit2/UIProcess/ios/WKContentViewInteraction.h
Source/WebKit2/UIProcess/ios/WKContentViewInteraction.mm
Source/WebKit2/WebProcess/WebPage/WebPage.cpp
Source/WebKit2/WebProcess/WebPage/ios/WebPageIOS.mm
Tools/ChangeLog
Tools/DumpRenderTree/ios/UIScriptControllerIOS.mm
Tools/DumpRenderTree/mac/UIScriptControllerMac.mm
Tools/TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl
Tools/TestRunnerShared/UIScriptContext/UIScriptController.cpp
Tools/TestRunnerShared/UIScriptContext/UIScriptController.h
Tools/WebKitTestRunner/cocoa/TestRunnerWKWebView.mm
Tools/WebKitTestRunner/ios/UIScriptControllerIOS.mm
Tools/WebKitTestRunner/mac/UIScriptControllerMac.mm

index ab62ea1..f4f5326 100644 (file)
@@ -1,3 +1,27 @@
+2016-12-16  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        Visual viewports: carets and selection UI are incorrectly positioned when editing fixed elements
+        https://bugs.webkit.org/show_bug.cgi?id=165767
+        <rdar://problem/29602382>
+
+        Reviewed by Simon Fraser.
+
+        Adds new layout tests verifying that scrolling selected text (non-editable) and a text caret (in editable
+        content) results in the selection/caret rects having the correct location relative to the document, in both
+        cases where the selected/focused element has fixed position or absolute position. For fixed position elements,
+        this means that the rects must "move" down in the document as the document is scrolled, but for absolute
+        elements, these rects must remain in place.
+
+        * TestExpectations:
+        * editing/caret/ios/absolute-caret-position-after-scroll-expected.txt: Added.
+        * editing/caret/ios/absolute-caret-position-after-scroll.html: Added.
+        * editing/caret/ios/fixed-caret-position-after-scroll-expected.txt: Added.
+        * editing/caret/ios/fixed-caret-position-after-scroll.html: Added.
+        * editing/selection/ios/absolute-selection-after-scroll-expected.txt: Added.
+        * editing/selection/ios/absolute-selection-after-scroll.html: Added.
+        * editing/selection/ios/fixed-selection-after-scroll-expected.txt: Added.
+        * editing/selection/ios/fixed-selection-after-scroll.html: Added.
+
 2016-12-16  Zalan Bujtas  <zalan@apple.com>
 
         Defer certain accessibility callbacks until after layout is finished.
index f90d1a1..bf9b135 100644 (file)
@@ -13,7 +13,9 @@ accessibility/mac [ Skip ]
 accessibility/win [ Skip ]
 displaylists [ Skip ]
 editing/mac [ Skip ]
+editing/caret/ios [ Skip ]
 editing/pasteboard/gtk [ Skip ]
+editing/selection/ios [ Skip ]
 tiled-drawing [ Skip ]
 fast/visual-viewport/tiled-drawing [ Skip ]
 swipe [ Skip ]
diff --git a/LayoutTests/editing/caret/ios/absolute-caret-position-after-scroll-expected.txt b/LayoutTests/editing/caret/ios/absolute-caret-position-after-scroll-expected.txt
new file mode 100644 (file)
index 0000000..33999f8
--- /dev/null
@@ -0,0 +1,10 @@
+PASS successfullyParsed is true
+
+TEST COMPLETE
+The initial caret rect is: [6 17 ; 3 15]
+The caret rect after scrolling 1000px down is: [6 17 ; 3 15]
+PASS finalCaretRect.top is initialCaretRect.top
+PASS finalCaretRect.left is initialCaretRect.left
+PASS finalCaretRect.width is initialCaretRect.width
+PASS finalCaretRect.height is initialCaretRect.height
+
diff --git a/LayoutTests/editing/caret/ios/absolute-caret-position-after-scroll.html b/LayoutTests/editing/caret/ios/absolute-caret-position-after-scroll.html
new file mode 100644 (file)
index 0000000..fb97521
--- /dev/null
@@ -0,0 +1,88 @@
+<html>
+<meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no">
+<head>
+    <script src="../../../resources/js-test-pre.js"></script>
+    <style>
+        body {
+            margin: 0;
+        }
+
+        input {
+            width: 100%;
+            height: 50px;
+            position: absolute;
+            left: 0;
+            top: 0;
+        }
+
+        div {
+            background-image: linear-gradient(0deg, blue, red);
+            height: 4000px;
+        }
+    </style>
+    <script>
+    if (window.testRunner) {
+        testRunner.dumpAsText();
+        testRunner.waitUntilDone();
+    }
+
+    function tapInInputScript(tapX, tapY)
+    {
+        return `(function() {
+            uiController.didShowKeyboardCallback = function() {
+                uiController.doAfterNextStablePresentationUpdate(function() {
+                    uiController.uiScriptComplete(JSON.stringify(uiController.textSelectionCaretRect));
+                });
+            };
+            uiController.singleTapAtPoint(${tapX}, ${tapY}, function() { });
+        })()`;
+    }
+
+    function simulateScrollingScript(distance)
+    {
+        return `(function() {
+            uiController.stableStateOverride = false;
+            uiController.immediateScrollToOffset(0, ${distance});
+            uiController.stableStateOverride = true;
+            uiController.doAfterNextStablePresentationUpdate(function() {
+                uiController.uiScriptComplete(JSON.stringify(uiController.textSelectionCaretRect));
+            });
+        })()`;
+    }
+
+    function toString(rect)
+    {
+        return `[${rect.left} ${rect.top} ; ${rect.width} ${rect.height}]`;
+    }
+
+    function run()
+    {
+        if (!window.testRunner || !testRunner.runUIScript) {
+            description("To manually test, tap this input field and scroll up. The text caret should not end up outside of the input.");
+            return;
+        }
+
+        testRunner.runUIScript(tapInInputScript(window.innerWidth / 2, 30), initialCaretRect => {
+            initialCaretRect = JSON.parse(initialCaretRect);
+            window.initialCaretRect = initialCaretRect;
+            debug(`The initial caret rect is: ${toString(initialCaretRect)}`);
+            testRunner.runUIScript(simulateScrollingScript(1000), finalCaretRect => {
+                finalCaretRect = JSON.parse(finalCaretRect);
+                window.finalCaretRect = finalCaretRect;
+                debug(`The caret rect after scrolling 1000px down is: ${toString(finalCaretRect)}`);
+                shouldBe("finalCaretRect.top", "initialCaretRect.top");
+                shouldBe("finalCaretRect.left", "initialCaretRect.left");
+                shouldBe("finalCaretRect.width", "initialCaretRect.width");
+                shouldBe("finalCaretRect.height", "initialCaretRect.height");
+                testRunner.notifyDone();
+            });
+        });
+    }
+    </script>
+</head>
+<body onload=run()>
+    <input></input>
+    <script src="../../../resources/js-test-post.js"></script>
+</body>
+
+</html>
diff --git a/LayoutTests/editing/caret/ios/fixed-caret-position-after-scroll-expected.txt b/LayoutTests/editing/caret/ios/fixed-caret-position-after-scroll-expected.txt
new file mode 100644 (file)
index 0000000..58f5654
--- /dev/null
@@ -0,0 +1,7 @@
+PASS successfullyParsed is true
+
+TEST COMPLETE
+The initial caret rect is: [6 17 ; 3 15]
+The caret rect after scrolling 1000px down is: [6 1017 ; 3 15]
+PASS finalCaretRect.top - initialCaretRect.top is 1000
+
diff --git a/LayoutTests/editing/caret/ios/fixed-caret-position-after-scroll.html b/LayoutTests/editing/caret/ios/fixed-caret-position-after-scroll.html
new file mode 100644 (file)
index 0000000..e7f37d6
--- /dev/null
@@ -0,0 +1,85 @@
+<html>
+<meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no">
+<head>
+    <script src="../../../resources/js-test-pre.js"></script>
+    <style>
+        body {
+            margin: 0;
+        }
+
+        input {
+            width: 100%;
+            height: 50px;
+            position: fixed;
+            left: 0;
+            top: 0;
+        }
+
+        div {
+            background-image: linear-gradient(blue, red);
+            height: 4000px;
+        }
+    </style>
+    <script>
+    if (window.testRunner) {
+        testRunner.dumpAsText();
+        testRunner.waitUntilDone();
+    }
+
+    function tapInInputScript(tapX, tapY)
+    {
+        return `(function() {
+            uiController.didShowKeyboardCallback = function() {
+                uiController.doAfterNextStablePresentationUpdate(function() {
+                    uiController.uiScriptComplete(JSON.stringify(uiController.textSelectionCaretRect));
+                });
+            };
+            uiController.singleTapAtPoint(${tapX}, ${tapY}, function() { });
+        })()`;
+    }
+
+    function simulateScrollingScript(distance)
+    {
+        return `(function() {
+            uiController.stableStateOverride = false;
+            uiController.immediateScrollToOffset(0, ${distance});
+            uiController.stableStateOverride = true;
+            uiController.doAfterNextStablePresentationUpdate(function() {
+                uiController.uiScriptComplete(JSON.stringify(uiController.textSelectionCaretRect));
+            });
+        })()`;
+    }
+
+    function toString(rect)
+    {
+        return `[${rect.left} ${rect.top} ; ${rect.width} ${rect.height}]`;
+    }
+
+    function run()
+    {
+        if (!window.testRunner || !testRunner.runUIScript) {
+            description("To manually test, tap this input field and scroll up. The text caret should not end up outside of the input.");
+            return;
+        }
+
+        testRunner.runUIScript(tapInInputScript(window.innerWidth / 2, 30), initialCaretRect => {
+            initialCaretRect = JSON.parse(initialCaretRect);
+            window.initialCaretRect = initialCaretRect;
+            debug(`The initial caret rect is: ${toString(initialCaretRect)}`);
+            testRunner.runUIScript(simulateScrollingScript(1000), finalCaretRect => {
+                finalCaretRect = JSON.parse(finalCaretRect);
+                window.finalCaretRect = finalCaretRect;
+                debug(`The caret rect after scrolling 1000px down is: ${toString(finalCaretRect)}`);
+                shouldBe("finalCaretRect.top - initialCaretRect.top", "1000");
+                testRunner.notifyDone();
+            });
+        });
+    }
+    </script>
+</head>
+<body onload=run()>
+    <input></input>
+    <script src="../../../resources/js-test-post.js"></script>
+</body>
+
+</html>
diff --git a/LayoutTests/editing/selection/ios/absolute-selection-after-scroll-expected.txt b/LayoutTests/editing/selection/ios/absolute-selection-after-scroll-expected.txt
new file mode 100644 (file)
index 0000000..a0761db
--- /dev/null
@@ -0,0 +1,11 @@
+PASS successfullyParsed is true
+
+TEST COMPLETE
+After long pressing, the selection rects are: [0 0 ; 309 114]
+After scrolling 1000px down, the selection rects are: [0 0 ; 309 114]
+PASS finalSelectionRects[0].top is initialSelectionRects[0].top
+PASS finalSelectionRects[0].left is initialSelectionRects[0].left
+PASS finalSelectionRects[0].width is initialSelectionRects[0].width
+PASS finalSelectionRects[0].height is initialSelectionRects[0].height
+WebKit
+
diff --git a/LayoutTests/editing/selection/ios/absolute-selection-after-scroll.html b/LayoutTests/editing/selection/ios/absolute-selection-after-scroll.html
new file mode 100644 (file)
index 0000000..473ab1e
--- /dev/null
@@ -0,0 +1,94 @@
+<html>
+<meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no">
+<head>
+    <script src="../../../resources/js-test-pre.js"></script>
+    <style>
+        body {
+            margin: 0;
+        }
+
+        #fixed {
+            width: 100vw;
+            height: 50px;
+            position: absolute;
+            left: 0;
+            top: 0;
+            font-size: 100px;
+        }
+
+        #content {
+            background-image: linear-gradient(blue, red);
+            height: 4000px;
+        }
+    </style>
+    <script>
+    if (window.testRunner) {
+        testRunner.dumpAsText();
+        testRunner.waitUntilDone();
+    }
+
+    function toString(rect)
+    {
+        return `[${rect.left} ${rect.top} ; ${rect.width} ${rect.height}]`;
+    }
+
+    function selectFixedTextScript(tapX, tapY)
+    {
+        return `
+        (function() {
+            uiController.longPressAtPoint(${tapX}, ${tapY}, function() {
+                uiController.doAfterNextStablePresentationUpdate(function() {
+                    uiController.uiScriptComplete(JSON.stringify(uiController.selectionRangeViewRects));
+                });
+            });
+        })();`;
+    }
+
+    function simulateScrollingScript(distance)
+    {
+        return `(function() {
+            uiController.stableStateOverride = false;
+            uiController.immediateScrollToOffset(0, ${distance});
+            uiController.stableStateOverride = true;
+            uiController.doAfterNextStablePresentationUpdate(function() {
+                uiController.uiScriptComplete(JSON.stringify(uiController.selectionRangeViewRects));
+            });
+        })()`;
+    }
+
+    function toString(rect)
+    {
+        return `[${rect.left} ${rect.top} ; ${rect.width} ${rect.height}]`;
+    }
+
+    function run()
+    {
+        if (!window.testRunner || !testRunner.runUIScript) {
+            description("To manually test, long press this text and scroll up. The selection rect should be in the expected place (at the very top of the page) after scrolling.");
+            return;
+        }
+
+        testRunner.runUIScript(selectFixedTextScript(50, 50), initialSelectionRects => {
+            initialSelectionRects = JSON.parse(initialSelectionRects);
+            window.initialSelectionRects = initialSelectionRects;
+            debug(`After long pressing, the selection rects are: ${initialSelectionRects.map(toString)}`);
+            testRunner.runUIScript(simulateScrollingScript(1000), finalSelectionRects => {
+                finalSelectionRects = JSON.parse(finalSelectionRects);
+                window.finalSelectionRects = finalSelectionRects;
+                debug(`After scrolling 1000px down, the selection rects are: ${finalSelectionRects.map(toString)}`);
+                shouldBe("finalSelectionRects[0].top", "initialSelectionRects[0].top");
+                shouldBe("finalSelectionRects[0].left", "initialSelectionRects[0].left");
+                shouldBe("finalSelectionRects[0].width", "initialSelectionRects[0].width");
+                shouldBe("finalSelectionRects[0].height", "initialSelectionRects[0].height");
+                testRunner.notifyDone();
+            });
+        });
+    }
+    </script>
+</head>
+<body onload=run()>
+    <div id="fixed">WebKit</div>
+    <div id="content"></div>
+    <script src="../../../resources/js-test-post.js"></script>
+</body>
+</html>
\ No newline at end of file
diff --git a/LayoutTests/editing/selection/ios/fixed-selection-after-scroll-expected.txt b/LayoutTests/editing/selection/ios/fixed-selection-after-scroll-expected.txt
new file mode 100644 (file)
index 0000000..23e0231
--- /dev/null
@@ -0,0 +1,10 @@
+PASS successfullyParsed is true
+
+TEST COMPLETE
+After long pressing, the selection rects are: [0 0 ; 309 114]
+After scrolling 1000px down, the selection rects are: [0 1000 ; 309 114]
+PASS initialSelectionRects.length is 1
+PASS finalSelectionRects.length is 1
+PASS finalSelectionRects[0].top - initialSelectionRects[0].top is 1000
+WebKit
+
diff --git a/LayoutTests/editing/selection/ios/fixed-selection-after-scroll.html b/LayoutTests/editing/selection/ios/fixed-selection-after-scroll.html
new file mode 100644 (file)
index 0000000..22f721b
--- /dev/null
@@ -0,0 +1,93 @@
+<html>
+<meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no">
+<head>
+    <script src="../../../resources/js-test-pre.js"></script>
+    <style>
+        body {
+            margin: 0;
+        }
+
+        #fixed {
+            width: 100vw;
+            height: 50px;
+            position: fixed;
+            left: 0;
+            top: 0;
+            font-size: 100px;
+        }
+
+        #content {
+            background-image: linear-gradient(blue, red);
+            height: 4000px;
+        }
+    </style>
+    <script>
+    if (window.testRunner) {
+        testRunner.dumpAsText();
+        testRunner.waitUntilDone();
+    }
+
+    function toString(rect)
+    {
+        return `[${rect.left} ${rect.top} ; ${rect.width} ${rect.height}]`;
+    }
+
+    function selectFixedTextScript(tapX, tapY)
+    {
+        return `
+        (function() {
+            uiController.longPressAtPoint(${tapX}, ${tapY}, function() {
+                uiController.doAfterNextStablePresentationUpdate(function() {
+                    uiController.uiScriptComplete(JSON.stringify(uiController.selectionRangeViewRects));
+                });
+            });
+        })();`;
+    }
+
+    function simulateScrollingScript(distance)
+    {
+        return `(function() {
+            uiController.stableStateOverride = false;
+            uiController.immediateScrollToOffset(0, ${distance});
+            uiController.stableStateOverride = true;
+            uiController.doAfterNextStablePresentationUpdate(function() {
+                uiController.uiScriptComplete(JSON.stringify(uiController.selectionRangeViewRects));
+            });
+        })()`;
+    }
+
+    function toString(rect)
+    {
+        return `[${rect.left} ${rect.top} ; ${rect.width} ${rect.height}]`;
+    }
+
+    function run()
+    {
+        if (!window.testRunner || !testRunner.runUIScript) {
+            description("To manually test, long press this text and scroll up. The selection rect should be in the expected place (in the fixed div) after scrolling.");
+            return;
+        }
+
+        testRunner.runUIScript(selectFixedTextScript(50, 50), initialSelectionRects => {
+            initialSelectionRects = JSON.parse(initialSelectionRects);
+            window.initialSelectionRects = initialSelectionRects;
+            debug(`After long pressing, the selection rects are: ${initialSelectionRects.map(toString)}`);
+            testRunner.runUIScript(simulateScrollingScript(1000), finalSelectionRects => {
+                finalSelectionRects = JSON.parse(finalSelectionRects);
+                window.finalSelectionRects = finalSelectionRects;
+                debug(`After scrolling 1000px down, the selection rects are: ${finalSelectionRects.map(toString)}`);
+                shouldBe("initialSelectionRects.length", "1");
+                shouldBe("finalSelectionRects.length", "1");
+                shouldBe("finalSelectionRects[0].top - initialSelectionRects[0].top", "1000");
+                testRunner.notifyDone();
+            });
+        });
+    }
+    </script>
+</head>
+<body onload=run()>
+    <div id="fixed">WebKit</div>
+    <div id="content"></div>
+    <script src="../../../resources/js-test-post.js"></script>
+</body>
+</html>
\ No newline at end of file
index 0823269..ea28009 100644 (file)
@@ -1,3 +1,26 @@
+2016-12-16  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        Visual viewports: carets and selection UI are incorrectly positioned when editing fixed elements
+        https://bugs.webkit.org/show_bug.cgi?id=165767
+        <rdar://problem/29602382>
+
+        Reviewed by Simon Fraser.
+
+        When changing the layout viewport override, mark viewport-constrained objects as needing layout. If only the
+        width and height of the old and new layout viewports are compared, EditorState info (namely selection and caret
+        rects) that depends on the document location of fixed elements may be stale immediately after the layout
+        viewport override changes and before layout occurs.
+
+        This caused one of the tests (fixed-caret-position-after-scroll.html) to occasionally fail.
+
+        Tests: editing/caret/ios/absolute-caret-position-after-scroll.html
+               editing/caret/ios/fixed-caret-position-after-scroll.html
+               editing/selection/ios/absolute-selection-after-scroll.html
+               editing/selection/ios/fixed-selection-after-scroll.html
+
+        * page/FrameView.cpp:
+        (WebCore::FrameView::setLayoutViewportOverrideRect):
+
 2016-12-14  Sam Weinig  <sam@webkit.org>
 
         [ApplePay] Remove remaining custom bindings from the ApplePay code
index 167429a..ff79742 100644 (file)
@@ -1849,8 +1849,7 @@ void FrameView::setLayoutViewportOverrideRect(std::optional<LayoutRect> rect)
 
     LOG_WITH_STREAM(Scrolling, stream << "\nFrameView " << this << " setLayoutViewportOverrideRect() - changing layout viewport from " << oldRect << " to " << m_layoutViewportOverrideRect.value());
 
-    // FIXME: do we need to also do this if the origin changes?
-    if (oldRect.size() != layoutViewportRect().size())
+    if (oldRect != layoutViewportRect())
         setViewportConstrainedObjectsNeedLayout();
 }
 
index c4ba26b..45491c3 100644 (file)
@@ -1,3 +1,99 @@
+2016-12-16  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        Visual viewports: carets and selection UI are incorrectly positioned when editing fixed elements
+        https://bugs.webkit.org/show_bug.cgi?id=165767
+        <rdar://problem/29602382>
+
+        Reviewed by Simon Fraser.
+
+        When focusing an input, the position of the caret on iOS is determined by the overridden layout viewport rect in
+        the web process. However, this rect is not updated until the end the scroll gesture. Whereas this is fine for
+        non-fixed inputs since their document location does not change, fixed inputs effectively change their position
+        in the document as the user scrolls. This causes the caret to be 'left behind' in the document position it was
+        in at the start of the scroll. To fix this, we deactivate the selection when exiting stable state if the
+        assisted node is in a fixed position container, and reenable it upon receiving the next stable state EditorState
+        update (as indicated by postLayoutData().isStableStateUpdate). Additionally, we apply similar treatment to the
+        web selection assistant -- this time, we need to force the selection view to hide (analagous to deactivating
+        the text selection assistant) and show it again upon receiving the next selection change update when the WebPage
+        (in the web process) is stable.
+
+        Furthermore, adds test support for querying text caret and selection rects, as well as perform a callback after
+        the WebPage has indicated that it is stable, both as SPI on the WKWebView.
+
+        Covered by 4 new layout tests in fast/editing/caret/ios and fast/editing/selection/ios.
+
+        * Platform/spi/ios/UIKitSPI.h:
+        * Shared/EditorState.cpp:
+        (WebKit::EditorState::PostLayoutData::encode):
+        (WebKit::EditorState::PostLayoutData::decode):
+        * Shared/EditorState.h:
+
+        Introduce isStableStateUpdate, which is true when the WebPage is known to be in stable state, as well as
+        insideFixedPosition, which is true when the current selection is inside a fixed position container.
+
+        * Shared/mac/RemoteLayerTreeTransaction.h:
+        (WebKit::RemoteLayerTreeTransaction::isInStableState):
+        (WebKit::RemoteLayerTreeTransaction::setIsInStableState):
+        * Shared/mac/RemoteLayerTreeTransaction.mm:
+        (WebKit::RemoteLayerTreeTransaction::encode):
+        (WebKit::RemoteLayerTreeTransaction::decode):
+        (WebKit::RemoteLayerTreeTransaction::description):
+        * UIProcess/API/Cocoa/WKWebView.mm:
+        (-[WKWebView _didCommitLayerTree:]):
+        (-[WKWebView _uiTextCaretRect]):
+
+        Introduced a new SPI method for fetching the current rect of the text assistant's caret view, at keyPath
+        "selectionView.selection.caretRect".
+
+        (-[WKWebView _uiTextSelectionRects]):
+
+        Renamed (and refactored) from _uiTextSelectionRectViews, which was previously fetching an array of UIViews. I
+        found this to cause character-granularity-rect.html to fail due to the array of UIViews here being empty, so I
+        refactored this to simply return an array of rects from the keyPath "selectionView.selection.selectionRects" for
+        the text selection assistant and @"selectionView.selectionRects" for the web selection assistant.
+
+        (-[WKWebView _doAfterNextStablePresentationUpdate:]):
+
+        Runs the given block after both the UI process and web processes agree that the visible content rect state is 
+        stable. To do this, we fire presentation updates until the UI process (via RemoteLayerTreeTransactions)
+        discovers that the web page is in stable state. This is used solely for testing purposes.
+
+        (-[WKWebView _firePresentationUpdateForPendingStableStatePresentationCallbacks]):
+        (-[WKWebView _uiTextSelectionRectViews]): Deleted.
+        * UIProcess/API/Cocoa/WKWebViewPrivate.h:
+        * UIProcess/WebPageProxy.h:
+        (WebKit::WebPageProxy::inStableState):
+        * UIProcess/ios/WKContentView.mm:
+        (-[WKContentView _didExitStableState]):
+
+        Deactivate the text selection if the assisted node is inside a fixed container.
+
+        (-[WKContentView didUpdateVisibleRect:unobscuredRect:unobscuredRectInScrollViewCoordinates:obscuredInset:scale:minimumScale:inStableState:isChangingObscuredInsetsInteractively:enclosedInScrollableAncestorView:]):
+        * UIProcess/ios/WKContentViewInteraction.h:
+        * UIProcess/ios/WKContentViewInteraction.mm:
+        (-[WKContentView setupInteraction]):
+        (-[WKContentView cleanupInteraction]):
+        (-[WKContentView shouldHideSelectionWhenScrolling]):
+        (-[WKContentView _uiTextSelectionRects]):
+        (-[WKContentView _didEndScrollingOrZooming]):
+        (-[WKContentView _updateChangedSelection:]):
+
+        If the EditorState was created after a stable state update, reactivate the text selection assistant if it exists.
+        Additionally, if we are deferring the end scrolling selection update until after the first stable editor state
+        update arrives from the web process, we need to also call [_textSelectionAssistant didEndScrollingOverflow]
+        and [_webSelectionAssistant didEndScrollingOrZoomingPage] here instead of doing so immediately after scrolling
+        finishes. This ensures that selection UI (the callout and selection highlights) do not flicker from their old
+        position to the new position when scrolling finishes.
+
+        * WebProcess/WebPage/WebPage.cpp:
+        (WebKit::WebPage::willCommitLayerTree):
+        * WebProcess/WebPage/ios/WebPageIOS.mm:
+        (WebKit::WebPage::platformEditorState):
+        (WebKit::WebPage::updateVisibleContentRects):
+
+        When updating the layout viewport override rect, also recompute the caret if needed and send an updated
+        EditorState over to the UI process.
+
 2016-12-16  Claudio Saavedra  <csaavedra@igalia.com>
 
         [WK2] SharedMemory: include cleanups
index 53b2740..08f97a6 100644 (file)
@@ -450,9 +450,16 @@ typedef NS_ENUM (NSInteger, _UIBackdropMaskViewFlags) {
 @property (nonatomic, assign, setter=_setBackdropMaskViewFlags:) NSInteger _backdropMaskViewFlags;
 @end
 
+@interface UIWebSelectionView : UIView
+@end
+
 @interface UIWebSelectionAssistant : NSObject <UIGestureRecognizerDelegate>
 @end
 
+@protocol UISelectionInteractionAssistant
+- (void)showSelectionCommands;
+@end
+
 @interface UIWebSelectionAssistant ()
 - (BOOL)isSelectionGestureRecognizer:(UIGestureRecognizer *)gestureRecognizer;
 - (id)initWithView:(UIView *)view;
@@ -464,6 +471,7 @@ typedef NS_ENUM (NSInteger, _UIBackdropMaskViewFlags) {
 - (void)setGestureRecognizers;
 - (void)willStartScrollingOrZoomingPage;
 - (void)willStartScrollingOverflow;
+@property (nonatomic, retain) UIWebSelectionView *selectionView;
 @property (nonatomic, readonly) CGRect selectionFrame;
 @end
 
index e184560..ae7ede9 100644 (file)
@@ -125,6 +125,8 @@ void EditorState::PostLayoutData::encode(IPC::Encoder& encoder) const
     encoder << twoCharacterBeforeSelection;
     encoder << isReplaceAllowed;
     encoder << hasContent;
+    encoder << isStableStateUpdate;
+    encoder << insideFixedPosition;
 #endif
 #if PLATFORM(MAC)
     encoder << candidateRequestStartPosition;
@@ -170,6 +172,10 @@ bool EditorState::PostLayoutData::decode(IPC::Decoder& decoder, PostLayoutData&
         return false;
     if (!decoder.decode(result.hasContent))
         return false;
+    if (!decoder.decode(result.isStableStateUpdate))
+        return false;
+    if (!decoder.decode(result.insideFixedPosition))
+        return false;
 #endif
 #if PLATFORM(MAC)
     if (!decoder.decode(result.candidateRequestStartPosition))
index ddd0243..3dd039a 100644 (file)
@@ -99,6 +99,8 @@ struct EditorState {
         UChar32 twoCharacterBeforeSelection { 0 };
         bool isReplaceAllowed { false };
         bool hasContent { false };
+        bool isStableStateUpdate { false };
+        bool insideFixedPosition { false };
 #endif
 #if PLATFORM(MAC)
         uint64_t candidateRequestStartPosition { 0 };
index b5e9055..df722fe 100644 (file)
@@ -249,6 +249,9 @@ public:
     bool viewportMetaTagCameFromImageDocument() const { return m_viewportMetaTagCameFromImageDocument; }
     void setViewportMetaTagCameFromImageDocument(bool cameFromImageDocument) { m_viewportMetaTagCameFromImageDocument = cameFromImageDocument; }
 
+    bool isInStableState() const { return m_isInStableState; }
+    void setIsInStableState(bool isInStableState) { m_isInStableState = isInStableState; }
+
     bool allowsUserScaling() const { return m_allowsUserScaling; }
     void setAllowsUserScaling(bool allowsUserScaling) { m_allowsUserScaling = allowsUserScaling; }
 
@@ -295,6 +298,7 @@ private:
     bool m_allowsUserScaling { false };
     bool m_viewportMetaTagWidthWasExplicit { false };
     bool m_viewportMetaTagCameFromImageDocument { false };
+    bool m_isInStableState { false };
 };
 
 } // namespace WebKit
index 09f9488..c9679dd 100644 (file)
@@ -553,6 +553,8 @@ void RemoteLayerTreeTransaction::encode(IPC::Encoder& encoder) const
     encoder << m_viewportMetaTagWidthWasExplicit;
     encoder << m_viewportMetaTagCameFromImageDocument;
 
+    encoder << m_isInStableState;
+
     encoder << m_callbackIDs;
 }
 
@@ -660,6 +662,9 @@ bool RemoteLayerTreeTransaction::decode(IPC::Decoder& decoder, RemoteLayerTreeTr
     if (!decoder.decode(result.m_viewportMetaTagCameFromImageDocument))
         return false;
 
+    if (!decoder.decode(result.m_isInStableState))
+        return false;
+
     if (!decoder.decode(result.m_callbackIDs))
         return false;
 
@@ -873,6 +878,7 @@ CString RemoteLayerTreeTransaction::description() const
     ts.dumpProperty("viewportMetaTagWidth", m_viewportMetaTagWidth);
     ts.dumpProperty("viewportMetaTagWidthWasExplicit", m_viewportMetaTagWidthWasExplicit);
     ts.dumpProperty("viewportMetaTagCameFromImageDocument", m_viewportMetaTagCameFromImageDocument);
+    ts.dumpProperty("isInStableState", m_isInStableState);
     ts.dumpProperty("renderTreeSize", m_renderTreeSize);
 
     ts << "root-layer " << m_rootLayerID << ")";
index 16102dd..c11abb3 100644 (file)
@@ -272,6 +272,7 @@ WKWebView* fromWebPageProxy(WebKit::WebPageProxy& page)
     BOOL _hadDelayedUpdateVisibleContentRects;
 
     Vector<std::function<void ()>> _snapshotsDeferredDuringResize;
+    RetainPtr<NSMutableArray> _stableStatePresentationUpdateCallbacks;
 #endif
 #if PLATFORM(MAC)
     std::unique_ptr<WebKit::WebViewImpl> _impl;
@@ -1296,6 +1297,13 @@ static inline bool areEssentiallyEqualAsFloat(float a, float b)
     _viewportMetaTagWidthWasExplicit = layerTreeTransaction.viewportMetaTagWidthWasExplicit();
     _viewportMetaTagCameFromImageDocument = layerTreeTransaction.viewportMetaTagCameFromImageDocument();
     _initialScaleFactor = layerTreeTransaction.initialScaleFactor();
+    if (_page->inStableState() && layerTreeTransaction.isInStableState() && [_stableStatePresentationUpdateCallbacks count]) {
+        for (dispatch_block_t action in _stableStatePresentationUpdateCallbacks.get())
+            action();
+
+        [_stableStatePresentationUpdateCallbacks removeAllObjects];
+        _stableStatePresentationUpdateCallbacks = nil;
+    }
 
     BOOL needUpdateVisbleContentRects = _page->updateLayoutViewportParameters(layerTreeTransaction);
 
@@ -4736,9 +4744,20 @@ static WebCore::UserInterfaceLayoutDirection toUserInterfaceLayoutDirection(UISe
     // For subclasses to override.
 }
 
-- (NSArray<UIView *> *)_uiTextSelectionRectViews
+- (CGRect)_uiTextCaretRect
+{
+    // Force the selection view to update if needed.
+    [_contentView _updateChangedSelection];
+
+    return [[_contentView valueForKeyPath:@"interactionAssistant.selectionView.selection.caretRect"] CGRectValue];
+}
+
+- (NSArray<NSValue *> *)_uiTextSelectionRects
 {
-    return [_contentView valueForKeyPath:@"interactionAssistant.selectionView.rangeView.m_rectViews"];
+    // Force the selection view to update if needed.
+    [_contentView _updateChangedSelection];
+
+    return [_contentView _uiTextSelectionRects];
 }
 
 - (NSString *)_scrollingTreeAsText
@@ -4756,6 +4775,27 @@ static WebCore::UserInterfaceLayoutDirection toUserInterfaceLayoutDirection(UISe
     return nil;
 }
 
+- (void)_doAfterNextStablePresentationUpdate:(dispatch_block_t)updateBlock
+{
+    updateBlock = Block_copy(updateBlock);
+    if (_stableStatePresentationUpdateCallbacks)
+        [_stableStatePresentationUpdateCallbacks addObject:updateBlock];
+    else {
+        _stableStatePresentationUpdateCallbacks = adoptNS([[NSMutableArray alloc] initWithObjects:Block_copy(updateBlock), nil]);
+        [self _firePresentationUpdateForPendingStableStatePresentationCallbacks];
+    }
+    Block_release(updateBlock);
+}
+
+- (void)_firePresentationUpdateForPendingStableStatePresentationCallbacks
+{
+    RetainPtr<WKWebView> strongSelf = self;
+    [self _doAfterNextPresentationUpdate:^() {
+        if ([strongSelf->_stableStatePresentationUpdateCallbacks count])
+            [strongSelf _firePresentationUpdateForPendingStableStatePresentationCallbacks];
+    }];
+}
+
 #endif // PLATFORM(IOS)
 
 #if PLATFORM(MAC)
index fd9ef9f..9f61b03 100644 (file)
@@ -287,8 +287,10 @@ typedef NS_ENUM(NSInteger, _WKImmediateActionType) {
 
 - (void)_didShowForcePressPreview WK_API_AVAILABLE(ios(WK_IOS_TBA));
 - (void)_didDismissForcePressPreview WK_API_AVAILABLE(ios(WK_IOS_TBA));
+- (void)_doAfterNextStablePresentationUpdate:(dispatch_block_t)updateBlock WK_API_AVAILABLE(ios(WK_IOS_TBA));
 
-@property (nonatomic, readonly) NSArray<UIView *> *_uiTextSelectionRectViews WK_API_AVAILABLE(ios(WK_IOS_TBA));
+@property (nonatomic, readonly) NSArray<NSValue *> *_uiTextSelectionRects WK_API_AVAILABLE(ios(WK_IOS_TBA));
+@property (nonatomic, readonly) CGRect _uiTextCaretRect WK_API_AVAILABLE(ios(WK_IOS_TBA));
 
 @property (nonatomic, readonly) NSString *_scrollingTreeAsText WK_API_AVAILABLE(ios(WK_IOS_TBA));
 
index ff02f4a..5273790 100644 (file)
@@ -467,6 +467,7 @@ public:
     double displayedContentScale() const { return m_lastVisibleContentRectUpdate.scale(); }
     const WebCore::FloatRect& exposedContentRect() const { return m_lastVisibleContentRectUpdate.exposedContentRect(); }
     const WebCore::FloatRect& unobscuredContentRect() const { return m_lastVisibleContentRectUpdate.unobscuredContentRect(); }
+    bool inStableState() const { return m_lastVisibleContentRectUpdate.inStableState(); }
     // When visual viewports are enabled, this is the layout viewport rect.
     const WebCore::FloatRect& customFixedPositionRect() const { return m_lastVisibleContentRectUpdate.customFixedPositionRect(); }
 
index d0bebbd..4bfbd4c 100644 (file)
@@ -362,6 +362,16 @@ private:
     [_fixedClippingView setBounds:clippingBounds];
 }
 
+- (void)_didExitStableState
+{
+    _needsDeferredEndScrollingSelectionUpdate = self.shouldHideSelectionWhenScrolling;
+    if (!_needsDeferredEndScrollingSelectionUpdate)
+        return;
+
+    [_textSelectionAssistant deactivateSelection];
+    [[_webSelectionAssistant selectionView] setHidden:YES];
+}
+
 - (void)didUpdateVisibleRect:(CGRect)visibleContentRect
     unobscuredRect:(CGRect)unobscuredContentRect
     unobscuredRectInScrollViewCoordinates:(CGRect)unobscuredRectInScrollViewCoordinates
@@ -405,6 +415,7 @@ private:
 
     LOG_WITH_STREAM(VisibleRects, stream << "-[WKContentView didUpdateVisibleRect]" << visibleContentRectUpdateInfo.dump());
 
+    bool wasStableState = _page->inStableState();
     _page->updateVisibleContentRects(visibleContentRectUpdateInfo);
 
     _sizeChangedSinceLastVisibleContentRectUpdate = NO;
@@ -415,6 +426,9 @@ private:
     drawingArea->updateDebugIndicator();
     
     [self updateFixedClippingView:fixedPositionRect];
+
+    if (wasStableState && !isStableState)
+        [self _didExitStableState];
 }
 
 - (void)didFinishScrolling
index 250621d..b400340 100644 (file)
@@ -178,6 +178,7 @@ struct WKAutoCorrectionData {
     BOOL _showDebugTapHighlightsForFastClicking;
 
     BOOL _resigningFirstResponder;
+    BOOL _needsDeferredEndScrollingSelectionUpdate;
 }
 
 @end
@@ -186,6 +187,7 @@ struct WKAutoCorrectionData {
 
 @property (nonatomic, readonly) CGPoint lastInteractionLocation;
 @property (nonatomic, readonly) BOOL isEditable;
+@property (nonatomic, readonly) BOOL shouldHideSelectionWhenScrolling;
 @property (nonatomic, readonly) const WebKit::InteractionInformationAtPosition& positionInformation;
 @property (nonatomic, readonly) const WebKit::WKAutoCorrectionData& autocorrectionData;
 @property (nonatomic, readonly) const WebKit::AssistedNodeInformation& assistedNodeInformation;
@@ -228,6 +230,7 @@ struct WKAutoCorrectionData {
 - (void)_becomeFirstResponderWithSelectionMovingForward:(BOOL)selectingForward completionHandler:(void (^)(BOOL didBecomeFirstResponder))completionHandler;
 - (void)_setDoubleTapGesturesEnabled:(BOOL)enabled;
 - (NSArray *)_dataDetectionResults;
+- (NSArray<NSValue *> *)_uiTextSelectionRects;
 @end
 
 @interface WKContentView (WKTesting)
index a5cc97f..f64850c 100644 (file)
@@ -589,6 +589,7 @@ static UIWebSelectionMode toUIWebSelectionMode(WKSelectionGranularity granularit
     _potentialTapInProgress = NO;
     _isDoubleTapPending = NO;
     _showDebugTapHighlightsForFastClicking = [[NSUserDefaults standardUserDefaults] boolForKey:@"WebKitShowFastClickDebugTapHighlights"];
+    _needsDeferredEndScrollingSelectionUpdate = NO;
 }
 
 - (void)cleanupInteraction
@@ -599,6 +600,7 @@ static UIWebSelectionMode toUIWebSelectionMode(WKSelectionGranularity granularit
     _smartMagnificationController = nil;
     _didAccessoryTabInitiateFocus = NO;
     _isExpectingFastSingleTapCommit = NO;
+    _needsDeferredEndScrollingSelectionUpdate = NO;
     [_formInputSession invalidate];
     _formInputSession = nil;
     [_highlightView removeFromSuperview];
@@ -765,6 +767,15 @@ static UIWebSelectionMode toUIWebSelectionMode(WKSelectionGranularity granularit
     return _lastInteractionLocation;
 }
 
+- (BOOL)shouldHideSelectionWhenScrolling
+{
+    if (_isEditable)
+        return _assistedNodeInformation.insideFixedPosition;
+
+    auto& editorState = _page->editorState();
+    return !editorState.isMissingPostLayoutData && editorState.postLayoutData().insideFixedPosition;
+}
+
 - (BOOL)isEditable
 {
     return _isEditable;
@@ -1261,6 +1272,21 @@ static inline bool isSamePair(UIGestureRecognizer *a, UIGestureRecognizer *b, UI
 }
 #endif
 
+- (NSArray<NSValue *> *)_uiTextSelectionRects
+{
+    NSMutableArray *textSelectionRects = [NSMutableArray array];
+
+    if (_textSelectionAssistant) {
+        for (WKTextSelectionRect *selectionRect in [_textSelectionAssistant valueForKeyPath:@"selectionView.selection.selectionRects"])
+            [textSelectionRects addObject:[NSValue valueWithCGRect:selectionRect.webRect.rect]];
+    } else if (_webSelectionAssistant) {
+        for (WebSelectionRect *selectionRect in [_webSelectionAssistant valueForKeyPath:@"selectionView.selectionRects"])
+            [textSelectionRects addObject:[NSValue valueWithCGRect:selectionRect.rect]];
+    }
+
+    return textSelectionRects;
+}
+
 - (BOOL)gestureRecognizerShouldBegin:(UIGestureRecognizer *)gestureRecognizer
 {
     CGPoint point = [gestureRecognizer locationInView:self];
@@ -1642,8 +1668,10 @@ static void cancelPotentialTapIfNecessary(WKContentView* contentView)
 
 - (void)_didEndScrollingOrZooming
 {
-    [_webSelectionAssistant didEndScrollingOrZoomingPage];
-    [_textSelectionAssistant didEndScrollingOverflow];
+    if (!_needsDeferredEndScrollingSelectionUpdate) {
+        [_webSelectionAssistant didEndScrollingOrZoomingPage];
+        [_textSelectionAssistant didEndScrollingOverflow];
+    }
     _page->setIsScrollingOrZooming(false);
 }
 
@@ -3735,25 +3763,37 @@ static bool isAssistableInputType(InputType type)
         return;
 
     WKSelectionDrawingInfo selectionDrawingInfo(_page->editorState());
-    if (!force && selectionDrawingInfo == _lastSelectionDrawingInfo)
-        return;
+    if (force || selectionDrawingInfo != _lastSelectionDrawingInfo) {
+        LOG_WITH_STREAM(Selection, stream << "_updateChangedSelection " << selectionDrawingInfo);
 
-    LOG_WITH_STREAM(Selection, stream << "_updateChangedSelection " << selectionDrawingInfo);
+        _lastSelectionDrawingInfo = selectionDrawingInfo;
 
-    _lastSelectionDrawingInfo = selectionDrawingInfo;
+        // FIXME: We need to figure out what to do if the selection is changed by Javascript.
+        if (_textSelectionAssistant) {
+            _markedText = (_page->editorState().hasComposition) ? _page->editorState().markedText : String();
+            if (!_showingTextStyleOptions)
+                [_textSelectionAssistant selectionChanged];
+        } else if (!_page->editorState().isContentEditable)
+            [_webSelectionAssistant selectionChanged];
+
+        _selectionNeedsUpdate = NO;
+        if (_shouldRestoreSelection) {
+            [_webSelectionAssistant didEndScrollingOverflow];
+            [_textSelectionAssistant didEndScrollingOverflow];
+            _shouldRestoreSelection = NO;
+        }
+    }
 
-    // FIXME: We need to figure out what to do if the selection is changed by Javascript.
-    if (_textSelectionAssistant) {
-        _markedText = (_page->editorState().hasComposition) ? _page->editorState().markedText : String();
-        if (!_showingTextStyleOptions)
-            [_textSelectionAssistant selectionChanged];
-    } else if (!_page->editorState().isContentEditable)
-        [_webSelectionAssistant selectionChanged];
-    _selectionNeedsUpdate = NO;
-    if (_shouldRestoreSelection) {
-        [_webSelectionAssistant didEndScrollingOverflow];
+    auto& state = _page->editorState();
+    if (!state.isMissingPostLayoutData && state.postLayoutData().isStableStateUpdate && _needsDeferredEndScrollingSelectionUpdate && _page->inStableState()) {
+        [[self selectionInteractionAssistant] showSelectionCommands];
+        [_webSelectionAssistant didEndScrollingOrZoomingPage];
+        [[_webSelectionAssistant selectionView] setHidden:NO];
+
+        [_textSelectionAssistant activateSelection];
         [_textSelectionAssistant didEndScrollingOverflow];
-        _shouldRestoreSelection = NO;
+
+        _needsDeferredEndScrollingSelectionUpdate = NO;
     }
 }
 
index 9ef9ef2..70a26f8 100644 (file)
@@ -3273,6 +3273,7 @@ void WebPage::willCommitLayerTree(RemoteLayerTreeTransaction& layerTransaction)
     layerTransaction.setViewportMetaTagWidth(m_viewportConfiguration.viewportArguments().width);
     layerTransaction.setViewportMetaTagWidthWasExplicit(m_viewportConfiguration.viewportArguments().widthWasExplicit);
     layerTransaction.setViewportMetaTagCameFromImageDocument(m_viewportConfiguration.viewportArguments().type == ViewportArguments::ImageDocument);
+    layerTransaction.setIsInStableState(m_isInStableState);
     layerTransaction.setAllowsUserScaling(allowsUserScaling());
 #endif
 
index e0c8e3d..18c029d 100644 (file)
@@ -166,8 +166,12 @@ void WebPage::platformEditorState(Frame& frame, EditorState& result, IncludePost
     auto& postLayoutData = result.postLayoutData();
     FrameView* view = frame.view();
     const VisibleSelection& selection = frame.selection().selection();
+    postLayoutData.isStableStateUpdate = m_isInStableState;
+    bool startNodeIsInsideFixedPosition = false;
+    bool endNodeIsInsideFixedPosition = false;
     if (selection.isCaret()) {
-        postLayoutData.caretRectAtStart = view->contentsToRootView(frame.selection().absoluteCaretBounds());
+        postLayoutData.caretRectAtStart = view->contentsToRootView(frame.selection().absoluteCaretBounds(&startNodeIsInsideFixedPosition));
+        endNodeIsInsideFixedPosition = startNodeIsInsideFixedPosition;
         postLayoutData.caretRectAtEnd = postLayoutData.caretRectAtStart;
         // FIXME: The following check should take into account writing direction.
         postLayoutData.isReplaceAllowed = result.isContentEditable && atBoundaryOfGranularity(selection.start(), WordGranularity, DirectionForward);
@@ -178,8 +182,8 @@ void WebPage::platformEditorState(Frame& frame, EditorState& result, IncludePost
             postLayoutData.hasContent = root && root->hasChildNodes() && !isEndOfEditableOrNonEditableContent(firstPositionInNode(root));
         }
     } else if (selection.isRange()) {
-        postLayoutData.caretRectAtStart = view->contentsToRootView(VisiblePosition(selection.start()).absoluteCaretBounds());
-        postLayoutData.caretRectAtEnd = view->contentsToRootView(VisiblePosition(selection.end()).absoluteCaretBounds());
+        postLayoutData.caretRectAtStart = view->contentsToRootView(VisiblePosition(selection.start()).absoluteCaretBounds(&startNodeIsInsideFixedPosition));
+        postLayoutData.caretRectAtEnd = view->contentsToRootView(VisiblePosition(selection.end()).absoluteCaretBounds(&endNodeIsInsideFixedPosition));
         RefPtr<Range> selectedRange = selection.toNormalizedRange();
         String selectedText;
         if (selectedRange) {
@@ -194,6 +198,7 @@ void WebPage::platformEditorState(Frame& frame, EditorState& result, IncludePost
         // FIXME: We should disallow replace when the string contains only CJ characters.
         postLayoutData.isReplaceAllowed = result.isContentEditable && !result.isInPasswordField && !selectedText.containsOnlyWhitespace();
     }
+    postLayoutData.insideFixedPosition = startNodeIsInsideFixedPosition || endNodeIsInsideFixedPosition;
     if (!selection.isNone()) {
         if (m_assistedNode && m_assistedNode->renderer())
             postLayoutData.selectionClipRect = view->contentsToRootView(m_assistedNode->renderer()->absoluteBoundingBoxRect());
@@ -3081,9 +3086,14 @@ void WebPage::updateVisibleContentRects(const VisibleContentRectUpdateInfo& visi
     frameView.setScrollVelocity(horizontalVelocity, verticalVelocity, scaleChangeRate, visibleContentRectUpdateInfo.timestamp());
 
     if (m_isInStableState) {
-        if (frameView.frame().settings().visualViewportEnabled())
+        if (frameView.frame().settings().visualViewportEnabled()) {
             frameView.setLayoutViewportOverrideRect(LayoutRect(visibleContentRectUpdateInfo.customFixedPositionRect()));
-        else
+            const auto& state = editorState();
+            if (!state.isMissingPostLayoutData && state.postLayoutData().insideFixedPosition) {
+                frameView.frame().selection().setCaretRectNeedsUpdate();
+                send(Messages::WebPageProxy::EditorStateChanged(state));
+            }
+        } else
             frameView.setCustomFixedPositionLayoutRect(enclosingIntRect(visibleContentRectUpdateInfo.customFixedPositionRect()));
     }
 
index da372d6..90a34ae 100644 (file)
@@ -1,3 +1,37 @@
+2016-12-16  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        Visual viewports: carets and selection UI are incorrectly positioned when editing fixed elements
+        https://bugs.webkit.org/show_bug.cgi?id=165767
+        <rdar://problem/29602382>
+
+        Reviewed by Simon Fraser.
+
+        Introduces two new UIScriptController methods: doAfterWebPageIsInStableState and textSelectionCaretRect. See
+        WebKit2 ChangeLog for more details.
+
+        * DumpRenderTree/ios/UIScriptControllerIOS.mm:
+        (WTR::UIScriptController::doAfterNextStablePresentationUpdate):
+        (WTR::UIScriptController::textSelectionCaretRect):
+        * DumpRenderTree/mac/UIScriptControllerMac.mm:
+        (WTR::UIScriptController::doAfterNextStablePresentationUpdate):
+        * TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl:
+        * TestRunnerShared/UIScriptContext/UIScriptController.cpp:
+        (WTR::UIScriptController::doAfterNextStablePresentationUpdate):
+        (WTR::UIScriptController::textSelectionCaretRect):
+        * TestRunnerShared/UIScriptContext/UIScriptController.h:
+        * WebKitTestRunner/cocoa/TestRunnerWKWebView.mm:
+        (-[TestRunnerWKWebView _setStableStateOverride:]):
+
+        Force the WKWebView to update its visible content rects when changing the stable state override.
+
+        * WebKitTestRunner/ios/UIScriptControllerIOS.mm:
+        (WTR::toNSDictionary):
+        (WTR::UIScriptController::doAfterNextStablePresentationUpdate):
+        (WTR::UIScriptController::selectionRangeViewRects):
+        (WTR::UIScriptController::textSelectionCaretRect):
+        * WebKitTestRunner/mac/UIScriptControllerMac.mm:
+        (WTR::UIScriptController::doAfterNextStablePresentationUpdate):
+
 2016-12-15  Brent Fulgham  <bfulgham@apple.com>
 
         Arguments called in wrong order
index ff9d57a..2f2506b 100644 (file)
@@ -54,6 +54,11 @@ void UIScriptController::doAfterPresentationUpdate(JSValueRef callback)
     return doAsyncTask(callback);
 }
 
+void UIScriptController::doAfterNextStablePresentationUpdate(JSValueRef callback)
+{
+    doAsyncTask(callback);
+}
+
 void UIScriptController::zoomToScale(double scale, JSValueRef callback)
 {
     RefPtr<UIScriptController> protectedThis(this);
@@ -261,6 +266,11 @@ JSObjectRef UIScriptController::selectionRangeViewRects() const
     return nullptr;
 }
 
+JSObjectRef UIScriptController::textSelectionCaretRect() const
+{
+    return nullptr;
+}
+
 void UIScriptController::removeAllDynamicDictionaries()
 {
 }
index acff4a8..4887c7d 100644 (file)
@@ -54,6 +54,11 @@ void UIScriptController::doAfterPresentationUpdate(JSValueRef callback)
     return doAsyncTask(callback);
 }
 
+void UIScriptController::doAfterNextStablePresentationUpdate(JSValueRef callback)
+{
+    doAsyncTask(callback);
+}
+
 void UIScriptController::insertText(JSStringRef, int, int)
 {
 }
index a467c22..8ad3b9a 100644 (file)
@@ -27,6 +27,7 @@ interface UIScriptController {
 
     void doAsyncTask(object callback); // Used to test the harness.
     void doAfterPresentationUpdate(object callback); // Call the callback after sending a message to the WebProcess and receiving a subsequent update.
+    void doAfterNextStablePresentationUpdate(object callback);
 
     void simulateAccessibilitySettingsChangeNotification(object callback);
 
@@ -204,6 +205,7 @@ interface UIScriptController {
     readonly attribute object contentVisibleRect; // Returned object has 'left', 'top', 'width', 'height' properties.
 
     readonly attribute object selectionRangeViewRects; // An array of objects with 'left', 'top', 'width', and 'height' properties.
+    readonly attribute object textSelectionCaretRect; // An object with 'left', 'top', 'width', 'height' properties.
 
     void insertText(DOMString text, long location, long length);
     void removeAllDynamicDictionaries();
index 1691145..46603cb 100644 (file)
@@ -64,6 +64,10 @@ void simulateAccessibilitySettingsChangeNotification(JSValueRef)
 void UIScriptController::doAfterPresentationUpdate(JSValueRef)
 {
 }
+
+void UIScriptController::doAfterNextStablePresentationUpdate(JSValueRef)
+{
+}
 #endif
 
 void UIScriptController::setDidStartFormControlInteractionCallback(JSValueRef callback)
@@ -307,6 +311,11 @@ JSObjectRef UIScriptController::selectionRangeViewRects() const
     return nullptr;
 }
 
+JSObjectRef UIScriptController::textSelectionCaretRect() const
+{
+    return nullptr;
+}
+
 void UIScriptController::removeAllDynamicDictionaries()
 {
 }
index 0493269..2993d16 100644 (file)
@@ -52,6 +52,7 @@ public:
     
     void doAsyncTask(JSValueRef callback);
     void doAfterPresentationUpdate(JSValueRef callback);
+    void doAfterNextStablePresentationUpdate(JSValueRef callback);
 
     void zoomToScale(double scale, JSValueRef callback);
 
@@ -128,6 +129,7 @@ public:
     JSObjectRef contentVisibleRect() const;
     
     JSObjectRef selectionRangeViewRects() const;
+    JSObjectRef textSelectionCaretRect() const;
 
     void insertText(JSStringRef, int location, int length);
     void removeAllDynamicDictionaries();
index 1b09ef8..5d6d308 100644 (file)
@@ -37,6 +37,7 @@
 - (void)scrollViewWillBeginZooming:(UIScrollView *)scrollView withView:(UIView *)view;
 - (void)scrollViewDidEndZooming:(UIScrollView *)scrollView withView:(UIView *)view atScale:(CGFloat)scale;
 - (void)_didFinishScrolling;
+- (void)_updateVisibleContentRects;
 
 @end
 #endif
 - (void)_setStableStateOverride:(NSNumber *)overrideBoolean
 {
     m_stableStateOverride = overrideBoolean;
+    [self _updateVisibleContentRects];
 }
 
 #endif
index 794e91a..bf077e8 100644 (file)
 
 namespace WTR {
 
+static NSDictionary *toNSDictionary(CGRect rect)
+{
+    return @{
+        @"left": @(rect.origin.x),
+        @"top": @(rect.origin.y),
+        @"width": @(rect.size.width),
+        @"height": @(rect.size.height)
+    };
+}
+
 void UIScriptController::doAsyncTask(JSValueRef callback)
 {
     unsigned callbackID = m_context->prepareForAsyncTask(callback, CallbackTypeNonPersistent);
@@ -67,6 +77,17 @@ void UIScriptController::doAfterPresentationUpdate(JSValueRef callback)
     }];
 }
 
+void UIScriptController::doAfterNextStablePresentationUpdate(JSValueRef callback)
+{
+    TestRunnerWKWebView *webView = TestController::singleton().mainWebView()->platformView();
+
+    unsigned callbackID = m_context->prepareForAsyncTask(callback, CallbackTypeNonPersistent);
+    [webView _doAfterNextStablePresentationUpdate:^() {
+        if (m_context)
+            m_context->asyncTaskComplete(callbackID);
+    }];
+}
+
 void UIScriptController::zoomToScale(double scale, JSValueRef callback)
 {
     TestRunnerWKWebView *webView = TestController::singleton().mainWebView()->platformView();
@@ -462,21 +483,18 @@ JSObjectRef UIScriptController::contentVisibleRect() const
 JSObjectRef UIScriptController::selectionRangeViewRects() const
 {
     NSMutableArray *selectionRects = [[NSMutableArray alloc] init];
-    for (UIView *rectView in TestController::singleton().mainWebView()->platformView()._uiTextSelectionRectViews) {
-        if (rectView.hidden)
-            continue;
+    NSArray *rects = TestController::singleton().mainWebView()->platformView()._uiTextSelectionRects;
+    for (NSValue *rect in rects)
+        [selectionRects addObject:toNSDictionary([rect CGRectValue])];
 
-        CGRect frame = rectView.frame;
-        [selectionRects addObject:@{
-            @"left": @(frame.origin.x),
-            @"top": @(frame.origin.y),
-            @"width": @(frame.size.width),
-            @"height": @(frame.size.height),
-        }];
-    }
     return JSValueToObject(m_context->jsContext(), [JSValue valueWithObject:selectionRects inContext:[JSContext contextWithJSGlobalContextRef:m_context->jsContext()]].JSValueRef, nullptr);
 }
 
+JSObjectRef UIScriptController::textSelectionCaretRect() const
+{
+    return JSValueToObject(m_context->jsContext(), [JSValue valueWithObject:toNSDictionary(TestController::singleton().mainWebView()->platformView()._uiTextCaretRect) inContext:[JSContext contextWithJSGlobalContextRef:m_context->jsContext()]].JSValueRef, nullptr);
+}
+
 void UIScriptController::removeAllDynamicDictionaries()
 {
     [UIKeyboard removeAllDynamicDictionaries];
index d503b4a..69840d2 100644 (file)
@@ -61,6 +61,11 @@ void UIScriptController::doAfterPresentationUpdate(JSValueRef callback)
     return doAsyncTask(callback);
 }
 
+void UIScriptController::doAfterNextStablePresentationUpdate(JSValueRef callback)
+{
+    doAsyncTask(callback);
+}
+
 void UIScriptController::insertText(JSStringRef text, int location, int length)
 {
 #if WK_API_ENABLED