[iOS] Add a fast path that avoids hit testing during context menu interactions on...
authoraestes@apple.com <aestes@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 8 Oct 2019 23:23:01 +0000 (23:23 +0000)
committeraestes@apple.com <aestes@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 8 Oct 2019 23:23:01 +0000 (23:23 +0000)
https://bugs.webkit.org/show_bug.cgi?id=202510
<rdar://problem/55939313>

Reviewed by Tim Horton.

When computing position information for an element, we can store sufficient context
information in InteractionInformationAtPosition such that the same element could be
retrieved again without hit testing. The existing TextInputContext can already store an
element's page identifier, document identifier, and element identifier, which is sufficient
context for such a task.

This patch renames TextInputContext to ElementContext and uses it when populating new
InteractionInformationAtPosition structs in elementPositionInformation(). When a context
menu interaction occurs, WebPage::startInteractionWithElementContextOrPosition() uses this
context to look up the interaction element by its identifier triplet, falling back to hit
testing if this lookup fails.

* Scripts/webkit/messages.py:
* Shared/DocumentEditingContext.h:
* Shared/DocumentEditingContext.mm:
(IPC::ArgumentCoder<WebKit::DocumentEditingContextRequest>::decode):
* Shared/ElementContext.cpp: Renamed from Source/WebKit/Shared/TextInputContext.cpp.
(IPC::ArgumentCoder<WebKit::ElementContext>::encode):
(IPC::ArgumentCoder<WebKit::ElementContext>::decode):
* Shared/ElementContext.h: Renamed from Source/WebKit/Shared/TextInputContext.h.
(WebKit::operator==):
* Shared/ios/InteractionInformationAtPosition.h:
* Shared/ios/InteractionInformationAtPosition.mm:
(WebKit::InteractionInformationAtPosition::encode const):
(WebKit::InteractionInformationAtPosition::decode):
* Sources.txt:
* UIProcess/API/Cocoa/WKWebView.mm:
(-[WKWebView _requestTextInputContextsInRect:completionHandler:]):
* UIProcess/API/Cocoa/_WKTextInputContext.mm:
(-[_WKTextInputContext _initWithTextInputContext:]):
(-[_WKTextInputContext _textInputContext]):
* UIProcess/API/Cocoa/_WKTextInputContextInternal.h:
* UIProcess/WebPageProxy.cpp:
(WebKit::WebPageProxy::textInputContextsInRect):
(WebKit::WebPageProxy::focusTextInputContext):
* UIProcess/WebPageProxy.h:
* UIProcess/ios/WKContentViewInteraction.mm:
(-[WKContentView actionSheetAssistant:willStartInteractionWithElement:]):
(-[WKContentView assignLegacyDataForContextMenuInteraction]):
(-[WKContentView continueContextMenuInteraction:]):
(-[WKContentView continueContextMenuInteractionWithDataDetectors:]):
(-[WKContentView _presentedViewControllerForPreviewItemController:]):
* UIProcess/ios/WebPageProxyIOS.mm:
(WebKit::WebPageProxy::startInteractionWithPositionInformation):
(WebKit::WebPageProxy::startInteractionWithElementAtPosition): Deleted.
* WebKit.xcodeproj/project.pbxproj:
* WebProcess/WebPage/WebPage.cpp:
(WebKit::WebPage::textInputContextsInRect):
(WebKit::WebPage::focusTextInputContext):
(WebKit::WebPage::elementForContext const):
(WebKit::WebPage::contextForElement const):
(WebKit::WebPage::elementForTextInputContext): Deleted.
* WebProcess/WebPage/WebPage.h:
* WebProcess/WebPage/WebPage.messages.in:
* WebProcess/WebPage/ios/WebPageIOS.mm:
(WebKit::elementPositionInformation):
(WebKit::WebPage::startInteractionWithElementContextOrPosition):
(WebKit::WebPage::requestDocumentEditingContext):
(WebKit::WebPage::startInteractionWithElementAtPosition): Deleted.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@250876 268f45cc-cd09-0410-ab3c-d52691b4dbfc

21 files changed:
Source/WebKit/ChangeLog
Source/WebKit/Scripts/webkit/messages.py
Source/WebKit/Shared/DocumentEditingContext.h
Source/WebKit/Shared/DocumentEditingContext.mm
Source/WebKit/Shared/ElementContext.cpp [moved from Source/WebKit/Shared/TextInputContext.cpp with 86% similarity]
Source/WebKit/Shared/ElementContext.h [moved from Source/WebKit/Shared/TextInputContext.h with 86% similarity]
Source/WebKit/Shared/ios/InteractionInformationAtPosition.h
Source/WebKit/Shared/ios/InteractionInformationAtPosition.mm
Source/WebKit/Sources.txt
Source/WebKit/UIProcess/API/Cocoa/WKWebView.mm
Source/WebKit/UIProcess/API/Cocoa/_WKTextInputContext.mm
Source/WebKit/UIProcess/API/Cocoa/_WKTextInputContextInternal.h
Source/WebKit/UIProcess/WebPageProxy.cpp
Source/WebKit/UIProcess/WebPageProxy.h
Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm
Source/WebKit/UIProcess/ios/WebPageProxyIOS.mm
Source/WebKit/WebKit.xcodeproj/project.pbxproj
Source/WebKit/WebProcess/WebPage/WebPage.cpp
Source/WebKit/WebProcess/WebPage/WebPage.h
Source/WebKit/WebProcess/WebPage/WebPage.messages.in
Source/WebKit/WebProcess/WebPage/ios/WebPageIOS.mm

index d51c62e..755484f 100644 (file)
@@ -1,3 +1,71 @@
+2019-10-08  Andy Estes  <aestes@apple.com>
+
+        [iOS] Add a fast path that avoids hit testing during context menu interactions on elements
+        https://bugs.webkit.org/show_bug.cgi?id=202510
+        <rdar://problem/55939313>
+
+        Reviewed by Tim Horton.
+
+        When computing position information for an element, we can store sufficient context
+        information in InteractionInformationAtPosition such that the same element could be
+        retrieved again without hit testing. The existing TextInputContext can already store an
+        element's page identifier, document identifier, and element identifier, which is sufficient
+        context for such a task.
+
+        This patch renames TextInputContext to ElementContext and uses it when populating new
+        InteractionInformationAtPosition structs in elementPositionInformation(). When a context
+        menu interaction occurs, WebPage::startInteractionWithElementContextOrPosition() uses this
+        context to look up the interaction element by its identifier triplet, falling back to hit
+        testing if this lookup fails.
+
+        * Scripts/webkit/messages.py:
+        * Shared/DocumentEditingContext.h:
+        * Shared/DocumentEditingContext.mm:
+        (IPC::ArgumentCoder<WebKit::DocumentEditingContextRequest>::decode):
+        * Shared/ElementContext.cpp: Renamed from Source/WebKit/Shared/TextInputContext.cpp.
+        (IPC::ArgumentCoder<WebKit::ElementContext>::encode):
+        (IPC::ArgumentCoder<WebKit::ElementContext>::decode):
+        * Shared/ElementContext.h: Renamed from Source/WebKit/Shared/TextInputContext.h.
+        (WebKit::operator==):
+        * Shared/ios/InteractionInformationAtPosition.h:
+        * Shared/ios/InteractionInformationAtPosition.mm:
+        (WebKit::InteractionInformationAtPosition::encode const):
+        (WebKit::InteractionInformationAtPosition::decode):
+        * Sources.txt:
+        * UIProcess/API/Cocoa/WKWebView.mm:
+        (-[WKWebView _requestTextInputContextsInRect:completionHandler:]):
+        * UIProcess/API/Cocoa/_WKTextInputContext.mm:
+        (-[_WKTextInputContext _initWithTextInputContext:]):
+        (-[_WKTextInputContext _textInputContext]):
+        * UIProcess/API/Cocoa/_WKTextInputContextInternal.h:
+        * UIProcess/WebPageProxy.cpp:
+        (WebKit::WebPageProxy::textInputContextsInRect):
+        (WebKit::WebPageProxy::focusTextInputContext):
+        * UIProcess/WebPageProxy.h:
+        * UIProcess/ios/WKContentViewInteraction.mm:
+        (-[WKContentView actionSheetAssistant:willStartInteractionWithElement:]):
+        (-[WKContentView assignLegacyDataForContextMenuInteraction]):
+        (-[WKContentView continueContextMenuInteraction:]):
+        (-[WKContentView continueContextMenuInteractionWithDataDetectors:]):
+        (-[WKContentView _presentedViewControllerForPreviewItemController:]):
+        * UIProcess/ios/WebPageProxyIOS.mm:
+        (WebKit::WebPageProxy::startInteractionWithPositionInformation):
+        (WebKit::WebPageProxy::startInteractionWithElementAtPosition): Deleted.
+        * WebKit.xcodeproj/project.pbxproj:
+        * WebProcess/WebPage/WebPage.cpp:
+        (WebKit::WebPage::textInputContextsInRect):
+        (WebKit::WebPage::focusTextInputContext):
+        (WebKit::WebPage::elementForContext const):
+        (WebKit::WebPage::contextForElement const):
+        (WebKit::WebPage::elementForTextInputContext): Deleted.
+        * WebProcess/WebPage/WebPage.h:
+        * WebProcess/WebPage/WebPage.messages.in:
+        * WebProcess/WebPage/ios/WebPageIOS.mm:
+        (WebKit::elementPositionInformation):
+        (WebKit::WebPage::startInteractionWithElementContextOrPosition):
+        (WebKit::WebPage::requestDocumentEditingContext):
+        (WebKit::WebPage::startInteractionWithElementAtPosition): Deleted.
+
 2019-10-08  Adrian Perez de Castro  <aperez@igalia.com>
 
         Unreviewed. Update OptionsWPE.cmake and NEWS for the 2.27.1 release
index a78da31..df3bb4b 100644 (file)
@@ -486,7 +486,7 @@ def headers_for_type(type):
         'WebKit::WebMouseEvent': ['"WebEvent.h"'],
         'WebKit::WebTouchEvent': ['"WebEvent.h"'],
         'WebKit::WebWheelEvent': ['"WebEvent.h"'],
-        'struct WebKit::TextInputContext': ['"TextInputContext.h"'],
+        'struct WebKit::ElementContext': ['"ElementContext.h"'],
         'struct WebKit::WebUserScriptData': ['"WebUserContentControllerDataTypes.h"'],
         'struct WebKit::WebUserStyleSheetData': ['"WebUserContentControllerDataTypes.h"'],
         'struct WebKit::WebScriptMessageHandlerData': ['"WebUserContentControllerDataTypes.h"'],
index 1c30f64..861c891 100644 (file)
@@ -29,7 +29,7 @@
 
 #include "ArgumentCoders.h"
 #include "AttributedString.h"
-#include "TextInputContext.h"
+#include "ElementContext.h"
 #include <WebCore/FloatRect.h>
 #include <WebCore/TextGranularity.h>
 #include <wtf/OptionSet.h>
@@ -56,7 +56,7 @@ struct DocumentEditingContextRequest {
 
     WebCore::FloatRect rect;
 
-    Optional<WebKit::TextInputContext> textInputContext;
+    Optional<WebKit::ElementContext> textInputContext;
 };
 
 struct DocumentEditingContext {
index 316c013..482e8c7 100644 (file)
@@ -28,7 +28,7 @@
 
 #if PLATFORM(IOS_FAMILY)
 
-#include "TextInputContext.h"
+#include "ElementContext.h"
 #include "UIKitSPI.h"
 #include "WebCoreArgumentCoders.h"
 
@@ -198,7 +198,7 @@ Optional<WebKit::DocumentEditingContextRequest> ArgumentCoder<WebKit::DocumentEd
     if (!decoder.decode(request.rect))
         return WTF::nullopt;
 
-    Optional<Optional<WebKit::TextInputContext>> optionalTextInputContext;
+    Optional<Optional<WebKit::ElementContext>> optionalTextInputContext;
     decoder >> optionalTextInputContext;
     if (!optionalTextInputContext)
         return WTF::nullopt;
similarity index 86%
rename from Source/WebKit/Shared/TextInputContext.cpp
rename to Source/WebKit/Shared/ElementContext.cpp
index c30fafa..3f74e7f 100644 (file)
  */
 
 #include "config.h"
-#include "TextInputContext.h"
+#include "ElementContext.h"
 
 #include "WebCoreArgumentCoders.h"
 #include <WebCore/Element.h>
 
 namespace WebKit {
 
-TextInputContext::~TextInputContext() = default;
+ElementContext::~ElementContext() = default;
 
 }
 
 namespace IPC {
 
-void ArgumentCoder<WebKit::TextInputContext>::encode(Encoder& encoder, const WebKit::TextInputContext& context)
+void ArgumentCoder<WebKit::ElementContext>::encode(Encoder& encoder, const WebKit::ElementContext& context)
 {
     encoder << context.boundingRect;
     encoder << context.webPageIdentifier;
@@ -45,9 +45,9 @@ void ArgumentCoder<WebKit::TextInputContext>::encode(Encoder& encoder, const Web
     encoder << context.elementIdentifier;
 }
 
-Optional<WebKit::TextInputContext> ArgumentCoder<WebKit::TextInputContext>::decode(Decoder& decoder)
+Optional<WebKit::ElementContext> ArgumentCoder<WebKit::ElementContext>::decode(Decoder& decoder)
 {
-    WebKit::TextInputContext context;
+    WebKit::ElementContext context;
     if (!decoder.decode(context.boundingRect))
         return WTF::nullopt;
     if (!decoder.decode(context.webPageIdentifier))
similarity index 86%
rename from Source/WebKit/Shared/TextInputContext.h
rename to Source/WebKit/Shared/ElementContext.h
index 54d4277..215cc42 100644 (file)
 
 namespace WebKit {
 
-struct TextInputContext {
+struct ElementContext {
     WebCore::FloatRect boundingRect;
 
     WebCore::PageIdentifier webPageIdentifier;
     WebCore::DocumentIdentifier documentIdentifier;
     WebCore::ElementIdentifier elementIdentifier;
 
-    ~TextInputContext();
+    ~ElementContext();
 };
 
-inline bool operator==(const TextInputContext& a, const TextInputContext& b)
+inline bool operator==(const ElementContext& a, const ElementContext& b)
 {
     return a.boundingRect == b.boundingRect
         && a.webPageIdentifier == b.webPageIdentifier
@@ -54,8 +54,8 @@ inline bool operator==(const TextInputContext& a, const TextInputContext& b)
 }
 
 namespace IPC {
-template<> struct ArgumentCoder<WebKit::TextInputContext> {
-    static void encode(Encoder&, const WebKit::TextInputContext&);
-    static Optional<WebKit::TextInputContext> decode(Decoder&);
+template<> struct ArgumentCoder<WebKit::ElementContext> {
+    static void encode(Encoder&, const WebKit::ElementContext&);
+    static Optional<WebKit::ElementContext> decode(Decoder&);
 };
 }
index fe92cd1..a287138 100644 (file)
@@ -28,6 +28,7 @@
 #if PLATFORM(IOS_FAMILY)
 
 #include "ArgumentCoders.h"
+#include "ElementContext.h"
 #include "InteractionInformationRequest.h"
 #include "ShareableBitmap.h"
 #include <WebCore/IntPoint.h>
@@ -89,6 +90,8 @@ struct InteractionInformationAtPosition {
     RetainPtr<NSArray> dataDetectorResults;
 #endif
 
+    Optional<ElementContext> elementContext;
+
     // Copy compatible optional bits forward (for example, if we have a InteractionInformationAtPosition
     // with snapshots in it, and perform another request for the same point without requesting the snapshots,
     // we can fetch the cheap information and copy the snapshots into the new response).
index ca4cddb..9c01309 100644 (file)
@@ -85,6 +85,7 @@ void InteractionInformationAtPosition::encode(IPC::Encoder& encoder) const
 #if ENABLE(DATALIST_ELEMENT)
     encoder << preventTextInteraction;
 #endif
+    encoder << elementContext;
 }
 
 bool InteractionInformationAtPosition::decode(IPC::Decoder& decoder, InteractionInformationAtPosition& result)
@@ -196,6 +197,9 @@ bool InteractionInformationAtPosition::decode(IPC::Decoder& decoder, Interaction
         return false;
 #endif
 
+    if (!decoder.decode(result.elementContext))
+        return false;
+
     return true;
 }
 
index 373d655..936c0cf 100644 (file)
@@ -121,6 +121,7 @@ Shared/CacheModel.cpp
 Shared/ContextMenuContextData.cpp
 Shared/EditingRange.cpp
 Shared/EditorState.cpp
+Shared/ElementContext.cpp
 Shared/FontInfo.cpp
 Shared/FrameInfoData.cpp
 Shared/LayerTreeContext.cpp
@@ -138,7 +139,6 @@ Shared/SharedStringHashStore.cpp
 Shared/SharedStringHashTableReadOnly.cpp
 Shared/SharedStringHashTable.cpp
 Shared/StatisticsData.cpp
-Shared/TextInputContext.cpp
 Shared/TouchBarMenuData.cpp
 Shared/TouchBarMenuItemData.cpp
 Shared/URLSchemeTaskParameters.cpp
index 4fec750..a5cfb72 100644 (file)
@@ -33,6 +33,7 @@
 #import "CompletionHandlerCallChecker.h"
 #import "DiagnosticLoggingClient.h"
 #import "DynamicViewportSizeUpdate.h"
+#import "ElementContext.h"
 #import "FindClient.h"
 #import "FullscreenClient.h"
 #import "GlobalFindInPageState.h"
@@ -49,7 +50,6 @@
 #import "RemoteObjectRegistry.h"
 #import "RemoteObjectRegistryMessages.h"
 #import "SafeBrowsingWarning.h"
-#import "TextInputContext.h"
 #import "UIDelegate.h"
 #import "UserMediaProcessManager.h"
 #import "VersionChecks.h"
@@ -5054,12 +5054,12 @@ FOR_EACH_PRIVATE_WKCONTENTVIEW_ACTION(FORWARD_ACTION_TO_WKCONTENTVIEW)
 
     CGRect rectInRootViewCoordinates = [self _convertRectToRootViewCoordinates:rectInWebViewCoordinates];
     auto weakSelf = WeakObjCPtr<WKWebView>(self);
-    _page->textInputContextsInRect(rectInRootViewCoordinates, [weakSelf, capturedCompletionHandler = makeBlockPtr(completionHandler)] (const Vector<WebKit::TextInputContext>& contexts) {
+    _page->textInputContextsInRect(rectInRootViewCoordinates, [weakSelf, capturedCompletionHandler = makeBlockPtr(completionHandler)] (const Vector<WebKit::ElementContext>& contexts) {
         RetainPtr<NSMutableArray> elements = adoptNS([[NSMutableArray alloc] initWithCapacity:contexts.size()]);
 
         auto strongSelf = weakSelf.get();
         for (const auto& context : contexts) {
-            WebKit::TextInputContext contextWithWebViewBoundingRect = context;
+            WebKit::ElementContext contextWithWebViewBoundingRect = context;
             contextWithWebViewBoundingRect.boundingRect = [strongSelf _convertRectFromRootViewCoordinates:context.boundingRect];
             [elements addObject:adoptNS([[_WKTextInputContext alloc] _initWithTextInputContext:contextWithWebViewBoundingRect]).get()];
         }
index c7ece2d..f61d727 100644 (file)
 #import "config.h"
 #import "_WKTextInputContext.h"
 
-#import "TextInputContext.h"
+#import "ElementContext.h"
 #import "_WKTextInputContextInternal.h"
 
 @implementation _WKTextInputContext {
-    WebKit::TextInputContext _textInputContext;
+    WebKit::ElementContext _textInputContext;
 }
 
 - (instancetype)init
@@ -38,7 +38,7 @@
     return nil;
 }
 
-- (instancetype)_initWithTextInputContext:(const WebKit::TextInputContext&)context
+- (instancetype)_initWithTextInputContext:(const WebKit::ElementContext&)context
 {
     self = [super init];
     if (!self)
@@ -54,7 +54,7 @@
     return _textInputContext.boundingRect;
 }
 
-- (const WebKit::TextInputContext&)_textInputContext
+- (const WebKit::ElementContext&)_textInputContext
 {
     return _textInputContext;
 }
index bf32a41..4ac7002 100644 (file)
 #import "_WKTextInputContext.h"
 
 namespace WebKit {
-struct TextInputContext;
+struct ElementContext;
 }
 
 @interface _WKTextInputContext ()
 
-- (instancetype)_initWithTextInputContext:(const WebKit::TextInputContext&)context;
+- (instancetype)_initWithTextInputContext:(const WebKit::ElementContext&)context;
 
-- (const WebKit::TextInputContext&)_textInputContext;
+- (const WebKit::ElementContext&)_textInputContext;
 
 @end
index 04182b9..87820d3 100644 (file)
@@ -58,6 +58,7 @@
 #include "DownloadProxy.h"
 #include "DrawingAreaMessages.h"
 #include "DrawingAreaProxy.h"
+#include "ElementContext.h"
 #include "EventDispatcherMessages.h"
 #include "FormDataReference.h"
 #include "FrameInfoData.h"
@@ -85,7 +86,6 @@
 #include "SyntheticEditingCommandType.h"
 #include "TextChecker.h"
 #include "TextCheckerState.h"
-#include "TextInputContext.h"
 #include "URLSchemeTaskParameters.h"
 #include "UndoOrRedo.h"
 #include "UserMediaPermissionRequestProxy.h"
@@ -9339,7 +9339,7 @@ void WebPageProxy::webViewDidMoveToWindow()
     }
 }
 
-void WebPageProxy::textInputContextsInRect(WebCore::FloatRect rect, CompletionHandler<void(const Vector<WebKit::TextInputContext>&)>&& completionHandler)
+void WebPageProxy::textInputContextsInRect(WebCore::FloatRect rect, CompletionHandler<void(const Vector<WebKit::ElementContext>&)>&& completionHandler)
 {
     if (!hasRunningProcess()) {
         completionHandler({ });
@@ -9349,7 +9349,7 @@ void WebPageProxy::textInputContextsInRect(WebCore::FloatRect rect, CompletionHa
     m_process->connection()->sendWithAsyncReply(Messages::WebPage::TextInputContextsInRect(rect), WTFMove(completionHandler), m_webPageID);
 }
 
-void WebPageProxy::focusTextInputContext(const TextInputContext& context, CompletionHandler<void(bool)>&& completionHandler)
+void WebPageProxy::focusTextInputContext(const ElementContext& context, CompletionHandler<void(bool)>&& completionHandler)
 {
     if (!hasRunningProcess()) {
         completionHandler(false);
index a41697c..8c77509 100644 (file)
@@ -288,6 +288,7 @@ struct DocumentEditingContext;
 struct DocumentEditingContextRequest;
 struct EditingRange;
 struct EditorState;
+struct ElementContext;
 struct FontInfo;
 struct FrameInfoData;
 struct InsertTextOptions;
@@ -295,7 +296,6 @@ struct InteractionInformationRequest;
 struct LoadParameters;
 struct PlatformPopupMenuData;
 struct PrintInfo;
-struct TextInputContext;
 struct WebAutocorrectionData;
 struct WebPopupItem;
 struct URLSchemeTaskParameters;
@@ -645,8 +645,8 @@ public:
     void requestFontAttributesAtSelectionStart(Function<void(const WebCore::FontAttributes&, CallbackBase::Error)>&&);
     void fontAttributesCallback(const WebCore::FontAttributes&, CallbackID);
 
-    void textInputContextsInRect(WebCore::FloatRect, CompletionHandler<void(const Vector<TextInputContext>&)>&&);
-    void focusTextInputContext(const TextInputContext&, CompletionHandler<void(bool)>&&);
+    void textInputContextsInRect(WebCore::FloatRect, CompletionHandler<void(const Vector<ElementContext>&)>&&);
+    void focusTextInputContext(const ElementContext&, CompletionHandler<void(bool)>&&);
 
 #if PLATFORM(IOS_FAMILY)
     double displayedContentScale() const { return m_lastVisibleContentRectUpdate.scale(); }
@@ -703,7 +703,7 @@ public:
     void replaceSelectedText(const String& oldText, const String& newText);
     void didReceivePositionInformation(const InteractionInformationAtPosition&);
     void requestPositionInformation(const InteractionInformationRequest&);
-    void startInteractionWithElementAtPosition(const WebCore::IntPoint&);
+    void startInteractionWithPositionInformation(const InteractionInformationAtPosition&);
     void stopInteraction();
     void performActionOnElement(uint32_t action);
     void saveImageToLibrary(const SharedMemory::Handle& imageHandle, uint64_t imageSize);
index de46501..52cdfa5 100644 (file)
@@ -6516,7 +6516,7 @@ static BOOL allPasteboardItemOriginsMatchOrigin(UIPasteboard *pasteboard, const
 
 - (void)actionSheetAssistant:(WKActionSheetAssistant *)assistant willStartInteractionWithElement:(_WKActivatedElementInfo *)element
 {
-    _page->startInteractionWithElementAtPosition(_positionInformation.request.point);
+    _page->startInteractionWithPositionInformation(_positionInformation);
 }
 
 - (void)actionSheetAssistantDidStopInteraction:(WKActionSheetAssistant *)assistant
@@ -8093,7 +8093,7 @@ static UIMenu *menuFromLegacyPreviewOrDefaultActions(UIViewController *previewVi
 
     const auto& url = _positionInformation.url;
 
-    _page->startInteractionWithElementAtPosition(_positionInformation.request.point);
+    _page->startInteractionWithPositionInformation(_positionInformation);
 
     UIViewController *previewViewController = nil;
 
@@ -8266,7 +8266,7 @@ static UIMenu *menuFromLegacyPreviewOrDefaultActions(UIViewController *previewVi
             return strongSelf->_contextMenuLegacyPreviewController.get();
         };
 
-        _page->startInteractionWithElementAtPosition(_positionInformation.request.point);
+        _page->startInteractionWithPositionInformation(_positionInformation);
 
         continueWithContextMenuConfiguration([UIContextMenuConfiguration configurationWithIdentifier:nil previewProvider:contentPreviewProvider actionProvider:actionMenuProvider]);
         return;
@@ -8288,7 +8288,7 @@ static UIMenu *menuFromLegacyPreviewOrDefaultActions(UIViewController *previewVi
         }
 
         if (configurationFromWKUIDelegate) {
-            strongSelf->_page->startInteractionWithElementAtPosition(strongSelf->_positionInformation.request.point);
+            strongSelf->_page->startInteractionWithPositionInformation(strongSelf->_positionInformation);
             strongSelf->_contextMenuActionProviderDelegateNeedsOverride = YES;
             continueWithContextMenuConfiguration(configurationFromWKUIDelegate);
             return;
@@ -8383,7 +8383,7 @@ static UIMenu *menuFromLegacyPreviewOrDefaultActions(UIViewController *previewVi
         NSDictionary *context = [self dataDetectionContextForPositionInformation:_positionInformation];
         UIContextMenuConfiguration *configurationFromDD = [ddContextMenuActionClass contextMenuConfigurationForURL:linkURL identifier:_positionInformation.dataDetectorIdentifier selectedText:[self selectedText] results:_positionInformation.dataDetectorResults.get() inView:self context:context menuIdentifier:nil];
         _contextMenuActionProviderDelegateNeedsOverride = YES;
-        _page->startInteractionWithElementAtPosition(_positionInformation.request.point);
+        _page->startInteractionWithPositionInformation(_positionInformation);
         continueWithContextMenuConfiguration(configurationFromDD);
         return YES;
     }
@@ -8816,7 +8816,7 @@ static RetainPtr<UITargetedPreview> createFallbackTargetedPreview(UIView *rootVi
 
     if ([_previewItemController type] == UIPreviewItemTypeLink) {
         _longPressCanClick = NO;
-        _page->startInteractionWithElementAtPosition(_positionInformation.request.point);
+        _page->startInteractionWithPositionInformation(_positionInformation);
 
         // Treat animated images like a link preview
         if (isValidURLForImagePreview && _positionInformation.isAnimatedImage) {
@@ -8876,7 +8876,7 @@ static RetainPtr<UITargetedPreview> createFallbackTargetedPreview(UIView *rootVi
         }
 
         RetainPtr<_WKActivatedElementInfo> elementInfo = adoptNS([[_WKActivatedElementInfo alloc] _initWithType:_WKActivatedElementTypeImage URL:alternateURL.get() imageURL:nil location:_positionInformation.request.point title:_positionInformation.title ID:_positionInformation.idAttribute rect:_positionInformation.bounds image:_positionInformation.image.get() userInfo:imageInfo.get()]);
-        _page->startInteractionWithElementAtPosition(_positionInformation.request.point);
+        _page->startInteractionWithPositionInformation(_positionInformation);
 
         ALLOW_DEPRECATED_DECLARATIONS_BEGIN
         if ([uiDelegate respondsToSelector:@selector(_webView:willPreviewImageWithURL:)])
index 83b886e..042bb5d 100644 (file)
@@ -624,9 +624,9 @@ void WebPageProxy::requestPositionInformation(const InteractionInformationReques
     m_process->send(Messages::WebPage::RequestPositionInformation(request), m_webPageID);
 }
 
-void WebPageProxy::startInteractionWithElementAtPosition(const WebCore::IntPoint& point)
+void WebPageProxy::startInteractionWithPositionInformation(const InteractionInformationAtPosition& positionInformation)
 {
-    m_process->send(Messages::WebPage::StartInteractionWithElementAtPosition(point), m_webPageID);
+    m_process->send(Messages::WebPage::StartInteractionWithElementContextOrPosition(positionInformation.elementContext, positionInformation.request.point), m_webPageID);
 }
 
 void WebPageProxy::stopInteraction()
index daaa17e..05a1eaa 100644 (file)
                2DDF731518E95060004F5A66 /* RemoteLayerBackingStoreCollection.h in Headers */ = {isa = PBXBuildFile; fileRef = 2DDF731318E95060004F5A66 /* RemoteLayerBackingStoreCollection.h */; };
                2DE6943D18BD2A68005C15E5 /* SmartMagnificationControllerMessageReceiver.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 2DE6943B18BD2A68005C15E5 /* SmartMagnificationControllerMessageReceiver.cpp */; };
                2DE6943E18BD2A68005C15E5 /* SmartMagnificationControllerMessages.h in Headers */ = {isa = PBXBuildFile; fileRef = 2DE6943C18BD2A68005C15E5 /* SmartMagnificationControllerMessages.h */; };
-               2DE9B1352231B5B2005287B7 /* TextInputContext.h in Headers */ = {isa = PBXBuildFile; fileRef = 2DE9B1332231B5B2005287B7 /* TextInputContext.h */; };
+               2DE9B1352231B5B2005287B7 /* ElementContext.h in Headers */ = {isa = PBXBuildFile; fileRef = 2DE9B1332231B5B2005287B7 /* ElementContext.h */; };
                2DE9B13A2231F61C005287B7 /* _WKTextInputContext.h in Headers */ = {isa = PBXBuildFile; fileRef = 2DE9B1382231F61C005287B7 /* _WKTextInputContext.h */; settings = {ATTRIBUTES = (Private, ); }; };
                2DE9B13C2231F77C005287B7 /* _WKTextInputContextInternal.h in Headers */ = {isa = PBXBuildFile; fileRef = 2DE9B13B2231F77C005287B7 /* _WKTextInputContextInternal.h */; };
                2DEAC5CF1AC368BB00A195D8 /* _WKFindOptions.h in Headers */ = {isa = PBXBuildFile; fileRef = 2DEAC5CE1AC368BB00A195D8 /* _WKFindOptions.h */; settings = {ATTRIBUTES = (Private, ); }; };
                2DDF731418E95060004F5A66 /* RemoteLayerBackingStoreCollection.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = RemoteLayerBackingStoreCollection.mm; sourceTree = "<group>"; };
                2DE6943B18BD2A68005C15E5 /* SmartMagnificationControllerMessageReceiver.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = SmartMagnificationControllerMessageReceiver.cpp; path = DerivedSources/WebKit2/SmartMagnificationControllerMessageReceiver.cpp; sourceTree = BUILT_PRODUCTS_DIR; };
                2DE6943C18BD2A68005C15E5 /* SmartMagnificationControllerMessages.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = SmartMagnificationControllerMessages.h; path = DerivedSources/WebKit2/SmartMagnificationControllerMessages.h; sourceTree = BUILT_PRODUCTS_DIR; };
-               2DE9B1332231B5B2005287B7 /* TextInputContext.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = TextInputContext.h; sourceTree = "<group>"; };
-               2DE9B1342231B5B2005287B7 /* TextInputContext.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = TextInputContext.cpp; sourceTree = "<group>"; };
+               2DE9B1332231B5B2005287B7 /* ElementContext.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ElementContext.h; sourceTree = "<group>"; };
+               2DE9B1342231B5B2005287B7 /* ElementContext.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ElementContext.cpp; sourceTree = "<group>"; };
                2DE9B1372231F61C005287B7 /* _WKTextInputContext.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = _WKTextInputContext.mm; sourceTree = "<group>"; };
                2DE9B1382231F61C005287B7 /* _WKTextInputContext.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = _WKTextInputContext.h; sourceTree = "<group>"; };
                2DE9B13B2231F77C005287B7 /* _WKTextInputContextInternal.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = _WKTextInputContextInternal.h; sourceTree = "<group>"; };
                                E105FE5318D7B9DE008F57A8 /* EditingRange.h */,
                                8CFECE931490F140002AAA32 /* EditorState.cpp */,
                                1AA41AB412C02EC4002BE67B /* EditorState.h */,
+                               2DE9B1342231B5B2005287B7 /* ElementContext.cpp */,
+                               2DE9B1332231B5B2005287B7 /* ElementContext.h */,
                                C59C4A5618B81174007BDCB6 /* FocusedElementInformation.cpp */,
                                C59C4A5718B81174007BDCB6 /* FocusedElementInformation.h */,
                                BCE81D8A1319F7EF00241910 /* FontInfo.cpp */,
                                5272B2881406985D0096A5D0 /* StatisticsData.cpp */,
                                5272B2891406985D0096A5D0 /* StatisticsData.h */,
                                1A5E4DA312D3BD3D0099A2BB /* TextCheckerState.h */,
-                               2DE9B1342231B5B2005287B7 /* TextInputContext.cpp */,
-                               2DE9B1332231B5B2005287B7 /* TextInputContext.h */,
                                2FD43B921FA006A30083F51C /* TouchBarMenuData.cpp */,
                                2FD43B911FA006A10083F51C /* TouchBarMenuData.h */,
                                2F809DD51FBD1BC9005FE63A /* TouchBarMenuItemData.cpp */,
                                2D5875BF219B53150070B9AA /* EditableImageController.h in Headers */,
                                E105FE5418D7B9DE008F57A8 /* EditingRange.h in Headers */,
                                1AA41AB512C02EC4002BE67B /* EditorState.h in Headers */,
+                               2DE9B1352231B5B2005287B7 /* ElementContext.h in Headers */,
                                BC032DA810F437D10058C15A /* Encoder.h in Headers */,
                                51B15A8513843A3900321AD8 /* EnvironmentUtilities.h in Headers */,
                                1AA575FB1496B52600A4EE06 /* EventDispatcher.h in Headers */,
                                1AA417CB12C00CCA002BE67B /* TextChecker.h in Headers */,
                                53CFBBC82224D1B500266546 /* TextCheckerCompletion.h in Headers */,
                                1A5E4DA412D3BD3D0099A2BB /* TextCheckerState.h in Headers */,
-                               2DE9B1352231B5B2005287B7 /* TextInputContext.h in Headers */,
                                CE1A0BD71A48E6C60054EF74 /* TextInputSPI.h in Headers */,
                                1AAF263914687C39004A1E8A /* TiledCoreAnimationDrawingArea.h in Headers */,
                                1AF05D8714688348008B1E81 /* TiledCoreAnimationDrawingAreaProxy.h in Headers */,
index 210a46e..c35c202 100644 (file)
@@ -6749,9 +6749,9 @@ static bool isEditableTextInputElement(Element& element)
     return element.isRootEditableElement();
 }
 
-void WebPage::textInputContextsInRect(WebCore::FloatRect searchRect, CompletionHandler<void(const Vector<TextInputContext>&)>&& completionHandler)
+void WebPage::textInputContextsInRect(WebCore::FloatRect searchRect, CompletionHandler<void(const Vector<ElementContext>&)>&& completionHandler)
 {
-    Vector<WebKit::TextInputContext> textInputContexts;
+    Vector<WebKit::ElementContext> textInputContexts;
 
     for (Frame* frame = &m_page->mainFrame(); frame; frame = frame->tree().traverseNext()) {
         Document* document = frame->document();
@@ -6777,7 +6777,7 @@ void WebPage::textInputContextsInRect(WebCore::FloatRect searchRect, CompletionH
             if (!searchRect.intersects(elementRect))
                 continue;
 
-            WebKit::TextInputContext context;
+            WebKit::ElementContext context;
             context.webPageIdentifier = m_identifier;
             context.documentIdentifier = document->identifier();
             context.elementIdentifier = document->identifierForElement(element);
@@ -6790,9 +6790,9 @@ void WebPage::textInputContextsInRect(WebCore::FloatRect searchRect, CompletionH
     completionHandler(textInputContexts);
 }
 
-void WebPage::focusTextInputContext(const TextInputContext& textInputContext, CompletionHandler<void(bool)>&& completionHandler)
+void WebPage::focusTextInputContext(const ElementContext& textInputContext, CompletionHandler<void(bool)>&& completionHandler)
 {
-    RefPtr<Element> element = elementForTextInputContext(textInputContext);
+    RefPtr<Element> element = elementForContext(textInputContext);
 
     if (element)
         element->focus();
@@ -6800,19 +6800,32 @@ void WebPage::focusTextInputContext(const TextInputContext& textInputContext, Co
     completionHandler(element);
 }
 
-Element* WebPage::elementForTextInputContext(const TextInputContext& textInputContext)
+Element* WebPage::elementForContext(const ElementContext& elementContext) const
 {
-    if (textInputContext.webPageIdentifier != m_identifier)
+    if (elementContext.webPageIdentifier != m_identifier)
         return nullptr;
 
-    auto* document = Document::allDocumentsMap().get(textInputContext.documentIdentifier);
+    auto* document = Document::allDocumentsMap().get(elementContext.documentIdentifier);
     if (!document)
         return nullptr;
 
     if (document->page() != m_page.get())
         return nullptr;
 
-    return document->searchForElementByIdentifier(textInputContext.elementIdentifier);
+    return document->searchForElementByIdentifier(elementContext.elementIdentifier);
+}
+
+Optional<ElementContext> WebPage::contextForElement(WebCore::Element& element) const
+{
+    auto& document = element.document();
+    if (!m_page || document.page() != m_page.get())
+        return WTF::nullopt;
+
+    auto frame = document.frame();
+    if (!frame)
+        return WTF::nullopt;
+
+    return ElementContext { elementRectInRootViewCoordinates(element, *frame), m_identifier, document.identifier(), document.identifierForElement(element) };
 }
 
 PAL::SessionID WebPage::sessionID() const
index df0f364..b1765c1 100644 (file)
@@ -261,13 +261,13 @@ struct AttributedString;
 struct BackForwardListItemState;
 struct DataDetectionResult;
 struct EditorState;
+struct ElementContext;
 struct FontInfo;
 struct InsertTextOptions;
 struct InteractionInformationAtPosition;
 struct InteractionInformationRequest;
 struct LoadParameters;
 struct PrintInfo;
-struct TextInputContext;
 struct WebAutocorrectionData;
 struct WebAutocorrectionContext;
 struct WebPageCreationParameters;
@@ -611,8 +611,8 @@ public:
     void executeEditCommandWithCallback(const String&, const String& argument, CallbackID);
     void selectAll();
 
-    void textInputContextsInRect(WebCore::FloatRect, CompletionHandler<void(const Vector<WebKit::TextInputContext>&)>&&);
-    void focusTextInputContext(const TextInputContext&, CompletionHandler<void(bool)>&&);
+    void textInputContextsInRect(WebCore::FloatRect, CompletionHandler<void(const Vector<WebKit::ElementContext>&)>&&);
+    void focusTextInputContext(const ElementContext&, CompletionHandler<void(bool)>&&);
 
 #if PLATFORM(IOS_FAMILY)
     WebCore::FloatSize screenSize() const;
@@ -668,7 +668,7 @@ public:
     void requestAutocorrectionContext();
     void getPositionInformation(const InteractionInformationRequest&, CompletionHandler<void(InteractionInformationAtPosition&&)>&&);
     void requestPositionInformation(const InteractionInformationRequest&);
-    void startInteractionWithElementAtPosition(const WebCore::IntPoint&);
+    void startInteractionWithElementContextOrPosition(Optional<ElementContext>&&, WebCore::IntPoint&&);
     void stopInteraction();
     void performActionOnElement(uint32_t action);
     void focusNextFocusedElement(bool isForward, CallbackID);
@@ -1188,7 +1188,8 @@ public:
 
     void configureLoggingChannel(const String&, WTFLogChannelState, WTFLogLevel);
 
-    WebCore::Element* elementForTextInputContext(const TextInputContext&);
+    WebCore::Element* elementForContext(const ElementContext&) const;
+    Optional<ElementContext> contextForElement(WebCore::Element&) const;
 
 #if ENABLE(APPLE_PAY)
     WebPaymentCoordinator* paymentCoordinator();
index 2ee1889..3096e2e 100644 (file)
@@ -86,7 +86,7 @@ messages -> WebPage LegacyReceiver {
     RequestEvasionRectsAboveSelection() -> (Vector<WebCore::FloatRect> rects) Async
     GetPositionInformation(struct WebKit::InteractionInformationRequest request) -> (struct WebKit::InteractionInformationAtPosition information) Synchronous
     RequestPositionInformation(struct WebKit::InteractionInformationRequest request)
-    StartInteractionWithElementAtPosition(WebCore::IntPoint point)
+    StartInteractionWithElementContextOrPosition(Optional<WebKit::ElementContext> elementContext, WebCore::IntPoint point)
     StopInteraction()
     PerformActionOnElement(uint32_t action)
     FocusNextFocusedElement(bool isForward, WebKit::CallbackID callbackID)
@@ -569,8 +569,8 @@ GenerateSyntheticEditingCommand(enum:uint8_t WebKit::SyntheticEditingCommandType
     VoicesDidChange()
 #endif
 
-    TextInputContextsInRect(WebCore::FloatRect rect) -> (Vector<struct WebKit::TextInputContext> contexts) Async
-    FocusTextInputContext(struct WebKit::TextInputContext context) -> (bool success) Async
+    TextInputContextsInRect(WebCore::FloatRect rect) -> (Vector<struct WebKit::ElementContext> contexts) Async
+    FocusTextInputContext(struct WebKit::ElementContext context) -> (bool success) Async
 
 #if ENABLE(RESOURCE_LOAD_STATISTICS)
     WasLoadedWithDataTransferFromPrevalentResource()
index 32ab847..ae5591e 100644 (file)
@@ -2715,6 +2715,8 @@ static void elementPositionInformation(WebPage& page, Element& element, const In
             imagePositionInformation(page, element, request, info);
         boundsPositionInformation(*renderer, info);
     }
+
+    info.elementContext = page.contextForElement(element);
 }
     
 static void selectionPositionInformation(WebPage& page, const InteractionInformationRequest& request, InteractionInformationAtPosition& info)
@@ -2813,14 +2815,14 @@ void WebPage::requestPositionInformation(const InteractionInformationRequest& re
     send(Messages::WebPageProxy::DidReceivePositionInformation(positionInformation(request)));
 }
 
-void WebPage::startInteractionWithElementAtPosition(const WebCore::IntPoint& point)
+void WebPage::startInteractionWithElementContextOrPosition(Optional<ElementContext>&& elementContext, WebCore::IntPoint&& point)
 {
-    // FIXME: We've already performed a hit test when the long-press gesture was recognized and we
-    // used that result to generate the targeted preview, but now we are hit testing again in order
-    // to perform the selected action. Since an arbitrary amount of time can elapse between
-    // generating the targeted preview and the user selecting an action, it's possible that this
-    // second hit test will find a different element than the first one, leading to bugs like
-    // <rdar://problem/54723131>. We should re-use the results of the first hit test instead.
+    if (elementContext) {
+        m_interactionNode = elementForContext(*elementContext);
+        if (m_interactionNode)
+            return;
+    }
+
     FloatPoint adjustedPoint;
     m_interactionNode = m_page->mainFrame().nodeRespondingToInteraction(point, adjustedPoint);
 }
@@ -3935,7 +3937,7 @@ void WebPage::requestDocumentEditingContext(DocumentEditingContextRequest reques
     bool wantsRects = request.options.contains(DocumentEditingContextRequest::Options::Rects);
 
     if (auto textInputContext = request.textInputContext) {
-        RefPtr<Element> element = elementForTextInputContext(*textInputContext);
+        RefPtr<Element> element = elementForContext(*textInputContext);
         if (!element) {
             completionHandler({ });
             return;