https://bugs.webkit.org/show_bug.cgi?id=176348
Work towards <rdar://problem/
31144674>
Reviewed by Tim Horton.
Source/WebKit:
Adds boilerplate plumbing to request additional items to an existing session. This implements some UI-side logic
in WKContentView to send an additional drag items request and handle a response from the web process.
To start, WebPageProxy::requestAdditionalItemsForDragSession is invoked by -_dragInteraction:
itemsForAddingToSession:withTouchAtPoint:completion: in WKContentView, and the response from the web process is
handled in -[WKContentView _didHandleAdditionalDragItemsRequest:].
* UIProcess/API/Cocoa/WKWebView.mm:
(-[WKWebView _simulateItemsForAddingToSession:atLocation:completion:]):
* UIProcess/API/Cocoa/WKWebViewPrivate.h:
Add SPI to simulate a request for additional drag items. See Tools/ChangeLog for more detail.
* UIProcess/PageClient.h:
* UIProcess/WebPageProxy.h:
* UIProcess/WebPageProxy.messages.in:
Add an IPC message handler for the additional drag items response from the web process.
* UIProcess/ios/DragDropInteractionState.h:
(WebKit::DragDropInteractionState::BlockPtr<void):
* UIProcess/ios/DragDropInteractionState.mm:
(WebKit::DragDropInteractionState::shouldRequestAdditionalItemForDragSession const):
(WebKit::DragDropInteractionState::dragSessionWillRequestAdditionalItem):
Invoke the new additional items completion handler when tearing down to avoid getting UIKit into a bad state.
(WebKit::DragDropInteractionState::dragAndDropSessionsDidEnd):
* UIProcess/ios/PageClientImplIOS.h:
* UIProcess/ios/PageClientImplIOS.mm:
(WebKit::PageClientImpl::didHandleAdditionalDragItemsRequest):
* UIProcess/ios/WKContentViewInteraction.h:
* UIProcess/ios/WKContentViewInteraction.mm:
(-[WKContentView _didHandleAdditionalDragItemsRequest:]):
(-[WKContentView _itemsForBeginningOrAddingToSessionWithRegistrationList:stagedDragSource:]):
Factors common logic to convert a staged WebItemProviderRegistrationInfoList and DragSourceState into a list of
drag items, invoking the private UI delegate in the process. This is called both when starting a drag session,
and adding items to an existing drag session.
(-[WKContentView _dragInteraction:itemsForAddingToSession:withTouchAtPoint:completion:]):
Implements a (proposed) additional drag item delegate that serves as an asynchronous alternative to the API
variant, -_dragInteraction:itemsForAddingToSession:withTouchAtPoint:. See <rdar://problem/
33146803> for more
information.
(-[WKContentView dragInteraction:itemsForBeginningSession:]):
Refactored to use -_itemsForBeginningOrAddingToSessionWithRegistrationList:stagedDragSource:.
(-[WKContentView _simulateItemsForAddingToSession:atLocation:completion:]):
* UIProcess/ios/WebPageProxyIOS.mm:
(WebKit::WebPageProxy::didHandleAdditionalDragItemsRequest):
(WebKit::WebPageProxy::requestAdditionalItemsForDragSession):
* WebProcess/WebPage/WebPage.h:
* WebProcess/WebPage/WebPage.messages.in:
Add an IPC message handler for the additional drag items request from the UI process.
* WebProcess/WebPage/ios/WebPageIOS.mm:
(WebKit::WebPage::requestAdditionalItemsForDragSession):
Tools:
Introduces new test infrastructure to simulate tapping to add additional items to the current drag session. We
add -[DataInteractionSimulator runFrom:to:additionalItemRequestLocations:], for which the last argument is a
dictionary mapping progress (a double between 0 and 1) to NSValues representing the location from which the drag
and drop simulation will request an additional drag item. During the simulated drag and drop, when the progress
value exceeds a value in the map of remaining additional item request locations, we halt drag simulation
progress for that runloop and instead request additional items from the location specified.
The only (useful) passing test we can create using the new machinery is one that verifies that preventDefault()
on dragstart works as expected, preventing additional items from being added. While this trivially passes now,
since the web-process-side of the additional items flow is not yet implemented, it should continue to pass after
the web process portion is implemented.
* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WebKitCocoa/selected-text-image-link-and-editable.html: Added.
Add a new test page that contains some non-editable text, an image, a link, and a rich text editable area.
* TestWebKitAPI/Tests/ios/DataInteractionTests.mm:
(TestWebKitAPI::TEST):
* TestWebKitAPI/ios/DataInteractionSimulator.h:
* TestWebKitAPI/ios/DataInteractionSimulator.mm:
(-[MockDragDropSession addItems:]):
Convenience method to add additional mock drag items to a drag or drop session.
(-[DataInteractionSimulator _resetSimulatedState]):
(-[DataInteractionSimulator runFrom:to:]):
Converted into a convenience wrapper around the latter version, passing in nil for the additional item request
locations dictionary.
(-[DataInteractionSimulator runFrom:to:additionalItemRequestLocations:]):
(-[DataInteractionSimulator _enqueuePendingAdditionalItemRequestLocations]):
(-[DataInteractionSimulator _sendQueuedAdditionalItemRequest]):
New helper methods to detect when and where additional items should be "added", and subsequently simulate adding
drag items at these given locations.
(-[DataInteractionSimulator _advanceProgress]):
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@221660
268f45cc-cd09-0410-ab3c-
d52691b4dbfc
+2017-09-05 Wenson Hsieh <wenson_hsieh@apple.com>
+
+ [iOS WK2] Add plumbing for WKContentView to ask the web process for additional drag items
+ https://bugs.webkit.org/show_bug.cgi?id=176348
+ Work towards <rdar://problem/31144674>
+
+ Reviewed by Tim Horton.
+
+ Adds boilerplate plumbing to request additional items to an existing session. This implements some UI-side logic
+ in WKContentView to send an additional drag items request and handle a response from the web process.
+ To start, WebPageProxy::requestAdditionalItemsForDragSession is invoked by -_dragInteraction:
+ itemsForAddingToSession:withTouchAtPoint:completion: in WKContentView, and the response from the web process is
+ handled in -[WKContentView _didHandleAdditionalDragItemsRequest:].
+
+ * UIProcess/API/Cocoa/WKWebView.mm:
+ (-[WKWebView _simulateItemsForAddingToSession:atLocation:completion:]):
+ * UIProcess/API/Cocoa/WKWebViewPrivate.h:
+
+ Add SPI to simulate a request for additional drag items. See Tools/ChangeLog for more detail.
+
+ * UIProcess/PageClient.h:
+ * UIProcess/WebPageProxy.h:
+ * UIProcess/WebPageProxy.messages.in:
+
+ Add an IPC message handler for the additional drag items response from the web process.
+
+ * UIProcess/ios/DragDropInteractionState.h:
+ (WebKit::DragDropInteractionState::BlockPtr<void):
+ * UIProcess/ios/DragDropInteractionState.mm:
+ (WebKit::DragDropInteractionState::shouldRequestAdditionalItemForDragSession const):
+ (WebKit::DragDropInteractionState::dragSessionWillRequestAdditionalItem):
+
+ Invoke the new additional items completion handler when tearing down to avoid getting UIKit into a bad state.
+
+ (WebKit::DragDropInteractionState::dragAndDropSessionsDidEnd):
+ * UIProcess/ios/PageClientImplIOS.h:
+ * UIProcess/ios/PageClientImplIOS.mm:
+ (WebKit::PageClientImpl::didHandleAdditionalDragItemsRequest):
+ * UIProcess/ios/WKContentViewInteraction.h:
+ * UIProcess/ios/WKContentViewInteraction.mm:
+ (-[WKContentView _didHandleAdditionalDragItemsRequest:]):
+ (-[WKContentView _itemsForBeginningOrAddingToSessionWithRegistrationList:stagedDragSource:]):
+
+ Factors common logic to convert a staged WebItemProviderRegistrationInfoList and DragSourceState into a list of
+ drag items, invoking the private UI delegate in the process. This is called both when starting a drag session,
+ and adding items to an existing drag session.
+
+ (-[WKContentView _dragInteraction:itemsForAddingToSession:withTouchAtPoint:completion:]):
+
+ Implements a (proposed) additional drag item delegate that serves as an asynchronous alternative to the API
+ variant, -_dragInteraction:itemsForAddingToSession:withTouchAtPoint:. See <rdar://problem/33146803> for more
+ information.
+
+ (-[WKContentView dragInteraction:itemsForBeginningSession:]):
+
+ Refactored to use -_itemsForBeginningOrAddingToSessionWithRegistrationList:stagedDragSource:.
+
+ (-[WKContentView _simulateItemsForAddingToSession:atLocation:completion:]):
+ * UIProcess/ios/WebPageProxyIOS.mm:
+ (WebKit::WebPageProxy::didHandleAdditionalDragItemsRequest):
+ (WebKit::WebPageProxy::requestAdditionalItemsForDragSession):
+ * WebProcess/WebPage/WebPage.h:
+ * WebProcess/WebPage/WebPage.messages.in:
+
+ Add an IPC message handler for the additional drag items request from the UI process.
+
+ * WebProcess/WebPage/ios/WebPageIOS.mm:
+ (WebKit::WebPage::requestAdditionalItemsForDragSession):
+
2017-09-05 Alex Christensen <achristensen@webkit.org>
Add WKUIDelegatePrivate equivalent of WKPageUIClient's mouseDidMoveOverElement
#endif
}
+- (void)_simulateItemsForAddingToSession:(id)session atLocation:(CGPoint)location completion:(void(^)(NSArray *))completion
+{
+#if ENABLE(DATA_INTERACTION)
+ [_contentView _simulateItemsForAddingToSession:session atLocation:location completion:completion];
+#endif
+}
+
- (void)_simulatePrepareForDataInteractionSession:(id)session completion:(dispatch_block_t)completion
{
#if ENABLE(DATA_INTERACTION)
- (void)_simulateDataInteractionSessionDidEnd:(id)session WK_API_AVAILABLE(ios(WK_IOS_TBA));
- (void)_simulateWillBeginDataInteractionWithSession:(id)session WK_API_AVAILABLE(ios(WK_IOS_TBA));
- (NSArray *)_simulatedItemsForSession:(id)session WK_API_AVAILABLE(ios(WK_IOS_TBA));
+- (void)_simulateItemsForAddingToSession:(id)session atLocation:(CGPoint)location completion:(void(^)(NSArray *))completion WK_API_AVAILABLE(ios(WK_IOS_TBA));
- (void)_simulatePrepareForDataInteractionSession:(id)session completion:(dispatch_block_t)completion WK_API_AVAILABLE(ios(WK_IOS_TBA));
- (void)_simulateLongPressActionAtLocation:(CGPoint)location;
#if ENABLE(DATA_INTERACTION)
virtual void didPerformDataInteractionControllerOperation(bool handled) = 0;
virtual void didHandleStartDataInteractionRequest(bool started) = 0;
+ virtual void didHandleAdditionalDragItemsRequest(bool added) = 0;
virtual void startDrag(const WebCore::DragItem&, const ShareableBitmap::Handle& image) = 0;
virtual void didConcludeEditDataInteraction(std::optional<WebCore::TextIndicatorData>) = 0;
virtual void didChangeDataInteractionCaretRect(const WebCore::IntRect& previousCaretRect, const WebCore::IntRect& caretRect) = 0;
#if ENABLE(DATA_INTERACTION)
void didPerformDataInteractionControllerOperation(bool handled);
void didHandleStartDataInteractionRequest(bool started);
+ void didHandleAdditionalDragItemsRequest(bool added);
void requestStartDataInteraction(const WebCore::IntPoint& clientPosition, const WebCore::IntPoint& globalPosition);
+ void requestAdditionalItemsForDragSession(const WebCore::IntPoint& clientPosition, const WebCore::IntPoint& globalPosition);
void didConcludeEditDataInteraction(std::optional<WebCore::TextIndicatorData>);
#endif
#endif
#if ENABLE(DATA_INTERACTION)
DidPerformDataInteractionControllerOperation(bool handled)
DidHandleStartDataInteractionRequest(bool started)
+ DidHandleAdditionalDragItemsRequest(bool added)
DidConcludeEditDataInteraction(std::optional<WebCore::TextIndicatorData> textIndicator)
#endif
void clearStagedDragSource(DidBecomeActive = DidBecomeActive::No);
UITargetedDragPreview *previewForDragItem(UIDragItem *, UIView *contentView, UIView *previewContainer) const;
void dragSessionWillDelaySetDownAnimation(dispatch_block_t completion);
+ bool shouldRequestAdditionalItemForDragSession(id <UIDragSession>) const;
+ void dragSessionWillRequestAdditionalItem(void (^completion)(NSArray <UIDragItem *> *));
// These helper methods are unique to UIDropInteraction.
void dropSessionDidEnterOrUpdate(id <UIDropSession>, const WebCore::DragData&);
id<UIDropSession> dropSession() const { return m_dropSession.get(); }
BlockPtr<void()> takeDragStartCompletionBlock() { return WTFMove(m_dragStartCompletionBlock); }
BlockPtr<void()> takeDragCancelSetDownBlock() { return WTFMove(m_dragCancelSetDownBlock); }
+ BlockPtr<void(NSArray<UIDragItem *> *)> takeAddDragItemCompletionBlock() { return WTFMove(m_addDragItemCompletionBlock); }
private:
void updatePreviewsForActiveDragSources();
RetainPtr<id <UIDropSession>> m_dropSession;
BlockPtr<void()> m_dragStartCompletionBlock;
BlockPtr<void()> m_dragCancelSetDownBlock;
+ BlockPtr<void(NSArray<UIDragItem *> *)> m_addDragItemCompletionBlock;
std::optional<DragSourceState> m_stagedDragSource;
Vector<DragSourceState> m_activeDragSources;
m_dragCancelSetDownBlock = completion;
}
+bool DragDropInteractionState::shouldRequestAdditionalItemForDragSession(id <UIDragSession> session) const
+{
+ return m_dragSession == session && !m_addDragItemCompletionBlock && !m_dragStartCompletionBlock;
+}
+
+void DragDropInteractionState::dragSessionWillRequestAdditionalItem(void (^completion)(NSArray <UIDragItem *> *))
+{
+ clearStagedDragSource();
+ m_addDragItemCompletionBlock = completion;
+}
+
void DragDropInteractionState::dropSessionDidEnterOrUpdate(id <UIDropSession> session, const DragData& dragData)
{
m_dropSession = session;
if (auto completionBlock = takeDragCancelSetDownBlock())
completionBlock();
+ if (auto completionBlock = takeAddDragItemCompletionBlock())
+ completionBlock(@[ ]);
+
if (auto completionBlock = takeDragStartCompletionBlock())
completionBlock();
}
#if ENABLE(DATA_INTERACTION)
void didPerformDataInteractionControllerOperation(bool handled) override;
void didHandleStartDataInteractionRequest(bool started) override;
+ void didHandleAdditionalDragItemsRequest(bool added) override;
void startDrag(const WebCore::DragItem&, const ShareableBitmap::Handle& image) override;
void didConcludeEditDataInteraction(std::optional<WebCore::TextIndicatorData>) override;
void didChangeDataInteractionCaretRect(const WebCore::IntRect& previousCaretRect, const WebCore::IntRect& caretRect) override;
[m_contentView _didHandleStartDataInteractionRequest:started];
}
+void PageClientImpl::didHandleAdditionalDragItemsRequest(bool added)
+{
+ [m_contentView _didHandleAdditionalDragItemsRequest:added];
+}
+
void PageClientImpl::startDrag(const DragItem& item, const ShareableBitmap::Handle& image)
{
[m_contentView _startDrag:ShareableBitmap::create(image)->makeCGImageCopy() item:item];
- (void)_didChangeDragInteractionPolicy;
- (void)_didPerformDataInteractionControllerOperation:(BOOL)handled;
- (void)_didHandleStartDataInteractionRequest:(BOOL)started;
+- (void)_didHandleAdditionalDragItemsRequest:(BOOL)added;
- (void)_startDrag:(RetainPtr<CGImageRef>)image item:(const WebCore::DragItem&)item;
- (void)_didConcludeEditDataInteraction:(std::optional<WebCore::TextIndicatorData>)data;
- (void)_didChangeDataInteractionCaretRect:(CGRect)previousRect currentRect:(CGRect)rect;
- (void)_simulateWillBeginDataInteractionWithSession:(id)session;
- (NSArray *)_simulatedItemsForSession:(id)session;
- (void)_simulatePrepareForDataInteractionSession:(id)session completion:(dispatch_block_t)completion;
+- (void)_simulateItemsForAddingToSession:(id)session atLocation:(CGPoint)location completion:(void(^)(NSArray *))completion;
#endif
- (void)_simulateLongPressActionAtLocation:(CGPoint)location;
_dragDropInteractionState.stageDragItem(item, dragImage.get());
}
+- (void)_didHandleAdditionalDragItemsRequest:(BOOL)added
+{
+ auto completion = _dragDropInteractionState.takeAddDragItemCompletionBlock();
+ if (!completion)
+ return;
+
+ WebItemProviderRegistrationInfoList *registrationList = [[WebItemProviderPasteboard sharedInstance] takeRegistrationList];
+ if (!added || !registrationList || !_dragDropInteractionState.hasStagedDragSource()) {
+ _dragDropInteractionState.clearStagedDragSource();
+ completion(@[ ]);
+ return;
+ }
+
+ auto stagedDragSource = _dragDropInteractionState.stagedDragSource();
+ NSArray *dragItemsToAdd = [self _itemsForBeginningOrAddingToSessionWithRegistrationList:registrationList stagedDragSource:stagedDragSource];
+
+ RELEASE_LOG(DragAndDrop, "Drag session: %p adding %tu items", _dragDropInteractionState.dragSession(), dragItemsToAdd.count);
+ _dragDropInteractionState.clearStagedDragSource(dragItemsToAdd.count ? DragDropInteractionState::DidBecomeActive::Yes : DragDropInteractionState::DidBecomeActive::No);
+
+ completion(dragItemsToAdd);
+
+ if (dragItemsToAdd.count)
+ _page->didStartDrag();
+}
+
- (void)_didHandleStartDataInteractionRequest:(BOOL)started
{
BlockPtr<void()> savedCompletionBlock = _dragDropInteractionState.takeDragStartCompletionBlock();
_shouldRestoreCalloutBarAfterDrop = NO;
}
+- (NSArray<UIDragItem *> *)_itemsForBeginningOrAddingToSessionWithRegistrationList:(WebItemProviderRegistrationInfoList *)registrationList stagedDragSource:(const DragSourceState&)stagedDragSource
+{
+ UIItemProvider *defaultItemProvider = registrationList.itemProvider;
+ if (!defaultItemProvider)
+ return @[ ];
+
+ NSArray *adjustedItemProviders;
+ id <WKUIDelegatePrivate> uiDelegate = self.webViewUIDelegate;
+ if ([uiDelegate respondsToSelector:@selector(_webView:adjustedDataInteractionItemProvidersForItemProvider:representingObjects:additionalData:)]) {
+ auto representingObjects = adoptNS([[NSMutableArray alloc] init]);
+ auto additionalData = adoptNS([[NSMutableDictionary alloc] init]);
+ [registrationList enumerateItems:[representingObjects, additionalData] (WebItemProviderRegistrationInfo *item, NSUInteger) {
+ if (item.representingObject)
+ [representingObjects addObject:item.representingObject];
+ if (item.typeIdentifier && item.data)
+ [additionalData setObject:item.data forKey:item.typeIdentifier];
+ }];
+ adjustedItemProviders = [uiDelegate _webView:_webView adjustedDataInteractionItemProvidersForItemProvider:defaultItemProvider representingObjects:representingObjects.get() additionalData:additionalData.get()];
+ } else
+ adjustedItemProviders = @[ defaultItemProvider ];
+
+ NSMutableArray *dragItems = [NSMutableArray arrayWithCapacity:adjustedItemProviders.count];
+ for (UIItemProvider *itemProvider in adjustedItemProviders) {
+ auto item = adoptNS([[UIDragItem alloc] initWithItemProvider:itemProvider]);
+ [item _setPrivateLocalContext:@(stagedDragSource.itemIdentifier)];
+ [dragItems addObject:item.autorelease()];
+ }
+
+ return dragItems;
+}
+
#pragma mark - UIDragInteractionDelegate
- (BOOL)_dragInteraction:(UIDragInteraction *)interaction shouldDelayCompetingGestureRecognizer:(UIGestureRecognizer *)competingGestureRecognizer
return dataOwner;
}
+- (void)_dragInteraction:(UIDragInteraction *)interaction itemsForAddingToSession:(id <UIDragSession>)session withTouchAtPoint:(CGPoint)point completion:(void(^)(NSArray<UIDragItem *> *))completion
+{
+ if (!_dragDropInteractionState.shouldRequestAdditionalItemForDragSession(session)) {
+ completion(@[ ]);
+ return;
+ }
+
+ _dragDropInteractionState.dragSessionWillRequestAdditionalItem(completion);
+ _page->requestAdditionalItemsForDragSession(roundedIntPoint(point), roundedIntPoint(point));
+}
+
- (void)_dragInteraction:(UIDragInteraction *)interaction prepareForSession:(id <UIDragSession>)session completion:(dispatch_block_t)completion
{
[self _cancelLongPressGestureRecognizer];
auto stagedDragSource = _dragDropInteractionState.stagedDragSource();
WebItemProviderRegistrationInfoList *registrationList = [[WebItemProviderPasteboard sharedInstance] takeRegistrationList];
- UIItemProvider *defaultItemProvider = registrationList.itemProvider;
- if (!defaultItemProvider) {
- RELEASE_LOG(DragAndDrop, "Drag session failed: %p (no item providers generated before adjustment)", session);
- _page->dragCancelled();
- _dragDropInteractionState.clearStagedDragSource();
- return @[ ];
- }
-
- // Give internal clients such as Mail one final chance to augment the contents of each UIItemProvider before sending the drag items off to UIKit.
- NSArray *adjustedItemProviders;
- id <WKUIDelegatePrivate> uiDelegate = self.webViewUIDelegate;
- if ([uiDelegate respondsToSelector:@selector(_webView:adjustedDataInteractionItemProvidersForItemProvider:representingObjects:additionalData:)]) {
- auto representingObjects = adoptNS([[NSMutableArray alloc] init]);
- auto additionalData = adoptNS([[NSMutableDictionary alloc] init]);
- [registrationList enumerateItems:[representingObjects, additionalData] (WebItemProviderRegistrationInfo *item, NSUInteger) {
- if (item.representingObject)
- [representingObjects addObject:item.representingObject];
- if (item.typeIdentifier && item.data)
- [additionalData setObject:item.data forKey:item.typeIdentifier];
- }];
- adjustedItemProviders = [uiDelegate _webView:_webView adjustedDataInteractionItemProvidersForItemProvider:defaultItemProvider representingObjects:representingObjects.get() additionalData:additionalData.get()];
- } else
- adjustedItemProviders = @[ defaultItemProvider ];
-
- NSMutableArray *dragItems = [NSMutableArray arrayWithCapacity:adjustedItemProviders.count];
- for (UIItemProvider *itemProvider in adjustedItemProviders) {
- auto item = adoptNS([[UIDragItem alloc] initWithItemProvider:itemProvider]);
- [item _setPrivateLocalContext:@(stagedDragSource.itemIdentifier)];
- [dragItems addObject:item.autorelease()];
- }
-
+ NSArray *dragItems = [self _itemsForBeginningOrAddingToSessionWithRegistrationList:registrationList stagedDragSource:stagedDragSource];
if (![dragItems count])
_page->dragCancelled();
[self _dragInteraction:_dragInteraction.get() prepareForSession:session completion:completion];
}
+- (void)_simulateItemsForAddingToSession:(id)session atLocation:(CGPoint)location completion:(void(^)(NSArray *))completion
+{
+ [self _dragInteraction:_dragInteraction.get() itemsForAddingToSession:session withTouchAtPoint:location completion:completion];
+}
+
#endif
- (void)_simulateLongPressActionAtLocation:(CGPoint)location
m_pageClient.didHandleStartDataInteractionRequest(started);
}
+void WebPageProxy::didHandleAdditionalDragItemsRequest(bool added)
+{
+ m_pageClient.didHandleAdditionalDragItemsRequest(added);
+}
+
void WebPageProxy::requestStartDataInteraction(const WebCore::IntPoint& clientPosition, const WebCore::IntPoint& globalPosition)
{
if (isValid())
m_process->send(Messages::WebPage::RequestStartDataInteraction(clientPosition, globalPosition), m_pageID);
}
+void WebPageProxy::requestAdditionalItemsForDragSession(const IntPoint& clientPosition, const IntPoint& globalPosition)
+{
+ if (isValid())
+ m_process->send(Messages::WebPage::RequestAdditionalItemsForDragSession(clientPosition, globalPosition), m_pageID);
+}
+
void WebPageProxy::didConcludeEditDataInteraction(std::optional<TextIndicatorData> data)
{
m_pageClient.didConcludeEditDataInteraction(data);
RefPtr<WebCore::Range> switchToBlockSelectionAtPoint(const WebCore::IntPoint&, SelectionHandlePosition);
#if ENABLE(DATA_INTERACTION)
void requestStartDataInteraction(const WebCore::IntPoint& clientPosition, const WebCore::IntPoint& globalPosition);
+ void requestAdditionalItemsForDragSession(const WebCore::IntPoint& clientPosition, const WebCore::IntPoint& globalPosition);
#endif
#endif
#if ENABLE(DATA_INTERACTION)
RequestStartDataInteraction(WebCore::IntPoint clientPosition, WebCore::IntPoint globalPosition)
+ RequestAdditionalItemsForDragSession(WebCore::IntPoint clientPosition, WebCore::IntPoint globalPosition)
#endif
# Popup menu.
send(Messages::WebPageProxy::DidHandleStartDataInteractionRequest(didStart));
}
+void WebPage::requestAdditionalItemsForDragSession(const IntPoint& clientPosition, const IntPoint& globalPosition)
+{
+ notImplemented();
+
+ send(Messages::WebPageProxy::DidHandleAdditionalDragItemsRequest(false));
+}
+
void WebPage::didConcludeEditDataInteraction()
{
std::optional<TextIndicatorData> textIndicatorData;
+2017-09-05 Wenson Hsieh <wenson_hsieh@apple.com>
+
+ [iOS WK2] Add plumbing for WKContentView to ask the web process for additional drag items
+ https://bugs.webkit.org/show_bug.cgi?id=176348
+ Work towards <rdar://problem/31144674>
+
+ Reviewed by Tim Horton.
+
+ Introduces new test infrastructure to simulate tapping to add additional items to the current drag session. We
+ add -[DataInteractionSimulator runFrom:to:additionalItemRequestLocations:], for which the last argument is a
+ dictionary mapping progress (a double between 0 and 1) to NSValues representing the location from which the drag
+ and drop simulation will request an additional drag item. During the simulated drag and drop, when the progress
+ value exceeds a value in the map of remaining additional item request locations, we halt drag simulation
+ progress for that runloop and instead request additional items from the location specified.
+
+ The only (useful) passing test we can create using the new machinery is one that verifies that preventDefault()
+ on dragstart works as expected, preventing additional items from being added. While this trivially passes now,
+ since the web-process-side of the additional items flow is not yet implemented, it should continue to pass after
+ the web process portion is implemented.
+
+ * TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
+ * TestWebKitAPI/Tests/WebKitCocoa/selected-text-image-link-and-editable.html: Added.
+
+ Add a new test page that contains some non-editable text, an image, a link, and a rich text editable area.
+
+ * TestWebKitAPI/Tests/ios/DataInteractionTests.mm:
+ (TestWebKitAPI::TEST):
+ * TestWebKitAPI/ios/DataInteractionSimulator.h:
+ * TestWebKitAPI/ios/DataInteractionSimulator.mm:
+ (-[MockDragDropSession addItems:]):
+
+ Convenience method to add additional mock drag items to a drag or drop session.
+
+ (-[DataInteractionSimulator _resetSimulatedState]):
+ (-[DataInteractionSimulator runFrom:to:]):
+
+ Converted into a convenience wrapper around the latter version, passing in nil for the additional item request
+ locations dictionary.
+
+ (-[DataInteractionSimulator runFrom:to:additionalItemRequestLocations:]):
+ (-[DataInteractionSimulator _enqueuePendingAdditionalItemRequestLocations]):
+ (-[DataInteractionSimulator _sendQueuedAdditionalItemRequest]):
+
+ New helper methods to detect when and where additional items should be "added", and subsequently simulate adding
+ drag items at these given locations.
+
+ (-[DataInteractionSimulator _advanceProgress]):
+
2017-09-05 Alex Christensen <achristensen@webkit.org>
Add WKUIDelegatePrivate equivalent of WKPageUIClient's mouseDidMoveOverElement
F4D4F3B61E4E2BCB00BB2767 /* DataInteractionSimulator.mm in Sources */ = {isa = PBXBuildFile; fileRef = F4D4F3B41E4E2BCB00BB2767 /* DataInteractionSimulator.mm */; };
F4D4F3B91E4E36E400BB2767 /* DataInteractionTests.mm in Sources */ = {isa = PBXBuildFile; fileRef = F4D4F3B71E4E36E400BB2767 /* DataInteractionTests.mm */; };
F4D5E4E81F0C5D38008C1A49 /* dragstart-clear-selection.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = F4D5E4E71F0C5D27008C1A49 /* dragstart-clear-selection.html */; };
+ F4D65DA81F5E4704009D8C27 /* selected-text-image-link-and-editable.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = F4D65DA71F5E46C0009D8C27 /* selected-text-image-link-and-editable.html */; };
F4DEF6ED1E9B4DB60048EF61 /* image-in-link-and-input.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = F4DEF6EC1E9B4D950048EF61 /* image-in-link-and-input.html */; };
F4F137921D9B683E002BEC57 /* large-video-test-now-playing.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = F4F137911D9B6832002BEC57 /* large-video-test-now-playing.html */; };
F4F405BC1D4C0D1C007A9707 /* full-size-autoplaying-video-with-audio.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = F4F405BA1D4C0CF8007A9707 /* full-size-autoplaying-video-with-audio.html */; };
F6FDDDD614241C6F004F1729 /* push-state.html in Copy Resources */,
A12DDC001E8373E700CF6CAE /* rendered-image-excluding-overflow.html in Copy Resources */,
F46849C01EEF5EF300B937FE /* rich-and-plain-text.html in Copy Resources */,
+ F4D65DA81F5E4704009D8C27 /* selected-text-image-link-and-editable.html in Copy Resources */,
7A66BDB81EAF18D500CCC924 /* set-long-title.html in Copy Resources */,
52B8CF9815868D9100281053 /* SetDocumentURI.html in Copy Resources */,
CEBABD491B71687C0051210A /* should-open-external-schemes.html in Copy Resources */,
F4D4F3B51E4E2BCB00BB2767 /* DataInteractionSimulator.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = DataInteractionSimulator.h; sourceTree = "<group>"; };
F4D4F3B71E4E36E400BB2767 /* DataInteractionTests.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = DataInteractionTests.mm; sourceTree = "<group>"; };
F4D5E4E71F0C5D27008C1A49 /* dragstart-clear-selection.html */ = {isa = PBXFileReference; lastKnownFileType = text.html; path = "dragstart-clear-selection.html"; sourceTree = "<group>"; };
+ F4D65DA71F5E46C0009D8C27 /* selected-text-image-link-and-editable.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = "selected-text-image-link-and-editable.html"; sourceTree = "<group>"; };
F4DEF6EC1E9B4D950048EF61 /* image-in-link-and-input.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = "image-in-link-and-input.html"; sourceTree = "<group>"; };
F4F137911D9B6832002BEC57 /* large-video-test-now-playing.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = "large-video-test-now-playing.html"; sourceTree = "<group>"; };
F4F405BA1D4C0CF8007A9707 /* full-size-autoplaying-video-with-audio.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = "full-size-autoplaying-video-with-audio.html"; sourceTree = "<group>"; };
F41AB99A1EF4692C0083FA08 /* prevent-start.html */,
A12DDBFF1E8373C100CF6CAE /* rendered-image-excluding-overflow.html */,
F46849BF1EEF5EDC00B937FE /* rich-and-plain-text.html */,
+ F4D65DA71F5E46C0009D8C27 /* selected-text-image-link-and-editable.html */,
C9B4AD291ECA6EA500F5FEA0 /* silence-long.m4a */,
F4F405BB1D4C0CF8007A9707 /* skinny-autoplaying-video-with-audio.html */,
515BE16E1D4288FF00DD7C68 /* StoreBlobToBeDeleted.html */,
--- /dev/null
+<!DOCTYPE html>
+<html>
+<head>
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<style>
+html, body {
+ width: 100%;
+ height: 100%;
+ margin: 0;
+}
+
+#text, #image, #link {
+ height: 100px;
+ font-size: 80px;
+ white-space: nowrap;
+}
+
+#editor {
+ border: black 1px solid;
+ height: calc(100% - 300px);
+}
+</style>
+</head>
+
+<body>
+<div id="text">ABCD</div>
+<div><img id="image" src="apple.gif"></img></div>
+<div><a id="link" href="https://www.apple.com/">A link</a></div>
+<div contenteditable id="editor"></div>
+<script>
+getSelection().setBaseAndExtent(text, 0, text, 1);
+</script>
+</body>
+</html>
EXPECT_WK_STREQ("FAR OFFSCREEN", [webView stringByEvaluatingJavaScript:@"getSelection().getRangeAt(0).toString()"]);
}
+TEST(DataInteractionTests, AdditionalItemsCanBePreventedOnDragStart)
+{
+ auto webView = adoptNS([[TestWKWebView alloc] initWithFrame:CGRectMake(0, 0, 320, 500)]);
+ [webView synchronouslyLoadTestPageNamed:@"selected-text-image-link-and-editable"];
+ [webView stringByEvaluatingJavaScript:@"link.addEventListener('dragstart', e => e.preventDefault())"];
+ [webView stringByEvaluatingJavaScript:@"image.addEventListener('dragstart', e => e.preventDefault())"];
+
+ auto simulator = adoptNS([[DataInteractionSimulator alloc] initWithWebView:webView.get()]);
+ [simulator runFrom:CGPointMake(50, 50) to:CGPointMake(50, 400) additionalItemRequestLocations:@{
+ @0.33: [NSValue valueWithCGPoint:CGPointMake(50, 150)],
+ @0.66: [NSValue valueWithCGPoint:CGPointMake(50, 250)]
+ }];
+ EXPECT_WK_STREQ("ABCD", [webView stringByEvaluatingJavaScript:@"editor.textContent"]);
+}
+
} // namespace TestWebKitAPI
#endif // ENABLE(DATA_INTERACTION)
extern NSString * const DataInteractionLeaveEventName;
extern NSString * const DataInteractionStartEventName;
+typedef NSDictionary<NSNumber *, NSValue *> *ProgressToCGPointValueMap;
+
typedef NS_ENUM(NSInteger, DataInteractionPhase) {
DataInteractionCancelled = 0,
DataInteractionBeginning = 1,
CGPoint _endLocation;
CGRect _lastKnownDragCaretRect;
+ RetainPtr<NSMutableDictionary<NSNumber *, NSValue *>>_remainingAdditionalItemRequestLocationsByProgress;
+ RetainPtr<NSMutableArray<NSValue *>>_queuedAdditionalItemRequestLocations;
+
bool _isDoneWaitingForInputSession;
BOOL _shouldPerformOperation;
double _currentProgress;
}
- (instancetype)initWithWebView:(TestWKWebView *)webView;
+// The start location, end location, and locations of additional item requests are all in window coordinates.
- (void)runFrom:(CGPoint)startLocation to:(CGPoint)endLocation;
+- (void)runFrom:(CGPoint)startLocation to:(CGPoint)endLocation additionalItemRequestLocations:(ProgressToCGPointValueMap)additionalItemRequestLocations;
- (void)waitForInputSession;
@property (nonatomic) BOOL allowsFocusToStartInputSession;
_mockItems = items;
}
+- (void)addItems:(NSArray<UIDragItem *> *)items
+{
+ if (![items count])
+ return;
+
+ if (![_mockItems count])
+ _mockItems = items;
+ else
+ _mockItems = [_mockItems arrayByAddingObjectsFromArray:items];
+}
+
- (CGPoint)locationInView:(UIView *)view
{
return [_window convertPoint:_mockLocationInWindow toView:view];
_dataOperationSession = nil;
_shouldPerformOperation = NO;
_lastKnownDragCaretRect = CGRectZero;
+ _remainingAdditionalItemRequestLocationsByProgress = nil;
+ _queuedAdditionalItemRequestLocations = adoptNS([[NSMutableArray alloc] init]);
}
- (NSArray *)observedEventNames
}
- (void)runFrom:(CGPoint)startLocation to:(CGPoint)endLocation
+{
+ [self runFrom:startLocation to:endLocation additionalItemRequestLocations:nil];
+}
+
+- (void)runFrom:(CGPoint)startLocation to:(CGPoint)endLocation additionalItemRequestLocations:(ProgressToCGPointValueMap)additionalItemRequestLocations
{
NSNotificationCenter *defaultCenter = [NSNotificationCenter defaultCenter];
[defaultCenter addObserver:self selector:@selector(simulateAllTouchesCanceled:) name:TestWebKitAPISimulateCancelAllTouchesNotificationName object:nil];
[self _resetSimulatedState];
+ if (additionalItemRequestLocations)
+ _remainingAdditionalItemRequestLocationsByProgress = adoptNS([additionalItemRequestLocations mutableCopy]);
+
RetainPtr<DataInteractionSimulator> strongSelf = self;
for (NSString *eventName in dataInteractionEventNames()) {
DataInteractionSimulator *weakSelf = strongSelf.get();
[_webView _simulateDataInteractionSessionDidEnd:_dataInteractionSession.get()];
}
+- (void)_enqueuePendingAdditionalItemRequestLocations
+{
+ NSMutableArray *progressValuesToRemove = [NSMutableArray array];
+ for (NSNumber *progressValue in _remainingAdditionalItemRequestLocationsByProgress.get()) {
+ double progress = progressValue.doubleValue;
+ if (progress > _currentProgress)
+ continue;
+ [progressValuesToRemove addObject:progressValue];
+ [_queuedAdditionalItemRequestLocations addObject:[_remainingAdditionalItemRequestLocationsByProgress objectForKey:progressValue]];
+ }
+
+ for (NSNumber *progressToRemove in progressValuesToRemove)
+ [_remainingAdditionalItemRequestLocationsByProgress removeObjectForKey:progressToRemove];
+}
+
+- (BOOL)_sendQueuedAdditionalItemRequest
+{
+ if (![_queuedAdditionalItemRequestLocations count])
+ return NO;
+
+ RetainPtr<NSValue> requestLocationValue = [_queuedAdditionalItemRequestLocations objectAtIndex:0];
+ [_queuedAdditionalItemRequestLocations removeObjectAtIndex:0];
+
+ auto requestLocation = [[_webView window] convertPoint:[requestLocationValue CGPointValue] toView:_webView.get()];
+ [_webView _simulateItemsForAddingToSession:_dataInteractionSession.get() atLocation:requestLocation completion:[dragSession = _dataInteractionSession, dropSession = _dataOperationSession] (NSArray *items) {
+ [dragSession addItems:items];
+ [dropSession addItems:items];
+ }];
+ return YES;
+}
+
- (void)_advanceProgress
{
+ [self _enqueuePendingAdditionalItemRequestLocations];
+ if ([self _sendQueuedAdditionalItemRequest]) {
+ [self _scheduleAdvanceProgress];
+ return;
+ }
+
_lastKnownDragCaretRect = [_webView _dragCaretRect];
_currentProgress += progressIncrementStep;
CGPoint locationInWindow = self._currentLocation;
git://git.webkit.org / WebKit-https.git / commitdiff