[iOS DnD] Support DataTransfer.setDragImage when starting a drag on iOS
authorwenson_hsieh@apple.com <wenson_hsieh@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 12 Sep 2017 06:59:48 +0000 (06:59 +0000)
committerwenson_hsieh@apple.com <wenson_hsieh@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 12 Sep 2017 06:59:48 +0000 (06:59 +0000)
https://bugs.webkit.org/show_bug.cgi?id=176721
<rdar://problem/34373660>

Reviewed by Tim Horton.

Source/WebCore:

Adds support for setting the drag lift preview frame in the case where DataTransfer.setDragImage is being used
to override the default drag preview. Currently, the frame of the drag preview we supply in this case is the
same as the bounds of the source element in root view coordinates, but this means that any custom drag image
the page supplies will be stretched to fill the frame of the source element. Instead, when handling a DHTML drag,
position the lift and cancel drag previews relative to the event location, respecting any drag offset specified
in setDragImage. The size of this preview matches the size of the drag image source (since this is all in root
view coordinates, this means the drag preview will also enlarge if the user pinches to zoom in). If a
disconnected image source element was provided, then we just fall back to the image size.

Additionally, renames DragItem's elementBounds to dragPreviewFrameInRootViewCoordinates to better reflect the
purpose of this variable. This patch also introduces API test plumbing to grab targeted drag previews from the
drag interaction delegate (i.e. WKContentView), and uses this in a new API test that checks the frame of the
resulting UITargetedDragPreview after initiating a drag in various circumstances (see changes in Tools/ for more
detail).

Test: DataInteractionTests.DragLiftPreviewDataTransferSetDragImage

* dom/DataTransfer.cpp:
(WebCore::DataTransfer::dragImageElement const):
* dom/DataTransfer.h:
* page/DragController.cpp:
(WebCore::dragLocForDHTMLDrag):

The logic to flip the y offset when computing the drag location is only relevant on Mac, but currently, this is
guarded by #if PLATFORM(COCOA), which causes the y offset to shift the drag image in the opposite direction on
iOS. To fix this, simply change the platform define to Mac.

(WebCore::DragController::doSystemDrag):
* platform/DragItem.h:
(WebCore::DragItem::encode const):
(WebCore::DragItem::decode):

Source/WebKit:

Rename elementBounds => dragPreviewFrameInRootViewCoordinates.

* UIProcess/ios/DragDropInteractionState.h:
* UIProcess/ios/DragDropInteractionState.mm:
(WebKit::DragDropInteractionState::previewForDragItem const):
(WebKit::DragDropInteractionState::stageDragItem):

Source/WebKitLegacy/mac:

Rename elementBounds => dragPreviewFrameInRootViewCoordinates. (Note: Unfortunately, _draggedElementBounds in
WebViewPrivate.h can't be renamed yet, due to binary and SDK compatibility with UIKit).

* WebView/WebView.mm:
(-[WebView _startDrag:]):
(-[WebView _draggedElementBounds]):
(-[WebView _endedDataInteraction:global:]):
* WebView/WebViewData.h:

Tools:

In DataInteractionSimulator, ask the UIDragInteractionDelegate (WKContentView) for targeted drag previews after
retrieving UIDragItems during a lift. Remember these previews after the drag and drop simulation is complete, so
API tests (currently, just DragLiftPreviewDataTransferSetDragImage) can verify that the generated drag previews
are as expected.

* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WebKitCocoa/DataTransfer-setDragImage.html: Added.

Adds a new test harness containing 5 draggable elements, each generating a drag image using different codepaths.
DragLiftPreviewDataTransferSetDragImage uses this page to check that the frame of the targeted drag preview in
each scenario is as expected.

* TestWebKitAPI/Tests/ios/DataInteractionTests.mm:
(checkCGRectIsEqualToCGRectWithLogging):
(TestWebKitAPI::TEST):
* TestWebKitAPI/ios/DataInteractionSimulator.h:
* TestWebKitAPI/ios/DataInteractionSimulator.mm:
(-[DataInteractionSimulator _resetSimulatedState]):
(-[DataInteractionSimulator _advanceProgress]):
(-[DataInteractionSimulator liftPreviews]):

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@221907 268f45cc-cd09-0410-ab3c-d52691b4dbfc

17 files changed:
Source/WebCore/ChangeLog
Source/WebCore/dom/DataTransfer.cpp
Source/WebCore/dom/DataTransfer.h
Source/WebCore/page/DragController.cpp
Source/WebCore/platform/DragItem.h
Source/WebKit/ChangeLog
Source/WebKit/UIProcess/ios/DragDropInteractionState.h
Source/WebKit/UIProcess/ios/DragDropInteractionState.mm
Source/WebKitLegacy/mac/ChangeLog
Source/WebKitLegacy/mac/WebView/WebView.mm
Source/WebKitLegacy/mac/WebView/WebViewData.h
Tools/ChangeLog
Tools/TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj
Tools/TestWebKitAPI/Tests/WebKitCocoa/DataTransfer-setDragImage.html [new file with mode: 0644]
Tools/TestWebKitAPI/Tests/ios/DataInteractionTests.mm
Tools/TestWebKitAPI/ios/DataInteractionSimulator.h
Tools/TestWebKitAPI/ios/DataInteractionSimulator.mm

index 258863a25fb544ff7049b9129e59361f81143778..8b4006b9d6dc99123e8e6dc309a900544a9d0347 100644 (file)
@@ -1,3 +1,43 @@
+2017-09-11  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        [iOS DnD] Support DataTransfer.setDragImage when starting a drag on iOS
+        https://bugs.webkit.org/show_bug.cgi?id=176721
+        <rdar://problem/34373660>
+
+        Reviewed by Tim Horton.
+
+        Adds support for setting the drag lift preview frame in the case where DataTransfer.setDragImage is being used
+        to override the default drag preview. Currently, the frame of the drag preview we supply in this case is the
+        same as the bounds of the source element in root view coordinates, but this means that any custom drag image
+        the page supplies will be stretched to fill the frame of the source element. Instead, when handling a DHTML drag,
+        position the lift and cancel drag previews relative to the event location, respecting any drag offset specified
+        in setDragImage. The size of this preview matches the size of the drag image source (since this is all in root
+        view coordinates, this means the drag preview will also enlarge if the user pinches to zoom in). If a
+        disconnected image source element was provided, then we just fall back to the image size.
+
+        Additionally, renames DragItem's elementBounds to dragPreviewFrameInRootViewCoordinates to better reflect the
+        purpose of this variable. This patch also introduces API test plumbing to grab targeted drag previews from the
+        drag interaction delegate (i.e. WKContentView), and uses this in a new API test that checks the frame of the
+        resulting UITargetedDragPreview after initiating a drag in various circumstances (see changes in Tools/ for more
+        detail).
+
+        Test: DataInteractionTests.DragLiftPreviewDataTransferSetDragImage
+
+        * dom/DataTransfer.cpp:
+        (WebCore::DataTransfer::dragImageElement const):
+        * dom/DataTransfer.h:
+        * page/DragController.cpp:
+        (WebCore::dragLocForDHTMLDrag):
+
+        The logic to flip the y offset when computing the drag location is only relevant on Mac, but currently, this is
+        guarded by #if PLATFORM(COCOA), which causes the y offset to shift the drag image in the opposite direction on
+        iOS. To fix this, simply change the platform define to Mac.
+
+        (WebCore::DragController::doSystemDrag):
+        * platform/DragItem.h:
+        (WebCore::DragItem::encode const):
+        (WebCore::DragItem::decode):
+
 2017-09-11  Wenson Hsieh  <wenson_hsieh@apple.com>
 
         [iOS WK2] Support tapping to add items to the current drag session in web content
index e238045628ce0a182b4c5b55d57fc65b21f4ff5b..0e6b57f18d0ff14fe49fcc850f199149fa5afead 100644 (file)
@@ -303,6 +303,11 @@ void DataTransfer::updateDragImage()
     m_pasteboard->setDragImage(WTFMove(computedImage), computedHotSpot);
 }
 
+RefPtr<Element> DataTransfer::dragImageElement() const
+{
+    return m_dragImageElement;
+}
+
 #if !PLATFORM(MAC)
 
 DragImageRef DataTransfer::createDragImage(IntPoint& location) const
index be23127932078907ea1e8261a4a01370fb89e5c8..675c6e2731690c5a43d042d82e483b4ed9a61852 100644 (file)
@@ -92,6 +92,7 @@ public:
     void setDragHasStarted() { m_shouldUpdateDragImage = true; }
     DragImageRef createDragImage(IntPoint& dragLocation) const;
     void updateDragImage();
+    RefPtr<Element> dragImageElement() const;
 #endif
 
 private:
index a1d1e574acd33da772a4300d5db4bdb908017aa6..ef566795aad0bd04338bd38df66045dfa9f63646 100644 (file)
@@ -830,7 +830,7 @@ static void selectElement(Element& element)
 static IntPoint dragLocForDHTMLDrag(const IntPoint& mouseDraggedPoint, const IntPoint& dragOrigin, const IntPoint& dragImageOffset, bool isLinkImage)
 {
     // dragImageOffset is the cursor position relative to the lower-left corner of the image.
-#if PLATFORM(COCOA)
+#if PLATFORM(MAC)
     // We add in the Y dimension because we are a flipped view, so adding moves the image down.
     const int yOffset = dragImageOffset.y();
 #else
@@ -1228,12 +1228,31 @@ void DragController::doSystemDrag(DragImage image, const IntPoint& dragLoc, cons
     DragItem item;
     item.image = WTFMove(image);
     item.sourceAction = state.type;
-    item.eventPositionInContentCoordinates = viewProtector->rootViewToContents(frame.view()->contentsToRootView(eventPos));
-    item.dragLocationInContentCoordinates = viewProtector->rootViewToContents(frame.view()->contentsToRootView(dragLoc));
+
+    auto eventPositionInRootViewCoordinates = frame.view()->contentsToRootView(eventPos);
+    auto dragLocationInRootViewCoordinates = frame.view()->contentsToRootView(dragLoc);
+    item.eventPositionInContentCoordinates = viewProtector->rootViewToContents(eventPositionInRootViewCoordinates);
+    item.dragLocationInContentCoordinates = viewProtector->rootViewToContents(dragLocationInRootViewCoordinates);
     item.eventPositionInWindowCoordinates = frame.view()->contentsToWindow(item.eventPositionInContentCoordinates);
     item.dragLocationInWindowCoordinates = frame.view()->contentsToWindow(item.dragLocationInContentCoordinates);
     if (auto element = state.source) {
-        item.elementBounds = element->boundsInRootViewSpace();
+        auto dataTransferImageElement = state.dataTransfer->dragImageElement();
+        if (state.type == DragSourceActionDHTML) {
+            // If the drag image has been customized, fall back to positioning the preview relative to the drag event location.
+            IntSize dragPreviewSize;
+            if (dataTransferImageElement)
+                dragPreviewSize = dataTransferImageElement->boundsInRootViewSpace().size();
+            else {
+                dragPreviewSize = dragImageSize(item.image.get());
+                if (auto* page = frame.page())
+                    dragPreviewSize.scale(1 / page->deviceScaleFactor());
+            }
+            item.dragPreviewFrameInRootViewCoordinates = { dragLocationInRootViewCoordinates, WTFMove(dragPreviewSize) };
+        } else {
+            // We can position the preview using the bounds of the drag source element.
+            item.dragPreviewFrameInRootViewCoordinates = element->boundsInRootViewSpace();
+        }
+
         RefPtr<Element> link;
         if (element->isLink())
             link = element;
index 28f67c679ee7cbd5e4fe6723f7f490a105659e20..b0faa0369a1815469b5696a30e86cd78ac50c03b 100644 (file)
@@ -46,7 +46,7 @@ struct DragItem final {
     IntPoint dragLocationInWindowCoordinates;
     String title;
     URL url;
-    IntRect elementBounds;
+    IntRect dragPreviewFrameInRootViewCoordinates;
 
     PasteboardWriterData data;
 
@@ -60,7 +60,7 @@ void DragItem::encode(Encoder& encoder) const
     // FIXME(173815): We should encode and decode PasteboardWriterData and platform drag image data
     // here too, as part of moving off of the legacy dragging codepath.
     encoder.encodeEnum(sourceAction);
-    encoder << imageAnchorPoint << eventPositionInContentCoordinates << dragLocationInContentCoordinates << eventPositionInWindowCoordinates << dragLocationInWindowCoordinates << title << url << elementBounds;
+    encoder << imageAnchorPoint << eventPositionInContentCoordinates << dragLocationInContentCoordinates << eventPositionInWindowCoordinates << dragLocationInWindowCoordinates << title << url << dragPreviewFrameInRootViewCoordinates;
     bool hasIndicatorData = image.hasIndicatorData();
     encoder << hasIndicatorData;
     if (hasIndicatorData)
@@ -86,7 +86,7 @@ bool DragItem::decode(Decoder& decoder, DragItem& result)
         return false;
     if (!decoder.decode(result.url))
         return false;
-    if (!decoder.decode(result.elementBounds))
+    if (!decoder.decode(result.dragPreviewFrameInRootViewCoordinates))
         return false;
     bool hasIndicatorData;
     if (!decoder.decode(hasIndicatorData))
index e111b72b802a9b91396e283397eec221df9fc7da..76105801827d6aaf6d090be4566a10dbba480b74 100644 (file)
@@ -1,3 +1,18 @@
+2017-09-11  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        [iOS DnD] Support DataTransfer.setDragImage when starting a drag on iOS
+        https://bugs.webkit.org/show_bug.cgi?id=176721
+        <rdar://problem/34373660>
+
+        Reviewed by Tim Horton.
+
+        Rename elementBounds => dragPreviewFrameInRootViewCoordinates.
+
+        * UIProcess/ios/DragDropInteractionState.h:
+        * UIProcess/ios/DragDropInteractionState.mm:
+        (WebKit::DragDropInteractionState::previewForDragItem const):
+        (WebKit::DragDropInteractionState::stageDragItem):
+
 2017-09-11  Tim Horton  <timothy_horton@apple.com>
 
         REGRESSION (r221272): WKWebView gets stuck zoomed in if the web process crashes
index 3b1c329b72a42369f8400330f57fecbb1affbdbb..3b784f75ae69f9c60796adfad8fdc1aa78b1989b 100644 (file)
@@ -46,7 +46,7 @@ namespace WebKit {
 struct DragSourceState {
     WebCore::DragSourceAction action { WebCore::DragSourceActionNone };
     CGPoint adjustedOrigin { CGPointZero };
-    CGRect elementBounds { CGRectZero };
+    CGRect dragPreviewFrameInRootViewCoordinates { CGRectZero };
     RetainPtr<UIImage> image;
     std::optional<WebCore::TextIndicatorData> indicatorData;
     String linkTitle;
index cd30d2bfaf781b2f81fd4e4da168e22da26aedda..5943f1ec5fbc3e932a8ad0ee3f1df3bc193cffac 100644 (file)
@@ -159,10 +159,8 @@ UITargetedDragPreview *DragDropInteractionState::previewForDragItem(UIDragItem *
         return nil;
 
     auto& source = foundSource.value();
-    if (shouldUseDragImageToCreatePreviewForDragSource(source)) {
-        Vector<FloatRect> emptyClippingRects;
-        return createTargetedDragPreview(source.image.get(), contentView, previewContainer, source.elementBounds, emptyClippingRects, nil);
-    }
+    if (shouldUseDragImageToCreatePreviewForDragSource(source))
+        return createTargetedDragPreview(source.image.get(), contentView, previewContainer, source.dragPreviewFrameInRootViewCoordinates, { }, nil);
 
     if (shouldUseTextIndicatorToCreatePreviewForDragSource(source)) {
         auto indicator = source.indicatorData.value();
@@ -203,7 +201,7 @@ void DragDropInteractionState::stageDragItem(const DragItem& item, UIImage *drag
     m_stagedDragSource = {{
         static_cast<DragSourceAction>(item.sourceAction),
         item.eventPositionInContentCoordinates,
-        item.elementBounds,
+        item.dragPreviewFrameInRootViewCoordinates,
         dragImage,
         item.image.indicatorData(),
         item.title.isEmpty() ? nil : (NSString *)item.title,
index 856a1aee981a9d6f0431d1f4c5af0ca9ad2a2c70..dcab4462fcb74de89ed510f87d770686de4e420d 100644 (file)
@@ -1,3 +1,20 @@
+2017-09-11  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        [iOS DnD] Support DataTransfer.setDragImage when starting a drag on iOS
+        https://bugs.webkit.org/show_bug.cgi?id=176721
+        <rdar://problem/34373660>
+
+        Reviewed by Tim Horton.
+
+        Rename elementBounds => dragPreviewFrameInRootViewCoordinates. (Note: Unfortunately, _draggedElementBounds in
+        WebViewPrivate.h can't be renamed yet, due to binary and SDK compatibility with UIKit).
+
+        * WebView/WebView.mm:
+        (-[WebView _startDrag:]):
+        (-[WebView _draggedElementBounds]):
+        (-[WebView _endedDataInteraction:global:]):
+        * WebView/WebViewData.h:
+
 2017-09-11  Andy Estes  <aestes@apple.com>
 
         [Mac] Upstream QTKit-related WebKitSystemInterface functions
index c7233be1332db8b5fae7787c9a68e613f4ca3bf9..fb760a2ca488756a0d77c7924ddfc8b648346d4a 100644 (file)
@@ -1812,7 +1812,7 @@ static void WebKitInitializeGamepadProviderIfNecessary()
         _private->textIndicatorData = adoptNS([[WebUITextIndicatorData alloc] initWithImage:image scale:_private->page->deviceScaleFactor()]);
     _private->draggedLinkURL = dragItem.url.isEmpty() ? nil : (NSURL *)dragItem.url;
     _private->draggedLinkTitle = dragItem.title.isEmpty() ? nil : (NSString *)dragItem.title;
-    _private->draggedElementBounds = dragItem.elementBounds;
+    _private->dragPreviewFrameInRootViewCoordinates = dragItem.dragPreviewFrameInRootViewCoordinates;
     _private->dragSourceAction = static_cast<WebDragSourceAction>(dragItem.sourceAction);
 }
 
@@ -1847,7 +1847,7 @@ static void WebKitInitializeGamepadProviderIfNecessary()
 
 - (CGRect)_draggedElementBounds
 {
-    return _private->draggedElementBounds;
+    return _private->dragPreviewFrameInRootViewCoordinates;
 }
 
 - (WebUITextIndicatorData *)_getDataInteractionData
@@ -1903,7 +1903,7 @@ static void WebKitInitializeGamepadProviderIfNecessary()
     WebThreadLock();
     _private->page->dragController().dragEnded();
     _private->dataOperationTextIndicator = nullptr;
-    _private->draggedElementBounds = CGRectNull;
+    _private->dragPreviewFrameInRootViewCoordinates = CGRectNull;
     _private->dragSourceAction = WebDragSourceActionNone;
     _private->draggedLinkTitle = nil;
     _private->draggedLinkURL = nil;
index 3f214ca83151d033ebabaebc9d3ab4130d103f9a..5a73f1ac6ab2a9926519c72befc2e6daba93c749 100644 (file)
@@ -301,7 +301,7 @@ private:
 #if ENABLE(DATA_INTERACTION)
     RetainPtr<WebUITextIndicatorData> textIndicatorData;
     RetainPtr<WebUITextIndicatorData> dataOperationTextIndicator;
-    CGRect draggedElementBounds;
+    CGRect dragPreviewFrameInRootViewCoordinates;
     WebDragSourceAction dragSourceAction;
     RetainPtr<NSURL> draggedLinkURL;
     RetainPtr<NSString> draggedLinkTitle;
index 0f8fc2e491055b705f675ca0b4b9daebb4e441c1..4576efd64fd2caf6b799c9611897ea6f6046628f 100644 (file)
@@ -1,3 +1,32 @@
+2017-09-11  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        [iOS DnD] Support DataTransfer.setDragImage when starting a drag on iOS
+        https://bugs.webkit.org/show_bug.cgi?id=176721
+        <rdar://problem/34373660>
+
+        Reviewed by Tim Horton.
+
+        In DataInteractionSimulator, ask the UIDragInteractionDelegate (WKContentView) for targeted drag previews after
+        retrieving UIDragItems during a lift. Remember these previews after the drag and drop simulation is complete, so
+        API tests (currently, just DragLiftPreviewDataTransferSetDragImage) can verify that the generated drag previews
+        are as expected.
+
+        * TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
+        * TestWebKitAPI/Tests/WebKitCocoa/DataTransfer-setDragImage.html: Added.
+
+        Adds a new test harness containing 5 draggable elements, each generating a drag image using different codepaths.
+        DragLiftPreviewDataTransferSetDragImage uses this page to check that the frame of the targeted drag preview in
+        each scenario is as expected.
+
+        * TestWebKitAPI/Tests/ios/DataInteractionTests.mm:
+        (checkCGRectIsEqualToCGRectWithLogging):
+        (TestWebKitAPI::TEST):
+        * TestWebKitAPI/ios/DataInteractionSimulator.h:
+        * TestWebKitAPI/ios/DataInteractionSimulator.mm:
+        (-[DataInteractionSimulator _resetSimulatedState]):
+        (-[DataInteractionSimulator _advanceProgress]):
+        (-[DataInteractionSimulator liftPreviews]):
+
 2017-09-11  Tim Horton  <timothy_horton@apple.com>
 
         REGRESSION (r221272): WKWebView gets stuck zoomed in if the web process crashes
index edcb243e27d49464f1aa58e3ff77c928d2773730..d738cb16d2a5b54bec600793b5bba3da30ae7575 100644 (file)
                F46A095B1ED8A6E600D4AA55 /* gif-and-file-input.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = F47D30ED1ED28A6C000482E1 /* gif-and-file-input.html */; };
                F47728991E4AE3C1007ABF6A /* full-page-contenteditable.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = F47728981E4AE3AD007ABF6A /* full-page-contenteditable.html */; };
                F4856CA31E649EA8009D7EE7 /* attachment-element.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = F4856CA21E6498A8009D7EE7 /* attachment-element.html */; };
+               F486B1D01F67952300F34BDD /* DataTransfer-setDragImage.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = F486B1CF1F6794FF00F34BDD /* DataTransfer-setDragImage.html */; };
                F4A32EC41F05F3850047C544 /* dragstart-change-selection-offscreen.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = F4A32EC31F05F3780047C544 /* dragstart-change-selection-offscreen.html */; };
                F4A32ECB1F0643370047C544 /* contenteditable-in-iframe.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = F4A32ECA1F0642F40047C544 /* contenteditable-in-iframe.html */; };
                F4AB578A1F65165400DB0DA1 /* custom-draggable-div.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = F4AB57891F65164B00DB0DA1 /* custom-draggable-div.html */; };
                                7AEAD4811E20122700416EFE /* CrossPartitionFileSchemeAccess.html in Copy Resources */,
                                F4AB578A1F65165400DB0DA1 /* custom-draggable-div.html in Copy Resources */,
                                290F4275172A221C00939FF0 /* custom-protocol-sync-xhr.html in Copy Resources */,
+                               F486B1D01F67952300F34BDD /* DataTransfer-setDragImage.html in Copy Resources */,
                                F4512E131F60C44600BB369E /* DataTransferItem-getAsEntry.html in Copy Resources */,
                                C07E6CB213FD73930038B22B /* devicePixelRatio.html in Copy Resources */,
                                0799C34B1EBA3301003B7532 /* disableGetUserMedia.html in Copy Resources */,
                F47D30EB1ED28619000482E1 /* apple.gif */ = {isa = PBXFileReference; lastKnownFileType = image.gif; path = apple.gif; sourceTree = "<group>"; };
                F47D30ED1ED28A6C000482E1 /* gif-and-file-input.html */ = {isa = PBXFileReference; lastKnownFileType = text.html; path = "gif-and-file-input.html"; sourceTree = "<group>"; };
                F4856CA21E6498A8009D7EE7 /* attachment-element.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = "attachment-element.html"; sourceTree = "<group>"; };
+               F486B1CF1F6794FF00F34BDD /* DataTransfer-setDragImage.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = "DataTransfer-setDragImage.html"; sourceTree = "<group>"; };
                F493247C1F44DF8D006F4336 /* UIKitSPI.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = UIKitSPI.h; sourceTree = "<group>"; };
                F4A32EC31F05F3780047C544 /* dragstart-change-selection-offscreen.html */ = {isa = PBXFileReference; lastKnownFileType = text.html; path = "dragstart-change-selection-offscreen.html"; sourceTree = "<group>"; };
                F4A32ECA1F0642F40047C544 /* contenteditable-in-iframe.html */ = {isa = PBXFileReference; lastKnownFileType = text.html; path = "contenteditable-in-iframe.html"; sourceTree = "<group>"; };
                                A16F66B91C40EA2000BD4D24 /* ContentFiltering.html */,
                                5C2936941D5BFD1900DEAB1E /* CookieMessage.html */,
                                F4AB57891F65164B00DB0DA1 /* custom-draggable-div.html */,
+                               F486B1CF1F6794FF00F34BDD /* DataTransfer-setDragImage.html */,
                                F4512E121F60C43400BB369E /* DataTransferItem-getAsEntry.html */,
                                0799C34A1EBA32F4003B7532 /* disableGetUserMedia.html */,
                                F41AB99E1EF4692C0083FA08 /* div-and-large-image.html */,
diff --git a/Tools/TestWebKitAPI/Tests/WebKitCocoa/DataTransfer-setDragImage.html b/Tools/TestWebKitAPI/Tests/WebKitCocoa/DataTransfer-setDragImage.html
new file mode 100644 (file)
index 0000000..4d3b970
--- /dev/null
@@ -0,0 +1,70 @@
+<!DOCTYPE html>
+<html>
+<meta name="viewport" content="width=device-width">
+<head>
+<style>
+    body, html {
+        width: 100%;
+        height: 100%;
+    }
+
+    body {
+        margin: 0;
+    }
+
+    #red {
+        background-color: red;
+        width: 300px;
+        height: 100px;
+    }
+
+    #green {
+        background-color: green;
+        width: 300px;
+        height: 100px;
+    }
+
+    #blue {
+        background-color: blue;
+        width: 300px;
+        height: 100px;
+    }
+
+    #cyan {
+        background-color: cyan;
+        width: 300px;
+        height: 100px;
+    }
+</style>
+</head>
+
+<div id="red" draggable="true"></div>
+<div id="green" draggable="true"></div>
+<div id="blue" draggable="true"></div>
+<div id="cyan" draggable="true"></div>
+<img id="icon" src="icon.png"></img>
+<div><code>To manually test, attempt to drag each of the colored blocks.</code></div>
+
+<script>
+red.addEventListener("dragstart", event => {
+    event.dataTransfer.setDragImage(icon, 0, 0);
+    event.dataTransfer.setData("text/plain", "red");
+});
+
+green.addEventListener("dragstart", event => {
+    event.dataTransfer.setDragImage(icon, 100, 100);
+    event.dataTransfer.setData("text/plain", "green");
+});
+
+blue.addEventListener("dragstart", event => {
+    let image = new Image();
+    image.src = icon.src;
+    event.dataTransfer.setDragImage(image, 0, 0);
+    event.dataTransfer.setData("text/plain", "blue");
+});
+
+cyan.addEventListener("dragstart", event => {
+    event.dataTransfer.setData("text/plain", "cyan");
+});
+</script>
+</html>
index 5d55c01454871206ab0971c1da629cfd0135583e..574e3eb0e48f62a13a5aa199051dc7ab0d4e2ef5 100644 (file)
@@ -102,6 +102,14 @@ static NSValue *makeCGRectValue(CGFloat x, CGFloat y, CGFloat width, CGFloat hei
     return [NSValue valueWithCGRect:CGRectMake(x, y, width, height)];
 }
 
+static void checkCGRectIsEqualToCGRectWithLogging(CGRect expected, CGRect observed)
+{
+    BOOL isEqual = CGRectEqualToRect(expected, observed);
+    EXPECT_TRUE(isEqual);
+    if (!isEqual)
+        NSLog(@"Expected: %@ but observed: %@", NSStringFromCGRect(expected), NSStringFromCGRect(observed));
+}
+
 static void checkSelectionRectsWithLogging(NSArray *expected, NSArray *observed)
 {
     if (![expected isEqualToArray:observed])
@@ -1409,6 +1417,33 @@ TEST(DataInteractionTests, AdditionalLinkAndImageIntoContentEditable)
     EXPECT_WK_STREQ("https://www.apple.com/", [webView stringByEvaluatingJavaScript:@"editor.querySelector('a').href"]);
 }
 
+TEST(DataInteractionTests, DragLiftPreviewDataTransferSetDragImage)
+{
+    auto webView = adoptNS([[TestWKWebView alloc] initWithFrame:CGRectMake(0, 0, 320, 500)]);
+    [webView synchronouslyLoadTestPageNamed:@"DataTransfer-setDragImage"];
+    auto simulator = adoptNS([[DataInteractionSimulator alloc] initWithWebView:webView.get()]);
+
+    // Use DataTransfer.setDragImage to specify an existing image element in the DOM.
+    [simulator runFrom:CGPointMake(100, 50) to:CGPointMake(250, 50)];
+    checkCGRectIsEqualToCGRectWithLogging({{100, 50}, {215, 174}}, [simulator liftPreviews][0].view.frame);
+
+    // Use DataTransfer.setDragImage to specify an existing image element in the DOM, with x and y offsets.
+    [simulator runFrom:CGPointMake(10, 150) to:CGPointMake(250, 150)];
+    checkCGRectIsEqualToCGRectWithLogging({{-90, 50}, {215, 174}}, [simulator liftPreviews][0].view.frame);
+
+    // Use DataTransfer.setDragImage to specify a newly created Image, disconnected from the DOM.
+    [simulator runFrom:CGPointMake(100, 250) to:CGPointMake(250, 250)];
+    checkCGRectIsEqualToCGRectWithLogging({{100, 250}, {215, 174}}, [simulator liftPreviews][0].view.frame);
+
+    // Don't use DataTransfer.setDragImage and fall back to snapshotting the dragged element.
+    [simulator runFrom:CGPointMake(50, 350) to:CGPointMake(250, 350)];
+    checkCGRectIsEqualToCGRectWithLogging({{0, 300}, {300, 100}}, [simulator liftPreviews][0].view.frame);
+
+    // Perform a normal drag on an image.
+    [simulator runFrom:CGPointMake(50, 450) to:CGPointMake(250, 450)];
+    checkCGRectIsEqualToCGRectWithLogging({{0, 400}, {215, 174}}, [simulator liftPreviews][0].view.frame);
+}
+
 } // namespace TestWebKitAPI
 
 #endif // ENABLE(DATA_INTERACTION)
index 55f67855b1c37df43b52cec06b97b1c7b798386f..710ec76c1c14ee6f777aed845f1e25fe854b920e 100644 (file)
@@ -136,6 +136,7 @@ typedef NS_ENUM(NSInteger, DataInteractionPhase) {
 
     RetainPtr<NSMutableDictionary<NSNumber *, NSValue *>>_remainingAdditionalItemRequestLocationsByProgress;
     RetainPtr<NSMutableArray<NSValue *>>_queuedAdditionalItemRequestLocations;
+    RetainPtr<NSMutableArray<UITargetedDragPreview *>> _liftPreviews;
 
     bool _isDoneWaitingForInputSession;
     BOOL _shouldPerformOperation;
@@ -165,6 +166,7 @@ typedef NS_ENUM(NSInteger, DataInteractionPhase) {
 @property (nonatomic, readonly) NSArray *finalSelectionRects;
 @property (nonatomic, readonly) DataInteractionPhase phase;
 @property (nonatomic, readonly) CGRect lastKnownDragCaretRect;
+@property (nonatomic, readonly) NSArray<UITargetedDragPreview *> *liftPreviews;
 
 @end
 
index 51eeddae9b5dcfea95f278ada6bf765ea3c0af79..f1df7868c63d2c0cdc3ba16345ceece85efee31f 100644 (file)
@@ -343,6 +343,7 @@ static NSArray *dataInteractionEventNames()
     _lastKnownDragCaretRect = CGRectZero;
     _remainingAdditionalItemRequestLocationsByProgress = nil;
     _queuedAdditionalItemRequestLocations = adoptNS([[NSMutableArray alloc] init]);
+    _liftPreviews = adoptNS([[NSMutableArray alloc] init]);
 }
 
 - (NSArray *)observedEventNames
@@ -501,6 +502,8 @@ static NSArray *dataInteractionEventNames()
             [itemProviders addObject:item.itemProvider];
             UITargetedDragPreview *liftPreview = [[_webView dragInteractionDelegate] dragInteraction:[_webView dragInteraction] previewForLiftingItem:item session:_dragSession.get()];
             EXPECT_TRUE(!!liftPreview);
+            if (liftPreview)
+                [_liftPreviews addObject:liftPreview];
         }
 
         _dropSession = adoptNS([[MockDropSession alloc] initWithProviders:itemProviders location:self._currentLocation window:[_webView window] allowMove:self.shouldAllowMoveOperation]);
@@ -572,6 +575,11 @@ static NSArray *dataInteractionEventNames()
     return _phase;
 }
 
+- (NSArray<UITargetedDragPreview *> *)liftPreviews
+{
+    return _liftPreviews.get();
+}
+
 - (CGRect)lastKnownDragCaretRect
 {
     return _lastKnownDragCaretRect;