[Attachment Support] Add plumbing for starting a drag with promised blob data
authorwenson_hsieh@apple.com <wenson_hsieh@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 4 Jan 2018 05:58:03 +0000 (05:58 +0000)
committerwenson_hsieh@apple.com <wenson_hsieh@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 4 Jan 2018 05:58:03 +0000 (05:58 +0000)
https://bugs.webkit.org/show_bug.cgi?id=181201

Reviewed by Tim Horton.

Source/WebCore:

Adds logic to allow dragging an attachment element as a file by sending promised blob information to the UI
process. See comments below for more detail.

The only change in behavior is that dragging an attachment element will no longer write web content and injected
bundle data to the pasteboard if the attachment element's file attribute is nonnull. This will cause one
existing WK1 layout test to fail, but will otherwise not affect any attachment editing clients. On iOS,
attachment elements in the Mail viewer can be dragged, but each attachment's file is null, so we fall back to
current behavior; on macOS, Mail currently overrides the drag completely, beginning at -mouseDown:, so this
doesn't make a difference to macOS Mail either.

* editing/Editor.h:
* editing/cocoa/EditorCocoa.mm:
(WebCore::Editor::getPasteboardTypesAndDataForAttachment):

Add a helper method to retrieve an attachment element as web archive data, for moving attachments within the
same document. Also gives the injected editor bundle a chance to supply custom pasteboard types.

* loader/EmptyClients.cpp:
* page/DragClient.h:
(WebCore::DragClient::prepareToDragPromisedBlob):

Add new DragClient methods to send information about a promised blob to the UI process.

* page/DragController.cpp:
(WebCore::DragController::startDrag):

Call dragAttachmentElement when starting a drag on an attachment element.

(WebCore::DragController::dragAttachmentElement):

Try to begin dragging a given attachment element, propagating promised blob information to the client layers.
Returns true iff the attachment is backed by blob data (i.e. the file is nonnull).

* platform/PromisedBlobInfo.h:

Add a list of additional types and data to PromisedBlobInfo. In addition to the promised blob info, this would
allow injected bundle data and other private types alongside the main attachment data on the pasteboard.

Source/WebKit:

Add boilerplate plumbing for PrepareToDragPromisedBlob, which delivers blob promises to the UI process when
dragging, respectively.

* Scripts/webkit/messages.py:
(headers_for_type):
* Shared/WebCoreArgumentCoders.cpp:
(IPC::encodeTypesAndData):
(IPC::decodeTypesAndData):
(IPC::ArgumentCoder<PasteboardWebContent>::encode):
(IPC::ArgumentCoder<PasteboardWebContent>::decode):
(IPC::ArgumentCoder<PasteboardImage>::encode):
(IPC::ArgumentCoder<PasteboardImage>::decode):
(IPC::ArgumentCoder<PromisedBlobInfo>::encode):
(IPC::ArgumentCoder<PromisedBlobInfo>::decode):

Add IPC support PromisedBlobInfo's additionalTypes and additionalData.

(IPC::encodeClientTypesAndData): Deleted.
(IPC::decodeClientTypesAndData): Deleted.

Rename these helper functions and move them to the top of the file.

* UIProcess/Cocoa/WebViewImpl.h:
* UIProcess/Cocoa/WebViewImpl.mm:
(WebKit::WebViewImpl::prepareToDragPromisedBlob):
* UIProcess/PageClient.h:
(WebKit::PageClient::prepareToDragPromisedBlob):
* UIProcess/WebPageProxy.cpp:
(WebKit::WebPageProxy::prepareToDragPromisedBlob):
* UIProcess/WebPageProxy.h:
* UIProcess/WebPageProxy.messages.in:
* UIProcess/ios/PageClientImplIOS.h:
* UIProcess/ios/PageClientImplIOS.mm:
(WebKit::PageClientImpl::prepareToDragPromisedBlob):
* UIProcess/ios/WKContentViewInteraction.h:
* UIProcess/ios/WKContentViewInteraction.mm:
(-[WKContentView _prepareToDragPromisedBlob:]):
* UIProcess/mac/PageClientImplMac.h:
* UIProcess/mac/PageClientImplMac.mm:
(WebKit::PageClientImpl::prepareToDragPromisedBlob):
* WebProcess/WebCoreSupport/WebDragClient.cpp:
(WebKit::WebDragClient::prepareToDragPromisedBlob):
* WebProcess/WebCoreSupport/WebDragClient.h:
* WebProcess/WebPage/WebPage.cpp:
(WebKit::WebPage::prepareToDragPromisedBlob):
* WebProcess/WebPage/WebPage.h:

Source/WebKitLegacy/mac:

Minor adjustment to account for a DragClient interface change. See WebCore ChangeLog for more details.

* WebCoreSupport/WebDragClient.h:

LayoutTests:

Remove a WK1 LayoutTest testing drag and drop of an attachment element into a contenteditable. This test no
longer passes because the implementation of attachment dragging on macOS is not yet implemented. Subsequent
patches will test this scenario once again, but as a WK2 macOS drag and drop API test.

* editing/pasteboard/drag-and-drop-attachment-contenteditable-expected.txt: Removed.
* editing/pasteboard/drag-and-drop-attachment-contenteditable.html: Removed.
* platform/gtk/TestExpectations:
* platform/ios/TestExpectations:
* platform/mac-wk2/TestExpectations:
* platform/win/TestExpectations:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@226393 268f45cc-cd09-0410-ab3c-d52691b4dbfc

36 files changed:
LayoutTests/ChangeLog
LayoutTests/editing/pasteboard/drag-and-drop-attachment-contenteditable-expected.txt [deleted file]
LayoutTests/editing/pasteboard/drag-and-drop-attachment-contenteditable.html [deleted file]
LayoutTests/platform/gtk/TestExpectations
LayoutTests/platform/ios/TestExpectations
LayoutTests/platform/mac-wk2/TestExpectations
LayoutTests/platform/win/TestExpectations
Source/WebCore/ChangeLog
Source/WebCore/editing/Editor.h
Source/WebCore/editing/cocoa/EditorCocoa.mm
Source/WebCore/loader/EmptyClients.cpp
Source/WebCore/page/DragClient.h
Source/WebCore/page/DragController.cpp
Source/WebCore/page/DragController.h
Source/WebCore/platform/PromisedBlobInfo.h
Source/WebKit/ChangeLog
Source/WebKit/Scripts/webkit/messages.py
Source/WebKit/Shared/WebCoreArgumentCoders.cpp
Source/WebKit/UIProcess/Cocoa/WebViewImpl.h
Source/WebKit/UIProcess/Cocoa/WebViewImpl.mm
Source/WebKit/UIProcess/PageClient.h
Source/WebKit/UIProcess/WebPageProxy.cpp
Source/WebKit/UIProcess/WebPageProxy.h
Source/WebKit/UIProcess/WebPageProxy.messages.in
Source/WebKit/UIProcess/ios/PageClientImplIOS.h
Source/WebKit/UIProcess/ios/PageClientImplIOS.mm
Source/WebKit/UIProcess/ios/WKContentViewInteraction.h
Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm
Source/WebKit/UIProcess/mac/PageClientImplMac.h
Source/WebKit/UIProcess/mac/PageClientImplMac.mm
Source/WebKit/WebProcess/WebCoreSupport/WebDragClient.cpp
Source/WebKit/WebProcess/WebCoreSupport/WebDragClient.h
Source/WebKit/WebProcess/WebPage/WebPage.cpp
Source/WebKit/WebProcess/WebPage/WebPage.h
Source/WebKitLegacy/mac/ChangeLog
Source/WebKitLegacy/mac/WebCoreSupport/WebDragClient.h

index 9e49347b61fba493303a06476d880ef019fd4deb..e68adc1e27be36121eb3f76d00277eaf93210964 100644 (file)
@@ -1,3 +1,21 @@
+2018-01-03  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        [Attachment Support] Add plumbing for starting a drag with promised blob data
+        https://bugs.webkit.org/show_bug.cgi?id=181201
+
+        Reviewed by Tim Horton.
+
+        Remove a WK1 LayoutTest testing drag and drop of an attachment element into a contenteditable. This test no
+        longer passes because the implementation of attachment dragging on macOS is not yet implemented. Subsequent
+        patches will test this scenario once again, but as a WK2 macOS drag and drop API test.
+
+        * editing/pasteboard/drag-and-drop-attachment-contenteditable-expected.txt: Removed.
+        * editing/pasteboard/drag-and-drop-attachment-contenteditable.html: Removed.
+        * platform/gtk/TestExpectations:
+        * platform/ios/TestExpectations:
+        * platform/mac-wk2/TestExpectations:
+        * platform/win/TestExpectations:
+
 2018-01-03  John Wilander  <wilander@apple.com>
 
         Storage Access API: Refactor XPC for access removal to go straight from the web process to the network process
diff --git a/LayoutTests/editing/pasteboard/drag-and-drop-attachment-contenteditable-expected.txt b/LayoutTests/editing/pasteboard/drag-and-drop-attachment-contenteditable-expected.txt
deleted file mode 100644 (file)
index 6e4f068..0000000
+++ /dev/null
@@ -1,13 +0,0 @@
-These tests dragging an attachment element into contenteditable
-
-On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
-
-
-
-
-PASS target.getElementsByTagName("attachment").length is 1
-PASS target.getElementsByTagName("attachment")[0].file.name is "apple.gif"
-PASS successfullyParsed is true
-
-TEST COMPLETE
-
diff --git a/LayoutTests/editing/pasteboard/drag-and-drop-attachment-contenteditable.html b/LayoutTests/editing/pasteboard/drag-and-drop-attachment-contenteditable.html
deleted file mode 100644 (file)
index 178803b..0000000
+++ /dev/null
@@ -1,59 +0,0 @@
-<!DOCTYPE html><!-- webkit-test-runner [ enableAttachmentElement=true ] -->
-<html>
-<head>
-<script src="../../resources/js-test.js"></script>
-<style>
-#target {
-    height: 200px;
-    width: 220px;
-    border: 5px solid blue;
-}
-</style>
-</head>
-<body onload="runTest()">
-<p id="description"></p>
-<div id="test1"><attachment id='attachment'></attachment></div>
-<div contenteditable=true id="target"></div>
-<div id="console"></div>
-
-<script>
-    description('These tests dragging an attachment element into contenteditable');
-var jsTestIsAsync = true;
-var target = document.getElementById('target');
-
-function contentInserted() {
-    shouldBe('target.getElementsByTagName("attachment").length', '1');
-    shouldBeEqualToString('target.getElementsByTagName("attachment")[0].file.name', 'apple.gif');
-    finishJSTest();
-}
-
-function setupAttachment() {
-    if (window.internals) {
-        var attachment = document.getElementById('attachment');
-        attachment.file = window.internals.createFile("resources/apple.gif");
-    }
-}
-
-function runTest() {
-    setupAttachment();
-    target.addEventListener('DOMNodeInserted', contentInserted, false);
-    if (!window.testRunner)
-        return;
-
-    e = document.getElementById("attachment");
-    x = e.offsetLeft + e.offsetWidth / 2;
-    y = e.offsetTop + e.offsetHeight / 2;
-
-    eventSender.mouseMoveTo(x, y);
-    eventSender.mouseDown();
-    eventSender.leapForward(100);
-    e = document.getElementById("target");
-    x = e.offsetLeft + e.offsetWidth / 2;
-    y = e.offsetTop + e.offsetHeight / 2;
-
-    eventSender.mouseMoveTo(x, y);
-    eventSender.mouseUp();
-}
-</script>
-</body>
-</html>
index 820009061917c7d2be9b35f12ca8609a9e591af3..6504c886a6a3d1e5ccbf324fa2744b16d210a600 100644 (file)
@@ -2111,7 +2111,6 @@ webkit.org/b/141959 http/tests/media/clearkey/clear-key-hls-aes128.html [ Crash
 webkit.org/b/142489 http/tests/media/video-play-waiting.html [ Timeout ]
 
 webkit.org/b/142820 fast/images/animated-gif-body-outside-viewport.html [ Timeout Pass ]
-webkit.org/b/143473 editing/pasteboard/drag-and-drop-attachment-contenteditable.html [ Timeout ]
 
 webkit.org/b/143477 fast/writing-mode/broken-ideograph-small-caps.html [ Timeout Pass ]
 
index dccc4a20b3bb984377ef24e96b3cfcf3bb45f004..b7c485f7e543cc54150caeefdf78724cebdbeb70 100644 (file)
@@ -985,7 +985,6 @@ editing/pasteboard/drop-file-svg.html [ Skip ]
 editing/pasteboard/drop-inputtext-acquires-style.html [ Skip ]
 editing/pasteboard/drop-link.html [ Skip ]
 editing/pasteboard/drop-text-events.html [ Skip ]
-editing/pasteboard/drag-and-drop-attachment-contenteditable.html [ Skip ]
 editing/pasteboard/drag-and-drop-image-contenteditable.html [ Skip ]
 editing/pasteboard/drag-and-drop-inputimage-contenteditable.html [ Skip ]
 editing/pasteboard/drag-and-drop-objectimage-contenteditable.html [ Skip ]
index a7937c878ebcbfad4ebb8277e24d2c1ab9dec155..4961b129acb655069018b55338d3969108c294b2 100644 (file)
@@ -89,7 +89,6 @@ editing/pasteboard/copy-standalone-image-crash.html
 editing/pasteboard/drag-and-drop-image-contenteditable.html
 editing/pasteboard/drag-and-drop-inputimage-contenteditable.html
 editing/pasteboard/drag-and-drop-objectimage-contenteditable.html
-editing/pasteboard/drag-and-drop-attachment-contenteditable.html
 editing/pasteboard/drag-drop-copy-content.html
 editing/pasteboard/drag-drop-dead-frame.html
 editing/pasteboard/drag-drop-input-textarea.html
index 096cd457df714c73d9092c8cf916e474fc319071..2f4a6c90de3fd9070a4e95d037a1af7998280839 100644 (file)
@@ -592,7 +592,6 @@ webkit.org/b/35013 svg/zoom/text/ [ Skip ]
 
 # TODO Attachment elements do not provide a custom renderer
 webkit.org/b/142260 fast/attachment [ Skip ]
-webkit.org/b/142260 editing/pasteboard/drag-and-drop-attachment-contenteditable.html [ Skip ]
 
 # TODO APNG is not supported
 webkit.org/b/142731 fast/images/animated-png.html [ Skip ]
index fcee36f3053fa452e50ad00bc210af706881c4fd..8ad51d92330ae0bcbea685fecc92a8b6f4e7dfcc 100644 (file)
@@ -1,3 +1,48 @@
+2018-01-03  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        [Attachment Support] Add plumbing for starting a drag with promised blob data
+        https://bugs.webkit.org/show_bug.cgi?id=181201
+
+        Reviewed by Tim Horton.
+
+        Adds logic to allow dragging an attachment element as a file by sending promised blob information to the UI
+        process. See comments below for more detail.
+
+        The only change in behavior is that dragging an attachment element will no longer write web content and injected
+        bundle data to the pasteboard if the attachment element's file attribute is nonnull. This will cause one
+        existing WK1 layout test to fail, but will otherwise not affect any attachment editing clients. On iOS,
+        attachment elements in the Mail viewer can be dragged, but each attachment's file is null, so we fall back to
+        current behavior; on macOS, Mail currently overrides the drag completely, beginning at -mouseDown:, so this
+        doesn't make a difference to macOS Mail either.
+
+        * editing/Editor.h:
+        * editing/cocoa/EditorCocoa.mm:
+        (WebCore::Editor::getPasteboardTypesAndDataForAttachment):
+
+        Add a helper method to retrieve an attachment element as web archive data, for moving attachments within the
+        same document. Also gives the injected editor bundle a chance to supply custom pasteboard types.
+
+        * loader/EmptyClients.cpp:
+        * page/DragClient.h:
+        (WebCore::DragClient::prepareToDragPromisedBlob):
+
+        Add new DragClient methods to send information about a promised blob to the UI process.
+
+        * page/DragController.cpp:
+        (WebCore::DragController::startDrag):
+
+        Call dragAttachmentElement when starting a drag on an attachment element.
+
+        (WebCore::DragController::dragAttachmentElement):
+
+        Try to begin dragging a given attachment element, propagating promised blob information to the client layers.
+        Returns true iff the attachment is backed by blob data (i.e. the file is nonnull).
+
+        * platform/PromisedBlobInfo.h:
+
+        Add a list of additional types and data to PromisedBlobInfo. In addition to the promised blob info, this would
+        allow injected bundle data and other private types alongside the main attachment data on the pasteboard.
+
 2018-01-03  Simon Fraser  <simon.fraser@apple.com>
 
         Remove the 'resolutionScale' parameter from ImageBufferDataCG get/putBytes
index ed99802d065665601194f9df93e78adae85eaa5d..e6a8967ed94b4f855bbc3a5c5549178aafa0dbd5 100644 (file)
@@ -509,6 +509,10 @@ public:
     WEBCORE_EXPORT void insertAttachment(const String& identifier, const AttachmentDisplayOptions&, const String& filename, Ref<SharedBuffer>&& data, std::optional<String> contentType = std::nullopt);
     void didInsertAttachmentElement(HTMLAttachmentElement&);
     void didRemoveAttachmentElement(HTMLAttachmentElement&);
+
+#if PLATFORM(COCOA)
+    void getPasteboardTypesAndDataForAttachment(HTMLAttachmentElement&, Vector<String>& outTypes, Vector<RefPtr<SharedBuffer>>& outData);
+#endif
 #endif
 
 private:
index 0d8db1775101fabeb215a5cfc789ec80eb869f19..fafe58922594f19da5a800d0dd651d3b9d4643df 100644 (file)
@@ -40,6 +40,7 @@
 #import "Frame.h"
 #import "FrameLoader.h"
 #import "FrameSelection.h"
+#import "HTMLAttachmentElement.h"
 #import "HTMLConverter.h"
 #import "HTMLImageElement.h"
 #import "HTMLSpanElement.h"
@@ -152,6 +153,20 @@ String Editor::selectionInHTMLFormat()
     return { };
 }
 
+#if ENABLE(ATTACHMENT_ELEMENT)
+
+void Editor::getPasteboardTypesAndDataForAttachment(HTMLAttachmentElement& attachment, Vector<String>& outTypes, Vector<RefPtr<SharedBuffer>>& outData)
+{
+    auto attachmentRange = Range::create(attachment.document(), { &attachment, Position::PositionIsBeforeAnchor }, { &attachment, Position::PositionIsAfterAnchor });
+    client()->getClientPasteboardDataForRange(attachmentRange.ptr(), outTypes, outData);
+    if (auto archive = LegacyWebArchive::create(attachmentRange.ptr())) {
+        outTypes.append(WebArchivePboardType);
+        outData.append(SharedBuffer::create(archive->rawDataRepresentation().get()));
+    }
+}
+
+#endif
+
 void Editor::writeSelectionToPasteboard(Pasteboard& pasteboard)
 {
     NSAttributedString *attributedString = attributedStringFromRange(*selectedRange());
index f24ad6aa0130ff8a460965808a00d8f71f9b60dc..15f2a40f3acb262718609fdd4f1500ffa3b33961 100644 (file)
@@ -134,6 +134,7 @@ class EmptyDragClient final : public DragClient {
     DragSourceAction dragSourceActionMaskForPoint(const IntPoint&) final { return DragSourceActionNone; }
     void startDrag(DragItem, DataTransfer&, Frame&) final { }
     void dragControllerDestroyed() final { }
+    void prepareToDragPromisedBlob(const PromisedBlobInfo&) final { }
 };
 
 #endif // ENABLE(DRAG_SUPPORT)
index 7cd62534e99955270056c92a4094674ff70b5c5a..6ff546eb68ef630566b1b5cffe1cde47b3773bb8 100644 (file)
@@ -37,6 +37,7 @@ class DataTransfer;
 class Element;
 class Frame;
 class Image;
+struct PromisedBlobInfo;
 
 class DragClient {
 public:
@@ -60,6 +61,8 @@ public:
     virtual void declareAndWriteDragImage(const String&, Element&, const URL&, const String&, Frame*) { }
 #endif
 
+    virtual void prepareToDragPromisedBlob(const PromisedBlobInfo&) { }
+
     virtual ~DragClient() = default;
 };
     
index b7e4643fb8f51fdf7175249cc2be9fd5b0416c26..b6ce6a2f618d69bd53f511b26bd06cf4dafaa0a4 100644 (file)
@@ -45,6 +45,7 @@
 #include "EditorClient.h"
 #include "ElementAncestorIterator.h"
 #include "EventHandler.h"
+#include "File.h"
 #include "FloatRect.h"
 #include "FocusController.h"
 #include "FrameLoadRequest.h"
@@ -68,6 +69,7 @@
 #include "PluginDocument.h"
 #include "PluginViewBase.h"
 #include "Position.h"
+#include "PromisedBlobInfo.h"
 #include "RenderAttachment.h"
 #include "RenderFileUploadControl.h"
 #include "RenderImage.h"
@@ -1067,7 +1069,8 @@ bool DragController::startDrag(Frame& src, const DragState& state, DragOperation
 
 #if ENABLE(ATTACHMENT_ELEMENT)
     if (is<HTMLAttachmentElement>(element) && m_dragSourceAction & DragSourceActionAttachment) {
-        auto* attachmentRenderer = downcast<HTMLAttachmentElement>(element).attachmentRenderer();
+        auto& attachment = downcast<HTMLAttachmentElement>(element);
+        auto* attachmentRenderer = attachment.attachmentRenderer();
         if (!attachmentRenderer)
             return false;
 
@@ -1075,7 +1078,7 @@ bool DragController::startDrag(Frame& src, const DragState& state, DragOperation
         auto previousSelection = src.selection().selection();
         if (hasData == HasNonDefaultPasteboardData::No) {
             selectElement(element);
-            if (src.editor().client()) {
+            if (!dragAttachmentElement(src, attachment) && src.editor().client()) {
 #if PLATFORM(COCOA)
                 // Otherwise, if no file URL is specified, call out to the injected bundle to populate the pasteboard with data.
                 auto& editor = src.editor();
@@ -1271,6 +1274,31 @@ bool DragController::shouldUseCachedImageForDragImage(const Image& image) const
 #endif
 }
 
+#if ENABLE(ATTACHMENT_ELEMENT)
+
+bool DragController::dragAttachmentElement(Frame& frame, HTMLAttachmentElement& attachment)
+{
+    if (!attachment.file())
+        return false;
+
+    Vector<String> additionalTypes;
+    Vector<RefPtr<SharedBuffer>> additionalData;
+#if PLATFORM(COCOA)
+    if (frame.editor().client())
+        frame.editor().getPasteboardTypesAndDataForAttachment(attachment, additionalTypes, additionalData);
+#endif
+
+    auto& file = *attachment.file();
+    auto blobURL = file.url().string();
+    bool isFileBacked = !file.path().isEmpty();
+    auto blobType = isFileBacked ? PromisedBlobType::FileBacked : PromisedBlobType::DataBacked;
+    m_client.prepareToDragPromisedBlob({ blobURL, file.type(), file.name(), blobType, additionalTypes, additionalData });
+
+    return true;
+}
+
+#endif // ENABLE(ATTACHMENT_ELEMENT)
+
 #endif // ENABLE(DRAG_SUPPORT)
 
 } // namespace WebCore
index 32ce33851894a97ac8be4df09ab0d1615020567d..d77e648182a8cb9ae9f3a1f2b2dd95cc8a981413 100644 (file)
@@ -40,6 +40,7 @@ class DragData;
 class Element;
 class Frame;
 class FrameSelection;
+class HTMLAttachmentElement;
 class HTMLInputElement;
 class IntRect;
 class Page;
@@ -129,6 +130,10 @@ struct DragState;
 
         void cleanupAfterSystemDrag();
         void declareAndWriteDragImage(DataTransfer&, Element&, const URL&, const String& label);
+
+#if ENABLE(ATTACHMENT_ELEMENT)
+        bool dragAttachmentElement(Frame&, HTMLAttachmentElement&);
+#endif
         Page& m_page;
         DragClient& m_client;
 
index eb348592991a8c2a3814ef1bfc89be19cba35eb8..ae846fc5711dd75b3b7f12b230750f570d60ce80 100644 (file)
@@ -25,7 +25,7 @@
 
 #pragma once
 
-#include <WebCore/SharedBuffer.h>
+#include "SharedBuffer.h"
 
 namespace WebCore {
 
@@ -37,6 +37,9 @@ struct PromisedBlobInfo {
     String filename;
     PromisedBlobType blobType;
 
+    Vector<String> additionalTypes;
+    Vector<RefPtr<SharedBuffer>> additionalData;
+
     operator bool() const { return !blobURL.isEmpty(); }
 };
 
index d0f6757bb91d47ce5720f6e18c710d9943c5bfff..0582254d1046c7c06e2c53e691ea62a78375a732 100644 (file)
@@ -1,3 +1,57 @@
+2018-01-03  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        [Attachment Support] Add plumbing for starting a drag with promised blob data
+        https://bugs.webkit.org/show_bug.cgi?id=181201
+
+        Reviewed by Tim Horton.
+
+        Add boilerplate plumbing for PrepareToDragPromisedBlob, which delivers blob promises to the UI process when
+        dragging, respectively.
+
+        * Scripts/webkit/messages.py:
+        (headers_for_type):
+        * Shared/WebCoreArgumentCoders.cpp:
+        (IPC::encodeTypesAndData):
+        (IPC::decodeTypesAndData):
+        (IPC::ArgumentCoder<PasteboardWebContent>::encode):
+        (IPC::ArgumentCoder<PasteboardWebContent>::decode):
+        (IPC::ArgumentCoder<PasteboardImage>::encode):
+        (IPC::ArgumentCoder<PasteboardImage>::decode):
+        (IPC::ArgumentCoder<PromisedBlobInfo>::encode):
+        (IPC::ArgumentCoder<PromisedBlobInfo>::decode):
+
+        Add IPC support PromisedBlobInfo's additionalTypes and additionalData.
+
+        (IPC::encodeClientTypesAndData): Deleted.
+        (IPC::decodeClientTypesAndData): Deleted.
+
+        Rename these helper functions and move them to the top of the file.
+
+        * UIProcess/Cocoa/WebViewImpl.h:
+        * UIProcess/Cocoa/WebViewImpl.mm:
+        (WebKit::WebViewImpl::prepareToDragPromisedBlob):
+        * UIProcess/PageClient.h:
+        (WebKit::PageClient::prepareToDragPromisedBlob):
+        * UIProcess/WebPageProxy.cpp:
+        (WebKit::WebPageProxy::prepareToDragPromisedBlob):
+        * UIProcess/WebPageProxy.h:
+        * UIProcess/WebPageProxy.messages.in:
+        * UIProcess/ios/PageClientImplIOS.h:
+        * UIProcess/ios/PageClientImplIOS.mm:
+        (WebKit::PageClientImpl::prepareToDragPromisedBlob):
+        * UIProcess/ios/WKContentViewInteraction.h:
+        * UIProcess/ios/WKContentViewInteraction.mm:
+        (-[WKContentView _prepareToDragPromisedBlob:]):
+        * UIProcess/mac/PageClientImplMac.h:
+        * UIProcess/mac/PageClientImplMac.mm:
+        (WebKit::PageClientImpl::prepareToDragPromisedBlob):
+        * WebProcess/WebCoreSupport/WebDragClient.cpp:
+        (WebKit::WebDragClient::prepareToDragPromisedBlob):
+        * WebProcess/WebCoreSupport/WebDragClient.h:
+        * WebProcess/WebPage/WebPage.cpp:
+        (WebKit::WebPage::prepareToDragPromisedBlob):
+        * WebProcess/WebPage/WebPage.h:
+
 2018-01-03  David Kilzer  <ddkilzer@apple.com>
 
         REGRESSION (r212929): WKSnapshotConfiguration may leak an NSNumber when deallocated
index 08d24f2e2cb1ca7434192a745dc0c52712e9c85a..c4840d9733ea36e6c82d596593e1e63b4367eb28 100644 (file)
@@ -376,7 +376,6 @@ def headers_for_type(type):
         'WebCore::IncludeSecureCookies': ['<WebCore/CookiesStrategy.h>'],
         'WebCore::KeyframeValueList': ['<WebCore/GraphicsLayer.h>'],
         'WebCore::KeypressCommand': ['<WebCore/KeyboardEvent.h>'],
-        'WebCore::MediaConstraints': ['<WebCore/MediaConstraints.h>'],
         'WebCore::PasteboardCustomData': ['<WebCore/Pasteboard.h>'],
         'WebCore::PasteboardImage': ['<WebCore/Pasteboard.h>'],
         'WebCore::PasteboardURL': ['<WebCore/Pasteboard.h>'],
index 8864df27097f318a1dc6ef2bbd3796f8564af9a6..49ce56aa55a212e9d63dd96690146c52a514e8e9 100644 (file)
@@ -146,6 +146,33 @@ static bool decodeSharedBuffer(Decoder& decoder, RefPtr<SharedBuffer>& buffer)
     return true;
 }
 
+static void encodeTypesAndData(Encoder& encoder, const Vector<String>& types, const Vector<RefPtr<SharedBuffer>>& data)
+{
+    ASSERT(types.size() == data.size());
+    encoder << types;
+    encoder << static_cast<uint64_t>(data.size());
+    for (auto& buffer : data)
+        encodeSharedBuffer(encoder, buffer.get());
+}
+
+static bool decodeTypesAndData(Decoder& decoder, Vector<String>& types, Vector<RefPtr<SharedBuffer>>& data)
+{
+    if (!decoder.decode(types))
+        return false;
+
+    uint64_t dataSize;
+    if (!decoder.decode(dataSize))
+        return false;
+
+    ASSERT(dataSize == types.size());
+
+    data.resize(dataSize);
+    for (auto& buffer : data)
+        decodeSharedBuffer(decoder, buffer);
+
+    return true;
+}
+
 void ArgumentCoder<MonotonicTime>::encode(Encoder& encoder, const MonotonicTime& time)
 {
     encoder << time.secondsSinceEpoch().value();
@@ -1649,33 +1676,6 @@ bool ArgumentCoder<PasteboardURL>::decode(Decoder& decoder, PasteboardURL& conte
     return true;
 }
 
-static void encodeClientTypesAndData(Encoder& encoder, const Vector<String>& types, const Vector<RefPtr<SharedBuffer>>& data)
-{
-    ASSERT(types.size() == data.size());
-    encoder << types;
-    encoder << static_cast<uint64_t>(data.size());
-    for (auto& buffer : data)
-        encodeSharedBuffer(encoder, buffer.get());
-}
-
-static bool decodeClientTypesAndData(Decoder& decoder, Vector<String>& types, Vector<RefPtr<SharedBuffer>>& data)
-{
-    if (!decoder.decode(types))
-        return false;
-
-    uint64_t dataSize;
-    if (!decoder.decode(dataSize))
-        return false;
-
-    ASSERT(dataSize == types.size());
-
-    data.resize(dataSize);
-    for (auto& buffer : data)
-        decodeSharedBuffer(decoder, buffer);
-
-    return true;
-}
-
 void ArgumentCoder<PasteboardWebContent>::encode(Encoder& encoder, const PasteboardWebContent& content)
 {
     encoder << content.contentOrigin;
@@ -1688,7 +1688,7 @@ void ArgumentCoder<PasteboardWebContent>::encode(Encoder& encoder, const Pastebo
     encodeSharedBuffer(encoder, content.dataInRTFFormat.get());
     encodeSharedBuffer(encoder, content.dataInAttributedStringFormat.get());
 
-    encodeClientTypesAndData(encoder, content.clientTypes, content.clientData);
+    encodeTypesAndData(encoder, content.clientTypes, content.clientData);
 }
 
 bool ArgumentCoder<PasteboardWebContent>::decode(Decoder& decoder, PasteboardWebContent& content)
@@ -1709,7 +1709,7 @@ bool ArgumentCoder<PasteboardWebContent>::decode(Decoder& decoder, PasteboardWeb
         return false;
     if (!decodeSharedBuffer(decoder, content.dataInAttributedStringFormat))
         return false;
-    if (!decodeClientTypesAndData(decoder, content.clientTypes, content.clientData))
+    if (!decodeTypesAndData(decoder, content.clientTypes, content.clientData))
         return false;
     return true;
 }
@@ -1724,7 +1724,7 @@ void ArgumentCoder<PasteboardImage>::encode(Encoder& encoder, const PasteboardIm
     encoder << pasteboardImage.imageSize;
     if (pasteboardImage.resourceData)
         encodeSharedBuffer(encoder, pasteboardImage.resourceData.get());
-    encodeClientTypesAndData(encoder, pasteboardImage.clientTypes, pasteboardImage.clientData);
+    encodeTypesAndData(encoder, pasteboardImage.clientTypes, pasteboardImage.clientData);
 }
 
 bool ArgumentCoder<PasteboardImage>::decode(Decoder& decoder, PasteboardImage& pasteboardImage)
@@ -1743,7 +1743,7 @@ bool ArgumentCoder<PasteboardImage>::decode(Decoder& decoder, PasteboardImage& p
         return false;
     if (!decodeSharedBuffer(decoder, pasteboardImage.resourceData))
         return false;
-    if (!decodeClientTypesAndData(decoder, pasteboardImage.clientTypes, pasteboardImage.clientData))
+    if (!decodeTypesAndData(decoder, pasteboardImage.clientTypes, pasteboardImage.clientData))
         return false;
     return true;
 }
@@ -2814,6 +2814,7 @@ void ArgumentCoder<PromisedBlobInfo>::encode(Encoder& encoder, const PromisedBlo
     encoder << info.contentType;
     encoder << info.filename;
     encoder.encodeEnum(info.blobType);
+    encodeTypesAndData(encoder, info.additionalTypes, info.additionalData);
 }
 
 bool ArgumentCoder<PromisedBlobInfo>::decode(Decoder& decoder, PromisedBlobInfo& info)
@@ -2830,6 +2831,9 @@ bool ArgumentCoder<PromisedBlobInfo>::decode(Decoder& decoder, PromisedBlobInfo&
     if (!decoder.decode(info.blobType))
         return false;
 
+    if (!decodeTypesAndData(decoder, info.additionalTypes, info.additionalData))
+        return false;
+
     return true;
 }
 
index a5472bb1cc52dff110addfeb847d973f6e628a8e..8fd3712388a4412fd7e55aa01d84d94acb691c03 100644 (file)
@@ -33,6 +33,7 @@
 #include "WeakObjCPtr.h"
 #include "WebPageProxy.h"
 #include "_WKOverlayScrollbarStyle.h"
+#include <WebCore/PromisedBlobInfo.h>
 #include <WebCore/TextIndicatorWindow.h>
 #include <WebCore/UserInterfaceLayoutDirection.h>
 #include <pal/spi/cocoa/AVKitSPI.h>
@@ -534,6 +535,8 @@ public:
     void setIsCustomizingTouchBar(bool isCustomizingTouchBar) { m_isCustomizingTouchBar = isCustomizingTouchBar; };
 #endif // HAVE(TOUCH_BAR)
 
+    void prepareToDragPromisedBlob(const WebCore::PromisedBlobInfo&);
+
 private:
 #if HAVE(TOUCH_BAR)
     void setUpTextTouchBar(NSTouchBar *);
index 48d623ef91f279be602a86aa9e7f380a6900f159..8526d67bd122cf876cca10dd7e6f850baf7f97cc 100644 (file)
@@ -3765,6 +3765,13 @@ void WebViewImpl::registerDraggedTypes()
     [types addObject:PasteboardTypes::WebDummyPboardType];
     [m_view registerForDraggedTypes:[types allObjects]];
 }
+
+void WebViewImpl::prepareToDragPromisedBlob(const WebCore::PromisedBlobInfo& info)
+{
+    // FIXME: Add macOS support for dragging promised blob data as file promises.
+    UNUSED_PARAM(info);
+}
+
 #endif // ENABLE(DRAG_SUPPORT)
 
 void WebViewImpl::startWindowDrag()
index c2ec872db16783a627cf2c57bb941bdd18abde96..341cba4cc898598dd21d5bf3e6524e5c70f26db5 100644 (file)
@@ -193,6 +193,8 @@ public:
     virtual void setPromisedDataForImage(const String& pasteboardName, Ref<WebCore::SharedBuffer>&& imageBuffer, const String& filename, const String& extension, const String& title, const String& url, const String& visibleUrl, RefPtr<WebCore::SharedBuffer>&& archiveBuffer) = 0;
 #endif
 
+    virtual void prepareToDragPromisedBlob(const WebCore::PromisedBlobInfo&) { }
+
     virtual WebCore::FloatRect convertToDeviceSpace(const WebCore::FloatRect&) = 0;
     virtual WebCore::FloatRect convertToUserSpace(const WebCore::FloatRect&) = 0;
     virtual WebCore::IntPoint screenToRootView(const WebCore::IntPoint&) = 0;
index 6e80c2bf77d19d5d716132bab38271142168a0a5..c150129514d53916346049b313c5f61359a5cf6e 100644 (file)
@@ -7199,6 +7199,11 @@ void WebPageProxy::requestStorageAccess(String&& subFrameHost, String&& topFrame
 }
 #endif
 
+void WebPageProxy::prepareToDragPromisedBlob(const PromisedBlobInfo& info)
+{
+    m_pageClient.prepareToDragPromisedBlob(info);
+}
+
 #if PLATFORM(COCOA)
 void WebPageProxy::touchBarMenuDataChanged(const TouchBarMenuData& touchBarMenuData)
 {
index 3cb3dccc1a1003b24547cef8602d9c6cb5e51b6a..2dce6ba3181c63c3684e507869de7d25b19cbbd0 100644 (file)
@@ -1249,6 +1249,8 @@ public:
     void requestStorageAccess(String&& subFrameHost, String&& topFrameHost, uint64_t frameID, uint64_t pageID, uint64_t webProcessContextId);
 #endif
 
+    void prepareToDragPromisedBlob(const WebCore::PromisedBlobInfo&);
+
 #if ENABLE(ATTACHMENT_ELEMENT)
     void insertAttachment(const String& identifier, const WebCore::AttachmentDisplayOptions&, const String& filename, std::optional<String> contentType, WebCore::SharedBuffer& data, Function<void(CallbackBase::Error)>&&);
     void requestAttachmentData(const String& identifier, Function<void(RefPtr<WebCore::SharedBuffer>, CallbackBase::Error)>&&);
index b8b058139357ced6f07818f9a6c230ce1a3c542e..7ea6b2b45b15bae07e0245b1c5a1be499b212b21 100644 (file)
@@ -511,4 +511,6 @@ messages -> WebPageProxy {
     DidInsertAttachment(String identifier)
     DidRemoveAttachment(String identifier)
 #endif
+
+    PrepareToDragPromisedBlob(struct WebCore::PromisedBlobInfo blobInfo)
 }
index c1a137a6b9bb9b57401145edbe0524f19fa22b08..4e111a8ab761431a58fbb0a80a7f5f2180c14712 100644 (file)
 OBJC_CLASS WKContentView;
 OBJC_CLASS WKEditorUndoTargetObjC;
 
+namespace WebCore {
+struct PromisedBlobInfo;
+}
+
 namespace WebKit {
     
 class PageClientImpl : public PageClientImplCocoa
@@ -204,6 +208,8 @@ private:
     void didChangeDataInteractionCaretRect(const WebCore::IntRect& previousCaretRect, const WebCore::IntRect& caretRect) override;
 #endif
 
+    void prepareToDragPromisedBlob(const WebCore::PromisedBlobInfo&) final;
+
     WKContentView *m_contentView;
     RetainPtr<WKEditorUndoTargetObjC> m_undoTarget;
 };
index 15b24d3302539f06b3c6163d811c91441fa3bd11..b58405d6a52b890f48cad6122932148b03e7cc04 100644 (file)
@@ -52,6 +52,7 @@
 #import "_WKDownloadInternal.h"
 #import <WebCore/NotImplemented.h>
 #import <WebCore/PlatformScreen.h>
+#import <WebCore/PromisedBlobInfo.h>
 #import <WebCore/SharedBuffer.h>
 #import <WebCore/TextIndicator.h>
 #import <WebCore/ValidationBubble.h>
@@ -813,6 +814,11 @@ void PageClientImpl::requestPasswordForQuickLookDocument(const String& fileName,
 }
 #endif
 
+void PageClientImpl::prepareToDragPromisedBlob(const WebCore::PromisedBlobInfo& info)
+{
+    [m_contentView _prepareToDragPromisedBlob:info];
+}
+
 } // namespace WebKit
 
 #endif // PLATFORM(IOS)
index be4c3d365cc799c4ad2a4efd1b0126ca0c1a8e77..86fab02d3b04209ae9734943dffa90623c1cbd9f 100644 (file)
@@ -59,6 +59,7 @@ class Color;
 class FloatQuad;
 class IntSize;
 class SelectionRect;
+struct PromisedBlobInfo;
 }
 
 #if ENABLE(DRAG_SUPPORT)
@@ -326,6 +327,7 @@ FOR_EACH_WKCONTENTVIEW_ACTION(DECLARE_WKCONTENTVIEW_ACTION_FOR_WEB_VIEW)
 - (void)_startDrag:(RetainPtr<CGImageRef>)image item:(const WebCore::DragItem&)item;
 - (void)_didConcludeEditDataInteraction:(std::optional<WebCore::TextIndicatorData>)data;
 - (void)_didChangeDataInteractionCaretRect:(CGRect)previousRect currentRect:(CGRect)rect;
+- (void)_prepareToDragPromisedBlob:(const WebCore::PromisedBlobInfo&)info;
 #endif
 
 @end
index f75ed256b3123e978b8aae78a589bb17ba6b5aaa..b588725829107f6576f8b8256b8bb990f1151d33 100644 (file)
@@ -72,6 +72,7 @@
 #import <WebCore/Pasteboard.h>
 #import <WebCore/Path.h>
 #import <WebCore/PathUtilities.h>
+#import <WebCore/PromisedBlobInfo.h>
 #import <WebCore/RuntimeApplicationChecks.h>
 #import <WebCore/Scrollbar.h>
 #import <WebCore/TextIndicator.h>
@@ -4469,6 +4470,12 @@ static NSArray<UIItemProvider *> *extractItemProvidersFromDropSession(id <UIDrop
     [_editDropCaretView updateToPosition:[WKTextPosition textPositionWithRect:rect]];
 }
 
+- (void)_prepareToDragPromisedBlob:(const PromisedBlobInfo&)info
+{
+    // FIXME: Add iOS support for dragging promised blob data as file promises.
+    UNUSED_PARAM(info);
+}
+
 - (WKDragDestinationAction)_dragDestinationActionForDropSession:(id <UIDropSession>)session
 {
     id <WKUIDelegatePrivate> uiDelegate = self.webViewUIDelegate;
index ae836401cfa6d8217051ede694653ab5d85e9c04..4179ab334c7b3b746d98de7222dd39009f0abde2 100644 (file)
@@ -37,6 +37,7 @@
 
 namespace WebCore {
 class AlternativeTextUIController;
+struct PromisedBlobInfo;
 }
 
 namespace WebKit {
@@ -249,6 +250,8 @@ private:
 
     void didRestoreScrollPosition() override;
     bool windowIsFrontWindowUnderMouse(const NativeWebMouseEvent&) override;
+
+    void prepareToDragPromisedBlob(const WebCore::PromisedBlobInfo&) final;
 };
 
 } // namespace WebKit
index addff701bfa982b7a46456cde66d577a701f9add..4e3185c675632291eec8e209ba4d8f3b0a2ee3cc 100644 (file)
@@ -64,6 +64,7 @@
 #import <WebCore/Image.h>
 #import <WebCore/KeyboardEvent.h>
 #import <WebCore/NotImplemented.h>
+#import <WebCore/PromisedBlobInfo.h>
 #import <WebCore/SharedBuffer.h>
 #import <WebCore/TextIndicator.h>
 #import <WebCore/TextIndicatorWindow.h>
@@ -876,6 +877,11 @@ WebCore::UserInterfaceLayoutDirection PageClientImpl::userInterfaceLayoutDirecti
     return (m_view.userInterfaceLayoutDirection == NSUserInterfaceLayoutDirectionLeftToRight) ? WebCore::UserInterfaceLayoutDirection::LTR : WebCore::UserInterfaceLayoutDirection::RTL;
 }
 
+void PageClientImpl::prepareToDragPromisedBlob(const WebCore::PromisedBlobInfo& info)
+{
+    m_impl->prepareToDragPromisedBlob(info);
+}
+
 } // namespace WebKit
 
 #endif // PLATFORM(MAC)
index 590571895c81c84f4dce702871a0f5d563aec30d..ec71c9bf3a666ed09d19981849f3192b85acba56 100644 (file)
@@ -66,6 +66,11 @@ void WebDragClient::dragControllerDestroyed()
     delete this;
 }
 
+void WebDragClient::prepareToDragPromisedBlob(const WebCore::PromisedBlobInfo& info)
+{
+    m_page->prepareToDragPromisedBlob(info);
+}
+
 } // namespace WebKit
 
 #endif // ENABLE(DRAG_SUPPORT)
index 9f7d7fc50b6ec8aee56bb259b73340c40d936727..2b2b2977532126e3d9aa79883e7049367ebadde7 100644 (file)
@@ -53,6 +53,8 @@ private:
     void declareAndWriteDragImage(const String& pasteboardName, WebCore::Element&, const WebCore::URL&, const String&, WebCore::Frame*) override;
 #endif
 
+    void prepareToDragPromisedBlob(const WebCore::PromisedBlobInfo&) final;
+
     void dragControllerDestroyed() override;
 
     WebPage* m_page;
index 82468527d26319d18fd6914088433034f4b1fb10..029b34ad333768cb3423ada5cc923d3134b04975 100644 (file)
 #include <WebCore/PlatformKeyboardEvent.h>
 #include <WebCore/PluginDocument.h>
 #include <WebCore/PrintContext.h>
+#include <WebCore/PromisedBlobInfo.h>
 #include <WebCore/Range.h>
 #include <WebCore/RenderLayer.h>
 #include <WebCore/RenderTreeAsText.h>
@@ -3288,6 +3289,11 @@ bool WebPage::handleEditingKeyboardEvent(KeyboardEvent* evt)
 
 #if ENABLE(DRAG_SUPPORT)
 
+void WebPage::prepareToDragPromisedBlob(const WebCore::PromisedBlobInfo& info)
+{
+    send(Messages::WebPageProxy::PrepareToDragPromisedBlob(info));
+}
+
 #if PLATFORM(GTK)
 void WebPage::performDragControllerAction(DragControllerAction action, const IntPoint& clientPosition, const IntPoint& globalPosition, uint64_t draggingSourceOperationMask, WebSelectionData&& selection, uint32_t flags)
 {
index a398953d7695871a5cb9af52d7175c86b9f3a37c..d8c37a66cc5ea80b5eaa28c439009106425a8ad5 100644 (file)
@@ -153,6 +153,7 @@ struct CompositionUnderline;
 struct DictationAlternative;
 struct Highlight;
 struct KeypressCommand;
+struct PromisedBlobInfo;
 struct TextCheckingResult;
 struct ViewportArguments;
 
@@ -777,6 +778,8 @@ public:
     void willStartDrag() { ASSERT(!m_isStartingDrag); m_isStartingDrag = true; }
     void didStartDrag();
     void dragCancelled();
+
+    void prepareToDragPromisedBlob(const WebCore::PromisedBlobInfo&);
 #endif
 
     void beginPrinting(uint64_t frameID, const PrintInfo&);
index 90761b1f3e4bac0b77e63d6f2603fc82df32f6c6..33341b8c9b9fde7c345c9e1c7e4047d99debded1 100644 (file)
@@ -1,3 +1,14 @@
+2018-01-03  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        [Attachment Support] Add plumbing for starting a drag with promised blob data
+        https://bugs.webkit.org/show_bug.cgi?id=181201
+
+        Reviewed by Tim Horton.
+
+        Minor adjustment to account for a DragClient interface change. See WebCore ChangeLog for more details.
+
+        * WebCoreSupport/WebDragClient.h:
+
 2018-01-02  Jiewen Tan  <jiewen_tan@apple.com>
 
         Add a WebAuthentication runtime feature flag
index 3e9a07ef5383e20bfc9dd7ea51d5547a6564b5cf..ced0dbc45fe16692e61367c8c3e402bc95e58079 100644 (file)
@@ -45,6 +45,9 @@ public:
 
     void declareAndWriteDragImage(const String& pasteboardName, WebCore::Element&, const WebCore::URL&, const String&, WebCore::Frame*) override;
     void didConcludeEditDrag() override;
+
+    void prepareToDragPromisedBlob(const WebCore::PromisedBlobInfo&) final { }
+
 private:
     WebView* m_webView;
 };