[Attachment Support] Dropping and pasting images should insert inline image elements...
authorwenson_hsieh@apple.com <wenson_hsieh@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 27 Aug 2018 02:37:22 +0000 (02:37 +0000)
committerwenson_hsieh@apple.com <wenson_hsieh@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 27 Aug 2018 02:37:22 +0000 (02:37 +0000)
https://bugs.webkit.org/show_bug.cgi?id=188933
<rdar://problem/43699724>

Reviewed by Darin Adler.

Source/WebCore:

Support the ability to drop and paste images as image elements, with attachment elements, only if attachment
elements are enabled. See changes below for more detail.

Tests:  WKAttachmentTests.CutAndPastePastedImage
        WKAttachmentTests.MovePastedImageByDragging
        WKAttachmentTests.RemoveNewlinesBeforePastedImage

* editing/Editor.h:
* editing/cocoa/EditorCocoa.mm:
(WebCore::Editor::getPasteboardTypesAndDataForAttachment):

Adjust this helper to take an Element& rather than an HTMLAttachmentElement&, and address a FIXME by writing the
document origin identifier to the pasteboard via custom pasteboard data when dragging an attachment. This allows
us to avoid creating extra image and attachment elements when dragging an image backed by an attachment within
the same document.

* editing/cocoa/WebContentReaderCocoa.mm:
(WebCore::contentTypeIsSuitableForInlineImageRepresentation):

Add a helper to determine whether a content type (UTI or MIME type) should be read as an inline image.

(WebCore::createFragmentForImageAttachment):
(WebCore::replaceRichContentWithAttachments):
(WebCore::WebContentReader::readFilePaths):

Teach codepaths where we currently create attachment elements to instead create image elements if the MIME type,
is something suitable for display via an inline image element; add the attachment element under the shadow root
of the image element.

* editing/markup.cpp:
(WebCore::StyledMarkupAccumulator::appendCustomAttributes):
(WebCore::restoreAttachmentElementsInFragment):

When dragging or copying an image element, we need to make sure that any attachment element backing the image
is preserved in the pasted or dropped fragment. To do this, we use a technique similar to what was done for
r180785 and r224593 and write a temporary "webkitattachmentid" attribute to the serialized markup on copy. Upon
deserializing the markup back to a fragment, we then create an attachment element with the same identifier under
the image.

(WebCore::createFragmentFromMarkup):
* html/HTMLAttachmentElement.h:
* html/HTMLImageElement.cpp:
(WebCore::HTMLImageElement::setAttachmentElement):
(WebCore::HTMLImageElement::attachmentElement const):

Helper methods to get and set an attachment element under an image element. Setting an image's attachment
element puts that attachment element under the shadow root of the image, and also hides the attachment element.

(WebCore::HTMLImageElement::attachmentIdentifier const):

Returns the identifier of an attachment element associated with the image element, or null.

* html/HTMLImageElement.h:
* html/HTMLImageElement.idl:

Add HTMLImageElement.webkitAttachmentIdentifier, a readonly attribute guarded by runtime-enabled attachment
element feature.

* page/DragController.cpp:
(WebCore::DragController::startDrag):

In the case of dragging an image, if that image element is backed by an attachment element, don't bother writing
the image data to the clipboard; instead, write the attachment data as a promise.

(WebCore::DragController::doImageDrag):

Plumb promised attachment information to DragController::doSystemDrag.

(WebCore::DragController::promisedAttachmentInfo):

Teach this to handle attachment elements as well as image elements that are backed by attachment elements.

* page/DragController.h:
* platform/PromisedAttachmentInfo.h:
(WebCore::PromisedAttachmentInfo::operator bool const):

A valid PromisedAttachmentInfo no longer requires a contentType to be set; instead, an attachment identifier
alone is sufficient, since an up-to-date content type can be requested in the UI process from the API attachment
object.

Source/WebKit:

Support the ability to drop and paste images as image elements, with attachment elements, only if attachment
elements are enabled. See changes below for more detail.

* Shared/WebCoreArgumentCoders.cpp:
(IPC::ArgumentCoder<PromisedAttachmentInfo>::encode):
(IPC::ArgumentCoder<PromisedAttachmentInfo>::decode):

Rename "filename" to "fileName", for consistency with WKContentView and WebViewImpl.

* UIProcess/API/APIAttachment.cpp:
(API::Attachment::mimeType const):
(API::Attachment::fileName const):
* UIProcess/API/APIAttachment.h:

Push getters for MIME type, UTI, and the file name down from _WKAttachment to API::Attachment. This allows
WKContentView and WebViewImpl to ask an API::Attachment questions about its UTI and file name without
additionally creating a wrapper object.

* UIProcess/API/Cocoa/APIAttachmentCocoa.mm: Added.
(API::mimeTypeInferredFromFileExtension):
(API::isDeclaredOrDynamicTypeIdentifier):
(API::Attachment::mimeType const):
(API::Attachment::utiType const):
(API::Attachment::fileName const):
* UIProcess/API/Cocoa/WKUIDelegatePrivate.h:

Add a private delegate hook to notify the UI delegate when a drop has been performed. This is used by tests to
know when a drop with file promises has been processed by the page.

* UIProcess/API/Cocoa/WKWebView.mm:
(-[WKWebView _web_didPerformDragOperation:]):
* UIProcess/API/Cocoa/_WKAttachment.mm:
(-[_WKAttachmentInfo initWithFileWrapper:filePath:mimeType:utiType:]):
(-[_WKAttachmentInfo fileWrapper]):
(-[_WKAttachmentInfo contentType]):
(-[_WKAttachment info]):
(-[_WKAttachmentInfo initWithFileWrapper:filePath:contentType:]): Deleted.
(isDeclaredOrDynamicTypeIdentifier): Deleted.
(-[_WKAttachmentInfo _typeIdentifierFromPathExtension]): Deleted.
(-[_WKAttachmentInfo mimeType]): Deleted.
(-[_WKAttachmentInfo utiType]): Deleted.

Moved to APIAttachmentCocoa.mm.

* UIProcess/API/mac/WKView.mm:
(-[WKView _web_didPerformDragOperation:]):
* UIProcess/Cocoa/WebViewImpl.h:
* UIProcess/Cocoa/WebViewImpl.mm:
(-[WKPromisedAttachmentContext initWithIdentifier:blobURL:fileName:]):

Adjust this constructor to take each piece of data separately. This is because, in the case where our
PromisedAttachmentInfo contains an attachment identifier, we determine the UTI and file name from the associated
file wrapper.

(-[WKPromisedAttachmentContext fileName]):
(WebKit::WebViewImpl::fileNameForFilePromiseProvider):
(WebKit::WebViewImpl::didPerformDragOperation):
(WebKit::WebViewImpl::startDrag):

Determine UTI and file name from the attachment element corresponding to the attachment identifier when
dragging. This is because the attachment element in the web process shouldn't need to have type and title
attributes set when starting a drag if it already has an identifier that maps to attachment data in the UI
process. An example of this is in inline images backed by attachments, for which we don't need to bother keeping
specifying display attributes, since they are never visible.

(-[WKPromisedAttachmentContext initWithAttachmentInfo:]): Deleted.
(-[WKPromisedAttachmentContext filename]): Deleted.
* UIProcess/PageClient.h:
(WebKit::PageClient::didPerformDragOperation):
* UIProcess/WebPageProxy.cpp:
(WebKit::WebPageProxy::didPerformDragOperation):
* UIProcess/WebPageProxy.h:
* UIProcess/WebPageProxy.messages.in:

Rename DidPerformDataInteractionControllerOperation to DidPerformDragOperation, and make it cross-platform (this
was previously only sent on iOS). Add plumbing through PageClient and friends on macOS to notify the UI
delegate when a drop is handled by the web process.

* UIProcess/ios/PageClientImplIOS.h:
* UIProcess/ios/PageClientImplIOS.mm:
(WebKit::PageClientImpl::didPerformDragOperation):
(WebKit::PageClientImpl::didPerformDataInteractionControllerOperation): Deleted.
* UIProcess/ios/WKContentViewInteraction.h:
* UIProcess/ios/WKContentViewInteraction.mm:
(-[WKContentView _didPerformDragOperation:]):
(-[WKContentView _prepareToDragPromisedAttachment:]):

Just like in WebViewImpl::startDrag, infer content type and file name from the API attachment object.

(-[WKContentView _didPerformDataInteractionControllerOperation:]): Deleted.
* UIProcess/ios/WebPageProxyIOS.mm:
(WebKit::WebPageProxy::didPerformDataInteractionControllerOperation): Deleted.
* UIProcess/mac/PageClientImplMac.h:
* UIProcess/mac/PageClientImplMac.mm:
(WebKit::PageClientImpl::didPerformDragOperation):
* WebKit.xcodeproj/project.pbxproj:
* WebProcess/WebPage/WebPage.cpp:
(WebKit::WebPage::performDragControllerAction):

Tools:

Rebaseline existing API tests that involve dropping or pasting image files, and additionally write some new
tests. These new tests exercise the following cases:
•       Inserting and removing newlines before an inline image with an attachment element does not cause new
        _WKAttachments to be created and destroyed.
•       Pasting an image, cutting it, and then pasting it again propagates an attachment update to the UI
        process with the original _WKAttachment.
•       A pasted attachment in the document can be moved around by dragging, and doing so does not cause us to
        lose a _WKAttachment.

* TestWebKitAPI/Tests/WebKitCocoa/WKAttachmentTests.mm:
(-[TestWKWebView expectElementCount:tagName:]):
(TestWebKitAPI::TEST):

Add the new tests described above, and also adjust existing tests to check that images are dropped or pasted
as image elements, but still have associated attachment elements whose attachment identifiers (observed via
script) match that of the corresponding _WKAttachment's uniqueIdentifier in the UI process.

* TestWebKitAPI/mac/DragAndDropSimulatorMac.mm:
(-[DragAndDropSimulator runFrom:to:]):
(-[DragAndDropSimulator continueDragSession]):
(-[DragAndDropSimulator performDragInWebView:atLocation:withImage:pasteboard:source:]):

Teach DragAndDropSimulator on macOS to wait until the drop has been handled by the web process before returning
execution to the caller. This ensures that tests which involve dropping promised files as attachments aren't
flaky, due to how the promised data is retrieved asynchronously when performing the drop.

(-[DragAndDropSimulator _webView:didPerformDragOperation:]):

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@235343 268f45cc-cd09-0410-ab3c-d52691b4dbfc

39 files changed:
Source/WebCore/ChangeLog
Source/WebCore/editing/Editor.h
Source/WebCore/editing/cocoa/EditorCocoa.mm
Source/WebCore/editing/cocoa/WebContentReaderCocoa.mm
Source/WebCore/editing/markup.cpp
Source/WebCore/html/HTMLAttachmentElement.h
Source/WebCore/html/HTMLImageElement.cpp
Source/WebCore/html/HTMLImageElement.h
Source/WebCore/html/HTMLImageElement.idl
Source/WebCore/page/DragController.cpp
Source/WebCore/page/DragController.h
Source/WebCore/platform/PromisedAttachmentInfo.h
Source/WebKit/ChangeLog
Source/WebKit/Shared/WebCoreArgumentCoders.cpp
Source/WebKit/UIProcess/API/APIAttachment.cpp
Source/WebKit/UIProcess/API/APIAttachment.h
Source/WebKit/UIProcess/API/Cocoa/APIAttachmentCocoa.mm [new file with mode: 0644]
Source/WebKit/UIProcess/API/Cocoa/WKUIDelegatePrivate.h
Source/WebKit/UIProcess/API/Cocoa/WKWebView.mm
Source/WebKit/UIProcess/API/Cocoa/_WKAttachment.mm
Source/WebKit/UIProcess/API/mac/WKView.mm
Source/WebKit/UIProcess/Cocoa/WebViewImpl.h
Source/WebKit/UIProcess/Cocoa/WebViewImpl.mm
Source/WebKit/UIProcess/PageClient.h
Source/WebKit/UIProcess/WebPageProxy.cpp
Source/WebKit/UIProcess/WebPageProxy.h
Source/WebKit/UIProcess/WebPageProxy.messages.in
Source/WebKit/UIProcess/ios/PageClientImplIOS.h
Source/WebKit/UIProcess/ios/PageClientImplIOS.mm
Source/WebKit/UIProcess/ios/WKContentViewInteraction.h
Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm
Source/WebKit/UIProcess/ios/WebPageProxyIOS.mm
Source/WebKit/UIProcess/mac/PageClientImplMac.h
Source/WebKit/UIProcess/mac/PageClientImplMac.mm
Source/WebKit/WebKit.xcodeproj/project.pbxproj
Source/WebKit/WebProcess/WebPage/WebPage.cpp
Tools/ChangeLog
Tools/TestWebKitAPI/Tests/WebKitCocoa/WKAttachmentTests.mm
Tools/TestWebKitAPI/mac/DragAndDropSimulatorMac.mm

index 92df4f7..845e8d2 100644 (file)
@@ -1,3 +1,91 @@
+2018-08-26  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        [Attachment Support] Dropping and pasting images should insert inline image elements with _WKAttachments
+        https://bugs.webkit.org/show_bug.cgi?id=188933
+        <rdar://problem/43699724>
+
+        Reviewed by Darin Adler.
+
+        Support the ability to drop and paste images as image elements, with attachment elements, only if attachment
+        elements are enabled. See changes below for more detail.
+
+        Tests:  WKAttachmentTests.CutAndPastePastedImage
+                WKAttachmentTests.MovePastedImageByDragging
+                WKAttachmentTests.RemoveNewlinesBeforePastedImage
+
+        * editing/Editor.h:
+        * editing/cocoa/EditorCocoa.mm:
+        (WebCore::Editor::getPasteboardTypesAndDataForAttachment):
+
+        Adjust this helper to take an Element& rather than an HTMLAttachmentElement&, and address a FIXME by writing the
+        document origin identifier to the pasteboard via custom pasteboard data when dragging an attachment. This allows
+        us to avoid creating extra image and attachment elements when dragging an image backed by an attachment within
+        the same document.
+
+        * editing/cocoa/WebContentReaderCocoa.mm:
+        (WebCore::contentTypeIsSuitableForInlineImageRepresentation):
+
+        Add a helper to determine whether a content type (UTI or MIME type) should be read as an inline image.
+
+        (WebCore::createFragmentForImageAttachment):
+        (WebCore::replaceRichContentWithAttachments):
+        (WebCore::WebContentReader::readFilePaths):
+
+        Teach codepaths where we currently create attachment elements to instead create image elements if the MIME type,
+        is something suitable for display via an inline image element; add the attachment element under the shadow root
+        of the image element.
+
+        * editing/markup.cpp:
+        (WebCore::StyledMarkupAccumulator::appendCustomAttributes):
+        (WebCore::restoreAttachmentElementsInFragment):
+
+        When dragging or copying an image element, we need to make sure that any attachment element backing the image
+        is preserved in the pasted or dropped fragment. To do this, we use a technique similar to what was done for
+        r180785 and r224593 and write a temporary "webkitattachmentid" attribute to the serialized markup on copy. Upon
+        deserializing the markup back to a fragment, we then create an attachment element with the same identifier under
+        the image.
+
+        (WebCore::createFragmentFromMarkup):
+        * html/HTMLAttachmentElement.h:
+        * html/HTMLImageElement.cpp:
+        (WebCore::HTMLImageElement::setAttachmentElement):
+        (WebCore::HTMLImageElement::attachmentElement const):
+
+        Helper methods to get and set an attachment element under an image element. Setting an image's attachment
+        element puts that attachment element under the shadow root of the image, and also hides the attachment element.
+
+        (WebCore::HTMLImageElement::attachmentIdentifier const):
+
+        Returns the identifier of an attachment element associated with the image element, or null.
+
+        * html/HTMLImageElement.h:
+        * html/HTMLImageElement.idl:
+
+        Add HTMLImageElement.webkitAttachmentIdentifier, a readonly attribute guarded by runtime-enabled attachment
+        element feature.
+
+        * page/DragController.cpp:
+        (WebCore::DragController::startDrag):
+
+        In the case of dragging an image, if that image element is backed by an attachment element, don't bother writing
+        the image data to the clipboard; instead, write the attachment data as a promise.
+
+        (WebCore::DragController::doImageDrag):
+
+        Plumb promised attachment information to DragController::doSystemDrag.
+
+        (WebCore::DragController::promisedAttachmentInfo):
+
+        Teach this to handle attachment elements as well as image elements that are backed by attachment elements.
+
+        * page/DragController.h:
+        * platform/PromisedAttachmentInfo.h:
+        (WebCore::PromisedAttachmentInfo::operator bool const):
+
+        A valid PromisedAttachmentInfo no longer requires a contentType to be set; instead, an attachment identifier
+        alone is sufficient, since an up-to-date content type can be requested in the UI process from the API attachment
+        object.
+
 2018-08-26  Andy Estes  <aestes@apple.com>
 
         [Apple Pay] PaymentRequest.show() should reject when an unsupported ApplePayRequest version is specified
index deef53c..1ec120b 100644 (file)
@@ -511,7 +511,7 @@ public:
     void didRemoveAttachmentElement(HTMLAttachmentElement&);
 
 #if PLATFORM(COCOA)
-    void getPasteboardTypesAndDataForAttachment(HTMLAttachmentElement&, Vector<String>& outTypes, Vector<RefPtr<SharedBuffer>>& outData);
+    void getPasteboardTypesAndDataForAttachment(Element&, Vector<String>& outTypes, Vector<RefPtr<SharedBuffer>>& outData);
 #endif
 #endif
 
index 6bc903b..03f1115 100644 (file)
@@ -160,26 +160,21 @@ String Editor::selectionInHTMLFormat()
 
 #if ENABLE(ATTACHMENT_ELEMENT)
 
-void Editor::getPasteboardTypesAndDataForAttachment(HTMLAttachmentElement& attachment, Vector<String>& outTypes, Vector<RefPtr<SharedBuffer>>& outData)
+void Editor::getPasteboardTypesAndDataForAttachment(Element& element, Vector<String>& outTypes, Vector<RefPtr<SharedBuffer>>& outData)
 {
-    auto attachmentRange = Range::create(attachment.document(), { &attachment, Position::PositionIsBeforeAnchor }, { &attachment, Position::PositionIsAfterAnchor });
-    client()->getClientPasteboardDataForRange(attachmentRange.ptr(), outTypes, outData);
-    // FIXME: We should additionally write the attachment as a web archive here, such that drag and drop within the
-    // same page doesn't destroy and recreate attachments unnecessarily. This is also needed to preserve the attachment
-    // display mode when dragging and dropping or cutting and pasting. For the time being, this is disabled because
-    // inserting attachment elements from web archive data sometimes causes attachment data to be lost; this requires
-    // further investigation.
-#if PLATFORM(MAC)
-    // On macOS, we currently write the attachment as a web archive; we can't do the same for iOS and remove the platform guard above
-    // quite yet without breaking drag moves. This investigation is tracked in <https://bugs.webkit.org/show_bug.cgi?id=181514>.
-    // See the above FIXME for more details.
-    if (auto archive = LegacyWebArchive::create(attachmentRange.ptr())) {
+    auto& document = element.document();
+    auto elementRange = Range::create(document, { &element, Position::PositionIsBeforeAnchor }, { &element, Position::PositionIsAfterAnchor });
+    client()->getClientPasteboardDataForRange(elementRange.ptr(), outTypes, outData);
+
+    outTypes.append(PasteboardCustomData::cocoaType());
+    outData.append(PasteboardCustomData { document.originIdentifierForPasteboard(), { }, { }, { } }.createSharedBuffer());
+
+    if (auto archive = LegacyWebArchive::create(elementRange.ptr())) {
         if (auto webArchiveData = archive->rawDataRepresentation()) {
             outTypes.append(WebArchivePboardType);
             outData.append(SharedBuffer::create(webArchiveData.get()));
         }
     }
-#endif
 }
 
 #endif
index b0cb796..1c91351 100644 (file)
@@ -47,6 +47,7 @@
 #import "HTMLImageElement.h"
 #import "HTMLObjectElement.h"
 #import "LegacyWebArchive.h"
+#import "MIMETypeRegistry.h"
 #import "Page.h"
 #import "PublicURLManager.h"
 #import "RuntimeEnabledFeatures.h"
@@ -207,6 +208,11 @@ static bool shouldReplaceRichContentWithAttachments()
 
 #if ENABLE(ATTACHMENT_ELEMENT)
 
+static bool contentTypeIsSuitableForInlineImageRepresentation(const String& contentType)
+{
+    return MIMETypeRegistry::isSupportedImageMIMEType(isDeclaredUTI(contentType) ? MIMETypeFromUTI(contentType) : contentType);
+}
+
 static bool supportsClientSideAttachmentData(const Frame& frame)
 {
     if (auto* client = frame.editor().client())
@@ -224,15 +230,22 @@ static Ref<DocumentFragment> createFragmentForImageAttachment(Frame& frame, Docu
     // FIXME: This fallback image name needs to be a localized string.
     String defaultImageAttachmentName { "image"_s };
 
+    auto fragment = document.createDocumentFragment();
     if (supportsClientSideAttachmentData(frame)) {
-        attachment->updateAttributes(buffer->size(), contentType, defaultImageAttachmentName);
         frame.editor().registerAttachmentIdentifier(attachment->ensureUniqueIdentifier(), contentType, defaultImageAttachmentName, WTFMove(buffer));
-    } else
+        if (contentTypeIsSuitableForInlineImageRepresentation(contentType)) {
+            auto image = HTMLImageElement::create(document);
+            image->setAttributeWithoutSynchronization(HTMLNames::srcAttr, DOMURL::createObjectURL(document, Blob::create(buffer.get(), contentType)));
+            image->setAttachmentElement(WTFMove(attachment));
+            fragment->appendChild(WTFMove(image));
+        } else {
+            attachment->updateAttributes(buffer->size(), contentType, defaultImageAttachmentName);
+            fragment->appendChild(WTFMove(attachment));
+        }
+    } else {
         attachment->setFile(File::create(Blob::create(buffer.get(), contentType), defaultImageAttachmentName), HTMLAttachmentElement::UpdateDisplayAttributes::Yes);
-
-    auto fragment = document.createDocumentFragment();
-    fragment->appendChild(attachment);
-
+        fragment->appendChild(WTFMove(attachment));
+    }
     return fragment;
 #else
     UNUSED_PARAM(blob);
@@ -243,11 +256,11 @@ static Ref<DocumentFragment> createFragmentForImageAttachment(Frame& frame, Docu
 static void replaceRichContentWithAttachments(Frame& frame, DocumentFragment& fragment, const Vector<Ref<ArchiveResource>>& subresources)
 {
 #if ENABLE(ATTACHMENT_ELEMENT)
-    struct AttachmentReplacementInfo {
+    struct AttachmentInsertionInfo {
         String fileName;
         String contentType;
         Ref<SharedBuffer> data;
-        Ref<Element> elementToReplace;
+        Ref<Element> originalElement;
     };
 
     ASSERT(RuntimeEnabledFeatures::sharedFeatures().attachmentElementEnabled());
@@ -263,7 +276,7 @@ static void replaceRichContentWithAttachments(Frame& frame, DocumentFragment& fr
     }
 
     Vector<Ref<Element>> elementsToRemove;
-    Vector<AttachmentReplacementInfo> attachmentReplacementInfo;
+    Vector<AttachmentInsertionInfo> attachmentInsertionInfo;
     for (auto& image : descendantsOfType<HTMLImageElement>(fragment)) {
         auto resourceURLString = image.attributeWithoutSynchronization(HTMLNames::srcAttr);
         if (resourceURLString.isEmpty())
@@ -277,7 +290,7 @@ static void replaceRichContentWithAttachments(Frame& frame, DocumentFragment& fr
         if (name.isEmpty())
             name = AtomicString("media");
 
-        attachmentReplacementInfo.append({ name, resource->value->mimeType(), resource->value->data(), image });
+        attachmentInsertionInfo.append({ name, resource->value->mimeType(), resource->value->data(), image });
     }
 
     for (auto& object : descendantsOfType<HTMLObjectElement>(fragment)) {
@@ -295,23 +308,30 @@ static void replaceRichContentWithAttachments(Frame& frame, DocumentFragment& fr
         if (name.isEmpty())
             name = AtomicString("file");
 
-        attachmentReplacementInfo.append({ name, resource->value->mimeType(), resource->value->data(), object });
+        attachmentInsertionInfo.append({ name, resource->value->mimeType(), resource->value->data(), object });
     }
 
-    for (auto& info : attachmentReplacementInfo) {
-        auto elementToReplace = WTFMove(info.elementToReplace);
-        auto parent = makeRefPtr(elementToReplace->parentNode());
+    for (auto& info : attachmentInsertionInfo) {
+        auto originalElement = WTFMove(info.originalElement);
+        auto parent = makeRefPtr(originalElement->parentNode());
         if (!parent)
             continue;
 
         auto attachment = HTMLAttachmentElement::create(HTMLNames::attachmentTag, fragment.document());
         if (supportsClientSideAttachmentData(frame)) {
-            attachment->updateAttributes(info.data->size(), info.contentType, info.fileName);
+            if (is<HTMLImageElement>(originalElement.get()) && contentTypeIsSuitableForInlineImageRepresentation(info.contentType)) {
+                auto& image = downcast<HTMLImageElement>(originalElement.get());
+                image.setAttributeWithoutSynchronization(HTMLNames::srcAttr, DOMURL::createObjectURL(*frame.document(), Blob::create(info.data, info.contentType)));
+                image.setAttachmentElement(WTFMove(attachment));
+            } else {
+                attachment->updateAttributes(info.data->size(), info.contentType, info.fileName);
+                parent->replaceChild(attachment, WTFMove(originalElement));
+            }
             frame.editor().registerAttachmentIdentifier(attachment->ensureUniqueIdentifier(), WTFMove(info.contentType), WTFMove(info.fileName), WTFMove(info.data));
-        } else
-            attachment->setFile(File::create(Blob::create(info.data, info.contentType), info.fileName), HTMLAttachmentElement::UpdateDisplayAttributes::Yes);
-
-        parent->replaceChild(attachment, elementToReplace);
+        } else {
+            attachment->setFile(File::create(Blob::create(WTFMove(info.data), WTFMove(info.contentType)), WTFMove(info.fileName)), HTMLAttachmentElement::UpdateDisplayAttributes::Yes);
+            parent->replaceChild(WTFMove(attachment), WTFMove(originalElement));
+        }
     }
 
     for (auto& elementToRemove : elementsToRemove)
@@ -684,12 +704,20 @@ bool WebContentReader::readFilePaths(const Vector<String>& paths)
                 long long fileSize { 0 };
                 FileSystem::getFileSize(path, fileSize);
                 auto contentType = File::contentTypeForFile(path);
-                attachment->updateAttributes(fileSize, contentType, FileSystem::pathGetFileName(path));
                 frame.editor().registerAttachmentIdentifier(attachment->ensureUniqueIdentifier(), contentType, path);
-            } else
+                if (contentTypeIsSuitableForInlineImageRepresentation(contentType)) {
+                    auto image = HTMLImageElement::create(document);
+                    image->setAttributeWithoutSynchronization(HTMLNames::srcAttr, DOMURL::createObjectURL(document, File::create(path)));
+                    image->setAttachmentElement(WTFMove(attachment));
+                    fragment->appendChild(image);
+                } else {
+                    attachment->updateAttributes(fileSize, contentType, FileSystem::pathGetFileName(path));
+                    fragment->appendChild(attachment);
+                }
+            } else {
                 attachment->setFile(File::create(path), HTMLAttachmentElement::UpdateDisplayAttributes::Yes);
-
-            fragment->appendChild(attachment);
+                fragment->appendChild(attachment);
+            }
         }
     }
 #endif
index d1ec589..7f4afe3 100644 (file)
@@ -67,6 +67,7 @@
 #include "PageConfiguration.h"
 #include "Range.h"
 #include "RenderBlock.h"
+#include "RuntimeEnabledFeatures.h"
 #include "Settings.h"
 #include "SocketProvider.h"
 #include "StyleProperties.h"
@@ -406,16 +407,21 @@ String StyledMarkupAccumulator::stringValueForRange(const Node& node, const Rang
 void StyledMarkupAccumulator::appendCustomAttributes(StringBuilder& out, const Element& element, Namespaces* namespaces)
 {
 #if ENABLE(ATTACHMENT_ELEMENT)
-    if (!is<HTMLAttachmentElement>(element))
+    if (!RuntimeEnabledFeatures::sharedFeatures().attachmentElementEnabled())
         return;
     
-    const HTMLAttachmentElement& attachment = downcast<HTMLAttachmentElement>(element);
-    appendAttribute(out, element, { webkitattachmentidAttr, attachment.uniqueIdentifier() }, namespaces);
-    if (auto* file = attachment.file()) {
-        // These attributes are only intended for File deserialization, and are removed from the generated attachment
-        // element after we've deserialized and set its backing File.
-        appendAttribute(out, element, { webkitattachmentpathAttr, file->path() }, namespaces);
-        appendAttribute(out, element, { webkitattachmentbloburlAttr, file->url().string() }, namespaces);
+    if (is<HTMLAttachmentElement>(element)) {
+        auto& attachment = downcast<HTMLAttachmentElement>(element);
+        appendAttribute(out, element, { webkitattachmentidAttr, attachment.uniqueIdentifier() }, namespaces);
+        if (auto* file = attachment.file()) {
+            // These attributes are only intended for File deserialization, and are removed from the generated attachment
+            // element after we've deserialized and set its backing File, in restoreAttachmentElementsInFragment.
+            appendAttribute(out, element, { webkitattachmentpathAttr, file->path() }, namespaces);
+            appendAttribute(out, element, { webkitattachmentbloburlAttr, file->url().string() }, namespaces);
+        }
+    } else if (is<HTMLImageElement>(element)) {
+        if (auto attachment = downcast<HTMLImageElement>(element).attachmentElement())
+            appendAttribute(out, element, { webkitattachmentidAttr, attachment->uniqueIdentifier() }, namespaces);
     }
 #else
     UNUSED_PARAM(out);
@@ -881,15 +887,12 @@ String sanitizedMarkupForFragmentInDocument(Ref<DocumentFragment>&& fragment, Do
     return result;
 }
 
-Ref<DocumentFragment> createFragmentFromMarkup(Document& document, const String& markup, const String& baseURL, ParserContentPolicy parserContentPolicy)
+static void restoreAttachmentElementsInFragment(DocumentFragment& fragment)
 {
-    // We use a fake body element here to trick the HTML parser to using the InBody insertion mode.
-    auto fakeBody = HTMLBodyElement::create(document);
-    auto fragment = DocumentFragment::create(document);
-
-    fragment->parseHTML(markup, fakeBody.ptr(), parserContentPolicy);
-
 #if ENABLE(ATTACHMENT_ELEMENT)
+    if (!RuntimeEnabledFeatures::sharedFeatures().attachmentElementEnabled())
+        return;
+
     // When creating a fragment we must strip the webkit-attachment-path attribute after restoring the File object.
     Vector<Ref<HTMLAttachmentElement>> attachments;
     for (auto& attachment : descendantsOfType<HTMLAttachmentElement>(fragment))
@@ -905,11 +908,39 @@ Ref<DocumentFragment> createFragmentFromMarkup(Document& document, const String&
         else if (!blobURL.isEmpty())
             attachment->setFile(File::deserialize({ }, blobURL, attachment->attachmentType(), attachment->attachmentTitle()));
 
+        // Remove temporary attributes that were previously added in StyledMarkupAccumulator::appendCustomAttributes.
         attachment->removeAttribute(webkitattachmentidAttr);
         attachment->removeAttribute(webkitattachmentpathAttr);
         attachment->removeAttribute(webkitattachmentbloburlAttr);
     }
+
+    Vector<Ref<HTMLImageElement>> images;
+    for (auto& image : descendantsOfType<HTMLImageElement>(fragment))
+        images.append(image);
+
+    for (auto& image : images) {
+        auto attachmentIdentifier = image->attributeWithoutSynchronization(webkitattachmentidAttr);
+        if (attachmentIdentifier.isEmpty())
+            continue;
+
+        auto attachment = HTMLAttachmentElement::create(HTMLNames::attachmentTag, *fragment.ownerDocument());
+        attachment->setUniqueIdentifier(attachmentIdentifier);
+        image->setAttachmentElement(WTFMove(attachment));
+        image->removeAttribute(webkitattachmentidAttr);
+    }
+#else
+    UNUSED_PARAM(fragment);
 #endif
+}
+
+Ref<DocumentFragment> createFragmentFromMarkup(Document& document, const String& markup, const String& baseURL, ParserContentPolicy parserContentPolicy)
+{
+    // We use a fake body element here to trick the HTML parser into using the InBody insertion mode.
+    auto fakeBody = HTMLBodyElement::create(document);
+    auto fragment = DocumentFragment::create(document);
+
+    fragment->parseHTML(markup, fakeBody.ptr(), parserContentPolicy);
+    restoreAttachmentElementsInFragment(fragment);
     if (!baseURL.isEmpty() && baseURL != blankURL() && baseURL != document.baseURL())
         completeURLs(fragment.ptr(), baseURL);
 
index db806ac..5b64192 100644 (file)
@@ -46,7 +46,7 @@ public:
     enum class UpdateDisplayAttributes { No, Yes };
     void setFile(RefPtr<File>&&, UpdateDisplayAttributes = UpdateDisplayAttributes::No);
 
-    String uniqueIdentifier() const { return m_uniqueIdentifier; }
+    const String& uniqueIdentifier() const { return m_uniqueIdentifier; }
     void setUniqueIdentifier(const String& uniqueIdentifier) { m_uniqueIdentifier = uniqueIdentifier; }
 
     WEBCORE_EXPORT void updateAttributes(uint64_t fileSize, std::optional<String>&& newContentType = std::nullopt, std::optional<String>&& newFilename = std::nullopt);
index ed6cbd2..d52744c 100644 (file)
 #include "CSSPropertyNames.h"
 #include "CSSValueKeywords.h"
 #include "CachedImage.h"
+#include "ElementIterator.h"
 #include "FrameView.h"
 #include "HTMLAnchorElement.h"
+#include "HTMLAttachmentElement.h"
 #include "HTMLDocument.h"
 #include "HTMLFormElement.h"
 #include "HTMLParserIdioms.h"
@@ -603,6 +605,35 @@ String HTMLImageElement::crossOrigin() const
     return parseCORSSettingsAttribute(attributeWithoutSynchronization(crossoriginAttr));
 }
 
+#if ENABLE(ATTACHMENT_ELEMENT)
+
+void HTMLImageElement::setAttachmentElement(Ref<HTMLAttachmentElement>&& attachment)
+{
+    if (auto existingAttachment = attachmentElement())
+        existingAttachment->remove();
+
+    attachment->setInlineStyleProperty(CSSPropertyDisplay, CSSValueNone, true);
+    ensureUserAgentShadowRoot().appendChild(WTFMove(attachment));
+}
+
+RefPtr<HTMLAttachmentElement> HTMLImageElement::attachmentElement() const
+{
+    if (auto shadowRoot = userAgentShadowRoot())
+        return childrenOfType<HTMLAttachmentElement>(*shadowRoot).first();
+
+    return nullptr;
+}
+
+const String& HTMLImageElement::attachmentIdentifier() const
+{
+    if (auto attachment = attachmentElement())
+        return attachment->uniqueIdentifier();
+
+    return nullAtom();
+}
+
+#endif // ENABLE(ATTACHMENT_ELEMENT)
+
 #if ENABLE(SERVICE_CONTROLS)
 void HTMLImageElement::updateImageControls()
 {
index 848edea..75c173d 100644 (file)
@@ -31,6 +31,7 @@
 
 namespace WebCore {
 
+class HTMLAttachmentElement;
 class HTMLFormElement;
 class HTMLMapElement;
 
@@ -91,6 +92,12 @@ public:
     bool willRespondToMouseClickEvents() override;
 #endif
 
+#if ENABLE(ATTACHMENT_ELEMENT)
+    void setAttachmentElement(Ref<HTMLAttachmentElement>&&);
+    RefPtr<HTMLAttachmentElement> attachmentElement() const;
+    const String& attachmentIdentifier() const;
+#endif
+
     bool hasPendingActivity() const { return m_imageLoader.hasPendingActivity(); }
 
     bool canContainRangeEndPoint() const override { return false; }
index af72d3b..3790ac7 100644 (file)
@@ -42,6 +42,8 @@
     attribute unsigned long width;
     [Reflect] attribute DOMString decoding;
 
+    [Conditional=ATTACHMENT_ELEMENT, EnabledAtRuntime=AttachmentElement, ImplementedAs=attachmentIdentifier] readonly attribute DOMString webkitAttachmentIdentifier;
+
     // Extensions
     readonly attribute boolean complete;
     [Reflect,URL] attribute USVString lowsrc;
index d50e118..64f200a 100644 (file)
@@ -910,9 +910,6 @@ bool DragController::startDrag(Frame& src, const DragState& state, DragOperation
 #if ENABLE(VIDEO)
     includeShadowDOM = state.source->isMediaElement();
 #endif
-#if ENABLE(ATTACHMENT_ELEMENT)
-    includeShadowDOM = includeShadowDOM || is<HTMLAttachmentElement>(state.source.get());
-#endif
 #if ENABLE(INPUT_TYPE_COLOR)
     bool isColorControl = is<HTMLInputElement>(state.source) && downcast<HTMLInputElement>(*state.source).isColorControl();
     includeShadowDOM = includeShadowDOM || isColorControl;
@@ -1048,20 +1045,28 @@ bool DragController::startDrag(Frame& src, const DragState& state, DragOperation
         // We shouldn't be starting a drag for an image that can't provide an extension.
         // This is an early detection for problems encountered later upon drop.
         ASSERT(!image->filenameExtension().isEmpty());
+
+#if ENABLE(ATTACHMENT_ELEMENT)
+        auto attachmentInfo = promisedAttachmentInfo(src, element);
+#else
+        PromisedAttachmentInfo attachmentInfo;
+#endif
+
         if (hasData == HasNonDefaultPasteboardData::No) {
             m_draggingImageURL = imageURL;
             if (element.isContentRichlyEditable())
                 selectElement(element);
-            declareAndWriteDragImage(dataTransfer, element, !linkURL.isEmpty() ? linkURL : imageURL, hitTestResult.altDisplayString());
+            if (!attachmentInfo)
+                declareAndWriteDragImage(dataTransfer, element, !linkURL.isEmpty() ? linkURL : imageURL, hitTestResult.altDisplayString());
         }
 
         m_client.willPerformDragSourceAction(DragSourceActionImage, dragOrigin, dataTransfer);
 
         if (!dragImage)
-            doImageDrag(element, dragOrigin, hitTestResult.imageRect(), src, m_dragOffset, state);
+            doImageDrag(element, dragOrigin, hitTestResult.imageRect(), src, m_dragOffset, state, WTFMove(attachmentInfo));
         else {
             // DHTML defined drag image
-            doSystemDrag(WTFMove(dragImage), dragLoc, dragOrigin, src, state, { });
+            doSystemDrag(WTFMove(dragImage), dragLoc, dragOrigin, src, state, WTFMove(attachmentInfo));
         }
 
         return true;
@@ -1200,7 +1205,7 @@ bool DragController::startDrag(Frame& src, const DragState& state, DragOperation
     return false;
 }
 
-void DragController::doImageDrag(Element& element, const IntPoint& dragOrigin, const IntRect& layoutRect, Frame& frame, IntPoint& dragImageOffset, const DragState& state)
+void DragController::doImageDrag(Element& element, const IntPoint& dragOrigin, const IntRect& layoutRect, Frame& frame, IntPoint& dragImageOffset, const DragState& state, PromisedAttachmentInfo&& attachmentInfo)
 {
     IntPoint mouseDownPoint = dragOrigin;
     DragImage dragImage;
@@ -1243,7 +1248,7 @@ void DragController::doImageDrag(Element& element, const IntPoint& dragOrigin, c
         return;
 
     dragImageOffset = mouseDownPoint + scaledOrigin;
-    doSystemDrag(WTFMove(dragImage), dragImageOffset, dragOrigin, frame, state, { });
+    doSystemDrag(WTFMove(dragImage), dragImageOffset, dragOrigin, frame, state, WTFMove(attachmentInfo));
 }
 
 void DragController::beginDrag(DragItem dragItem, Frame& frame, const IntPoint& mouseDownPoint, const IntPoint& mouseDraggedPoint, DataTransfer& dataTransfer, DragSourceAction dragSourceAction)
@@ -1367,19 +1372,31 @@ String DragController::platformContentTypeForBlobType(const String& type) const
 
 #if ENABLE(ATTACHMENT_ELEMENT)
 
-PromisedAttachmentInfo DragController::promisedAttachmentInfo(Frame& frame, HTMLAttachmentElement& attachment)
+PromisedAttachmentInfo DragController::promisedAttachmentInfo(Frame& frame, Element& element)
 {
+    auto* client = frame.editor().client();
+    if (!client || !client->supportsClientSideAttachmentData())
+        return { };
+
+    RefPtr<HTMLAttachmentElement> attachment;
+    if (is<HTMLAttachmentElement>(element))
+        attachment = &downcast<HTMLAttachmentElement>(element);
+    else if (is<HTMLImageElement>(element))
+        attachment = downcast<HTMLImageElement>(element).attachmentElement();
+
+    if (!attachment)
+        return { };
+
     Vector<String> additionalTypes;
     Vector<RefPtr<SharedBuffer>> additionalData;
 #if PLATFORM(COCOA)
-    if (frame.editor().client())
-        frame.editor().getPasteboardTypesAndDataForAttachment(attachment, additionalTypes, additionalData);
+    frame.editor().getPasteboardTypesAndDataForAttachment(element, additionalTypes, additionalData);
 #endif
 
-    if (auto* file = attachment.file())
-        return { file->url(), platformContentTypeForBlobType(file->type()), file->name(), attachment.uniqueIdentifier(), WTFMove(additionalTypes), WTFMove(additionalData) };
+    if (auto* file = attachment->file())
+        return { file->url(), platformContentTypeForBlobType(file->type()), file->name(), { }, WTFMove(additionalTypes), WTFMove(additionalData) };
 
-    return { { }, platformContentTypeForBlobType(attachment.attachmentType()), attachment.attachmentTitle(), attachment.uniqueIdentifier(), WTFMove(additionalTypes), WTFMove(additionalData) };
+    return { { }, { }, { }, attachment->uniqueIdentifier(), WTFMove(additionalTypes), WTFMove(additionalData) };
 }
 
 #endif // ENABLE(ATTACHMENT_ELEMENT)
index 22a6958..c02d5c8 100644 (file)
@@ -40,7 +40,6 @@ class DragData;
 class Element;
 class Frame;
 class FrameSelection;
-class HTMLAttachmentElement;
 class HTMLInputElement;
 class IntRect;
 class Page;
@@ -115,7 +114,7 @@ struct PromisedAttachmentInfo;
         void mouseMovedIntoDocument(Document*);
         bool shouldUseCachedImageForDragImage(const Image&) const;
 
-        void doImageDrag(Element&, const IntPoint&, const IntRect&, Frame&, IntPoint&, const DragState&);
+        void doImageDrag(Element&, const IntPoint&, const IntRect&, Frame&, IntPoint&, const DragState&, PromisedAttachmentInfo&&);
         void doSystemDrag(DragImage, const IntPoint&, const IntPoint&, Frame&, const DragState&, PromisedAttachmentInfo&&);
 
         void beginDrag(DragItem, Frame&, const IntPoint& mouseDownPoint, const IntPoint& mouseDraggedPoint, DataTransfer&, DragSourceAction);
@@ -135,7 +134,7 @@ struct PromisedAttachmentInfo;
         void declareAndWriteDragImage(DataTransfer&, Element&, const URL&, const String& label);
 
 #if ENABLE(ATTACHMENT_ELEMENT)
-        PromisedAttachmentInfo promisedAttachmentInfo(Frame&, HTMLAttachmentElement&);
+        PromisedAttachmentInfo promisedAttachmentInfo(Frame&, Element&);
 #endif
         Page& m_page;
         DragClient& m_client;
index d38a31d..dbecfc1 100644 (file)
@@ -36,7 +36,7 @@ class URL;
 struct PromisedAttachmentInfo {
     URL blobURL;
     String contentType;
-    String filename;
+    String fileName;
 
 #if ENABLE(ATTACHMENT_ELEMENT)
     String attachmentIdentifier;
@@ -47,15 +47,12 @@ struct PromisedAttachmentInfo {
 
     operator bool() const
     {
-        if (contentType.isEmpty())
-            return false;
-
 #if ENABLE(ATTACHMENT_ELEMENT)
         if (!attachmentIdentifier.isEmpty())
             return true;
 #endif
 
-        return !blobURL.isEmpty();
+        return !contentType.isEmpty() && !blobURL.isEmpty();
     }
 };
 
index 509808a..5aa2e3d 100644 (file)
@@ -1,3 +1,110 @@
+2018-08-26  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        [Attachment Support] Dropping and pasting images should insert inline image elements with _WKAttachments
+        https://bugs.webkit.org/show_bug.cgi?id=188933
+        <rdar://problem/43699724>
+
+        Reviewed by Darin Adler.
+
+        Support the ability to drop and paste images as image elements, with attachment elements, only if attachment
+        elements are enabled. See changes below for more detail.
+
+        * Shared/WebCoreArgumentCoders.cpp:
+        (IPC::ArgumentCoder<PromisedAttachmentInfo>::encode):
+        (IPC::ArgumentCoder<PromisedAttachmentInfo>::decode):
+
+        Rename "filename" to "fileName", for consistency with WKContentView and WebViewImpl.
+
+        * UIProcess/API/APIAttachment.cpp:
+        (API::Attachment::mimeType const):
+        (API::Attachment::fileName const):
+        * UIProcess/API/APIAttachment.h:
+
+        Push getters for MIME type, UTI, and the file name down from _WKAttachment to API::Attachment. This allows
+        WKContentView and WebViewImpl to ask an API::Attachment questions about its UTI and file name without
+        additionally creating a wrapper object.
+
+        * UIProcess/API/Cocoa/APIAttachmentCocoa.mm: Added.
+        (API::mimeTypeInferredFromFileExtension):
+        (API::isDeclaredOrDynamicTypeIdentifier):
+        (API::Attachment::mimeType const):
+        (API::Attachment::utiType const):
+        (API::Attachment::fileName const):
+        * UIProcess/API/Cocoa/WKUIDelegatePrivate.h:
+
+        Add a private delegate hook to notify the UI delegate when a drop has been performed. This is used by tests to
+        know when a drop with file promises has been processed by the page.
+
+        * UIProcess/API/Cocoa/WKWebView.mm:
+        (-[WKWebView _web_didPerformDragOperation:]):
+        * UIProcess/API/Cocoa/_WKAttachment.mm:
+        (-[_WKAttachmentInfo initWithFileWrapper:filePath:mimeType:utiType:]):
+        (-[_WKAttachmentInfo fileWrapper]):
+        (-[_WKAttachmentInfo contentType]):
+        (-[_WKAttachment info]):
+        (-[_WKAttachmentInfo initWithFileWrapper:filePath:contentType:]): Deleted.
+        (isDeclaredOrDynamicTypeIdentifier): Deleted.
+        (-[_WKAttachmentInfo _typeIdentifierFromPathExtension]): Deleted.
+        (-[_WKAttachmentInfo mimeType]): Deleted.
+        (-[_WKAttachmentInfo utiType]): Deleted.
+
+        Moved to APIAttachmentCocoa.mm.
+
+        * UIProcess/API/mac/WKView.mm:
+        (-[WKView _web_didPerformDragOperation:]):
+        * UIProcess/Cocoa/WebViewImpl.h:
+        * UIProcess/Cocoa/WebViewImpl.mm:
+        (-[WKPromisedAttachmentContext initWithIdentifier:blobURL:fileName:]):
+
+        Adjust this constructor to take each piece of data separately. This is because, in the case where our
+        PromisedAttachmentInfo contains an attachment identifier, we determine the UTI and file name from the associated
+        file wrapper.
+
+        (-[WKPromisedAttachmentContext fileName]):
+        (WebKit::WebViewImpl::fileNameForFilePromiseProvider):
+        (WebKit::WebViewImpl::didPerformDragOperation):
+        (WebKit::WebViewImpl::startDrag):
+
+        Determine UTI and file name from the attachment element corresponding to the attachment identifier when
+        dragging. This is because the attachment element in the web process shouldn't need to have type and title
+        attributes set when starting a drag if it already has an identifier that maps to attachment data in the UI
+        process. An example of this is in inline images backed by attachments, for which we don't need to bother keeping
+        specifying display attributes, since they are never visible.
+
+        (-[WKPromisedAttachmentContext initWithAttachmentInfo:]): Deleted.
+        (-[WKPromisedAttachmentContext filename]): Deleted.
+        * UIProcess/PageClient.h:
+        (WebKit::PageClient::didPerformDragOperation):
+        * UIProcess/WebPageProxy.cpp:
+        (WebKit::WebPageProxy::didPerformDragOperation):
+        * UIProcess/WebPageProxy.h:
+        * UIProcess/WebPageProxy.messages.in:
+
+        Rename DidPerformDataInteractionControllerOperation to DidPerformDragOperation, and make it cross-platform (this
+        was previously only sent on iOS). Add plumbing through PageClient and friends on macOS to notify the UI
+        delegate when a drop is handled by the web process.
+
+        * UIProcess/ios/PageClientImplIOS.h:
+        * UIProcess/ios/PageClientImplIOS.mm:
+        (WebKit::PageClientImpl::didPerformDragOperation):
+        (WebKit::PageClientImpl::didPerformDataInteractionControllerOperation): Deleted.
+        * UIProcess/ios/WKContentViewInteraction.h:
+        * UIProcess/ios/WKContentViewInteraction.mm:
+        (-[WKContentView _didPerformDragOperation:]):
+        (-[WKContentView _prepareToDragPromisedAttachment:]):
+
+        Just like in WebViewImpl::startDrag, infer content type and file name from the API attachment object.
+
+        (-[WKContentView _didPerformDataInteractionControllerOperation:]): Deleted.
+        * UIProcess/ios/WebPageProxyIOS.mm:
+        (WebKit::WebPageProxy::didPerformDataInteractionControllerOperation): Deleted.
+        * UIProcess/mac/PageClientImplMac.h:
+        * UIProcess/mac/PageClientImplMac.mm:
+        (WebKit::PageClientImpl::didPerformDragOperation):
+        * WebKit.xcodeproj/project.pbxproj:
+        * WebProcess/WebPage/WebPage.cpp:
+        (WebKit::WebPage::performDragControllerAction):
+
 2018-08-23  Jeff Miller  <jeffm@apple.com>
 
         Remove -[WKNavigationDelegate _webView:decidePolicyForPluginLoadWithCurrentPolicy:pluginInfo:unavailabilityDescription:]
index 365f592..353d328 100644 (file)
@@ -2867,7 +2867,7 @@ void ArgumentCoder<PromisedAttachmentInfo>::encode(Encoder& encoder, const Promi
 {
     encoder << info.blobURL;
     encoder << info.contentType;
-    encoder << info.filename;
+    encoder << info.fileName;
 #if ENABLE(ATTACHMENT_ELEMENT)
     encoder << info.attachmentIdentifier;
 #endif
@@ -2882,7 +2882,7 @@ bool ArgumentCoder<PromisedAttachmentInfo>::decode(Decoder& decoder, PromisedAtt
     if (!decoder.decode(info.contentType))
         return false;
 
-    if (!decoder.decode(info.filename))
+    if (!decoder.decode(info.fileName))
         return false;
 
 #if ENABLE(ATTACHMENT_ELEMENT)
index 6c2af52..b615048 100644 (file)
@@ -81,6 +81,20 @@ void Attachment::invalidate()
 #endif
 }
 
+#if !PLATFORM(COCOA)
+
+WTF::String Attachment::mimeType() const
+{
+    return m_contentType;
+}
+
+WTF::String Attachment::fileName() const
+{
+    return { };
+}
+
+#endif // !PLATFORM(COCOA)
+
 }
 
 #endif // ENABLE(ATTACHMENT_ELEMENT)
index bf35f45..71741ff 100644 (file)
@@ -64,10 +64,13 @@ public:
 #if PLATFORM(COCOA)
     NSFileWrapper *fileWrapper() const { return m_fileWrapper.get(); }
     void setFileWrapper(NSFileWrapper *fileWrapper) { m_fileWrapper = fileWrapper; }
+    WTF::String utiType() const;
 #endif
+    WTF::String mimeType() const;
 
     const WTF::String& filePath() const { return m_filePath; }
     void setFilePath(const WTF::String& filePath) { m_filePath = filePath; }
+    WTF::String fileName() const;
 
     const WTF::String& contentType() const { return m_contentType; }
     void setContentType(const WTF::String& contentType) { m_contentType = contentType; }
diff --git a/Source/WebKit/UIProcess/API/Cocoa/APIAttachmentCocoa.mm b/Source/WebKit/UIProcess/API/Cocoa/APIAttachmentCocoa.mm
new file mode 100644 (file)
index 0000000..9e8b588
--- /dev/null
@@ -0,0 +1,77 @@
+/*
+ * Copyright (C) 2018 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#import "config.h"
+#import "APIAttachment.h"
+
+#import <WebCore/MIMETypeRegistry.h>
+#if PLATFORM(IOS)
+#import <MobileCoreServices/MobileCoreServices.h>
+#else
+#import <CoreServices/CoreServices.h>
+#endif
+
+namespace API {
+
+static WTF::String mimeTypeInferredFromFileExtension(const API::Attachment& attachment)
+{
+    if (NSString *fileExtension = [(NSString *)attachment.fileName() pathExtension])
+        return WebCore::MIMETypeRegistry::getMIMETypeForExtension(fileExtension);
+
+    return { };
+}
+
+static BOOL isDeclaredOrDynamicTypeIdentifier(NSString *type)
+{
+    return UTTypeIsDeclared((__bridge CFStringRef)type) || UTTypeIsDynamic((__bridge CFStringRef)type);
+}
+
+WTF::String Attachment::mimeType() const
+{
+    NSString *contentType = m_contentType.isEmpty() ? mimeTypeInferredFromFileExtension(*this) : m_contentType;
+    if (!isDeclaredOrDynamicTypeIdentifier(contentType))
+        return contentType;
+
+    return adoptCF(UTTypeCopyPreferredTagWithClass((__bridge CFStringRef)contentType, kUTTagClassMIMEType)).get();
+}
+
+WTF::String Attachment::utiType() const
+{
+    NSString *contentType = m_contentType.isEmpty() ? mimeTypeInferredFromFileExtension(*this) : m_contentType;
+    if (isDeclaredOrDynamicTypeIdentifier(contentType))
+        return contentType;
+
+    return adoptCF(UTTypeCreatePreferredIdentifierForTag(kUTTagClassMIMEType, (__bridge CFStringRef)contentType, nullptr)).get();
+}
+
+WTF::String Attachment::fileName() const
+{
+    if ([m_fileWrapper filename].length)
+        return [m_fileWrapper filename];
+
+    return [m_fileWrapper preferredFilename];
+}
+
+} // namespace API
index 44989f3..cdc98f0 100644 (file)
@@ -191,6 +191,7 @@ struct UIEdgeInsets;
 - (NSMenu *)_webView:(WKWebView *)webView contextMenu:(NSMenu *)menu forElement:(_WKContextMenuElementInfo *)element WK_API_DEPRECATED_WITH_REPLACEMENT("_webView:getContextMenuFromProposedMenu:forElement:userInfo:completionHandler:", macosx(10.12, WK_MAC_TBA));
 - (NSMenu *)_webView:(WKWebView *)webView contextMenu:(NSMenu *)menu forElement:(_WKContextMenuElementInfo *)element userInfo:(id <NSSecureCoding>)userInfo WK_API_DEPRECATED_WITH_REPLACEMENT("_webView:getContextMenuFromProposedMenu:forElement:userInfo:completionHandler:", macosx(10.12, WK_MAC_TBA));
 - (void)_webView:(WKWebView *)webView getContextMenuFromProposedMenu:(NSMenu *)menu forElement:(_WKContextMenuElementInfo *)element userInfo:(id <NSSecureCoding>)userInfo completionHandler:(void (^)(NSMenu *))completionHandler WK_API_AVAILABLE(macosx(WK_MAC_TBA));
+- (void)_webView:(WKWebView *)webView didPerformDragOperation:(BOOL)handled WK_API_AVAILABLE(macosx(WK_MAC_TBA));
 #endif // TARGET_OS_IPHONE
 
 @end
index 003a193..0f2a4b8 100644 (file)
@@ -4068,6 +4068,13 @@ WEBCORE_COMMAND(yankAndSelect)
     return WKDragDestinationActionAny & ~WKDragDestinationActionLoad;
 }
 
+- (void)_web_didPerformDragOperation:(BOOL)handled
+{
+    id <WKUIDelegatePrivate> uiDelegate = (id <WKUIDelegatePrivate>)self.UIDelegate;
+    if ([uiDelegate respondsToSelector:@selector(_webView:didPerformDragOperation:)])
+        [uiDelegate _webView:self didPerformDragOperation:handled];
+}
+
 #endif
 
 - (void)_web_dismissContentRelativeChildWindows
index f495c71..7df48b2 100644 (file)
@@ -56,18 +56,20 @@ static const NSInteger InvalidAttachmentErrorCode = 2;
 
 @implementation _WKAttachmentInfo {
     RetainPtr<NSFileWrapper> _fileWrapper;
-    RetainPtr<NSString> _contentType;
+    RetainPtr<NSString> _mimeType;
+    RetainPtr<NSString> _utiType;
     RetainPtr<NSString> _filePath;
 }
 
-- (instancetype)initWithFileWrapper:(NSFileWrapper *)fileWrapper filePath:(NSString *)filePath contentType:(NSString *)contentType
+- (instancetype)initWithFileWrapper:(NSFileWrapper *)fileWrapper filePath:(NSString *)filePath mimeType:(NSString *)mimeType utiType:(NSString *)utiType
 {
     if (!(self = [super init]))
         return nil;
 
     _fileWrapper = fileWrapper;
     _filePath = filePath;
-    _contentType = contentType;
+    _mimeType = mimeType;
+    _utiType = utiType;
     return self;
 }
 
@@ -94,48 +96,17 @@ static const NSInteger InvalidAttachmentErrorCode = 2;
     return _filePath.get();
 }
 
-static BOOL isDeclaredOrDynamicTypeIdentifier(NSString *type)
-{
-    return UTTypeIsDeclared((CFStringRef)type) || UTTypeIsDynamic((CFStringRef)type);
-}
-
-- (NSString *)_typeIdentifierFromPathExtension
+- (NSFileWrapper *)fileWrapper
 {
-    if (NSString *extension = self.name.pathExtension)
-        return WebCore::MIMETypeRegistry::getMIMETypeForExtension(extension);
-
-    return nil;
+    return _fileWrapper.get();
 }
 
 - (NSString *)contentType
 {
-    // A "content type" can refer to either a UTI or a MIME type. We prefer MIME type here to preserve existing behavior.
-    return self.mimeType ?: self.utiType;
-}
-
-- (NSString *)mimeType
-{
-    NSString *contentType = [_contentType length] ? _contentType.get() : [self _typeIdentifierFromPathExtension];
-    if (!isDeclaredOrDynamicTypeIdentifier(contentType))
-        return contentType;
+    if ([_mimeType length])
+        return _mimeType.get();
 
-    auto mimeType = adoptCF(UTTypeCopyPreferredTagWithClass((CFStringRef)contentType, kUTTagClassMIMEType));
-    return (NSString *)mimeType.autorelease();
-}
-
-- (NSString *)utiType
-{
-    NSString *contentType = [_contentType length] ? _contentType.get() : [self _typeIdentifierFromPathExtension];
-    if (isDeclaredOrDynamicTypeIdentifier(contentType))
-        return contentType;
-
-    auto utiType = adoptCF(UTTypeCreatePreferredIdentifierForTag(kUTTagClassMIMEType, (CFStringRef)contentType, nil));
-    return (NSString *)utiType.autorelease();
-}
-
-- (NSFileWrapper *)fileWrapper
-{
-    return _fileWrapper.get();
+    return _utiType.get();
 }
 
 @end
@@ -152,7 +123,7 @@ static BOOL isDeclaredOrDynamicTypeIdentifier(NSString *type)
     if (!_attachment->isValid())
         return nil;
 
-    return [[[_WKAttachmentInfo alloc] initWithFileWrapper:_attachment->fileWrapper() filePath:_attachment->filePath() contentType:_attachment->contentType()] autorelease];
+    return [[[_WKAttachmentInfo alloc] initWithFileWrapper:_attachment->fileWrapper() filePath:_attachment->filePath() mimeType:_attachment->mimeType() utiType:_attachment->utiType()] autorelease];
 }
 
 - (void)requestInfo:(void(^)(_WKAttachmentInfo *, NSError *))completionHandler
index f776fdc..e983286 100644 (file)
@@ -1031,6 +1031,11 @@ Some other editing-related methods still unimplemented:
 
 #if ENABLE(DRAG_SUPPORT) && WK_API_ENABLED
 
+- (void)_web_didPerformDragOperation:(BOOL)handled
+{
+    UNUSED_PARAM(handled);
+}
+
 - (WKDragDestinationAction)_web_dragDestinationActionForDraggingInfo:(id <NSDraggingInfo>)draggingInfo
 {
     return WKDragDestinationActionAny;
index a778b9a..63e5f66 100644 (file)
@@ -104,6 +104,7 @@ class PageConfiguration;
 
 #if ENABLE(DRAG_SUPPORT) && WK_API_ENABLED
 - (WKDragDestinationAction)_web_dragDestinationActionForDraggingInfo:(id <NSDraggingInfo>)draggingInfo;
+- (void)_web_didPerformDragOperation:(BOOL)handled;
 #endif
 
 @optional
@@ -430,6 +431,8 @@ public:
 
     NSString *fileNameForFilePromiseProvider(NSFilePromiseProvider *, NSString *fileType);
     void writeToURLForFilePromiseProvider(NSFilePromiseProvider *, NSURL *, void(^)(NSError *));
+
+    void didPerformDragOperation(bool handled);
 #endif
 
     void startWindowDrag();
index 6ed86cf..8ead862 100644 (file)
@@ -880,28 +880,28 @@ static const NSUInteger orderedListSegment = 2;
 @interface WKPromisedAttachmentContext : NSObject {
 @private
     RetainPtr<NSURL> _blobURL;
-    RetainPtr<NSString> _filename;
+    RetainPtr<NSString> _fileName;
     RetainPtr<NSString> _attachmentIdentifier;
 }
 
-- (instancetype)initWithAttachmentInfo:(const WebCore::PromisedAttachmentInfo&)info;
+- (instancetype)initWithIdentifier:(NSString *)identifier blobURL:(NSURL *)url fileName:(NSString *)fileName;
 
 @property (nonatomic, readonly) NSURL *blobURL;
-@property (nonatomic, readonly) NSString *filename;
+@property (nonatomic, readonly) NSString *fileName;
 @property (nonatomic, readonly) NSString *attachmentIdentifier;
 
 @end
 
 @implementation WKPromisedAttachmentContext
 
-- (instancetype)initWithAttachmentInfo:(const WebCore::PromisedAttachmentInfo&)info
+- (instancetype)initWithIdentifier:(NSString *)identifier blobURL:(NSURL *)blobURL fileName:(NSString *)fileName
 {
     if (!(self = [super init]))
         return nil;
 
-    _blobURL = info.blobURL;
-    _filename = info.filename;
-    _attachmentIdentifier = info.attachmentIdentifier;
+    _blobURL = blobURL;
+    _fileName = fileName;
+    _attachmentIdentifier = identifier;
     return self;
 }
 
@@ -910,9 +910,9 @@ static const NSUInteger orderedListSegment = 2;
     return _blobURL.get();
 }
 
-- (NSString *)filename
+- (NSString *)fileName
 {
-    return _filename.get();
+    return _fileName.get();
 }
 
 - (NSString *)attachmentIdentifier
@@ -3908,7 +3908,7 @@ NSString *WebViewImpl::fileNameForFilePromiseProvider(NSFilePromiseProvider *pro
     if (![userInfo isKindOfClass:[WKPromisedAttachmentContext class]])
         return nil;
 
-    return [(WKPromisedAttachmentContext *)userInfo filename];
+    return [(WKPromisedAttachmentContext *)userInfo fileName];
 }
 
 static NSError *webKitUnknownError()
@@ -3920,6 +3920,15 @@ static NSError *webKitUnknownError()
 #endif
 }
 
+void WebViewImpl::didPerformDragOperation(bool handled)
+{
+#if WK_API_ENABLED
+    [m_view _web_didPerformDragOperation:handled];
+#else
+    UNUSED_PARAM(handled);
+#endif
+}
+
 void WebViewImpl::writeToURLForFilePromiseProvider(NSFilePromiseProvider *provider, NSURL *fileURL, void(^completionHandler)(NSError *))
 {
     id userInfo = provider.userInfo;
@@ -3992,18 +4001,26 @@ void WebViewImpl::startDrag(const WebCore::DragItem& item, const ShareableBitmap
     NSPasteboard *pasteboard = [NSPasteboard pasteboardWithName:NSDragPboard];
 #pragma clang diagnostic pop
 
-    if (auto& attachmentInfo = item.promisedAttachmentInfo) {
-        auto provider = adoptNS([[NSFilePromiseProvider alloc] initWithFileType:attachmentInfo.contentType delegate:(id <NSFilePromiseProviderDelegate>)m_view.getAutoreleased()]);
-        [provider setUserInfo:[[[WKPromisedAttachmentContext alloc] initWithAttachmentInfo:attachmentInfo] autorelease]];
+    if (auto& info = item.promisedAttachmentInfo) {
+        NSString *utiType = info.contentType;
+        NSString *fileName = info.fileName;
+        if (auto attachment = m_page->attachmentForIdentifier(info.attachmentIdentifier)) {
+            utiType = attachment->utiType();
+            fileName = attachment->fileName();
+        }
+
+        auto provider = adoptNS([[NSFilePromiseProvider alloc] initWithFileType:utiType delegate:(id <NSFilePromiseProviderDelegate>)m_view.getAutoreleased()]);
+        auto context = adoptNS([[WKPromisedAttachmentContext alloc] initWithIdentifier:info.attachmentIdentifier blobURL:info.blobURL fileName:fileName]);
+        [provider setUserInfo:context.get()];
         auto draggingItem = adoptNS([[NSDraggingItem alloc] initWithPasteboardWriter:provider.get()]);
         [draggingItem setDraggingFrame:NSMakeRect(clientDragLocation.x(), clientDragLocation.y() - size.height(), size.width(), size.height()) contents:dragNSImage.get()];
         [m_view beginDraggingSessionWithItems:@[draggingItem.get()] event:m_lastMouseDownEvent.get() source:(id <NSDraggingSource>)m_view.getAutoreleased()];
 
-        ASSERT(attachmentInfo.additionalTypes.size() == attachmentInfo.additionalData.size());
-        if (attachmentInfo.additionalTypes.size() == attachmentInfo.additionalData.size()) {
-            for (size_t index = 0; index < attachmentInfo.additionalTypes.size(); ++index) {
-                auto nsData = attachmentInfo.additionalData[index]->createNSData();
-                [pasteboard setData:nsData.get() forType:attachmentInfo.additionalTypes[index]];
+        ASSERT(info.additionalTypes.size() == info.additionalData.size());
+        if (info.additionalTypes.size() == info.additionalData.size()) {
+            for (size_t index = 0; index < info.additionalTypes.size(); ++index) {
+                auto nsData = info.additionalData[index]->createNSData();
+                [pasteboard setData:nsData.get() forType:info.additionalTypes[index]];
             }
         }
         m_page->didStartDrag();
index 2683fc8..883d4bb 100644 (file)
@@ -210,6 +210,7 @@ public:
 #else
     virtual void startDrag(const WebCore::DragItem&, const ShareableBitmap::Handle&) { }
 #endif
+    virtual void didPerformDragOperation(bool) { }
 #endif // ENABLE(DRAG_SUPPORT)
 
     virtual void setCursor(const WebCore::Cursor&) = 0;
@@ -430,7 +431,6 @@ public:
 #endif
 
 #if ENABLE(DATA_INTERACTION)
-    virtual void didPerformDataInteractionControllerOperation(bool handled) = 0;
     virtual void didHandleStartDataInteractionRequest(bool started) = 0;
     virtual void didHandleAdditionalDragItemsRequest(bool added) = 0;
     virtual void didConcludeEditDataInteraction(std::optional<WebCore::TextIndicatorData>) = 0;
index e072bc2..002becf 100644 (file)
@@ -1870,6 +1870,11 @@ void WebPageProxy::dragEnded(const IntPoint& clientPosition, const IntPoint& glo
     setDragCaretRect({ });
 }
 
+void WebPageProxy::didPerformDragOperation(bool handled)
+{
+    m_pageClient.didPerformDragOperation(handled);
+}
+
 void WebPageProxy::didStartDrag()
 {
     if (isValid())
index d6934d4..2eb241e 100644 (file)
@@ -628,7 +628,6 @@ public:
     void startAutoscrollAtPosition(const WebCore::FloatPoint& positionInWindow);
     void cancelAutoscroll();
 #if ENABLE(DATA_INTERACTION)
-    void didPerformDataInteractionControllerOperation(bool handled);
     void didHandleStartDataInteractionRequest(bool started);
     void didHandleAdditionalDragItemsRequest(bool added);
     void requestStartDataInteraction(const WebCore::IntPoint& clientPosition, const WebCore::IntPoint& globalPosition);
@@ -923,6 +922,7 @@ public:
     void dragUpdated(WebCore::DragData&, const String& dragStorageName = String());
     void dragExited(WebCore::DragData&, const String& dragStorageName = String());
     void performDragOperation(WebCore::DragData&, const String& dragStorageName, SandboxExtension::Handle&&, SandboxExtension::HandleArray&&);
+    void didPerformDragOperation(bool handled);
 
     void didPerformDragControllerAction(uint64_t dragOperation, bool mouseIsOverFileInput, unsigned numberOfItemsToBeAccepted, const WebCore::IntRect& insertionRect);
     void dragEnded(const WebCore::IntPoint& clientPosition, const WebCore::IntPoint& globalPosition, uint64_t operation);
index b8c01c0..ed11e37 100644 (file)
@@ -321,8 +321,11 @@ messages -> WebPageProxy {
     StartDrag(struct WebKit::WebSelectionData selection, uint64_t dragOperation, WebKit::ShareableBitmap::Handle dragImage)
 #endif
 
+#if ENABLE(DRAG_SUPPORT)
+    DidPerformDragOperation(bool handled)
+#endif
+
 #if ENABLE(DATA_INTERACTION)
-    DidPerformDataInteractionControllerOperation(bool handled)
     DidHandleStartDataInteractionRequest(bool started)
     DidHandleAdditionalDragItemsRequest(bool added)
     DidConcludeEditDataInteraction(std::optional<WebCore::TextIndicatorData> textIndicator)
index 415e669..13fbd29 100644 (file)
@@ -210,7 +210,7 @@ private:
 #endif
 
 #if ENABLE(DATA_INTERACTION)
-    void didPerformDataInteractionControllerOperation(bool handled) override;
+    void didPerformDragOperation(bool handled) override;
     void didHandleStartDataInteractionRequest(bool started) override;
     void didHandleAdditionalDragItemsRequest(bool added) override;
     void startDrag(const WebCore::DragItem&, const ShareableBitmap::Handle& image) override;
index 629b299..d79d0c2 100644 (file)
@@ -778,9 +778,9 @@ RefPtr<WebDataListSuggestionsDropdown> PageClientImpl::createDataListSuggestions
 #endif
 
 #if ENABLE(DATA_INTERACTION)
-void PageClientImpl::didPerformDataInteractionControllerOperation(bool handled)
+void PageClientImpl::didPerformDragOperation(bool handled)
 {
-    [m_contentView _didPerformDataInteractionControllerOperation:handled];
+    [m_contentView _didPerformDragOperation:handled];
 }
 
 void PageClientImpl::didHandleStartDataInteractionRequest(bool started)
index a22830a..bfa67be 100644 (file)
@@ -341,7 +341,7 @@ FOR_EACH_WKCONTENTVIEW_ACTION(DECLARE_WKCONTENTVIEW_ACTION_FOR_WEB_VIEW)
 
 #if ENABLE(DATA_INTERACTION)
 - (void)_didChangeDragInteractionPolicy;
-- (void)_didPerformDataInteractionControllerOperation:(BOOL)handled;
+- (void)_didPerformDragOperation:(BOOL)handled;
 - (void)_didHandleStartDataInteractionRequest:(BOOL)started;
 - (void)_didHandleAdditionalDragItemsRequest:(BOOL)added;
 - (void)_startDrag:(RetainPtr<CGImageRef>)image item:(const WebCore::DragItem&)item;
index f75c13b..306f38b 100644 (file)
@@ -5076,7 +5076,7 @@ static NSArray<UIItemProvider *> *extractItemProvidersFromDropSession(id <UIDrop
     }];
 }
 
-- (void)_didPerformDataInteractionControllerOperation:(BOOL)handled
+- (void)_didPerformDragOperation:(BOOL)handled
 {
     RELEASE_LOG(DragAndDrop, "Finished performing drag controller operation (handled: %d)", handled);
     [[WebItemProviderPasteboard sharedInstance] decrementPendingOperationCount];
@@ -5129,10 +5129,17 @@ static NSArray<UIItemProvider *> *extractItemProvidersFromDropSession(id <UIDrop
 
     RELEASE_LOG(DragAndDrop, "Drag session: %p preparing to drag blob: %s with attachment identifier: %s", session.get(), info.blobURL.string().utf8().data(), info.attachmentIdentifier.utf8().data());
 
+    NSString *utiType = info.contentType;
+    NSString *fileName = info.fileName;
+    if (auto attachment = _page->attachmentForIdentifier(info.attachmentIdentifier)) {
+        utiType = attachment->utiType();
+        fileName = attachment->fileName();
+    }
+
     auto registrationList = adoptNS([[WebItemProviderRegistrationInfoList alloc] init]);
     [registrationList setPreferredPresentationStyle:WebPreferredPresentationStyleAttachment];
-    if (!info.filename.isEmpty())
-        [registrationList setSuggestedName:info.filename];
+    if ([fileName length])
+        [registrationList setSuggestedName:fileName];
     if (numberOfAdditionalTypes == info.additionalData.size() && numberOfAdditionalTypes) {
         for (size_t index = 0; index < numberOfAdditionalTypes; ++index) {
             auto nsData = info.additionalData[index]->createNSData();
@@ -5140,7 +5147,7 @@ static NSArray<UIItemProvider *> *extractItemProvidersFromDropSession(id <UIDrop
         }
     }
 
-    [registrationList addPromisedType:info.contentType fileCallback:[session = WTFMove(session), weakSelf = WeakObjCPtr<WKContentView>(self), info] (WebItemProviderFileCallback callback) {
+    [registrationList addPromisedType:utiType fileCallback:[session = WTFMove(session), weakSelf = WeakObjCPtr<WKContentView>(self), info] (WebItemProviderFileCallback callback) {
         auto strongSelf = weakSelf.get();
         if (!strongSelf) {
             callback(nil, [NSError errorWithDomain:WKErrorDomain code:WKErrorWebViewInvalidated userInfo:nil]);
index c12754c..b96703d 100644 (file)
@@ -1088,11 +1088,6 @@ void WebPageProxy::setIsScrollingOrZooming(bool isScrollingOrZooming)
 
 #if ENABLE(DATA_INTERACTION)
 
-void WebPageProxy::didPerformDataInteractionControllerOperation(bool handled)
-{
-    m_pageClient.didPerformDataInteractionControllerOperation(handled);
-}
-
 void WebPageProxy::didHandleStartDataInteractionRequest(bool started)
 {
     m_pageClient.didHandleStartDataInteractionRequest(started);
index c884fa2..7e462e8 100644 (file)
@@ -233,6 +233,10 @@ private:
     WebCore::UserInterfaceLayoutDirection userInterfaceLayoutDirection() override;
     bool effectiveAppearanceIsDark() const override;
 
+#if ENABLE(DRAG_SUPPORT)
+    void didPerformDragOperation(bool handled) final;
+#endif
+
 #if WK_API_ENABLED
     NSView *inspectorAttachmentView() override;
     _WKRemoteObjectRegistry *remoteObjectRegistry() override;
index a0bc9eb..e15b442 100644 (file)
@@ -874,6 +874,15 @@ NSWindow *PageClientImpl::platformWindow()
     return m_impl->window();
 }
 
+#if ENABLE(DRAG_SUPPORT)
+
+void PageClientImpl::didPerformDragOperation(bool handled)
+{
+    m_impl->didPerformDragOperation(handled);
+}
+
+#endif
+
 #if WK_API_ENABLED
 NSView *PageClientImpl::inspectorAttachmentView()
 {
index 924a747..c771559 100644 (file)
                ED82A7F2128C6FAF004477B3 /* WKBundlePageOverlay.h in Headers */ = {isa = PBXBuildFile; fileRef = 1A22F0FF1289FCD90085E74F /* WKBundlePageOverlay.h */; settings = {ATTRIBUTES = (Private, ); }; };
                EDCA71B7128DDA8C00201B26 /* WKBundlePageOverlay.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 1A22F1001289FCD90085E74F /* WKBundlePageOverlay.cpp */; };
                F409BA181E6E64BC009DA28E /* WKDragDestinationAction.h in Headers */ = {isa = PBXBuildFile; fileRef = F409BA171E6E64B3009DA28E /* WKDragDestinationAction.h */; settings = {ATTRIBUTES = (Private, ); }; };
+               F41056622130699A0092281D /* APIAttachmentCocoa.h in Headers */ = {isa = PBXBuildFile; fileRef = F41056602130699A0092281D /* APIAttachmentCocoa.h */; };
+               F41056632130699A0092281D /* APIAttachmentCocoa.mm in Sources */ = {isa = PBXBuildFile; fileRef = F41056612130699A0092281D /* APIAttachmentCocoa.mm */; };
                F44291921FA591C9002CC93E /* _WKAttachment.h in Headers */ = {isa = PBXBuildFile; fileRef = F44291911FA59107002CC93E /* _WKAttachment.h */; settings = {ATTRIBUTES = (Private, ); }; };
                F44291941FA59311002CC93E /* _WKAttachment.mm in Sources */ = {isa = PBXBuildFile; fileRef = F44291931FA59311002CC93E /* _WKAttachment.mm */; };
                F44291961FA5942A002CC93E /* _WKAttachmentInternal.h in Headers */ = {isa = PBXBuildFile; fileRef = F44291951FA5942A002CC93E /* _WKAttachmentInternal.h */; };
                ECBFC1DB1E6A4D66000300C7 /* ExtraPublicSymbolsForTAPI.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = ExtraPublicSymbolsForTAPI.h; sourceTree = "<group>"; };
                F036978715F4BF0500C3A80E /* WebColorPicker.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = WebColorPicker.cpp; sourceTree = "<group>"; };
                F409BA171E6E64B3009DA28E /* WKDragDestinationAction.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WKDragDestinationAction.h; sourceTree = "<group>"; };
+               F41056602130699A0092281D /* APIAttachmentCocoa.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = APIAttachmentCocoa.h; sourceTree = "<group>"; };
+               F41056612130699A0092281D /* APIAttachmentCocoa.mm */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.objcpp; path = APIAttachmentCocoa.mm; sourceTree = "<group>"; };
                F44291911FA59107002CC93E /* _WKAttachment.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = _WKAttachment.h; sourceTree = "<group>"; };
                F44291931FA59311002CC93E /* _WKAttachment.mm */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.objcpp; path = _WKAttachment.mm; sourceTree = "<group>"; };
                F44291951FA5942A002CC93E /* _WKAttachmentInternal.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = _WKAttachmentInternal.h; sourceTree = "<group>"; };
                                5CB237891DF0DD4300117AA3 /* _WKWebsitePolicies.h */,
                                5CB2378A1DF0DD4300117AA3 /* _WKWebsitePolicies.mm */,
                                5CB2378D1DF0E0C200117AA3 /* _WKWebsitePoliciesInternal.h */,
+                               F41056602130699A0092281D /* APIAttachmentCocoa.h */,
+                               F41056612130699A0092281D /* APIAttachmentCocoa.mm */,
                                7CEFA9601AC0999300B910FD /* APIContentRuleListStoreCocoa.mm */,
                                FED3C1DA1B447AE800E0EB7F /* APISerializedScriptValueCocoa.mm */,
                                1A3635AB1A3145E500ED6197 /* APIWebsiteDataStoreCocoa.mm */,
                                634842511FB26E7100946E3C /* APIApplicationManifest.h in Headers */,
                                BC64697011DBE603006455B0 /* APIArray.h in Headers */,
                                2E5C770E1FA7D429005932C3 /* APIAttachment.h in Headers */,
+                               F41056622130699A0092281D /* APIAttachmentCocoa.h in Headers */,
                                99C81D5D1C21F38B005C4C82 /* APIAutomationClient.h in Headers */,
                                990D28C01C6553F100986977 /* APIAutomationSessionClient.h in Headers */,
                                1A3DD206125E5A2F004515E6 /* APIClient.h in Headers */,
                                2D92A784212B6AB100F493FD /* ActivityAssertion.cpp in Sources */,
                                BC64696F11DBE603006455B0 /* APIArray.cpp in Sources */,
                                2E5C770F1FA7D429005932C3 /* APIAttachment.cpp in Sources */,
+                               F41056632130699A0092281D /* APIAttachmentCocoa.mm in Sources */,
                                7C89D2B31A6B068C003A5FDE /* APIContentRuleList.cpp in Sources */,
                                7C3A06A71AAB903E009D74BA /* APIContentRuleListStore.cpp in Sources */,
                                7CEFA9621AC0999300B910FD /* APIContentRuleListStoreCocoa.mm in Sources */,
index 9af688c..b5b711a 100644 (file)
@@ -3459,11 +3459,7 @@ void WebPage::performDragControllerAction(DragControllerAction action, const Web
         m_pendingDropSandboxExtension = nullptr;
 
         m_pendingDropExtensionsForFileUpload.clear();
-#if ENABLE(DATA_INTERACTION)
-        send(Messages::WebPageProxy::DidPerformDataInteractionControllerOperation(handled));
-#else
-        UNUSED_PARAM(handled);
-#endif
+        send(Messages::WebPageProxy::DidPerformDragOperation(handled));
         return;
     }
     }
index e811ef4..d75a222 100644 (file)
@@ -1,3 +1,39 @@
+2018-08-26  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        [Attachment Support] Dropping and pasting images should insert inline image elements with _WKAttachments
+        https://bugs.webkit.org/show_bug.cgi?id=188933
+        <rdar://problem/43699724>
+
+        Reviewed by Darin Adler.
+
+        Rebaseline existing API tests that involve dropping or pasting image files, and additionally write some new
+        tests. These new tests exercise the following cases:
+        •       Inserting and removing newlines before an inline image with an attachment element does not cause new
+                _WKAttachments to be created and destroyed.
+        •       Pasting an image, cutting it, and then pasting it again propagates an attachment update to the UI
+                process with the original _WKAttachment.
+        •       A pasted attachment in the document can be moved around by dragging, and doing so does not cause us to
+                lose a _WKAttachment.
+
+        * TestWebKitAPI/Tests/WebKitCocoa/WKAttachmentTests.mm:
+        (-[TestWKWebView expectElementCount:tagName:]):
+        (TestWebKitAPI::TEST):
+
+        Add the new tests described above, and also adjust existing tests to check that images are dropped or pasted
+        as image elements, but still have associated attachment elements whose attachment identifiers (observed via
+        script) match that of the corresponding _WKAttachment's uniqueIdentifier in the UI process.
+
+        * TestWebKitAPI/mac/DragAndDropSimulatorMac.mm:
+        (-[DragAndDropSimulator runFrom:to:]):
+        (-[DragAndDropSimulator continueDragSession]):
+        (-[DragAndDropSimulator performDragInWebView:atLocation:withImage:pasteboard:source:]):
+
+        Teach DragAndDropSimulator on macOS to wait until the drop has been handled by the web process before returning
+        execution to the caller. This ensures that tests which involve dropping promised files as attachments aren't
+        flaky, due to how the promised data is retrieved asynchronously when performing the drop.
+
+        (-[DragAndDropSimulator _webView:didPerformDragOperation:]):
+
 2018-08-26  Lucas Forschler  <lforschler@apple.com>
 
         Open svn.webkit.org for commits.
index 8aaa1da..dd26879 100644 (file)
@@ -225,6 +225,12 @@ static NSData *testPDFData()
         NSLog(@"Expected to find ordered tags: %@ in: %@", tagNames, tagsInBody);
 }
 
+- (void)expectElementCount:(NSInteger)count tagName:(NSString *)tagName
+{
+    NSString *script = [NSString stringWithFormat:@"document.querySelectorAll('%@').length", tagName];
+    EXPECT_EQ(count, [self stringByEvaluatingJavaScript:script].integerValue);
+}
+
 - (void)expectElementTag:(NSString *)tagName toComeBefore:(NSString *)otherTagName
 {
     [self expectElementTagsInOrder:@[tagName, otherTagName]];
@@ -779,7 +785,7 @@ TEST(WKAttachmentTests, ChangeAttachmentDataAndFileInformation)
     [webView expectUpdatesAfterCommand:@"DeleteBackward" withArgument:nil expectedRemovals:@[attachment.get()] expectedInsertions:@[]];
 }
 
-TEST(WKAttachmentTests, InsertPastedImageAsAttachment)
+TEST(WKAttachmentTests, RemoveNewlinesBeforePastedImage)
 {
     platformCopyPNG();
 
@@ -795,16 +801,76 @@ TEST(WKAttachmentTests, InsertPastedImageAsAttachment)
     auto size = platformImageWithData([attachment info].data).size;
     EXPECT_EQ(215., size.width);
     EXPECT_EQ(174., size.height);
-    EXPECT_WK_STREQ([attachment uniqueIdentifier], [webView stringByEvaluatingJavaScript:@"document.querySelector('attachment').uniqueIdentifier"]);
+    EXPECT_WK_STREQ([attachment uniqueIdentifier], [webView stringByEvaluatingJavaScript:@"document.querySelector('img').webkitAttachmentIdentifier"]);
 
+    [webView stringByEvaluatingJavaScript:@"getSelection().collapse(document.body, 0)"];
+    {
+        ObserveAttachmentUpdatesForScope observer(webView.get());
+        [webView _synchronouslyExecuteEditCommand:@"InsertParagraph" argument:nil];
+        [webView _synchronouslyExecuteEditCommand:@"InsertParagraph" argument:nil];
+        observer.expectAttachmentUpdates(@[ ], @[ ]);
+        [webView expectElementTagsInOrder:@[ @"BR", @"BR", @"IMG" ]];
+    }
     {
         ObserveAttachmentUpdatesForScope observer(webView.get());
-        [webView _synchronouslyExecuteEditCommand:@"SelectAll" argument:nil];
         [webView _synchronouslyExecuteEditCommand:@"DeleteBackward" argument:nil];
-        observer.expectAttachmentUpdates(@[attachment.get()], @[]);
+        [webView _synchronouslyExecuteEditCommand:@"DeleteBackward" argument:nil];
+        observer.expectAttachmentUpdates(@[ ], @[ ]);
+        [webView expectElementCount:0 tagName:@"BR"];
+    }
+}
+
+TEST(WKAttachmentTests, CutAndPastePastedImage)
+{
+    platformCopyPNG();
+
+    RetainPtr<_WKAttachment> attachment;
+    auto webView = webViewForTestingAttachments();
+    {
+        ObserveAttachmentUpdatesForScope observer(webView.get());
+        [webView _synchronouslyExecuteEditCommand:@"Paste" argument:nil];
+        EXPECT_EQ(1U, observer.observer().inserted.count);
+        attachment = observer.observer().inserted[0];
+    }
+    {
+        ObserveAttachmentUpdatesForScope observer(webView.get());
+        [webView _synchronouslyExecuteEditCommand:@"SelectAll" argument:nil];
+        [webView _synchronouslyExecuteEditCommand:@"Cut" argument:nil];
+        observer.expectAttachmentUpdates(@[ attachment.get() ], @[ ]);
+    }
+    {
+        ObserveAttachmentUpdatesForScope observer(webView.get());
+        [webView _synchronouslyExecuteEditCommand:@"Paste" argument:nil];
+        observer.expectAttachmentUpdates(@[ ], @[ attachment.get() ]);
     }
 }
 
+TEST(WKAttachmentTests, MovePastedImageByDragging)
+{
+    auto configuration = adoptNS([[WKWebViewConfiguration alloc] init]);
+    [configuration _setAttachmentElementEnabled:YES];
+    auto simulator = adoptNS([[DragAndDropSimulator alloc] initWithWebViewFrame:NSMakeRect(0, 0, 400, 400) configuration:configuration.get()]);
+    TestWKWebView *webView = [simulator webView];
+    [webView synchronouslyLoadHTMLString:attachmentEditingTestMarkup];
+
+    platformCopyPNG();
+    [webView _synchronouslyExecuteEditCommand:@"Paste" argument:nil];
+    [webView _executeEditCommand:@"InsertParagraph" argument:nil completion:nil];
+    [webView _executeEditCommand:@"InsertHTML" argument:@"<strong>text</strong>" completion:nil];
+    [webView _synchronouslyExecuteEditCommand:@"InsertParagraph" argument:nil];
+    [webView expectElementTag:@"IMG" toComeBefore:@"STRONG"];
+    [webView expectElementCount:1 tagName:@"IMG"];
+
+    // Drag the attachment element to somewhere below the strong text.
+    [simulator runFrom:CGPointMake(50, 50) to:CGPointMake(50, 350)];
+
+    [webView expectElementTag:@"STRONG" toComeBefore:@"IMG"];
+    [webView expectElementCount:1 tagName:@"IMG"];
+    EXPECT_EQ([simulator insertedAttachments].count, [simulator removedAttachments].count);
+
+    [simulator endDataTransfer];
+}
+
 TEST(WKAttachmentTests, InsertPastedAttributedStringContainingImage)
 {
     platformCopyRichTextWithImage();
@@ -821,8 +887,7 @@ TEST(WKAttachmentTests, InsertPastedAttributedStringContainingImage)
 
     [attachment expectRequestedDataToBe:testImageData()];
     EXPECT_WK_STREQ("Lorem ipsum  dolor sit amet.", [webView stringByEvaluatingJavaScript:@"document.body.textContent"]);
-    EXPECT_WK_STREQ("image/png", [webView valueOfAttribute:@"type" forQuerySelector:@"attachment"]);
-    EXPECT_WK_STREQ([attachment uniqueIdentifier], [webView stringByEvaluatingJavaScript:@"document.querySelector('attachment').uniqueIdentifier"]);
+    EXPECT_WK_STREQ([attachment uniqueIdentifier], [webView stringByEvaluatingJavaScript:@"document.querySelector('img').webkitAttachmentIdentifier"]);
 
     {
         ObserveAttachmentUpdatesForScope observer(webView.get());
@@ -857,20 +922,20 @@ TEST(WKAttachmentTests, InsertPastedAttributedStringContainingMultipleAttachment
     }
 
     EXPECT_TRUE(zipAttachment && imageAttachment && pdfAttachment);
-    EXPECT_EQ(3, [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment').length"].integerValue);
-    EXPECT_WK_STREQ("image/png", [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[0].getAttribute('type')"]);
-    EXPECT_WK_STREQ("application/pdf", [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[1].getAttribute('type')"]);
+    [webView expectElementCount:2 tagName:@"ATTACHMENT"];
+    [webView expectElementCount:1 tagName:@"IMG"];
+    EXPECT_WK_STREQ("application/pdf", [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[0].getAttribute('type')"]);
 
-    NSString *zipAttachmentType = [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[2].getAttribute('type')"];
+    NSString *zipAttachmentType = [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[1].getAttribute('type')"];
 #if USES_MODERN_ATTRIBUTED_STRING_CONVERSION
     EXPECT_WK_STREQ("application/zip", zipAttachmentType);
 #else
     EXPECT_WK_STREQ("application/octet-stream", zipAttachmentType);
 #endif
 
-    EXPECT_WK_STREQ([imageAttachment uniqueIdentifier], [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[0].uniqueIdentifier"]);
-    EXPECT_WK_STREQ([pdfAttachment uniqueIdentifier], [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[1].uniqueIdentifier"]);
-    EXPECT_WK_STREQ([zipAttachment uniqueIdentifier], [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[2].uniqueIdentifier"]);
+    EXPECT_WK_STREQ([imageAttachment uniqueIdentifier], [webView stringByEvaluatingJavaScript:@"document.querySelector('img').webkitAttachmentIdentifier"]);
+    EXPECT_WK_STREQ([pdfAttachment uniqueIdentifier], [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[0].uniqueIdentifier"]);
+    EXPECT_WK_STREQ([zipAttachment uniqueIdentifier], [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[1].uniqueIdentifier"]);
 
     {
         ObserveAttachmentUpdatesForScope observer(webView.get());
@@ -992,6 +1057,7 @@ TEST(WKAttachmentTests, InjectedBundleReplaceURLsWhenPastingAttributedString)
     }
     [webView expectElementTagsInOrder:@[ @"IMG", @"ATTACHMENT", @"ATTACHMENT" ]];
     EXPECT_WK_STREQ("cid:foo-bar", [webView valueOfAttribute:@"src" forQuerySelector:@"img"]);
+    EXPECT_WK_STREQ(@"", [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('img')[0].webkitAttachmentIdentifier"]);
 }
 
 TEST(WKAttachmentTests, InjectedBundleReplaceURLWhenPastingImage)
@@ -1124,10 +1190,10 @@ TEST(WKAttachmentTests, MoveAttachmentElementAsIconByDragging)
     // Drag the attachment element to somewhere below the strong text.
     [simulator runFrom:[webView attachmentElementMidPoint] to:CGPointMake(50, 300)];
 
-    EXPECT_EQ([simulator insertedAttachments].count, [simulator removedAttachments].count);
-    [attachment expectRequestedDataToBe:data.get()];
     EXPECT_WK_STREQ("document.pdf", [webView valueOfAttribute:@"title" forQuerySelector:@"attachment"]);
     EXPECT_WK_STREQ("application/pdf", [webView valueOfAttribute:@"type" forQuerySelector:@"attachment"]);
+    [attachment expectRequestedDataToBe:data.get()];
+    EXPECT_EQ([simulator insertedAttachments].count, [simulator removedAttachments].count);
 #if PLATFORM(MAC)
     EXPECT_FALSE(isCompletelyTransparent([simulator draggingInfo].draggedImage));
 #endif
@@ -1156,13 +1222,16 @@ TEST(WKAttachmentTestsMac, InsertPastedFileURLsAsAttachments)
         EXPECT_EQ(2U, [insertedAttachments count]);
     }
 
-    NSArray<NSData *> *expectedAttachmentData = @[ testPDFData(), testImageData() ];
-    EXPECT_TRUE([expectedAttachmentData containsObject:[insertedAttachments firstObject].info.data]);
-    EXPECT_TRUE([expectedAttachmentData containsObject:[insertedAttachments lastObject].info.data]);
-    EXPECT_WK_STREQ("application/pdf", [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[0].getAttribute('type')"]);
-    EXPECT_WK_STREQ("test.pdf", [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[0].getAttribute('title')"]);
-    EXPECT_WK_STREQ("image/png", [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[1].getAttribute('type')"]);
-    EXPECT_WK_STREQ("icon.png", [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[1].getAttribute('title')"]);
+    [webView expectElementCount:1 tagName:@"ATTACHMENT"];
+    [webView expectElementCount:1 tagName:@"IMG"];
+    EXPECT_WK_STREQ("application/pdf", [webView stringByEvaluatingJavaScript:@"document.querySelector('attachment').getAttribute('type')"]);
+    EXPECT_WK_STREQ("test.pdf", [webView stringByEvaluatingJavaScript:@"document.querySelector('attachment').getAttribute('title')"]);
+
+    NSString *imageAttachmentIdentifier = [webView stringByEvaluatingJavaScript:@"document.querySelector('img').webkitAttachmentIdentifier"];
+    if ([testImageData() isEqualToData:[insertedAttachments firstObject].info.data])
+        EXPECT_WK_STREQ([insertedAttachments firstObject].uniqueIdentifier, imageAttachmentIdentifier);
+    else
+        EXPECT_WK_STREQ([insertedAttachments lastObject].uniqueIdentifier, imageAttachmentIdentifier);
 
     for (_WKAttachment *attachment in insertedAttachments.get())
         EXPECT_GT(attachment.info.filePath.length, 0U);
@@ -1188,11 +1257,10 @@ TEST(WKAttachmentTestsMac, InsertDroppedFilePromisesAsAttachments)
     [simulator writePromisedFiles:@[ testPDFFileURL(), testImageFileURL() ]];
 
     [simulator runFrom:CGPointMake(0, 0) to:CGPointMake(50, 50)];
-    while ([[NSRunLoop currentRunLoop] runMode:NSDefaultRunLoopMode beforeDate:[NSDate distantPast]]) {
-        if ([simulator insertedAttachments].count == 2)
-            break;
-    }
-    EXPECT_EQ(2, [[webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment').length"] intValue]);
+
+    [webView expectElementCount:1 tagName:@"ATTACHMENT"];
+    [webView expectElementCount:1 tagName:@"IMG"];
+    EXPECT_EQ(2U, [simulator insertedAttachments].count);
 
     auto insertedAttachments = retainPtr([simulator insertedAttachments]);
     NSArray<NSData *> *expectedData = @[ testPDFData(), testImageData() ];
@@ -1201,15 +1269,18 @@ TEST(WKAttachmentTestsMac, InsertDroppedFilePromisesAsAttachments)
         EXPECT_TRUE([expectedData containsObject:attachment.info.data]);
         if ([testPDFData() isEqualToData:attachment.info.data])
             EXPECT_WK_STREQ("application/pdf", attachment.info.contentType);
-        else if ([testImageData() isEqualToData:attachment.info.data])
+        else if ([testImageData() isEqualToData:attachment.info.data]) {
             EXPECT_WK_STREQ("image/png", attachment.info.contentType);
+            EXPECT_WK_STREQ(attachment.uniqueIdentifier, [webView stringByEvaluatingJavaScript:@"document.querySelector('img').webkitAttachmentIdentifier"]);
+        }
     }
 
     [webView _synchronouslyExecuteEditCommand:@"SelectAll" argument:nil];
     [webView _synchronouslyExecuteEditCommand:@"DeleteBackward" argument:nil];
     auto removedAttachments = retainPtr([simulator removedAttachments]);
     EXPECT_EQ(2U, [removedAttachments count]);
-    EXPECT_EQ(0, [[webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment').length"] intValue]);
+    [webView expectElementCount:0 tagName:@"ATTACHMENT"];
+    [webView expectElementCount:0 tagName:@"IMG"];
     EXPECT_TRUE([removedAttachments containsObject:[insertedAttachments firstObject]]);
     EXPECT_TRUE([removedAttachments containsObject:[insertedAttachments lastObject]]);
 }
@@ -1228,6 +1299,7 @@ TEST(WKAttachmentTestsMac, DragAttachmentAsFilePromise)
 
     NSArray<NSURL *> *urls = [simulator receivePromisedFiles];
     EXPECT_EQ(1U, urls.count);
+    EXPECT_WK_STREQ("test.pdf", urls.lastObject.lastPathComponent);
     EXPECT_TRUE([[NSData dataWithContentsOfURL:urls.firstObject] isEqualToData:testPDFData()]);
     EXPECT_FALSE(isCompletelyTransparent([simulator draggingInfo].draggedImage));
 }
@@ -1249,7 +1321,7 @@ TEST(WKAttachmentTestsIOS, InsertDroppedImageAsAttachment)
     EXPECT_EQ(0U, [dragAndDropSimulator removedAttachments].count);
     auto attachment = retainPtr([dragAndDropSimulator insertedAttachments].firstObject);
     [attachment expectRequestedDataToBe:testImageData()];
-    EXPECT_WK_STREQ("public.png", [webView valueOfAttribute:@"type" forQuerySelector:@"attachment"]);
+    EXPECT_WK_STREQ([attachment uniqueIdentifier], [webView stringByEvaluatingJavaScript:@"document.querySelector('img').webkitAttachmentIdentifier"]);
 
     {
         ObserveAttachmentUpdatesForScope observer(webView.get());
@@ -1277,7 +1349,7 @@ TEST(WKAttachmentTestsIOS, InsertDroppedAttributedStringContainingAttachment)
     auto size = platformImageWithData([attachment info].data).size;
     EXPECT_EQ(215., size.width);
     EXPECT_EQ(174., size.height);
-    EXPECT_WK_STREQ("image/png", [webView valueOfAttribute:@"type" forQuerySelector:@"attachment"]);
+    EXPECT_WK_STREQ([attachment uniqueIdentifier], [webView stringByEvaluatingJavaScript:@"document.querySelector('img').webkitAttachmentIdentifier"]);
 
     {
         ObserveAttachmentUpdatesForScope observer(webView.get());
@@ -1315,7 +1387,7 @@ TEST(WKAttachmentTestsIOS, InsertDroppedRichAndPlainTextFilesAsAttachments)
     for (_WKAttachment *attachment in [dragAndDropSimulator insertedAttachments])
         EXPECT_GT([attachment info].data.length, 0U);
 
-    EXPECT_EQ(2, [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment').length"].intValue);
+    [webView expectElementCount:2 tagName:@"ATTACHMENT"];
     EXPECT_WK_STREQ("hello.rtf", [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[0].getAttribute('title')"]);
     EXPECT_WK_STREQ("text/rtf", [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[0].getAttribute('type')"]);
     EXPECT_WK_STREQ("world.txt", [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[1].getAttribute('title')"]);
@@ -1340,7 +1412,7 @@ TEST(WKAttachmentTestsIOS, InsertDroppedZipArchiveAsAttachment)
     EXPECT_EQ(1U, [dragAndDropSimulator insertedAttachments].count);
     EXPECT_EQ(0U, [dragAndDropSimulator removedAttachments].count);
     [[dragAndDropSimulator insertedAttachments].firstObject expectRequestedDataToBe:data];
-    EXPECT_EQ(1, [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment').length"].intValue);
+    [webView expectElementCount:1 tagName:@"ATTACHMENT"];
     EXPECT_WK_STREQ("archive.zip", [webView valueOfAttribute:@"title" forQuerySelector:@"attachment"]);
     EXPECT_WK_STREQ("application/zip", [webView valueOfAttribute:@"type" forQuerySelector:@"attachment"]);
 }
index efd70bd..e528301 100644 (file)
@@ -103,6 +103,7 @@ static NSImage *defaultExternalDragImage()
     NSPoint _endLocationInWindow;
     double _progress;
     bool _doneWaitingForDraggingSession;
+    bool _doneWaitingForDrop;
 }
 
 @synthesize currentDragOperation=_currentDragOperation;
@@ -155,6 +156,7 @@ static NSImage *defaultExternalDragImage()
     _insertedAttachments = adoptNS([NSMutableArray new]);
     _removedAttachments = adoptNS([NSMutableArray new]);
     _doneWaitingForDraggingSession = true;
+    _doneWaitingForDrop = true;
     _startLocationInWindow = [self flipAboutXAxisInHostWindow:flippedStartLocation];
     _endLocationInWindow = [self flipAboutXAxisInHostWindow:flippedEndLocation];
     _currentDragOperation = NSDragOperationNone;
@@ -167,6 +169,7 @@ static NSImage *defaultExternalDragImage()
         NSPoint startLocationInView = [_webView convertPoint:_startLocationInWindow fromView:nil];
         NSImage *dragImage = self.externalDragImage ?: defaultExternalDragImage();
         [self performDragInWebView:_webView.get() atLocation:startLocationInView withImage:dragImage pasteboard:pasteboard source:nil];
+        TestWebKitAPI::Util::run(&_doneWaitingForDrop);
         return;
     }
 
@@ -186,6 +189,8 @@ static NSImage *defaultExternalDragImage()
 
     [_webView mouseUpAtPoint:_endLocationInWindow];
     [_webView waitForPendingMouseEvents];
+
+    TestWebKitAPI::Util::run(&_doneWaitingForDrop);
 }
 
 - (void)beginDraggingSessionInWebView:(DragAndDropTestWKWebView *)webView withItems:(NSArray<NSDraggingItem *> *)items source:(id<NSDraggingSource>)source
@@ -238,9 +243,10 @@ static NSImage *defaultExternalDragImage()
     if (_willEndDraggingHandler)
         _willEndDraggingHandler();
 
-    if (_currentDragOperation != NSDragOperationNone && [_webView prepareForDragOperation:_draggingInfo.get()])
+    if (_currentDragOperation != NSDragOperationNone && [_webView prepareForDragOperation:_draggingInfo.get()]) {
+        _doneWaitingForDrop = false;
         [_webView performDragOperation:_draggingInfo.get()];
-    else if (_currentDragOperation == NSDragOperationNone)
+    else if (_currentDragOperation == NSDragOperationNone)
         [_webView draggingExited:_draggingInfo.get()];
     [_webView waitForNextPresentationUpdate];
     [(id <NSDraggingSource>)_webView.get() draggingSession:_draggingSession.get() endedAtPoint:_endLocationInWindow operation:_currentDragOperation];
@@ -268,9 +274,10 @@ static NSImage *defaultExternalDragImage()
     if (_willEndDraggingHandler)
         _willEndDraggingHandler();
 
-    if (_currentDragOperation != NSDragOperationNone && [_webView prepareForDragOperation:_draggingInfo.get()])
+    if (_currentDragOperation != NSDragOperationNone && [_webView prepareForDragOperation:_draggingInfo.get()]) {
+        _doneWaitingForDrop = false;
         [_webView performDragOperation:_draggingInfo.get()];
-    else if (_currentDragOperation == NSDragOperationNone)
+    else if (_currentDragOperation == NSDragOperationNone)
         [_webView draggingExited:_draggingInfo.get()];
     [_webView waitForNextPresentationUpdate];
 
@@ -437,6 +444,8 @@ static BOOL getFilePathsAndTypeIdentifiers(NSArray<NSURL *> *fileURLs, NSArray<N
 {
 }
 
+#pragma mark - WKUIDelegatePrivate
+
 - (void)_webView:(WKWebView *)webView didInsertAttachment:(_WKAttachment *)attachment withSource:(NSString *)source
 {
     [_insertedAttachments addObject:attachment];
@@ -447,6 +456,11 @@ static BOOL getFilePathsAndTypeIdentifiers(NSArray<NSURL *> *fileURLs, NSArray<N
     [_removedAttachments addObject:attachment];
 }
 
+- (void)_webView:(WKWebView *)webView didPerformDragOperation:(BOOL)handled
+{
+    _doneWaitingForDrop = true;
+}
+
 @end
 
 #endif // ENABLE(DRAG_SUPPORT) && PLATFORM(MAC) && WK_API_ENABLED