[Attachment Support] Dropping and pasting images should insert inline image elements...
authorwenson_hsieh@apple.com <wenson_hsieh@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 27 Aug 2018 02:37:22 +0000 (02:37 +0000)
committerwenson_hsieh@apple.com <wenson_hsieh@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 27 Aug 2018 02:37:22 +0000 (02:37 +0000)
commitfb37dfb128b53e4a39fc5eeb883fd93ee7824471
treeca6d4086784a99b7882bf68c46f96807426c812f
parentf5b3e440baacafac5375ee7c11ab4bd3fb521fec
[Attachment Support] Dropping and pasting images should insert inline image elements with _WKAttachments
https://bugs.webkit.org/show_bug.cgi?id=188933
<rdar://problem/43699724>

Reviewed by Darin Adler.

Source/WebCore:

Support the ability to drop and paste images as image elements, with attachment elements, only if attachment
elements are enabled. See changes below for more detail.

Tests:  WKAttachmentTests.CutAndPastePastedImage
        WKAttachmentTests.MovePastedImageByDragging
        WKAttachmentTests.RemoveNewlinesBeforePastedImage

* editing/Editor.h:
* editing/cocoa/EditorCocoa.mm:
(WebCore::Editor::getPasteboardTypesAndDataForAttachment):

Adjust this helper to take an Element& rather than an HTMLAttachmentElement&, and address a FIXME by writing the
document origin identifier to the pasteboard via custom pasteboard data when dragging an attachment. This allows
us to avoid creating extra image and attachment elements when dragging an image backed by an attachment within
the same document.

* editing/cocoa/WebContentReaderCocoa.mm:
(WebCore::contentTypeIsSuitableForInlineImageRepresentation):

Add a helper to determine whether a content type (UTI or MIME type) should be read as an inline image.

(WebCore::createFragmentForImageAttachment):
(WebCore::replaceRichContentWithAttachments):
(WebCore::WebContentReader::readFilePaths):

Teach codepaths where we currently create attachment elements to instead create image elements if the MIME type,
is something suitable for display via an inline image element; add the attachment element under the shadow root
of the image element.

* editing/markup.cpp:
(WebCore::StyledMarkupAccumulator::appendCustomAttributes):
(WebCore::restoreAttachmentElementsInFragment):

When dragging or copying an image element, we need to make sure that any attachment element backing the image
is preserved in the pasted or dropped fragment. To do this, we use a technique similar to what was done for
r180785 and r224593 and write a temporary "webkitattachmentid" attribute to the serialized markup on copy. Upon
deserializing the markup back to a fragment, we then create an attachment element with the same identifier under
the image.

(WebCore::createFragmentFromMarkup):
* html/HTMLAttachmentElement.h:
* html/HTMLImageElement.cpp:
(WebCore::HTMLImageElement::setAttachmentElement):
(WebCore::HTMLImageElement::attachmentElement const):

Helper methods to get and set an attachment element under an image element. Setting an image's attachment
element puts that attachment element under the shadow root of the image, and also hides the attachment element.

(WebCore::HTMLImageElement::attachmentIdentifier const):

Returns the identifier of an attachment element associated with the image element, or null.

* html/HTMLImageElement.h:
* html/HTMLImageElement.idl:

Add HTMLImageElement.webkitAttachmentIdentifier, a readonly attribute guarded by runtime-enabled attachment
element feature.

* page/DragController.cpp:
(WebCore::DragController::startDrag):

In the case of dragging an image, if that image element is backed by an attachment element, don't bother writing
the image data to the clipboard; instead, write the attachment data as a promise.

(WebCore::DragController::doImageDrag):

Plumb promised attachment information to DragController::doSystemDrag.

(WebCore::DragController::promisedAttachmentInfo):

Teach this to handle attachment elements as well as image elements that are backed by attachment elements.

* page/DragController.h:
* platform/PromisedAttachmentInfo.h:
(WebCore::PromisedAttachmentInfo::operator bool const):

A valid PromisedAttachmentInfo no longer requires a contentType to be set; instead, an attachment identifier
alone is sufficient, since an up-to-date content type can be requested in the UI process from the API attachment
object.

Source/WebKit:

Support the ability to drop and paste images as image elements, with attachment elements, only if attachment
elements are enabled. See changes below for more detail.

* Shared/WebCoreArgumentCoders.cpp:
(IPC::ArgumentCoder<PromisedAttachmentInfo>::encode):
(IPC::ArgumentCoder<PromisedAttachmentInfo>::decode):

Rename "filename" to "fileName", for consistency with WKContentView and WebViewImpl.

* UIProcess/API/APIAttachment.cpp:
(API::Attachment::mimeType const):
(API::Attachment::fileName const):
* UIProcess/API/APIAttachment.h:

Push getters for MIME type, UTI, and the file name down from _WKAttachment to API::Attachment. This allows
WKContentView and WebViewImpl to ask an API::Attachment questions about its UTI and file name without
additionally creating a wrapper object.

* UIProcess/API/Cocoa/APIAttachmentCocoa.mm: Added.
(API::mimeTypeInferredFromFileExtension):
(API::isDeclaredOrDynamicTypeIdentifier):
(API::Attachment::mimeType const):
(API::Attachment::utiType const):
(API::Attachment::fileName const):
* UIProcess/API/Cocoa/WKUIDelegatePrivate.h:

Add a private delegate hook to notify the UI delegate when a drop has been performed. This is used by tests to
know when a drop with file promises has been processed by the page.

* UIProcess/API/Cocoa/WKWebView.mm:
(-[WKWebView _web_didPerformDragOperation:]):
* UIProcess/API/Cocoa/_WKAttachment.mm:
(-[_WKAttachmentInfo initWithFileWrapper:filePath:mimeType:utiType:]):
(-[_WKAttachmentInfo fileWrapper]):
(-[_WKAttachmentInfo contentType]):
(-[_WKAttachment info]):
(-[_WKAttachmentInfo initWithFileWrapper:filePath:contentType:]): Deleted.
(isDeclaredOrDynamicTypeIdentifier): Deleted.
(-[_WKAttachmentInfo _typeIdentifierFromPathExtension]): Deleted.
(-[_WKAttachmentInfo mimeType]): Deleted.
(-[_WKAttachmentInfo utiType]): Deleted.

Moved to APIAttachmentCocoa.mm.

* UIProcess/API/mac/WKView.mm:
(-[WKView _web_didPerformDragOperation:]):
* UIProcess/Cocoa/WebViewImpl.h:
* UIProcess/Cocoa/WebViewImpl.mm:
(-[WKPromisedAttachmentContext initWithIdentifier:blobURL:fileName:]):

Adjust this constructor to take each piece of data separately. This is because, in the case where our
PromisedAttachmentInfo contains an attachment identifier, we determine the UTI and file name from the associated
file wrapper.

(-[WKPromisedAttachmentContext fileName]):
(WebKit::WebViewImpl::fileNameForFilePromiseProvider):
(WebKit::WebViewImpl::didPerformDragOperation):
(WebKit::WebViewImpl::startDrag):

Determine UTI and file name from the attachment element corresponding to the attachment identifier when
dragging. This is because the attachment element in the web process shouldn't need to have type and title
attributes set when starting a drag if it already has an identifier that maps to attachment data in the UI
process. An example of this is in inline images backed by attachments, for which we don't need to bother keeping
specifying display attributes, since they are never visible.

(-[WKPromisedAttachmentContext initWithAttachmentInfo:]): Deleted.
(-[WKPromisedAttachmentContext filename]): Deleted.
* UIProcess/PageClient.h:
(WebKit::PageClient::didPerformDragOperation):
* UIProcess/WebPageProxy.cpp:
(WebKit::WebPageProxy::didPerformDragOperation):
* UIProcess/WebPageProxy.h:
* UIProcess/WebPageProxy.messages.in:

Rename DidPerformDataInteractionControllerOperation to DidPerformDragOperation, and make it cross-platform (this
was previously only sent on iOS). Add plumbing through PageClient and friends on macOS to notify the UI
delegate when a drop is handled by the web process.

* UIProcess/ios/PageClientImplIOS.h:
* UIProcess/ios/PageClientImplIOS.mm:
(WebKit::PageClientImpl::didPerformDragOperation):
(WebKit::PageClientImpl::didPerformDataInteractionControllerOperation): Deleted.
* UIProcess/ios/WKContentViewInteraction.h:
* UIProcess/ios/WKContentViewInteraction.mm:
(-[WKContentView _didPerformDragOperation:]):
(-[WKContentView _prepareToDragPromisedAttachment:]):

Just like in WebViewImpl::startDrag, infer content type and file name from the API attachment object.

(-[WKContentView _didPerformDataInteractionControllerOperation:]): Deleted.
* UIProcess/ios/WebPageProxyIOS.mm:
(WebKit::WebPageProxy::didPerformDataInteractionControllerOperation): Deleted.
* UIProcess/mac/PageClientImplMac.h:
* UIProcess/mac/PageClientImplMac.mm:
(WebKit::PageClientImpl::didPerformDragOperation):
* WebKit.xcodeproj/project.pbxproj:
* WebProcess/WebPage/WebPage.cpp:
(WebKit::WebPage::performDragControllerAction):

Tools:

Rebaseline existing API tests that involve dropping or pasting image files, and additionally write some new
tests. These new tests exercise the following cases:
•       Inserting and removing newlines before an inline image with an attachment element does not cause new
        _WKAttachments to be created and destroyed.
•       Pasting an image, cutting it, and then pasting it again propagates an attachment update to the UI
        process with the original _WKAttachment.
•       A pasted attachment in the document can be moved around by dragging, and doing so does not cause us to
        lose a _WKAttachment.

* TestWebKitAPI/Tests/WebKitCocoa/WKAttachmentTests.mm:
(-[TestWKWebView expectElementCount:tagName:]):
(TestWebKitAPI::TEST):

Add the new tests described above, and also adjust existing tests to check that images are dropped or pasted
as image elements, but still have associated attachment elements whose attachment identifiers (observed via
script) match that of the corresponding _WKAttachment's uniqueIdentifier in the UI process.

* TestWebKitAPI/mac/DragAndDropSimulatorMac.mm:
(-[DragAndDropSimulator runFrom:to:]):
(-[DragAndDropSimulator continueDragSession]):
(-[DragAndDropSimulator performDragInWebView:atLocation:withImage:pasteboard:source:]):

Teach DragAndDropSimulator on macOS to wait until the drop has been handled by the web process before returning
execution to the caller. This ensures that tests which involve dropping promised files as attachments aren't
flaky, due to how the promised data is retrieved asynchronously when performing the drop.

(-[DragAndDropSimulator _webView:didPerformDragOperation:]):

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@235343 268f45cc-cd09-0410-ab3c-d52691b4dbfc
39 files changed:
Source/WebCore/ChangeLog
Source/WebCore/editing/Editor.h
Source/WebCore/editing/cocoa/EditorCocoa.mm
Source/WebCore/editing/cocoa/WebContentReaderCocoa.mm
Source/WebCore/editing/markup.cpp
Source/WebCore/html/HTMLAttachmentElement.h
Source/WebCore/html/HTMLImageElement.cpp
Source/WebCore/html/HTMLImageElement.h
Source/WebCore/html/HTMLImageElement.idl
Source/WebCore/page/DragController.cpp
Source/WebCore/page/DragController.h
Source/WebCore/platform/PromisedAttachmentInfo.h
Source/WebKit/ChangeLog
Source/WebKit/Shared/WebCoreArgumentCoders.cpp
Source/WebKit/UIProcess/API/APIAttachment.cpp
Source/WebKit/UIProcess/API/APIAttachment.h
Source/WebKit/UIProcess/API/Cocoa/APIAttachmentCocoa.mm [new file with mode: 0644]
Source/WebKit/UIProcess/API/Cocoa/WKUIDelegatePrivate.h
Source/WebKit/UIProcess/API/Cocoa/WKWebView.mm
Source/WebKit/UIProcess/API/Cocoa/_WKAttachment.mm
Source/WebKit/UIProcess/API/mac/WKView.mm
Source/WebKit/UIProcess/Cocoa/WebViewImpl.h
Source/WebKit/UIProcess/Cocoa/WebViewImpl.mm
Source/WebKit/UIProcess/PageClient.h
Source/WebKit/UIProcess/WebPageProxy.cpp
Source/WebKit/UIProcess/WebPageProxy.h
Source/WebKit/UIProcess/WebPageProxy.messages.in
Source/WebKit/UIProcess/ios/PageClientImplIOS.h
Source/WebKit/UIProcess/ios/PageClientImplIOS.mm
Source/WebKit/UIProcess/ios/WKContentViewInteraction.h
Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm
Source/WebKit/UIProcess/ios/WebPageProxyIOS.mm
Source/WebKit/UIProcess/mac/PageClientImplMac.h
Source/WebKit/UIProcess/mac/PageClientImplMac.mm
Source/WebKit/WebKit.xcodeproj/project.pbxproj
Source/WebKit/WebProcess/WebPage/WebPage.cpp
Tools/ChangeLog
Tools/TestWebKitAPI/Tests/WebKitCocoa/WKAttachmentTests.mm
Tools/TestWebKitAPI/mac/DragAndDropSimulatorMac.mm