[Attachment Support] Create attachment elements when dropping files on iOS
authorwenson_hsieh@apple.com <wenson_hsieh@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 4 Jan 2018 07:44:55 +0000 (07:44 +0000)
committerwenson_hsieh@apple.com <wenson_hsieh@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 4 Jan 2018 07:44:55 +0000 (07:44 +0000)
commitd5ea6cd1d8d2e464347d1b4da3e1b04439db10cc
treed7ebe586c386fbb8cf832f70f9e84ea38b13f446
parentaa36072da7d7b96011716b531a6c7a348c61ef86
[Attachment Support] Create attachment elements when dropping files on iOS
https://bugs.webkit.org/show_bug.cgi?id=181192
<rdar://problem/36280945>

Reviewed by Tim Horton.

Source/WebCore:

Implements support for dropping data as attachment elements on iOS. See comments below for more detail.

Tests:  WKAttachmentTests.InsertDroppedRichAndPlainTextFilesAsAttachments
        WKAttachmentTests.InsertDroppedZipArchiveAsAttachment
        WKAttachmentTests.InsertDroppedItemProvidersInOrder

* WebCore.xcodeproj/project.pbxproj:
* editing/WebContentReader.cpp:
(WebCore::WebContentReader::ensureFragment):

Add a new helper to create the WebContentReader's fragment, if it hasn't already been created.

* editing/WebContentReader.h:
* editing/cocoa/WebContentReaderCocoa.mm:
(WebCore::WebContentReader::readFilePaths):

Rename readFilenames to readFilePaths (which better reflects its parameters, which are file paths). Also, move
the implementation of readFilePaths to shared iOS/macOS code in WebContentReaderCocoa, and remove the stub
implementation on iOS.

There's a bit of code here that I kept macOS-only which deals with inserting file paths as plain text in
editable areas, but it's unclear to me why and if WebKit clients currently find this useful, so I left a FIXME
to investigate removing this altogether. Code for handling this plain text insertion of file paths on Mac was
introduced in r67403.

* editing/ios/WebContentReaderIOS.mm:
(WebCore::WebContentReader::readFilenames): Deleted.
* editing/mac/WebContentReaderMac.mm:
(WebCore::WebContentReader::readFilenames): Deleted.
* page/mac/DragControllerMac.mm:
(WebCore::DragController::updateSupportedTypeIdentifiersForDragHandlingMethod const):

Teach DragController to accept all types conforming to "public.item" and "public.content" on iOS, only when
attachment elements are enabled. This allows us to load content from item providers that we otherwise would not
have loaded, since we now have the ability to fall back to attachment element insertion if the type is not have
a default representation using standard web content.

* platform/Pasteboard.h:
* platform/PasteboardItemInfo.h: Added.
(WebCore::PasteboardItemInfo::encode const):
(WebCore::PasteboardItemInfo::decode):

Add PasteboardItemInfo, a struct that describes an item on the pasteboard. Also, implement encoding and decoding
support for PasteboardItemInfo. So far, the item info only describes file information about the pasteboard item,
and flags indicating whether the item prefers attachment or inline presentation.

* platform/PasteboardStrategy.h:

Replace getFilenamesForDataInteraction with informationForItemAtIndex. Instead of returning all of the file
paths associated with any item on the pasteboard, fetch a PasteboardItemInfo at a given item index, which
includes information about the file path as well as some other metadata we'll need when deciding how to read
pasteboard contents as a document fragment.

* platform/PlatformPasteboard.h:
* platform/cocoa/PasteboardCocoa.mm:
(WebCore::Pasteboard::read):
* platform/ios/AbstractPasteboard.h:
* platform/ios/PasteboardIOS.mm:
(WebCore::Pasteboard::read):
(WebCore::Pasteboard::readRespectingUTIFidelities):

Teach the iOS Pasteboard to read web content using attachment elements, if enabled. There are two scenarios in
which we would want to insert an attachment element:
(1) The item provider uses a preferred presentation style of attachment, in which case we bail out of trying to
    handle the drop using the default mechanisms, and simply insert it as an attachment. We need this to deal
    with the case where we drop text or HTML files from the Files app, so that we don't try and insert the
    contents of the text or HTML as inline web content.
(2) The item provider doesn't have a preferred attachment presentation style, but there's nothing WebKit would
    otherwise do with the dropped content, so insert an attachment element as a fallback. Examples where this is
    relevant are dropping a PDF or ZIP archive without attachment presentation style explicitly set.
We first check if we fall into case (1). If so, we can bail early by inserting an attachment; otherwise, we
proceed normally and see if we can read the contents of the drop as web content. If, at the end of default drop
handling, we don't still have a way to represent the dropped content, enter case (2).

(WebCore::Pasteboard::readFilePaths):
(WebCore::Pasteboard::readFilenames): Deleted.

Rename readFilenames to readFilePaths, and reimplement it using informationForItemAtIndex.

* platform/ios/PlatformPasteboardIOS.mm:
(WebCore::pasteboardItemPresentationStyle):
(WebCore::PlatformPasteboard::informationForItemAtIndex):
(WebCore::PlatformPasteboard::filenamesForDataInteraction): Deleted.

Implement informationForItemAtIndex and remove filenamesForDataInteraction. As before, we ask the pasteboard
(i.e. WebItemProviderPasteboard) for information about dropped file URLs. This time, we limit this to a single
file, so we don't end up creating multiple attachment elements for each representation of a single item
provider. See below for -preferredFileUploadURLAtIndex:fileType: for more detail.

* platform/ios/WebItemProviderPasteboard.h:
* platform/ios/WebItemProviderPasteboard.mm:
(-[WebItemProviderLoadResult initWithItemProvider:typesToLoad:]):
(-[WebItemProviderLoadResult canBeRepresentedAsFileUpload]):

Remove this synthesized instance variable and instead just check the item provider's preferredPresentationStyle.

(-[WebItemProviderLoadResult description]):

Add a verbose -description to the load result object. Useful for debugging what was content was loaded from an
item provider on drop.

(-[WebItemProviderPasteboard preferredFileUploadURLAtIndex:fileType:]):

Return the highest fidelity loaded type identifier for a given item.

(-[WebItemProviderPasteboard allDroppedFileURLs]):
(-[WebItemProviderPasteboard typeIdentifiersToLoadForRegisteredTypeIdentfiers:]):

Prefer flat RTFD to RTFD. In the case where attachments are enabled and we're accepting all types of content
using attachment elements as a fallback representation, if the source writes attributed strings to the
pasteboard with com.apple.rtfd at a higher fidelity than com.apple.flat-rtfd, we'll end up loading only
com.apple.rtfd and dropping the text as an attachment element because we cannot convert the dropped content to
markup. Instead, if flat RTFD is present in the item provider, always prefer that over RTFD so that dropping as
regular web content isn't overridden when attachment elements are enabled.

(-[WebItemProviderPasteboard doAfterLoadingProvidedContentIntoFileURLs:synchronousTimeout:]):
(-[WebItemProviderPasteboard droppedFileURLs]): Deleted.
* platform/mac/DragDataMac.mm:
(WebCore::DragData::containsCompatibleContent const):

DragData::containsCompatibleContent should be true when attachment elements are enabled, and there are files we
can drop as attachment elements.

* platform/mac/PasteboardMac.mm:
(WebCore::Pasteboard::read):
(WebCore::Pasteboard::readFilePaths):
(WebCore::Pasteboard::readFilenames): Deleted.

Source/WebKit:

Make some minor adjustments for changes to the pasteboard in WebCore. See WebCore/ChangeLog for more detail.
Teaches WebPasteboardProxy et. al. to plumb PasteboardItemInfo from the UI process to the web process via the
new `InformationForItemAtIndex` codepath.

* UIProcess/Cocoa/WebPasteboardProxyCocoa.mm:
(WebKit::WebPasteboardProxy::informationForItemAtIndex):
(WebKit::WebPasteboardProxy::getFilenamesForDataInteraction): Deleted.
* UIProcess/WebPasteboardProxy.h:
* UIProcess/WebPasteboardProxy.messages.in:
* WebProcess/WebCoreSupport/WebPlatformStrategies.cpp:
(WebKit::WebPlatformStrategies::informationForItemAtIndex):
(WebKit::WebPlatformStrategies::getFilenamesForDataInteraction): Deleted.
* WebProcess/WebCoreSupport/WebPlatformStrategies.h:

Source/WebKitLegacy/mac:

Make some minor adjustments for changes to the pasteboard in WebCore. See WebCore/ChangeLog for more detail.

* WebCoreSupport/WebPlatformStrategies.h:
* WebCoreSupport/WebPlatformStrategies.mm:
(WebPlatformStrategies::informationForItemAtIndex):
(WebPlatformStrategies::getFilenamesForDataInteraction): Deleted.

Tools:

Adds 3 new API tests to exercise different use cases of dropping content as attachment elements when the runtime
switch is enabled. See below for more details.

* TestWebKitAPI/Tests/WebKitCocoa/WKAttachmentTests.mm:
(-[NSItemProvider registerData:type:]):
(platformCopyPNG):
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/ios/DataInteractionTests.mm:

Fix some currently failing iOS drag and drop tests. In this case, there's no reason RTFD should appear in the
source item provider when dragging rich text *without* attachments, so this should have been a check for just
kUTTypeRTF instead.

(TestWebKitAPI::TEST):

Tests a few cases of inserting attachment elements via drop:
1.  We should distinguish between drops containing rich/plain text files from just dropping rich/plain text.
    Instead of inserting the contents as inline web content, this should generate attachment elements.
2.  Test the fallback mechanism for inserting attachment elements. If the preferred presentation style is not
    explicitly set, but there's nothing WebKit would otherwise do with the dropped content, then we should fall
    back to inserting the content as an attachment.
3.  Test that if multiple attachments and inline item providers are present, WebKit will respect the order in
    which they were inserted by the source (as opposed to, for instance, putting all of the attachments in front
    or at the end).

* TestWebKitAPI/cocoa/TestWKWebView.h:
* TestWebKitAPI/cocoa/TestWKWebView.mm:
(-[TestWKWebView objectByEvaluatingJavaScript:]):

Add a helper method to return an object that represents the result of evaluating some given script, and rewrite
-stringByEvaluatingJavaScript to just turn around and call this.

(-[TestWKWebView stringByEvaluatingJavaScript:]):

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@226396 268f45cc-cd09-0410-ab3c-d52691b4dbfc
34 files changed:
Source/WebCore/ChangeLog
Source/WebCore/WebCore.xcodeproj/project.pbxproj
Source/WebCore/editing/WebContentReader.cpp
Source/WebCore/editing/WebContentReader.h
Source/WebCore/editing/cocoa/WebContentReaderCocoa.mm
Source/WebCore/editing/ios/WebContentReaderIOS.mm
Source/WebCore/editing/mac/WebContentReaderMac.mm
Source/WebCore/page/mac/DragControllerMac.mm
Source/WebCore/platform/Pasteboard.h
Source/WebCore/platform/PasteboardItemInfo.h [new file with mode: 0644]
Source/WebCore/platform/PasteboardStrategy.h
Source/WebCore/platform/PlatformPasteboard.h
Source/WebCore/platform/cocoa/PasteboardCocoa.mm
Source/WebCore/platform/ios/AbstractPasteboard.h
Source/WebCore/platform/ios/PasteboardIOS.mm
Source/WebCore/platform/ios/PlatformPasteboardIOS.mm
Source/WebCore/platform/ios/WebItemProviderPasteboard.h
Source/WebCore/platform/ios/WebItemProviderPasteboard.mm
Source/WebCore/platform/mac/DragDataMac.mm
Source/WebCore/platform/mac/PasteboardMac.mm
Source/WebKit/ChangeLog
Source/WebKit/UIProcess/Cocoa/WebPasteboardProxyCocoa.mm
Source/WebKit/UIProcess/WebPasteboardProxy.h
Source/WebKit/UIProcess/WebPasteboardProxy.messages.in
Source/WebKit/WebProcess/WebCoreSupport/WebPlatformStrategies.cpp
Source/WebKit/WebProcess/WebCoreSupport/WebPlatformStrategies.h
Source/WebKitLegacy/mac/ChangeLog
Source/WebKitLegacy/mac/WebCoreSupport/WebPlatformStrategies.h
Source/WebKitLegacy/mac/WebCoreSupport/WebPlatformStrategies.mm
Tools/ChangeLog
Tools/TestWebKitAPI/Tests/WebKitCocoa/WKAttachmentTests.mm
Tools/TestWebKitAPI/Tests/ios/DataInteractionTests.mm
Tools/TestWebKitAPI/cocoa/TestWKWebView.h
Tools/TestWebKitAPI/cocoa/TestWKWebView.mm