Image pasting is not working on tineye.com / gmail.com / GitHub.com due to lack of...
authorrniwa@webkit.org <rniwa@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 29 Sep 2017 18:46:51 +0000 (18:46 +0000)
committerrniwa@webkit.org <rniwa@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 29 Sep 2017 18:46:51 +0000 (18:46 +0000)
commit2337917314d754f212f108a7c254efcd51304e5b
tree9c52af63f8de1362f2ad0be7c19b449f40ba9720
parenta0484573d68b145f3d374f21fdbec4dc978c3bc0
Image pasting is not working on tineye.com / gmail.com / GitHub.com due to lack of support for DataTransfer.items
https://bugs.webkit.org/show_bug.cgi?id=170449
<rdar://problem/31432525>

Reviewed by Wenson Hsieh.

Source/WebCore:

The bug was caused by image types in NSPasteboard or UIPasteboard not being treated as file items in dataTransfer.
Because there is no Web API to get binary data out of dataTransfer unlike text data, we need to treat any image
data as files even if they're entirely in the memory.

This patch introduces the notion of pasteboard types to be treated as files and expose them on dataTransfer.files
as well as dataTransfer.items of type "file". Because in-memory images are stored as TIFF in macOS and websites
don't typically support image/tiff, we convert all such in-memory TIFF images into PNG images ourselves for
a better web compatibility. This is done inside read(PasteboardFileReader&) in PasteboardCocoa.mm.

Note that PasteboardFileReader cannot directly have RefPtr<File> as a member variable as code in WebCore/platform
including Pasteboard code is not supposed to depend on WebCore types. WebCorePasteboardFileReader, a subclass of
PasteboardFileReader was introduced to resolve this reverse dependency.

In addition, this patch removes the restriction on dataTransfer.items that it only includes files of the supported
MIME types. This was unwarranted since 1. we don't have any restriction on what an user can drag & drop into a web
page via input element so there is no security benefit in this, and 2. the user should be able to copy & paste
whatever file he/she desires regardless of the MIME type on websites like Google Drive.

Tests: PasteImage

* CMakeLists.txt:
* WebCore.xcodeproj/project.pbxproj:
* WebCore/PlatformMac.cmake:
* dom/DataTransfer.cpp:
(WebCore::DataTransfer::types const): Now excludes image/gif, image/png, image/jpeg, and image/tiff.
(WebCore::DataTransfer::files const): Add fake files we create for in-memory images but only when there are no real
files in the pasteboard since it's expensive to copy image data across UI/Web processes to create a blob URL.
* dom/DataTransferItemList.cpp:
(WebCore::DataTransferItemList::ensureItems const): Just expose every file type. If the user had decided to paste
a file, it's okay for the page to access that file regardless of whether it's a zip file, JPEG image, etc...
* editing/WebCorePasteboardFileReader.cpp:
(WebCorePasteboardFileReader::~WebCorePasteboardFileReader):
(WebCorePasteboardFileReader::read):
* editing/WebCorePasteboardFileReader.h:
(WebCorePasteboardFileReader):
* platform/Pasteboard.cpp:
(WebCore::PasteboardImage::PasteboardImage): Moved from platform specific translation units.
(WebCore::PasteboardImage::~PasteboardImage): Ditto.
* platform/Pasteboard.h:
(PasteboardFileReader): Added.
(* platform/StaticPasteboard.h:
(StaticPasteboard::typesForBindings): Added.
(StaticPasteboard::typesTreatedAsFiles): Added. Returns an empty list we don't support the web content writing image
files into pasteboard at the moment.
* platform/cocoa/PasteboardCocoa.mm: Added.
(WebCore::PasteboardWebContent::PasteboardWebContent): Moved from PasteboardMac.mm and PasteboardIOS.mm.
(WebCore::PasteboardWebContent::~PasteboardWebContent):
(WebCore::cocoaTypeToImageType): Added.
(WebCore::imageTypeToMIMEType): Added. Pretends to have image/png when the Cocoa type is image/tiff since most of
websites don't support image/tiff.
(WebCore::imageTypeToFakeFilename): Added.
(WebCore::mimeTypeToImageType): Added.
(WebCore::Pasteboard::shouldTreatCocoaTypeAsFile): Added. Pasteboard::typesForBindings excludes the type for which
this function returns true.
(WebCore::Pasteboard::typesTreatedAsFiles): Returns the list of all in-memory image types in the pasteboard.
(WebCore::Pasteboard::read): Added. On macOS, we convert TIFF to PNG for web compatibility. We don't do this rather
memory intensive coercion on iOS where most of apps like Photos put PNG file into the pasteboard in the first place.
* platform/gtk/PasteboardGtk.cpp:
(WebCore::PasteboardImage::PasteboardImage): Deleted.
(WebCore::PasteboardImage::~PasteboardImage): Deleted.
(WebCore::Pasteboard::read):
(WebCore::Pasteboard::typesForBindings): Renamed from types.
(WebCore::Pasteboard::typesTreatedAsFiles):
* platform/ios/PasteboardIOS.mm:
(WebCore::addHTMLClipboardTypesForCocoaType):
(WebCore::Pasteboard::typesForBindings):
(WebCore::PasteboardWebContent::PasteboardWebContent): Deleted.
(WebCore::PasteboardWebContent::~PasteboardWebContent): Deleted.
(WebCore::PasteboardImage::PasteboardImage): Deleted.
(WebCore::PasteboardImage::~PasteboardImage): Deleted.
(WebCore::Pasteboard::types): Deleted.
* platform/ios/PlatformPasteboardIOS.mm:
(WebCore::safeTypeForDOMToReadAndWriteForPlatformType): Add "Files" to dataTransfer.types when there is an in-memory
image type in the pasteboard.
* platform/mac/PasteboardMac.mm:
(WebCore::PasteboardWebContent::PasteboardWebContent): Deleted.
(WebCore::PasteboardWebContent::~PasteboardWebContent): Deleted.
(WebCore::PasteboardImage::PasteboardImage): Deleted.
(WebCore::PasteboardImage::~PasteboardImage): Deleted.
(WebCore::addHTMLClipboardTypesForCocoaType): Moved the check for the legacy NeXT plain text check here. Also add
"Files" to dataTransfer.types when there is an in-memory image type in the pasteboard.
(WebCore::Pasteboard::typesForBindings): Renamed from types.
* platform/mac/PlatformPasteboardMac.mm:
(WebCore::safeTypeForDOMToReadAndWriteForPlatformType): Ditto to add "Files".
* platform/win/PasteboardWin.cpp:
(WebCore::Pasteboard::typesForBindings): Renamed from types.
(WebCore::Pasteboard::typesTreatedAsFiles):
(WebCore::Pasteboard::read):
* platform/wpe/PasteboardWPE.cpp:
(WebCore::Pasteboard::typesForBindings): Renamed from types.
(WebCore::Pasteboard::typesTreatedAsFiles):
(WebCore::Pasteboard::read):

Source/WebKit:

Add sandbox extensions for files in the pasteboard to make copying & pasting image files work.
This is what we do for drag & drop but we should consider adding a mechanism to rekoke the extension in the future.

* UIProcess/Cocoa/WebPasteboardProxyCocoa.mm:
(WebKit::WebPasteboardProxy::getPasteboardPathnamesForType): Add sandbox extensions to the pasted files.
* UIProcess/WebPasteboardProxy.h:
* UIProcess/WebPasteboardProxy.messages.in:
* WebProcess/WebCoreSupport/WebPlatformStrategies.cpp:
(WebKit::WebPlatformStrategies::getPathnamesForType): Consume the sandbox tokens sent by the UI process permanently
since WebCore will now create File objects for these pasted files.

Tools:

Added an API test to paste an image from pasteboard. The test is shared between iOS and macOS.

The tests to paste image files are only enabled on macOS since putting files into pasteboard isn't a thing on iOS.

* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WebKitCocoa/PasteImage.mm: Added.
(writeImageDataToPasteboard):
(writeBundleFileToPasteboard):
* TestWebKitAPI/Tests/WebKitCocoa/paste-image.html: Added.
* TestWebKitAPI/Tests/WebKitCocoa/sunset-in-cupertino-100px.tiff: Added.
* TestWebKitAPI/Tests/WebKitCocoa/sunset-in-cupertino-200px.png: Added.
* TestWebKitAPI/Tests/WebKitCocoa/sunset-in-cupertino-400px.gif: Added.
* TestWebKitAPI/Tests/WebKitCocoa/sunset-in-cupertino-600px.jpg: Added.
* TestWebKitAPI/Tests/ios/DataInteractionTests.mm: Rebaselined the test now that types contain "Files".

git-svn-id: http://svn.webkit.org/repository/webkit/trunk@222656 268f45cc-cd09-0410-ab3c-d52691b4dbfc
34 files changed:
Source/WebCore/CMakeLists.txt
Source/WebCore/ChangeLog
Source/WebCore/PlatformMac.cmake
Source/WebCore/WebCore.xcodeproj/project.pbxproj
Source/WebCore/dom/DataTransfer.cpp
Source/WebCore/dom/DataTransferItemList.cpp
Source/WebCore/editing/WebCorePasteboardFileReader.cpp [new file with mode: 0644]
Source/WebCore/editing/WebCorePasteboardFileReader.h [new file with mode: 0644]
Source/WebCore/editing/wpe/EditorWPE.cpp
Source/WebCore/platform/Pasteboard.cpp
Source/WebCore/platform/Pasteboard.h
Source/WebCore/platform/StaticPasteboard.h
Source/WebCore/platform/cocoa/PasteboardCocoa.mm [new file with mode: 0644]
Source/WebCore/platform/gtk/PasteboardGtk.cpp
Source/WebCore/platform/ios/PasteboardIOS.mm
Source/WebCore/platform/ios/PlatformPasteboardIOS.mm
Source/WebCore/platform/mac/PasteboardMac.mm
Source/WebCore/platform/mac/PlatformPasteboardMac.mm
Source/WebCore/platform/win/PasteboardWin.cpp
Source/WebCore/platform/wpe/PasteboardWPE.cpp
Source/WebKit/ChangeLog
Source/WebKit/UIProcess/Cocoa/WebPasteboardProxyCocoa.mm
Source/WebKit/UIProcess/WebPasteboardProxy.h
Source/WebKit/UIProcess/WebPasteboardProxy.messages.in
Source/WebKit/WebProcess/WebCoreSupport/WebPlatformStrategies.cpp
Tools/ChangeLog
Tools/TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj
Tools/TestWebKitAPI/Tests/WebKitCocoa/PasteImage.mm [new file with mode: 0644]
Tools/TestWebKitAPI/Tests/WebKitCocoa/paste-image.html [new file with mode: 0644]
Tools/TestWebKitAPI/Tests/WebKitCocoa/sunset-in-cupertino-100px.tiff [new file with mode: 0644]
Tools/TestWebKitAPI/Tests/WebKitCocoa/sunset-in-cupertino-200px.png [new file with mode: 0644]
Tools/TestWebKitAPI/Tests/WebKitCocoa/sunset-in-cupertino-400px.gif [new file with mode: 0644]
Tools/TestWebKitAPI/Tests/WebKitCocoa/sunset-in-cupertino-600px.jpg [new file with mode: 0644]
Tools/TestWebKitAPI/Tests/ios/DataInteractionTests.mm