[iOS DnD] Web process uses too much memory when beginning a drag on a very large...
authorwenson_hsieh@apple.com <wenson_hsieh@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 17 Jul 2017 23:17:09 +0000 (23:17 +0000)
committerwenson_hsieh@apple.com <wenson_hsieh@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 17 Jul 2017 23:17:09 +0000 (23:17 +0000)
commitdd5aa9ef3ebd9ddfe97b808e18552d9f3488d5c5
tree051d12cf996856cd1d025cab0dbfe2a7b9d03944
parent916b13a7af44c64c6f877866bdb2a4a22dac74e2
[iOS DnD] Web process uses too much memory when beginning a drag on a very large image
https://bugs.webkit.org/show_bug.cgi?id=174585
<rdar://problem/33302541>

Reviewed by Tim Horton.

Source/WebCore:

Currently, attempting to drag a very large image fails, either due to us telling CoreGraphics to create an image
buffer that is too large, or because the web process exceeds its memory limit and gets jetsamed. There are two
places where we can optimize our memory use during the drag initialization sequence, and this patch improves
both.

First, on iOS, we attempt to encode and send over a WebCore::Image in the PasteboardImage when writing to the
item providers upon starting a drag. Currently, this Image is only used in the drag and drop codepath, in
PlatformPasteboard::writeObjectRepresentations, to grab the size of the image being written for the purpose of
specifying estimated display size. Serializing and deserializing an Image calls into Image::nativeImage, which
attempts to draw the contents of the image into a buffer so that it can be shipped across to the UI process.
Instead, we can simply compute the size in the web process while we already have the Image, and simply send that
across. For copy/paste, this doesn't result in any behavior change, since we don't use the PasteboardImage's
image in the first place.

Secondly, when starting a drag, we try to allocate create an image buffer the size of the WebCore::Image for the
purpose of generating the drag preview. Instead, this patch establishes a limit on the size of this drag preview
image, such that if the Image's size is larger, we'll scale down the drag preview image to be the maximum
allowed size.

Test: DataInteractionTests.CanStartDragOnEnormousImage.

* editing/ios/EditorIOS.mm:
(WebCore::Editor::writeImageToPasteboard):
* platform/Pasteboard.h:
* platform/graphics/GeometryUtilities.cpp:
(WebCore::sizeWithAreaAndAspectRatio):

Introduce a new helper function to compute a size with the given aspect ratio and area.

* platform/graphics/GeometryUtilities.h:
* platform/ios/DragImageIOS.mm:
(WebCore::createDragImageFromImage):
* platform/ios/PlatformPasteboardIOS.mm:
(WebCore::PlatformPasteboard::writeObjectRepresentations):

Source/WebKit:

Add IPC support for serializing/deserializing the size of an image written to the pasteboard. See WebCore
ChangeLogs for more details.

* Shared/WebCoreArgumentCoders.cpp:
(IPC::ArgumentCoder<PasteboardImage>::encode):
(IPC::ArgumentCoder<PasteboardImage>::decode):

Tools:

Adds a new test verifying that we don't try to allocate any image buffer equal to the true size of the image
being dragged when initiating a drag.

* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WebKit2Cocoa/enormous.svg: Added.
* TestWebKitAPI/Tests/ios/DataInteractionTests.mm:
(TestWebKitAPI::TEST):
* TestWebKitAPI/cocoa/TestWKWebView.h:

Add a new -synchronouslyLoadHTMLString: helper that works like -synchronouslyLoadTestPage:, but takes markup.

* TestWebKitAPI/cocoa/TestWKWebView.mm:
(-[TestWKWebView synchronouslyLoadHTMLString:]):

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@219585 268f45cc-cd09-0410-ab3c-d52691b4dbfc
15 files changed:
Source/WebCore/ChangeLog
Source/WebCore/editing/ios/EditorIOS.mm
Source/WebCore/platform/Pasteboard.h
Source/WebCore/platform/graphics/GeometryUtilities.cpp
Source/WebCore/platform/graphics/GeometryUtilities.h
Source/WebCore/platform/ios/DragImageIOS.mm
Source/WebCore/platform/ios/PlatformPasteboardIOS.mm
Source/WebKit/ChangeLog
Source/WebKit/Shared/WebCoreArgumentCoders.cpp
Tools/ChangeLog
Tools/TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj
Tools/TestWebKitAPI/Tests/WebKit2Cocoa/enormous.svg [new file with mode: 0644]
Tools/TestWebKitAPI/Tests/ios/DataInteractionTests.mm
Tools/TestWebKitAPI/cocoa/TestWKWebView.h
Tools/TestWebKitAPI/cocoa/TestWKWebView.mm