Make it possible to edit images inline
authortimothy_horton@apple.com <timothy_horton@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 12 Nov 2018 22:04:47 +0000 (22:04 +0000)
committertimothy_horton@apple.com <timothy_horton@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 12 Nov 2018 22:04:47 +0000 (22:04 +0000)
commit65682e95262aba91ae009c8a589ef3228c4214d3
tree3ef527b094c7662eb006e57d9fdc5664edced05a
parent2d82cdfcd055b25edfd5de0e29fd914ddd71bc1e
Make it possible to edit images inline
https://bugs.webkit.org/show_bug.cgi?id=191352
<rdar://problem/30107985>

Reviewed by Dean Jackson.

Source/WebCore:

Tests: editing/images/basic-editable-image.html
       editing/images/reparent-editable-image-maintains-strokes.html

Add the beginnings of a mechanism to replace images with a special attribute
with a native drawing view in the UI process.

* page/Settings.yaml:
Add a setting to control whether images become natively editable when they
have the x-apple-editable-image attribute.

* html/HTMLImageElement.cpp:
(WebCore::HTMLImageElement::editableImageViewID const):
Lazily generate an EmbeddedViewID and persist it on the <img> element.

* html/HTMLImageElement.h:
Rearrange the service controls methods to sit before the members.
Add m_editableImageViewID and editableImageViewID().

* platform/graphics/GraphicsLayer.cpp:
(WebCore::GraphicsLayer::nextEmbeddedViewID):
* platform/graphics/GraphicsLayer.h:
(WebCore::GraphicsLayer::setContentsToEmbeddedView):
Add a new ContentsLayerPurpose, EmbeddedView, which is only supported
on Cocoa platforms and when using RemoteLayerTree.
Add ContentsLayerEmbeddedViewType, which currently only has the EditableImage type.
Add setContentsToEmbeddedView, which takes a ContentsLayerEmbeddedViewType
and an EmbeddedViewID to uniquely identify and communicate about the
embedded view (which may move between layers, since it is tied to an element).

* platform/graphics/ca/GraphicsLayerCA.cpp:
(WebCore::GraphicsLayerCA::createPlatformCALayerForEmbeddedView):
(WebCore::GraphicsLayerCA::setContentsToEmbeddedView):
When setting GraphicsLayer's contents to an embedded view, we use
a special PlatformCALayer factory that takes the EmbeddedViewID and type.
GraphicsLayerCARemote will override this and make a correctly-initialized
PlatformCALayerRemote that keeps track of the EmbeddedViewID.

* platform/graphics/ca/GraphicsLayerCA.h:
* platform/graphics/ca/PlatformCALayer.cpp:
(WebCore::operator<<):
* platform/graphics/ca/PlatformCALayer.h:
* platform/graphics/ca/cocoa/PlatformCALayerCocoa.h:
* platform/graphics/ca/cocoa/PlatformCALayerCocoa.mm:
(WebCore::PlatformCALayerCocoa::PlatformCALayerCocoa):
(WebCore::PlatformCALayerCocoa::embeddedViewID const):
Add stubs and logging for EmbeddedViewID on PlatformCALayer.
These will be overridden by PlatformCALayerRemote to do more interesting things.

* rendering/RenderImage.cpp:
(WebCore::RenderImage::isEditableImage const):
Add a getter that return true if the setting is enabled and
x-apple-editable-image is empty or true.

(WebCore::RenderImage::requiresLayer const):
RenderImage requires a layer either if RenderReplaced does, or we are an
editable image.

* rendering/RenderImage.h:
* rendering/RenderLayer.cpp:
(WebCore::RenderLayer::shouldBeNormalFlowOnly const):
(WebCore::RenderLayer::calculateClipRects const):
* rendering/RenderLayerBacking.cpp:
(WebCore::RenderLayerBacking::updateConfiguration):
Push the EmbeddedViewID and type down to GraphicsLayer for editable images.

* rendering/RenderLayerCompositor.cpp:
(WebCore::RenderLayerCompositor::requiresCompositingLayer const):
(WebCore::RenderLayerCompositor::requiresOwnBackingStore const):
(WebCore::RenderLayerCompositor::reasonsForCompositing const):
(WebCore::RenderLayerCompositor::requiresCompositingForEditableImage const):
* rendering/RenderLayerCompositor.h:
Make editable images require compositing implicitly.

Source/WebKit:

* Platform/spi/ios/PencilKitSPI.h: Added.
* Shared/RemoteLayerTree/RemoteLayerBackingStore.mm:
(WebKit::RemoteLayerBackingStore::drawInContext):
* Shared/RemoteLayerTree/RemoteLayerTreeTransaction.h:
* Shared/RemoteLayerTree/RemoteLayerTreeTransaction.mm:
(WebKit::RemoteLayerTreeTransaction::LayerCreationProperties::LayerCreationProperties):
(WebKit::RemoteLayerTreeTransaction::LayerCreationProperties::encode const):
(WebKit::RemoteLayerTreeTransaction::LayerCreationProperties::decode):
* WebProcess/WebPage/RemoteLayerTree/GraphicsLayerCARemote.cpp:
(WebKit::GraphicsLayerCARemote::createPlatformCALayerForEmbeddedView):
* WebProcess/WebPage/RemoteLayerTree/GraphicsLayerCARemote.h:
* WebProcess/WebPage/RemoteLayerTree/PlatformCALayerRemote.cpp:
(WebKit::PlatformCALayerRemote::createForEmbeddedView):
(WebKit::PlatformCALayerRemote::PlatformCALayerRemote):
(WebKit::PlatformCALayerRemote::embeddedViewID const):
* WebProcess/WebPage/RemoteLayerTree/PlatformCALayerRemote.h:
* WebProcess/WebPage/RemoteLayerTree/RemoteLayerTreeContext.mm:
(WebKit::RemoteLayerTreeContext::layerWasCreated):
Propagate EmbeddedViewID through the PlatformCALayer constructor and
through the layer creation parameters to the UI process.

* Shared/WebPreferences.yaml:
* UIProcess/API/Cocoa/WKWebView.mm:
(-[WKWebView _initializeWithConfiguration:]):
* UIProcess/API/Cocoa/WKWebViewConfiguration.mm:
(-[WKWebViewConfiguration init]):
(-[WKWebViewConfiguration copyWithZone:]):
(-[WKWebViewConfiguration _setEditableImagesEnabled:]):
(-[WKWebViewConfiguration _editableImagesEnabled]):
* UIProcess/API/Cocoa/WKWebViewConfigurationPrivate.h:
Add a preference to enable editable images.

* UIProcess/RemoteLayerTree/RemoteLayerTreeHost.h:
* UIProcess/RemoteLayerTree/RemoteLayerTreeHost.mm:
(WebKit::RemoteLayerTreeHost::layerWillBeRemoved):
(WebKit::RemoteLayerTreeHost::clearLayers):
(WebKit::RemoteLayerTreeHost::createLayer):
Keep track of "embedded views" in two maps: embeddedViewID->UIView,
and layerID->embeddedViewID. Clean them up when layers go away.
If a embedded view is reparented, currently it must be added to a new
layer in the same commit as it is removed from the previous layer
in order to persist the view's state (otherwise the view will be
destroyed and recreated). This will be less of a problem after future
patches introduce serialization of image data and whatnot.

* UIProcess/RemoteLayerTree/ios/RemoteLayerTreeHostIOS.mm:
(WebKit::RemoteLayerTreeHost::createLayer):
(WebKit::RemoteLayerTreeHost::createEmbeddedView):
Move the various remote layer tree UIView subclasses out into a separate file.

Add createEmbeddedView, which is used for LayerTypeEditableImageLayer,
and creates a WKDrawingView and sticks it in the maps.

* UIProcess/RemoteLayerTree/ios/RemoteLayerTreeViews.h: Added.
* UIProcess/RemoteLayerTree/ios/RemoteLayerTreeViews.mm: Added.
(-[UIView _web_recursiveFindDescendantInteractibleViewAtPoint:withEvent:]):
(-[UIView _web_findDescendantViewAtPoint:withEvent:]):
(-[WKCompositingView hitTest:withEvent:]):
(-[WKCompositingView description]):
(+[WKTransformView layerClass]):
(+[WKSimpleBackdropView layerClass]):
(+[WKShapeView layerClass]):
(-[WKRemoteView initWithFrame:contextID:]):
(+[WKRemoteView layerClass]):
(-[WKBackdropView hitTest:withEvent:]):
(-[WKBackdropView description]):
(-[WKChildScrollView initWithFrame:]):
Move various remote layer tree UIView subclasses here, to their own file.
Make our UIView hit testing override test for views that conform to the
protocol "WKNativelyInteractible", which switches to normal UIView hit
testing. WKDrawingView will be the one such view.

Add WKChildScrollView and pull the one thing we customize out into it,
to make RemoteLayerTreeHost::createLayer less logic-ful.

* UIProcess/ios/WKDrawingView.h: Added.
* UIProcess/ios/WKDrawingView.mm: Added.
(-[WKDrawingView init]):
(-[WKDrawingView layoutSubviews]):
Add a very simple WKDrawingView, which uses PKCanvasView to edit the image.

* WebKit.xcodeproj/project.pbxproj:
* SourcesCocoa.txt:
Add the new files.

Tools:

* WebKitTestRunner/TestController.cpp:
(WTR::updateTestOptionsFromTestHeader):
* WebKitTestRunner/TestOptions.h:
(WTR::TestOptions::hasSameInitializationOptions const):
* WebKitTestRunner/WebKitTestRunner.xcodeproj/project.pbxproj:
* WebKitTestRunner/cocoa/TestControllerCocoa.mm:
(WTR::TestController::platformCreateWebView):
Add a test option to enable editable images.

* DumpRenderTree/ios/UIScriptControllerIOS.mm:
(WTR::UIScriptController::drawSquareInEditableImage):
(WTR::UIScriptController::numberOfStrokesInEditableImage):
* TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl:
* TestRunnerShared/UIScriptContext/UIScriptController.cpp:
(WTR::UIScriptController::drawSquareInEditableImage):
(WTR::UIScriptController::numberOfStrokesInEditableImage):
* TestRunnerShared/UIScriptContext/UIScriptController.h:
* TestRunnerShared/spi/PencilKitTestSPI.h: Added.
* WebKitTestRunner/ios/UIScriptControllerIOS.mm:
(WTR::findEditableImageCanvas):
(WTR::UIScriptController::drawSquareInEditableImage):
(WTR::UIScriptController::numberOfStrokesInEditableImage):
Add the ability to draw on a PKCanvasView that is a subview of the WKWebView,
and also to retrieve the number of strokes currently on the PKCanvasView.
Currently this just takes the first canvas; we might need to make it
take an identifier or something in the future if we need tests with multiple
canvases. The indirect testing mechanism is required because PKCanvasView
can currently not actually paint its strokes in the Simulator.

LayoutTests:

* TestExpectations:
* editing/images/basic-editable-image-expected.txt: Added.
* editing/images/basic-editable-image.html: Added.
* editing/images/reparent-editable-image-maintains-strokes-expected.txt: Added.
* editing/images/reparent-editable-image-maintains-strokes.html: Added.
* platform/ios-wk2/TestExpectations:
* resources/ui-helper.js:
(window.UIHelper.drawSquareInEditableImage):
(window.UIHelper.numberOfStrokesInEditableImage):
(window.UIHelper):
Add tests that we can find and draw in editable images, and that if
the element is moved around in the DOM, it persists its strokes.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@238108 268f45cc-cd09-0410-ab3c-d52691b4dbfc
60 files changed:
LayoutTests/ChangeLog
LayoutTests/TestExpectations
LayoutTests/editing/images/basic-editable-image-expected.txt [new file with mode: 0644]
LayoutTests/editing/images/basic-editable-image.html [new file with mode: 0644]
LayoutTests/editing/images/reparent-editable-image-maintains-strokes-expected.txt [new file with mode: 0644]
LayoutTests/editing/images/reparent-editable-image-maintains-strokes.html [new file with mode: 0644]
LayoutTests/resources/ui-helper.js
Source/WebCore/ChangeLog
Source/WebCore/html/HTMLAttributeNames.in
Source/WebCore/html/HTMLImageElement.cpp
Source/WebCore/html/HTMLImageElement.h
Source/WebCore/page/Settings.yaml
Source/WebCore/platform/graphics/GraphicsLayer.cpp
Source/WebCore/platform/graphics/GraphicsLayer.h
Source/WebCore/platform/graphics/ca/GraphicsLayerCA.cpp
Source/WebCore/platform/graphics/ca/GraphicsLayerCA.h
Source/WebCore/platform/graphics/ca/PlatformCALayer.cpp
Source/WebCore/platform/graphics/ca/PlatformCALayer.h
Source/WebCore/platform/graphics/ca/cocoa/PlatformCALayerCocoa.h
Source/WebCore/platform/graphics/ca/cocoa/PlatformCALayerCocoa.mm
Source/WebCore/rendering/RenderImage.cpp
Source/WebCore/rendering/RenderImage.h
Source/WebCore/rendering/RenderLayer.cpp
Source/WebCore/rendering/RenderLayerBacking.cpp
Source/WebCore/rendering/RenderLayerCompositor.cpp
Source/WebCore/rendering/RenderLayerCompositor.h
Source/WebKit/ChangeLog
Source/WebKit/Platform/spi/ios/PencilKitSPI.h [new file with mode: 0644]
Source/WebKit/Shared/RemoteLayerTree/RemoteLayerBackingStore.mm
Source/WebKit/Shared/RemoteLayerTree/RemoteLayerTreeTransaction.h
Source/WebKit/Shared/RemoteLayerTree/RemoteLayerTreeTransaction.mm
Source/WebKit/Shared/WebPreferences.yaml
Source/WebKit/SourcesCocoa.txt
Source/WebKit/UIProcess/API/Cocoa/WKWebView.mm
Source/WebKit/UIProcess/API/Cocoa/WKWebViewConfiguration.mm
Source/WebKit/UIProcess/API/Cocoa/WKWebViewConfigurationPrivate.h
Source/WebKit/UIProcess/RemoteLayerTree/RemoteLayerTreeHost.h
Source/WebKit/UIProcess/RemoteLayerTree/RemoteLayerTreeHost.mm
Source/WebKit/UIProcess/RemoteLayerTree/ios/RemoteLayerTreeHostIOS.mm
Source/WebKit/UIProcess/RemoteLayerTree/ios/RemoteLayerTreeViews.h [new file with mode: 0644]
Source/WebKit/UIProcess/RemoteLayerTree/ios/RemoteLayerTreeViews.mm [new file with mode: 0644]
Source/WebKit/UIProcess/ios/WKDrawingView.h [new file with mode: 0644]
Source/WebKit/UIProcess/ios/WKDrawingView.mm [new file with mode: 0644]
Source/WebKit/WebKit.xcodeproj/project.pbxproj
Source/WebKit/WebProcess/WebPage/RemoteLayerTree/GraphicsLayerCARemote.cpp
Source/WebKit/WebProcess/WebPage/RemoteLayerTree/GraphicsLayerCARemote.h
Source/WebKit/WebProcess/WebPage/RemoteLayerTree/PlatformCALayerRemote.cpp
Source/WebKit/WebProcess/WebPage/RemoteLayerTree/PlatformCALayerRemote.h
Source/WebKit/WebProcess/WebPage/RemoteLayerTree/RemoteLayerTreeContext.mm
Tools/ChangeLog
Tools/DumpRenderTree/ios/UIScriptControllerIOS.mm
Tools/TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl
Tools/TestRunnerShared/UIScriptContext/UIScriptController.cpp
Tools/TestRunnerShared/UIScriptContext/UIScriptController.h
Tools/TestRunnerShared/spi/PencilKitTestSPI.h [new file with mode: 0644]
Tools/WebKitTestRunner/TestController.cpp
Tools/WebKitTestRunner/TestOptions.h
Tools/WebKitTestRunner/WebKitTestRunner.xcodeproj/project.pbxproj
Tools/WebKitTestRunner/cocoa/TestControllerCocoa.mm
Tools/WebKitTestRunner/ios/UIScriptControllerIOS.mm