Enable async image decoding for large images
authorcommit-queue@webkit.org <commit-queue@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 9 Mar 2017 02:16:51 +0000 (02:16 +0000)
committercommit-queue@webkit.org <commit-queue@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 9 Mar 2017 02:16:51 +0000 (02:16 +0000)
commitbc82d48ef04bc1ffd972024b781994172bf187a4
tree3eadd41bd2a72c2d3545b647f4950c4f84583d1c
parentb70b2c9280444b7f17a1c58c284f3a27faa8892f
Enable async image decoding for large images
https://bugs.webkit.org/show_bug.cgi?id=165039

Patch by Said Abou-Hallawa <sabouhallawa@apple.com> on 2017-03-08
Reviewed by Simon Fraser.

Source/WebCore:

Once FrameView finishes flushing compositing, it will request all its
images, which intersect with the tileCoverageRect (aka the futureRect)
of the  TileController, to start asynchronously decoding their image frames.
This should improve the image first time paint and the scrolling scenarios.
It also makes the scrolling more responsive by removing the decoding step
from the main thread.

For now we are going to disable the asynchronous image decoding for the
webkit test runner because drawing the image does not block the page rendering
anymore. An image can be repainted later when its frame is ready for painting.
This can cause a test to fail because the webkit test runner may capture
an image for the page before painting all the images. The asynchronous image
decoding can to be explicitly enabled from the test page. Once the specs
of the image 'async' attribute and 'ready' event is approved, this should
be revisited. It is important to test what we ship and eventually async
image decoding should be enabled in the webkit test runner.

* html/HTMLImageElement.cpp:
(WebCore::HTMLImageElement::didAttachRenderers):
(WebCore::HTMLImageElement::willDetachRenderers):
Register/unregister the renderer of the HTMLImageElement for async image decoding
from the RenderView. I followed what HTMLMediaElement to registerForVisibleInViewportCallback().
* html/HTMLImageElement.h:

* page/FrameView.cpp:
(WebCore::FrameView::flushCompositingStateForThisFrame): For all the images inside
the  tileCoverageRect of the TileController, request async image decoding.
(WebCore::FrameView::applyRecursivelyWithAbsoluteRect): Request the async image decoding for the
the images inside this FrameView and then recursively do the same thing for all sub FrameViews.
(WebCore::FrameView::requestAsyncDecodingForImagesInAbsoluteRect): Calls the RenderView to
make the request.
(WebCore::FrameView::requestAsyncDecodingForImagesInAbsoluteRectIncludingSubframes): Calls
applyRecursivelyWithAbsoluteRect giving an apply lambda which calls requestAsyncDecodingForImagesInAbsoluteRect.
* page/FrameView.h:

* platform/graphics/BitmapImage.cpp:
(WebCore::BitmapImage::destroyDecodedData): ImageSource::hasDecodingQueue() is renamed to ImageSource::hasAsyncDecodingQueue().
(WebCore::BitmapImage::dataChanged): Use String::utf8().data() instead of String::characters8().
(WebCore::BitmapImage::draw): If drawing the current frame is called while it is being
decoded, draw the the image if the current frame was decoded but for a different size
and and will not invoke decoding while painting. If the frame is being decoded and there
isn't a decoded frame, return without drawing but set a flag that that this image needs
a repaint.
(WebCore::BitmapImage::internalStartAnimation): Use String::utf8().data() instead of String::characters8().
(WebCore::BitmapImage::advanceAnimation): Ditto.
(WebCore::BitmapImage::internalAdvanceAnimation): Ditto.
(WebCore::BitmapImage::newFrameNativeImageAvailableAtIndex): Now this callback can be
called form the ImageFrameCache when finishing a frame of an animated image or the
frame of a large image. For large images, we need to call CachedImage::changedInRect()
if this image needs a repaint. If the decoding queue is idle, we should close it.
(WebCore::BitmapImage::requestAsyncDecoding): This is called form
RenderView::requestAsyncDecodingForImagesInRect().
* platform/graphics/BitmapImage.h:

* platform/graphics/Image.h:
(WebCore::Image::requestAsyncDecoding): Add the default implementation of a virtual function.

* platform/graphics/ImageFrameCache.cpp:
(WebCore::ImageFrameCache::~ImageFrameCache): hasDecodingQueue() was renamed to hasAsyncDecodingQueue().
(WebCore::ImageFrameCache::decodingQueue): Change the QNS of the decoding thread to be WorkQueue::QOS::Default.
WorkQueue::QOS::UserInteractive causes the scrolling thread to preempted  which can make the scrolling choppy.
(WebCore::ImageFrameCache::startAsyncDecodingQueue): hasDecodingQueue() was renamed to hasAsyncDecodingQueue().
(WebCore::ImageFrameCache::requestFrameAsyncDecodingAtIndex): Ditto.
(WebCore::ImageFrameCache::isAsyncDecodingQueueIdle): A helper function to tell whether the decoding thread is idle.
(WebCore::ImageFrameCache::stopAsyncDecodingQueue): hasDecodingQueue() was renamed to hasAsyncDecodingQueue().
* platform/graphics/ImageFrameCache.h:
(WebCore::ImageFrameCache::hasAsyncDecodingQueue): Rename this function to be consistent with the rest of the functions.
(WebCore::ImageFrameCache::hasDecodingQueue): Deleted.

* platform/graphics/ImageSource.h:
(WebCore::ImageSource::hasAsyncDecodingQueue): hasDecodingQueue() was renamed to hasAsyncDecodingQueue().
(WebCore::ImageSource::isAsyncDecodingQueueIdle): A wrapper for ImageFrameCache::isAsyncDecodingQueueIdle().
(WebCore::ImageSource::hasDecodingQueue): Deleted.

* platform/graphics/TiledBacking.h: Fix a comment. tileGridExtent() is the only one that is used for testing.

* rendering/RenderElement.cpp:
(WebCore::RenderElement::~RenderElement):
(WebCore::RenderElement::intersectsAbsoluteRect): Make this function does what shouldRepaintForImageAnimation()
was doing expect the check for document.activeDOMObjectsAreSuspended().
(WebCore::RenderElement::newImageAnimationFrameAvailable): Replace the call to
shouldRepaintForImageAnimation() by checking document().activeDOMObjectsAreSuspended() then a call to
RenderElement::intersectsAbsoluteRect().
(WebCore::RenderElement::repaintForPausedImageAnimationsIfNeeded): Ditto.
(WebCore::RenderElement::registerForAsyncImageDecodingCallback): Call the RenderView to register the renderer..
(WebCore::RenderElement::unregisterForAsyncImageDecodingCallback): Call the RenderView to unregister the renderer..
(WebCore::shouldRepaintForImageAnimation): Deleted.
* rendering/RenderElement.h:

* rendering/RenderObject.cpp:
(WebCore::RenderObject::setIsRegisteredForAsyncImageDecodingCallback):
* rendering/RenderObject.h:
(WebCore::RenderObject::isRegisteredForAsyncImageDecodingCallback):
(WebCore::RenderObject::RenderObjectRareData::RenderObjectRareData):
Mark/unmark the renderer for being RegisteredForAsyncImageDecodingCallback.

* rendering/RenderReplaced.h: Make intrinsicSize() be a public function.

* rendering/RenderView.cpp:
(WebCore::RenderView::registerForAsyncImageDecodingCallback): Register a renderer for async image decoding.
(WebCore::RenderView::unregisterForAsyncImageDecodingCallback): Remove a renderer from the list of async image decoding.
(WebCore::RenderView::requestAsyncDecodingForImagesInAbsoluteRect):This loops through all the registered RenderImages
inside this RenderView and if any of them intersects with the rectangle, requestAsyncDecoding for it.
* rendering/RenderView.h:

Source/WebKit2:

Add WK2 preferences for setting/getting LargeImageAsyncDecoding and
AnimatedImageAsyncDecoding.

* UIProcess/API/C/WKPreferences.cpp:
(WKPreferencesSetLargeImageAsyncDecodingEnabled):
(WKPreferencesGetLargeImageAsyncDecodingEnabled):
(WKPreferencesSetAnimatedImageAsyncDecodingEnabled):
(WKPreferencesGetAnimatedImageAsyncDecodingEnabled):
* UIProcess/API/C/WKPreferencesRefPrivate.h:

Tools:

Disable LargeImageAsyncDecoding for DRT/WTR.

* DumpRenderTree/mac/DumpRenderTree.mm:
(resetWebPreferencesToConsistentValues):
* WebKitTestRunner/TestController.cpp:
(WTR::TestController::resetPreferencesToConsistentValues):

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@213618 268f45cc-cd09-0410-ab3c-d52691b4dbfc
25 files changed:
Source/WebCore/ChangeLog
Source/WebCore/html/HTMLImageElement.cpp
Source/WebCore/html/HTMLImageElement.h
Source/WebCore/page/FrameView.cpp
Source/WebCore/page/FrameView.h
Source/WebCore/platform/graphics/BitmapImage.cpp
Source/WebCore/platform/graphics/BitmapImage.h
Source/WebCore/platform/graphics/Image.h
Source/WebCore/platform/graphics/ImageFrameCache.cpp
Source/WebCore/platform/graphics/ImageFrameCache.h
Source/WebCore/platform/graphics/ImageSource.h
Source/WebCore/platform/graphics/TiledBacking.h
Source/WebCore/rendering/RenderElement.cpp
Source/WebCore/rendering/RenderElement.h
Source/WebCore/rendering/RenderObject.cpp
Source/WebCore/rendering/RenderObject.h
Source/WebCore/rendering/RenderReplaced.h
Source/WebCore/rendering/RenderView.cpp
Source/WebCore/rendering/RenderView.h
Source/WebKit2/ChangeLog
Source/WebKit2/UIProcess/API/C/WKPreferences.cpp
Source/WebKit2/UIProcess/API/C/WKPreferencesRefPrivate.h
Tools/ChangeLog
Tools/DumpRenderTree/mac/DumpRenderTree.mm
Tools/WebKitTestRunner/TestController.cpp