2009-03-25 Dean Jackson <dino@apple.com>
authordino@apple.com <dino@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 26 Mar 2009 01:38:52 +0000 (01:38 +0000)
committerdino@apple.com <dino@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 26 Mar 2009 01:38:52 +0000 (01:38 +0000)
        Reviewed by Simon Fraser

        https://bugs.webkit.org/show_bug.cgi?id=23361

        When using hardware compositing, some images can be directly
        rendered by the hardware - no need to draw them into a separate
        context, therefore saving memory.
        Only images with certain style properties can be directly
        composited - basically anything that is not a simple image requires
        the usual rendering path (eg. if the image has borders).

        Test: compositing/direct-image-compositing.html

        * manual-tests/resources/simple_image.png: Added.
        * manual-tests/simple-image-compositing.html: Added.
        * platform/graphics/Image.h:
        (WebCore::Image::startAnimation):
            - move this to public
        * rendering/RenderImage.cpp:
        (WebCore::RenderImage::imageChanged):
            - poke compositing layer if image has changed
        (WebCore::RenderImage::notifyFinished):
            - let the compositing layer know that it can render the image
        * rendering/RenderImage.h:
        * rendering/RenderLayer.cpp:
        (WebCore::RenderLayer::rendererContentChanged):
        * rendering/RenderLayer.h:
        * rendering/RenderLayerBacking.cpp:
        (WebCore::RenderLayerBacking::canUseInnerContentLayer):
        (WebCore::RenderLayerBacking::detectDrawingOptimizations):
        (WebCore::RenderLayerBacking::rendererContentChanged):
        * rendering/RenderLayerBacking.h:
            - code to hook up direct compositing of images where
              possible

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@41996 268f45cc-cd09-0410-ab3c-d52691b4dbfc

16 files changed:
LayoutTests/ChangeLog
LayoutTests/compositing/direct-image-compositing.html [new file with mode: 0644]
LayoutTests/compositing/resources/simple_image.png [new file with mode: 0644]
LayoutTests/platform/mac/compositing/direct-image-compositing-expected.checksum [new file with mode: 0644]
LayoutTests/platform/mac/compositing/direct-image-compositing-expected.png [new file with mode: 0644]
LayoutTests/platform/mac/compositing/direct-image-compositing-expected.txt [new file with mode: 0644]
WebCore/ChangeLog
WebCore/manual-tests/resources/simple_image.png [new file with mode: 0644]
WebCore/manual-tests/simple-image-compositing.html [new file with mode: 0644]
WebCore/platform/graphics/Image.h
WebCore/rendering/RenderImage.cpp
WebCore/rendering/RenderImage.h
WebCore/rendering/RenderLayer.cpp
WebCore/rendering/RenderLayer.h
WebCore/rendering/RenderLayerBacking.cpp
WebCore/rendering/RenderLayerBacking.h

index d5fbf51..bb89c14 100644 (file)
@@ -1,3 +1,17 @@
+2009-03-25  Dean Jackson  <dino@apple.com>
+
+        Reviewed by Simon Fraser.
+
+        https://bugs.webkit.org/show_bug.cgi?id=23361
+        
+        Test for direct compositing of images using hardware acceleration.
+
+        * compositing/direct-image-compositing.html: Added.
+        * compositing/resources/simple_image.png: Added.
+        * platform/mac/compositing/direct-image-compositing-expected.checksum: Added.
+        * platform/mac/compositing/direct-image-compositing-expected.png: Added.
+        * platform/mac/compositing/direct-image-compositing-expected.txt: Added.
+
 2009-03-25  Simon Fraser  <simon.fraser@apple.com>
 
         Reviewed by Anders Carlsson.
diff --git a/LayoutTests/compositing/direct-image-compositing.html b/LayoutTests/compositing/direct-image-compositing.html
new file mode 100644 (file)
index 0000000..2fd9141
--- /dev/null
@@ -0,0 +1,66 @@
+<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
+   "http://www.w3.org/TR/html4/loose.dtd">
+
+<html lang="en">
+<head>
+  <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
+  <title>Testing direct image layer optimisation</title>
+  <style type="text/css" media="screen">
+    img {
+      float: left;
+      width: 150px;
+      height: 150px;
+    }
+    img {
+      -webkit-transform: rotate3d(0, 0, 1, 0);
+    }
+    .test {
+      float: left;
+      height: 200px;
+      width: 260px;
+    }
+   </style>
+</head>
+<body>
+
+  <h1>Image optimisation in layers</h1>
+
+  <p>
+    This test exercises direct compositing of images with hardware acceleration. The visual results
+    using ACCELERATED_COMPOSITING and regular TOT should be identical. Running this test manually with
+    the correct debug options will show which elements are directly composited. See
+    <a href="https://bugs.webkit.org/show_bug.cgi?id=23361">https://bugs.webkit.org/show_bug.cgi?id=23361</a>
+  </p>
+
+  <div class="test">
+    <img src="resources/simple_image.png">
+    Basic image - no style - can be directly composited
+  </div>
+
+  <div class="test">
+    <img src="resources/simple_image.png" style="border: 5px solid blue;">
+    5px blue border - can NOT be directly composited
+  </div>
+
+  <div class="test">
+    <img src="resources/simple_image.png" style="margin: 5px 5px;">
+    margin - can NOT be directly composited
+  </div>
+
+  <div class="test">
+    <img src="resources/simple_image.png" style="background-color: grey;">
+    solid background - can be directly composited
+  </div>
+
+  <div class="test">
+    <img src="resources/simple_image.png" style="background: orange url(resources/simple_image.png) -50px -50px;">
+    background image - can NOT be directly composited
+  </div>
+
+  <div class="test">
+    <img src="resources/simple_image.png" style="-webkit-transform: rotate3d(0, 0, 1, 10deg);">
+    rotated but otherwise no style - can be directly composited
+  </div>
+
+</body>
+</html>
diff --git a/LayoutTests/compositing/resources/simple_image.png b/LayoutTests/compositing/resources/simple_image.png
new file mode 100644 (file)
index 0000000..4685399
Binary files /dev/null and b/LayoutTests/compositing/resources/simple_image.png differ
diff --git a/LayoutTests/platform/mac/compositing/direct-image-compositing-expected.checksum b/LayoutTests/platform/mac/compositing/direct-image-compositing-expected.checksum
new file mode 100644 (file)
index 0000000..d57e3a1
--- /dev/null
@@ -0,0 +1 @@
+357ab3861a0e3b02ff1f2a3532a9fca4
\ No newline at end of file
diff --git a/LayoutTests/platform/mac/compositing/direct-image-compositing-expected.png b/LayoutTests/platform/mac/compositing/direct-image-compositing-expected.png
new file mode 100644 (file)
index 0000000..1e2e3b5
Binary files /dev/null and b/LayoutTests/platform/mac/compositing/direct-image-compositing-expected.png differ
diff --git a/LayoutTests/platform/mac/compositing/direct-image-compositing-expected.txt b/LayoutTests/platform/mac/compositing/direct-image-compositing-expected.txt
new file mode 100644 (file)
index 0000000..ef249c7
--- /dev/null
@@ -0,0 +1,67 @@
+layer at (0,0) size 785x749
+  RenderView at (0,0) size 785x600
+layer at (0,0) size 785x749
+  RenderBlock {HTML} at (0,0) size 785x149
+    RenderBody {BODY} at (8,21) size 769x112
+      RenderBlock {H1} at (0,0) size 769x37
+        RenderText {#text} at (0,0) size 389x37
+          text run at (0,0) width 389: "Image optimisation in layers"
+      RenderBlock {P} at (0,58) size 769x54
+        RenderText {#text} at (0,0) size 749x54
+          text run at (0,0) width 595: "This test exercises direct compositing of images with hardware acceleration. The visual results "
+          text run at (595,0) width 34: "using"
+          text run at (0,18) width 683: "ACCELERATED_COMPOSITING and regular TOT should be identical. Running this test manually with "
+          text run at (683,18) width 66: "the correct"
+          text run at (0,36) width 442: "debug options will show which elements are directly composited. See "
+        RenderInline {A} at (0,0) size 305x18 [color=#0000EE]
+          RenderText {#text} at (442,36) size 305x18
+            text run at (442,36) width 305: "https://bugs.webkit.org/show_bug.cgi?id=23361"
+        RenderText {#text} at (0,0) size 0x0
+      RenderBlock (floating) {DIV} at (0,128) size 260x200
+        RenderText {#text} at (150,0) size 106x72
+          text run at (150,0) width 106: "Basic image - no"
+          text run at (150,18) width 83: "style - can be"
+          text run at (150,36) width 47: "directly"
+          text run at (150,54) width 72: "composited"
+      RenderBlock (floating) {DIV} at (260,128) size 260x200
+        RenderText {#text} at (160,0) size 100x72
+          text run at (160,0) width 100: "5px blue border"
+          text run at (160,18) width 88: "- can NOT be"
+          text run at (160,36) width 47: "directly"
+          text run at (160,54) width 72: "composited"
+      RenderBlock (floating) {DIV} at (0,328) size 260x200
+        RenderText {#text} at (160,0) size 79x72
+          text run at (160,0) width 79: "margin - can"
+          text run at (160,18) width 53: "NOT be"
+          text run at (160,36) width 47: "directly"
+          text run at (160,54) width 72: "composited"
+      RenderBlock (floating) {DIV} at (260,328) size 260x200
+        RenderText {#text} at (150,0) size 109x54
+          text run at (150,0) width 109: "solid background"
+          text run at (150,18) width 101: "- can be directly"
+          text run at (150,36) width 72: "composited"
+      RenderBlock (floating) {DIV} at (0,528) size 260x200
+        RenderText {#text} at (150,0) size 104x72
+          text run at (150,0) width 75: "background"
+          text run at (150,18) width 73: "image - can"
+          text run at (150,36) width 104: "NOT be directly"
+          text run at (150,54) width 72: "composited"
+      RenderBlock (floating) {DIV} at (260,528) size 260x200
+        RenderText {#text} at (150,0) size 83x90
+          text run at (150,0) width 67: "rotated but"
+          text run at (150,18) width 81: "otherwise no"
+          text run at (150,36) width 83: "style - can be"
+          text run at (150,54) width 47: "directly"
+          text run at (150,72) width 72: "composited"
+layer at (8,149) size 150x150
+  RenderImage {IMG} at (0,0) size 150x150
+layer at (268,149) size 160x160
+  RenderImage {IMG} at (0,0) size 160x160 [border: (5px solid #0000FF)]
+layer at (13,354) size 150x150
+  RenderImage {IMG} at (5,5) size 150x150
+layer at (268,349) size 150x150
+  RenderImage {IMG} at (0,0) size 150x150 [bgcolor=#808080]
+layer at (8,549) size 150x150
+  RenderImage {IMG} at (0,0) size 150x150 [bgcolor=#FFA500]
+layer at (268,549) size 150x150
+  RenderImage {IMG} at (0,0) size 150x150
index 7f315d3..f7c474b 100644 (file)
@@ -1,3 +1,40 @@
+2009-03-25  Dean Jackson  <dino@apple.com>
+
+        Reviewed by Simon Fraser
+
+        https://bugs.webkit.org/show_bug.cgi?id=23361
+
+        When using hardware compositing, some images can be directly
+        rendered by the hardware - no need to draw them into a separate
+        context, therefore saving memory.
+        Only images with certain style properties can be directly
+        composited - basically anything that is not a simple image requires
+        the usual rendering path (eg. if the image has borders).
+
+        Test: compositing/direct-image-compositing.html
+
+        * manual-tests/resources/simple_image.png: Added.
+        * manual-tests/simple-image-compositing.html: Added.
+        * platform/graphics/Image.h:
+        (WebCore::Image::startAnimation):
+            - move this to public
+        * rendering/RenderImage.cpp:
+        (WebCore::RenderImage::imageChanged):
+            - poke compositing layer if image has changed
+        (WebCore::RenderImage::notifyFinished):
+            - let the compositing layer know that it can render the image
+        * rendering/RenderImage.h:
+        * rendering/RenderLayer.cpp:
+        (WebCore::RenderLayer::rendererContentChanged):
+        * rendering/RenderLayer.h:
+        * rendering/RenderLayerBacking.cpp:
+        (WebCore::RenderLayerBacking::canUseInnerContentLayer):
+        (WebCore::RenderLayerBacking::detectDrawingOptimizations):
+        (WebCore::RenderLayerBacking::rendererContentChanged):
+        * rendering/RenderLayerBacking.h:
+            - code to hook up direct compositing of images where
+              possible
+
 2009-03-25  David Levin  <levin@chromium.org>
 
         Reviewed by Dimitri Glazkov.
diff --git a/WebCore/manual-tests/resources/simple_image.png b/WebCore/manual-tests/resources/simple_image.png
new file mode 100644 (file)
index 0000000..4685399
Binary files /dev/null and b/WebCore/manual-tests/resources/simple_image.png differ
diff --git a/WebCore/manual-tests/simple-image-compositing.html b/WebCore/manual-tests/simple-image-compositing.html
new file mode 100644 (file)
index 0000000..e44ffed
--- /dev/null
@@ -0,0 +1,76 @@
+<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
+   "http://www.w3.org/TR/html4/loose.dtd">
+
+<html lang="en">
+<head>
+  <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
+  <title>Checking for simple image layer optimisation</title>
+  <style type="text/css" media="screen">
+    img {
+      float: left;
+      width: 150px;
+      height: 150px;
+    }
+    p {
+      clear: both;
+      margin: 1em 2em;
+      height: 180px;
+    }
+    img {
+      -webkit-transform: rotate3d(0, 0, 1, 0);
+    }
+   </style>
+</head>
+<body>
+
+  <h1>Image optimisation in layers</h1>
+
+  <p style="height: auto">
+    In order to run this test you should enable the debugging options that indicate
+    what type of compositing layer is being used.
+  </p>
+  
+<pre>
+defaults write com.apple.Safari WebCoreLayerRepaintCounter -bool yes
+defaults write com.apple.Safari WebCoreLayerBorders -bool yes
+</pre>
+  
+  <p style="height: auto">
+    Directly composited image layers will have a yellow border and no repaint counter.
+  </p>
+
+  <p>
+    <img src="resources/simple_image.png">
+    Basic image - no style - can be directly composited
+  </p>
+
+  <p>
+    <img src="resources/simple_image.png" style="border: 5px solid blue;">
+    5px blue border - can NOT be directly composited
+  </p>
+
+  <p>
+    <img src="resources/simple_image.png" style="margin: 10px 20px;">
+    margin - can NOT be directly composited
+  </p>
+
+  <p>
+    <img src="resources/simple_image.png" style="background-color: grey;">
+    solid background - can be directly composited
+  </p>
+
+  <p>
+    <img src="resources/simple_image.png" style="background: orange url(resources/simple_image.png) -50px -50px;">
+    background image - can NOT be directly composited
+  </p>
+
+  <p>
+    <img src="resources/simple_image.png" style="-webkit-transform: rotate3d(0, 0, 1, 10deg);">
+    rotated but otherwise no style - can be directly composited
+  </p>
+
+
+
+
+</body>
+</html>
index b832be9..d6bde11 100644 (file)
@@ -114,9 +114,9 @@ public:
 
     SharedBuffer* data() { return m_data.get(); }
 
-    // It may look unusual that there is no start animation call as public API.  This is because
-    // we start and stop animating lazily.  Animation begins whenever someone draws the image.  It will
-    // automatically pause once all observers no longer want to render the image anywhere.
+    // Animation begins whenever someone draws the image, so startAnimation() is not normally called.
+    // It will automatically pause once all observers no longer want to render the image anywhere.
+    virtual void startAnimation(bool /*catchUpIfNecessary*/ = true) { }
     virtual void stopAnimation() {}
     virtual void resetAnimation() {}
     
@@ -158,8 +158,6 @@ protected:
     virtual bool mayFillWithSolidColor() { return false; }
     virtual Color solidColor() const { return Color(); }
     
-    virtual void startAnimation(bool /*catchUpIfNecessary*/ = true) { }
-    
     virtual void drawPattern(GraphicsContext*, const FloatRect& srcRect, const TransformationMatrix& patternTransform,
                              const FloatPoint& phase, CompositeOperator, const FloatRect& destRect);
 #if PLATFORM(CG)
index 82f3741..56ad490 100644 (file)
@@ -38,6 +38,7 @@
 #include "Page.h"
 #include "RenderView.h"
 #include <wtf/CurrentTime.h>
+#include <wtf/UnusedParam.h>
 
 #if ENABLE(WML)
 #include "WMLImageElement.h"
@@ -311,9 +312,32 @@ void RenderImage::imageChanged(WrappedImagePtr newImage, const IntRect* rect)
             repaintRect = contentBoxRect();
         
         repaintRectangle(repaintRect);
+
+#if USE(ACCELERATED_COMPOSITING)
+        if (hasLayer()) {
+            // Tell any potential compositing layers that the image needs updating.
+            layer()->rendererContentChanged();
+        }
+#endif
     }
 }
 
+void RenderImage::notifyFinished(CachedResource* newImage)
+{
+    if (documentBeingDestroyed())
+        return;
+
+#if USE(ACCELERATED_COMPOSITING)
+    if ((newImage == m_cachedImage) && hasLayer()) {
+        // tell any potential compositing layers
+        // that the image is done and they can reference it directly.
+        layer()->rendererContentChanged();
+    }
+#else
+    UNUSED_PARAM(newImage);
+#endif
+}
+    
 void RenderImage::resetAnimation()
 {
     if (m_cachedImage) {
index f3e0e40..042452f 100644 (file)
@@ -48,6 +48,7 @@ public:
     virtual int minimumReplacedHeight() const;
 
     virtual void imageChanged(WrappedImagePtr, const IntRect* = 0);
+    virtual void notifyFinished(CachedResource*);
     
     bool setImageSizeForAltText(CachedImage* newImage = 0);
 
index eea98ef..695eaf8 100644 (file)
@@ -224,6 +224,12 @@ RenderLayerCompositor* RenderLayer::compositor() const
     ASSERT(renderer()->view());
     return renderer()->view()->compositor();
 }
+    
+void RenderLayer::rendererContentChanged()
+{
+    if (m_backing)
+        m_backing->rendererContentChanged();
+}    
 #endif // USE(ACCELERATED_COMPOSITING)
 
 void RenderLayer::setStaticY(int staticY)
index 0754c94..b8ce0d3 100644 (file)
@@ -282,6 +282,10 @@ public:
     
 #if USE(ACCELERATED_COMPOSITING)
     RenderLayerCompositor* compositor() const;
+    
+    // Notification from the renderer that its content changed (e.g. current frame of image changed).
+    // Allows updates of layer content without repainting.
+    void rendererContentChanged();
 #endif
     
     void updateLayerPosition();
index 29c4676..36cec87 100644 (file)
@@ -555,6 +555,34 @@ bool RenderLayerBacking::hasNonCompositingContent() const
     return false;
 }
 
+// A layer can use an inner content layer if the render layer's object is a replaced object and has no children.
+// This allows the GraphicsLayer to display the RenderLayer contents directly; it's used for images.
+bool RenderLayerBacking::canUseInnerContentLayer() const
+{
+    RenderObject* renderObject = renderer();
+    
+    // Reject anything that isn't a RenderReplaced.
+    if (!renderObject->isReplaced())
+        return false;
+    
+    if (renderObject->hasMask())
+        return false;
+    
+    RenderStyle* style = renderObject->style();
+    
+    // Reject anything that has a background other than a solid color.
+    if (!hasSimpleBackground(style))
+        return false;
+    
+    // Only optimize images that are unadorned.
+    if (renderObject->isImage())
+        // Reject anything that has a border, a border-radius, outline, margin or padding.
+        return !hasBorderOutlineOrShadow(style) && !style->hasMargin() && !style->hasPadding();
+    
+    return false;
+}
+    
+    
 // A "simple container layer" is a RenderLayer which has no visible content to render.
 // It may have no children, or all its children may be themselves composited.
 // This is a useful optimization, because it allows us to avoid allocating backing store.
@@ -572,9 +600,36 @@ void RenderLayerBacking::detectDrawingOptimizations()
 {
     bool drawsContent = true;
 
-    if (isSimpleContainerCompositingLayer() || paintingGoesToWindow())
+    // Check if a replaced layer can be further simplified.
+    bool hasImageBackgroundColor = false;
+    if (canUseInnerContentLayer()) {
+        if (renderer()->isImage()) {
+            RenderImage* imageRenderer = (RenderImage*)renderer();
+            if (imageRenderer && imageRenderer->cachedImage() && imageRenderer->cachedImage()->image())
+                rendererContentChanged();
+            
+            drawsContent = false;
+        }
+        
+        if (rendererHasBackground()) {
+            m_graphicsLayer->setBackgroundColor(rendererBackgroundColor());
+            hasImageBackgroundColor = true;
+        }
+    } else {
+        m_graphicsLayer->clearContents();
+        drawsContent = true;
+        
+        if (isSimpleContainerCompositingLayer())
+            drawsContent = false;
+        else if (!hasImageBackgroundColor) {
+            // Clear the background color in case we are swapping away from a simple layer.
+            m_graphicsLayer->clearBackgroundColor();
+        }
+    }
+    
+    if (paintingGoesToWindow())
         drawsContent = false;
-
+    
     m_graphicsLayer->setDrawsContent(drawsContent);
 }
 
@@ -583,6 +638,27 @@ void RenderLayerBacking::invalidateDrawingOptimizations()
     m_simpleCompositingLayerStatusDirty = true;
 }
 
+void RenderLayerBacking::rendererContentChanged()
+{
+    if (canUseInnerContentLayer() && renderer()->isImage()) {
+        RenderImage* imageRenderer = (RenderImage*)renderer();
+        if (imageRenderer &&
+            imageRenderer->cachedImage() &&
+            imageRenderer->cachedImage()->image() &&
+            imageRenderer->cachedImage()->isLoaded()) {
+            // We have to wait until the image is fully loaded before setting it on the layer.
+            
+            // This is a no-op if the layer doesn't have an inner layer for the image.
+            m_graphicsLayer->setContentsToImage(imageRenderer->cachedImage()->image());
+            
+            // Image animation is "lazy", in that it automatically stops unless someone is drawing
+            // the image. So we have to kick the animation each time; this has the downside that the
+            // image will keep animating, even if its layer is not visible.
+            imageRenderer->cachedImage()->image()->startAnimation();
+        }
+    }
+}
+
 FloatPoint3D RenderLayerBacking::computeTransformOrigin(const IntRect& borderBox) const
 {
     RenderStyle* style = renderer()->style();
index 0f7ddcc..496f445 100644 (file)
@@ -88,6 +88,9 @@ public:
     // r is in the coordinate space of the layer's render object
     void setContentsNeedDisplayInRect(const IntRect& r);
 
+    // Notification from the renderer that its content changed; used by RenderImage.
+    void rendererContentChanged();
+
     // Interface to start, finish, suspend and resume animations and transitions
     bool startAnimation(double beginTime, const Animation* anim, const KeyframeList& keyframes);
     bool startTransition(double beginTime, int property, const RenderStyle* fromStyle, const RenderStyle* toStyle);
@@ -135,6 +138,9 @@ private:
     // Returns true if this RenderLayer only has content that can be rendered directly
     // by the compositing layer, without drawing (e.g. solid background color).
     bool isSimpleContainerCompositingLayer();
+    // Returns true if we can optimize the RenderLayer to draw the replaced content
+    // directly into a compositing buffer
+    bool canUseInnerContentLayer() const;
 
     bool rendererHasBackground() const;
     const Color& rendererBackgroundColor() const;