[CG] Add the option to immediately decode an image frame and control its memory caching
authorcommit-queue@webkit.org <commit-queue@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 12 Oct 2016 01:40:14 +0000 (01:40 +0000)
committercommit-queue@webkit.org <commit-queue@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 12 Oct 2016 01:40:14 +0000 (01:40 +0000)
https://bugs.webkit.org/show_bug.cgi?id=163298

Patch by Said Abou-Hallawa <sabouhallawa@apple.com> on 2016-10-11
Reviewed by Simon Fraser.

This patch fixes two things. (1) An option is added to immediately decode an
image frame. This can be done by adding kCGImageSourceShouldCacheImmediately
to the ImageSource options dictionary. (2) BitmapImage should also control
when the image frame is actually deleted from memory. This can be done by
calling CGImageSourceCreateThumbnailAtIndex(). CG does not keep a reference
to the returned CGImageRef.

* platform/graphics/ImageFrame.h: Adding the DecodingMode enum class.
Ideally this should be a member of ImageDecoder class. But since we
have three header files for ImageDecoder, this can be added here till
the three files combined in one header file.

* platform/graphics/ImageFrameCache.cpp:
(WebCore::ImageFrameCache::setRenderTarget): Deleted.
* platform/graphics/ImageFrameCache.h:
* platform/graphics/ImageSource.cpp:
(WebCore::ImageSource::setRenderTarget):
* platform/graphics/ImageSource.h:
(WebCore::ImageSource::setRenderTarget): Deleted.
Unrelated change. The native image decoder is available from the ImageSource.
ImageSource::setTarget() does not need not to get it through ImageFrameCache.

* platform/graphics/cg/ImageDecoderCG.cpp:
(WebCore::createImageSourceOptions): Clean this function by using CFMutableDictionary.
(WebCore::imageSourceOptions): Cache two default ImageSource options and create new
ones for the non default cases.

(WebCore::ImageDecoder::createFrameImageAtIndex): Use the appropriate ImageSource function

* platform/graphics/cg/ImageDecoderCG.h:
* platform/graphics/win/ImageDecoderDirect2D.cpp:
(WebCore::ImageDecoder::createFrameImageAtIndex):
* platform/graphics/win/ImageDecoderDirect2D.h:
* platform/image-decoders/ImageDecoder.cpp:
(WebCore::ImageDecoder::createFrameImageAtIndex):
* platform/image-decoders/ImageDecoder.h:
 Change functions' signature to include a DecodingMode argument.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@207182 268f45cc-cd09-0410-ab3c-d52691b4dbfc

12 files changed:
Source/WebCore/ChangeLog
Source/WebCore/platform/graphics/ImageFrame.h
Source/WebCore/platform/graphics/ImageFrameCache.cpp
Source/WebCore/platform/graphics/ImageFrameCache.h
Source/WebCore/platform/graphics/ImageSource.cpp
Source/WebCore/platform/graphics/ImageSource.h
Source/WebCore/platform/graphics/cg/ImageDecoderCG.cpp
Source/WebCore/platform/graphics/cg/ImageDecoderCG.h
Source/WebCore/platform/graphics/win/ImageDecoderDirect2D.cpp
Source/WebCore/platform/graphics/win/ImageDecoderDirect2D.h
Source/WebCore/platform/image-decoders/ImageDecoder.cpp
Source/WebCore/platform/image-decoders/ImageDecoder.h

index 1feaf33..5d1a044 100644 (file)
@@ -1,3 +1,48 @@
+2016-10-11  Said Abou-Hallawa  <sabouhallawa@apple.com>
+
+        [CG] Add the option to immediately decode an image frame and control its memory caching
+        https://bugs.webkit.org/show_bug.cgi?id=163298
+
+        Reviewed by Simon Fraser.
+
+        This patch fixes two things. (1) An option is added to immediately decode an
+        image frame. This can be done by adding kCGImageSourceShouldCacheImmediately
+        to the ImageSource options dictionary. (2) BitmapImage should also control
+        when the image frame is actually deleted from memory. This can be done by
+        calling CGImageSourceCreateThumbnailAtIndex(). CG does not keep a reference
+        to the returned CGImageRef.
+
+        * platform/graphics/ImageFrame.h: Adding the DecodingMode enum class.
+        Ideally this should be a member of ImageDecoder class. But since we
+        have three header files for ImageDecoder, this can be added here till
+        the three files combined in one header file.
+
+        * platform/graphics/ImageFrameCache.cpp:
+        (WebCore::ImageFrameCache::setRenderTarget): Deleted.
+        * platform/graphics/ImageFrameCache.h:
+        * platform/graphics/ImageSource.cpp:
+        (WebCore::ImageSource::setRenderTarget):
+        * platform/graphics/ImageSource.h:
+        (WebCore::ImageSource::setRenderTarget): Deleted.
+        Unrelated change. The native image decoder is available from the ImageSource.
+        ImageSource::setTarget() does not need not to get it through ImageFrameCache.
+        
+        * platform/graphics/cg/ImageDecoderCG.cpp:
+        (WebCore::createImageSourceOptions): Clean this function by using CFMutableDictionary.
+        (WebCore::imageSourceOptions): Cache two default ImageSource options and create new
+        ones for the non default cases.
+
+        (WebCore::ImageDecoder::createFrameImageAtIndex): Use the appropriate ImageSource function
+
+        * platform/graphics/cg/ImageDecoderCG.h:
+        * platform/graphics/win/ImageDecoderDirect2D.cpp:
+        (WebCore::ImageDecoder::createFrameImageAtIndex):
+        * platform/graphics/win/ImageDecoderDirect2D.h:
+        * platform/image-decoders/ImageDecoder.cpp:
+        (WebCore::ImageDecoder::createFrameImageAtIndex):
+        * platform/image-decoders/ImageDecoder.h:
+         Change functions' signature to include a DecodingMode argument.
+
 2016-10-11  Chris Dumez  <cdumez@apple.com>
 
         select.options may return too many option elements
index 0805e30..b21e7f9 100644 (file)
@@ -73,6 +73,11 @@ enum class GammaAndColorProfileOption {
     Ignored
 };
 
+enum class DecodingMode {
+    OnDemand,
+    Immediate
+};
+
 class ImageFrame {
     friend class ImageFrameCache;
 public:
index e7a4171..fbfd601 100644 (file)
 
 #if USE(CG)
 #include "ImageDecoderCG.h"
-#elif USE(DIRECT2D)
-#include "GraphicsContext.h"
-#include "ImageDecoderDirect2D.h"
-#include <WinCodec.h>
 #else
 #include "ImageDecoder.h"
 #endif
@@ -378,12 +374,4 @@ NativeImagePtr ImageFrameCache::frameImageAtIndex(size_t index, SubsamplingLevel
     return frameMetadataAtIndex<NativeImagePtr, (&ImageFrame::nativeImage)>(index, subsamplingLevel, ImageFrame::Caching::MetadataAndImage);
 }
 
-#if USE(DIRECT2D)
-void ImageFrameCache::setRenderTarget(GraphicsContext& context)
-{
-    if (m_decoder)
-        m_decoder->setRenderTarget(context.platformContext());
-}
-#endif
-
 }
index 7ba9798..61775c8 100644 (file)
@@ -82,10 +82,6 @@ public:
     ImageOrientation frameOrientationAtIndex(size_t);
     NativeImagePtr frameImageAtIndex(size_t, SubsamplingLevel = SubsamplingLevel::Default);
 
-#if USE(DIRECT2D)
-    void setRenderTarget(GraphicsContext&);
-#endif
-
 private:
     template<typename T, T (ImageDecoder::*functor)() const>
     T metadata(const T& defaultValue, Optional<T>* cachedValue = nullptr);
index 92baba8..8256034 100644 (file)
@@ -32,6 +32,7 @@
 #if USE(CG)
 #include "ImageDecoderCG.h"
 #elif USE(DIRECT2D)
+#include "GraphicsContext.h"
 #include "ImageDecoderDirect2D.h"
 #include <WinCodec.h>
 #else
@@ -114,6 +115,16 @@ bool ImageSource::ensureDecoderAvailable(SharedBuffer* data)
     return true;
 }
 
+#if USE(DIRECT2D)
+void ImageSource::setRenderTarget(GraphicsContext& context)
+{
+    if (!isDecoderAvailable())
+        return;
+
+    m_decoder->setRenderTarget(context.platformContext());
+}
+#endif
+
 void ImageSource::setData(SharedBuffer* data, bool allDataReceived)
 {
     if (!data || !ensureDecoderAvailable(data))
index 31648dc..ccdf7f1 100644 (file)
@@ -59,6 +59,10 @@ public:
     bool ensureDecoderAvailable(SharedBuffer*);
     bool isDecoderAvailable() const { return m_decoder.get(); }
 
+#if USE(DIRECT2D)
+    void setRenderTarget(GraphicsContext&);
+#endif
+
     void setData(SharedBuffer* data, bool allDataReceived);
     bool dataChanged(SharedBuffer* data, bool allDataReceived);
 
@@ -96,10 +100,6 @@ public:
     void setAllowSubsampling(bool allowSubsampling) { m_allowSubsampling = allowSubsampling; }
     NativeImagePtr createFrameImageAtIndex(size_t, SubsamplingLevel = SubsamplingLevel::Default);
 
-#if USE(DIRECT2D)
-    void setRenderTarget(GraphicsContext& context) { m_frameCache.setRenderTarget(context); }
-#endif
-
 private:
     void clearFrameBufferCache(size_t);
     void clear(bool destroyAll, size_t count, SharedBuffer* data);
index 7ad1aab..b88f3ac 100644 (file)
@@ -57,32 +57,39 @@ const CFStringRef WebCoreCGImagePropertyAPNGLoopCount = CFSTR("LoopCount");
 const CFStringRef kCGImageSourceShouldPreferRGB32 = CFSTR("kCGImageSourceShouldPreferRGB32");
 const CFStringRef kCGImageSourceSkipMetadata = CFSTR("kCGImageSourceSkipMetadata");
 
-static RetainPtr<CFDictionaryRef> createImageSourceOptions(SubsamplingLevel subsamplingLevel)
+static RetainPtr<CFDictionaryRef> createImageSourceOptions(SubsamplingLevel subsamplingLevel, DecodingMode decodingMode)
 {
-    if (subsamplingLevel == SubsamplingLevel::First) {
-        const unsigned numOptions = 3;
-        const void* keys[numOptions] = { kCGImageSourceShouldCache, kCGImageSourceShouldPreferRGB32, kCGImageSourceSkipMetadata };
-        const void* values[numOptions] = { kCFBooleanTrue, kCFBooleanTrue, kCFBooleanTrue };
-        return CFDictionaryCreate(nullptr, keys, values, numOptions, &kCFTypeDictionaryKeyCallBacks, &kCFTypeDictionaryValueCallBacks);
+    RetainPtr<CFMutableDictionaryRef> options = adoptCF(CFDictionaryCreateMutable(nullptr, 0, &kCFTypeDictionaryKeyCallBacks, &kCFTypeDictionaryValueCallBacks));
+
+    CFDictionarySetValue(options.get(), kCGImageSourceShouldCache, kCFBooleanTrue);
+    CFDictionarySetValue(options.get(), kCGImageSourceShouldPreferRGB32, kCFBooleanTrue);
+    CFDictionarySetValue(options.get(), kCGImageSourceSkipMetadata, kCFBooleanTrue);
+
+    if (subsamplingLevel > SubsamplingLevel::First) {
+        RetainPtr<CFNumberRef> subsampleNumber;
+        subsamplingLevel = std::min(SubsamplingLevel::Last, std::max(SubsamplingLevel::First, subsamplingLevel));
+        int subsampleInt = 1 << static_cast<int>(subsamplingLevel); // [0..3] => [1, 2, 4, 8]
+        subsampleNumber = adoptCF(CFNumberCreate(nullptr,  kCFNumberIntType,  &subsampleInt));
+        CFDictionarySetValue(options.get(), kCGImageSourceSubsampleFactor, subsampleNumber.get());
     }
-    
-    subsamplingLevel = std::min(SubsamplingLevel::Last, std::max(SubsamplingLevel::First, subsamplingLevel));
-    int subsampleInt = 1 << static_cast<int>(subsamplingLevel); // [0..3] => [1, 2, 4, 8]
-    
-    RetainPtr<CFNumberRef> subsampleNumber = adoptCF(CFNumberCreate(nullptr,  kCFNumberIntType,  &subsampleInt));
-    const CFIndex numOptions = 4;
-    const void* keys[numOptions] = { kCGImageSourceShouldCache, kCGImageSourceShouldPreferRGB32, kCGImageSourceSkipMetadata, kCGImageSourceSubsampleFactor };
-    const void* values[numOptions] = { kCFBooleanTrue, kCFBooleanTrue, kCFBooleanTrue, subsampleNumber.get() };
-    return adoptCF(CFDictionaryCreate(nullptr, keys, values, numOptions, &kCFTypeDictionaryKeyCallBacks, &kCFTypeDictionaryValueCallBacks));
+
+    if (decodingMode == DecodingMode::Immediate) {
+        CFDictionarySetValue(options.get(), kCGImageSourceShouldCacheImmediately, kCFBooleanTrue);
+        CFDictionarySetValue(options.get(), kCGImageSourceCreateThumbnailFromImageAlways, kCFBooleanTrue);
+    }
+
+    return options;
 }
 
-static RetainPtr<CFDictionaryRef> imageSourceOptions(SubsamplingLevel subsamplingLevel = SubsamplingLevel::Default)
+static RetainPtr<CFDictionaryRef> imageSourceOptions(SubsamplingLevel subsamplingLevel = SubsamplingLevel::Default, DecodingMode decodingMode = DecodingMode::OnDemand)
 {
     if (subsamplingLevel > SubsamplingLevel::First)
-        return createImageSourceOptions(subsamplingLevel);
-    
-    static NeverDestroyed<RetainPtr<CFDictionaryRef>> options = createImageSourceOptions(SubsamplingLevel::First);
-    return options;
+        return createImageSourceOptions(subsamplingLevel, decodingMode);
+
+    static NeverDestroyed<RetainPtr<CFDictionaryRef>> optionsOnDemand = createImageSourceOptions(SubsamplingLevel::First, DecodingMode::OnDemand);
+    static NeverDestroyed<RetainPtr<CFDictionaryRef>> optionsImmediate = createImageSourceOptions(SubsamplingLevel::First, DecodingMode::Immediate);
+
+    return decodingMode == DecodingMode::OnDemand ? optionsOnDemand : optionsImmediate;
 }
 
 static ImageOrientation orientationFromProperties(CFDictionaryRef imageProperties)
@@ -331,11 +338,17 @@ unsigned ImageDecoder::frameBytesAtIndex(size_t index, SubsamplingLevel subsampl
     return frameSize.area() * 4;
 }
 
-NativeImagePtr ImageDecoder::createFrameImageAtIndex(size_t index, SubsamplingLevel subsamplingLevel) const
+NativeImagePtr ImageDecoder::createFrameImageAtIndex(size_t index, SubsamplingLevel subsamplingLevel, DecodingMode decodingMode) const
 {
     LOG(Images, "ImageDecoder %p createFrameImageAtIndex %lu", this, index);
 
-    RetainPtr<CGImageRef> image = adoptCF(CGImageSourceCreateImageAtIndex(m_nativeDecoder.get(), index, imageSourceOptions(subsamplingLevel).get()));
+    const auto* options = imageSourceOptions(subsamplingLevel, decodingMode).get();
+    RetainPtr<CGImageRef> image;
+
+    if (decodingMode == DecodingMode::OnDemand)
+        image = adoptCF(CGImageSourceCreateImageAtIndex(m_nativeDecoder.get(), index, options));
+    else
+        image = adoptCF(CGImageSourceCreateThumbnailAtIndex(m_nativeDecoder.get(), index, options));
     
 #if PLATFORM(IOS)
     // <rdar://problem/7371198> - CoreGraphics changed the default caching behaviour in iOS 4.0 to kCGImageCachingTransient
index 19329be..6bc2368 100644 (file)
@@ -62,7 +62,7 @@ public:
     bool frameAllowSubsamplingAtIndex(size_t) const;
     unsigned frameBytesAtIndex(size_t, SubsamplingLevel = SubsamplingLevel::Default) const;
     
-    NativeImagePtr createFrameImageAtIndex(size_t, SubsamplingLevel = SubsamplingLevel::Default) const;
+    NativeImagePtr createFrameImageAtIndex(size_t, SubsamplingLevel = SubsamplingLevel::Default, DecodingMode = DecodingMode::OnDemand) const;
     
     void setData(SharedBuffer&, bool allDataReceived);
     bool isAllDataReceived() const { return m_isAllDataReceived; }
index e6ae277..0f1bcb1 100644 (file)
@@ -192,7 +192,7 @@ void ImageDecoder::setRenderTarget(ID2D1RenderTarget* renderTarget)
     m_renderTarget = renderTarget;
 }
 
-NativeImagePtr ImageDecoder::createFrameImageAtIndex(size_t index, SubsamplingLevel subsamplingLevel) const
+NativeImagePtr ImageDecoder::createFrameImageAtIndex(size_t index, SubsamplingLevel subsamplingLevel, DecodingMode) const
 {
     if (!m_nativeDecoder)
         return nullptr;
index 61ca4da..dd002cb 100644 (file)
@@ -67,7 +67,7 @@ public:
     bool frameAllowSubsamplingAtIndex(size_t) const;
     unsigned frameBytesAtIndex(size_t, SubsamplingLevel = SubsamplingLevel::Default) const;
     
-    NativeImagePtr createFrameImageAtIndex(size_t, SubsamplingLevel = SubsamplingLevel::Default) const;
+    NativeImagePtr createFrameImageAtIndex(size_t, SubsamplingLevel = SubsamplingLevel::Default, DecodingMode = DecodingMode::OnDemand) const;
     
     void setData(SharedBuffer&, bool allDataReceived);
     bool isAllDataReceived() const { return m_isAllDataReceived; }
index 54bd1f8..436ed8d 100644 (file)
@@ -207,7 +207,7 @@ float ImageDecoder::frameDurationAtIndex(size_t index)
     return duration;
 }
 
-NativeImagePtr ImageDecoder::createFrameImageAtIndex(size_t index, SubsamplingLevel)
+NativeImagePtr ImageDecoder::createFrameImageAtIndex(size_t index, SubsamplingLevel, DecodingMode)
 {
     // Zero-height images can cause problems for some ports. If we have an empty image dimension, just bail.
     if (size().isEmpty())
index 26746e4..5fd4f1b 100644 (file)
@@ -137,7 +137,7 @@ namespace WebCore {
         
         float frameDurationAtIndex(size_t);
         
-        NativeImagePtr createFrameImageAtIndex(size_t, SubsamplingLevel);
+        NativeImagePtr createFrameImageAtIndex(size_t, SubsamplingLevel = SubsamplingLevel::Default, DecodingMode = DecodingMode::OnDemand);
 
         void setIgnoreGammaAndColorProfile(bool flag) { m_ignoreGammaAndColorProfile = flag; }
         bool ignoresGammaAndColorProfile() const { return m_ignoreGammaAndColorProfile; }