Network Cache: Disk cache getting filled by YouTube video data
authorantti@apple.com <antti@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 28 Apr 2015 12:49:30 +0000 (12:49 +0000)
committerantti@apple.com <antti@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 28 Apr 2015 12:49:30 +0000 (12:49 +0000)
https://bugs.webkit.org/show_bug.cgi?id=144259

Reviewed by Darin Adler.

Source/WebCore:

MSE media is loaded via XHR and tends to eventually fill the cache.

YouTube serves the media chunks cacheable, however they are rarely (if ever) reused.
We can reduce disk writes and keep more useful resources around by not caching them

Test: http/tests/cache/disk-cache/disk-cache-media.html

* loader/DocumentLoader.cpp:
(WebCore::DocumentLoader::startLoadingMainResource):

    Set the requester.

* loader/cache/CachedRawResource.cpp:
(WebCore::CachedRawResource::CachedRawResource):
* page/DiagnosticLoggingKeys.cpp:
(WebCore::DiagnosticLoggingKeys::streamingMedia):
* page/DiagnosticLoggingKeys.h:
* platform/network/ResourceRequestBase.cpp:
(WebCore::ResourceRequestBase::adopt):
(WebCore::ResourceRequestBase::copyData):
(WebCore::equalIgnoringHeaderFields):
* platform/network/ResourceRequestBase.h:
(WebCore::ResourceRequestBase::requester):
(WebCore::ResourceRequestBase::setRequester):

    Add requester type to the request object. Currently this is main resource, xhr or unspecified.

(WebCore::ResourceRequestBase::encodeWithoutPlatformData):
(WebCore::ResourceRequestBase::decodeWithoutPlatformData):
* platform/network/cf/ResourceRequest.h:
(WebCore::ResourceRequest::deprecatedSetMainResourceRequest): Deleted.
(WebCore::ResourceRequest::deprecatedIsMainResourceRequest): Deleted.

    Replace this iOS only field with shared mechanism.

* platform/network/ios/QuickLook.mm:
(WebCore::QuickLookHandle::create):
* xml/XMLHttpRequest.cpp:
(WebCore::XMLHttpRequest::createRequest):

    Set the requester.

Source/WebKit2:

MSE media is loaded via XHR and tends to eventually fill the cache.

YouTube serves the media chunks cacheable, however they are rarely (if ever) reused.
We can reduce disk writes and keep more useful resources around by not caching them

* NetworkProcess/NetworkResourceLoader.cpp:
(WebKit::NetworkResourceLoader::didReceiveResponseAsync):
(WebKit::NetworkResourceLoader::didRetrieveCacheEntry):
* NetworkProcess/cache/NetworkCache.cpp:
(WebKit::NetworkCache::makeStoreDecision):

    Don't store if the requester is XHR and response has video or audio content type.

* NetworkProcess/cache/NetworkCache.h:
* NetworkProcess/cache/NetworkCacheStatistics.cpp:
(WebKit::NetworkCache::storeDecisionToDiagnosticKey):
* Shared/Network/NetworkResourceLoadParameters.cpp:
(WebKit::NetworkResourceLoadParameters::NetworkResourceLoadParameters):
(WebKit::NetworkResourceLoadParameters::encode):
(WebKit::NetworkResourceLoadParameters::decode):
* Shared/Network/NetworkResourceLoadParameters.h:

    The requester type is now part of the ResourceRequest, no need for separate parameter.

* Shared/mac/WebCoreArgumentCodersMac.mm:
(IPC::ArgumentCoder<ResourceRequest>::encodePlatformData):
(IPC::ArgumentCoder<ResourceRequest>::decodePlatformData):
* WebProcess/Network/WebResourceLoadScheduler.cpp:
(WebKit::WebResourceLoadScheduler::scheduleLoad):

LayoutTests:

* http/tests/cache/disk-cache/disk-cache-media-expected.txt: Added.
* http/tests/cache/disk-cache/disk-cache-media.html: Added.
* http/tests/cache/disk-cache/resources/cache-test.js:
(generateTestURL):

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@183467 268f45cc-cd09-0410-ab3c-d52691b4dbfc

24 files changed:
LayoutTests/ChangeLog
LayoutTests/http/tests/cache/disk-cache/disk-cache-media-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/cache/disk-cache/disk-cache-media.html [new file with mode: 0644]
LayoutTests/http/tests/cache/disk-cache/resources/cache-test.js
Source/WebCore/ChangeLog
Source/WebCore/loader/DocumentLoader.cpp
Source/WebCore/loader/cache/CachedRawResource.cpp
Source/WebCore/page/DiagnosticLoggingKeys.cpp
Source/WebCore/page/DiagnosticLoggingKeys.h
Source/WebCore/platform/network/ResourceRequestBase.cpp
Source/WebCore/platform/network/ResourceRequestBase.h
Source/WebCore/platform/network/cf/ResourceRequest.h
Source/WebCore/platform/network/ios/QuickLook.mm
Source/WebCore/xml/XMLHttpRequest.cpp
Source/WebKit/mac/WebCoreSupport/WebFrameLoaderClient.mm
Source/WebKit2/ChangeLog
Source/WebKit2/NetworkProcess/NetworkResourceLoader.cpp
Source/WebKit2/NetworkProcess/cache/NetworkCache.cpp
Source/WebKit2/NetworkProcess/cache/NetworkCache.h
Source/WebKit2/NetworkProcess/cache/NetworkCacheStatistics.cpp
Source/WebKit2/Shared/Network/NetworkResourceLoadParameters.cpp
Source/WebKit2/Shared/Network/NetworkResourceLoadParameters.h
Source/WebKit2/Shared/mac/WebCoreArgumentCodersMac.mm
Source/WebKit2/WebProcess/Network/WebResourceLoadScheduler.cpp

index 74a3e55..bad96cf 100644 (file)
@@ -1,3 +1,15 @@
+2015-04-28  Antti Koivisto  <antti@apple.com>
+
+        Network Cache: Disk cache getting filled by YouTube video data
+        https://bugs.webkit.org/show_bug.cgi?id=144259
+
+        Reviewed by Darin Adler.
+
+        * http/tests/cache/disk-cache/disk-cache-media-expected.txt: Added.
+        * http/tests/cache/disk-cache/disk-cache-media.html: Added.
+        * http/tests/cache/disk-cache/resources/cache-test.js:
+        (generateTestURL):
+
 2015-04-25  Simon Fraser  <simon.fraser@apple.com>
 
         Eliminate styleDidChange with StyleDifferenceEqual when updates are actually necessary
diff --git a/LayoutTests/http/tests/cache/disk-cache/disk-cache-media-expected.txt b/LayoutTests/http/tests/cache/disk-cache/disk-cache-media-expected.txt
new file mode 100644 (file)
index 0000000..fa19f6d
--- /dev/null
@@ -0,0 +1,29 @@
+Tests that media resources loaded via XHR is not cached.
+
+On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
+
+
+running 6 tests
+
+response headers: {"Cache-control":"max-age=0","Content-Type":"text/plain"}
+response source: Network
+
+response headers: {"Cache-control":"max-age=100","Content-Type":"text/plain"}
+response source: Disk cache
+
+response headers: {"Cache-control":"max-age=0","Content-Type":"video/mp4"}
+response source: Network
+
+response headers: {"Cache-control":"max-age=100","Content-Type":"video/mp4"}
+response source: Network
+
+response headers: {"Cache-control":"max-age=0","Content-Type":"audio/mp4"}
+response source: Network
+
+response headers: {"Cache-control":"max-age=100","Content-Type":"audio/mp4"}
+response source: Network
+
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
diff --git a/LayoutTests/http/tests/cache/disk-cache/disk-cache-media.html b/LayoutTests/http/tests/cache/disk-cache/disk-cache-media.html
new file mode 100644 (file)
index 0000000..ccf506b
--- /dev/null
@@ -0,0 +1,29 @@
+<script src="/js-test-resources/js-test-pre.js"></script>
+<script src="resources/cache-test.js"></script>
+<body>
+<script>
+
+var testMatrix =
+[
+ [
+  { responseHeaders: {'Cache-control': 'max-age=0' } },
+  { responseHeaders: {'Cache-control': 'max-age=100' } },
+  ],
+ [
+  { responseHeaders: {'Content-Type': 'text/plain' } },
+  { responseHeaders: {'Content-Type': 'video/mp4' } },
+  { responseHeaders: {'Content-Type': 'audio/mp4' } },
+  ],
+ ];
+
+description("Tests that media resources loaded via XHR is not cached.");
+
+var tests = generateTests(testMatrix);
+
+debug("running " + tests.length + " tests");
+debug("");
+
+runTests(tests);
+
+</script>
+<script src="/js-test-resources/js-test-post.js"></script>
index 76a8b8e..2332d86 100644 (file)
@@ -46,7 +46,9 @@ function generateTestURL(test, includeBody, expiresInFutureIn304)
     var testURL = "resources/generate-response.cgi?include-body=" + (includeBody ? "1" : "0");
     if (expiresInFutureIn304)
         testURL += "&expires-in-future-in-304=1";
-    testURL += "&uniqueId=" + uniqueTestId++ + "&Content-type=text/plain";
+    testURL += "&uniqueId=" + uniqueTestId++;
+    if (!test.responseHeaders || !test.responseHeaders["Content-Type"])
+        testURL += "&Content-Type=text/plain";
     for (var header in test.responseHeaders)
         testURL += '&' + header + '=' + makeHeaderValue(test.responseHeaders[header]);
     return testURL;
index df9baa9..16fc816 100644 (file)
@@ -1,3 +1,52 @@
+2015-04-28  Antti Koivisto  <antti@apple.com>
+
+        Network Cache: Disk cache getting filled by YouTube video data
+        https://bugs.webkit.org/show_bug.cgi?id=144259
+
+        Reviewed by Darin Adler.
+
+        MSE media is loaded via XHR and tends to eventually fill the cache.
+
+        YouTube serves the media chunks cacheable, however they are rarely (if ever) reused.
+        We can reduce disk writes and keep more useful resources around by not caching them
+
+        Test: http/tests/cache/disk-cache/disk-cache-media.html
+
+        * loader/DocumentLoader.cpp:
+        (WebCore::DocumentLoader::startLoadingMainResource):
+
+            Set the requester.
+
+        * loader/cache/CachedRawResource.cpp:
+        (WebCore::CachedRawResource::CachedRawResource):
+        * page/DiagnosticLoggingKeys.cpp:
+        (WebCore::DiagnosticLoggingKeys::streamingMedia):
+        * page/DiagnosticLoggingKeys.h:
+        * platform/network/ResourceRequestBase.cpp:
+        (WebCore::ResourceRequestBase::adopt):
+        (WebCore::ResourceRequestBase::copyData):
+        (WebCore::equalIgnoringHeaderFields):
+        * platform/network/ResourceRequestBase.h:
+        (WebCore::ResourceRequestBase::requester):
+        (WebCore::ResourceRequestBase::setRequester):
+
+            Add requester type to the request object. Currently this is main resource, xhr or unspecified.
+
+        (WebCore::ResourceRequestBase::encodeWithoutPlatformData):
+        (WebCore::ResourceRequestBase::decodeWithoutPlatformData):
+        * platform/network/cf/ResourceRequest.h:
+        (WebCore::ResourceRequest::deprecatedSetMainResourceRequest): Deleted.
+        (WebCore::ResourceRequest::deprecatedIsMainResourceRequest): Deleted.
+
+            Replace this iOS only field with shared mechanism.
+
+        * platform/network/ios/QuickLook.mm:
+        (WebCore::QuickLookHandle::create):
+        * xml/XMLHttpRequest.cpp:
+        (WebCore::XMLHttpRequest::createRequest):
+
+            Set the requester.
+
 2015-04-28  Namhoon Kim  <nakim@ea.com>
 
         Fix windows build error in WebCore related to bulk build.
index 699e0c7..1b0baa8 100644 (file)
@@ -934,11 +934,6 @@ void DocumentLoader::clearMainResourceLoader()
 {
     m_loadingMainResource = false;
 
-#if PLATFORM(IOS)
-    // FIXME: Remove PLATFORM(IOS)-guard once we upstream the iOS changes to ResourceRequest.h.
-    m_request.deprecatedSetMainResourceRequest(false);
-#endif
-
     if (this == frameLoader()->activeDocumentLoader())
         checkLoadComplete();
 }
@@ -1403,12 +1398,9 @@ void DocumentLoader::startLoadingMainResource()
         return;
     }
 
-#if PLATFORM(IOS)
-    // FIXME: Remove PLATFORM(IOS)-guard once we upstream the iOS changes to ResourceRequest.h.
-    m_request.deprecatedSetMainResourceRequest(true);
-#endif
-
     ResourceRequest request(m_request);
+    request.setRequester(ResourceRequest::Requester::Main);
+    
     static NeverDestroyed<ResourceLoaderOptions> mainResourceLoadOptions(SendCallbacks, SniffContent, BufferData, AllowStoredCredentials, AskClientForAllCredentials, SkipSecurityCheck, UseDefaultOriginRestrictionsForType, IncludeCertificateInfo);
     CachedResourceRequest cachedResourceRequest(request, mainResourceLoadOptions);
     cachedResourceRequest.setInitiator(*this);
index 316127b..3c1b295 100644 (file)
@@ -42,7 +42,6 @@ CachedRawResource::CachedRawResource(ResourceRequest& resourceRequest, Type type
     , m_identifier(0)
     , m_allowEncodedDataReplacement(true)
 {
-    // FIXME: The wrong CachedResource::Type here may cause a bad cast elsewhere.
     ASSERT(isMainOrRawResource());
 }
 
index fa7a486..5e91d10 100644 (file)
@@ -373,6 +373,11 @@ String DiagnosticLoggingKeys::sourceKey()
     return ASCIILiteral("sourceKey");
 }
 
+String DiagnosticLoggingKeys::streamingMedia()
+{
+    return ASCIILiteral("streamingMedia");
+}
+
 String DiagnosticLoggingKeys::styleSheetKey()
 {
     return ASCIILiteral("styleSheet");
index 03b823b..0d06e72 100644 (file)
@@ -106,6 +106,7 @@ public:
     static String sameLoadKey();
     static String scriptKey();
     static String sourceKey();
+    WEBCORE_EXPORT static String streamingMedia();
     static String styleSheetKey();
     static String svgDocumentKey();
     WEBCORE_EXPORT static String uncacheableStatusCodeKey();
index 9508ffd..3dbdbcf 100644 (file)
@@ -57,6 +57,7 @@ std::unique_ptr<ResourceRequest> ResourceRequestBase::adopt(std::unique_ptr<Cros
     request->setFirstPartyForCookies(data->m_firstPartyForCookies);
     request->setHTTPMethod(data->m_httpMethod);
     request->setPriority(data->m_priority);
+    request->setRequester(data->m_requester);
 
     request->updateResourceRequest();
     request->m_httpHeaderFields.adopt(WTF::move(data->m_httpHeaders));
@@ -90,6 +91,7 @@ std::unique_ptr<CrossThreadResourceRequestData> ResourceRequestBase::copyData()
     data->m_httpMethod = httpMethod().isolatedCopy();
     data->m_httpHeaders = httpHeaderFields().copyData();
     data->m_priority = priority();
+    data->m_requester = m_requester;
 
     data->m_responseContentDispositionEncodingFallbackArray.reserveInitialCapacity(m_responseContentDispositionEncodingFallbackArray.size());
     size_t encodingArraySize = m_responseContentDispositionEncodingFallbackArray.size();
@@ -496,6 +498,9 @@ bool equalIgnoringHeaderFields(const ResourceRequestBase& a, const ResourceReque
     if (a.priority() != b.priority())
         return false;
 
+    if (a.requester() != b.requester())
+        return false;
+
     FormData* formDataA = a.httpBody();
     FormData* formDataB = b.httpBody();
     
index c69aaf9..23bc8c2 100644 (file)
@@ -147,6 +147,10 @@ namespace WebCore {
         bool hiddenFromInspector() const { return m_hiddenFromInspector; }
         void setHiddenFromInspector(bool hiddenFromInspector) { m_hiddenFromInspector = hiddenFromInspector; }
 
+        enum class Requester { Unspecified, Main, XHR };
+        WEBCORE_EXPORT Requester requester() const { return m_requester; }
+        WEBCORE_EXPORT void setRequester(Requester requester) { m_requester = requester; }
+
 #if !PLATFORM(COCOA)
         bool encodingRequiresPlatformData() const { return true; }
 #endif
@@ -224,6 +228,7 @@ namespace WebCore {
         unsigned m_reportRawHeaders : 1;
         unsigned m_hiddenFromInspector : 1;
         unsigned m_priority : 4;
+        Requester m_requester { Requester::Unspecified };
 
     private:
         const ResourceRequest& asResourceRequest() const;
@@ -255,6 +260,7 @@ namespace WebCore {
         RefPtr<FormData> m_httpBody;
         bool m_allowCookies;
         ResourceLoadPriority m_priority;
+        ResourceRequestBase::Requester m_requester;
     };
     
     unsigned initializeMaximumHTTPConnectionCountPerHost();
@@ -276,6 +282,7 @@ void ResourceRequestBase::encodeWithoutPlatformData(Encoder& encoder) const
     encoder.encodeEnum(m_cachePolicy);
     encoder << m_allowCookies;
     encoder.encodeEnum(m_priority);
+    encoder.encodeEnum(m_requester);
 }
 
 template<class Decoder>
@@ -318,6 +325,9 @@ bool ResourceRequestBase::decodeWithoutPlatformData(Decoder& decoder)
         return false;
     m_priority = priority;
 
+    if (!decoder.decodeEnum(m_requester))
+        return false;
+
     return true;
 }
 
index b140a87..b1e2475 100644 (file)
@@ -118,17 +118,6 @@ namespace WebCore {
 
         static bool resourcePrioritiesEnabled();
 
-#if PLATFORM(IOS)
-        // FIXME: deprecatedIsMainResourceRequest() does not return the correct value if the ResourceRequest has been
-        // deserialized from an IPC message. As a result this function can only be relied on when networking is not in a
-        // separate process.
-        void deprecatedSetMainResourceRequest(bool isMainResourceRequest) const { m_mainResourceRequest = isMainResourceRequest; }
-        bool deprecatedIsMainResourceRequest() const { return m_mainResourceRequest; }
-
-    private:
-        mutable bool m_mainResourceRequest = false;
-#endif
-
     private:
         friend class ResourceRequestBase;
 
index 8c088f8..a9adf36 100644 (file)
@@ -459,7 +459,7 @@ QuickLookHandle::QuickLookHandle(NSURL *firstRequestURL, NSURLConnection *connec
 std::unique_ptr<QuickLookHandle> QuickLookHandle::create(ResourceHandle* handle, NSURLConnection *connection, NSURLResponse *nsResponse, id delegate)
 {
     ASSERT_ARG(handle, handle);
-    if (!handle->firstRequest().deprecatedIsMainResourceRequest() || ![WebCore::QLPreviewGetSupportedMIMETypesSet() containsObject:[nsResponse MIMEType]])
+    if (handle->firstRequest().requester() != ResourceRequest::Requester::Main || ![WebCore::QLPreviewGetSupportedMIMETypesSet() containsObject:[nsResponse MIMEType]])
         return nullptr;
 
     std::unique_ptr<QuickLookHandle> quickLookHandle(new QuickLookHandle([handle->firstRequest().nsURLRequest(DoNotUpdateHTTPBody) URL], connection, nsResponse, delegate));
@@ -471,7 +471,7 @@ std::unique_ptr<QuickLookHandle> QuickLookHandle::create(ResourceHandle* handle,
 std::unique_ptr<QuickLookHandle> QuickLookHandle::create(ResourceHandle* handle, SynchronousResourceHandleCFURLConnectionDelegate* connectionDelegate, CFURLResponseRef cfResponse)
 {
     ASSERT_ARG(handle, handle);
-    if (!handle->firstRequest().deprecatedIsMainResourceRequest() || ![WebCore::QLPreviewGetSupportedMIMETypesSet() containsObject:(NSString *)CFURLResponseGetMIMEType(cfResponse)])
+    if (handle->firstRequest().requester() != ResourceRequest::Requester::Main || ![WebCore::QLPreviewGetSupportedMIMETypesSet() containsObject:(NSString *)CFURLResponseGetMIMEType(cfResponse)])
         return nullptr;
 
     NSURLResponse *nsResponse = [NSURLResponse _responseWithCFURLResponse:cfResponse];
index ba450ce..46d5aab 100644 (file)
@@ -745,6 +745,7 @@ void XMLHttpRequest::createRequest(ExceptionCode& ec)
     m_uploadEventsAllowed = m_sameOriginRequest || uploadEvents || !isSimpleCrossOriginAccessRequest(m_method, m_requestHeaders);
 
     ResourceRequest request(m_url);
+    request.setRequester(ResourceRequest::Requester::XHR);
     request.setHTTPMethod(m_method);
 
     if (m_requestEntityBody) {
index 83b32fa..d8d4846 100644 (file)
@@ -363,8 +363,8 @@ void WebFrameLoaderClient::dispatchWillSendRequest(DocumentLoader* loader, unsig
     NSURLRequest *newURLRequest = currentURLRequest;
     ResourceLoadPriority priority = request.priority();
     bool isHiddenFromInspector = request.hiddenFromInspector();
+    auto requester = request.requester();
 #if PLATFORM(IOS)
-    bool isMainResourceRequest = request.deprecatedIsMainResourceRequest();
     if (implementations->webThreadWillSendRequestFunc) {
         newURLRequest = (NSURLRequest *)CallResourceLoadDelegateInWebThread(implementations->webThreadWillSendRequestFunc, webView, @selector(webThreadWebView:resource:willSendRequest:redirectResponse:fromDataSource:), [webView _objectForIdentifier:identifier], currentURLRequest, redirectResponse.nsURLResponse(), dataSource(loader));
     } else
@@ -375,10 +375,8 @@ void WebFrameLoaderClient::dispatchWillSendRequest(DocumentLoader* loader, unsig
     if (newURLRequest != currentURLRequest)
         request = newURLRequest;
     request.setHiddenFromInspector(isHiddenFromInspector);
-#if PLATFORM(IOS)
-    request.deprecatedSetMainResourceRequest(isMainResourceRequest);
-#endif
     request.setPriority(priority);
+    request.setRequester(requester);
 }
 
 bool WebFrameLoaderClient::shouldUseCredentialStorage(DocumentLoader* loader, unsigned long identifier)
index 0036a27..1978fef 100644 (file)
@@ -1,3 +1,40 @@
+2015-04-28  Antti Koivisto  <antti@apple.com>
+
+        Network Cache: Disk cache getting filled by YouTube video data
+        https://bugs.webkit.org/show_bug.cgi?id=144259
+
+        Reviewed by Darin Adler.
+
+        MSE media is loaded via XHR and tends to eventually fill the cache.
+
+        YouTube serves the media chunks cacheable, however they are rarely (if ever) reused.
+        We can reduce disk writes and keep more useful resources around by not caching them
+
+        * NetworkProcess/NetworkResourceLoader.cpp:
+        (WebKit::NetworkResourceLoader::didReceiveResponseAsync):
+        (WebKit::NetworkResourceLoader::didRetrieveCacheEntry):
+        * NetworkProcess/cache/NetworkCache.cpp:
+        (WebKit::NetworkCache::makeStoreDecision):
+
+            Don't store if the requester is XHR and response has video or audio content type.
+
+        * NetworkProcess/cache/NetworkCache.h:
+        * NetworkProcess/cache/NetworkCacheStatistics.cpp:
+        (WebKit::NetworkCache::storeDecisionToDiagnosticKey):
+        * Shared/Network/NetworkResourceLoadParameters.cpp:
+        (WebKit::NetworkResourceLoadParameters::NetworkResourceLoadParameters):
+        (WebKit::NetworkResourceLoadParameters::encode):
+        (WebKit::NetworkResourceLoadParameters::decode):
+        * Shared/Network/NetworkResourceLoadParameters.h:
+
+            The requester type is now part of the ResourceRequest, no need for separate parameter.
+
+        * Shared/mac/WebCoreArgumentCodersMac.mm:
+        (IPC::ArgumentCoder<ResourceRequest>::encodePlatformData):
+        (IPC::ArgumentCoder<ResourceRequest>::decodePlatformData):
+        * WebProcess/Network/WebResourceLoadScheduler.cpp:
+        (WebKit::WebResourceLoadScheduler::scheduleLoad):
+
 2015-04-28  Zan Dobersek  <zdobersek@igalia.com>
 
         [WK2] API::UserContentURLPattern creation functions should return Ref<>
index 04f073b..61024a0 100644 (file)
@@ -253,17 +253,18 @@ void NetworkResourceLoader::didReceiveResponseAsync(ResourceHandle* handle, cons
     shouldSendDidReceiveResponse = !m_cacheEntryForValidation;
 #endif
 
+    bool shouldWaitContinueDidReceiveResponse = originalRequest().requester() == ResourceRequest::Requester::Main;
     if (shouldSendDidReceiveResponse) {
         if (isSynchronous())
             m_synchronousLoadData->response = m_response;
         else {
-            if (!sendAbortingOnFailure(Messages::WebResourceLoader::DidReceiveResponse(m_response, m_parameters.isMainResource)))
+            if (!sendAbortingOnFailure(Messages::WebResourceLoader::DidReceiveResponse(m_response, shouldWaitContinueDidReceiveResponse)))
                 return;
         }
     }
 
     // For main resources, the web process is responsible for sending back a NetworkResourceLoader::ContinueDidReceiveResponse message.
-    bool shouldContinueDidReceiveResponse = !m_parameters.isMainResource;
+    bool shouldContinueDidReceiveResponse = !shouldWaitContinueDidReceiveResponse;
 #if ENABLE(NETWORK_CACHE)
     shouldContinueDidReceiveResponse = shouldContinueDidReceiveResponse || m_cacheEntryForValidation;
 #endif
@@ -543,7 +544,8 @@ void NetworkResourceLoader::didRetrieveCacheEntry(std::unique_ptr<NetworkCache::
         m_synchronousLoadData->response = entry->response();
         sendReplyToSynchronousRequest(*m_synchronousLoadData, entry->buffer());
     } else {
-        sendAbortingOnFailure(Messages::WebResourceLoader::DidReceiveResponse(entry->response(), m_parameters.isMainResource));
+        bool needsContinueDidReceiveResponseMessage = originalRequest().requester() == ResourceRequest::Requester::Main;
+        sendAbortingOnFailure(Messages::WebResourceLoader::DidReceiveResponse(entry->response(), needsContinueDidReceiveResponseMessage));
 
 #if ENABLE(SHAREABLE_RESOURCE)
         if (!entry->shareableResourceHandle().isNull())
index dae4792..855f252 100644 (file)
@@ -267,6 +267,15 @@ static bool isStatusCodePotentiallyCacheable(int statusCode)
     }
 }
 
+static bool isMediaMIMEType(const String& mimeType)
+{
+    if (mimeType.startsWith("video/", /*caseSensitive*/ false))
+        return true;
+    if (mimeType.startsWith("audio/", /*caseSensitive*/ false))
+        return true;
+    return false;
+}
+
 static StoreDecision makeStoreDecision(const WebCore::ResourceRequest& originalRequest, const WebCore::ResourceResponse& response)
 {
     if (!originalRequest.url().protocolIsInHTTPFamily() || !response.isHTTP())
@@ -290,8 +299,8 @@ static StoreDecision makeStoreDecision(const WebCore::ResourceRequest& originalR
             return StoreDecision::NoDueToHTTPStatusCode;
     }
 
-    // Main resource has ResourceLoadPriorityVeryHigh.
-    bool storeUnconditionallyForHistoryNavigation = originalRequest.priority() == WebCore::ResourceLoadPriorityVeryHigh;
+    bool isMainResource = originalRequest.requester() == WebCore::ResourceRequest::Requester::Main;
+    bool storeUnconditionallyForHistoryNavigation = isMainResource || originalRequest.priority() == WebCore::ResourceLoadPriorityVeryHigh;
     if (!storeUnconditionallyForHistoryNavigation) {
         auto now = std::chrono::system_clock::now();
         bool hasNonZeroLifetime = !response.cacheControlContainsNoCache() && WebCore::computeFreshnessLifetimeForHTTPFamily(response, now) > 0_ms;
@@ -301,6 +310,14 @@ static StoreDecision makeStoreDecision(const WebCore::ResourceRequest& originalR
             return StoreDecision::NoDueToUnlikelyToReuse;
     }
 
+    // Media loaded via XHR is likely being used for MSE streaming (YouTube and Netflix for example).
+    // Streaming media fills the cache quickly and is unlikely to be reused.
+    // FIXME: We should introduce a separate media cache partition that doesn't affect other resources.
+    // FIXME: We should also make sure make the MSE paths are copy-free so we can use mapped buffers from disk effectively.
+    bool isLikelyStreamingMedia = originalRequest.requester() == WebCore::ResourceRequest::Requester::XHR && isMediaMIMEType(response.mimeType());
+    if (isLikelyStreamingMedia)
+        return StoreDecision::NoDueToStreamingMedia;
+
     return StoreDecision::Yes;
 }
 
index ff37a68..5995db3 100644 (file)
@@ -71,7 +71,8 @@ enum class StoreDecision {
     NoDueToNoStoreResponse,
     NoDueToHTTPStatusCode,
     NoDueToNoStoreRequest,
-    NoDueToUnlikelyToReuse
+    NoDueToUnlikelyToReuse,
+    NoDueToStreamingMedia
 };
 
 enum class UseDecision {
index f436713..a1d9aa5 100644 (file)
@@ -244,6 +244,8 @@ static String storeDecisionToDiagnosticKey(StoreDecision storeDecision)
         return WebCore::DiagnosticLoggingKeys::uncacheableStatusCodeKey();
     case StoreDecision::NoDueToUnlikelyToReuse:
         return WebCore::DiagnosticLoggingKeys::unlikelyToReuseKey();
+    case StoreDecision::NoDueToStreamingMedia:
+        return WebCore::DiagnosticLoggingKeys::streamingMedia();
     case StoreDecision::Yes:
         // It was stored but could not be retrieved so it must have been pruned from the cache.
         return WebCore::DiagnosticLoggingKeys::noLongerInCacheKey();
index aaae7b4..c5d8465 100644 (file)
@@ -45,7 +45,6 @@ NetworkResourceLoadParameters::NetworkResourceLoadParameters()
     , allowStoredCredentials(DoNotAllowStoredCredentials)
     , clientCredentialPolicy(DoNotAskClientForAnyCredentials)
     , shouldClearReferrerOnHTTPSToHTTPRedirect(true)
-    , isMainResource(false)
     , defersLoading(false)
     , needsCertificateInfo(false)
     , maximumBufferingTime(0_ms)
@@ -94,7 +93,6 @@ void NetworkResourceLoadParameters::encode(IPC::ArgumentEncoder& encoder) const
     encoder.encodeEnum(allowStoredCredentials);
     encoder.encodeEnum(clientCredentialPolicy);
     encoder << shouldClearReferrerOnHTTPSToHTTPRedirect;
-    encoder << isMainResource;
     encoder << defersLoading;
     encoder << needsCertificateInfo;
     encoder << maximumBufferingTime;
@@ -151,8 +149,6 @@ bool NetworkResourceLoadParameters::decode(IPC::ArgumentDecoder& decoder, Networ
         return false;
     if (!decoder.decode(result.shouldClearReferrerOnHTTPSToHTTPRedirect))
         return false;
-    if (!decoder.decode(result.isMainResource))
-        return false;
     if (!decoder.decode(result.defersLoading))
         return false;
     if (!decoder.decode(result.needsCertificateInfo))
index e0b5d4c..2e8b91c 100644 (file)
@@ -61,7 +61,6 @@ public:
     WebCore::StoredCredentials allowStoredCredentials;
     WebCore::ClientCredentialPolicy clientCredentialPolicy;
     bool shouldClearReferrerOnHTTPSToHTTPRedirect;
-    bool isMainResource;
     bool defersLoading;
     bool needsCertificateInfo;
     std::chrono::milliseconds maximumBufferingTime;
index 80a3558..2cafae3 100644 (file)
@@ -84,6 +84,7 @@ void ArgumentCoder<ResourceRequest>::encodePlatformData(ArgumentEncoder& encoder
 
     // The fallback array is part of CFURLRequest, but it is not encoded by WKCFURLRequestCreateSerializableRepresentation.
     encoder << resourceRequest.responseContentDispositionEncodingFallbackArray();
+    encoder.encodeEnum(resourceRequest.requester());
 }
 #else
 void ArgumentCoder<ResourceRequest>::encodePlatformData(ArgumentEncoder& encoder, const ResourceRequest& resourceRequest)
@@ -109,6 +110,7 @@ void ArgumentCoder<ResourceRequest>::encodePlatformData(ArgumentEncoder& encoder
 
     // The fallback array is part of NSURLRequest, but it is not encoded by WKNSURLRequestCreateSerializableRepresentation.
     encoder << resourceRequest.responseContentDispositionEncodingFallbackArray();
+    encoder.encodeEnum(resourceRequest.requester());
 }
 #endif
 
@@ -151,6 +153,11 @@ bool ArgumentCoder<ResourceRequest>::decodePlatformData(ArgumentDecoder& decoder
         responseContentDispositionEncodingFallbackArray.size() > 2 ? responseContentDispositionEncodingFallbackArray[2] : String()
     );
 
+    ResourceRequest::Requester requester;
+    if (!decoder.decodeEnum(requester))
+        return false;
+    resourceRequest.setRequester(requester);
+
     return true;
 }
 
index 925279e..0a16bc5 100644 (file)
@@ -173,7 +173,6 @@ void WebResourceLoadScheduler::scheduleLoad(ResourceLoader* resourceLoader, Cach
     // If there is no WebFrame then this resource cannot be authenticated with the client.
     loadParameters.clientCredentialPolicy = (webFrame && webPage && resourceLoader->isAllowedToAskUserForCredentials()) ? AskClientForAllCredentials : DoNotAskClientForAnyCredentials;
     loadParameters.shouldClearReferrerOnHTTPSToHTTPRedirect = shouldClearReferrerOnHTTPSToHTTPRedirect;
-    loadParameters.isMainResource = resource && resource->type() == CachedResource::MainResource;
     loadParameters.defersLoading = resourceLoader->defersLoading();
     loadParameters.needsCertificateInfo = resourceLoader->shouldIncludeCertificateInfo();
     loadParameters.maximumBufferingTime = maximumBufferingTime(resource);