Use normal loading path for ping loads
authoryouenn@apple.com <youenn@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 26 Apr 2019 18:10:17 +0000 (18:10 +0000)
committeryouenn@apple.com <youenn@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 26 Apr 2019 18:10:17 +0000 (18:10 +0000)
https://bugs.webkit.org/show_bug.cgi?id=196807

Reviewed by Alex Christensen.

LayoutTests/imported/w3c:

* web-platform-tests/beacon/headers/header-content-type-expected.txt:

Source/WebCore:

Make use of regular code path for ping loads and beacon.
This is done conditionally on KeepAlive flag.
The benefits are a single loading code path and service worker interception.

For that purpose, introduce a LoaderStrategy switch based on KeepAlive runtime flag.
This switch is used to use ping loads when keepAlive is set or regular loads.
In case of regular loads, the keepAlive flag should be used to extend the lifetime of the load.

Migrate ping loads to use CachedResourceLoader instead of PingLoad.
For that purpose, introduce a new Ping CachedResource type.

Covered by existing tests.

* Modules/beacon/NavigatorBeacon.cpp:
(WebCore::NavigatorBeacon::sendBeacon):
* inspector/agents/InspectorPageAgent.cpp:
(WebCore::InspectorPageAgent::inspectorResourceType):
* loader/LinkLoader.cpp:
(WebCore::createLinkPreloadResourceClient):
* loader/LoaderStrategy.h:
* loader/PingLoader.cpp:
(WebCore::PingLoader::loadImage):
(WebCore::PingLoader::sendPing):
(WebCore::PingLoader::sendViolationReport):
(WebCore::PingLoader::startPingLoad):
* loader/PingLoader.h:
* loader/ResourceLoadInfo.cpp:
(WebCore::toResourceType):
* loader/SubresourceLoader.cpp:
(WebCore::logResourceLoaded):
* loader/cache/CachedResource.cpp:
(WebCore::CachedResource::defaultPriorityForResourceType):
(WebCore::CachedResource::load):
(WebCore::CachedResource::cancelLoad):
* loader/cache/CachedResource.h:
(WebCore::CachedResource::shouldUsePingLoad):
(WebCore::CachedResource::isMainOrMediaOrIconOrRawResource const):
* loader/cache/CachedResourceLoader.cpp:
(WebCore::createResource):
(WebCore::CachedResourceLoader::requestPingResource):
(WebCore::contentTypeFromResourceType):
(WebCore::CachedResourceLoader::checkInsecureContent const):
(WebCore::CachedResourceLoader::allowedByContentSecurityPolicy const):
(WebCore::CachedResourceLoader::canRequest):
(WebCore::isResourceSuitableForDirectReuse):
(WebCore::destinationForType):
* loader/cache/CachedResourceLoader.h:

Source/WebKit:

In case a NetworkResourceLoader has the keepAlive option we do the following:
- Always use NetworkLoadChecker as we might need it to do checks after the Web context is gone.
- In case of aborting a KeepAlive loader, remove it from NetworkConnectionToWebProcess map
and add it to a kept-alive NetworkSession load set. The loader is only kept alive if it
has not yet received a response. Mark the loader as kept-alive.
- In case loader is kept-alive, cancel the load as soon as a response is gathered.

* NetworkProcess/NetworkConnectionToWebProcess.cpp:
(WebKit::NetworkConnectionToWebProcess::transferKeptAliveLoad):
* NetworkProcess/NetworkConnectionToWebProcess.h:
* NetworkProcess/NetworkProcess.cpp:
(WebKit::NetworkProcess::addKeptAliveLoad):
(WebKit::NetworkProcess::removeKeptAliveLoad):
* NetworkProcess/NetworkProcess.h:
* NetworkProcess/NetworkResourceLoadMap.cpp:
(WebKit::NetworkResourceLoadMap::remove):
(WebKit::NetworkResourceLoadMap::take):
* NetworkProcess/NetworkResourceLoadMap.h:
* NetworkProcess/NetworkResourceLoader.cpp:
(WebKit::m_shouldCaptureExtraNetworkLoadMetrics):
(WebKit::NetworkResourceLoader::cleanup):
(WebKit::NetworkResourceLoader::abort):
(WebKit::NetworkResourceLoader::didReceiveResponse):
(WebKit::NetworkResourceLoader::continueWillSendRedirectedRequest):
* NetworkProcess/NetworkResourceLoader.h:
* WebProcess/Network/WebLoaderStrategy.cpp:
(WebKit::maximumBufferingTime):
(WebKit::WebLoaderStrategy::usePingLoad const):
* WebProcess/Network/WebLoaderStrategy.h:

LayoutTests:

* TestExpectations:
* http/tests/blink/sendbeacon/beacon-cross-origin.https-expected.txt:
* http/tests/blink/sendbeacon/connect-src-beacon-allowed.html:
* http/tests/security/contentSecurityPolicy/connect-src-beacon-allowed.html:
* http/tests/security/contentSecurityPolicy/report-only-connect-src-beacon-redirect-blocked-expected.txt:
* http/tests/security/contentSecurityPolicy/report-only-connect-src-beacon-redirect-blocked.php:
* http/tests/security/contentSecurityPolicy/user-style-sheet-font-crasher-expected.txt:
* http/wpt/beacon/beacon-async-error-logging-expected.txt:
* http/wpt/beacon/beacon-async-error-logging.html:
* http/wpt/beacon/connect-src-beacon-redirect-blocked.sub-expected.txt:
* http/wpt/beacon/contentextensions/beacon-redirect-blocked-expected.txt:
* http/wpt/beacon/resources/beacon-preflight.py:
(respondToCORSPreflight):
(main):
* platform/wk2/http/tests/security/contentSecurityPolicy/block-all-mixed-content/insecure-css-in-iframe-report-only-expected.txt:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@244700 268f45cc-cd09-0410-ab3c-d52691b4dbfc

40 files changed:
LayoutTests/ChangeLog
LayoutTests/TestExpectations
LayoutTests/http/tests/blink/sendbeacon/beacon-cross-origin.https-expected.txt
LayoutTests/http/tests/security/contentSecurityPolicy/user-style-sheet-font-crasher-expected.txt
LayoutTests/http/wpt/beacon/beacon-async-error-logging-expected.txt
LayoutTests/http/wpt/beacon/beacon-async-error-logging.html
LayoutTests/http/wpt/beacon/connect-src-beacon-redirect-blocked.sub-expected.txt
LayoutTests/http/wpt/beacon/contentextensions/beacon-redirect-blocked-expected.txt
LayoutTests/http/wpt/beacon/resources/beacon-preflight.py
LayoutTests/imported/w3c/ChangeLog
LayoutTests/imported/w3c/web-platform-tests/beacon/headers/header-content-type-expected.txt
LayoutTests/platform/mac-wk1/http/tests/security/contentSecurityPolicy/user-style-sheet-font-crasher-expected.txt [new file with mode: 0644]
LayoutTests/platform/win/http/tests/security/contentSecurityPolicy/user-style-sheet-font-crasher-expected.txt [new file with mode: 0644]
LayoutTests/platform/wk2/http/tests/security/contentSecurityPolicy/block-all-mixed-content/insecure-css-in-iframe-report-only-expected.txt
Source/WebCore/ChangeLog
Source/WebCore/Modules/beacon/NavigatorBeacon.cpp
Source/WebCore/inspector/agents/InspectorPageAgent.cpp
Source/WebCore/loader/LinkLoader.cpp
Source/WebCore/loader/LoaderStrategy.h
Source/WebCore/loader/PingLoader.cpp
Source/WebCore/loader/PingLoader.h
Source/WebCore/loader/ResourceLoadInfo.cpp
Source/WebCore/loader/SubresourceLoader.cpp
Source/WebCore/loader/cache/CachedResource.cpp
Source/WebCore/loader/cache/CachedResource.h
Source/WebCore/loader/cache/CachedResourceLoader.cpp
Source/WebCore/loader/cache/CachedResourceLoader.h
Source/WebKit/ChangeLog
Source/WebKit/NetworkProcess/NetworkConnectionToWebProcess.cpp
Source/WebKit/NetworkProcess/NetworkConnectionToWebProcess.h
Source/WebKit/NetworkProcess/NetworkProcess.cpp
Source/WebKit/NetworkProcess/NetworkProcess.h
Source/WebKit/NetworkProcess/NetworkResourceLoadMap.cpp
Source/WebKit/NetworkProcess/NetworkResourceLoadMap.h
Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp
Source/WebKit/NetworkProcess/NetworkResourceLoader.h
Source/WebKit/NetworkProcess/NetworkSession.cpp
Source/WebKit/NetworkProcess/NetworkSession.h
Source/WebKit/WebProcess/Network/WebLoaderStrategy.cpp
Source/WebKit/WebProcess/Network/WebLoaderStrategy.h

index 9d5b8be..52b7b78 100644 (file)
@@ -1,5 +1,28 @@
 2019-04-26  Youenn Fablet  <youenn@apple.com>
 
+        Use normal loading path for ping loads
+        https://bugs.webkit.org/show_bug.cgi?id=196807
+
+        Reviewed by Alex Christensen.
+
+        * TestExpectations:
+        * http/tests/blink/sendbeacon/beacon-cross-origin.https-expected.txt:
+        * http/tests/blink/sendbeacon/connect-src-beacon-allowed.html:
+        * http/tests/security/contentSecurityPolicy/connect-src-beacon-allowed.html:
+        * http/tests/security/contentSecurityPolicy/report-only-connect-src-beacon-redirect-blocked-expected.txt:
+        * http/tests/security/contentSecurityPolicy/report-only-connect-src-beacon-redirect-blocked.php:
+        * http/tests/security/contentSecurityPolicy/user-style-sheet-font-crasher-expected.txt:
+        * http/wpt/beacon/beacon-async-error-logging-expected.txt:
+        * http/wpt/beacon/beacon-async-error-logging.html:
+        * http/wpt/beacon/connect-src-beacon-redirect-blocked.sub-expected.txt:
+        * http/wpt/beacon/contentextensions/beacon-redirect-blocked-expected.txt:
+        * http/wpt/beacon/resources/beacon-preflight.py:
+        (respondToCORSPreflight):
+        (main):
+        * platform/wk2/http/tests/security/contentSecurityPolicy/block-all-mixed-content/insecure-css-in-iframe-report-only-expected.txt:
+
+2019-04-26  Youenn Fablet  <youenn@apple.com>
+
         Mark some cache-storage as slow on iOS-simulator
         https://bugs.webkit.org/show_bug.cgi?id=197316
 
index bdc45ae..f0c42eb 100644 (file)
@@ -1787,6 +1787,16 @@ webkit.org/b/168238 imported/w3c/web-platform-tests/dom/events/EventListener-inv
 imported/w3c/web-platform-tests/fetch/nosniff [ DumpJSConsoleLogInStdErr ]
 imported/w3c/web-platform-tests/html/semantics/embedded-content/the-iframe-element/sandbox-ascii-case-insensitive.html [ DumpJSConsoleLogInStdErr ]
 
+http/wpt/beacon/cors/cors-preflight-blob-failure.html [ DumpJSConsoleLogInStdErr ]
+http/wpt/beacon/cors/cors-preflight-blob-success.html [ DumpJSConsoleLogInStdErr ]
+http/wpt/beacon/cors/cors-preflight-cookie.html [ DumpJSConsoleLogInStdErr ]
+http/wpt/beacon/cors/cors-preflight-redirect-failure.html [ DumpJSConsoleLogInStdErr ]
+http/wpt/beacon/cors/cors-preflight-redirect-from-crossorigin-to-sameorigin.html [ DumpJSConsoleLogInStdErr ]
+http/wpt/beacon/cors/cors-preflight-redirect-success.html [ DumpJSConsoleLogInStdErr ]
+http/wpt/beacon/cors/crossorigin-arraybufferview-no-preflight.html [ DumpJSConsoleLogInStdErr ]
+
+http/tests/security/contentSecurityPolicy/block-all-mixed-content/insecure-css-in-iframe-report-only.html [ Pass Failure ]
+
 # Imported css-text test suite from WPT
 webkit.org/b/183258 imported/w3c/web-platform-tests/css/css-text/hanging-punctuation/hanging-punctuation-first-001.xht [ ImageOnlyFailure ]
 webkit.org/b/183258 imported/w3c/web-platform-tests/css/css-text/hanging-punctuation/hanging-punctuation-force-end-001.xht [ ImageOnlyFailure ]
index dcf06b4..c5664ed 100644 (file)
@@ -1,11 +1,12 @@
+Blocked access to external URL http://example.test:8000/blink/sendbeacon/resources/save-beacon.php?name=cross-origin
+CONSOLE MESSAGE: line 1: Beacon API cannot load http://example.test:8000/blink/sendbeacon/resources/save-beacon.php?name=cross-origin due to access control checks.
 Verify navigator.sendBeacon() mixed content checking.
 
 On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
 
 
-FAIL navigator.sendBeacon("http://example.test:8000/blink/sendbeacon/resources/save-beacon.php?name=cross-origin", "CrossOrigin"); should be false. Was true.
+PASS navigator.sendBeacon("http://example.test:8000/blink/sendbeacon/resources/save-beacon.php?name=cross-origin", "CrossOrigin"); is false
 PASS successfullyParsed is true
-Some tests failed.
 
 TEST COMPLETE
 
index 2b96adf..5b71768 100644 (file)
@@ -1,4 +1,6 @@
 CONSOLE MESSAGE: Refused to load http://127.0.0.1:8000/security/contentSecurityPolicy/example_font.woff because it does not appear in the font-src directive of the Content Security Policy.
+Blocked access to external URL http://webkit.org/report
 CONSOLE MESSAGE: Refused to load http://127.0.0.1:8000/security/contentSecurityPolicy/example_font.woff because it does not appear in the font-src directive of the Content Security Policy.
+Blocked access to external URL http://webkit.org/report
 The iframe below triggers a violation report creating the initial empty document. It should not crash the web process.
 
index 82008d9..49d1e74 100644 (file)
@@ -1,4 +1,4 @@
-CONSOLE MESSAGE: Beacon API cannot load http://invalid.localhost/. A server with the specified hostname could not be found.
+CONSOLE MESSAGE: Beacon API cannot load http://localhost:1/. Not allowed to use restricted network port
 
 PASS Should log an error message in the console 
 
index 00bc9d7..701799f 100644 (file)
@@ -14,8 +14,8 @@
         } else {
             setTimeout(function() { t.done(); }, 500);
         }
-        let invalidHost = "http://invalid.localhost";
-        assert_true(navigator.sendBeacon(invalidHost, 'test'), "sendBeacon should return true");
+        let hostWithInvalidPort = "http://localhost:1";
+        assert_true(navigator.sendBeacon(hostWithInvalidPort, 'test'), "sendBeacon should return true");
     }, "Should log an error message in the console");
 </script>
 </body>
index fc75d81..935e479 100644 (file)
@@ -1,3 +1,5 @@
+CONSOLE MESSAGE: Refused to connect to http://127.0.0.1:8800/WebKit/beacon/resources/beacon-preflight.py?allowCors=1&cmd=put&id=2539e883-7dfb-4dde-a227-a41c670d5fe1&redirect_status=307&location=http%3A%2F%2F127.0.0.1%3A8800%2FWebKit%2Fbeacon%2Fresources%2Fbeacon-preflight.py%3FallowCors%3D1%26cmd%3Dput%26id%3D2539e883-7dfb-4dde-a227-a41c670d5fe1&count=1 because it does not appear in the connect-src directive of the Content Security Policy.
+CONSOLE MESSAGE: Blocked by Content Security Policy.
 CONSOLE MESSAGE: Beacon API cannot load http://127.0.0.1:8800/WebKit/beacon/resources/beacon-preflight.py?allowCors=1&cmd=put&id=2539e883-7dfb-4dde-a227-a41c670d5fe1&redirect_status=307&location=http%3A%2F%2F127.0.0.1%3A8800%2FWebKit%2Fbeacon%2Fresources%2Fbeacon-preflight.py%3FallowCors%3D1%26cmd%3Dput%26id%3D2539e883-7dfb-4dde-a227-a41c670d5fe1&count=1. Blocked by Content Security Policy.
 
 PASS Redirect is blocked by CSP 
index 97b7952..e46327e 100644 (file)
@@ -1,4 +1,5 @@
-CONSOLE MESSAGE: Beacon API cannot load http://127.0.0.1:8800/WebKit/beacon/resources/beacon-preflight.py?allowCors=1&cmd=put&id=f470f43c-258c-4c82-b880-ace3bcdb211c&redirect_status=307&location=http%3A%2F%2F127.0.0.1%3A8800%2FWebKit%2Fbeacon%2Fresources%2Fbeacon-preflight.py%3FallowCors%3D1%26cmd%3Dput%26id%3Df470f43c-258c-4c82-b880-ace3bcdb211c&count=1. Blocked by content extension
+CONSOLE MESSAGE: Content blocker prevented frame displaying http://localhost:8800/WebKit/beacon/contentextensions/beacon-redirect-blocked.html from loading a resource from http://127.0.0.1:8800/WebKit/beacon/resources/beacon-preflight.py?allowCors=1&cmd=put&id=f470f43c-258c-4c82-b880-ace3bcdb211c&redirect_status=307&location=http%3A%2F%2F127.0.0.1%3A8800%2FWebKit%2Fbeacon%2Fresources%2Fbeacon-preflight.py%3FallowCors%3D1%26cmd%3Dput%26id%3Df470f43c-258c-4c82-b880-ace3bcdb211c&count=1
+CONSOLE MESSAGE: Beacon API cannot load http://localhost:8800/WebKit/beacon/resources/redirect.py?redirect_status=307&location=http%3A%2F%2F127.0.0.1%3A8800%2FWebKit%2Fbeacon%2Fresources%2Fbeacon-preflight.py%3FallowCors%3D1%26cmd%3Dput%26id%3Df470f43c-258c-4c82-b880-ace3bcdb211c. The URL was blocked by a content blocker
 
 PASS Content extensions should be able to block beacon redirects 
 
index 9e64e76..1d49cf3 100644 (file)
@@ -1,17 +1,17 @@
 import json
 
 def respondToCORSPreflight(request, response):
+  headers = [("Content-Type", "text/plain")]
   allow_cors = int(request.GET.first("allowCors", 0)) != 0;
   
   if not allow_cors:
     response.set_error(400, "Not allowed")
-    return "ERROR: Not allowed"
+    return headers, "ERROR: Not allowed"
   
   if not "Access-Control-Request-Method" in request.headers:
     response.set_error(400, "No Access-Control-Request-Method header")
-    return "ERROR: No access-control-request-method in preflight!"
+    return headers, "ERROR: No access-control-request-method in preflight!"
   
-  headers = [("Content-Type", "text/plain")]
   headers.append(("Access-Control-Allow-Origin", request.headers.get("Origin", "*")))
   headers.append(("Access-Control-Allow-Credentials", "true"))
   requested_method = request.headers.get("Access-Control-Request-Method", None)
@@ -51,4 +51,4 @@ def main(request, response):
     return [("Content-Type", "text/plain")], ""
 
   response.set_error(400, "Bad Command")
-  return "ERROR: Bad Command!"
+  return [("Content-Type", "text/plain")], "ERROR: Bad Command!"
index 22480e7..3396ced 100644 (file)
@@ -1,5 +1,14 @@
 2019-04-26  Youenn Fablet  <youenn@apple.com>
 
+        Use normal loading path for ping loads
+        https://bugs.webkit.org/show_bug.cgi?id=196807
+
+        Reviewed by Alex Christensen.
+
+        * web-platform-tests/beacon/headers/header-content-type-expected.txt:
+
+2019-04-26  Youenn Fablet  <youenn@apple.com>
+
         [Mac WK2 iOS Sim] Layout Test imported/w3c/web-platform-tests/webrtc/RTCRtpReceiver-getSynchronizationSources.https.html is a flaky failure
         https://bugs.webkit.org/show_bug.cgi?id=196633
         <rdar://problem/49627667>
index 3d8e9dc..60ee6e5 100644 (file)
@@ -1,10 +1,8 @@
 
-Harness Error (TIMEOUT), message = null
-
-TIMEOUT Test content-type header for a body string Test timed out
-NOTRUN Test content-type header for a body ArrayBufferView 
-NOTRUN Test content-type header for a body ArrayBuffer 
-NOTRUN Test content-type header for a body Blob 
-NOTRUN Test content-type header for a body FormData 
-NOTRUN Test content-type header for a body URLSearchParams 
+PASS Test content-type header for a body string 
+PASS Test content-type header for a body ArrayBufferView 
+PASS Test content-type header for a body ArrayBuffer 
+PASS Test content-type header for a body Blob 
+PASS Test content-type header for a body FormData 
+PASS Test content-type header for a body URLSearchParams 
 
diff --git a/LayoutTests/platform/mac-wk1/http/tests/security/contentSecurityPolicy/user-style-sheet-font-crasher-expected.txt b/LayoutTests/platform/mac-wk1/http/tests/security/contentSecurityPolicy/user-style-sheet-font-crasher-expected.txt
new file mode 100644 (file)
index 0000000..2b96adf
--- /dev/null
@@ -0,0 +1,4 @@
+CONSOLE MESSAGE: Refused to load http://127.0.0.1:8000/security/contentSecurityPolicy/example_font.woff because it does not appear in the font-src directive of the Content Security Policy.
+CONSOLE MESSAGE: Refused to load http://127.0.0.1:8000/security/contentSecurityPolicy/example_font.woff because it does not appear in the font-src directive of the Content Security Policy.
+The iframe below triggers a violation report creating the initial empty document. It should not crash the web process.
+
diff --git a/LayoutTests/platform/win/http/tests/security/contentSecurityPolicy/user-style-sheet-font-crasher-expected.txt b/LayoutTests/platform/win/http/tests/security/contentSecurityPolicy/user-style-sheet-font-crasher-expected.txt
new file mode 100644 (file)
index 0000000..2b96adf
--- /dev/null
@@ -0,0 +1,4 @@
+CONSOLE MESSAGE: Refused to load http://127.0.0.1:8000/security/contentSecurityPolicy/example_font.woff because it does not appear in the font-src directive of the Content Security Policy.
+CONSOLE MESSAGE: Refused to load http://127.0.0.1:8000/security/contentSecurityPolicy/example_font.woff because it does not appear in the font-src directive of the Content Security Policy.
+The iframe below triggers a violation report creating the initial empty document. It should not crash the web process.
+
index 8e5f651..060a4bf 100644 (file)
@@ -8,13 +8,13 @@ frame "<!--frame1-->" - willPerformClientRedirectToURL: https://127.0.0.1:8443/s
 frame "<!--frame1-->" - didFinishDocumentLoadForFrame
 main frame - didHandleOnloadEventsForFrame
 frame "<!--frame1-->" - didFinishLoadForFrame
-main frame - didFinishLoadForFrame
 frame "<!--frame1-->" - didStartProvisionalLoadForFrame
 frame "<!--frame1-->" - didCancelClientRedirectForFrame
 frame "<!--frame1-->" - didCommitLoadForFrame
 frame "<!--frame1-->" - didFinishDocumentLoadForFrame
 frame "<!--frame1-->" - didHandleOnloadEventsForFrame
 frame "<!--frame1-->" - didFinishLoadForFrame
+main frame - didFinishLoadForFrame
 This test loads a secure iframe that loads an insecure stylesheet. We should trigger a mixed content block even though the child frame has a report only CSP block-all-mixed-content directive because an active network attacker can use CSS3 to breach the confidentiality of the HTTPS security origin.
 
 
index 3b4335a..4593a44 100644 (file)
@@ -1,3 +1,58 @@
+2019-04-26  Youenn Fablet  <youenn@apple.com>
+
+        Use normal loading path for ping loads
+        https://bugs.webkit.org/show_bug.cgi?id=196807
+
+        Reviewed by Alex Christensen.
+
+        Make use of regular code path for ping loads and beacon.
+        This is done conditionally on KeepAlive flag.
+        The benefits are a single loading code path and service worker interception.
+
+        For that purpose, introduce a LoaderStrategy switch based on KeepAlive runtime flag.
+        This switch is used to use ping loads when keepAlive is set or regular loads.
+        In case of regular loads, the keepAlive flag should be used to extend the lifetime of the load.
+
+        Migrate ping loads to use CachedResourceLoader instead of PingLoad.
+        For that purpose, introduce a new Ping CachedResource type.
+
+        Covered by existing tests.
+
+        * Modules/beacon/NavigatorBeacon.cpp:
+        (WebCore::NavigatorBeacon::sendBeacon):
+        * inspector/agents/InspectorPageAgent.cpp:
+        (WebCore::InspectorPageAgent::inspectorResourceType):
+        * loader/LinkLoader.cpp:
+        (WebCore::createLinkPreloadResourceClient):
+        * loader/LoaderStrategy.h:
+        * loader/PingLoader.cpp:
+        (WebCore::PingLoader::loadImage):
+        (WebCore::PingLoader::sendPing):
+        (WebCore::PingLoader::sendViolationReport):
+        (WebCore::PingLoader::startPingLoad):
+        * loader/PingLoader.h:
+        * loader/ResourceLoadInfo.cpp:
+        (WebCore::toResourceType):
+        * loader/SubresourceLoader.cpp:
+        (WebCore::logResourceLoaded):
+        * loader/cache/CachedResource.cpp:
+        (WebCore::CachedResource::defaultPriorityForResourceType):
+        (WebCore::CachedResource::load):
+        (WebCore::CachedResource::cancelLoad):
+        * loader/cache/CachedResource.h:
+        (WebCore::CachedResource::shouldUsePingLoad):
+        (WebCore::CachedResource::isMainOrMediaOrIconOrRawResource const):
+        * loader/cache/CachedResourceLoader.cpp:
+        (WebCore::createResource):
+        (WebCore::CachedResourceLoader::requestPingResource):
+        (WebCore::contentTypeFromResourceType):
+        (WebCore::CachedResourceLoader::checkInsecureContent const):
+        (WebCore::CachedResourceLoader::allowedByContentSecurityPolicy const):
+        (WebCore::CachedResourceLoader::canRequest):
+        (WebCore::isResourceSuitableForDirectReuse):
+        (WebCore::destinationForType):
+        * loader/cache/CachedResourceLoader.h:
+
 2019-04-26  Alex Christensen  <achristensen@webkit.org>
 
         Fix Windows build after r244695
index cf1e3fc..007f721 100644 (file)
@@ -122,10 +122,12 @@ ExceptionOr<bool> NavigatorBeacon::sendBeacon(Document& document, const String&
     ResourceRequest request(parsedUrl);
     request.setHTTPMethod("POST"_s);
 
-    FetchOptions options;
+    ResourceLoaderOptions options;
     options.credentials = FetchOptions::Credentials::Include;
     options.cache = FetchOptions::Cache::NoCache;
     options.keepAlive = true;
+    options.sendLoadCallbacks = SendCallbackPolicy::SendCallbacks;
+
     if (body) {
         options.mode = FetchOptions::Mode::Cors;
         String mimeType;
index dbdfa36..f44f0fe 100644 (file)
@@ -280,6 +280,8 @@ InspectorPageAgent::ResourceType InspectorPageAgent::inspectorResourceType(Cache
     case CachedResource::Type::ApplicationManifest:
         return InspectorPageAgent::ApplicationManifestResource;
 #endif
+    case CachedResource::Type::Ping:
+        return InspectorPageAgent::PingResource;
     case CachedResource::Type::MediaResource:
     case CachedResource::Type::Icon:
     case CachedResource::Type::RawResource:
index 9f55ae8..91861e1 100644 (file)
@@ -163,6 +163,7 @@ static std::unique_ptr<LinkPreloadResourceClient> createLinkPreloadResourceClien
     case CachedResource::Type::XSLStyleSheet:
 #endif
     case CachedResource::Type::Beacon:
+    case CachedResource::Type::Ping:
     case CachedResource::Type::LinkPrefetch:
 #if ENABLE(APPLICATION_MANIFEST)
     case CachedResource::Type::ApplicationManifest:
index b35c6fb..8a82381 100644 (file)
@@ -67,6 +67,7 @@ public:
     virtual void suspendPendingRequests() = 0;
     virtual void resumePendingRequests() = 0;
 
+    virtual bool usePingLoad() const { return true; }
     using PingLoadCompletionHandler = WTF::Function<void(const ResourceError&, const ResourceResponse&)>;
     virtual void startPingLoad(Frame&, ResourceRequest&, const HTTPHeaderMap& originalRequestHeaders, const FetchOptions&, ContentSecurityPolicyImposition, PingLoadCompletionHandler&& = { }) = 0;
 
index 83ab359..67768be 100644 (file)
@@ -34,6 +34,8 @@
 #include "config.h"
 #include "PingLoader.h"
 
+#include "CachedResourceLoader.h"
+#include "CachedResourceRequest.h"
 #include "ContentRuleListResults.h"
 #include "ContentSecurityPolicy.h"
 #include "Document.h"
@@ -104,7 +106,7 @@ void PingLoader::loadImage(Frame& frame, const URL& url)
         request.setHTTPReferrer(referrer);
     frame.loader().addExtraFieldsToSubresourceRequest(request);
 
-    startPingLoad(frame, request, WTFMove(originalRequestHeader), ShouldFollowRedirects::Yes, ContentSecurityPolicyImposition::DoPolicyCheck);
+    startPingLoad(frame, request, WTFMove(originalRequestHeader), ShouldFollowRedirects::Yes, ContentSecurityPolicyImposition::DoPolicyCheck, ReferrerPolicy::EmptyString);
 }
 
 // http://www.whatwg.org/specs/web-apps/current-work/multipage/links.html#hyperlink-auditing
@@ -145,7 +147,7 @@ void PingLoader::sendPing(Frame& frame, const URL& pingURL, const URL& destinati
         }
     }
 
-    startPingLoad(frame, request, WTFMove(originalRequestHeader), ShouldFollowRedirects::Yes, ContentSecurityPolicyImposition::DoPolicyCheck);
+    startPingLoad(frame, request, WTFMove(originalRequestHeader), ShouldFollowRedirects::Yes, ContentSecurityPolicyImposition::DoPolicyCheck, request.httpReferrer().isEmpty() ? ReferrerPolicy::NoReferrer : ReferrerPolicy::UnsafeUrl);
 }
 
 void PingLoader::sendViolationReport(Frame& frame, const URL& reportURL, Ref<FormData>&& report, ViolationReportType reportType)
@@ -186,10 +188,10 @@ void PingLoader::sendViolationReport(Frame& frame, const URL& reportURL, Ref<For
     if (!referrer.isEmpty())
         request.setHTTPReferrer(referrer);
 
-    startPingLoad(frame, request, WTFMove(originalRequestHeader), ShouldFollowRedirects::No, ContentSecurityPolicyImposition::SkipPolicyCheck);
+    startPingLoad(frame, request, WTFMove(originalRequestHeader), ShouldFollowRedirects::No, ContentSecurityPolicyImposition::SkipPolicyCheck, ReferrerPolicy::EmptyString);
 }
 
-void PingLoader::startPingLoad(Frame& frame, ResourceRequest& request, HTTPHeaderMap&& originalRequestHeaders, ShouldFollowRedirects shouldFollowRedirects, ContentSecurityPolicyImposition policyCheck)
+void PingLoader::startPingLoad(Frame& frame, ResourceRequest& request, HTTPHeaderMap&& originalRequestHeaders, ShouldFollowRedirects shouldFollowRedirects, ContentSecurityPolicyImposition policyCheck, ReferrerPolicy referrerPolicy)
 {
     unsigned long identifier = frame.page()->progress().createUniqueIdentifier();
     // FIXME: Why activeDocumentLoader? I would have expected documentLoader().
@@ -198,22 +200,33 @@ void PingLoader::startPingLoad(Frame& frame, ResourceRequest& request, HTTPHeade
     // with the provisional DocumentLoader if there is a provisional
     // DocumentLoader.
     bool shouldUseCredentialStorage = frame.loader().client().shouldUseCredentialStorage(frame.loader().activeDocumentLoader(), identifier);
-    FetchOptions options;
+    ResourceLoaderOptions options;
     options.credentials = shouldUseCredentialStorage ? FetchOptions::Credentials::Include : FetchOptions::Credentials::Omit;
     options.redirect = shouldFollowRedirects == ShouldFollowRedirects::Yes ? FetchOptions::Redirect::Follow : FetchOptions::Redirect::Error;
+    options.keepAlive = true;
+    options.contentSecurityPolicyImposition = policyCheck;
+    options.referrerPolicy = referrerPolicy;
+    options.sendLoadCallbacks = SendCallbackPolicy::SendCallbacks;
+    options.cache = FetchOptions::Cache::NoCache;
+
+    // FIXME: Deprecate the ping load code path.
+    if (platformStrategies()->loaderStrategy()->usePingLoad()) {
+        InspectorInstrumentation::willSendRequestOfType(&frame, identifier, frame.loader().activeDocumentLoader(), request, InspectorInstrumentation::LoadType::Ping);
+
+        platformStrategies()->loaderStrategy()->startPingLoad(frame, request, WTFMove(originalRequestHeaders), options, policyCheck, [protectedFrame = makeRef(frame), identifier] (const ResourceError& error, const ResourceResponse& response) {
+            if (!response.isNull())
+                InspectorInstrumentation::didReceiveResourceResponse(protectedFrame, identifier, protectedFrame->loader().activeDocumentLoader(), response, nullptr);
+            if (!error.isNull()) {
+                InspectorInstrumentation::didFailLoading(protectedFrame.ptr(), protectedFrame->loader().activeDocumentLoader(), identifier, error);
+                return;
+            }
+            InspectorInstrumentation::didFinishLoading(protectedFrame.ptr(), protectedFrame->loader().activeDocumentLoader(), identifier, { }, nullptr);
+        });
+        return;
+    }
 
-    // FIXME: Move ping loads to normal subresource loading to get normal inspector request instrumentation hooks.
-    InspectorInstrumentation::willSendRequestOfType(&frame, identifier, frame.loader().activeDocumentLoader(), request, InspectorInstrumentation::LoadType::Ping);
-
-    platformStrategies()->loaderStrategy()->startPingLoad(frame, request, WTFMove(originalRequestHeaders), options, policyCheck, [protectedFrame = makeRef(frame), identifier] (const ResourceError& error, const ResourceResponse& response) {
-        if (!response.isNull())
-            InspectorInstrumentation::didReceiveResourceResponse(protectedFrame, identifier, protectedFrame->loader().activeDocumentLoader(), response, nullptr);
-        if (error.isNull()) {
-            NetworkLoadMetrics emptyMetrics;
-            InspectorInstrumentation::didFinishLoading(protectedFrame.ptr(), protectedFrame->loader().activeDocumentLoader(), identifier, emptyMetrics, nullptr);
-        } else
-            InspectorInstrumentation::didFailLoading(protectedFrame.ptr(), protectedFrame->loader().activeDocumentLoader(), identifier, error);
-    });
+    CachedResourceRequest cachedResourceRequest { ResourceRequest { request }, options };
+    frame.document()->cachedResourceLoader().requestPingResource(WTFMove(cachedResourceRequest));
 }
 
 }
index 8d82d71..5b4d6f2 100644 (file)
@@ -32,6 +32,7 @@
 
 #pragma once
 
+#include "ReferrerPolicy.h"
 #include <wtf/Forward.h>
 #include <wtf/Ref.h>
 
@@ -57,7 +58,7 @@ public:
 
 private:
     enum class ShouldFollowRedirects { No, Yes };
-    static void startPingLoad(Frame&, ResourceRequest&, HTTPHeaderMap&& originalRequestHeaders, ShouldFollowRedirects, ContentSecurityPolicyImposition);
+    static void startPingLoad(Frame&, ResourceRequest&, HTTPHeaderMap&& originalRequestHeaders, ShouldFollowRedirects, ContentSecurityPolicyImposition, ReferrerPolicy);
 };
 
 } // namespace WebCore
index 7783368..bf023b3 100644 (file)
@@ -62,6 +62,7 @@ ResourceType toResourceType(CachedResource::Type type)
         return ResourceType::Media;
 
     case CachedResource::Type::Beacon:
+    case CachedResource::Type::Ping:
     case CachedResource::Type::Icon:
     case CachedResource::Type::RawResource:
         return ResourceType::Raw;
index 976d805..36b06bb 100644 (file)
@@ -518,8 +518,7 @@ static void logResourceLoaded(Frame* frame, CachedResource::Type type)
         resourceType = DiagnosticLoggingKeys::fontKey();
         break;
     case CachedResource::Type::Beacon:
-        ASSERT_NOT_REACHED();
-        break;
+    case CachedResource::Type::Ping:
     case CachedResource::Type::MediaResource:
     case CachedResource::Type::Icon:
     case CachedResource::Type::RawResource:
@@ -687,10 +686,9 @@ void SubresourceLoader::didFail(const ResourceError& error)
     ASSERT(!reachedTerminalState());
     LOG(ResourceLoading, "Failed to load '%s'.\n", m_resource->url().string().latin1().data());
 
-    if (m_frame->document() && error.isAccessControl())
+    if (m_frame->document() && error.isAccessControl() && m_resource->type() != CachedResource::Type::Ping)
         m_frame->document()->addConsoleMessage(MessageSource::Security, MessageLevel::Error, error.localizedDescription());
 
-
     Ref<SubresourceLoader> protectedThis(*this);
     CachedResourceHandle<CachedResource> protectResource(m_resource);
     m_state = Finishing;
index d29e4ad..e430e44 100644 (file)
@@ -89,6 +89,7 @@ ResourceLoadPriority CachedResource::defaultPriorityForResourceType(Type type)
     case Type::SVGDocumentResource:
         return ResourceLoadPriority::Low;
     case Type::Beacon:
+    case Type::Ping:
         return ResourceLoadPriority::VeryLow;
     case Type::LinkPrefetch:
         return ResourceLoadPriority::VeryLow;
@@ -271,36 +272,34 @@ void CachedResource::load(CachedResourceLoader& cachedResourceLoader)
         m_fragmentIdentifierForRequest = String();
     }
 
-    if (m_options.keepAlive) {
-        if (!cachedResourceLoader.keepaliveRequestTracker().tryRegisterRequest(*this)) {
-            setResourceError({ errorDomainWebKitInternal, 0, request.url(), "Reached maximum amount of queued data of 64Kb for keepalive requests"_s, ResourceError::Type::AccessControl });
-            failBeforeStarting();
-            return;
-        }
-        // FIXME: We should not special-case Beacon here.
-        if (shouldUsePingLoad(type())) {
-            ASSERT(m_originalRequest);
-            CachedResourceHandle<CachedResource> protectedThis(this);
-
-            // FIXME: Move beacon loads to normal subresource loading to get normal inspector request instrumentation hooks.
-            unsigned long identifier = frame.page()->progress().createUniqueIdentifier();
-            InspectorInstrumentation::willSendRequestOfType(&frame, identifier, frameLoader.activeDocumentLoader(), request, InspectorInstrumentation::LoadType::Beacon);
-
-            platformStrategies()->loaderStrategy()->startPingLoad(frame, request, m_originalRequest->httpHeaderFields(), m_options, m_options.contentSecurityPolicyImposition, [this, protectedThis = WTFMove(protectedThis), protectedFrame = makeRef(frame), identifier] (const ResourceError& error, const ResourceResponse& response) {
-                if (!response.isNull())
-                    InspectorInstrumentation::didReceiveResourceResponse(protectedFrame, identifier, protectedFrame->loader().activeDocumentLoader(), response, nullptr);
-                if (error.isNull()) {
-                    finishLoading(nullptr);
-                    NetworkLoadMetrics emptyMetrics;
-                    InspectorInstrumentation::didFinishLoading(protectedFrame.ptr(), protectedFrame->loader().activeDocumentLoader(), identifier, emptyMetrics, nullptr);
-                } else {
-                    setResourceError(error);
-                    this->error(LoadError);
-                    InspectorInstrumentation::didFailLoading(protectedFrame.ptr(), protectedFrame->loader().activeDocumentLoader(), identifier, error);
-                }
-            });
-            return;
-        }
+    if (m_options.keepAlive && type() != Type::Ping && !cachedResourceLoader.keepaliveRequestTracker().tryRegisterRequest(*this)) {
+        setResourceError({ errorDomainWebKitInternal, 0, request.url(), "Reached maximum amount of queued data of 64Kb for keepalive requests"_s, ResourceError::Type::AccessControl });
+        failBeforeStarting();
+        return;
+    }
+
+    // FIXME: Deprecate that code path.
+    if (m_options.keepAlive && shouldUsePingLoad(type()) && platformStrategies()->loaderStrategy()->usePingLoad()) {
+        ASSERT(m_originalRequest);
+        CachedResourceHandle<CachedResource> protectedThis(this);
+
+        unsigned long identifier = frame.page()->progress().createUniqueIdentifier();
+        InspectorInstrumentation::willSendRequestOfType(&frame, identifier, frameLoader.activeDocumentLoader(), request, InspectorInstrumentation::LoadType::Beacon);
+
+        platformStrategies()->loaderStrategy()->startPingLoad(frame, request, m_originalRequest->httpHeaderFields(), m_options, m_options.contentSecurityPolicyImposition, [this, protectedThis = WTFMove(protectedThis), protectedFrame = makeRef(frame), identifier] (const ResourceError& error, const ResourceResponse& response) {
+            if (!response.isNull())
+                InspectorInstrumentation::didReceiveResourceResponse(protectedFrame, identifier, protectedFrame->loader().activeDocumentLoader(), response, nullptr);
+            if (!error.isNull()) {
+                setResourceError(error);
+                this->error(LoadError);
+                InspectorInstrumentation::didFailLoading(protectedFrame.ptr(), protectedFrame->loader().activeDocumentLoader(), identifier, error);
+                return;
+            }
+            finishLoading(nullptr);
+            NetworkLoadMetrics emptyMetrics;
+            InspectorInstrumentation::didFinishLoading(protectedFrame.ptr(), protectedFrame->loader().activeDocumentLoader(), identifier, emptyMetrics, nullptr);
+        });
+        return;
     }
 
     platformStrategies()->loaderStrategy()->loadResource(frame, *this, WTFMove(request), m_options, [this, protectedThis = CachedResourceHandle<CachedResource>(this), frame = makeRef(frame), loggingAllowed = cachedResourceLoader.isAlwaysOnLoggingAllowed()] (RefPtr<SubresourceLoader>&& loader) {
@@ -384,7 +383,12 @@ void CachedResource::cancelLoad()
     if (!isLoading() && !stillNeedsLoad())
         return;
 
-    setStatus(LoadError);
+    auto* documentLoader = (m_loader && m_loader->frame()) ? m_loader->frame()->loader().activeDocumentLoader() : nullptr;
+    if (m_options.keepAlive && (!documentLoader || documentLoader->isStopping()))
+        m_error = { };
+    else
+        setStatus(LoadError);
+
     setLoading(false);
     checkNotify();
 }
index aa8530d..9bf51e0 100644 (file)
@@ -74,6 +74,7 @@ public:
         RawResource,
         Icon,
         Beacon,
+        Ping,
         SVGDocumentResource
 #if ENABLE(XSLT)
         , XSLStyleSheet
@@ -122,7 +123,7 @@ public:
     String mimeType() const { return m_response.mimeType(); }
     long long expectedContentLength() const { return m_response.expectedContentLength(); }
 
-    static bool shouldUsePingLoad(Type type) { return type == Type::Beacon; }
+    static bool shouldUsePingLoad(Type type) { return type == Type::Beacon || type == Type::Ping; }
 
     ResourceLoadPriority loadPriority() const { return m_loadPriority; }
     void setLoadPriority(const Optional<ResourceLoadPriority>&);
@@ -168,7 +169,7 @@ public:
 
     bool isImage() const { return type() == Type::ImageResource; }
     // FIXME: CachedRawResource could be a main resource, an audio/video resource, or a raw XHR/icon resource.
-    bool isMainOrMediaOrIconOrRawResource() const { return type() == Type::MainResource || type() == Type::MediaResource || type() == Type::Icon || type() == Type::RawResource || type() == Type::Beacon; }
+    bool isMainOrMediaOrIconOrRawResource() const { return type() == Type::MainResource || type() == Type::MediaResource || type() == Type::Icon || type() == Type::RawResource || type() == Type::Beacon || type() == Type::Ping; }
 
     // Whether this request should impact request counting and delay window.onload.
     bool ignoreForRequestCount() const
@@ -176,6 +177,8 @@ public:
         return m_ignoreForRequestCount
             || type() == Type::MainResource
             || type() == Type::LinkPrefetch
+            || type() == Type::Beacon
+            || type() == Type::Ping
             || type() == Type::Icon
             || type() == Type::RawResource;
     }
index d0de65f..5b0d4e0 100644 (file)
@@ -120,6 +120,7 @@ static CachedResource* createResource(CachedResource::Type type, CachedResourceR
     case CachedResource::Type::FontResource:
         return new CachedFont(WTFMove(request), sessionID, cookieJar);
     case CachedResource::Type::Beacon:
+    case CachedResource::Type::Ping:
     case CachedResource::Type::MediaResource:
     case CachedResource::Type::RawResource:
     case CachedResource::Type::Icon:
@@ -308,6 +309,12 @@ ResourceErrorOr<CachedResourceHandle<CachedRawResource>> CachedResourceLoader::r
     return castCachedResourceTo<CachedRawResource>(requestResource(CachedResource::Type::Beacon, WTFMove(request)));
 }
 
+ResourceErrorOr<CachedResourceHandle<CachedRawResource>> CachedResourceLoader::requestPingResource(CachedResourceRequest&& request)
+{
+    ASSERT(request.options().destination == FetchOptions::Destination::EmptyString);
+    return castCachedResourceTo<CachedRawResource>(requestResource(CachedResource::Type::Ping, WTFMove(request)));
+}
+
 ResourceErrorOr<CachedResourceHandle<CachedRawResource>> CachedResourceLoader::requestMainResource(CachedResourceRequest&& request)
 {
     return castCachedResourceTo<CachedRawResource>(requestResource(CachedResource::Type::MainResource, WTFMove(request)));
@@ -341,6 +348,7 @@ static MixedContentChecker::ContentType contentTypeFromResourceType(CachedResour
 #endif
 
     case CachedResource::Type::Beacon:
+    case CachedResource::Type::Ping:
     case CachedResource::Type::RawResource:
     case CachedResource::Type::Icon:
     case CachedResource::Type::SVGDocumentResource:
@@ -412,6 +420,7 @@ bool CachedResourceLoader::checkInsecureContent(CachedResource::Type type, const
     }
     case CachedResource::Type::MainResource:
     case CachedResource::Type::Beacon:
+    case CachedResource::Type::Ping:
     case CachedResource::Type::LinkPrefetch:
         // Prefetch cannot affect the current document.
 #if ENABLE(APPLICATION_MANIFEST)
@@ -463,6 +472,7 @@ bool CachedResourceLoader::allowedByContentSecurityPolicy(CachedResource::Type t
             return false;
         break;
     case CachedResource::Type::Beacon:
+    case CachedResource::Type::Ping:
     case CachedResource::Type::RawResource:
         return true;
 #if ENABLE(APPLICATION_MANIFEST)
@@ -501,7 +511,7 @@ bool CachedResourceLoader::canRequest(CachedResource::Type type, const URL& url,
         return false;
     }
 
-    if (options.mode == FetchOptions::Mode::NoCors && options.redirect != FetchOptions::Redirect::Follow) {
+    if (options.mode == FetchOptions::Mode::NoCors && options.redirect != FetchOptions::Redirect::Follow && type != CachedResource::Type::Ping) {
         ASSERT(type != CachedResource::Type::MainResource);
         frame()->document()->addConsoleMessage(MessageSource::Security, MessageLevel::Error, "No-Cors mode requires follow redirect mode"_s);
         return false;
@@ -669,7 +679,7 @@ static inline bool isResourceSuitableForDirectReuse(const CachedResource& resour
     if (resource.type() == CachedResource::Type::RawResource || resource.type() == CachedResource::Type::MediaResource)
         return false;
 
-    if (resource.type() == CachedResource::Type::Beacon)
+    if (resource.type() == CachedResource::Type::Beacon || resource.type() == CachedResource::Type::Ping)
         return false;
 
     return true;
@@ -762,6 +772,7 @@ static FetchOptions::Destination destinationForType(CachedResource::Type type)
         return FetchOptions::Destination::Manifest;
 #endif
     case CachedResource::Type::Beacon:
+    case CachedResource::Type::Ping:
     case CachedResource::Type::LinkPrefetch:
     case CachedResource::Type::RawResource:
     case CachedResource::Type::MediaResource:
index 0f6d600..4e9c1a0 100644 (file)
@@ -87,6 +87,7 @@ public:
     ResourceErrorOr<CachedResourceHandle<CachedRawResource>> requestMedia(CachedResourceRequest&&);
     ResourceErrorOr<CachedResourceHandle<CachedRawResource>> requestIcon(CachedResourceRequest&&);
     ResourceErrorOr<CachedResourceHandle<CachedRawResource>> requestBeaconResource(CachedResourceRequest&&);
+    ResourceErrorOr<CachedResourceHandle<CachedRawResource>> requestPingResource(CachedResourceRequest&&);
     ResourceErrorOr<CachedResourceHandle<CachedRawResource>> requestMainResource(CachedResourceRequest&&);
     ResourceErrorOr<CachedResourceHandle<CachedSVGDocument>> requestSVGDocument(CachedResourceRequest&&);
 #if ENABLE(XSLT)
index 46847e3..f10cb92 100644 (file)
@@ -1,3 +1,40 @@
+2019-04-26  Youenn Fablet  <youenn@apple.com>
+
+        Use normal loading path for ping loads
+        https://bugs.webkit.org/show_bug.cgi?id=196807
+
+        Reviewed by Alex Christensen.
+
+        In case a NetworkResourceLoader has the keepAlive option we do the following:
+        - Always use NetworkLoadChecker as we might need it to do checks after the Web context is gone.
+        - In case of aborting a KeepAlive loader, remove it from NetworkConnectionToWebProcess map
+        and add it to a kept-alive NetworkSession load set. The loader is only kept alive if it
+        has not yet received a response. Mark the loader as kept-alive.
+        - In case loader is kept-alive, cancel the load as soon as a response is gathered.
+
+        * NetworkProcess/NetworkConnectionToWebProcess.cpp:
+        (WebKit::NetworkConnectionToWebProcess::transferKeptAliveLoad):
+        * NetworkProcess/NetworkConnectionToWebProcess.h:
+        * NetworkProcess/NetworkProcess.cpp:
+        (WebKit::NetworkProcess::addKeptAliveLoad):
+        (WebKit::NetworkProcess::removeKeptAliveLoad):
+        * NetworkProcess/NetworkProcess.h:
+        * NetworkProcess/NetworkResourceLoadMap.cpp:
+        (WebKit::NetworkResourceLoadMap::remove):
+        (WebKit::NetworkResourceLoadMap::take):
+        * NetworkProcess/NetworkResourceLoadMap.h:
+        * NetworkProcess/NetworkResourceLoader.cpp:
+        (WebKit::m_shouldCaptureExtraNetworkLoadMetrics):
+        (WebKit::NetworkResourceLoader::cleanup):
+        (WebKit::NetworkResourceLoader::abort):
+        (WebKit::NetworkResourceLoader::didReceiveResponse):
+        (WebKit::NetworkResourceLoader::continueWillSendRedirectedRequest):
+        * NetworkProcess/NetworkResourceLoader.h:
+        * WebProcess/Network/WebLoaderStrategy.cpp:
+        (WebKit::maximumBufferingTime):
+        (WebKit::WebLoaderStrategy::usePingLoad const):
+        * WebProcess/Network/WebLoaderStrategy.h:
+
 2019-04-26  Alex Christensen  <achristensen@webkit.org>
 
         Fix internal High Sierra build after r244653
index cab434e..dc74a61 100644 (file)
@@ -111,11 +111,25 @@ void NetworkConnectionToWebProcess::didCleanupResourceLoader(NetworkResourceLoad
 {
     RELEASE_ASSERT(loader.identifier());
     RELEASE_ASSERT(RunLoop::isMain());
-    ASSERT(m_networkResourceLoaders.get(loader.identifier()) == &loader);
 
+    if (loader.isKeptAlive()) {
+        networkProcess().removeKeptAliveLoad(loader);
+        return;
+    }
+
+    ASSERT(m_networkResourceLoaders.get(loader.identifier()) == &loader);
     m_networkResourceLoaders.remove(loader.identifier());
 }
 
+void NetworkConnectionToWebProcess::transferKeptAliveLoad(NetworkResourceLoader& loader)
+{
+    RELEASE_ASSERT(RunLoop::isMain());
+    ASSERT(loader.isKeptAlive());
+    ASSERT(m_networkResourceLoaders.get(loader.identifier()) == &loader);
+    if (auto takenLoader = m_networkResourceLoaders.take(loader.identifier()))
+        m_networkProcess->addKeptAliveLoad(takenLoader.releaseNonNull());
+}
+
 void NetworkConnectionToWebProcess::didReceiveMessage(IPC::Connection& connection, IPC::Decoder& decoder)
 {
     if (decoder.messageReceiverName() == Messages::NetworkConnectionToWebProcess::messageReceiverName()) {
index 98cf447..c69e325 100644 (file)
@@ -82,6 +82,7 @@ public:
     NetworkProcess& networkProcess() { return m_networkProcess.get(); }
 
     void didCleanupResourceLoader(NetworkResourceLoader&);
+    void transferKeptAliveLoad(NetworkResourceLoader&);
     void setOnLineState(bool);
 
     bool captureExtraNetworkLoadMetricsEnabled() const { return m_captureExtraNetworkLoadMetricsEnabled; }
index 5f73208..69226f0 100644 (file)
@@ -2574,4 +2574,16 @@ void NetworkProcess::markAdClickAttributionsAsExpiredForTesting(PAL::SessionID s
     completionHandler();
 }
 
+void NetworkProcess::addKeptAliveLoad(Ref<NetworkResourceLoader>&& loader)
+{
+    if (auto session = m_networkSessions.get(loader->sessionID()))
+        session->addKeptAliveLoad(WTFMove(loader));
+}
+
+void NetworkProcess::removeKeptAliveLoad(NetworkResourceLoader& loader)
+{
+    if (auto session = m_networkSessions.get(loader.sessionID()))
+        session->removeKeptAliveLoad(loader);
+}
+
 } // namespace WebKit
index c0449a1..d46443d 100644 (file)
@@ -92,6 +92,7 @@ class AuthenticationManager;
 class NetworkConnectionToWebProcess;
 class NetworkProcessSupplement;
 class NetworkProximityManager;
+class NetworkResourceLoader;
 class WebSWServerConnection;
 class WebSWServerToContextConnection;
 enum class ShouldGrandfatherStatistics : bool;
@@ -334,6 +335,9 @@ public:
 
     WebCore::StorageQuotaManager& storageQuotaManager(PAL::SessionID, const WebCore::ClientOrigin&);
 
+    void addKeptAliveLoad(Ref<NetworkResourceLoader>&&);
+    void removeKeptAliveLoad(NetworkResourceLoader&);
+
 private:
     void platformInitializeNetworkProcess(const NetworkProcessCreationParameters&);
     std::unique_ptr<WebCore::NetworkStorageSession> platformCreateDefaultStorageSession() const;
index b0868f1..e3332ce 100644 (file)
@@ -46,9 +46,14 @@ NetworkResourceLoadMap::MapType::AddResult NetworkResourceLoadMap::add(ResourceL
 
 bool NetworkResourceLoadMap::remove(ResourceLoadIdentifier identifier)
 {
+    return !!take(identifier);
+}
+
+RefPtr<NetworkResourceLoader> NetworkResourceLoadMap::take(ResourceLoadIdentifier identifier)
+{
     auto loader = m_loaders.take(identifier);
     if (!loader)
-        return false;
+        return nullptr;
 
     if ((*loader)->originalRequest().hasUpload()) {
         m_loadersWithUploads.remove(loader->ptr());
@@ -56,7 +61,7 @@ bool NetworkResourceLoadMap::remove(ResourceLoadIdentifier identifier)
             m_connectionToWebProcess.clearConnectionHasUploads();
     }
 
-    return true;
+    return WTFMove(*loader);
 }
 
 NetworkResourceLoader* NetworkResourceLoadMap::get(ResourceLoadIdentifier identifier) const
index e38cbdc..6d4079c 100644 (file)
@@ -56,6 +56,7 @@ public:
     MapType::AddResult add(ResourceLoadIdentifier, Ref<NetworkResourceLoader>&&);
     NetworkResourceLoader* get(ResourceLoadIdentifier) const;
     bool remove(ResourceLoadIdentifier);
+    RefPtr<NetworkResourceLoader> take(ResourceLoadIdentifier);
 
 private:
     NetworkConnectionToWebProcess& m_connectionToWebProcess;
index 1742c12..048f077 100644 (file)
@@ -106,7 +106,7 @@ NetworkResourceLoader::NetworkResourceLoader(NetworkResourceLoadParameters&& par
     //        Once bug 116233 is resolved, this ASSERT can just be "m_webPageID && m_webFrameID"
     ASSERT((m_parameters.webPageID && m_parameters.webFrameID) || m_parameters.clientCredentialPolicy == ClientCredentialPolicy::CannotAskClientForCredentials);
 
-    if (synchronousReply || parameters.shouldRestrictHTTPResponseAccess) {
+    if (synchronousReply || parameters.shouldRestrictHTTPResponseAccess || parameters.options.keepAlive) {
         NetworkLoadChecker::LoadType requestLoadType = isMainFrameLoad() ? NetworkLoadChecker::LoadType::MainFrame : NetworkLoadChecker::LoadType::Other;
         m_networkLoadChecker = std::make_unique<NetworkLoadChecker>(connection.networkProcess(), FetchOptions { m_parameters.options }, m_parameters.sessionID, m_parameters.webPageID, m_parameters.webFrameID, HTTPHeaderMap { m_parameters.originalRequestHeaders }, URL { m_parameters.request.url() }, m_parameters.sourceOrigin.copyRef(), m_parameters.preflightPolicy, originalRequest().httpReferrer(), m_parameters.isHTTPSUpgradeEnabled, shouldCaptureExtraNetworkLoadMetrics(), requestLoadType);
         if (m_parameters.cspResponseHeaders)
@@ -333,6 +333,12 @@ void NetworkResourceLoader::abort()
     RELEASE_LOG_IF_ALLOWED("abort: Canceling resource load (pageID = %" PRIu64 ", frameID = %" PRIu64 ", resourceID = %" PRIu64 ")",
         m_parameters.webPageID, m_parameters.webFrameID, m_parameters.identifier);
 
+    if (m_parameters.options.keepAlive && m_response.isNull() && !m_isKeptAlive) {
+        m_isKeptAlive = true;
+        m_connection->transferKeptAliveLoad(*this);
+        return;
+    }
+
     if (m_networkLoad) {
         if (canUseCache(m_networkLoad->currentRequest())) {
             // We might already have used data from this incomplete load. Ensure older versions don't remain in the cache after cancel.
@@ -467,10 +473,21 @@ void NetworkResourceLoader::didReceiveResponse(ResourceResponse&& receivedRespon
     // a main resource because the embedding client must decide whether to allow the load.
     bool willWaitForContinueDidReceiveResponse = isMainResource();
     send(Messages::WebResourceLoader::DidReceiveResponse { response, willWaitForContinueDidReceiveResponse });
-    if (willWaitForContinueDidReceiveResponse)
+
+    if (willWaitForContinueDidReceiveResponse) {
         m_responseCompletionHandler = WTFMove(completionHandler);
-    else
-        completionHandler(PolicyAction::Use);
+        return;
+    }
+
+    if (m_isKeptAlive) {
+        m_responseCompletionHandler = WTFMove(completionHandler);
+        RunLoop::main().dispatch([protectedThis = makeRef(*this)] {
+            protectedThis->didFinishLoading(NetworkLoadMetrics { });
+        });
+        return;
+    }
+
+    completionHandler(PolicyAction::Use);
 }
 
 void NetworkResourceLoader::didReceiveBuffer(Ref<SharedBuffer>&& buffer, int reportedEncodedDataLength)
@@ -650,6 +667,11 @@ void NetworkResourceLoader::continueWillSendRedirectedRequest(ResourceRequest&&
 {
     ASSERT(!isSynchronous());
 
+    if (m_isKeptAlive) {
+        continueWillSendRequest(WTFMove(request), false);
+        return;
+    }
+
     if (adClickConversion)
         handleAdClickAttributionConversion(WTFMove(*adClickConversion), request.url(), redirectRequest);
     send(Messages::WebResourceLoader::WillSendRequest(redirectRequest, sanitizeResponseIfPossible(WTFMove(redirectResponse), ResourceResponse::SanitizationType::Redirection)));
index 0298511..c7de3c0 100644 (file)
@@ -114,6 +114,8 @@ public:
 
     void disableExtraNetworkLoadMetricsCapture() { m_shouldCaptureExtraNetworkLoadMetrics = false; }
 
+    bool isKeptAlive() const { return m_isKeptAlive; }
+
 private:
     NetworkResourceLoader(NetworkResourceLoadParameters&&, NetworkConnectionToWebProcess&, Messages::NetworkConnectionToWebProcess::PerformSynchronousLoad::DelayedReply&&);
 
@@ -208,6 +210,7 @@ private:
     bool m_shouldRestartLoad { false };
     ResponseCompletionHandler m_responseCompletionHandler;
     bool m_shouldCaptureExtraNetworkLoadMetrics { false };
+    bool m_isKeptAlive { false };
 
     Optional<NetworkActivityTracker> m_networkActivityTracker;
 };
index 4261b6a..5b442bf 100644 (file)
@@ -30,6 +30,7 @@
 #include "NetworkProcess.h"
 #include "NetworkProcessProxyMessages.h"
 #include "NetworkResourceLoadParameters.h"
+#include "NetworkResourceLoader.h"
 #include "PingLoad.h"
 #include "WebPageProxy.h"
 #include "WebPageProxyMessages.h"
@@ -183,4 +184,18 @@ void NetworkSession::markAdClickAttributionsAsExpiredForTesting()
     m_adClickAttribution->markAllUnconvertedAsExpiredForTesting();
 }
 
+void NetworkSession::addKeptAliveLoad(Ref<NetworkResourceLoader>&& loader)
+{
+    ASSERT(m_sessionID == loader->sessionID());
+    ASSERT(!m_keptAliveLoads.contains(loader));
+    m_keptAliveLoads.add(WTFMove(loader));
+}
+
+void NetworkSession::removeKeptAliveLoad(NetworkResourceLoader& loader)
+{
+    ASSERT(m_sessionID == loader.sessionID());
+    ASSERT(m_keptAliveLoads.contains(loader));
+    m_keptAliveLoads.remove(loader);
+}
+
 } // namespace WebKit
index 32e1522..7099db8 100644 (file)
@@ -48,6 +48,7 @@ namespace WebKit {
 class AdClickAttributionManager;
 class NetworkDataTask;
 class NetworkProcess;
+class NetworkResourceLoader;
 class WebResourceLoadStatisticsStore;
 struct NetworkSessionCreationParameters;
 
@@ -88,6 +89,9 @@ public:
     void setAdClickAttributionConversionURLForTesting(URL&&);
     void markAdClickAttributionsAsExpiredForTesting();
 
+    void addKeptAliveLoad(Ref<NetworkResourceLoader>&&);
+    void removeKeptAliveLoad(NetworkResourceLoader&);
+
 protected:
     NetworkSession(NetworkProcess&, PAL::SessionID);
 
@@ -102,6 +106,8 @@ protected:
     WebCore::RegistrableDomain m_resourceLoadStatisticsManualPrevalentResource;
 #endif
     UniqueRef<AdClickAttributionManager> m_adClickAttribution;
+
+    HashSet<Ref<NetworkResourceLoader>> m_keptAliveLoads;
 };
 
 } // namespace WebKit
index 25ccca0..04ae0e4 100644 (file)
@@ -116,6 +116,7 @@ static Seconds maximumBufferingTime(CachedResource* resource)
 
     switch (resource->type()) {
     case CachedResource::Type::Beacon:
+    case CachedResource::Type::Ping:
     case CachedResource::Type::CSSStyleSheet:
     case CachedResource::Type::Script:
 #if ENABLE(SVG_FONTS)
@@ -286,7 +287,7 @@ void WebLoaderStrategy::scheduleLoadFromNetworkProcess(ResourceLoader& resourceL
     auto* document = resourceLoader.frame() ? resourceLoader.frame()->document() : nullptr;
     if (resourceLoader.options().cspResponseHeaders)
         loadParameters.cspResponseHeaders = resourceLoader.options().cspResponseHeaders;
-    else if (document && !document->shouldBypassMainWorldContentSecurityPolicy()) {
+    else if (document && !document->shouldBypassMainWorldContentSecurityPolicy() && resourceLoader.options().contentSecurityPolicyImposition == ContentSecurityPolicyImposition::DoPolicyCheck) {
         if (auto* contentSecurityPolicy = document->contentSecurityPolicy())
             loadParameters.cspResponseHeaders = contentSecurityPolicy->responseHeaders();
     }
@@ -571,6 +572,11 @@ static uint64_t generateLoadIdentifier()
     return ++identifier;
 }
 
+bool WebLoaderStrategy::usePingLoad() const
+{
+    return !RuntimeEnabledFeatures::sharedFeatures().fetchAPIKeepAliveEnabled();
+}
+
 void WebLoaderStrategy::startPingLoad(Frame& frame, ResourceRequest& request, const HTTPHeaderMap& originalRequestHeaders, const FetchOptions& options, ContentSecurityPolicyImposition policyCheck, PingLoadCompletionHandler&& completionHandler)
 {
     auto* document = frame.document();
index 1efc93c..4af5994 100644 (file)
@@ -62,6 +62,7 @@ public:
     void suspendPendingRequests() final;
     void resumePendingRequests() final;
 
+    bool usePingLoad() const final;
     void startPingLoad(WebCore::Frame&, WebCore::ResourceRequest&, const WebCore::HTTPHeaderMap& originalRequestHeaders, const WebCore::FetchOptions&, WebCore::ContentSecurityPolicyImposition, PingLoadCompletionHandler&&) final;
     void didFinishPingLoad(uint64_t pingLoadIdentifier, WebCore::ResourceError&&, WebCore::ResourceResponse&&);