Source/WebCore:
authorkatherine_cheney@apple.com <katherine_cheney@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 14 Feb 2020 03:41:06 +0000 (03:41 +0000)
committerkatherine_cheney@apple.com <katherine_cheney@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 14 Feb 2020 03:41:06 +0000 (03:41 +0000)
Expose prevalent domains on a per-page basis
https://bugs.webkit.org/show_bug.cgi?id=207523
<rdar://problem/59270758>

Reviewed by Chris Dumez.

Tests: http/tests/resourceLoadStatistics/prevalent-domains-per-page-database.html
       http/tests/resourceLoadStatistics/prevalent-domains-per-page.html

Logs subresource loads in CachedResourceLoader::requestResource, and
redirects in ResourceLoader::willSendRequestInternal,
and reports domains to the Web Process.

* loader/FrameLoaderClient.h:
* loader/cache/CachedResourceLoader.cpp:
(WebCore::CachedResourceLoader::requestResource):
* loader/cache/ResourceLoader.cpp:
(WebCore::ResourceLoader::willSendRequestInternal):

Source/WebKit:
Expose prevalent domains on a per-page basis
https://bugs.webkit.org/show_bug.cgi?id=207523
<rdar://problem/59270758>

Reviewed by Chris Dumez.

Tests: http/tests/resourceLoadStatistics/prevalent-domains-per-page-database.html
       http/tests/resourceLoadStatistics/prevalent-domains-per-page.html

Added an API to report prevalent domains. The Web Process sends all
new loads to the Network Process, which determines if the resources are
prevalent. When the API is queried, the Web Process reports these
domains to the UI Process.

The WebPage keeps a hash set of previously loaded domains to avoid
spamming IPC with repeat domains.

* NetworkProcess/NetworkConnectionToWebProcess.cpp:
(WebKit::NetworkConnectionToWebProcess::requestStorageAccessUnderOpener):
(WebKit::NetworkConnectionToWebProcess::reportSubresourceLoadToDomain):
* NetworkProcess/NetworkConnectionToWebProcess.h:
* NetworkProcess/NetworkConnectionToWebProcess.messages.in:
* UIProcess/API/Cocoa/WKWebsiteDataStore.mm:
(-[WKWebsiteDataStore _getPrevalentDomainsFor:completionHandler:]):
(-[WKWebsiteDataStore _clearPrevalentDomainsFor:]):
* UIProcess/API/Cocoa/WKWebsiteDataStorePrivate.h:
* UIProcess/WebPageProxy.cpp:
(WebKit::WebPageProxy::didCommitLoadForFrame):
* UIProcess/WebPageProxy.h:
* WebProcess/WebCoreSupport/WebFrameLoaderClient.cpp:
(WebKit::WebFrameLoaderClient::addLoadedRegistrableDomain):
* WebProcess/WebCoreSupport/WebFrameLoaderClient.h:
* WebProcess/WebPage/WebPage.cpp:
(WebKit::WebPage::setIsSuspended):
(WebKit::WebPage::addLoadedRegistrableDomain):
(WebKit::WebPage::getPrevalentDomains):
(WebKit::WebPage::clearPrevalentDomains):
* WebProcess/WebPage/WebPage.h:
* WebProcess/WebPage/WebPage.messages.in:

Tools:
 Expose prevalent domains on a per-page basis
 https://bugs.webkit.org/show_bug.cgi?id=207523
 <rdar://problem/59270758>

Reviewed by Chris Dumez.

Added WebKitTestRunner functions so this functionality can be tested.
clearPrevalentDomains should be called between tests to reset the
state.

* WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl:
* WebKitTestRunner/InjectedBundle/InjectedBundle.cpp:
(WTR::InjectedBundle::didReceiveMessageToPage):
* WebKitTestRunner/InjectedBundle/TestRunner.cpp:
(WTR::TestRunner::getPrevalentDomains):
(WTR::TestRunner::callDidReceivePrevalentDomainsCallback):
* WebKitTestRunner/InjectedBundle/TestRunner.h:
* WebKitTestRunner/TestController.cpp:
(WTR::TestController::resetStateToConsistentValues):
(WTR::TestController::getPrevalentDomains):
(WTR::TestController::clearPrevalentDomains):
* WebKitTestRunner/TestController.h:
* WebKitTestRunner/TestInvocation.cpp:
(WTR::TestInvocation::didReceiveMessageFromInjectedBundle):
(WTR::TestInvocation::didReceivePrevalentDomains):
* WebKitTestRunner/TestInvocation.h:
* WebKitTestRunner/cocoa/TestControllerCocoa.mm:
(WTR::TestController::getPrevalentDomains):
(WTR::TestController::clearPrevalentDomains):

LayoutTests:
Expose prevalent domains on a per-page basis
https://bugs.webkit.org/show_bug.cgi?id=207523
<rdar://problem/59270758>

Reviewed by Chris Dumez.

Added 2 new tests for the resource load statistics database and
memory stores. Added a basic iframe that both tests can use to test
if a resource load from a prevalent domain gets reported properly.

* http/tests/resourceLoadStatistics/prevalent-domains-per-page-database-expected.txt: Added.
* http/tests/resourceLoadStatistics/prevalent-domains-per-page-database.html: Added.
* http/tests/resourceLoadStatistics/prevalent-domains-per-page-expected.txt: Added.
* http/tests/resourceLoadStatistics/prevalent-domains-per-page.html: Added.
* http/tests/resourceLoadStatistics/resources/basic-iframe.html: Added.

git-svn-id: http://svn.webkit.org/repository/webkit/trunk@256583 268f45cc-cd09-0410-ab3c-d52691b4dbfc

34 files changed:
LayoutTests/ChangeLog
LayoutTests/http/tests/resourceLoadStatistics/prevalent-domains-per-page-database-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/resourceLoadStatistics/prevalent-domains-per-page-database.html [new file with mode: 0644]
LayoutTests/http/tests/resourceLoadStatistics/prevalent-domains-per-page-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/resourceLoadStatistics/prevalent-domains-per-page.html [new file with mode: 0644]
LayoutTests/http/tests/resourceLoadStatistics/resources/basic-iframe.html [new file with mode: 0644]
Source/WebCore/ChangeLog
Source/WebCore/loader/FrameLoaderClient.h
Source/WebCore/loader/ResourceLoader.cpp
Source/WebCore/loader/cache/CachedResourceLoader.cpp
Source/WebCore/platform/RegistrableDomain.h
Source/WebKit/ChangeLog
Source/WebKit/NetworkProcess/NetworkConnectionToWebProcess.cpp
Source/WebKit/NetworkProcess/NetworkConnectionToWebProcess.h
Source/WebKit/NetworkProcess/NetworkConnectionToWebProcess.messages.in
Source/WebKit/UIProcess/API/Cocoa/WKWebsiteDataStore.mm
Source/WebKit/UIProcess/API/Cocoa/WKWebsiteDataStorePrivate.h
Source/WebKit/UIProcess/WebPageProxy.cpp
Source/WebKit/UIProcess/WebPageProxy.h
Source/WebKit/WebProcess/WebCoreSupport/WebFrameLoaderClient.cpp
Source/WebKit/WebProcess/WebCoreSupport/WebFrameLoaderClient.h
Source/WebKit/WebProcess/WebPage/WebPage.cpp
Source/WebKit/WebProcess/WebPage/WebPage.h
Source/WebKit/WebProcess/WebPage/WebPage.messages.in
Tools/ChangeLog
Tools/WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl
Tools/WebKitTestRunner/InjectedBundle/InjectedBundle.cpp
Tools/WebKitTestRunner/InjectedBundle/TestRunner.cpp
Tools/WebKitTestRunner/InjectedBundle/TestRunner.h
Tools/WebKitTestRunner/TestController.cpp
Tools/WebKitTestRunner/TestController.h
Tools/WebKitTestRunner/TestInvocation.cpp
Tools/WebKitTestRunner/TestInvocation.h
Tools/WebKitTestRunner/cocoa/TestControllerCocoa.mm

index f7f9604..31868e1 100644 (file)
@@ -1,3 +1,21 @@
+2020-02-13  Kate Cheney  <katherine_cheney@apple.com>
+
+        Expose prevalent domains on a per-page basis
+        https://bugs.webkit.org/show_bug.cgi?id=207523
+        <rdar://problem/59270758>
+
+        Reviewed by Chris Dumez.
+
+        Added 2 new tests for the resource load statistics database and
+        memory stores. Added a basic iframe that both tests can use to test
+        if a resource load from a prevalent domain gets reported properly.
+
+        * http/tests/resourceLoadStatistics/prevalent-domains-per-page-database-expected.txt: Added.
+        * http/tests/resourceLoadStatistics/prevalent-domains-per-page-database.html: Added.
+        * http/tests/resourceLoadStatistics/prevalent-domains-per-page-expected.txt: Added.
+        * http/tests/resourceLoadStatistics/prevalent-domains-per-page.html: Added.
+        * http/tests/resourceLoadStatistics/resources/basic-iframe.html: Added.
+
 2020-02-13  Lauro Moura  <lmoura@igalia.com>
 
         [GTK] Gardening more flaky crashes
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/prevalent-domains-per-page-database-expected.txt b/LayoutTests/http/tests/resourceLoadStatistics/prevalent-domains-per-page-database-expected.txt
new file mode 100644 (file)
index 0000000..97e1abb
--- /dev/null
@@ -0,0 +1,10 @@
+Tests that a iframe from a prevalent domain will be reported as prevalent under this page.
+
+On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
+
+
+PASS Domain was successfully marked prevalent.
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/prevalent-domains-per-page-database.html b/LayoutTests/http/tests/resourceLoadStatistics/prevalent-domains-per-page-database.html
new file mode 100644 (file)
index 0000000..812eaee
--- /dev/null
@@ -0,0 +1,74 @@
+<!DOCTYPE html>
+<html>
+<head>
+    <script src="/js-test-resources/js-test.js"></script>
+    <script src="/js-test-resources/ui-helper.js"></script>
+    <script src="/resourceLoadStatistics/resources/util.js"></script>
+    <script>
+        description("Tests that a iframe from a prevalent domain will be reported as prevalent under this page.");
+        jsTestIsAsync = true;
+
+        function finishTest() {
+            setEnableFeature(false, finishJSTest);
+        }
+
+        const iframeID = "iFrameFromPrevalentDomain";
+        function askForPrevalentResources() {
+            testRunner.getPrevalentDomains(function (arrayOfDomains) {
+                var passed = true;
+                for (var i = 0; i < arrayOfDomains.length; ++i) {
+                    if (arrayOfDomains[i] === "localhost") {
+                        passed = true;
+                        break;
+                    }
+                }
+                if (passed)
+                    testPassed("Domain was successfully marked prevalent.");
+                else
+                    testFailed("Domain was not successfully marked prevalent.");
+                finishTest();
+            });
+        }
+
+        function activateElement(elementId) {
+            var element = document.getElementById(elementId);
+            var centerX = element.offsetLeft + element.offsetWidth / 2;
+            var centerY = element.offsetTop + element.offsetHeight / 2;
+            UIHelper.activateAt(centerX, centerY).then(
+                function () { },
+                function () {
+                    testFailed("Promise rejected.");
+                    finishTest();
+                }
+            );
+            askForPrevalentResources();
+        }
+
+        function runTest() {
+            if (document.location.hash !== "#elementActivated") {
+                document.location.hash = "elementActivated";
+                activateElement(iframeID);
+            }
+        }
+
+        const hostUnderTest = "localhost:8000";
+        const statisticsUrl = "http://" + hostUnderTest;
+        
+        testRunner.setUseITPDatabase(true);
+        setEnableFeature(true, function() {
+            testRunner.setStatisticsPrevalentResource(statisticsUrl, true, function() {
+                if (!testRunner.isStatisticsPrevalentResource(statisticsUrl))
+                    testFailed("Host did not get set as prevalent resource.");
+
+                let iframeElement = document.createElement("iframe");
+                iframeElement.onload = runTest;
+                iframeElement.id = "iFrameFromPrevalentDomain";
+                iframeElement.src = "http://localhost:8000/resourceLoadStatistics/resources/basic-iframe.html";
+                document.body.appendChild(iframeElement);
+            });
+        });
+    </script>
+</head>
+<body>
+</body>
+</html>
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/prevalent-domains-per-page-expected.txt b/LayoutTests/http/tests/resourceLoadStatistics/prevalent-domains-per-page-expected.txt
new file mode 100644 (file)
index 0000000..97e1abb
--- /dev/null
@@ -0,0 +1,10 @@
+Tests that a iframe from a prevalent domain will be reported as prevalent under this page.
+
+On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
+
+
+PASS Domain was successfully marked prevalent.
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/prevalent-domains-per-page.html b/LayoutTests/http/tests/resourceLoadStatistics/prevalent-domains-per-page.html
new file mode 100644 (file)
index 0000000..c18c6fb
--- /dev/null
@@ -0,0 +1,73 @@
+<!DOCTYPE html>
+<html>
+<head>
+    <script src="/js-test-resources/js-test.js"></script>
+    <script src="/js-test-resources/ui-helper.js"></script>
+    <script src="/resourceLoadStatistics/resources/util.js"></script>
+    <script>
+        description("Tests that a iframe from a prevalent domain will be reported as prevalent under this page.");
+        jsTestIsAsync = true;
+
+        function finishTest() {
+            setEnableFeature(false, finishJSTest);
+        }
+
+        const iframeID = "iFrameFromPrevalentDomain";
+        function askForPrevalentResources() {
+            testRunner.getPrevalentDomains(function (arrayOfDomains) {
+                var passed = false;
+                for (var i = 0; i < arrayOfDomains.length; ++i) {
+                    if (arrayOfDomains[i] === "localhost") {
+                        passed = true;
+                        break;
+                    }
+                }
+                if (passed)
+                    testPassed("Domain was successfully marked prevalent.");
+                else
+                    testFailed("Domain was not successfully marked prevalent.");
+                finishTest();
+            });
+        }
+
+        function activateElement(elementId) {
+            var element = document.getElementById(elementId);
+            var centerX = element.offsetLeft + element.offsetWidth / 2;
+            var centerY = element.offsetTop + element.offsetHeight / 2;
+            UIHelper.activateAt(centerX, centerY).then(
+                function () { },
+                function () {
+                    testFailed("Promise rejected.");
+                    finishTest();
+                }
+            );
+            askForPrevalentResources();
+        }
+
+        function runTest() {
+            if (document.location.hash !== "#elementActivated") {
+                document.location.hash = "elementActivated";
+                activateElement(iframeID);
+            }
+        }
+
+        const hostUnderTest = "localhost:8000";
+        const statisticsUrl = "http://" + hostUnderTest;
+        
+        setEnableFeature(true, function() {
+            testRunner.setStatisticsPrevalentResource(statisticsUrl, true, function() {
+                if (!testRunner.isStatisticsPrevalentResource(statisticsUrl))
+                    testFailed("Host did not get set as prevalent resource.");
+
+                let iframeElement = document.createElement("iframe");
+                iframeElement.onload = runTest;
+                iframeElement.id = "iFrameFromPrevalentDomain";
+                iframeElement.src = "http://localhost:8000/storageAccess/resourceLoadStatistics/basic-iframe.html";
+                document.body.appendChild(iframeElement);
+            });
+        });
+    </script>
+</head>
+<body>
+</body>
+</html>
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/resources/basic-iframe.html b/LayoutTests/http/tests/resourceLoadStatistics/resources/basic-iframe.html
new file mode 100644 (file)
index 0000000..2655f4e
--- /dev/null
@@ -0,0 +1,7 @@
+<!DOCTYPE html>
+<html>
+<head>
+</head>
+<body>
+</body>
+</html>
index 2e0633c..4c2d446 100644 (file)
@@ -1,3 +1,24 @@
+2020-02-13  Kate Cheney  <katherine_cheney@apple.com>
+
+        Expose prevalent domains on a per-page basis
+        https://bugs.webkit.org/show_bug.cgi?id=207523
+        <rdar://problem/59270758>
+
+        Reviewed by Chris Dumez.
+
+        Tests: http/tests/resourceLoadStatistics/prevalent-domains-per-page-database.html
+               http/tests/resourceLoadStatistics/prevalent-domains-per-page.html
+
+        Logs subresource loads in CachedResourceLoader::requestResource, and
+        redirects in ResourceLoader::willSendRequestInternal,
+        and reports domains to the Web Process.
+
+        * loader/FrameLoaderClient.h:
+        * loader/cache/CachedResourceLoader.cpp:
+        (WebCore::CachedResourceLoader::requestResource):
+        * loader/cache/ResourceLoader.cpp:
+        (WebCore::ResourceLoader::willSendRequestInternal):
+
 2020-02-13  Ryosuke Niwa  <rniwa@webkit.org>
 
         Dark Mode: In Notes, list item becomes invisible in dark mode after outdenting
index e9debe1..98d02cf 100644 (file)
@@ -90,6 +90,7 @@ class NavigationAction;
 class Page;
 class PluginViewBase;
 class ProtectionSpace;
+class RegistrableDomain;
 class RTCPeerConnectionHandler;
 class ResourceError;
 class ResourceHandle;
@@ -374,6 +375,7 @@ public:
 
 #if ENABLE(RESOURCE_LOAD_STATISTICS)
     virtual bool hasFrameSpecificStorageAccess() { return false; }
+    virtual void addLoadedRegistrableDomain(RegistrableDomain&&) { }
 #endif
 };
 
index a04e3d5..7321700 100644 (file)
@@ -410,9 +410,12 @@ void ResourceLoader::willSendRequestInternal(ResourceRequest&& request, const Re
 #endif
 
     bool isRedirect = !redirectResponse.isNull();
-    if (isRedirect)
+    if (isRedirect) {
         platformStrategies()->loaderStrategy()->crossOriginRedirectReceived(this, request.url());
-
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+        frameLoader()->client().addLoadedRegistrableDomain(RegistrableDomain(request.url()));
+#endif
+    }
     m_request = request;
 
     if (isRedirect) {
index c3d8e35..d42eab4 100644 (file)
@@ -982,6 +982,9 @@ ResourceErrorOr<CachedResourceHandle<CachedResource>> CachedResourceLoader::requ
 
     ASSERT(resource->url() == url.string());
     m_documentResources.set(resource->url(), resource);
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+    frame.loader().client().addLoadedRegistrableDomain(RegistrableDomain(resource->resourceRequest().url()));
+#endif
     return resource;
 }
 
index 90aec25..565e427 100644 (file)
@@ -49,6 +49,7 @@ public:
     }
 
     bool isEmpty() const { return m_registrableDomain.isEmpty() || m_registrableDomain == "nullOrigin"_s; }
+    String& string() { return m_registrableDomain; }
     const String& string() const { return m_registrableDomain; }
 
     bool operator!=(const RegistrableDomain& other) const { return m_registrableDomain != other.m_registrableDomain; }
index 3d0aa94..fb2bade 100644 (file)
@@ -1,3 +1,45 @@
+2020-02-13  Kate Cheney  <katherine_cheney@apple.com>
+
+        Expose prevalent domains on a per-page basis
+        https://bugs.webkit.org/show_bug.cgi?id=207523
+        <rdar://problem/59270758>
+
+        Reviewed by Chris Dumez.
+
+        Tests: http/tests/resourceLoadStatistics/prevalent-domains-per-page-database.html
+               http/tests/resourceLoadStatistics/prevalent-domains-per-page.html
+
+        Added an API to report prevalent domains. The Web Process sends all
+        new loads to the Network Process, which determines if the resources are
+        prevalent. When the API is queried, the Web Process reports these
+        domains to the UI Process.
+
+        The WebPage keeps a hash set of previously loaded domains to avoid
+        spamming IPC with repeat domains.
+
+        * NetworkProcess/NetworkConnectionToWebProcess.cpp:
+        (WebKit::NetworkConnectionToWebProcess::requestStorageAccessUnderOpener):
+        (WebKit::NetworkConnectionToWebProcess::reportSubresourceLoadToDomain):
+        * NetworkProcess/NetworkConnectionToWebProcess.h:
+        * NetworkProcess/NetworkConnectionToWebProcess.messages.in:
+        * UIProcess/API/Cocoa/WKWebsiteDataStore.mm:
+        (-[WKWebsiteDataStore _getPrevalentDomainsFor:completionHandler:]):
+        (-[WKWebsiteDataStore _clearPrevalentDomainsFor:]):
+        * UIProcess/API/Cocoa/WKWebsiteDataStorePrivate.h:
+        * UIProcess/WebPageProxy.cpp:
+        (WebKit::WebPageProxy::didCommitLoadForFrame):
+        * UIProcess/WebPageProxy.h:
+        * WebProcess/WebCoreSupport/WebFrameLoaderClient.cpp:
+        (WebKit::WebFrameLoaderClient::addLoadedRegistrableDomain):
+        * WebProcess/WebCoreSupport/WebFrameLoaderClient.h:
+        * WebProcess/WebPage/WebPage.cpp:
+        (WebKit::WebPage::setIsSuspended):
+        (WebKit::WebPage::addLoadedRegistrableDomain):
+        (WebKit::WebPage::getPrevalentDomains):
+        (WebKit::WebPage::clearPrevalentDomains):
+        * WebProcess/WebPage/WebPage.h:
+        * WebProcess/WebPage/WebPage.messages.in:
+
 2020-02-13  Youenn Fablet  <youenn@apple.com>
 
         Protect WebSWServerConnection::scheduleJobInServer from bad scopeURL
index e817344..75e0b4a 100644 (file)
@@ -787,6 +787,17 @@ void NetworkConnectionToWebProcess::requestStorageAccessUnderOpener(WebCore::Reg
             resourceLoadStatistics->requestStorageAccessUnderOpener(WTFMove(domainInNeedOfStorageAccess), openerPageID, WTFMove(openerDomain));
     }
 }
+
+void NetworkConnectionToWebProcess::isPrevalentSubresourceLoad(RegistrableDomain&& domain, CompletionHandler<void(bool)>&& completionHandler)
+{
+    if (auto* networkSession = this->networkSession()) {
+        if (auto* resourceLoadStatistics = networkSession->resourceLoadStatistics()) {
+            resourceLoadStatistics->isPrevalentResource(domain, WTFMove(completionHandler));
+            return;
+        }
+    }
+    completionHandler(false);
+}
 #endif
 
 void NetworkConnectionToWebProcess::addOriginAccessWhitelistEntry(const String& sourceOrigin, const String& destinationProtocol, const String& destinationHost, bool allowDestinationSubdomains)
index 86cc60d..eb6f55c 100644 (file)
@@ -258,6 +258,7 @@ private:
     void hasStorageAccess(const RegistrableDomain& subFrameDomain, const RegistrableDomain& topFrameDomain, WebCore::FrameIdentifier, WebCore::PageIdentifier, CompletionHandler<void(bool)>&&);
     void requestStorageAccess(const RegistrableDomain& subFrameDomain, const RegistrableDomain& topFrameDomain, WebCore::FrameIdentifier, WebCore::PageIdentifier, WebPageProxyIdentifier, CompletionHandler<void(WebCore::StorageAccessWasGranted, WebCore::StorageAccessPromptWasShown)>&&);
     void requestStorageAccessUnderOpener(WebCore::RegistrableDomain&& domainInNeedOfStorageAccess, WebCore::PageIdentifier openerPageID, WebCore::RegistrableDomain&& openerDomain);
+    void isPrevalentSubresourceLoad(RegistrableDomain&&, CompletionHandler<void(bool)>&&);
 #endif
 
     void addOriginAccessWhitelistEntry(const String& sourceOrigin, const String& destinationProtocol, const String& destinationHost, bool allowDestinationSubdomains);
index 43b5f67..ec9b0b8 100644 (file)
@@ -62,6 +62,7 @@ messages -> NetworkConnectionToWebProcess LegacyReceiver {
     HasStorageAccess(WebCore::RegistrableDomain subFrameDomain, WebCore::RegistrableDomain topFrameDomain, WebCore::FrameIdentifier frameID, WebCore::PageIdentifier pageID) -> (bool hasStorageAccess) Async
     RequestStorageAccess(WebCore::RegistrableDomain subFrameDomain, WebCore::RegistrableDomain topFrameDomain, WebCore::FrameIdentifier frameID, WebCore::PageIdentifier webPageID, WebKit::WebPageProxyIdentifier webPageProxyID) -> (enum:bool WebCore::StorageAccessWasGranted wasGranted, enum:bool WebCore::StorageAccessPromptWasShown promptWasShown) Async
     RequestStorageAccessUnderOpener(WebCore::RegistrableDomain domainInNeedOfStorageAccess, WebCore::PageIdentifier openerPageID, WebCore::RegistrableDomain openerDomain)
+    IsPrevalentSubresourceLoad(WebCore::RegistrableDomain domain) -> (bool isPrevalent) Async
 #endif
 
     AddOriginAccessWhitelistEntry(String sourceOrigin, String destinationProtocol, String destinationHost, bool allowDestinationSubdomains);
index 8667542..432c207 100644 (file)
@@ -407,6 +407,46 @@ static Vector<WebKit::WebsiteDataRecord> toWebsiteDataRecords(NSArray *dataRecor
     WebKit::WebsiteDataStore::allowWebsiteDataRecordsForAllOrigins();
 }
 
+- (void)_getPrevalentDomainsFor:(WKWebView *)webView completionHandler:(void (^)(NSArray<NSString *> *domains))completionHandler
+{
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+    if (!webView) {
+        completionHandler(nil);
+        return;
+    }
+
+    auto webPageProxy = [webView _page];
+    if (!webPageProxy) {
+        completionHandler(nil);
+        return;
+    }
+    
+    webPageProxy->getPrevalentDomains([completionHandler = makeBlockPtr(completionHandler)] (Vector<WebCore::RegistrableDomain>&& prevalentDomains) {
+        Vector<RefPtr<API::Object>> apiDomains = WTF::map(prevalentDomains, [](auto& domain) {
+            return RefPtr<API::Object>(API::String::create(WTFMove(domain.string())));
+        });
+        completionHandler(wrapper(API::Array::create(WTFMove(apiDomains))));
+    });
+#else
+    completionHandler(nil);
+#endif
+}
+
+- (void)_clearPrevalentDomainsFor:(WKWebView *)webView
+{
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+    if (!webView)
+        return;
+
+    auto webPageProxy = [webView _page];
+    if (!webPageProxy)
+        return;
+
+    webPageProxy->clearPrevalentDomains();
+#endif
+}
+
+
 - (void)_getAllStorageAccessEntriesFor:(WKWebView *)webView completionHandler:(void (^)(NSArray<NSString *> *domains))completionHandler
 {
     if (!webView) {
index 6e99619..7d3fd5f 100644 (file)
@@ -62,6 +62,8 @@ typedef NS_OPTIONS(NSUInteger, _WKWebsiteDataStoreFetchOptions) {
 
 - (void)_setResourceLoadStatisticsTestingCallback:(nullable void (^)(WKWebsiteDataStore *, NSString *))callback WK_API_AVAILABLE(macos(10.13), ios(11.0));
 - (void)_getAllStorageAccessEntriesFor:(WKWebView *)webView completionHandler:(void (^)(NSArray<NSString *> *domains))completionHandler WK_API_AVAILABLE(macos(10.14), ios(12.0));
+- (void)_getPrevalentDomainsFor:(WKWebView *)webView completionHandler:(void (^)(NSArray<NSString *> *domains))completionHandler WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
+- (void)_clearPrevalentDomainsFor:(WKWebView *)webView WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
 + (void)_allowWebsiteDataRecordsForAllOrigins WK_API_AVAILABLE(macos(10.13.4), ios(11.3));
 - (bool)_hasRegisteredServiceWorker WK_API_AVAILABLE(macos(10.14), ios(12.0));
 
index 7522ef1..163b8f7 100644 (file)
@@ -9856,6 +9856,18 @@ void WebPageProxy::setOrientationForMediaCapture(uint64_t orientation)
 #endif
 }
 
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+void WebPageProxy::getPrevalentDomains(CompletionHandler<void(Vector<RegistrableDomain>&&)>&& completionHandler)
+{
+    sendWithAsyncReply(Messages::WebPage::GetPrevalentDomains(), WTFMove(completionHandler));
+}
+
+void WebPageProxy::clearPrevalentDomains()
+{
+    send(Messages::WebPage::ClearPrevalentDomains());
+}
+#endif
+
 } // namespace WebKit
 
 #undef MERGE_WHEEL_EVENTS
index 7dfcfa3..5ff6792 100644 (file)
@@ -1546,6 +1546,8 @@ public:
 #if ENABLE(RESOURCE_LOAD_STATISTICS)
     void requestStorageAccessConfirm(const WebCore::RegistrableDomain& subFrameDomain, const WebCore::RegistrableDomain& topFrameDomain, WebCore::FrameIdentifier, CompletionHandler<void(bool)>&&);
     void didCommitCrossSiteLoadWithDataTransferFromPrevalentResource();
+    void getPrevalentDomains(CompletionHandler<void(Vector<WebCore::RegistrableDomain>&&)>&&);
+    void clearPrevalentDomains();
 #endif
 
 #if ENABLE(DEVICE_ORIENTATION)
index 121cdca..cfb3d43 100644 (file)
@@ -153,6 +153,16 @@ void WebFrameLoaderClient::setHasFrameSpecificStorageAccess(FrameSpecificStorage
 
     m_frameSpecificStorageAccessIdentifier = WTFMove(frameSpecificStorageAccessIdentifier);
 }
+
+void WebFrameLoaderClient::addLoadedRegistrableDomain(RegistrableDomain&& domain)
+{
+    auto* webPage = m_frame->page();
+    if (!webPage)
+        return;
+    
+    webPage->addLoadedRegistrableDomain(WTFMove(domain));
+}
+
 #endif
 
 void WebFrameLoaderClient::frameLoaderDestroyed()
index 9aa5a05..e377b81 100644 (file)
@@ -61,6 +61,7 @@ public:
         WebCore::PageIdentifier pageID;
     };
     void setHasFrameSpecificStorageAccess(FrameSpecificStorageAccessIdentifier&&);
+    void addLoadedRegistrableDomain(WebCore::RegistrableDomain&&) final;
 #endif
     
 private:
index e0284a8..aa99e9d 100644 (file)
@@ -5830,6 +5830,10 @@ void WebPage::didCommitLoad(WebFrame* frame)
     if (!frame->isMainFrame())
         return;
 
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+    clearPrevalentDomains();
+#endif
+    
     // If previous URL is invalid, then it's not a real page that's being navigated away from.
     // Most likely, this is actually the first load to be committed in this page.
     if (frame->coreFrame()->loader().previousURL().isValid())
@@ -6660,6 +6664,29 @@ void WebPage::wasLoadedWithDataTransferFromPrevalentResource()
 
     frame->document()->wasLoadedWithDataTransferFromPrevalentResource();
 }
+
+void WebPage::addLoadedRegistrableDomain(RegistrableDomain&& domain)
+{
+    auto addResult = m_loadedDomains.add(domain);
+    if (addResult.isNewEntry) {
+        WebProcess::singleton().ensureNetworkProcessConnection().connection().sendWithAsyncReply(Messages::NetworkConnectionToWebProcess::IsPrevalentSubresourceLoad(domain), [this, protectedThis = makeRef(*this), domain] (bool isPrevalent) {
+            if (isPrevalent)
+                m_prevalentDomains.add(domain);
+        });
+    }
+}
+
+void WebPage::getPrevalentDomains(CompletionHandler<void(Vector<RegistrableDomain>)>&& completionHandler)
+{
+    completionHandler(copyToVector(m_prevalentDomains));
+}
+
+void WebPage::clearPrevalentDomains()
+{
+    m_prevalentDomains.clear();
+    m_loadedDomains.clear();
+}
+
 #endif
 
 #if ENABLE(DEVICE_ORIENTATION)
index bdab81b..76d04a0 100644 (file)
@@ -1176,6 +1176,9 @@ public:
     bool hasPageLevelStorageAccess(const WebCore::RegistrableDomain& topLevelDomain, const WebCore::RegistrableDomain& resourceDomain) const;
     void addDomainWithPageLevelStorageAccess(const WebCore::RegistrableDomain& topLevelDomain, const WebCore::RegistrableDomain& resourceDomain);
     void wasLoadedWithDataTransferFromPrevalentResource();
+    void addLoadedRegistrableDomain(WebCore::RegistrableDomain&&);
+    void clearPrevalentDomains();
+    void getPrevalentDomains(CompletionHandler<void(Vector<WebCore::RegistrableDomain>)>&&);
 #endif
 
 #if ENABLE(DEVICE_ORIENTATION)
@@ -2045,6 +2048,8 @@ private:
 
 #if ENABLE(RESOURCE_LOAD_STATISTICS)
     HashMap<WebCore::RegistrableDomain, WebCore::RegistrableDomain> m_domainsWithPageLevelStorageAccess;
+    HashSet<WebCore::RegistrableDomain> m_loadedDomains;
+    HashSet<WebCore::RegistrableDomain> m_prevalentDomains;
 #endif
 
     String m_overriddenMediaType;
index 1b4618b..d016e41 100644 (file)
@@ -582,6 +582,8 @@ GenerateSyntheticEditingCommand(enum:uint8_t WebKit::SyntheticEditingCommandType
 
 #if ENABLE(RESOURCE_LOAD_STATISTICS)
     WasLoadedWithDataTransferFromPrevalentResource()
+    ClearPrevalentDomains()
+    GetPrevalentDomains() -> (Vector<WebCore::RegistrableDomain> domains) Async
 #endif
 
 #if USE(SYSTEM_PREVIEW)
index 2a00691..f01477b 100644 (file)
@@ -1,3 +1,35 @@
+2020-02-13  Kate Cheney  <katherine_cheney@apple.com>
+
+         Expose prevalent domains on a per-page basis
+         https://bugs.webkit.org/show_bug.cgi?id=207523
+         <rdar://problem/59270758>
+
+        Reviewed by Chris Dumez.
+
+        Added WebKitTestRunner functions so this functionality can be tested.
+        clearPrevalentDomains should be called between tests to reset the
+        state.
+
+        * WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl:
+        * WebKitTestRunner/InjectedBundle/InjectedBundle.cpp:
+        (WTR::InjectedBundle::didReceiveMessageToPage):
+        * WebKitTestRunner/InjectedBundle/TestRunner.cpp:
+        (WTR::TestRunner::getPrevalentDomains):
+        (WTR::TestRunner::callDidReceivePrevalentDomainsCallback):
+        * WebKitTestRunner/InjectedBundle/TestRunner.h:
+        * WebKitTestRunner/TestController.cpp:
+        (WTR::TestController::resetStateToConsistentValues):
+        (WTR::TestController::getPrevalentDomains):
+        (WTR::TestController::clearPrevalentDomains):
+        * WebKitTestRunner/TestController.h:
+        * WebKitTestRunner/TestInvocation.cpp:
+        (WTR::TestInvocation::didReceiveMessageFromInjectedBundle):
+        (WTR::TestInvocation::didReceivePrevalentDomains):
+        * WebKitTestRunner/TestInvocation.h:
+        * WebKitTestRunner/cocoa/TestControllerCocoa.mm:
+        (WTR::TestController::getPrevalentDomains):
+        (WTR::TestController::clearPrevalentDomains):
+
 2020-02-13  Fujii Hironori  <Hironori.Fujii@sony.com>
 
         [Win][MiniBrowser] Reimplement the toolbar by using toolbar common control
index 160c2e9..a7fd21e 100644 (file)
@@ -344,6 +344,7 @@ interface TestRunner {
     void setStatisticsShouldDowngradeReferrer(boolean value, object callback);
     void setStatisticsShouldBlockThirdPartyCookies(boolean value, object callback, optional boolean onlyOnSitesWithoutUserInteraction);
     void setStatisticsFirstPartyWebsiteDataRemovalMode(boolean value, object callback);
+    void getPrevalentDomains(object callback);
 
     // Injected bundle form client.
     void installTextDidChangeInTextFieldCallback(object callback);
index faedaf5..958a79b 100644 (file)
@@ -404,6 +404,24 @@ void InjectedBundle::didReceiveMessageToPage(WKBundlePageRef page, WKStringRef m
         m_testRunner->callDidReceiveAllStorageAccessEntriesCallback(domains);
         return;
     }
+    
+    if (WKStringIsEqualToUTF8CString(messageName, "CallDidReceivePrevalentDomains")) {
+        ASSERT(messageBody);
+        ASSERT(WKGetTypeID(messageBody) == WKArrayGetTypeID());
+
+        WKArrayRef domainsArray = static_cast<WKArrayRef>(messageBody);
+        auto size = WKArrayGetSize(domainsArray);
+        Vector<String> domains;
+        domains.reserveInitialCapacity(size);
+        for (size_t i = 0; i < size; ++i) {
+            WKTypeRef item = WKArrayGetItemAtIndex(domainsArray, i);
+            if (item && WKGetTypeID(item) == WKStringGetTypeID())
+                domains.uncheckedAppend(toWTFString(static_cast<WKStringRef>(item)));
+        }
+
+        m_testRunner->callDidReceivePrevalentDomainsCallback(WTFMove(domains));
+        return;
+    }
 
     if (WKStringIsEqualToUTF8CString(messageName, "CallDidRemoveAllSessionCredentialsCallback")) {
         m_testRunner->callDidRemoveAllSessionCredentialsCallback();
index 24569c9..521f5e5 100644 (file)
@@ -754,6 +754,7 @@ enum {
     StatisticsDidSetShouldBlockThirdPartyCookiesCallbackID,
     StatisticsDidSetFirstPartyWebsiteDataRemovalModeCallbackID,
     AllStorageAccessEntriesCallbackID,
+    GetPrevalentDomainsCallbackID,
     DidRemoveAllSessionCredentialsCallbackID,
     GetApplicationManifestCallbackID,
     TextDidChangeInTextFieldCallbackID,
@@ -2352,6 +2353,38 @@ void TestRunner::callDidReceiveAllStorageAccessEntriesCallback(Vector<String>& d
     callTestRunnerCallback(AllStorageAccessEntriesCallbackID, 1, &result);
 }
 
+void TestRunner::getPrevalentDomains(JSValueRef callback)
+{
+    cacheTestRunnerCallback(GetPrevalentDomainsCallbackID, callback);
+    
+    WKRetainPtr<WKStringRef> messageName = adoptWK(WKStringCreateWithUTF8CString("GetPrevalentDomains"));
+    WKBundlePostMessage(InjectedBundle::singleton().bundle(), messageName.get(), nullptr);
+}
+
+void TestRunner::callDidReceivePrevalentDomainsCallback(Vector<String>&& domains)
+{
+    WKBundleFrameRef mainFrame = WKBundlePageGetMainFrame(InjectedBundle::singleton().page()->page());
+    JSContextRef context = WKBundleFrameGetJavaScriptContext(mainFrame);
+    
+    StringBuilder stringBuilder;
+    stringBuilder.append('[');
+    bool isFirstDomain = true;
+    for (auto& domain : domains) {
+        if (isFirstDomain)
+            isFirstDomain = false;
+        else
+            stringBuilder.appendLiteral(", ");
+        stringBuilder.appendLiteral("\"");
+        stringBuilder.append(domain);
+        stringBuilder.appendLiteral("\"");
+    }
+    stringBuilder.append(']');
+    
+    JSValueRef result = JSValueMakeFromJSONString(context, adopt(JSStringCreateWithUTF8CString(stringBuilder.toString().utf8().data())).get());
+
+    callTestRunnerCallback(GetPrevalentDomainsCallbackID, 1, &result);
+}
+
 void TestRunner::addMockMediaDevice(JSStringRef persistentId, JSStringRef label, const char* type)
 {
     Vector<WKRetainPtr<WKStringRef>> keys;
index c3c1045..7228b5b 100644 (file)
@@ -448,6 +448,8 @@ public:
     void statisticsCallDidSetFirstPartyWebsiteDataRemovalModeCallback();
     void statisticsResetToConsistentState(JSValueRef completionHandler);
     void statisticsCallDidResetToConsistentStateCallback();
+    void getPrevalentDomains(JSValueRef callback);
+    void callDidReceivePrevalentDomainsCallback(Vector<String>&& domains);
 
     // Injected bundle form client.
     void installTextDidChangeInTextFieldCallback(JSValueRef callback);
index bf14227..5cd9237 100644 (file)
@@ -1109,7 +1109,8 @@ bool TestController::resetStateToConsistentValues(const TestOptions& options, Re
     setAllowsAnySSLCertificate(true);
 
     statisticsResetToConsistentState();
-    
+    clearPrevalentDomains();
+
     clearAdClickAttribution();
 
     m_didReceiveServerRedirectForProvisionalNavigation = false;
@@ -3113,6 +3114,14 @@ void TestController::getAllStorageAccessEntries()
 {
 }
 
+void TestController::getPrevalentDomains()
+{
+}
+
+void TestController::clearPrevalentDomains()
+{
+}
+
 #endif
 
 struct ClearServiceWorkerRegistrationsCallbackContext {
index fa60882..4e65112 100644 (file)
@@ -258,6 +258,8 @@ public:
     void statisticsResetToConsistentState();
 
     void getAllStorageAccessEntries();
+    void getPrevalentDomains();
+    void clearPrevalentDomains();
 
     WKArrayRef openPanelFileURLs() const { return m_openPanelFileURLs.get(); }
     void setOpenPanelFileURLs(WKArrayRef fileURLs) { m_openPanelFileURLs = fileURLs; }
index 29b99f8..81afb52 100644 (file)
@@ -807,6 +807,11 @@ void TestInvocation::didReceiveMessageFromInjectedBundle(WKStringRef messageName
         return;
     }
 #endif
+    
+    if (WKStringIsEqualToUTF8CString(messageName, "GetPrevalentDomains")) {
+        TestController::singleton().getPrevalentDomains();
+        return;
+    }
 
     ASSERT_NOT_REACHED();
 }
@@ -1942,6 +1947,17 @@ void TestInvocation::didReceiveAllStorageAccessEntries(Vector<String>& domains)
     WKPagePostMessageToInjectedBundle(TestController::singleton().mainWebView()->page(), messageName.get(), messageBody.get());
 }
 
+void TestInvocation::didReceivePrevalentDomains(Vector<String>&& domains)
+{
+    WKRetainPtr<WKStringRef> messageName = adoptWK(WKStringCreateWithUTF8CString("CallDidReceivePrevalentDomains"));
+    
+    WKRetainPtr<WKMutableArrayRef> messageBody = adoptWK(WKMutableArrayCreate());
+    for (auto& domain : domains)
+        WKArrayAppendItem(messageBody.get(), adoptWK(WKStringCreateWithUTF8CString(domain.utf8().data())).get());
+    
+    WKPagePostMessageToInjectedBundle(TestController::singleton().mainWebView()->page(), messageName.get(), messageBody.get());
+}
+
 void TestInvocation::didRemoveAllSessionCredentials()
 {
     WKRetainPtr<WKStringRef> messageName = adoptWK(WKStringCreateWithUTF8CString("CallDidRemoveAllSessionCredentialsCallback"));
index e1c2c5d..50337d6 100644 (file)
@@ -86,6 +86,7 @@ public:
     void didSetVeryPrevalentResource();
     void didSetHasHadUserInteraction();
     void didReceiveAllStorageAccessEntries(Vector<String>& domains);
+    void didReceivePrevalentDomains(Vector<String>&& domains);
 
     void didRemoveAllSessionCredentials();
     
index 3f9304e..81c0a06 100644 (file)
@@ -332,6 +332,30 @@ void TestController::getAllStorageAccessEntries()
     }];
 }
 
+void TestController::getPrevalentDomains()
+{
+    auto* parentView = mainWebView();
+    if (!parentView)
+        return;
+    
+    [globalWebViewConfiguration.websiteDataStore _getPrevalentDomainsFor:parentView->platformView() completionHandler:^(NSArray<NSString *> *nsDomains) {
+        Vector<String> domains;
+        domains.reserveInitialCapacity(nsDomains.count);
+        for (NSString *domain : nsDomains)
+            domains.uncheckedAppend(domain);
+        m_currentInvocation->didReceivePrevalentDomains(WTFMove(domains));
+    }];
+}
+
+void TestController::clearPrevalentDomains()
+{
+    auto* parentView = mainWebView();
+    if (!parentView)
+        return;
+
+    [globalWebViewConfiguration.websiteDataStore _clearPrevalentDomainsFor:parentView->platformView()];
+}
+
 void TestController::injectUserScript(WKStringRef script)
 {
     auto userScript = adoptNS([[WKUserScript alloc] initWithSource: toWTFString(script) injectionTime:WKUserScriptInjectionTimeAtDocumentStart forMainFrameOnly:NO]);