Resource Load Statistics: Make it possible exclude localhost from classification
authorwilander@apple.com <wilander@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 11 Mar 2019 17:01:27 +0000 (17:01 +0000)
committerwilander@apple.com <wilander@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 11 Mar 2019 17:01:27 +0000 (17:01 +0000)
https://bugs.webkit.org/show_bug.cgi?id=195474
<rdar://problem/47520577>

Reviewed by Brent Fulgham.

Source/WebKit:

This patch allows for localhost to be excluded from classification and
treatment as a prevalent resource.

* NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:
(WebKit::ResourceLoadStatisticsDatabaseStore::ResourceLoadStatisticsDatabaseStore):
(WebKit::ResourceLoadStatisticsDatabaseStore::reclassifyResources):
    Makes use of the new ResourceLoadStatisticsMemoryStore::shouldSkip().
(WebKit::ResourceLoadStatisticsDatabaseStore::setPrevalentResource):
    Makes use of the new ResourceLoadStatisticsMemoryStore::shouldSkip().
(WebKit::ResourceLoadStatisticsDatabaseStore::isPrevalentResource const):
    Makes use of the new ResourceLoadStatisticsMemoryStore::shouldSkip().
(WebKit::ResourceLoadStatisticsDatabaseStore::isVeryPrevalentResource const):
    Makes use of the new ResourceLoadStatisticsMemoryStore::shouldSkip().
(WebKit::ResourceLoadStatisticsDatabaseStore::setVeryPrevalentResource):
    Makes use of the new ResourceLoadStatisticsMemoryStore::shouldSkip().
* NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h:
* NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp:
(WebKit::ResourceLoadStatisticsMemoryStore::ResourceLoadStatisticsMemoryStore):
(WebKit::ResourceLoadStatisticsMemoryStore::classifyPrevalentResources):
    Makes use of the new ResourceLoadStatisticsMemoryStore::shouldSkip().
(WebKit::ResourceLoadStatisticsMemoryStore::setPrevalentResource):
    Makes use of the new ResourceLoadStatisticsMemoryStore::shouldSkip().
(WebKit::ResourceLoadStatisticsMemoryStore::isPrevalentResource const):
    Makes use of the new ResourceLoadStatisticsMemoryStore::shouldSkip().
(WebKit::ResourceLoadStatisticsMemoryStore::isVeryPrevalentResource const):
    Makes use of the new ResourceLoadStatisticsMemoryStore::shouldSkip().
(WebKit::ResourceLoadStatisticsMemoryStore::setVeryPrevalentResource):
    Makes use of the new ResourceLoadStatisticsMemoryStore::shouldSkip().
* NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h:
* NetworkProcess/Classifier/ResourceLoadStatisticsStore.cpp:
(WebKit::ResourceLoadStatisticsStore::ResourceLoadStatisticsStore):
    Now takes a ShouldIncludeLocalhost parameter.
(WebKit::ResourceLoadStatisticsStore::shouldSkip const):
    Convenience function, currently supporting the localhost exclusion.
(WebKit::ResourceLoadStatisticsStore::setIsRunningTest):
    Test infrastructure.
* NetworkProcess/Classifier/ResourceLoadStatisticsStore.h:
* NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:
(WebKit::WebResourceLoadStatisticsStore::setIsRunningTest):
    Test infrastructure.
(WebKit::WebResourceLoadStatisticsStore::WebResourceLoadStatisticsStore):
    Now takes a ShouldIncludeLocalhost parameter.
* NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h:
    Defines the new ShouldIncludeLocalhost boolean enum.
* NetworkProcess/NetworkProcess.cpp:
(WebKit::NetworkProcess::setIsRunningResourceLoadStatisticsTest):
    Test infrastructure.
* NetworkProcess/NetworkProcess.h:
* NetworkProcess/NetworkProcess.messages.in:
* NetworkProcess/NetworkSession.cpp:
(WebKit::NetworkSession::setResourceLoadStatisticsEnabled):
    Forwards the localhost setting to the create function.
* NetworkProcess/NetworkSession.h:
* NetworkProcess/NetworkSessionCreationParameters.cpp:
(WebKit::NetworkSessionCreationParameters::encode const):
(WebKit::NetworkSessionCreationParameters::decode):
* NetworkProcess/NetworkSessionCreationParameters.h:
    New parameter called shouldIncludeLocalhostInResourceLoadStatistics.
* NetworkProcess/cocoa/NetworkSessionCocoa.mm:
(WebKit::NetworkSessionCocoa::NetworkSessionCocoa):
    Picks up the localhost setting from the parameters.
* UIProcess/API/C/WKWebsiteDataStoreRef.cpp:
(WKWebsiteDataStoreSetStatisticsIsRunningTest):
    Test infrastructure.
* UIProcess/API/C/WKWebsiteDataStoreRef.h:
* UIProcess/Network/NetworkProcessProxy.cpp:
(WebKit::NetworkProcessProxy::setIsRunningResourceLoadStatisticsTest):
    Test infrastructure.
* UIProcess/Network/NetworkProcessProxy.h:
* UIProcess/WebProcessPool.cpp:
(WebKit::WebProcessPool::ensureNetworkProcess):
    Picks up the localhost setting from the WebsiteDataStore parameters.
* UIProcess/WebsiteData/Cocoa/WebsiteDataStoreCocoa.mm:
(WebKit::WebsiteDataStore::parameters):
    Makes sure Safari does not exclude localhost.
* UIProcess/WebsiteData/WebsiteDataStore.cpp:
(WebKit::WebsiteDataStore::setIsRunningResourceLoadStatisticsTest):
    Test infrastructure.
* UIProcess/WebsiteData/WebsiteDataStore.h:

Tools:

This patch allows for localhost to be excluded from classification and
treatment as a prevalent resource.

The WebKit Tools change adds a new function called
testRunner.setStatisticsIsRunningTest() which can be used to control this
behavior.

* WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl:
* WebKitTestRunner/InjectedBundle/TestRunner.cpp:
(WTR::TestRunner::setStatisticsIsRunningTest):
* WebKitTestRunner/InjectedBundle/TestRunner.h:
* WebKitTestRunner/TestController.cpp:
(WTR::TestController::setStatisticsIsRunningTest):
* WebKitTestRunner/TestController.h:
* WebKitTestRunner/TestInvocation.cpp:
(WTR::TestInvocation::didReceiveSynchronousMessageFromInjectedBundle):

LayoutTests:

This patch makes sure that all test cases that need to, call the enableFeature()
function in http/tests/resourceLoadStatistics/resources/util.js.

The enableFeature() now calls the new function testRunner.setStatisticsIsRunningTest().

* http/tests/resourceLoadStatistics/do-not-block-top-level-navigation-redirect.html:
* http/tests/resourceLoadStatistics/non-sandboxed-iframe-redirect-ip-to-localhost-to-ip.html:
* http/tests/resourceLoadStatistics/prevalent-resource-with-user-interaction.html:
* http/tests/resourceLoadStatistics/resources/set-cookie-on-redirect.php:
* http/tests/resourceLoadStatistics/resources/util.js:
(setEnableFeature):
* http/tests/storageAccess/has-storage-access-from-prevalent-domain-with-user-interaction.html:
* http/tests/storageAccess/request-and-grant-access-cross-origin-sandboxed-iframe-from-prevalent-domain-with-user-interaction.html:
* http/tests/storageAccess/request-and-grant-access-cross-origin-sandboxed-iframe-from-prevalent-domain-without-user-interaction.html:
* http/tests/storageAccess/request-and-grant-access-cross-origin-sandboxed-iframe.html:
* http/tests/storageAccess/request-storage-access-cross-origin-sandboxed-iframe-with-unique-origin.html:
* http/tests/storageAccess/request-storage-access-same-origin-iframe.html:
* http/tests/storageAccess/request-storage-access-same-origin-sandboxed-iframe.html:
* http/tests/storageAccess/request-storage-access-top-frame.html:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@242712 268f45cc-cd09-0410-ab3c-d52691b4dbfc

46 files changed:
LayoutTests/ChangeLog
LayoutTests/http/tests/resourceLoadStatistics/do-not-block-top-level-navigation-redirect.html
LayoutTests/http/tests/resourceLoadStatistics/non-sandboxed-iframe-redirect-ip-to-localhost-to-ip.html
LayoutTests/http/tests/resourceLoadStatistics/prevalent-resource-with-user-interaction.html
LayoutTests/http/tests/resourceLoadStatistics/resources/set-cookie-on-redirect.php
LayoutTests/http/tests/resourceLoadStatistics/resources/util.js
LayoutTests/http/tests/storageAccess/has-storage-access-from-prevalent-domain-with-user-interaction.html
LayoutTests/http/tests/storageAccess/request-and-grant-access-cross-origin-sandboxed-iframe-from-prevalent-domain-with-user-interaction.html
LayoutTests/http/tests/storageAccess/request-and-grant-access-cross-origin-sandboxed-iframe-from-prevalent-domain-without-user-interaction.html
LayoutTests/http/tests/storageAccess/request-and-grant-access-cross-origin-sandboxed-iframe.html
LayoutTests/http/tests/storageAccess/request-storage-access-cross-origin-sandboxed-iframe-with-unique-origin.html
LayoutTests/http/tests/storageAccess/request-storage-access-same-origin-iframe.html
LayoutTests/http/tests/storageAccess/request-storage-access-same-origin-sandboxed-iframe.html
LayoutTests/http/tests/storageAccess/request-storage-access-top-frame.html
Source/WebKit/ChangeLog
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsStore.cpp
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsStore.h
Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp
Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h
Source/WebKit/NetworkProcess/NetworkProcess.cpp
Source/WebKit/NetworkProcess/NetworkProcess.h
Source/WebKit/NetworkProcess/NetworkProcess.messages.in
Source/WebKit/NetworkProcess/NetworkSession.cpp
Source/WebKit/NetworkProcess/NetworkSession.h
Source/WebKit/NetworkProcess/NetworkSessionCreationParameters.cpp
Source/WebKit/NetworkProcess/NetworkSessionCreationParameters.h
Source/WebKit/NetworkProcess/cocoa/NetworkSessionCocoa.mm
Source/WebKit/UIProcess/API/C/WKWebsiteDataStoreRef.cpp
Source/WebKit/UIProcess/API/C/WKWebsiteDataStoreRef.h
Source/WebKit/UIProcess/Network/NetworkProcessProxy.cpp
Source/WebKit/UIProcess/Network/NetworkProcessProxy.h
Source/WebKit/UIProcess/WebProcessPool.cpp
Source/WebKit/UIProcess/WebsiteData/Cocoa/WebsiteDataStoreCocoa.mm
Source/WebKit/UIProcess/WebsiteData/WebsiteDataStore.cpp
Source/WebKit/UIProcess/WebsiteData/WebsiteDataStore.h
Tools/ChangeLog
Tools/WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl
Tools/WebKitTestRunner/InjectedBundle/TestRunner.cpp
Tools/WebKitTestRunner/InjectedBundle/TestRunner.h
Tools/WebKitTestRunner/TestController.cpp
Tools/WebKitTestRunner/TestController.h
Tools/WebKitTestRunner/TestInvocation.cpp

index 55a8a1f..f8fa3d9 100644 (file)
@@ -1,3 +1,31 @@
+2019-03-11  John Wilander  <wilander@apple.com>
+
+        Resource Load Statistics: Make it possible exclude localhost from classification
+        https://bugs.webkit.org/show_bug.cgi?id=195474
+        <rdar://problem/47520577>
+
+        Reviewed by Brent Fulgham.
+
+        This patch makes sure that all test cases that need to, call the enableFeature()
+        function in http/tests/resourceLoadStatistics/resources/util.js.
+
+        The enableFeature() now calls the new function testRunner.setStatisticsIsRunningTest().
+
+        * http/tests/resourceLoadStatistics/do-not-block-top-level-navigation-redirect.html:
+        * http/tests/resourceLoadStatistics/non-sandboxed-iframe-redirect-ip-to-localhost-to-ip.html:
+        * http/tests/resourceLoadStatistics/prevalent-resource-with-user-interaction.html:
+        * http/tests/resourceLoadStatistics/resources/set-cookie-on-redirect.php:
+        * http/tests/resourceLoadStatistics/resources/util.js:
+        (setEnableFeature):
+        * http/tests/storageAccess/has-storage-access-from-prevalent-domain-with-user-interaction.html:
+        * http/tests/storageAccess/request-and-grant-access-cross-origin-sandboxed-iframe-from-prevalent-domain-with-user-interaction.html:
+        * http/tests/storageAccess/request-and-grant-access-cross-origin-sandboxed-iframe-from-prevalent-domain-without-user-interaction.html:
+        * http/tests/storageAccess/request-and-grant-access-cross-origin-sandboxed-iframe.html:
+        * http/tests/storageAccess/request-storage-access-cross-origin-sandboxed-iframe-with-unique-origin.html:
+        * http/tests/storageAccess/request-storage-access-same-origin-iframe.html:
+        * http/tests/storageAccess/request-storage-access-same-origin-sandboxed-iframe.html:
+        * http/tests/storageAccess/request-storage-access-top-frame.html:
+
 2019-03-11  Javier Fernandez  <jfernandez@igalia.com>
 
         Unreviewed GTK test gardening
index 5183e69..f9c42cc 100644 (file)
@@ -3,6 +3,7 @@
 <head>
     <meta charset="UTF-8">
     <script src="/js-test-resources/js-test.js"></script>
+    <script src="resources/util.js"></script>
 <script>
     description("Tests that blocking is not applied to top-level navigation redirects.");
     jsTestIsAsync = true;
 
     function doRedirect()
     {
-        testRunner.setStatisticsPrevalentResource("http://localhost", true, function() {
-            if (!testRunner.isStatisticsPrevalentResource("http://localhost")) {
-                testFailed("Host did not get set as prevalent resource.");
-                finishJSTest();
-            }
-            else
-                document.location.href = "http://localhost:8000/resourceLoadStatistics/resources/set-cookie-on-redirect.php?step=1";
+        setEnableFeature(true, function () {
+            testRunner.setStatisticsPrevalentResource("http://localhost", true, function () {
+                if (!testRunner.isStatisticsPrevalentResource("http://localhost")) {
+                    testFailed("Host did not get set as prevalent resource.");
+                    setEnableFeature(false, finishJSTest);
+                } else
+                    document.location.href = "http://localhost:8000/resourceLoadStatistics/resources/set-cookie-on-redirect.php?step=1";
+            });
         });
     }
 </script>
 </head>
-<body onload="doRedirect()">
+<body onload="setTimeout('doRedirect()', 0)">
 </body>
 </html>
index 176ece0..aea7251 100644 (file)
@@ -41,7 +41,6 @@
 
         if (testRunner) {
             setEnableFeature(true, function() {
-                testRunner.setStatisticsNotifyPagesWhenDataRecordsWereScanned(true);
                 testRunner.installStatisticsDidScanDataRecordsCallback(checkStats);
             });
         }
index 8d56978..606d3bf 100644 (file)
@@ -4,9 +4,13 @@
     <meta charset="UTF-8">
     <title>Tests for Prevalent Resource With User Interaction</title>
     <script src="/js-test-resources/js-test.js"></script>
+    <script src="resources/util.js"></script>
 </head>
-<body>
+<body onload="setTimeout('setUpAndRun()', 0)">
 <script>
+    if (testRunner)
+        testRunner.waitUntilDone();
+
     const hostUnderTest = "127.0.0.1:8000";
     const statisticsUrl = "http://" + hostUnderTest + "/temp";
     const otherPrevalentUrl = "http://localhost:8000/temp";
@@ -37,8 +41,7 @@
                             testPassed("Cookie not deleted.");
                         else
                             testFailed("Cookie deleted or document.cookie contains other cookies: " + document.cookie);
-                        internals.setResourceLoadStatisticsEnabled(false);
-                        testRunner.statisticsResetToConsistentState(function() {
+                        setEnableFeature(false, function() {
                             testRunner.notifyDone();
                         });
                     });
         });
     }
 
-    if (document.location.host === hostUnderTest && window.testRunner && window.internals) {
-        testRunner.waitUntilDone();
-        internals.setResourceLoadStatisticsEnabled(true);
-        testRunner.setStatisticsNotifyPagesWhenDataRecordsWereScanned(true);
-        runTestRunnerTest();
+    function setUpAndRun() {
+        if (document.location.host === hostUnderTest && window.testRunner && window.internals) {
+            setEnableFeature(true, function () {
+                runTestRunnerTest();
+            });
+        } else {
+            testFailed("Wrong host, no testRunner, or no internals.");
+            testRunner.notifyDone();
+        }
     }
 </script>
 </body>
index 483ce2d..21052f8 100644 (file)
@@ -16,6 +16,7 @@ if ($_GET['step'] == 1) {
   } else {
     echo "FAILED: Cookie not set\n";
   }
-  echo "<script>if (window.testRunner && window.internals) testRunner.statisticsResetToConsistentState(function() { internals.setResourceLoadStatisticsEnabled(false); testRunner.notifyDone(); });</script>";
+  echo "<script src='util.js'></script>";
+  echo "<script>if (window.testRunner && window.internals) setEnableFeature(false, finishJSTest);</script>";
 }
 ?>
index dc42844..a456d09 100644 (file)
@@ -4,9 +4,11 @@ function setEnableFeature(enable, completionHandler) {
     testRunner.setStatisticsNotifyPagesWhenDataRecordsWereScanned(enable);
     if (enable) {
         internals.setResourceLoadStatisticsEnabled(true);
+        testRunner.setStatisticsIsRunningTest(true);
         completionHandler();
     } else {
         testRunner.statisticsResetToConsistentState(function() {
+            testRunner.setStatisticsIsRunningTest(false);
             internals.setResourceLoadStatisticsEnabled(false);
             completionHandler();
         });
index a707c8b..35ef2fc 100644 (file)
@@ -3,6 +3,7 @@
 <head>
     <script src="/js-test-resources/js-test.js"></script>
     <script src="/js-test-resources/ui-helper.js"></script>
+    <script src="/resourceLoadStatistics/resources/util.js"></script>
     <script>
         description("Tests that a cross-origin iframe from a prevalent domain with user interaction does not have storage access.");
         jsTestIsAsync = true;
                     testFailed(event.data);
             } else
                 testFailed("Received a message from an unexpected origin: " + event.origin);
-            finishJSTest();
+            setEnableFeature(false, finishJSTest);
         }
 
         const hostUnderTest = "localhost:8000";
         const statisticsUrl = "http://" + hostUnderTest + "/temp";
-        testRunner.setStatisticsPrevalentResource(statisticsUrl, true, function() {
-            if (!testRunner.isStatisticsPrevalentResource(statisticsUrl))
-                testFailed("Host did not get set as prevalent resource.");
-            testRunner.setStatisticsHasHadUserInteraction(statisticsUrl, true, function() {
-                if (!testRunner.isStatisticsHasHadUserInteraction(statisticsUrl))
-                    testFailed("Host did not get logged for user interaction.");
-                let iframeElement = document.createElement("iframe");
-                iframeElement.setAttribute("sandbox", "allow-storage-access-by-user-activation allow-scripts allow-same-origin allow-modals");
-                iframeElement.id = "TheIframeThatRequestsStorageAccess";
-                iframeElement.src = "http://localhost:8000/storageAccess/resources/has-storage-access-iframe.html#policyShouldNotGrantAccess";
-                document.body.appendChild(iframeElement);
+        function runTest() {
+            setEnableFeature(true, function() {
+                testRunner.setStatisticsPrevalentResource(statisticsUrl, true, function() {
+                    if (!testRunner.isStatisticsPrevalentResource(statisticsUrl))
+                        testFailed("Host did not get set as prevalent resource.");
+                    testRunner.setStatisticsHasHadUserInteraction(statisticsUrl, true, function() {
+                        if (!testRunner.isStatisticsHasHadUserInteraction(statisticsUrl))
+                            testFailed("Host did not get logged for user interaction.");
+                        let iframeElement = document.createElement("iframe");
+                        iframeElement.setAttribute("sandbox", "allow-storage-access-by-user-activation allow-scripts allow-same-origin allow-modals");
+                        iframeElement.id = "TheIframeThatRequestsStorageAccess";
+                        iframeElement.src = "http://localhost:8000/storageAccess/resources/has-storage-access-iframe.html#policyShouldNotGrantAccess";
+                        document.body.appendChild(iframeElement);
+                    });
+                });
+
             });
-        });
+        }
     </script>
 </head>
-<body>
+<body onload="runTest()">
 </body>
 </html>
\ No newline at end of file
index 60c8777..b746b26 100644 (file)
@@ -3,6 +3,7 @@
 <head>
     <script src="/js-test-resources/js-test.js"></script>
     <script src="/js-test-resources/ui-helper.js"></script>
+    <script src="/resourceLoadStatistics/resources/util.js"></script>
     <script>
         description("Tests that cross-origin iframe storage access is granted if the iframe is sandboxed, has the allow token, the iframe origin is a prevalent resource, the iframe origin has had recent user interaction, and the user opts in.");
         jsTestIsAsync = true;
@@ -17,7 +18,7 @@
                     testFailed(event.data);
             } else
                 testFailed("Received a message from an unexpected origin: " + event.origin);
-            finishJSTest();
+            setEnableFeature(false, finishJSTest);
         }
 
         function activateElement(elementId) {
                         eventSender.keyDown("escape");
                     else {
                         testFailed("No eventSender.");
-                        finishJSTest();
+                        setEnableFeature(false, finishJSTest);
                     }
                 },
                 function () {
                     testFailed("Promise rejected.");
-                    finishJSTest();
+                    setEnableFeature(false, finishJSTest);
                 }
             );
         }
 
         const hostUnderTest = "localhost:8000";
         const statisticsUrl = "http://" + hostUnderTest + "/temp";
-        testRunner.setStatisticsPrevalentResource(statisticsUrl, true, function() {
-            if (!testRunner.isStatisticsPrevalentResource(statisticsUrl))
-                testFailed("Host did not get set as prevalent resource.");
-            testRunner.setStatisticsHasHadUserInteraction(statisticsUrl, true, function() {
-                if (!testRunner.isStatisticsHasHadUserInteraction(statisticsUrl))
-                    testFailed("Host did not get logged for user interaction.");
-                let iframeElement = document.createElement("iframe");
-                iframeElement.setAttribute("sandbox", "allow-storage-access-by-user-activation allow-scripts allow-same-origin allow-modals");
-                iframeElement.onload = runTest;
-                iframeElement.id = "TheIframeThatRequestsStorageAccess";
-                iframeElement.src = "http://localhost:8000/storageAccess/resources/request-storage-access-iframe.html#userShouldGrantAccess,userShouldBeConsulted,policyShouldGrantAccess,isNotSameOriginIframe";
-                document.body.appendChild(iframeElement);
+        setEnableFeature(true, function() {
+            testRunner.setStatisticsPrevalentResource(statisticsUrl, true, function() {
+                if (!testRunner.isStatisticsPrevalentResource(statisticsUrl))
+                    testFailed("Host did not get set as prevalent resource.");
+                testRunner.setStatisticsHasHadUserInteraction(statisticsUrl, true, function() {
+                    if (!testRunner.isStatisticsHasHadUserInteraction(statisticsUrl))
+                        testFailed("Host did not get logged for user interaction.");
+                    let iframeElement = document.createElement("iframe");
+                    iframeElement.setAttribute("sandbox", "allow-storage-access-by-user-activation allow-scripts allow-same-origin allow-modals");
+                    iframeElement.onload = runTest;
+                    iframeElement.id = "TheIframeThatRequestsStorageAccess";
+                    iframeElement.src = "http://localhost:8000/storageAccess/resources/request-storage-access-iframe.html#userShouldGrantAccess,userShouldBeConsulted,policyShouldGrantAccess,isNotSameOriginIframe";
+                    document.body.appendChild(iframeElement);
+                });
             });
         });
     </script>
index a76aed9..d58f597 100644 (file)
@@ -3,6 +3,7 @@
 <head>
     <script src="/js-test-resources/js-test.js"></script>
     <script src="/js-test-resources/ui-helper.js"></script>
+    <script src="/resourceLoadStatistics/resources/util.js"></script>
     <script>
         description("Tests that cross-origin iframe storage access is denied if the iframe is sandboxed, has the allow token, the iframe origin is a prevalent resource, the iframe origin has not had user interaction, and the user opts in.");
         jsTestIsAsync = true;
@@ -17,7 +18,7 @@
                     testFailed(event.data);
             } else
                 testFailed("Received a message from an unexpected origin: " + event.origin);
-            finishJSTest();
+            setEnableFeature(false, finishJSTest);
         }
 
         function activateElement(elementId) {
                         eventSender.keyDown("escape");
                     else {
                         testFailed("No eventSender.");
-                        finishJSTest();
+                        setEnableFeature(false, finishJSTest);
                     }
                 },
                 function () {
                     testFailed("Promise rejected.");
-                    finishJSTest();
+                    setEnableFeature(false, finishJSTest);
                 }
             );
         }
 
         const hostUnderTest = "localhost:8000";
         const statisticsUrl = "http://" + hostUnderTest + "/temp";
-        testRunner.setStatisticsPrevalentResource(statisticsUrl, true, function() {
-            if (!testRunner.isStatisticsPrevalentResource(statisticsUrl))
-                testFailed("Host did not get set as prevalent resource.");
-            let iframeElement = document.createElement("iframe");
-            iframeElement.setAttribute("sandbox", "allow-storage-access-by-user-activation allow-scripts allow-same-origin allow-modals");
-            iframeElement.onload = runTest;
-            iframeElement.id = "TheIframeThatRequestsStorageAccess";
-            iframeElement.src = "http://localhost:8000/storageAccess/resources/request-storage-access-iframe.html#userShouldGrantAccess,userShouldBeConsulted,policyShouldNotGrantAccess,isNotSameOriginIframe";
-            document.body.appendChild(iframeElement);
+
+        setEnableFeature(true, function() {
+            testRunner.setStatisticsPrevalentResource(statisticsUrl, true, function() {
+                if (!testRunner.isStatisticsPrevalentResource(statisticsUrl))
+                    testFailed("Host did not get set as prevalent resource.");
+                let iframeElement = document.createElement("iframe");
+                iframeElement.setAttribute("sandbox", "allow-storage-access-by-user-activation allow-scripts allow-same-origin allow-modals");
+                iframeElement.onload = runTest;
+                iframeElement.id = "TheIframeThatRequestsStorageAccess";
+                iframeElement.src = "http://localhost:8000/storageAccess/resources/request-storage-access-iframe.html#userShouldGrantAccess,userShouldBeConsulted,policyShouldNotGrantAccess,isNotSameOriginIframe";
+                document.body.appendChild(iframeElement);
+            });
         });
     </script>
 </head>
index 83d2580..a69504c 100644 (file)
@@ -3,6 +3,7 @@
 <head>
     <script src="/js-test-resources/js-test.js"></script>
     <script src="/js-test-resources/ui-helper.js"></script>
+    <script src="/resourceLoadStatistics/resources/util.js"></script>
     <script>
         description("Tests that cross-origin iframe storage access is granted if the iframe is sandboxed, has the allow token, and the user opts in.");
         jsTestIsAsync = true;
@@ -17,7 +18,7 @@
                     testFailed(event.data);
             } else
                 testFailed("Received a message from an unexpected origin: " + event.origin);
-            finishJSTest();
+            setEnableFeature(false, finishJSTest);
         }
 
         function activateElement(elementId) {
                         eventSender.keyDown("escape");
                     else {
                         testFailed("No eventSender.");
-                        finishJSTest();
+                        setEnableFeature(false, finishJSTest);
                     }
                 },
                 function () {
                     testFailed("Promise rejected.");
-                    finishJSTest();
+                    setEnableFeature(false, finishJSTest);
                 }
             );
         }
 
         function runTest() {
-            activateElement("TheIframeThatRequestsStorageAccess");
+            setEnableFeature(true, function() {
+                activateElement("TheIframeThatRequestsStorageAccess");
+            });
         }
     </script>
 </head>
index a2f6d83..4ff7253 100644 (file)
@@ -3,6 +3,7 @@
 <head>
     <script src="/js-test-resources/js-test.js"></script>
     <script src="/js-test-resources/ui-helper.js"></script>
+    <script src="/resourceLoadStatistics/resources/util.js"></script>
     <script>
         description("Tests that cross-origin iframe storage access is denied if the iframe is sandboxed and has the unique origin because it lacks the allow-same-origin token.");
         jsTestIsAsync = true;
@@ -17,7 +18,7 @@
                     testFailed(event.data.replace("FAIL ", ""));
             } else
                 testFailed("Received a message from an unexpected origin: " + event.origin);
-            finishJSTest();
+            setEnableFeature(false, finishJSTest);
         }
 
         function activateElement(elementId) {
                         eventSender.keyDown("escape");
                     else {
                         testFailed("No eventSender.");
-                        finishJSTest();
+                        setEnableFeature(false, finishJSTest);
                     }
                 },
                 function () {
                     testFailed("Promise rejected.");
-                    finishJSTest();
+                    setEnableFeature(false, finishJSTest);
                 }
             );
         }
 
         function runTest() {
-            activateElement("TheIframeThatRequestsStorageAccess");
+            setEnableFeature(true, function() {
+                activateElement("TheIframeThatRequestsStorageAccess");
+            });
         }
     </script>
 </head>
index 09d45e2..4498d07 100644 (file)
@@ -3,6 +3,7 @@
 <head>
     <script src="/js-test-resources/js-test.js"></script>
     <script src="/js-test-resources/ui-helper.js"></script>
+    <script src="/resourceLoadStatistics/resources/util.js"></script>
     <script>
         description("Tests that same-origin iframe storage access is granted if the iframe is not sandboxed.");
         jsTestIsAsync = true;
@@ -17,7 +18,7 @@
                     testFailed(event.data.replace("FAIL ", ""));
             } else
                 testFailed("Received a message from an unexpected origin: " + event.origin);
-            finishJSTest();
+            setEnableFeature(false, finishJSTest);
         }
 
         function activateElement(elementId) {
                         eventSender.keyDown("escape");
                     else {
                         testFailed("No eventSender.");
-                        finishJSTest();
+                        setEnableFeature(false, finishJSTest);
                     }
                 },
                 function () {
                     testFailed("Promise rejected.");
-                    finishJSTest();
+                    setEnableFeature(false, finishJSTest);
                 }
             );
         }
 
         function runTest() {
-            activateElement("TheIframeThatRequestsStorageAccess");
+            setEnableFeature(true, function() {
+                activateElement("TheIframeThatRequestsStorageAccess");
+            });
         }
     </script>
 </head>
index b7eabf9..d87c698 100644 (file)
@@ -3,6 +3,7 @@
 <head>
     <script src="/js-test-resources/js-test.js"></script>
     <script src="/js-test-resources/ui-helper.js"></script>
+    <script src="/resourceLoadStatistics/resources/util.js"></script>
     <script>
         description("Tests that same-origin iframe storage access is granted if the iframe is sandboxed and has the allow token.");
         jsTestIsAsync = true;
@@ -17,7 +18,7 @@
                     testFailed(event.data.replace("FAIL ", ""));
             } else
                 testFailed("Received a message from an unexpected origin: " + event.origin);
-            finishJSTest();
+            setEnableFeature(false, finishJSTest);
         }
 
         function activateElement(elementId) {
                         eventSender.keyDown("escape");
                     else {
                         testFailed("No eventSender.");
-                        finishJSTest();
+                        setEnableFeature(false, finishJSTest);
                     }
                 },
                 function () {
                     testFailed("Promise rejected.");
-                    finishJSTest();
+                    setEnableFeature(false, finishJSTest);
                 }
             );
         }
 
         function runTest() {
-            activateElement("TheIframeThatRequestsStorageAccess");
+            setEnableFeature(true, function() {
+                activateElement("TheIframeThatRequestsStorageAccess");
+            });
         }
     </script>
 </head>
index 6a497da..913f95b 100644 (file)
     var requestStorageAccessResolved;
 
     function makeRequestWithUserGesture() {
-        var promise = document.requestStorageAccess();
-        promise.then(
-            function () {
-                requestStorageAccessResolved = true;
-                continueAfterRequestWithUserGesture();
-            },
-            function () {
-                requestStorageAccessResolved = false;
-                continueAfterRequestWithUserGesture();
-            }
-        );
+        setEnableFeature(true, function() {
+            var promise = document.requestStorageAccess();
+            promise.then(
+                function () {
+                    requestStorageAccessResolved = true;
+                    continueAfterRequestWithUserGesture();
+                },
+                function () {
+                    requestStorageAccessResolved = false;
+                    continueAfterRequestWithUserGesture();
+                }
+            );
+        });
     }
 
     function continueAfterRequestWithUserGesture() {
index adc72db..1473c2a 100644 (file)
@@ -1,3 +1,91 @@
+2019-03-11  John Wilander  <wilander@apple.com>
+
+        Resource Load Statistics: Make it possible exclude localhost from classification
+        https://bugs.webkit.org/show_bug.cgi?id=195474
+        <rdar://problem/47520577>
+
+        Reviewed by Brent Fulgham.
+
+        This patch allows for localhost to be excluded from classification and
+        treatment as a prevalent resource.
+
+        * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:
+        (WebKit::ResourceLoadStatisticsDatabaseStore::ResourceLoadStatisticsDatabaseStore):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::reclassifyResources):
+            Makes use of the new ResourceLoadStatisticsMemoryStore::shouldSkip().
+        (WebKit::ResourceLoadStatisticsDatabaseStore::setPrevalentResource):
+            Makes use of the new ResourceLoadStatisticsMemoryStore::shouldSkip().
+        (WebKit::ResourceLoadStatisticsDatabaseStore::isPrevalentResource const):
+            Makes use of the new ResourceLoadStatisticsMemoryStore::shouldSkip().
+        (WebKit::ResourceLoadStatisticsDatabaseStore::isVeryPrevalentResource const):
+            Makes use of the new ResourceLoadStatisticsMemoryStore::shouldSkip().
+        (WebKit::ResourceLoadStatisticsDatabaseStore::setVeryPrevalentResource):
+            Makes use of the new ResourceLoadStatisticsMemoryStore::shouldSkip().
+        * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h:
+        * NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp:
+        (WebKit::ResourceLoadStatisticsMemoryStore::ResourceLoadStatisticsMemoryStore):
+        (WebKit::ResourceLoadStatisticsMemoryStore::classifyPrevalentResources):
+            Makes use of the new ResourceLoadStatisticsMemoryStore::shouldSkip().
+        (WebKit::ResourceLoadStatisticsMemoryStore::setPrevalentResource):
+            Makes use of the new ResourceLoadStatisticsMemoryStore::shouldSkip().
+        (WebKit::ResourceLoadStatisticsMemoryStore::isPrevalentResource const):
+            Makes use of the new ResourceLoadStatisticsMemoryStore::shouldSkip().
+        (WebKit::ResourceLoadStatisticsMemoryStore::isVeryPrevalentResource const):
+            Makes use of the new ResourceLoadStatisticsMemoryStore::shouldSkip().
+        (WebKit::ResourceLoadStatisticsMemoryStore::setVeryPrevalentResource):
+            Makes use of the new ResourceLoadStatisticsMemoryStore::shouldSkip().
+        * NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h:
+        * NetworkProcess/Classifier/ResourceLoadStatisticsStore.cpp:
+        (WebKit::ResourceLoadStatisticsStore::ResourceLoadStatisticsStore):
+            Now takes a ShouldIncludeLocalhost parameter.
+        (WebKit::ResourceLoadStatisticsStore::shouldSkip const):
+            Convenience function, currently supporting the localhost exclusion.
+        (WebKit::ResourceLoadStatisticsStore::setIsRunningTest):
+            Test infrastructure.
+        * NetworkProcess/Classifier/ResourceLoadStatisticsStore.h:
+        * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:
+        (WebKit::WebResourceLoadStatisticsStore::setIsRunningTest):
+            Test infrastructure.
+        (WebKit::WebResourceLoadStatisticsStore::WebResourceLoadStatisticsStore):
+            Now takes a ShouldIncludeLocalhost parameter.
+        * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h:
+            Defines the new ShouldIncludeLocalhost boolean enum.
+        * NetworkProcess/NetworkProcess.cpp:
+        (WebKit::NetworkProcess::setIsRunningResourceLoadStatisticsTest):
+            Test infrastructure.
+        * NetworkProcess/NetworkProcess.h:
+        * NetworkProcess/NetworkProcess.messages.in:
+        * NetworkProcess/NetworkSession.cpp:
+        (WebKit::NetworkSession::setResourceLoadStatisticsEnabled):
+            Forwards the localhost setting to the create function.
+        * NetworkProcess/NetworkSession.h:
+        * NetworkProcess/NetworkSessionCreationParameters.cpp:
+        (WebKit::NetworkSessionCreationParameters::encode const):
+        (WebKit::NetworkSessionCreationParameters::decode):
+        * NetworkProcess/NetworkSessionCreationParameters.h:
+            New parameter called shouldIncludeLocalhostInResourceLoadStatistics.
+        * NetworkProcess/cocoa/NetworkSessionCocoa.mm:
+        (WebKit::NetworkSessionCocoa::NetworkSessionCocoa):
+            Picks up the localhost setting from the parameters.
+        * UIProcess/API/C/WKWebsiteDataStoreRef.cpp:
+        (WKWebsiteDataStoreSetStatisticsIsRunningTest):
+            Test infrastructure.
+        * UIProcess/API/C/WKWebsiteDataStoreRef.h:
+        * UIProcess/Network/NetworkProcessProxy.cpp:
+        (WebKit::NetworkProcessProxy::setIsRunningResourceLoadStatisticsTest):
+            Test infrastructure.
+        * UIProcess/Network/NetworkProcessProxy.h:
+        * UIProcess/WebProcessPool.cpp:
+        (WebKit::WebProcessPool::ensureNetworkProcess):
+            Picks up the localhost setting from the WebsiteDataStore parameters.
+        * UIProcess/WebsiteData/Cocoa/WebsiteDataStoreCocoa.mm:
+        (WebKit::WebsiteDataStore::parameters):
+            Makes sure Safari does not exclude localhost.
+        * UIProcess/WebsiteData/WebsiteDataStore.cpp:
+        (WebKit::WebsiteDataStore::setIsRunningResourceLoadStatisticsTest):
+            Test infrastructure.
+        * UIProcess/WebsiteData/WebsiteDataStore.h:
+
 2019-03-11  Alex Christensen  <achristensen@webkit.org>
 
         Unreviewed, rolling out r242698.
index 4fcce1b..3b6ee5c 100644 (file)
@@ -158,8 +158,8 @@ constexpr auto createSubresourceUniqueRedirectsFrom = "CREATE TABLE SubresourceU
     "FOREIGN KEY(subresourceDomainID) REFERENCES ObservedDomains(domainID) ON DELETE CASCADE, "
     "FOREIGN KEY(fromDomainID) REFERENCES ObservedDomains(domainID) ON DELETE CASCADE);"_s;
     
-ResourceLoadStatisticsDatabaseStore::ResourceLoadStatisticsDatabaseStore(WebResourceLoadStatisticsStore& store, WorkQueue& workQueue, const String& storageDirectoryPath)
-    : ResourceLoadStatisticsStore(store, workQueue)
+ResourceLoadStatisticsDatabaseStore::ResourceLoadStatisticsDatabaseStore(WebResourceLoadStatisticsStore& store, WorkQueue& workQueue, ShouldIncludeLocalhost shouldIncludeLocalhost, const String& storageDirectoryPath)
+    : ResourceLoadStatisticsStore(store, workQueue, shouldIncludeLocalhost)
     , m_storageDirectoryPath(storageDirectoryPath + "/observations.db")
     , m_observedDomainCount(m_database, observedDomainCountQuery)
     , m_insertObservedDomainStatement(m_database, insertObservedDomainQuery)
@@ -689,6 +689,9 @@ void ResourceLoadStatisticsDatabaseStore::reclassifyResources()
     auto notVeryPrevalentResources = findNotVeryPrevalentResources();
 
     for (auto& resourceStatistic : notVeryPrevalentResources.values()) {
+        if (shouldSkip(resourceStatistic.registerableDomain))
+            continue;
+
         auto newPrevalence = classifier().calculateResourcePrevalence(resourceStatistic.subresourceUnderTopFrameDomainsCount, resourceStatistic.subresourceUniqueRedirectsToCount, resourceStatistic.subframeUnderTopFrameDomainsCount, resourceStatistic.topFrameUniqueRedirectsToCount, resourceStatistic.prevalence);
         if (newPrevalence != resourceStatistic.prevalence)
             setPrevalentResource(resourceStatistic.registerableDomain, newPrevalence);
@@ -1066,6 +1069,8 @@ bool ResourceLoadStatisticsDatabaseStore::hasHadUserInteraction(const Registrabl
 void ResourceLoadStatisticsDatabaseStore::setPrevalentResource(const RegistrableDomain& domain, ResourceLoadPrevalence newPrevalence)
 {
     ASSERT(!RunLoop::isMain());
+    if (shouldSkip(domain))
+        return;
 
     if (m_updatePrevalentResourceStatement.bindInt(1, 1) != SQLITE_OK
         || m_updatePrevalentResourceStatement.bindText(2, domain.string()) != SQLITE_OK
@@ -1139,6 +1144,9 @@ bool ResourceLoadStatisticsDatabaseStore::isPrevalentResource(const RegistrableD
 {
     ASSERT(!RunLoop::isMain());
 
+    if (shouldSkip(domain))
+        return false;
+
     return predicateValueForDomain(m_isPrevalentResourceStatement, domain);
 }
 
@@ -1146,6 +1154,9 @@ bool ResourceLoadStatisticsDatabaseStore::isVeryPrevalentResource(const Registra
 {
     ASSERT(!RunLoop::isMain());
 
+    if (shouldSkip(domain))
+        return false;
+
     return predicateValueForDomain(m_isVeryPrevalentResourceStatement, domain);
 }
 
@@ -1584,6 +1595,9 @@ void ResourceLoadStatisticsDatabaseStore::setPrevalentResource(const Registrable
 {
     ASSERT(!RunLoop::isMain());
 
+    if (shouldSkip(domain))
+        return;
+
     ensureResourceStatisticsForRegistrableDomain(domain);
     setPrevalentResource(domain, ResourceLoadPrevalence::High);
 }
@@ -1592,6 +1606,9 @@ void ResourceLoadStatisticsDatabaseStore::setVeryPrevalentResource(const Registr
 {
     ASSERT(!RunLoop::isMain());
 
+    if (shouldSkip(domain))
+        return;
+    
     ensureResourceStatisticsForRegistrableDomain(domain);
     setPrevalentResource(domain, ResourceLoadPrevalence::VeryHigh);
 }
index bd95036..f6bc8bb 100644 (file)
@@ -50,7 +50,7 @@ class ResourceLoadStatisticsMemoryStore;
 // This is always constructed / used / destroyed on the WebResourceLoadStatisticsStore's statistics queue.
 class ResourceLoadStatisticsDatabaseStore final : public ResourceLoadStatisticsStore {
 public:
-    ResourceLoadStatisticsDatabaseStore(WebResourceLoadStatisticsStore&, WorkQueue&, const String& storageDirectoryPath);
+    ResourceLoadStatisticsDatabaseStore(WebResourceLoadStatisticsStore&, WorkQueue&, ShouldIncludeLocalhost, const String& storageDirectoryPath);
 
     void populateFromMemoryStore(const ResourceLoadStatisticsMemoryStore&);
 
index 9efbbd7..fbf9774 100644 (file)
@@ -67,8 +67,8 @@ static void pruneResources(HashMap<RegistrableDomain, ResourceLoadStatistics>& s
         statisticsMap.remove(statisticsToPrune[i].domain);
 }
 
-ResourceLoadStatisticsMemoryStore::ResourceLoadStatisticsMemoryStore(WebResourceLoadStatisticsStore& store, WorkQueue& workQueue)
-    : ResourceLoadStatisticsStore(store, workQueue)
+ResourceLoadStatisticsMemoryStore::ResourceLoadStatisticsMemoryStore(WebResourceLoadStatisticsStore& store, WorkQueue& workQueue, ShouldIncludeLocalhost shouldIncludeLocalhost)
+    : ResourceLoadStatisticsStore(store, workQueue, shouldIncludeLocalhost)
 {
     ASSERT(!RunLoop::isMain());
 
@@ -175,6 +175,8 @@ bool ResourceLoadStatisticsMemoryStore::isPrevalentDueToDebugMode(ResourceLoadSt
 void ResourceLoadStatisticsMemoryStore::classifyPrevalentResources()
 {
     for (auto& resourceStatistic : m_resourceStatisticsMap.values()) {
+        if (shouldSkip(resourceStatistic.registrableDomain))
+            continue;
         if (isPrevalentDueToDebugMode(resourceStatistic))
             setPrevalentResource(resourceStatistic, ResourceLoadPrevalence::High);
         else if (!resourceStatistic.isVeryPrevalentResource) {
@@ -458,6 +460,9 @@ void ResourceLoadStatisticsMemoryStore::setPrevalentResource(ResourceLoadStatist
 {
     ASSERT(!RunLoop::isMain());
 
+    if (shouldSkip(resourceStatistic.registrableDomain))
+        return;
+
     resourceStatistic.isPrevalentResource = true;
     resourceStatistic.isVeryPrevalentResource = newPrevalence == ResourceLoadPrevalence::VeryHigh;
     HashSet<RegistrableDomain> domainsThatHaveRedirectedTo;
@@ -486,6 +491,9 @@ bool ResourceLoadStatisticsMemoryStore::isPrevalentResource(const RegistrableDom
 {
     ASSERT(!RunLoop::isMain());
 
+    if (shouldSkip(domain))
+        return false;
+
     auto mapEntry = m_resourceStatisticsMap.find(domain);
     return mapEntry == m_resourceStatisticsMap.end() ? false : mapEntry->value.isPrevalentResource;
 }
@@ -494,6 +502,9 @@ bool ResourceLoadStatisticsMemoryStore::isVeryPrevalentResource(const Registrabl
 {
     ASSERT(!RunLoop::isMain());
 
+    if (shouldSkip(domain))
+        return false;
+    
     auto mapEntry = m_resourceStatisticsMap.find(domain);
     return mapEntry == m_resourceStatisticsMap.end() ? false : mapEntry->value.isPrevalentResource && mapEntry->value.isVeryPrevalentResource;
 }
@@ -850,6 +861,9 @@ void ResourceLoadStatisticsMemoryStore::setPrevalentResource(const RegistrableDo
 {
     ASSERT(!RunLoop::isMain());
 
+    if (shouldSkip(domain))
+        return;
+    
     auto& resourceStatistic = ensureResourceStatisticsForRegistrableDomain(domain);
     setPrevalentResource(resourceStatistic, ResourceLoadPrevalence::High);
 }
@@ -858,6 +872,9 @@ void ResourceLoadStatisticsMemoryStore::setVeryPrevalentResource(const Registrab
 {
     ASSERT(!RunLoop::isMain());
 
+    if (shouldSkip(domain))
+        return;
+    
     auto& resourceStatistic = ensureResourceStatisticsForRegistrableDomain(domain);
     setPrevalentResource(resourceStatistic, ResourceLoadPrevalence::VeryHigh);
 }
index f7fb162..58d3da3 100644 (file)
@@ -47,7 +47,7 @@ class ResourceLoadStatisticsPersistentStorage;
 // This is always constructed / used / destroyed on the WebResourceLoadStatisticsStore's statistics queue.
 class ResourceLoadStatisticsMemoryStore final : public ResourceLoadStatisticsStore {
 public:
-    ResourceLoadStatisticsMemoryStore(WebResourceLoadStatisticsStore&, WorkQueue&);
+    ResourceLoadStatisticsMemoryStore(WebResourceLoadStatisticsStore&, WorkQueue&, ShouldIncludeLocalhost);
 
     void setPersistentStorage(ResourceLoadStatisticsPersistentStorage&);
 
index 04c4790..4fd484e 100644 (file)
@@ -101,9 +101,10 @@ bool OperatingDate::operator<=(const OperatingDate& other) const
     return secondsSinceEpoch() <= other.secondsSinceEpoch();
 }
 
-ResourceLoadStatisticsStore::ResourceLoadStatisticsStore(WebResourceLoadStatisticsStore& store, WorkQueue& workQueue)
+ResourceLoadStatisticsStore::ResourceLoadStatisticsStore(WebResourceLoadStatisticsStore& store, WorkQueue& workQueue, ShouldIncludeLocalhost shouldIncludeLocalhost)
     : m_store(store)
     , m_workQueue(workQueue)
+    , m_shouldIncludeLocalhost(shouldIncludeLocalhost)
 {
     ASSERT(!RunLoop::isMain());
 
@@ -131,6 +132,19 @@ void ResourceLoadStatisticsStore::setNotifyPagesWhenDataRecordsWereScanned(bool
     m_parameters.shouldNotifyPagesWhenDataRecordsWereScanned = value;
 }
 
+bool ResourceLoadStatisticsStore::shouldSkip(const RegistrableDomain& domain) const
+{
+    ASSERT(!RunLoop::isMain());
+    return !(parameters().isRunningTest)
+    && m_shouldIncludeLocalhost == ShouldIncludeLocalhost::No && domain.string() == "localhost";
+}
+
+void ResourceLoadStatisticsStore::setIsRunningTest(bool value)
+{
+    ASSERT(!RunLoop::isMain());
+    m_parameters.isRunningTest = value;
+}
+
 void ResourceLoadStatisticsStore::setShouldClassifyResourcesBeforeDataRecordsRemoval(bool value)
 {
     ASSERT(!RunLoop::isMain());
index 23eea13..aff575d 100644 (file)
@@ -146,6 +146,8 @@ public:
     virtual void calculateAndSubmitTelemetry() const = 0;
 
     void setNotifyPagesWhenDataRecordsWereScanned(bool);
+    void setIsRunningTest(bool);
+    bool shouldSkip(const RegistrableDomain&) const;
     void setShouldClassifyResourcesBeforeDataRecordsRemoval(bool);
     void setShouldSubmitTelemetry(bool);
     void setTimeToLiveUserInteraction(Seconds);
@@ -184,7 +186,7 @@ protected:
     static Vector<OperatingDate> mergeOperatingDates(const Vector<OperatingDate>& existingDates, Vector<OperatingDate>&& newDates);
     static void debugLogDomainsInBatches(const char* action, const Vector<RegistrableDomain>& domains);
 
-    ResourceLoadStatisticsStore(WebResourceLoadStatisticsStore&, WorkQueue&);
+    ResourceLoadStatisticsStore(WebResourceLoadStatisticsStore&, WorkQueue&, ShouldIncludeLocalhost);
 
     bool hasStatisticsExpired(const ResourceLoadStatistics&) const;
     bool hasStatisticsExpired(WallTime mostRecentUserInteractionTime) const;
@@ -216,6 +218,7 @@ protected:
         bool shouldNotifyPagesWhenDataRecordsWereScanned { false };
         bool shouldClassifyResourcesBeforeDataRecordsRemoval { true };
         bool shouldSubmitTelemetry { true };
+        bool isRunningTest { false };
     };
     const Parameters& parameters() const { return m_parameters; }
     const Vector<OperatingDate>& operatingDates() const { return m_operatingDates; }
@@ -265,6 +268,7 @@ private:
     bool m_debugModeEnabled { false };
     bool m_storageAccessPromptsEnabled { false };
     bool m_dataRecordsBeingRemoved { false };
+    ShouldIncludeLocalhost m_shouldIncludeLocalhost { ShouldIncludeLocalhost::Yes };
 };
 
 } // namespace WebKit
index 48ff64f..bb3c00d 100644 (file)
@@ -108,6 +108,18 @@ void WebResourceLoadStatisticsStore::setNotifyPagesWhenDataRecordsWereScanned(bo
     });
 }
 
+void WebResourceLoadStatisticsStore::setIsRunningTest(bool value, CompletionHandler<void()>&& completionHandler)
+{
+    ASSERT(RunLoop::isMain());
+    
+    postTask([this, value, completionHandler = WTFMove(completionHandler)]() mutable {
+        if (m_statisticsStore)
+            m_statisticsStore->setIsRunningTest(value);
+        
+        postTaskReply(WTFMove(completionHandler));
+    });
+}
+
 void WebResourceLoadStatisticsStore::setShouldClassifyResourcesBeforeDataRecordsRemoval(bool value, CompletionHandler<void()>&& completionHandler)
 {
     ASSERT(RunLoop::isMain());
@@ -138,23 +150,23 @@ void WebResourceLoadStatisticsStore::setNotifyPagesWhenTelemetryWasCaptured(bool
     completionHandler();
 }
 
-WebResourceLoadStatisticsStore::WebResourceLoadStatisticsStore(NetworkSession& networkSession, const String& resourceLoadStatisticsDirectory)
+WebResourceLoadStatisticsStore::WebResourceLoadStatisticsStore(NetworkSession& networkSession, const String& resourceLoadStatisticsDirectory, ShouldIncludeLocalhost shouldIncludeLocalhost)
     : m_networkSession(makeWeakPtr(networkSession))
     , m_statisticsQueue(WorkQueue::create("WebResourceLoadStatisticsStore Process Data Queue", WorkQueue::Type::Serial, WorkQueue::QOS::Utility))
     , m_dailyTasksTimer(RunLoop::main(), this, &WebResourceLoadStatisticsStore::performDailyTasks)
 {
     ASSERT(RunLoop::isMain());
     
-    postTask([this, resourceLoadStatisticsDirectory = resourceLoadStatisticsDirectory.isolatedCopy()] {
+    postTask([this, resourceLoadStatisticsDirectory = resourceLoadStatisticsDirectory.isolatedCopy(), shouldIncludeLocalhost] {
         if (RuntimeEnabledFeatures::sharedFeatures().isITPDatabaseEnabled()) {
-            m_statisticsStore = std::make_unique<ResourceLoadStatisticsDatabaseStore>(*this, m_statisticsQueue, resourceLoadStatisticsDirectory);
+            m_statisticsStore = std::make_unique<ResourceLoadStatisticsDatabaseStore>(*this, m_statisticsQueue, shouldIncludeLocalhost, resourceLoadStatisticsDirectory);
 
-            auto memoryStore = std::make_unique<ResourceLoadStatisticsMemoryStore>(*this, m_statisticsQueue);
+            auto memoryStore = std::make_unique<ResourceLoadStatisticsMemoryStore>(*this, m_statisticsQueue, shouldIncludeLocalhost);
             auto persistentStore = std::make_unique<ResourceLoadStatisticsPersistentStorage>(*memoryStore, m_statisticsQueue, resourceLoadStatisticsDirectory);
 
             downcast<ResourceLoadStatisticsDatabaseStore>(*m_statisticsStore.get()).populateFromMemoryStore(*memoryStore);
         } else {
-            m_statisticsStore = std::make_unique<ResourceLoadStatisticsMemoryStore>(*this, m_statisticsQueue);
+            m_statisticsStore = std::make_unique<ResourceLoadStatisticsMemoryStore>(*this, m_statisticsQueue, shouldIncludeLocalhost);
             m_persistentStorage = std::make_unique<ResourceLoadStatisticsPersistentStorage>(downcast<ResourceLoadStatisticsMemoryStore>(*m_statisticsStore), m_statisticsQueue, resourceLoadStatisticsDirectory);
         }
 
index eafea3d..8147e66 100644 (file)
@@ -58,6 +58,7 @@ class ResourceLoadStatisticsPersistentStorage;
 class WebFrameProxy;
 class WebProcessProxy;
 enum class ShouldGrandfatherStatistics : bool;
+enum class ShouldIncludeLocalhost : bool { No, Yes };
 
 class WebResourceLoadStatisticsStore final : public ThreadSafeRefCounted<WebResourceLoadStatisticsStore, WTF::DestructionThread::Main>, public IPC::MessageReceiver {
 public:
@@ -77,9 +78,9 @@ public:
     using PageID = uint64_t;
     using FrameID = uint64_t;
 
-    static Ref<WebResourceLoadStatisticsStore> create(NetworkSession& networkSession, const String& resourceLoadStatisticsDirectory)
+    static Ref<WebResourceLoadStatisticsStore> create(NetworkSession& networkSession, const String& resourceLoadStatisticsDirectory, ShouldIncludeLocalhost shouldIncludeLocalhost)
     {
-        return adoptRef(*new WebResourceLoadStatisticsStore(networkSession, resourceLoadStatisticsDirectory));
+        return adoptRef(*new WebResourceLoadStatisticsStore(networkSession, resourceLoadStatisticsDirectory, shouldIncludeLocalhost));
     }
 
     ~WebResourceLoadStatisticsStore();
@@ -128,6 +129,7 @@ public:
     void isGrandfathered(const RegistrableDomain&, CompletionHandler<void(bool)>&&);
     void removePrevalentDomains(const Vector<RegistrableDomain>&);
     void setNotifyPagesWhenDataRecordsWereScanned(bool, CompletionHandler<void()>&&);
+    void setIsRunningTest(bool, CompletionHandler<void()>&&);
     void setSubframeUnderTopFrameDomain(const SubFrameDomain&, const TopFrameDomain&, CompletionHandler<void()>&&);
     void setSubresourceUnderTopFrameDomain(const SubResourceDomain&, const TopFrameDomain&, CompletionHandler<void()>&&);
     void setSubresourceUniqueRedirectTo(const SubResourceDomain&, const RedirectedToDomain&, CompletionHandler<void()>&&);
@@ -171,7 +173,7 @@ public:
     void notifyPageStatisticsTelemetryFinished(unsigned totalPrevalentResources, unsigned totalPrevalentResourcesWithUserInteraction, unsigned top3SubframeUnderTopFrameOrigins) const;
 
 private:
-    explicit WebResourceLoadStatisticsStore(NetworkSession&, const String&);
+    explicit WebResourceLoadStatisticsStore(NetworkSession&, const String&, ShouldIncludeLocalhost);
 
     void postTask(WTF::Function<void()>&&);
     static void postTaskReply(WTF::Function<void()>&&);
index c75b3bc..70d283b 100644 (file)
@@ -786,6 +786,19 @@ void NetworkProcess::setNotifyPagesWhenDataRecordsWereScanned(PAL::SessionID ses
     }
 }
 
+void NetworkProcess::setIsRunningResourceLoadStatisticsTest(PAL::SessionID sessionID, bool value, CompletionHandler<void()>&& completionHandler)
+{
+    if (auto* networkSession = this->networkSession(sessionID)) {
+        if (auto* resourceLoadStatistics = networkSession->resourceLoadStatistics())
+            resourceLoadStatistics->setIsRunningTest(value, WTFMove(completionHandler));
+        else
+            completionHandler();
+    } else {
+        ASSERT_NOT_REACHED();
+        completionHandler();
+    }
+}
+
 void NetworkProcess::setNotifyPagesWhenTelemetryWasCaptured(PAL::SessionID sessionID, bool value, CompletionHandler<void()>&& completionHandler)
 {
     if (auto* networkSession = this->networkSession(sessionID)) {
index 4b5fcc2..f8c556b 100644 (file)
@@ -244,6 +244,7 @@ public:
     void setLastSeen(PAL::SessionID, const RegistrableDomain&, Seconds, CompletionHandler<void()>&&);
     void setMinimumTimeBetweenDataRecordsRemoval(PAL::SessionID, Seconds, CompletionHandler<void()>&&);
     void setNotifyPagesWhenDataRecordsWereScanned(PAL::SessionID, bool value, CompletionHandler<void()>&&);
+    void setIsRunningResourceLoadStatisticsTest(PAL::SessionID, bool value, CompletionHandler<void()>&&);
     void setNotifyPagesWhenTelemetryWasCaptured(PAL::SessionID, bool value, CompletionHandler<void()>&&);
     void setResourceLoadStatisticsEnabled(bool);
     void setResourceLoadStatisticsDebugMode(PAL::SessionID, bool debugMode, CompletionHandler<void()>&&d);
index 2a91188..5bf0842 100644 (file)
@@ -121,6 +121,7 @@ messages -> NetworkProcess LegacyReceiver {
     SetPruneEntriesDownTo(PAL::SessionID sessionID, uint64_t maximumEntryCount) -> () Async
     SetShouldClassifyResourcesBeforeDataRecordsRemoval(PAL::SessionID sessionID, bool value) -> () Async
     SetNotifyPagesWhenDataRecordsWereScanned(PAL::SessionID sessionID, bool value) -> () Async
+    SetIsRunningResourceLoadStatisticsTest(PAL::SessionID sessionID, bool value) -> () Async
     SetNotifyPagesWhenTelemetryWasCaptured(PAL::SessionID sessionID, bool value) -> () Async
     SetResourceLoadStatisticsDebugMode(PAL::SessionID sessionID, bool debugMode) -> () Async
     SetVeryPrevalentResource(PAL::SessionID sessionID, WebCore::RegistrableDomain resourceDomain) -> () Async
index d854113..260732d 100644 (file)
@@ -30,7 +30,6 @@
 #include "NetworkProcess.h"
 #include "NetworkProcessProxyMessages.h"
 #include "WebProcessProxy.h"
-#include "WebResourceLoadStatisticsStore.h"
 #include <WebCore/AdClickAttribution.h>
 #include <WebCore/CookieJar.h>
 #include <WebCore/NetworkStorageSession.h>
@@ -99,8 +98,8 @@ void NetworkSession::setResourceLoadStatisticsEnabled(bool enable)
     // FIXME(193728): Support ResourceLoadStatistics for ephemeral sessions, too.
     if (m_sessionID.isEphemeral())
         return;
-    
-    m_resourceLoadStatistics = WebResourceLoadStatisticsStore::create(*this, m_resourceLoadStatisticsDirectory);
+
+    m_resourceLoadStatistics = WebResourceLoadStatisticsStore::create(*this, m_resourceLoadStatisticsDirectory, m_shouldIncludeLocalhostInResourceLoadStatistics);
 }
 
 void NetworkSession::notifyResourceLoadStatisticsProcessed()
index 8723d63..56617cb 100644 (file)
@@ -25,6 +25,7 @@
 
 #pragma once
 
+#include "WebResourceLoadStatisticsStore.h"
 #include <WebCore/RegistrableDomain.h>
 #include <pal/SessionID.h>
 #include <wtf/HashSet.h>
@@ -91,6 +92,7 @@ protected:
     String m_resourceLoadStatisticsDirectory;
 #if ENABLE(RESOURCE_LOAD_STATISTICS)
     RefPtr<WebResourceLoadStatisticsStore> m_resourceLoadStatistics;
+    ShouldIncludeLocalhost m_shouldIncludeLocalhostInResourceLoadStatistics { ShouldIncludeLocalhost::Yes };
 #endif
     UniqueRef<NetworkAdClickAttribution> m_adClickAttribution;
 };
index 6f5aa55..0812a31 100644 (file)
@@ -79,6 +79,7 @@ void NetworkSessionCreationParameters::encode(IPC::Encoder& encoder) const
     encoder << resourceLoadStatisticsDirectory;
     encoder << resourceLoadStatisticsDirectoryExtensionHandle;
     encoder << enableResourceLoadStatistics;
+    encoder << shouldIncludeLocalhostInResourceLoadStatistics;
 }
 
 Optional<NetworkSessionCreationParameters> NetworkSessionCreationParameters::decode(IPC::Decoder& decoder)
@@ -172,6 +173,11 @@ Optional<NetworkSessionCreationParameters> NetworkSessionCreationParameters::dec
     if (!enableResourceLoadStatistics)
         return WTF::nullopt;
 
+    Optional<bool> shouldIncludeLocalhostInResourceLoadStatistics;
+    decoder >> shouldIncludeLocalhostInResourceLoadStatistics;
+    if (!shouldIncludeLocalhostInResourceLoadStatistics)
+        return WTF::nullopt;
+
     return {{
         sessionID
         , WTFMove(*boundInterfaceIdentifier)
@@ -196,6 +202,7 @@ Optional<NetworkSessionCreationParameters> NetworkSessionCreationParameters::dec
         , WTFMove(*resourceLoadStatisticsDirectory)
         , WTFMove(*resourceLoadStatisticsDirectoryExtensionHandle)
         , WTFMove(*enableResourceLoadStatistics)
+        , WTFMove(*shouldIncludeLocalhostInResourceLoadStatistics)
     }};
 }
 
index 4fc7361..7c5ce70 100644 (file)
@@ -81,6 +81,7 @@ struct NetworkSessionCreationParameters {
     String resourceLoadStatisticsDirectory;
     SandboxExtension::Handle resourceLoadStatisticsDirectoryExtensionHandle;
     bool enableResourceLoadStatistics { false };
+    bool shouldIncludeLocalhostInResourceLoadStatistics { true };
 };
 
 } // namespace WebKit
index 76b62cc..ec3bf41 100644 (file)
@@ -948,6 +948,7 @@ NetworkSessionCocoa::NetworkSessionCocoa(NetworkProcess& networkProcess, Network
 
 #if ENABLE(RESOURCE_LOAD_STATISTICS)
     m_resourceLoadStatisticsDirectory = parameters.resourceLoadStatisticsDirectory;
+    m_shouldIncludeLocalhostInResourceLoadStatistics = parameters.shouldIncludeLocalhostInResourceLoadStatistics ? ShouldIncludeLocalhost::Yes : ShouldIncludeLocalhost::No;
     setResourceLoadStatisticsEnabled(parameters.enableResourceLoadStatistics);
 #endif
 
index 41e5a4e..eda8d22 100644 (file)
@@ -336,6 +336,17 @@ void WKWebsiteDataStoreSetStatisticsNotifyPagesWhenDataRecordsWereScanned(WKWebs
 #endif
 }
 
+void WKWebsiteDataStoreSetStatisticsIsRunningTest(WKWebsiteDataStoreRef dataStoreRef, bool value, void* context, WKWebsiteDataStoreSetStatisticsIsRunningTestFunction callback)
+{
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+    WebKit::toImpl(dataStoreRef)->websiteDataStore().setIsRunningResourceLoadStatisticsTest(value, [context, callback] {
+        callback(context);
+    });
+#else
+    callback(context);
+#endif
+}
+
 void WKWebsiteDataStoreSetStatisticsShouldClassifyResourcesBeforeDataRecordsRemoval(WKWebsiteDataStoreRef dataStoreRef, bool value)
 {
 #if ENABLE(RESOURCE_LOAD_STATISTICS)
index 5b88b67..f5f9a47 100644 (file)
@@ -79,6 +79,8 @@ typedef void (*WKWebsiteDataStoreStatisticsUpdateCookieBlockingFunction)(void* f
 WK_EXPORT void WKWebsiteDataStoreStatisticsUpdateCookieBlocking(WKWebsiteDataStoreRef dataStoreRef, void* context, WKWebsiteDataStoreStatisticsUpdateCookieBlockingFunction completionHandler);
 WK_EXPORT void WKWebsiteDataStoreStatisticsSubmitTelemetry(WKWebsiteDataStoreRef dataStoreRef);
 WK_EXPORT void WKWebsiteDataStoreSetStatisticsNotifyPagesWhenDataRecordsWereScanned(WKWebsiteDataStoreRef dataStoreRef, bool value);
+typedef void (*WKWebsiteDataStoreSetStatisticsIsRunningTestFunction)(void* functionContext);
+WK_EXPORT void WKWebsiteDataStoreSetStatisticsIsRunningTest(WKWebsiteDataStoreRef dataStoreRef, bool value, void* context, WKWebsiteDataStoreSetStatisticsIsRunningTestFunction callback);
 WK_EXPORT void WKWebsiteDataStoreSetStatisticsShouldClassifyResourcesBeforeDataRecordsRemoval(WKWebsiteDataStoreRef dataStoreRef, bool value);
 WK_EXPORT void WKWebsiteDataStoreSetStatisticsNotifyPagesWhenTelemetryWasCaptured(WKWebsiteDataStoreRef dataStoreRef, bool value);
 WK_EXPORT void WKWebsiteDataStoreSetStatisticsMinimumTimeBetweenDataRecordsRemoval(WKWebsiteDataStoreRef dataStoreRef, double seconds);
index c6392c5..bdd9201 100644 (file)
@@ -594,6 +594,16 @@ void NetworkProcessProxy::setNotifyPagesWhenDataRecordsWereScanned(PAL::SessionI
     sendWithAsyncReply(Messages::NetworkProcess::SetNotifyPagesWhenDataRecordsWereScanned(sessionID, value), WTFMove(completionHandler));
 }
 
+void NetworkProcessProxy::setIsRunningResourceLoadStatisticsTest(PAL::SessionID sessionID, bool value, CompletionHandler<void()>&& completionHandler)
+{
+    if (!canSendMessage()) {
+        completionHandler();
+        return;
+    }
+    
+    sendWithAsyncReply(Messages::NetworkProcess::SetIsRunningResourceLoadStatisticsTest(sessionID, value), WTFMove(completionHandler));
+}
+
 void NetworkProcessProxy::setSubframeUnderTopFrameDomain(PAL::SessionID sessionID, const RegistrableDomain& subFrameDomain, const RegistrableDomain& topFrameDomain, CompletionHandler<void()>&& completionHandler)
 {
     if (!canSendMessage()) {
index c127a0c..69470fe 100644 (file)
@@ -113,6 +113,7 @@ public:
     void setCacheMaxAgeCap(PAL::SessionID, Seconds, CompletionHandler<void()>&&);
     void setGrandfathered(PAL::SessionID, const RegistrableDomain&, bool isGrandfathered, CompletionHandler<void()>&&);
     void setNotifyPagesWhenDataRecordsWereScanned(PAL::SessionID, bool, CompletionHandler<void()>&&);
+    void setIsRunningResourceLoadStatisticsTest(PAL::SessionID, bool, CompletionHandler<void()>&&);
     void setNotifyPagesWhenTelemetryWasCaptured(PAL::SessionID, bool, CompletionHandler<void()>&&);
     void setSubframeUnderTopFrameDomain(PAL::SessionID, const SubFrameDomain&, const TopFrameDomain&, CompletionHandler<void()>&&);
     void setSubresourceUnderTopFrameDomain(PAL::SessionID, const SubResourceDomain&, const TopFrameDomain&, CompletionHandler<void()>&&);
index cd6a5a3..6d8fe86 100644 (file)
@@ -574,12 +574,17 @@ NetworkProcessProxy& WebProcessPool::ensureNetworkProcess(WebsiteDataStore* with
     SandboxExtension::createHandleForReadWriteDirectory(parameters.defaultDataStoreParameters.networkSessionParameters.resourceLoadStatisticsDirectory, parameters.defaultDataStoreParameters.networkSessionParameters.resourceLoadStatisticsDirectoryExtensionHandle);
 
     bool enableResourceLoadStatistics = false;
-    if (withWebsiteDataStore)
+    bool shouldIncludeLocalhost = true;
+    if (withWebsiteDataStore) {
         enableResourceLoadStatistics = withWebsiteDataStore->resourceLoadStatisticsEnabled();
-    else if (m_websiteDataStore)
+        shouldIncludeLocalhost = withWebsiteDataStore->parameters().networkSessionParameters.shouldIncludeLocalhostInResourceLoadStatistics;
+    } else if (m_websiteDataStore) {
         enableResourceLoadStatistics = m_websiteDataStore->resourceLoadStatisticsEnabled();
+        shouldIncludeLocalhost = m_websiteDataStore->websiteDataStore().parameters().networkSessionParameters.shouldIncludeLocalhostInResourceLoadStatistics;
+    }
 
     parameters.defaultDataStoreParameters.networkSessionParameters.enableResourceLoadStatistics = enableResourceLoadStatistics;
+    parameters.defaultDataStoreParameters.networkSessionParameters.shouldIncludeLocalhostInResourceLoadStatistics = shouldIncludeLocalhost;
 
     // Add any platform specific parameters
     platformInitializeNetworkProcess(parameters);
index 3acf50c..a285940 100644 (file)
@@ -102,7 +102,8 @@ WebsiteDataStoreParameters WebsiteDataStore::parameters()
         WTFMove(httpsProxy),
         WTFMove(resourceLoadStatisticsDirectory),
         WTFMove(resourceLoadStatisticsDirectoryHandle),
-        false
+        false,
+        isSafari
     };
     finalizeApplicationIdentifiers();
 
index 8f8f39f..3aa9c23 100644 (file)
@@ -1608,6 +1608,14 @@ void WebsiteDataStore::setNotifyPagesWhenDataRecordsWereScanned(bool value, Comp
         processPool->ensureNetworkProcess().setNotifyPagesWhenDataRecordsWereScanned(m_sessionID, value, [processPool, callbackAggregator = callbackAggregator.copyRef()] { });
 }
 
+void WebsiteDataStore::setIsRunningResourceLoadStatisticsTest(bool value, CompletionHandler<void()>&& completionHandler)
+{
+    auto callbackAggregator = CallbackAggregator::create(WTFMove(completionHandler));
+    
+    for (auto& processPool : processPools())
+        processPool->ensureNetworkProcess().setIsRunningResourceLoadStatisticsTest(m_sessionID, value, [processPool, callbackAggregator = callbackAggregator.copyRef()] { });
+}
+
 void WebsiteDataStore::setNotifyPagesWhenTelemetryWasCaptured(bool value, CompletionHandler<void()>&& completionHandler)
 {
     auto callbackAggregator = CallbackAggregator::create(WTFMove(completionHandler));
index f6149db..c89f29e 100644 (file)
@@ -147,6 +147,7 @@ public:
     void setGrandfatheringTime(Seconds, CompletionHandler<void()>&&);
     void setLastSeen(const URL&, Seconds, CompletionHandler<void()>&&);
     void setNotifyPagesWhenDataRecordsWereScanned(bool, CompletionHandler<void()>&&);
+    void setIsRunningResourceLoadStatisticsTest(bool, CompletionHandler<void()>&&);
     void setPruneEntriesDownTo(size_t, CompletionHandler<void()>&&);
     void setSubframeUnderTopFrameDomain(const URL& subframe, const URL& topFrame, CompletionHandler<void()>&&);
     void setSubresourceUnderTopFrameDomain(const URL& subresource, const URL& topFrame, CompletionHandler<void()>&&);
index 514ded6..85ac141 100644 (file)
@@ -1,3 +1,28 @@
+2019-03-11  John Wilander  <wilander@apple.com>
+
+        Resource Load Statistics: Make it possible exclude localhost from classification
+        https://bugs.webkit.org/show_bug.cgi?id=195474
+        <rdar://problem/47520577>
+
+        Reviewed by Brent Fulgham.
+
+        This patch allows for localhost to be excluded from classification and
+        treatment as a prevalent resource.
+
+        The WebKit Tools change adds a new function called
+        testRunner.setStatisticsIsRunningTest() which can be used to control this
+        behavior.
+
+        * WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl:
+        * WebKitTestRunner/InjectedBundle/TestRunner.cpp:
+        (WTR::TestRunner::setStatisticsIsRunningTest):
+        * WebKitTestRunner/InjectedBundle/TestRunner.h:
+        * WebKitTestRunner/TestController.cpp:
+        (WTR::TestController::setStatisticsIsRunningTest):
+        * WebKitTestRunner/TestController.h:
+        * WebKitTestRunner/TestInvocation.cpp:
+        (WTR::TestInvocation::didReceiveSynchronousMessageFromInjectedBundle):
+
 2019-03-11  Aakash Jain  <aakash_jain@apple.com>
 
         [ews-build] Disable waterfall and console view for ews-build.webkit.org
index c253289..6b61114 100644 (file)
@@ -309,6 +309,7 @@ interface TestRunner {
     void statisticsUpdateCookieBlocking(object completionHandler);
     void statisticsSubmitTelemetry();
     void setStatisticsNotifyPagesWhenDataRecordsWereScanned(boolean value);
+    void setStatisticsIsRunningTest(boolean value);
     void setStatisticsShouldClassifyResourcesBeforeDataRecordsRemoval(boolean value);
     void setStatisticsNotifyPagesWhenTelemetryWasCaptured(boolean value);
     void setStatisticsMinimumTimeBetweenDataRecordsRemoval(double seconds);
index b79d08a..db18e04 100644 (file)
@@ -1947,6 +1947,13 @@ void TestRunner::setStatisticsNotifyPagesWhenDataRecordsWereScanned(bool value)
     WKBundlePostSynchronousMessage(InjectedBundle::singleton().bundle(), messageName.get(), messageBody.get(), nullptr);
 }
 
+void TestRunner::setStatisticsIsRunningTest(bool value)
+{
+    WKRetainPtr<WKStringRef> messageName(AdoptWK, WKStringCreateWithUTF8CString("StatisticsSetIsRunningTest"));
+    WKRetainPtr<WKBooleanRef> messageBody(AdoptWK, WKBooleanCreate(value));
+    WKBundlePostSynchronousMessage(InjectedBundle::singleton().bundle(), messageName.get(), messageBody.get(), nullptr);
+}
+
 void TestRunner::setStatisticsShouldClassifyResourcesBeforeDataRecordsRemoval(bool value)
 {
     WKRetainPtr<WKStringRef> messageName(AdoptWK, WKStringCreateWithUTF8CString("StatisticsShouldClassifyResourcesBeforeDataRecordsRemoval"));
index 8dfcf6c..32bb6d1 100644 (file)
@@ -418,6 +418,7 @@ public:
     void setStatisticsTopFrameUniqueRedirectFrom(JSStringRef hostName, JSStringRef hostNameRedirectedFrom);
     void setStatisticsTimeToLiveUserInteraction(double seconds);
     void setStatisticsNotifyPagesWhenDataRecordsWereScanned(bool);
+    void setStatisticsIsRunningTest(bool);
     void setStatisticsShouldClassifyResourcesBeforeDataRecordsRemoval(bool);
     void setStatisticsNotifyPagesWhenTelemetryWasCaptured(bool value);
     void setStatisticsMinimumTimeBetweenDataRecordsRemoval(double);
index 08964cd..2dfab59 100644 (file)
@@ -3288,6 +3288,14 @@ void TestController::setStatisticsNotifyPagesWhenDataRecordsWereScanned(bool val
     WKWebsiteDataStoreSetStatisticsNotifyPagesWhenDataRecordsWereScanned(dataStore, value);
 }
 
+void TestController::setStatisticsIsRunningTest(bool value)
+{
+    auto* dataStore = WKContextGetWebsiteDataStore(platformContext());
+    ResourceStatisticsCallbackContext context(*this);
+    WKWebsiteDataStoreSetStatisticsIsRunningTest(dataStore, value, &context, resourceStatisticsVoidResultCallback);
+    runUntil(context.done, noTimeout);
+}
+
 void TestController::setStatisticsShouldClassifyResourcesBeforeDataRecordsRemoval(bool value)
 {
     auto* dataStore = WKContextGetWebsiteDataStore(platformContext());
index 3abe2db..3fa8764 100644 (file)
@@ -230,6 +230,7 @@ public:
     void statisticsUpdateCookieBlocking();
     void statisticsSubmitTelemetry();
     void setStatisticsNotifyPagesWhenDataRecordsWereScanned(bool);
+    void setStatisticsIsRunningTest(bool);
     void setStatisticsShouldClassifyResourcesBeforeDataRecordsRemoval(bool);
     void setStatisticsNotifyPagesWhenTelemetryWasCaptured(bool value);
     void setStatisticsMinimumTimeBetweenDataRecordsRemoval(double);
index 7732918..da9e1f4 100644 (file)
@@ -1333,6 +1333,13 @@ WKRetainPtr<WKTypeRef> TestInvocation::didReceiveSynchronousMessageFromInjectedB
         return nullptr;
     }
 
+    if (WKStringIsEqualToUTF8CString(messageName, "StatisticsSetIsRunningTest")) {
+        ASSERT(WKGetTypeID(messageBody) == WKBooleanGetTypeID());
+        WKBooleanRef value = static_cast<WKBooleanRef>(messageBody);
+        TestController::singleton().setStatisticsIsRunningTest(WKBooleanGetValue(value));
+        return nullptr;
+    }
+    
     if (WKStringIsEqualToUTF8CString(messageName, "StatisticsNotifyPagesWhenTelemetryWasCaptured")) {
         ASSERT(WKGetTypeID(messageBody) == WKBooleanGetTypeID());
         WKBooleanRef value = static_cast<WKBooleanRef>(messageBody);