Resource Load Statistics: Count third-party script loads under top frame
authorwilander@apple.com <wilander@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 19 Nov 2019 20:35:21 +0000 (20:35 +0000)
committerwilander@apple.com <wilander@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 19 Nov 2019 20:35:21 +0000 (20:35 +0000)
https://bugs.webkit.org/show_bug.cgi?id=204262
<rdar://problem/57244945>

Reviewed by Alex Christensen.

Source/WebCore:

Third-party scripts running in the first-party context are a significant privacy
and security risk. This change captures the number of such script loads which will
allow ITP to take action.

Tests: http/tests/resourceLoadStatistics/count-third-party-script-import-in-worker-database.html
       http/tests/resourceLoadStatistics/count-third-party-script-import-in-worker.html
       http/tests/resourceLoadStatistics/count-third-party-script-loads-database.html
       http/tests/resourceLoadStatistics/count-third-party-script-loads.html

* loader/FrameLoader.cpp:
(WebCore::FrameLoader::loadResourceSynchronously):
    Now sends the ResourceLoadObserver::FetchDestinationIsScriptLike parameter to
    ResourceLoadObserver::logSubresourceLoading().
* loader/ResourceLoadObserver.h:
(WebCore::ResourceLoadObserver::logSubresourceLoading):
    Now takes a FetchDestinationIsScriptLike parameter.
* loader/ResourceLoadStatistics.cpp:
(WebCore::ResourceLoadStatistics::encode const):
(WebCore::ResourceLoadStatistics::decode):
(WebCore::ResourceLoadStatistics::toString const):
    Output of the new topFrameLoadedThirdPartyScripts category.
    Removed the lastSeen output since it may differ between test runs.
(WebCore::ResourceLoadStatistics::merge):
    Handling of the new topFrameLoadedThirdPartyScripts category.
* loader/ResourceLoadStatistics.h:
    Added the new topFrameLoadedThirdPartyScripts category.
* loader/SubresourceLoader.cpp:
(WebCore::SubresourceLoader::willSendRequestInternal):
    Now sends the ResourceLoadObserver::FetchDestinationIsScriptLike parameter to
    ResourceLoadObserver::logSubresourceLoading().

Source/WebKit:

Third-party scripts running in the first-party context are a significant privacy
and security risk. This change captures the number of such script loads which will
allow ITP to take action.

* NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:
(WebKit::ResourceLoadStatisticsDatabaseStore::ResourceLoadStatisticsDatabaseStore):
(WebKit::ResourceLoadStatisticsDatabaseStore::createUniqueIndices):
(WebKit::ResourceLoadStatisticsDatabaseStore::createSchema):
(WebKit::ResourceLoadStatisticsDatabaseStore::prepareStatements):
(WebKit::ResourceLoadStatisticsDatabaseStore::insertDomainRelationships):
(WebKit::ResourceLoadStatisticsDatabaseStore::getSubStatisticStatement):
    Addition of the new category TopFrameLoadedThirdPartyScripts.
(WebKit::ResourceLoadStatisticsDatabaseStore::resourceToString):
    Removed the lastSeen output since it may differ between test runs.
* NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h:
* NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp:
    Bumped statisticsModelVersion to 17.
* Shared/WebCoreArgumentCoders.cpp:
(IPC::ArgumentCoder<ResourceLoadStatistics>::encode):
(IPC::ArgumentCoder<ResourceLoadStatistics>::decode):
    Encoding and decoding of the new category topFrameLoadedThirdPartyScripts.
* WebProcess/WebCoreSupport/WebResourceLoadObserver.cpp:
(WebKit::WebResourceLoadObserver::logSubresourceLoading):
    Now takes an additional enum parameter FetchDestinationIsScriptLike which
    is used to detect third-party script-like loads (script, worker, or
    service worker) from third-parties. If one is detected, it is stored
    in the new topFrameLoadedThirdPartyScripts category.
* WebProcess/WebCoreSupport/WebResourceLoadObserver.h:

LayoutTests:

* http/tests/resourceLoadStatistics/count-third-party-script-import-in-worker-database-expected.txt: Added.
* http/tests/resourceLoadStatistics/count-third-party-script-import-in-worker-database.html: Added.
* http/tests/resourceLoadStatistics/count-third-party-script-import-in-worker-expected.txt: Added.
* http/tests/resourceLoadStatistics/count-third-party-script-import-in-worker.html: Added.
* http/tests/resourceLoadStatistics/count-third-party-script-loads-database-expected.txt: Added.
* http/tests/resourceLoadStatistics/count-third-party-script-loads-database.html: Added.
* http/tests/resourceLoadStatistics/count-third-party-script-loads-expected.txt: Added.
* http/tests/resourceLoadStatistics/count-third-party-script-loads.html: Added.
* http/tests/resourceLoadStatistics/dont-count-third-party-image-as-third-party-script-database-expected.txt: Added.
* http/tests/resourceLoadStatistics/dont-count-third-party-image-as-third-party-script-database.html: Added.
* http/tests/resourceLoadStatistics/dont-count-third-party-image-as-third-party-script-expected.txt: Added.
* http/tests/resourceLoadStatistics/dont-count-third-party-image-as-third-party-script.html: Added.
* http/tests/resourceLoadStatistics/log-cross-site-load-with-link-decoration-database-expected.txt:
    Removed the lastSeen output since it may differ between test runs.
* http/tests/resourceLoadStatistics/log-cross-site-load-with-link-decoration-expected.txt:
    Removed the lastSeen output since it may differ between test runs.
* http/tests/resourceLoadStatistics/resources/dummy.js: Added.
* http/tests/resourceLoadStatistics/website-data-removal-for-site-navigated-to-with-link-decoration-database-expected.txt:
    Removed the lastSeen output since it may differ between test runs.
* http/tests/resourceLoadStatistics/website-data-removal-for-site-navigated-to-with-link-decoration-expected.txt:
    Removed the lastSeen output since it may differ between test runs.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@252641 268f45cc-cd09-0410-ab3c-d52691b4dbfc

31 files changed:
LayoutTests/ChangeLog
LayoutTests/http/tests/resourceLoadStatistics/count-third-party-script-import-in-worker-database-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/resourceLoadStatistics/count-third-party-script-import-in-worker-database.html [new file with mode: 0644]
LayoutTests/http/tests/resourceLoadStatistics/count-third-party-script-import-in-worker-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/resourceLoadStatistics/count-third-party-script-import-in-worker.html [new file with mode: 0644]
LayoutTests/http/tests/resourceLoadStatistics/count-third-party-script-loads-database-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/resourceLoadStatistics/count-third-party-script-loads-database.html [new file with mode: 0644]
LayoutTests/http/tests/resourceLoadStatistics/count-third-party-script-loads-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/resourceLoadStatistics/count-third-party-script-loads.html [new file with mode: 0644]
LayoutTests/http/tests/resourceLoadStatistics/dont-count-third-party-image-as-third-party-script-database-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/resourceLoadStatistics/dont-count-third-party-image-as-third-party-script-database.html [new file with mode: 0644]
LayoutTests/http/tests/resourceLoadStatistics/dont-count-third-party-image-as-third-party-script-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/resourceLoadStatistics/dont-count-third-party-image-as-third-party-script.html [new file with mode: 0644]
LayoutTests/http/tests/resourceLoadStatistics/log-cross-site-load-with-link-decoration-database-expected.txt
LayoutTests/http/tests/resourceLoadStatistics/log-cross-site-load-with-link-decoration-expected.txt
LayoutTests/http/tests/resourceLoadStatistics/resources/dummy.js [new file with mode: 0644]
LayoutTests/http/tests/resourceLoadStatistics/website-data-removal-for-site-navigated-to-with-link-decoration-database-expected.txt
LayoutTests/http/tests/resourceLoadStatistics/website-data-removal-for-site-navigated-to-with-link-decoration-expected.txt
Source/WebCore/ChangeLog
Source/WebCore/loader/FrameLoader.cpp
Source/WebCore/loader/ResourceLoadObserver.h
Source/WebCore/loader/ResourceLoadStatistics.cpp
Source/WebCore/loader/ResourceLoadStatistics.h
Source/WebCore/loader/SubresourceLoader.cpp
Source/WebKit/ChangeLog
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp
Source/WebKit/Shared/WebCoreArgumentCoders.cpp
Source/WebKit/WebProcess/WebCoreSupport/WebResourceLoadObserver.cpp
Source/WebKit/WebProcess/WebCoreSupport/WebResourceLoadObserver.h

index 2fd04da..9ff6ba8 100644 (file)
@@ -1,3 +1,33 @@
+2019-11-19  John Wilander  <wilander@apple.com>
+
+        Resource Load Statistics: Count third-party script loads under top frame
+        https://bugs.webkit.org/show_bug.cgi?id=204262
+        <rdar://problem/57244945>
+
+        Reviewed by Alex Christensen.
+
+        * http/tests/resourceLoadStatistics/count-third-party-script-import-in-worker-database-expected.txt: Added.
+        * http/tests/resourceLoadStatistics/count-third-party-script-import-in-worker-database.html: Added.
+        * http/tests/resourceLoadStatistics/count-third-party-script-import-in-worker-expected.txt: Added.
+        * http/tests/resourceLoadStatistics/count-third-party-script-import-in-worker.html: Added.
+        * http/tests/resourceLoadStatistics/count-third-party-script-loads-database-expected.txt: Added.
+        * http/tests/resourceLoadStatistics/count-third-party-script-loads-database.html: Added.
+        * http/tests/resourceLoadStatistics/count-third-party-script-loads-expected.txt: Added.
+        * http/tests/resourceLoadStatistics/count-third-party-script-loads.html: Added.
+        * http/tests/resourceLoadStatistics/dont-count-third-party-image-as-third-party-script-database-expected.txt: Added.
+        * http/tests/resourceLoadStatistics/dont-count-third-party-image-as-third-party-script-database.html: Added.
+        * http/tests/resourceLoadStatistics/dont-count-third-party-image-as-third-party-script-expected.txt: Added.
+        * http/tests/resourceLoadStatistics/dont-count-third-party-image-as-third-party-script.html: Added.
+        * http/tests/resourceLoadStatistics/log-cross-site-load-with-link-decoration-database-expected.txt:
+            Removed the lastSeen output since it may differ between test runs.
+        * http/tests/resourceLoadStatistics/log-cross-site-load-with-link-decoration-expected.txt:
+            Removed the lastSeen output since it may differ between test runs.
+        * http/tests/resourceLoadStatistics/resources/dummy.js: Added.
+        * http/tests/resourceLoadStatistics/website-data-removal-for-site-navigated-to-with-link-decoration-database-expected.txt:
+            Removed the lastSeen output since it may differ between test runs.
+        * http/tests/resourceLoadStatistics/website-data-removal-for-site-navigated-to-with-link-decoration-expected.txt:
+            Removed the lastSeen output since it may differ between test runs.
+
 2019-11-19  Sihui Liu  <sihui_liu@apple.com>
 
         Update expectations for bufferedAmount-unchanged-by-sync-xhr.any.worker.html
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/count-third-party-script-import-in-worker-database-expected.txt b/LayoutTests/http/tests/resourceLoadStatistics/count-third-party-script-import-in-worker-database-expected.txt
new file mode 100644 (file)
index 0000000..369c1e4
--- /dev/null
@@ -0,0 +1,26 @@
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
+Resource load statistics:
+
+Registrable domain: 127.0.0.1
+    hadUserInteraction: No
+    mostRecentUserInteraction: -1
+    grandfathered: No
+    TopFrameLoadedThirdPartyScripts:
+        localhost
+    IsScheduledForAllButCookieDataRemoval: No
+    isPrevalentResource: No
+    isVeryPrevalentResource: No
+    dataRecordsRemoved: 0
+Registrable domain: localhost
+    hadUserInteraction: No
+    mostRecentUserInteraction: -1
+    grandfathered: No
+    IsScheduledForAllButCookieDataRemoval: No
+    SubresourceUnderTopFrameDomains:
+        127.0.0.1
+    isPrevalentResource: No
+    isVeryPrevalentResource: No
+    dataRecordsRemoved: 0
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/count-third-party-script-import-in-worker-database.html b/LayoutTests/http/tests/resourceLoadStatistics/count-third-party-script-import-in-worker-database.html
new file mode 100644 (file)
index 0000000..c175701
--- /dev/null
@@ -0,0 +1,48 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <title>Test counting of third-party script imports in workers</title>
+    <script src="/js-test-resources/js-test.js"></script>
+    <script src="resources/util.js"></script>
+</head>
+<body onload="runTest()">
+<script>
+    function finishTest() {
+        testRunner.dumpResourceLoadStatistics();
+        setEnableFeature(false, function() {
+            testRunner.notifyDone();
+        });
+    }
+
+    function receiveMessage(event) {
+        if (event.data.indexOf("PASS") === -1)
+            testFailed(event.data.replace("FAIL ", ""));
+
+        testRunner.statisticsNotifyObserver();
+    }
+
+    function runTest() {
+        switch (document.location.hash) {
+            case "":
+                if (window.testRunner && window.internals) {
+                    setEnableFeature(true, function() {
+                        testRunner.setUseITPDatabase(true);
+                        testRunner.waitUntilDone();
+                        testRunner.setStatisticsNotifyPagesWhenDataRecordsWereScanned(true);
+                        testRunner.installStatisticsDidScanDataRecordsCallback(finishTest);
+                        document.location.hash = "LoadWorker";
+                        runTest();
+                    });
+                }
+                break;
+            case "#LoadWorker":
+                let w1 = new Worker("resources/worker-importing-localhost-script.js");
+                w1.onmessage = receiveMessage;
+                w1.postMessage("shouldNotReceiveCookies");
+                break;
+        }
+    }
+</script>
+</body>
+</html>
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/count-third-party-script-import-in-worker-expected.txt b/LayoutTests/http/tests/resourceLoadStatistics/count-third-party-script-import-in-worker-expected.txt
new file mode 100644 (file)
index 0000000..1211af0
--- /dev/null
@@ -0,0 +1,26 @@
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
+Resource load statistics:
+
+Registrable domain: localhost
+    hadUserInteraction: No
+    mostRecentUserInteraction: -1
+    grandfathered: No
+    gotLinkDecorationFromPrevalentResource: No
+    subresourceUnderTopFrameDomains:
+        127.0.0.1
+    isPrevalentResource: No
+    isVeryPrevalentResource: No
+    dataRecordsRemoved: 0
+Registrable domain: 127.0.0.1
+    hadUserInteraction: No
+    mostRecentUserInteraction: -1
+    grandfathered: No
+    gotLinkDecorationFromPrevalentResource: No
+    topFrameLoadedThirdPartyScripts:
+        localhost
+    isPrevalentResource: No
+    isVeryPrevalentResource: No
+    dataRecordsRemoved: 0
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/count-third-party-script-import-in-worker.html b/LayoutTests/http/tests/resourceLoadStatistics/count-third-party-script-import-in-worker.html
new file mode 100644 (file)
index 0000000..1668cca
--- /dev/null
@@ -0,0 +1,47 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <title>Test counting of third-party script imports in workers</title>
+    <script src="/js-test-resources/js-test.js"></script>
+    <script src="resources/util.js"></script>
+</head>
+<body onload="runTest()">
+<script>
+    function finishTest() {
+        testRunner.dumpResourceLoadStatistics();
+        setEnableFeature(false, function() {
+            testRunner.notifyDone();
+        });
+    }
+
+    function receiveMessage(event) {
+        if (event.data.indexOf("PASS") === -1)
+            testFailed(event.data.replace("FAIL ", ""));
+
+        testRunner.statisticsNotifyObserver();
+    }
+
+    function runTest() {
+        switch (document.location.hash) {
+            case "":
+                if (window.testRunner && window.internals) {
+                    setEnableFeature(true, function() {
+                        testRunner.waitUntilDone();
+                        testRunner.setStatisticsNotifyPagesWhenDataRecordsWereScanned(true);
+                        testRunner.installStatisticsDidScanDataRecordsCallback(finishTest);
+                        document.location.hash = "LoadWorker";
+                        runTest();
+                    });
+                }
+                break;
+            case "#LoadWorker":
+                let w1 = new Worker("resources/worker-importing-localhost-script.js");
+                w1.onmessage = receiveMessage;
+                w1.postMessage("shouldNotReceiveCookies");
+                break;
+        }
+    }
+</script>
+</body>
+</html>
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/count-third-party-script-loads-database-expected.txt b/LayoutTests/http/tests/resourceLoadStatistics/count-third-party-script-loads-database-expected.txt
new file mode 100644 (file)
index 0000000..369c1e4
--- /dev/null
@@ -0,0 +1,26 @@
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
+Resource load statistics:
+
+Registrable domain: 127.0.0.1
+    hadUserInteraction: No
+    mostRecentUserInteraction: -1
+    grandfathered: No
+    TopFrameLoadedThirdPartyScripts:
+        localhost
+    IsScheduledForAllButCookieDataRemoval: No
+    isPrevalentResource: No
+    isVeryPrevalentResource: No
+    dataRecordsRemoved: 0
+Registrable domain: localhost
+    hadUserInteraction: No
+    mostRecentUserInteraction: -1
+    grandfathered: No
+    IsScheduledForAllButCookieDataRemoval: No
+    SubresourceUnderTopFrameDomains:
+        127.0.0.1
+    isPrevalentResource: No
+    isVeryPrevalentResource: No
+    dataRecordsRemoved: 0
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/count-third-party-script-loads-database.html b/LayoutTests/http/tests/resourceLoadStatistics/count-third-party-script-loads-database.html
new file mode 100644 (file)
index 0000000..439ec49
--- /dev/null
@@ -0,0 +1,49 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <title>Test counting of third-party script loads</title>
+    <script src="/js-test-resources/js-test.js"></script>
+    <script src="resources/util.js"></script>
+</head>
+<body onload="runTest()">
+<script>
+    const thirdPartyOrigin = "http://localhost:8000";
+    const thirdPartyBaseUrl = thirdPartyOrigin + "/resourceLoadStatistics/resources";
+
+    function finishTest() {
+        testRunner.dumpResourceLoadStatistics();
+        setEnableFeature(false, function() {
+            testRunner.notifyDone();
+        });
+    }
+
+    function runTest() {
+        switch (document.location.hash) {
+            case "":
+                if (window.testRunner && window.internals) {
+                    setEnableFeature(true, function() {
+                        testRunner.setUseITPDatabase(true);
+                        testRunner.waitUntilDone();
+                        testRunner.setStatisticsNotifyPagesWhenDataRecordsWereScanned(true);
+                        testRunner.installStatisticsDidScanDataRecordsCallback(finishTest);
+                        document.location.hash = "step1";
+                        runTest();
+                    });
+                }
+                break;
+            case "#step1":
+                document.location.hash = "step2";
+                let scriptElement = document.createElement("script");
+                scriptElement.onload = runTest;
+                scriptElement.src = thirdPartyBaseUrl + "/dummy.js?dummyParam=" + Math.random();
+                document.body.appendChild(scriptElement);
+                break;
+            case "#step2":
+                testRunner.statisticsNotifyObserver();
+                break;
+        }
+    }
+</script>
+</body>
+</html>
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/count-third-party-script-loads-expected.txt b/LayoutTests/http/tests/resourceLoadStatistics/count-third-party-script-loads-expected.txt
new file mode 100644 (file)
index 0000000..1211af0
--- /dev/null
@@ -0,0 +1,26 @@
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
+Resource load statistics:
+
+Registrable domain: localhost
+    hadUserInteraction: No
+    mostRecentUserInteraction: -1
+    grandfathered: No
+    gotLinkDecorationFromPrevalentResource: No
+    subresourceUnderTopFrameDomains:
+        127.0.0.1
+    isPrevalentResource: No
+    isVeryPrevalentResource: No
+    dataRecordsRemoved: 0
+Registrable domain: 127.0.0.1
+    hadUserInteraction: No
+    mostRecentUserInteraction: -1
+    grandfathered: No
+    gotLinkDecorationFromPrevalentResource: No
+    topFrameLoadedThirdPartyScripts:
+        localhost
+    isPrevalentResource: No
+    isVeryPrevalentResource: No
+    dataRecordsRemoved: 0
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/count-third-party-script-loads.html b/LayoutTests/http/tests/resourceLoadStatistics/count-third-party-script-loads.html
new file mode 100644 (file)
index 0000000..e305073
--- /dev/null
@@ -0,0 +1,48 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <title>Test counting of third-party script loads</title>
+    <script src="/js-test-resources/js-test.js"></script>
+    <script src="resources/util.js"></script>
+</head>
+<body onload="runTest()">
+<script>
+    const thirdPartyOrigin = "http://localhost:8000";
+    const thirdPartyBaseUrl = thirdPartyOrigin + "/resourceLoadStatistics/resources";
+
+    function finishTest() {
+        testRunner.dumpResourceLoadStatistics();
+        setEnableFeature(false, function() {
+            testRunner.notifyDone();
+        });
+    }
+
+    function runTest() {
+        switch (document.location.hash) {
+            case "":
+                if (window.testRunner && window.internals) {
+                    setEnableFeature(true, function() {
+                        testRunner.waitUntilDone();
+                        testRunner.setStatisticsNotifyPagesWhenDataRecordsWereScanned(true);
+                        testRunner.installStatisticsDidScanDataRecordsCallback(finishTest);
+                        document.location.hash = "step1";
+                        runTest();
+                    });
+                }
+                break;
+            case "#step1":
+                document.location.hash = "step2";
+                let scriptElement = document.createElement("script");
+                scriptElement.onload = runTest;
+                scriptElement.src = thirdPartyBaseUrl + "/dummy.js?dummyParam=" + Math.random();
+                document.body.appendChild(scriptElement);
+                break;
+            case "#step2":
+                testRunner.statisticsNotifyObserver();
+                break;
+        }
+    }
+</script>
+</body>
+</html>
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/dont-count-third-party-image-as-third-party-script-database-expected.txt b/LayoutTests/http/tests/resourceLoadStatistics/dont-count-third-party-image-as-third-party-script-database-expected.txt
new file mode 100644 (file)
index 0000000..025c23f
--- /dev/null
@@ -0,0 +1,24 @@
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
+Resource load statistics:
+
+Registrable domain: 127.0.0.1
+    hadUserInteraction: No
+    mostRecentUserInteraction: -1
+    grandfathered: No
+    IsScheduledForAllButCookieDataRemoval: No
+    isPrevalentResource: No
+    isVeryPrevalentResource: No
+    dataRecordsRemoved: 0
+Registrable domain: localhost
+    hadUserInteraction: No
+    mostRecentUserInteraction: -1
+    grandfathered: No
+    IsScheduledForAllButCookieDataRemoval: No
+    SubresourceUnderTopFrameDomains:
+        127.0.0.1
+    isPrevalentResource: No
+    isVeryPrevalentResource: No
+    dataRecordsRemoved: 0
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/dont-count-third-party-image-as-third-party-script-database.html b/LayoutTests/http/tests/resourceLoadStatistics/dont-count-third-party-image-as-third-party-script-database.html
new file mode 100644 (file)
index 0000000..5216090
--- /dev/null
@@ -0,0 +1,49 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <title>Test not counting third-party images as third-party script loads</title>
+    <script src="/js-test-resources/js-test.js"></script>
+    <script src="resources/util.js"></script>
+</head>
+<body onload="runTest()">
+<script>
+    const thirdPartyOrigin = "http://localhost:8000";
+    const thirdPartyBaseUrl = thirdPartyOrigin + "/resources";
+
+    function finishTest() {
+        testRunner.dumpResourceLoadStatistics();
+        setEnableFeature(false, function() {
+            testRunner.notifyDone();
+        });
+    }
+
+    function runTest() {
+        switch (document.location.hash) {
+            case "":
+                if (window.testRunner && window.internals) {
+                    setEnableFeature(true, function() {
+                        testRunner.setUseITPDatabase(true);
+                        testRunner.waitUntilDone();
+                        testRunner.setStatisticsNotifyPagesWhenDataRecordsWereScanned(true);
+                        testRunner.installStatisticsDidScanDataRecordsCallback(finishTest);
+                        document.location.hash = "step1";
+                        runTest();
+                    });
+                }
+                break;
+            case "#step1":
+                document.location.hash = "step2";
+                let imgElement = document.createElement("img");
+                imgElement.onload = runTest;
+                imgElement.src = thirdPartyBaseUrl + "/square20.jpg?dummyParam=" + Math.random();
+                document.body.appendChild(imgElement);
+                break;
+            case "#step2":
+                testRunner.statisticsNotifyObserver();
+                break;
+        }
+    }
+</script>
+</body>
+</html>
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/dont-count-third-party-image-as-third-party-script-expected.txt b/LayoutTests/http/tests/resourceLoadStatistics/dont-count-third-party-image-as-third-party-script-expected.txt
new file mode 100644 (file)
index 0000000..4ecb973
--- /dev/null
@@ -0,0 +1,16 @@
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
+Resource load statistics:
+
+Registrable domain: localhost
+    hadUserInteraction: No
+    mostRecentUserInteraction: -1
+    grandfathered: No
+    gotLinkDecorationFromPrevalentResource: No
+    subresourceUnderTopFrameDomains:
+        127.0.0.1
+    isPrevalentResource: No
+    isVeryPrevalentResource: No
+    dataRecordsRemoved: 0
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/dont-count-third-party-image-as-third-party-script.html b/LayoutTests/http/tests/resourceLoadStatistics/dont-count-third-party-image-as-third-party-script.html
new file mode 100644 (file)
index 0000000..a51e64c
--- /dev/null
@@ -0,0 +1,48 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <title>Test not counting third-party images as third-party script loads</title>
+    <script src="/js-test-resources/js-test.js"></script>
+    <script src="resources/util.js"></script>
+</head>
+<body onload="runTest()">
+<script>
+    const thirdPartyOrigin = "http://localhost:8000";
+    const thirdPartyBaseUrl = thirdPartyOrigin + "/resources";
+
+    function finishTest() {
+        testRunner.dumpResourceLoadStatistics();
+        setEnableFeature(false, function() {
+            testRunner.notifyDone();
+        });
+    }
+
+    function runTest() {
+        switch (document.location.hash) {
+            case "":
+                if (window.testRunner && window.internals) {
+                    setEnableFeature(true, function() {
+                        testRunner.waitUntilDone();
+                        testRunner.setStatisticsNotifyPagesWhenDataRecordsWereScanned(true);
+                        testRunner.installStatisticsDidScanDataRecordsCallback(finishTest);
+                        document.location.hash = "step1";
+                        runTest();
+                    });
+                }
+                break;
+            case "#step1":
+                document.location.hash = "step2";
+                let imgElement = document.createElement("img");
+                imgElement.onload = runTest;
+                imgElement.src = thirdPartyBaseUrl + "/square20.jpg?dummyParam=" + Math.random();
+                document.body.appendChild(imgElement);
+                break;
+            case "#step2":
+                testRunner.statisticsNotifyObserver();
+                break;
+        }
+    }
+</script>
+</body>
+</html>
index 98d1daa..947250d 100644 (file)
@@ -3,29 +3,29 @@ Test logging of link decorated cross-site navigations from a prevalent resource.
 Resource load statistics:
 
 Registrable domain: 127.0.0.1
-    lastSeen: 0
     hadUserInteraction: No
     mostRecentUserInteraction: -1
     grandfathered: No
-    IsScheduledForAllButCookieDataRemoval: No    isPrevalentResource: Yes
+    IsScheduledForAllButCookieDataRemoval: No
+    isPrevalentResource: Yes
     isVeryPrevalentResource: No
     dataRecordsRemoved: 0
 Registrable domain: 127.0.0.2
-    lastSeen: 0
     hadUserInteraction: No
     mostRecentUserInteraction: -1
     grandfathered: No
-    IsScheduledForAllButCookieDataRemoval: No    isPrevalentResource: No
+    IsScheduledForAllButCookieDataRemoval: No
+    isPrevalentResource: No
     isVeryPrevalentResource: No
     dataRecordsRemoved: 0
 Registrable domain: localhost
-    lastSeen: 0
     hadUserInteraction: No
     mostRecentUserInteraction: -1
     grandfathered: No
     TopFrameLinkDecorationsFrom:
         127.0.0.1
         127.0.0.2
-    IsScheduledForAllButCookieDataRemoval: Yes    isPrevalentResource: No
+    IsScheduledForAllButCookieDataRemoval: Yes
+    isPrevalentResource: No
     isVeryPrevalentResource: No
     dataRecordsRemoved: 0
index 740a403..5ccd124 100644 (file)
@@ -3,20 +3,20 @@ Test logging of link decorated cross-site navigations from a prevalent resource.
 Resource load statistics:
 
 Registrable domain: localhost
-    lastSeen: 0
     hadUserInteraction: No
     mostRecentUserInteraction: -1
     grandfathered: No
     topFrameLinkDecorationsFrom:
         127.0.0.1
-    gotLinkDecorationFromPrevalentResource: Yes    isPrevalentResource: No
+    gotLinkDecorationFromPrevalentResource: Yes
+    isPrevalentResource: No
     isVeryPrevalentResource: No
     dataRecordsRemoved: 0
 Registrable domain: 127.0.0.1
-    lastSeen: 0
     hadUserInteraction: No
     mostRecentUserInteraction: -1
     grandfathered: No
-    gotLinkDecorationFromPrevalentResource: No    isPrevalentResource: Yes
+    gotLinkDecorationFromPrevalentResource: No
+    isPrevalentResource: Yes
     isVeryPrevalentResource: No
     dataRecordsRemoved: 0
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/resources/dummy.js b/LayoutTests/http/tests/resourceLoadStatistics/resources/dummy.js
new file mode 100644 (file)
index 0000000..e7414c2
--- /dev/null
@@ -0,0 +1,6 @@
+function getRandomInt(max) {
+    return Math.floor(Math.random() * Math.floor(max));
+}
+let value = getRandomInt(10);
+if (value > 5)
+    value = 5;
\ No newline at end of file
index d9b055b..de41a9a 100644 (file)
@@ -16,20 +16,20 @@ After deletion: IDB entry does not exist.
 Resource load statistics:
 
 Registrable domain: 127.0.0.1
-    lastSeen: 0
     hadUserInteraction: No
     mostRecentUserInteraction: -1
     grandfathered: No
     TopFrameLinkDecorationsFrom:
         localhost
-    IsScheduledForAllButCookieDataRemoval: No    isPrevalentResource: No
+    IsScheduledForAllButCookieDataRemoval: No
+    isPrevalentResource: No
     isVeryPrevalentResource: No
     dataRecordsRemoved: 1
 Registrable domain: localhost
-    lastSeen: 0
     hadUserInteraction: No
     mostRecentUserInteraction: -1
     grandfathered: No
-    IsScheduledForAllButCookieDataRemoval: No    isPrevalentResource: Yes
+    IsScheduledForAllButCookieDataRemoval: No
+    isPrevalentResource: Yes
     isVeryPrevalentResource: No
     dataRecordsRemoved: 0
index d36dfc4..f6e4054 100644 (file)
@@ -16,20 +16,20 @@ After deletion: IDB entry does not exist.
 Resource load statistics:
 
 Registrable domain: localhost
-    lastSeen: 0
     hadUserInteraction: No
     mostRecentUserInteraction: -1
     grandfathered: No
-    gotLinkDecorationFromPrevalentResource: No    isPrevalentResource: Yes
+    gotLinkDecorationFromPrevalentResource: No
+    isPrevalentResource: Yes
     isVeryPrevalentResource: No
     dataRecordsRemoved: 0
 Registrable domain: 127.0.0.1
-    lastSeen: 0
     hadUserInteraction: No
     mostRecentUserInteraction: -1
     grandfathered: No
     topFrameLinkDecorationsFrom:
         localhost
-    gotLinkDecorationFromPrevalentResource: No    isPrevalentResource: No
+    gotLinkDecorationFromPrevalentResource: No
+    isPrevalentResource: No
     isVeryPrevalentResource: No
     dataRecordsRemoved: 1
index b49dfdf..bd73c0e 100644 (file)
@@ -1,3 +1,42 @@
+2019-11-19  John Wilander  <wilander@apple.com>
+
+        Resource Load Statistics: Count third-party script loads under top frame
+        https://bugs.webkit.org/show_bug.cgi?id=204262
+        <rdar://problem/57244945>
+
+        Reviewed by Alex Christensen.
+
+        Third-party scripts running in the first-party context are a significant privacy
+        and security risk. This change captures the number of such script loads which will
+        allow ITP to take action.
+
+        Tests: http/tests/resourceLoadStatistics/count-third-party-script-import-in-worker-database.html
+               http/tests/resourceLoadStatistics/count-third-party-script-import-in-worker.html
+               http/tests/resourceLoadStatistics/count-third-party-script-loads-database.html
+               http/tests/resourceLoadStatistics/count-third-party-script-loads.html
+
+        * loader/FrameLoader.cpp:
+        (WebCore::FrameLoader::loadResourceSynchronously):
+            Now sends the ResourceLoadObserver::FetchDestinationIsScriptLike parameter to
+            ResourceLoadObserver::logSubresourceLoading().
+        * loader/ResourceLoadObserver.h:
+        (WebCore::ResourceLoadObserver::logSubresourceLoading):
+            Now takes a FetchDestinationIsScriptLike parameter.
+        * loader/ResourceLoadStatistics.cpp:
+        (WebCore::ResourceLoadStatistics::encode const):
+        (WebCore::ResourceLoadStatistics::decode):
+        (WebCore::ResourceLoadStatistics::toString const):
+            Output of the new topFrameLoadedThirdPartyScripts category.
+            Removed the lastSeen output since it may differ between test runs.
+        (WebCore::ResourceLoadStatistics::merge):
+            Handling of the new topFrameLoadedThirdPartyScripts category.
+        * loader/ResourceLoadStatistics.h:
+            Added the new topFrameLoadedThirdPartyScripts category.
+        * loader/SubresourceLoader.cpp:
+        (WebCore::SubresourceLoader::willSendRequestInternal):
+            Now sends the ResourceLoadObserver::FetchDestinationIsScriptLike parameter to
+            ResourceLoadObserver::logSubresourceLoading().
+
 2019-11-19  Zalan Bujtas  <zalan@apple.com>
 
         [LFC][IFC] Display::Run:TextContext should use StringView
index 101669e..7b10cb9 100644 (file)
@@ -3150,7 +3150,8 @@ unsigned long FrameLoader::loadResourceSynchronously(const ResourceRequest& requ
             platformStrategies()->loaderStrategy()->loadResourceSynchronously(*this, identifier, newRequest, clientCredentialPolicy, options, originalRequestHeaders, error, response, buffer);
             data = SharedBuffer::create(WTFMove(buffer));
             documentLoader()->applicationCacheHost().maybeLoadFallbackSynchronously(newRequest, error, response, data);
-            ResourceLoadObserver::shared().logSubresourceLoading(&m_frame, newRequest, response);
+            ResourceLoadObserver::shared().logSubresourceLoading(&m_frame, newRequest, response,
+                (isScriptLikeDestination(options.destination) ? ResourceLoadObserver::FetchDestinationIsScriptLike::Yes : ResourceLoadObserver::FetchDestinationIsScriptLike::No));
         }
     }
     notifier().sendRemainingDelegateMessages(m_documentLoader.get(), identifier, request, response, data ? data->data() : nullptr, data ? data->size() : 0, -1, error);
index fd65155..129c3c6 100644 (file)
@@ -38,13 +38,16 @@ class ResourceResponse;
 class ResourceLoadObserver {
     WTF_MAKE_FAST_ALLOCATED;
 public:
+    // https://fetch.spec.whatwg.org/#request-destination-script-like
+    enum class FetchDestinationIsScriptLike : bool { Yes, No };
+
     WEBCORE_EXPORT static ResourceLoadObserver& shared();
     WEBCORE_EXPORT static ResourceLoadObserver* sharedIfExists();
     WEBCORE_EXPORT static void setShared(ResourceLoadObserver&);
     
     virtual ~ResourceLoadObserver() { }
 
-    virtual void logSubresourceLoading(const Frame*, const ResourceRequest& /* newRequest */, const ResourceResponse& /* redirectResponse */) { }
+    virtual void logSubresourceLoading(const Frame*, const ResourceRequest& /* newRequest */, const ResourceResponse& /* redirectResponse */, FetchDestinationIsScriptLike) { }
     virtual void logWebSocketLoading(const URL& /* targetURL */, const URL& /* mainFrameURL */) { }
     virtual void logUserInteractionWithReducedTimeResolution(const Document&) { }
     virtual void logFontLoad(const Document&, const String& /* familyName */, bool /* loadStatus */) { }
index 9459776..54781f8 100644 (file)
@@ -95,6 +95,7 @@ void ResourceLoadStatistics::encode(KeyedEncoder& encoder) const
     encodeHashSet(encoder, "topFrameUniqueRedirectsFrom"_s, "domain"_s, topFrameUniqueRedirectsFrom);
     encodeHashSet(encoder, "topFrameLinkDecorationsFrom"_s, "domain", topFrameLinkDecorationsFrom);
     encoder.encodeBool("gotLinkDecorationFromPrevalentResource"_s, gotLinkDecorationFromPrevalentResource);
+    encodeHashSet(encoder, "topFrameLoadedThirdPartyScripts"_s, "domain", topFrameLoadedThirdPartyScripts);
 
     // Subframe stats
     encodeHashSet(encoder, "subframeUnderTopFrameDomains"_s, "domain"_s, subframeUnderTopFrameDomains);
@@ -225,6 +226,13 @@ bool ResourceLoadStatistics::decode(KeyedDecoder& decoder, unsigned modelVersion
             return false;
     }
 
+    if (modelVersion >= 17) {
+        HashCountedSet<RegistrableDomain> topFrameLoadedThirdPartyScriptsCounted;
+        decodeHashCountedSet(decoder, "topFrameLoadedThirdPartyScripts", topFrameLoadedThirdPartyScriptsCounted);
+        for (auto& domain : topFrameLoadedThirdPartyScriptsCounted.values())
+            topFrameLoadedThirdPartyScripts.add(domain);
+    }
+
     // Subframe stats
     if (modelVersion >= 15)
         decodeHashSet(decoder, "subframeUnderTopFrameDomains", "domain", subframeUnderTopFrameDomains);
@@ -411,10 +419,7 @@ String ResourceLoadStatistics::toString() const
     builder.appendLiteral("Registrable domain: ");
     builder.append(registrableDomain.string());
     builder.append('\n');
-    builder.appendLiteral("    lastSeen: ");
-    builder.appendFixedPrecisionNumber(lastSeen.secondsSinceEpoch().value());
-    builder.append('\n');
-    
+
     // User interaction
     appendBoolean(builder, "hadUserInteraction", hadUserInteraction);
     builder.append('\n');
@@ -432,6 +437,8 @@ String ResourceLoadStatistics::toString() const
     appendHashSet(builder, "topFrameUniqueRedirectsFrom", topFrameUniqueRedirectsFrom);
     appendHashSet(builder, "topFrameLinkDecorationsFrom", topFrameLinkDecorationsFrom);
     appendBoolean(builder, "gotLinkDecorationFromPrevalentResource", gotLinkDecorationFromPrevalentResource);
+    builder.append('\n');
+    appendHashSet(builder, "topFrameLoadedThirdPartyScripts", topFrameLoadedThirdPartyScripts);
 
     // Subframe stats
     appendHashSet(builder, "subframeUnderTopFrameDomains", subframeUnderTopFrameDomains);
@@ -508,6 +515,7 @@ void ResourceLoadStatistics::merge(const ResourceLoadStatistics& other)
     mergeHashSet(topFrameUniqueRedirectsFrom, other.topFrameUniqueRedirectsFrom);
     mergeHashSet(topFrameLinkDecorationsFrom, other.topFrameLinkDecorationsFrom);
     gotLinkDecorationFromPrevalentResource |= other.gotLinkDecorationFromPrevalentResource;
+    mergeHashSet(topFrameLoadedThirdPartyScripts, other.topFrameLoadedThirdPartyScripts);
 
     // Subframe stats
     mergeHashSet(subframeUnderTopFrameDomains, other.subframeUnderTopFrameDomains);
index 5b08e80..7e2653d 100644 (file)
@@ -80,6 +80,7 @@ struct ResourceLoadStatistics {
     HashSet<RegistrableDomain> topFrameUniqueRedirectsFrom;
     HashSet<RegistrableDomain> topFrameLinkDecorationsFrom;
     bool gotLinkDecorationFromPrevalentResource { false };
+    HashSet<RegistrableDomain> topFrameLoadedThirdPartyScripts;
 
     // Subframe stats
     HashSet<RegistrableDomain> subframeUnderTopFrameDomains;
index f478df0..cf64524 100644 (file)
@@ -195,7 +195,8 @@ void SubresourceLoader::willSendRequestInternal(ResourceRequest&& newRequest, co
 
     if (newRequest.requester() != ResourceRequestBase::Requester::Main) {
         tracePoint(SubresourceLoadWillStart);
-        ResourceLoadObserver::shared().logSubresourceLoading(m_frame.get(), newRequest, redirectResponse);
+        ResourceLoadObserver::shared().logSubresourceLoading(m_frame.get(), newRequest, redirectResponse,
+            (isScriptLikeDestination(options().destination) ? ResourceLoadObserver::FetchDestinationIsScriptLike::Yes : ResourceLoadObserver::FetchDestinationIsScriptLike::No));
     }
 
     auto continueWillSendRequest = [this, protectedThis = makeRef(*this), redirectResponse] (CompletionHandler<void(ResourceRequest&&)>&& completionHandler, ResourceRequest&& newRequest) mutable {
index bc51ff1..6efb63f 100644 (file)
@@ -1,3 +1,40 @@
+2019-11-19  John Wilander  <wilander@apple.com>
+
+        Resource Load Statistics: Count third-party script loads under top frame
+        https://bugs.webkit.org/show_bug.cgi?id=204262
+        <rdar://problem/57244945>
+
+        Reviewed by Alex Christensen.
+
+        Third-party scripts running in the first-party context are a significant privacy
+        and security risk. This change captures the number of such script loads which will
+        allow ITP to take action.
+
+        * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:
+        (WebKit::ResourceLoadStatisticsDatabaseStore::ResourceLoadStatisticsDatabaseStore):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::createUniqueIndices):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::createSchema):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::prepareStatements):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::insertDomainRelationships):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::getSubStatisticStatement):
+            Addition of the new category TopFrameLoadedThirdPartyScripts.
+        (WebKit::ResourceLoadStatisticsDatabaseStore::resourceToString):
+            Removed the lastSeen output since it may differ between test runs.
+        * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h:
+        * NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp:
+            Bumped statisticsModelVersion to 17.
+        * Shared/WebCoreArgumentCoders.cpp:
+        (IPC::ArgumentCoder<ResourceLoadStatistics>::encode):
+        (IPC::ArgumentCoder<ResourceLoadStatistics>::decode):
+            Encoding and decoding of the new category topFrameLoadedThirdPartyScripts.
+        * WebProcess/WebCoreSupport/WebResourceLoadObserver.cpp:
+        (WebKit::WebResourceLoadObserver::logSubresourceLoading):
+            Now takes an additional enum parameter FetchDestinationIsScriptLike which
+            is used to detect third-party script-like loads (script, worker, or
+            service worker) from third-parties. If one is detected, it is stored
+            in the new topFrameLoadedThirdPartyScripts category.
+        * WebProcess/WebCoreSupport/WebResourceLoadObserver.h:
+
 2019-11-19  Brian Burg  <bburg@apple.com>
 
         [Cocoa] Add _WKInspector SPI to set diagnostic logging delegate for a local Web Inspector
index fec1ccc..24db20b 100644 (file)
@@ -83,6 +83,7 @@ constexpr auto topFrameUniqueRedirectsToQuery = "INSERT OR IGNORE into TopFrameU
 constexpr auto subframeUnderTopFrameDomainsQuery = "INSERT OR IGNORE into SubframeUnderTopFrameDomains (subFrameDomainID, topFrameDomainID) SELECT ?, domainID FROM ObservedDomains where registrableDomain in ( "_s;
 constexpr auto topFrameUniqueRedirectsFromQuery = "INSERT OR IGNORE INTO TopFrameUniqueRedirectsFrom (targetDomainID, fromDomainID) SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain in ( "_s;
 constexpr auto topFrameLinkDecorationsFromQuery = "INSERT OR IGNORE INTO TopFrameLinkDecorationsFrom (toDomainID, fromDomainID) SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain in ( "_s;
+constexpr auto topFrameLoadedThirdPartyScriptsQuery = "INSERT OR IGNORE into TopFrameLoadedThirdPartyScripts (topFrameDomainID, subresourceDomainID) SELECT ?, domainID FROM ObservedDomains where registrableDomain in ( "_s;
 constexpr auto subresourceUnderTopFrameDomainsQuery = "INSERT OR IGNORE INTO SubresourceUnderTopFrameDomains (subresourceDomainID, topFrameDomainID) SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain in ( "_s;
 constexpr auto subresourceUniqueRedirectsToQuery = "INSERT OR IGNORE INTO SubresourceUniqueRedirectsTo (subresourceDomainID, toDomainID) SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain in ( "_s;
 constexpr auto subresourceUniqueRedirectsFromQuery = "INSERT OR IGNORE INTO SubresourceUniqueRedirectsFrom (subresourceDomainID, fromDomainID) SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain in ( "_s;
@@ -95,7 +96,9 @@ constexpr auto subresourceUnderTopFrameDomainExistsQuery = "SELECT EXISTS (SELEC
 constexpr auto subresourceUniqueRedirectsToExistsQuery = "SELECT EXISTS (SELECT 1 FROM SubresourceUniqueRedirectsTo WHERE subresourceDomainID = ? "
     "AND toDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
 constexpr auto topFrameLinkDecorationsFromExistsQuery = "SELECT EXISTS (SELECT 1 FROM TopFrameLinkDecorationsFrom WHERE toDomainID = ? "
-"AND fromDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
+    "AND fromDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
+constexpr auto topFrameLoadedThirdPartyScriptsExistsQuery = "SELECT EXISTS (SELECT 1 FROM TopFrameLoadedThirdPartyScripts WHERE topFrameDomainID = ? "
+    "AND subresourceDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
 
 // UPDATE Queries
 constexpr auto mostRecentUserInteractionQuery = "UPDATE ObservedDomains SET hadUserInteraction = ?, mostRecentUserInteractionTime = ? "
@@ -166,6 +169,11 @@ constexpr auto createTopFrameLinkDecorationsFrom = "CREATE TABLE TopFrameLinkDec
     "FOREIGN KEY(toDomainID) REFERENCES TopLevelDomains(topLevelDomainID) ON DELETE CASCADE, "
     "FOREIGN KEY(fromDomainID) REFERENCES TopLevelDomains(topLevelDomainID) ON DELETE CASCADE);"_s;
 
+constexpr auto createTopFrameLoadedThirdPartyScripts = "CREATE TABLE TopFrameLoadedThirdPartyScripts ("
+    "topFrameDomainID INTEGER NOT NULL, subresourceDomainID INTEGER NOT NULL, "
+    "FOREIGN KEY(topFrameDomainID) REFERENCES ObservedDomains(domainID) ON DELETE CASCADE, "
+    "FOREIGN KEY(subresourceDomainID) REFERENCES ObservedDomains(domainID) ON DELETE CASCADE);"_s;
+
 constexpr auto createSubframeUnderTopFrameDomains = "CREATE TABLE SubframeUnderTopFrameDomains ("
     "subFrameDomainID INTEGER NOT NULL, topFrameDomainID INTEGER NOT NULL, "
     "FOREIGN KEY(subFrameDomainID) REFERENCES ObservedDomains(domainID) ON DELETE CASCADE, "
@@ -191,6 +199,7 @@ constexpr auto createUniqueIndexStorageAccessUnderTopFrameDomains = "CREATE UNIQ
 constexpr auto createUniqueIndexTopFrameUniqueRedirectsTo = "CREATE UNIQUE INDEX IF NOT EXISTS TopFrameUniqueRedirectsTo_sourceDomainID_toDomainID on TopFrameUniqueRedirectsTo ( sourceDomainID, toDomainID );"_s;
 constexpr auto createUniqueIndexTopFrameUniqueRedirectsFrom = "CREATE UNIQUE INDEX IF NOT EXISTS TopFrameUniqueRedirectsFrom_targetDomainID_fromDomainID on TopFrameUniqueRedirectsFrom ( targetDomainID, fromDomainID );"_s;
 constexpr auto createUniqueIndexTopFrameLinkDecorationsFrom = "CREATE UNIQUE INDEX IF NOT EXISTS TopFrameLinkDecorationsFrom_toDomainID_fromDomainID on TopFrameLinkDecorationsFrom ( toDomainID, fromDomainID );"_s;
+constexpr auto createUniqueIndexTopFrameLoadedThirdPartyScripts = "CREATE UNIQUE INDEX IF NOT EXISTS TopFrameLoadedThirdPartyScripts_topFrameDomainID_subresourceDomainID on TopFrameLoadedThirdPartyScripts ( topFrameDomainID, subresourceDomainID );"_s;
 constexpr auto createUniqueIndexSubframeUnderTopFrameDomains = "CREATE UNIQUE INDEX IF NOT EXISTS SubframeUnderTopFrameDomains_subFrameDomainID_topFrameDomainID on SubframeUnderTopFrameDomains ( subFrameDomainID, topFrameDomainID );"_s;
 constexpr auto createUniqueIndexSubresourceUnderTopFrameDomains = "CREATE UNIQUE INDEX IF NOT EXISTS SubresourceUnderTopFrameDomains_subresourceDomainID_topFrameDomainID on SubresourceUnderTopFrameDomains ( subresourceDomainID, topFrameDomainID );"_s;
 constexpr auto createUniqueIndexSubresourceUniqueRedirectsTo = "CREATE UNIQUE INDEX IF NOT EXISTS SubresourceUniqueRedirectsTo_subresourceDomainID_toDomainID on SubresourceUniqueRedirectsTo ( subresourceDomainID, toDomainID );"_s;
@@ -216,6 +225,7 @@ ResourceLoadStatisticsDatabaseStore::ResourceLoadStatisticsDatabaseStore(WebReso
     , m_insertTopLevelDomainStatement(m_database, insertTopLevelDomainQuery)
     , m_domainIDFromStringStatement(m_database, domainIDFromStringQuery)
     , m_topFrameLinkDecorationsFromExists(m_database, topFrameLinkDecorationsFromExistsQuery)
+    , m_topFrameLoadedThirdPartyScriptsExists(m_database, topFrameLoadedThirdPartyScriptsExistsQuery)
     , m_subframeUnderTopFrameDomainExists(m_database, subframeUnderTopFrameDomainExistsQuery)
     , m_subresourceUnderTopFrameDomainExists(m_database, subresourceUnderTopFrameDomainExistsQuery)
     , m_subresourceUniqueRedirectsToExists(m_database, subresourceUniqueRedirectsToExistsQuery)
@@ -340,6 +350,7 @@ bool ResourceLoadStatisticsDatabaseStore::createUniqueIndices()
         || !m_database.executeCommand(createUniqueIndexTopFrameUniqueRedirectsTo)
         || !m_database.executeCommand(createUniqueIndexTopFrameUniqueRedirectsFrom)
         || !m_database.executeCommand(createUniqueIndexTopFrameLinkDecorationsFrom)
+        || !m_database.executeCommand(createUniqueIndexTopFrameLoadedThirdPartyScripts)
         || !m_database.executeCommand(createUniqueIndexSubframeUnderTopFrameDomains)
         || !m_database.executeCommand(createUniqueIndexSubresourceUnderTopFrameDomains)
         || !m_database.executeCommand(createUniqueIndexSubresourceUniqueRedirectsTo)
@@ -384,6 +395,11 @@ bool ResourceLoadStatisticsDatabaseStore::createSchema()
         return false;
     }
     
+    if (!m_database.executeCommand(createTopFrameLoadedThirdPartyScripts)) {
+        LOG_ERROR("Could not create TopFrameLoadedThirdPartyScripts table in database (%i) - %s", m_database.lastError(), m_database.lastErrorMsg());
+        return false;
+    }
+    
     if (!m_database.executeCommand(createSubframeUnderTopFrameDomains)) {
         LOG_ERROR("Could not create SubframeUnderTopFrameDomains table in database (%i) - %s", m_database.lastError(), m_database.lastErrorMsg());
         return false;
@@ -435,6 +451,7 @@ bool ResourceLoadStatisticsDatabaseStore::prepareStatements()
         || m_isGrandfatheredStatement.prepare() != SQLITE_OK
         || m_findExpiredUserInteractionStatement.prepare() != SQLITE_OK
         || m_topFrameLinkDecorationsFromExists.prepare() != SQLITE_OK
+        || m_topFrameLoadedThirdPartyScriptsExists.prepare() != SQLITE_OK
         || m_countPrevalentResourcesStatement.prepare() != SQLITE_OK
         || m_countPrevalentResourcesWithUserInteractionStatement.prepare() != SQLITE_OK
         || m_countPrevalentResourcesWithoutUserInteractionStatement.prepare() != SQLITE_OK
@@ -600,6 +617,7 @@ void ResourceLoadStatisticsDatabaseStore::insertDomainRelationships(const Resour
     insertDomainRelationshipList(subresourceUniqueRedirectsToQuery, loadStatistics.subresourceUniqueRedirectsTo, registrableDomainID.value());
     insertDomainRelationshipList(subresourceUniqueRedirectsFromQuery, loadStatistics.subresourceUniqueRedirectsFrom, registrableDomainID.value());
     insertDomainRelationshipList(topFrameLinkDecorationsFromQuery, loadStatistics.topFrameLinkDecorationsFrom, registrableDomainID.value());
+    insertDomainRelationshipList(topFrameLoadedThirdPartyScriptsQuery, loadStatistics.topFrameLoadedThirdPartyScripts, registrableDomainID.value());
 }
 
 void ResourceLoadStatisticsDatabaseStore::populateFromMemoryStore(const ResourceLoadStatisticsMemoryStore& memoryStore)
@@ -2227,6 +2245,8 @@ String ResourceLoadStatisticsDatabaseStore::getSubStatisticStatement(const Strin
         return "SELECT fromDomainID from TopFrameUniqueRedirectsFrom WHERE targetDomainID = ?";
     if (tableName == "TopFrameLinkDecorationsFrom")
         return "SELECT fromDomainID from TopFrameLinkDecorationsFrom WHERE toDomainID = ?";
+    if (tableName == "TopFrameLoadedThirdPartyScripts")
+        return "SELECT subresourceDomainID from TopFrameLoadedThirdPartyScripts WHERE topFrameDomainID = ?";
     if (tableName == "SubframeUnderTopFrameDomains")
         return "SELECT topFrameDomainID from SubframeUnderTopFrameDomains WHERE subFrameDomainID = ?";
     if (tableName == "SubresourceUnderTopFrameDomains")
@@ -2283,9 +2303,6 @@ void ResourceLoadStatisticsDatabaseStore::resourceToString(StringBuilder& builde
     builder.appendLiteral("Registrable domain: ");
     builder.append(domain);
     builder.append('\n');
-    builder.appendLiteral("    lastSeen: ");
-    builder.appendFixedPrecisionNumber(m_getResourceDataByDomainNameStatement.getColumnDouble(LastSeenIndex));
-    builder.append('\n');
     
     // User interaction
     appendBoolean(builder, "hadUserInteraction", m_getResourceDataByDomainNameStatement.getColumnInt(HadUserInteractionIndex));
@@ -2303,8 +2320,10 @@ void ResourceLoadStatisticsDatabaseStore::resourceToString(StringBuilder& builde
     appendSubStatisticList(builder, "TopFrameUniqueRedirectsTo", domain);
     appendSubStatisticList(builder, "TopFrameUniqueRedirectsFrom", domain);
     appendSubStatisticList(builder, "TopFrameLinkDecorationsFrom", domain);
+    appendSubStatisticList(builder, "TopFrameLoadedThirdPartyScripts", domain);
 
     appendBoolean(builder, "IsScheduledForAllButCookieDataRemoval", m_getResourceDataByDomainNameStatement.getColumnInt(IsScheduledForAllButCookieDataRemovalIndex));
+    builder.append('\n');
 
     // Subframe stats
     appendSubStatisticList(builder, "SubframeUnderTopFrameDomains", domain);
@@ -2326,7 +2345,6 @@ void ResourceLoadStatisticsDatabaseStore::resourceToString(StringBuilder& builde
     resetStatement(m_getResourceDataByDomainNameStatement);
 }
 
-
 } // namespace WebKit
 
 #endif
index bf3f3e9..7cbfa8c 100644 (file)
@@ -223,6 +223,7 @@ private:
     WebCore::SQLiteStatement m_insertTopLevelDomainStatement;
     mutable WebCore::SQLiteStatement m_domainIDFromStringStatement;
     mutable WebCore::SQLiteStatement m_topFrameLinkDecorationsFromExists;
+    mutable WebCore::SQLiteStatement m_topFrameLoadedThirdPartyScriptsExists;
     mutable WebCore::SQLiteStatement m_subframeUnderTopFrameDomainExists;
     mutable WebCore::SQLiteStatement m_subresourceUnderTopFrameDomainExists;
     mutable WebCore::SQLiteStatement m_subresourceUniqueRedirectsToExists;
index 9bc501d..030b1f1 100644 (file)
@@ -51,7 +51,7 @@
 namespace WebKit {
 using namespace WebCore;
 
-constexpr unsigned statisticsModelVersion { 16 };
+constexpr unsigned statisticsModelVersion { 17 };
 
 struct StatisticsLastSeen {
     RegistrableDomain domain;
index 3ee38d9..7132e88 100644 (file)
@@ -2684,6 +2684,7 @@ void ArgumentCoder<ResourceLoadStatistics>::encode(Encoder& encoder, const WebCo
     // Top frame stats
     encoder << statistics.topFrameUniqueRedirectsTo;
     encoder << statistics.topFrameUniqueRedirectsFrom;
+    encoder << statistics.topFrameLoadedThirdPartyScripts;
 
     // Subframe stats
     encoder << statistics.subframeUnderTopFrameDomains;
@@ -2756,6 +2757,12 @@ Optional<ResourceLoadStatistics> ArgumentCoder<ResourceLoadStatistics>::decode(D
         return WTF::nullopt;
     statistics.topFrameUniqueRedirectsFrom = WTFMove(*topFrameUniqueRedirectsFrom);
 
+    Optional<HashSet<RegistrableDomain>> topFrameLoadedThirdPartyScripts;
+    decoder >> topFrameLoadedThirdPartyScripts;
+    if (!topFrameLoadedThirdPartyScripts)
+        return WTF::nullopt;
+    statistics.topFrameLoadedThirdPartyScripts = WTFMove(*topFrameLoadedThirdPartyScripts);
+
     // Subframe stats
     Optional<HashSet<RegistrableDomain>> subframeUnderTopFrameDomains;
     decoder >> subframeUnderTopFrameDomains;
index 5e4f067..4263d19 100644 (file)
@@ -235,7 +235,7 @@ void WebResourceLoadObserver::logScreenAPIAccessed(const Document& document, con
 #endif
 }
 
-void WebResourceLoadObserver::logSubresourceLoading(const Frame* frame, const ResourceRequest& newRequest, const ResourceResponse& redirectResponse)
+void WebResourceLoadObserver::logSubresourceLoading(const Frame* frame, const ResourceRequest& newRequest, const ResourceResponse& redirectResponse, FetchDestinationIsScriptLike isScriptLike)
 {
     ASSERT(frame->page());
 
@@ -273,6 +273,13 @@ void WebResourceLoadObserver::logSubresourceLoading(const Frame* frame, const Re
         scheduleNotificationIfNeeded();
     }
 
+    if (frame->isMainFrame() && isScriptLike == FetchDestinationIsScriptLike::Yes) {
+        auto& topFrameStatistics = ensureResourceStatisticsForRegistrableDomain(topFrameDomain);
+        topFrameStatistics.topFrameLoadedThirdPartyScripts.add(targetDomain);
+
+        scheduleNotificationIfNeeded();
+    }
+
     if (isRedirect) {
         auto& redirectingOriginStatistics = ensureResourceStatisticsForRegistrableDomain(redirectedFromDomain);
         redirectingOriginStatistics.subresourceUniqueRedirectsTo.add(targetDomain);
index 92de7bc..fae48c6 100644 (file)
@@ -39,7 +39,7 @@ public:
     WebResourceLoadObserver();
     ~WebResourceLoadObserver();
 
-    void logSubresourceLoading(const WebCore::Frame*, const WebCore::ResourceRequest& newRequest, const WebCore::ResourceResponse& redirectResponse) final;
+    void logSubresourceLoading(const WebCore::Frame*, const WebCore::ResourceRequest& newRequest, const WebCore::ResourceResponse& redirectResponse, FetchDestinationIsScriptLike) final;
     void logWebSocketLoading(const URL& targetURL, const URL& mainFrameURL) final;
     void logUserInteractionWithReducedTimeResolution(const WebCore::Document&) final;
     void logFontLoad(const WebCore::Document&, const String& familyName, bool loadStatus) final;