Domain relationships in the ITP Database should be inserted in a single query and...
authorcommit-queue@webkit.org <commit-queue@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 7 Oct 2019 23:17:28 +0000 (23:17 +0000)
committercommit-queue@webkit.org <commit-queue@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 7 Oct 2019 23:17:28 +0000 (23:17 +0000)
https://bugs.webkit.org/show_bug.cgi?id=202604
<rdar://problem/55995831>

Patch by Kate Cheney <katherine_cheney@apple.com> on 2019-10-07
Source/WebKit:

Reviewed by Chris Dumez.

This patch addresses two clean-ups for the ITP SQLite Database Store.
First, by using INSERT OR IGNORE as opposed to INSERT, it eliminates
the need to check if a relationship already exists in the database
before inserting it. Second, instead of looping through domain lists
and inserting each relationship as a separate query, this patch now
converts lists to a string which SQLite can use to insert multiple
rows into a database using a single query.

Some Exists queries could not be deleted because they were being
used for testing.

* NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:
(WebKit::ResourceLoadStatisticsDatabaseStore::ResourceLoadStatisticsDatabaseStore):
(WebKit::ResourceLoadStatisticsDatabaseStore::prepareStatements):
Removed old insert queries and replaced them with queries able to
insert string-lists of domains in a single query. Also reorganized
the queries by functionality to make them easier to find and edit.

(WebKit::ResourceLoadStatisticsDatabaseStore::createUniqueIndices):
(WebKit::ResourceLoadStatisticsDatabaseStore::createSchema):
In order to properly take advantage of the INSERT OR IGNORE
functionality, unique indices must be established so the SQLite table
is aware of what patterns to look for when ignoring a new insert.

(WebKit::ResourceLoadStatisticsDatabaseStore::ensureAndmakeDomainList):
(WebKit::ResourceLoadStatisticsDatabaseStore::insertDomainRelationshipList):
(WebKit::ResourceLoadStatisticsDatabaseStore::insertDomainRelationships):
Looping through the domain list is no longer needed with the new query
changes. Additionally, ensuring a domain is in the Observed Domains
table of the database must be done before utilizing any inserting of
relationships because the relationship queries rely on fetching the
topFrame domainID from the ObservedDomains table.

(WebKit::ResourceLoadStatisticsDatabaseStore::grantStorageAccess):
(WebKit::ResourceLoadStatisticsDatabaseStore::logFrameNavigation):
(WebKit::ResourceLoadStatisticsDatabaseStore::logCrossSiteLoadWithLinkDecoration):
(WebKit::ResourceLoadStatisticsDatabaseStore::setSubframeUnderTopFrameDomain):
(WebKit::ResourceLoadStatisticsDatabaseStore::setSubresourceUnderTopFrameDomain):
(WebKit::ResourceLoadStatisticsDatabaseStore::setSubresourceUniqueRedirectTo):
(WebKit::ResourceLoadStatisticsDatabaseStore::setSubresourceUniqueRedirectFrom):
(WebKit::ResourceLoadStatisticsDatabaseStore::setTopFrameUniqueRedirectTo):
(WebKit::ResourceLoadStatisticsDatabaseStore::setTopFrameUniqueRedirectFrom):
With the ensure check being done once in the ensureAndmakeDomainList
function, these functions no longer have to make that check before
inserting.
* NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h:

* NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:
(WebKit::WebResourceLoadStatisticsStore::mergeStatisticForTesting):
(WebKit::WebResourceLoadStatisticsStore::isRelationshipOnlyInDatabaseOnce):
* NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h:
Changes in WeResourceLoadStatisticsStore were for testing only. This
patch updated the merge statistic testing to also test this change by
having a topFrameDomain list with more than one domain.

* NetworkProcess/NetworkProcess.cpp:
(WebKit::NetworkProcess::mergeStatisticForTesting):
(WebKit::NetworkProcess::isRelationshipOnlyInDatabaseOnce):
* NetworkProcess/NetworkProcess.h:
* NetworkProcess/NetworkProcess.messages.in:
* UIProcess/API/C/WKWebsiteDataStoreRef.cpp:
(WKWebsiteDataStoreSetStatisticsMergeStatistic):
(WKWebsiteDataStoreIsStatisticsOnlyInDatabaseOnce):
* UIProcess/API/C/WKWebsiteDataStoreRef.h:
* UIProcess/Network/NetworkProcessProxy.cpp:
(WebKit::NetworkProcessProxy::mergeStatisticForTesting):
(WebKit::NetworkProcessProxy::isRelationshipOnlyInDatabaseOnce):
* UIProcess/Network/NetworkProcessProxy.h:
* UIProcess/WebsiteData/WebsiteDataStore.cpp:
(WebKit::WebsiteDataStore::mergeStatisticForTesting):
(WebKit::WebsiteDataStore::isRelationshipOnlyInDatabaseOnce):
* UIProcess/WebsiteData/WebsiteDataStore.h:
Added a new function for testing that there are no repeat inserts
into the database. Updated mergeStatistics to test the list-insert
functionality.

Tools:

Reviewed by Chris Dumez.

Updates to testing infrastructure to test successful list-merging of
top domains into the ITP SQLite database and test against repeat
inserts.
* WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl:
* WebKitTestRunner/InjectedBundle/TestRunner.cpp:
(WTR::TestRunner::setStatisticsMergeStatistic):
(WTR::TestRunner::isStatisticsHasHadUserInteraction):
(WTR::TestRunner::isStatisticsOnlyInDatabaseOnce):
* WebKitTestRunner/InjectedBundle/TestRunner.h:
* WebKitTestRunner/TestController.cpp:
(WTR::TestController::setStatisticsMergeStatistic):
(WTR::TestController::isStatisticsOnlyInDatabaseOnce):
* WebKitTestRunner/TestController.h:
* WebKitTestRunner/TestInvocation.cpp:
(WTR::TestInvocation::didReceiveSynchronousMessageFromInjectedBundle):

LayoutTests:

Reviewed by Chris Dumez.

Edited the merge-statistics tests to also test the list-insertion
change made in this patch. Also added a new test to ensure repeat
inserts only result in one entry in the database.

* http/tests/resourceLoadStatistics/many-inserts-only-insert-once-expected.txt: Added.
* http/tests/resourceLoadStatistics/many-inserts-only-insert-once.html: Added.
* http/tests/resourceLoadStatistics/merge-statistic-does-not-overwrite-database.html:
* http/tests/resourceLoadStatistics/merge-statistic-does-overwrite-database-expected.txt:
* http/tests/resourceLoadStatistics/merge-statistic-does-overwrite-database.html:
* http/tests/resourceLoadStatistics/merge-statistic-does-partially-overwrite-database.html:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@250804 268f45cc-cd09-0410-ab3c-d52691b4dbfc

28 files changed:
LayoutTests/ChangeLog
LayoutTests/http/tests/resourceLoadStatistics/many-inserts-only-insert-once-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/resourceLoadStatistics/many-inserts-only-insert-once.html [new file with mode: 0644]
LayoutTests/http/tests/resourceLoadStatistics/merge-statistic-does-not-overwrite-database.html
LayoutTests/http/tests/resourceLoadStatistics/merge-statistic-does-overwrite-database-expected.txt
LayoutTests/http/tests/resourceLoadStatistics/merge-statistic-does-overwrite-database.html
LayoutTests/http/tests/resourceLoadStatistics/merge-statistic-does-partially-overwrite-database.html
Source/WebKit/ChangeLog
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h
Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp
Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h
Source/WebKit/NetworkProcess/NetworkProcess.cpp
Source/WebKit/NetworkProcess/NetworkProcess.h
Source/WebKit/NetworkProcess/NetworkProcess.messages.in
Source/WebKit/UIProcess/API/C/WKWebsiteDataStoreRef.cpp
Source/WebKit/UIProcess/API/C/WKWebsiteDataStoreRef.h
Source/WebKit/UIProcess/Network/NetworkProcessProxy.cpp
Source/WebKit/UIProcess/Network/NetworkProcessProxy.h
Source/WebKit/UIProcess/WebsiteData/WebsiteDataStore.cpp
Source/WebKit/UIProcess/WebsiteData/WebsiteDataStore.h
Tools/ChangeLog
Tools/WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl
Tools/WebKitTestRunner/InjectedBundle/TestRunner.cpp
Tools/WebKitTestRunner/InjectedBundle/TestRunner.h
Tools/WebKitTestRunner/TestController.cpp
Tools/WebKitTestRunner/TestController.h
Tools/WebKitTestRunner/TestInvocation.cpp

index 12b6dd0..56ab36e 100644 (file)
@@ -1,3 +1,22 @@
+2019-10-07  Kate Cheney  <katherine_cheney@apple.com>
+
+        Domain relationships in the ITP Database should be inserted in a single query and ignore repeat insert attempts. (202604)
+        https://bugs.webkit.org/show_bug.cgi?id=202604
+        <rdar://problem/55995831>
+
+        Reviewed by Chris Dumez.
+
+        Edited the merge-statistics tests to also test the list-insertion
+        change made in this patch. Also added a new test to ensure repeat
+        inserts only result in one entry in the database.
+
+        * http/tests/resourceLoadStatistics/many-inserts-only-insert-once-expected.txt: Added.
+        * http/tests/resourceLoadStatistics/many-inserts-only-insert-once.html: Added.
+        * http/tests/resourceLoadStatistics/merge-statistic-does-not-overwrite-database.html:
+        * http/tests/resourceLoadStatistics/merge-statistic-does-overwrite-database-expected.txt:
+        * http/tests/resourceLoadStatistics/merge-statistic-does-overwrite-database.html:
+        * http/tests/resourceLoadStatistics/merge-statistic-does-partially-overwrite-database.html:
+
 2019-10-07  Sihui Liu  <sihui_liu@apple.com>
 
         ASSERTION FAILED: m_transactionOperationsInProgressQueue.first() == &operation in IDBTransaction::operationCompletedOnClient
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/many-inserts-only-insert-once-expected.txt b/LayoutTests/http/tests/resourceLoadStatistics/many-inserts-only-insert-once-expected.txt
new file mode 100644 (file)
index 0000000..aa28052
--- /dev/null
@@ -0,0 +1,11 @@
+Tests that many inserts into the database only result in one row.
+
+On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
+
+
+PASS url set as subframe
+PASS Relationships are only in the database once.
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/many-inserts-only-insert-once.html b/LayoutTests/http/tests/resourceLoadStatistics/many-inserts-only-insert-once.html
new file mode 100644 (file)
index 0000000..c019b0d
--- /dev/null
@@ -0,0 +1,75 @@
+<!DOCTYPE html>
+<html>
+<head>
+    <meta charset="UTF-8">
+        <title>Tests that many inserts into the database only result in one row.</title>
+    <script src="/js-test-resources/js-test.js"></script>
+    <script src="resources/util.js"></script>
+</head>
+<body>
+<script>
+    testRunner.setUseITPDatabase(true);
+    description("Tests that many inserts into the database only result in one row.");
+    jsTestIsAsync = true;
+
+    const topFrameUrl = "http://127.0.0.1:8000/temp";
+    const subframeUrl = "http://127.0.1.1:8000/temp";
+
+    function checkClassificationAndContinue() {
+        if (!testRunner.isStatisticsRegisteredAsSubFrameUnder(subframeUrl, topFrameUrl)) {
+            testFailed("url did not get set as subframe.");
+            setEnableFeature(false, finishJSTest);
+        } else if (!testRunner.isStatisticsRegisteredAsRedirectingTo(subframeUrl, topFrameUrl)) {
+            testFailed("url did not get set as redirecting to top frame.");
+            setEnableFeature(false, finishJSTest);
+        } else if (!testRunner.isStatisticsRegisteredAsSubresourceUnder(subframeUrl, topFrameUrl)) {
+            testFailed("url did not get set as subresource.");
+            setEnableFeature(false, finishJSTest);
+        } else {
+            testPassed("url set as subframe");
+            checkCountStatistics();
+        }
+    }
+
+    function insertManyTimes(url) {
+        testRunner.setStatisticsSubframeUnderTopFrameOrigin(url, topFrameUrl);
+        testRunner.setStatisticsSubframeUnderTopFrameOrigin(url, topFrameUrl);
+        testRunner.setStatisticsSubframeUnderTopFrameOrigin(url, topFrameUrl);
+        testRunner.setStatisticsSubframeUnderTopFrameOrigin(url, topFrameUrl);
+        
+        testRunner.setStatisticsSubresourceUnderTopFrameOrigin(url, topFrameUrl);
+        testRunner.setStatisticsSubresourceUnderTopFrameOrigin(url, topFrameUrl);
+        testRunner.setStatisticsSubresourceUnderTopFrameOrigin(url, topFrameUrl);
+        testRunner.setStatisticsSubresourceUnderTopFrameOrigin(url, topFrameUrl);
+        
+        
+        testRunner.setStatisticsSubresourceUniqueRedirectTo(url, topFrameUrl);
+        testRunner.setStatisticsSubresourceUniqueRedirectTo(url, topFrameUrl);
+        testRunner.setStatisticsSubresourceUniqueRedirectTo(url, topFrameUrl);
+        testRunner.setStatisticsSubresourceUniqueRedirectTo(url, topFrameUrl);
+    }
+
+    function setUpStatisticsAndContinue() {
+        insertManyTimes(subframeUrl);
+        testRunner.installStatisticsDidScanDataRecordsCallback(checkClassificationAndContinue);
+        testRunner.statisticsProcessStatisticsAndDataRecords();
+    }
+
+    function checkCountStatistics(result) {
+        if (testRunner.isStatisticsOnlyInDatabaseOnce(subframeUrl, topFrameUrl))
+            testPassed("Relationships are only in the database once.");
+        else
+            testFailed("Relationships are in the database more than once.");
+
+        setEnableFeature(false, finishJSTest);
+    }
+
+    if (window.testRunner) {
+        setEnableFeature(true, function() {
+            testRunner.setStatisticsNotifyPagesWhenDataRecordsWereScanned(true);
+            setUpStatisticsAndContinue();
+        });
+    }
+</script>
+</body>
+</html>
index 4249eec..4d4a283 100644 (file)
@@ -16,7 +16,7 @@
     const newerTimestamp = olderTimestamp + 10;
                                       
     function insertSecondStatistic() {
-      testRunner.setStatisticsMergeStatistic(url, "", olderTimestamp, false, 99, false, false, false, 0, function() {
+      testRunner.setStatisticsMergeStatistic(url, "", "", olderTimestamp, false, 99, false, false, false, 0, function() {
           if (testRunner.isStatisticsPrevalentResource(url))
               testPassed("Host did not overwrite old prevalent resource value.");
           else
@@ -44,7 +44,7 @@
     }
     
     function runTestRunnerTest() {
-        testRunner.setStatisticsMergeStatistic(url, "", newerTimestamp, true, 100, true, true, true, 1, function() {
+        testRunner.setStatisticsMergeStatistic(url, "", "", newerTimestamp, true, 100, true, true, true, 1, function() {
             if (!testRunner.isStatisticsPrevalentResource(url))
                 testFailed("Host got set as prevalent resource.");
 
index adcf7e8..5fb1e4a 100644 (file)
@@ -7,7 +7,8 @@ PASS Host overwrote old prevalent resource value.
 PASS Host overwrote old very prevalent resource value.
 PASS Host overwrote old user interaction value.
 PASS Host overwrote old grandfathered value.
-PASS Host set as subframe under top frame.
+PASS Host set as subframe under top frame1.
+PASS Host set as subframe under top frame2.
 PASS successfullyParsed is true
 
 TEST COMPLETE
index 1412e15..1a19949 100644 (file)
     jsTestIsAsync = true;
 
     const url = "http://127.0.0.1:8000";
-    const sampleTopFrameURL = "http://topFrameDomain:8000";
+    const sampleTopFrameURL1 = "http://topFrameDomain1:8000";
+    const sampleTopFrameURL2 = "http://topFrameDomain2:8000";
     const olderTimestamp = Math.round((new Date()).getTime() / 1000);
     const newerTimestamp = olderTimestamp + 10;
     const mostRecentUIToTriggerFirstPartyInteractionCount = newerTimestamp + 90000;
                                       
     function insertSecondStatistic() {
     
-        // set this resource as prevalent so it is in the statistics store
-        testRunner.setStatisticsPrevalentResource(sampleTopFrameURL, true, function() {
-        
-              testRunner.setStatisticsMergeStatistic(url, sampleTopFrameURL, newerTimestamp, true, mostRecentUIToTriggerFirstPartyInteractionCount, true, true, true, 1, function() {
-                  if (testRunner.isStatisticsPrevalentResource(url))
-                      testPassed("Host overwrote old prevalent resource value.");
-                  else
-                      testFailed("Host not set as prevalent resource.");
+        // set url1 as prevalent so it is in the statistics store
+        // set url2 having UI so it is in the statistics store
+        testRunner.setStatisticsPrevalentResource(sampleTopFrameURL1, true, function() {
+            testRunner.setStatisticsHasHadUserInteraction(sampleTopFrameURL2, true, function () {
+                  testRunner.setStatisticsMergeStatistic(url, sampleTopFrameURL1, sampleTopFrameURL2, newerTimestamp, true, mostRecentUIToTriggerFirstPartyInteractionCount, true, true, true, 1, function() {
+                      if (testRunner.isStatisticsPrevalentResource(url))
+                          testPassed("Host overwrote old prevalent resource value.");
+                      else
+                          testFailed("Host not set as prevalent resource.");
 
-                  if (testRunner.isStatisticsVeryPrevalentResource(url))
-                      testPassed("Host overwrote old very prevalent resource value.");
-                  else
-                      testFailed("Host not set as very prevalent resource.");
-                                                     
-                  if (testRunner.isStatisticsHasHadUserInteraction(url))
-                      testPassed("Host overwrote old user interaction value.");
-                  else
-                      testFailed("Host not logged for user interaction.");
-                   
-                  if (testRunner.isStatisticsGrandfathered(url))
-                      testPassed("Host overwrote old grandfathered value.");
-                  else
-                      testFailed("Host not set as grandfathered.");
+                      if (testRunner.isStatisticsVeryPrevalentResource(url))
+                          testPassed("Host overwrote old very prevalent resource value.");
+                      else
+                          testFailed("Host not set as very prevalent resource.");
+                                                         
+                      if (testRunner.isStatisticsHasHadUserInteraction(url))
+                          testPassed("Host overwrote old user interaction value.");
+                      else
+                          testFailed("Host not logged for user interaction.");
+                       
+                      if (testRunner.isStatisticsGrandfathered(url))
+                          testPassed("Host overwrote old grandfathered value.");
+                      else
+                          testFailed("Host not set as grandfathered.");
 
-                  if (testRunner.isStatisticsRegisteredAsSubFrameUnder(url, sampleTopFrameURL))
-                      testPassed("Host set as subframe under top frame.");
-                  else
-                      testFailed("Host not set as subframe under top frame.");
+                      if (testRunner.isStatisticsRegisteredAsSubFrameUnder(url, sampleTopFrameURL1))
+                          testPassed("Host set as subframe under top frame1.");
+                      else
+                          testFailed("Host not set as subframe under top frame1.");
+                      if (testRunner.isStatisticsRegisteredAsSubFrameUnder(url, sampleTopFrameURL2))
+                          testPassed("Host set as subframe under top frame2.");
+                      else
+                          testFailed("Host not set as subframe under top frame2.");
 
-                  testRunner.statisticsResetToConsistentState(function() {
-                      finishJSTest();
+                      testRunner.statisticsResetToConsistentState(function() {
+                          finishJSTest();
+                      });
                   });
-              });
-         });
+             });
+        });
     }
     
     function runTestRunnerTest() {
-        testRunner.setStatisticsMergeStatistic(url, "", olderTimestamp, false, 0, false, false, false, 0, function() {
+        testRunner.setStatisticsMergeStatistic(url, "", "", olderTimestamp, false, 0, false, false, false, 0, function() {
             if (testRunner.isStatisticsPrevalentResource(url))
                 testFailed("Host got set as prevalent resource.");
 
index 44d16d6..ab3c72b 100644 (file)
@@ -21,7 +21,7 @@
         // set this resource as prevalent so it is in the statistics store
         testRunner.setStatisticsPrevalentResource(sampleTopFrameURL, true, function() {
         
-              testRunner.setStatisticsMergeStatistic(url, sampleTopFrameURL, newerTimestamp, false, mostRecentUIToTriggerFirstPartyInteractionCount, true, true, true, 1, function() {
+              testRunner.setStatisticsMergeStatistic(url, sampleTopFrameURL, "", newerTimestamp, false, mostRecentUIToTriggerFirstPartyInteractionCount, true, true, true, 1, function() {
                   if (testRunner.isStatisticsPrevalentResource(url))
                       testPassed("Host overwrote old prevalent resource value.");
                   else
@@ -55,7 +55,7 @@
     }
     
     function runTestRunnerTest() {
-        testRunner.setStatisticsMergeStatistic(url, "", olderTimestamp, false, 0, false, false, false, 0, function() {
+        testRunner.setStatisticsMergeStatistic(url, "", "", olderTimestamp, false, 0, false, false, false, 0, function() {
             if (testRunner.isStatisticsPrevalentResource(url))
                 testFailed("Host got set as prevalent resource.");
 
index c3f2594..12342c7 100644 (file)
@@ -1,3 +1,87 @@
+2019-10-07  Kate Cheney  <katherine_cheney@apple.com>
+
+        Domain relationships in the ITP Database should be inserted in a single query and ignore repeat insert attempts. (202604)
+        https://bugs.webkit.org/show_bug.cgi?id=202604
+        <rdar://problem/55995831>
+
+        Reviewed by Chris Dumez. 
+        
+        This patch addresses two clean-ups for the ITP SQLite Database Store.
+        First, by using INSERT OR IGNORE as opposed to INSERT, it eliminates
+        the need to check if a relationship already exists in the database
+        before inserting it. Second, instead of looping through domain lists
+        and inserting each relationship as a separate query, this patch now
+        converts lists to a string which SQLite can use to insert multiple
+        rows into a database using a single query.
+
+        Some Exists queries could not be deleted because they were being
+        used for testing.
+
+        * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:
+        (WebKit::ResourceLoadStatisticsDatabaseStore::ResourceLoadStatisticsDatabaseStore):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::prepareStatements):
+        Removed old insert queries and replaced them with queries able to
+        insert string-lists of domains in a single query. Also reorganized
+        the queries by functionality to make them easier to find and edit.
+
+        (WebKit::ResourceLoadStatisticsDatabaseStore::createUniqueIndices):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::createSchema):
+        In order to properly take advantage of the INSERT OR IGNORE
+        functionality, unique indices must be established so the SQLite table
+        is aware of what patterns to look for when ignoring a new insert.
+
+        (WebKit::ResourceLoadStatisticsDatabaseStore::ensureAndmakeDomainList):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::insertDomainRelationshipList):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::insertDomainRelationships):
+        Looping through the domain list is no longer needed with the new query
+        changes. Additionally, ensuring a domain is in the Observed Domains
+        table of the database must be done before utilizing any inserting of 
+        relationships because the relationship queries rely on fetching the
+        topFrame domainID from the ObservedDomains table.
+
+        (WebKit::ResourceLoadStatisticsDatabaseStore::grantStorageAccess):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::logFrameNavigation):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::logCrossSiteLoadWithLinkDecoration):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::setSubframeUnderTopFrameDomain):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::setSubresourceUnderTopFrameDomain):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::setSubresourceUniqueRedirectTo):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::setSubresourceUniqueRedirectFrom):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::setTopFrameUniqueRedirectTo):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::setTopFrameUniqueRedirectFrom):
+        With the ensure check being done once in the ensureAndmakeDomainList
+        function, these functions no longer have to make that check before
+        inserting.
+        * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h:
+
+        * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:
+        (WebKit::WebResourceLoadStatisticsStore::mergeStatisticForTesting):
+        (WebKit::WebResourceLoadStatisticsStore::isRelationshipOnlyInDatabaseOnce):
+        * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h:
+        Changes in WeResourceLoadStatisticsStore were for testing only. This
+        patch updated the merge statistic testing to also test this change by
+        having a topFrameDomain list with more than one domain.
+
+        * NetworkProcess/NetworkProcess.cpp:
+        (WebKit::NetworkProcess::mergeStatisticForTesting):
+        (WebKit::NetworkProcess::isRelationshipOnlyInDatabaseOnce):
+        * NetworkProcess/NetworkProcess.h:
+        * NetworkProcess/NetworkProcess.messages.in:
+        * UIProcess/API/C/WKWebsiteDataStoreRef.cpp:
+        (WKWebsiteDataStoreSetStatisticsMergeStatistic):
+        (WKWebsiteDataStoreIsStatisticsOnlyInDatabaseOnce):
+        * UIProcess/API/C/WKWebsiteDataStoreRef.h:
+        * UIProcess/Network/NetworkProcessProxy.cpp:
+        (WebKit::NetworkProcessProxy::mergeStatisticForTesting):
+        (WebKit::NetworkProcessProxy::isRelationshipOnlyInDatabaseOnce):
+        * UIProcess/Network/NetworkProcessProxy.h:
+        * UIProcess/WebsiteData/WebsiteDataStore.cpp:
+        (WebKit::WebsiteDataStore::mergeStatisticForTesting):
+        (WebKit::WebsiteDataStore::isRelationshipOnlyInDatabaseOnce):
+        * UIProcess/WebsiteData/WebsiteDataStore.h:
+        Added a new function for testing that there are no repeat inserts
+        into the database. Updated mergeStatistics to test the list-insert
+        functionality.
+
 2019-10-07  Yusuke Suzuki  <ysuzuki@apple.com>
 
         [JSC] Change signature of HostFunction to (JSGlobalObject*, CallFrame*)
index 88fdb7e..55c3dfb 100644 (file)
@@ -62,57 +62,56 @@ using namespace WebCore;
 #define RELEASE_LOG_ERROR_IF_ALLOWED(sessionID, fmt, ...)  ((void)0)
 #endif
 
+// COUNT Queries
 constexpr auto observedDomainCountQuery = "SELECT COUNT(*) FROM ObservedDomains"_s;
+constexpr auto countSubframeUnderTopFrameQuery = "SELECT COUNT(*) FROM SubframeUnderTopFrameDomains WHERE subFrameDomainID = ? AND topFrameDomainID = ?;"_s;
+constexpr auto countSubresourceUnderTopFrameQuery = "SELECT COUNT(*) FROM SubresourceUnderTopFrameDomains WHERE subresourceDomainID = ? AND topFrameDomainID = ?;"_s;
+constexpr auto countSubresourceUniqueRedirectsToQuery = "SELECT COUNT(*) FROM SubresourceUniqueRedirectsTo WHERE subresourceDomainID = ? AND toDomainID = ?;"_s;
+
+// INSERT Queries
 constexpr auto insertObservedDomainQuery = "INSERT INTO ObservedDomains (registrableDomain, lastSeen, hadUserInteraction,"
     "mostRecentUserInteractionTime, grandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, timesAccessedAsFirstPartyDueToUserInteraction,"
     "timesAccessedAsFirstPartyDueToStorageAccessAPI) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"_s;
 constexpr auto insertTopLevelDomainQuery = "INSERT INTO TopLevelDomains VALUES (?)"_s;
-constexpr auto domainIDFromStringQuery = "SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?"_s;
-constexpr auto storageAccessUnderTopFrameDomainsQuery = "INSERT INTO StorageAccessUnderTopFrameDomains (domainID, topLevelDomainID) "
-    "SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain == ?"_s;
-constexpr auto storageAccessUnderTopFrameDomainsExistsQuery = "SELECT EXISTS (SELECT 1 FROM StorageAccessUnderTopFrameDomains WHERE domainID = ? "
-"AND topLevelDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
-constexpr auto topFrameUniqueRedirectsToQuery = "INSERT INTO TopFrameUniqueRedirectsTo (sourceDomainID, toDomainID) SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain == ?"_s;
-constexpr auto topFrameUniqueRedirectsToExistsQuery = "SELECT EXISTS (SELECT 1 FROM TopFrameUniqueRedirectsTo WHERE sourceDomainID = ? "
-    "AND toDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
-constexpr auto topFrameUniqueRedirectsFromQuery = "INSERT INTO TopFrameUniqueRedirectsFrom (targetDomainID, fromDomainID) "
-    "SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain = ?"_s;
-constexpr auto topFrameUniqueRedirectsFromExistsQuery = "SELECT EXISTS (SELECT 1 FROM TopFrameUniqueRedirectsFrom WHERE targetDomainID = ? "
-    "AND fromDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
-constexpr auto topFrameLinkDecorationsFromQuery = "INSERT INTO TopFrameLinkDecorationsFrom (fromDomainID, toDomainID) "
-    "SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain = ?"_s;
-constexpr auto topFrameLinkDecorationsFromExistsQuery = "SELECT EXISTS (SELECT 1 FROM TopFrameLinkDecorationsFrom WHERE fromDomainID = ? "
-    "AND toDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
-constexpr auto subframeUnderTopFrameDomainsQuery = "INSERT INTO SubframeUnderTopFrameDomains (subFrameDomainID, topFrameDomainID) "
-    "SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain = ?"_s;
+constexpr auto storageAccessUnderTopFrameDomainsQuery = "INSERT OR IGNORE INTO StorageAccessUnderTopFrameDomains (domainID, topLevelDomainID) SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain in ( "_s;
+constexpr auto topFrameUniqueRedirectsToQuery = "INSERT OR IGNORE into TopFrameUniqueRedirectsTo (sourceDomainID, toDomainID) SELECT ?, domainID FROM ObservedDomains where registrableDomain in ( "_s;
+constexpr auto subframeUnderTopFrameDomainsQuery = "INSERT OR IGNORE into SubframeUnderTopFrameDomains (subFrameDomainID, topFrameDomainID) SELECT ?, domainID FROM ObservedDomains where registrableDomain in ( "_s;
+constexpr auto topFrameUniqueRedirectsFromQuery = "INSERT OR IGNORE INTO TopFrameUniqueRedirectsFrom (targetDomainID, fromDomainID) SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain in ( "_s;
+constexpr auto topFrameLinkDecorationsFromQuery = "INSERT OR IGNORE INTO TopFrameLinkDecorationsFrom (fromDomainID, toDomainID) SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain in ( "_s;
+constexpr auto subresourceUnderTopFrameDomainsQuery = "INSERT OR IGNORE INTO SubresourceUnderTopFrameDomains (subresourceDomainID, topFrameDomainID) SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain in ( "_s;
+constexpr auto subresourceUniqueRedirectsToQuery = "INSERT OR IGNORE INTO SubresourceUniqueRedirectsTo (subresourceDomainID, toDomainID) SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain in ( "_s;
+constexpr auto subresourceUniqueRedirectsFromQuery = "INSERT OR IGNORE INTO SubresourceUniqueRedirectsFrom (subresourceDomainID, fromDomainID) SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain in ( "_s;
+
+// EXISTS Queries
 constexpr auto subframeUnderTopFrameDomainExistsQuery = "SELECT EXISTS (SELECT 1 FROM SubframeUnderTopFrameDomains WHERE subFrameDomainID = ? "
     "AND topFrameDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
-constexpr auto subresourceUnderTopFrameDomainsQuery = "INSERT INTO SubresourceUnderTopFrameDomains (subresourceDomainID, topFrameDomainID) "
-    "SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain = ?"_s;
 constexpr auto subresourceUnderTopFrameDomainExistsQuery = "SELECT EXISTS (SELECT 1 FROM SubresourceUnderTopFrameDomains "
     "WHERE subresourceDomainID = ? AND topFrameDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
-constexpr auto subresourceUniqueRedirectsToQuery = "INSERT INTO SubresourceUniqueRedirectsTo (subresourceDomainID, toDomainID) "
-    "SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain == ?"_s;
 constexpr auto subresourceUniqueRedirectsToExistsQuery = "SELECT EXISTS (SELECT 1 FROM SubresourceUniqueRedirectsTo WHERE subresourceDomainID = ? "
     "AND toDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
-constexpr auto subresourceUniqueRedirectsFromQuery = "INSERT INTO SubresourceUniqueRedirectsFrom (subresourceDomainID, fromDomainID) "
-    "SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain == ?"_s;
-constexpr auto subresourceUniqueRedirectsFromExistsQuery = "SELECT EXISTS (SELECT 1 FROM SubresourceUniqueRedirectsFrom WHERE subresourceDomainID = ? "
-    "AND fromDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
+constexpr auto topFrameLinkDecorationsFromExistsQuery = "SELECT EXISTS (SELECT 1 FROM TopFrameLinkDecorationsFrom WHERE fromDomainID = ? "
+"AND toDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
+
+// UPDATE Queries
 constexpr auto mostRecentUserInteractionQuery = "UPDATE ObservedDomains SET hadUserInteraction = ?, mostRecentUserInteractionTime = ? "
     "WHERE registrableDomain = ?"_s;
 constexpr auto updateLastSeenQuery = "UPDATE ObservedDomains SET lastSeen = ? WHERE registrableDomain = ?"_s;
 constexpr auto updateDataRecordsRemovedQuery = "UPDATE ObservedDomains SET dataRecordsRemoved = ? WHERE registrableDomain = ?"_s;
 constexpr auto updatePrevalentResourceQuery = "UPDATE ObservedDomains SET isPrevalent = ? WHERE registrableDomain = ?"_s;
-constexpr auto isPrevalentResourceQuery = "SELECT isPrevalent FROM ObservedDomains WHERE registrableDomain = ?"_s;
 constexpr auto updateVeryPrevalentResourceQuery = "UPDATE ObservedDomains SET isVeryPrevalent = ? WHERE registrableDomain = ?"_s;
-constexpr auto isVeryPrevalentResourceQuery = "SELECT isVeryPrevalent FROM ObservedDomains WHERE registrableDomain = ?"_s;
 constexpr auto clearPrevalentResourceQuery = "UPDATE ObservedDomains SET isPrevalent = 0, isVeryPrevalent = 0 WHERE registrableDomain = ?"_s;
-constexpr auto hadUserInteractionQuery = "SELECT hadUserInteraction, mostRecentUserInteractionTime FROM ObservedDomains WHERE registrableDomain = ?"_s;
 constexpr auto updateGrandfatheredQuery = "UPDATE ObservedDomains SET grandfathered = ? WHERE registrableDomain = ?"_s;
+
+
+// SELECT Queries
+constexpr auto domainIDFromStringQuery = "SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?"_s;
+constexpr auto isPrevalentResourceQuery = "SELECT isPrevalent FROM ObservedDomains WHERE registrableDomain = ?"_s;
+constexpr auto isVeryPrevalentResourceQuery = "SELECT isVeryPrevalent FROM ObservedDomains WHERE registrableDomain = ?"_s;
+constexpr auto hadUserInteractionQuery = "SELECT hadUserInteraction, mostRecentUserInteractionTime FROM ObservedDomains WHERE registrableDomain = ?"_s;
 constexpr auto isGrandfatheredQuery = "SELECT grandfathered FROM ObservedDomains WHERE registrableDomain = ?"_s;
 constexpr auto findExpiredUserInteractionQuery = "SELECT domainID FROM ObservedDomains WHERE hadUserInteraction = 1 AND mostRecentUserInteractionTime < ?"_s;
 
+// CREATE TABLE Queries
 constexpr auto createObservedDomain = "CREATE TABLE ObservedDomains ("
     "domainID INTEGER PRIMARY KEY, registrableDomain TEXT NOT NULL UNIQUE ON CONFLICT FAIL, lastSeen REAL NOT NULL, "
     "hadUserInteraction INTEGER NOT NULL, mostRecentUserInteractionTime REAL NOT NULL, grandfathered INTEGER NOT NULL, "
@@ -176,6 +175,17 @@ constexpr auto createSubresourceUniqueRedirectsFrom = "CREATE TABLE SubresourceU
     "subresourceDomainID INTEGER NOT NULL, fromDomainID INTEGER NOT NULL, "
     "FOREIGN KEY(subresourceDomainID) REFERENCES ObservedDomains(domainID) ON DELETE CASCADE, "
     "FOREIGN KEY(fromDomainID) REFERENCES ObservedDomains(domainID) ON DELETE CASCADE);"_s;
+
+// CREATE UNIQUE INDEX Queries
+constexpr auto createUniqueIndexStorageAccessUnderTopFrameDomains = "CREATE UNIQUE INDEX IF NOT EXISTS StorageAccessUnderTopFrameDomains_domainID_topLevelDomainID on StorageAccessUnderTopFrameDomains ( domainID, topLevelDomainID );"_s;
+constexpr auto createUniqueIndexTopFrameUniqueRedirectsTo = "CREATE UNIQUE INDEX IF NOT EXISTS TopFrameUniqueRedirectsTo_sourceDomainID_toDomainID on TopFrameUniqueRedirectsTo ( sourceDomainID, toDomainID );"_s;
+constexpr auto createUniqueIndexTopFrameUniqueRedirectsFrom = "CREATE UNIQUE INDEX IF NOT EXISTS TopFrameUniqueRedirectsFrom_targetDomainID_fromDomainID on TopFrameUniqueRedirectsFrom ( targetDomainID, fromDomainID );"_s;
+constexpr auto createUniqueIndexTopFrameLinkDecorationsFrom = "CREATE UNIQUE INDEX IF NOT EXISTS TopFrameLinkDecorationsFrom_fromDomainID_toDomainID on TopFrameLinkDecorationsFrom ( fromDomainID, toDomainID );"_s;
+constexpr auto createUniqueIndexSubframeUnderTopFrameDomains = "CREATE UNIQUE INDEX IF NOT EXISTS SubframeUnderTopFrameDomains_subFrameDomainID_topFrameDomainID on SubframeUnderTopFrameDomains ( subFrameDomainID, topFrameDomainID );"_s;
+constexpr auto createUniqueIndexSubresourceUnderTopFrameDomains = "CREATE UNIQUE INDEX IF NOT EXISTS SubresourceUnderTopFrameDomains_subresourceDomainID_topFrameDomainID on SubresourceUnderTopFrameDomains ( subresourceDomainID, topFrameDomainID );"_s;
+constexpr auto createUniqueIndexSubresourceUniqueRedirectsTo = "CREATE UNIQUE INDEX IF NOT EXISTS SubresourceUniqueRedirectsTo_subresourceDomainID_toDomainID on SubresourceUniqueRedirectsTo ( subresourceDomainID, toDomainID );"_s;
+constexpr auto createUniqueIndexSubresourceUniqueRedirectsFrom = "CREATE UNIQUE INDEX IF NOT EXISTS SubresourceUniqueRedirectsFrom_subresourceDomainID_fromDomainID on SubresourceUnderTopFrameDomains ( subresourceDomainID, fromDomainID );"_s;
+
     
 ResourceLoadStatisticsDatabaseStore::ResourceLoadStatisticsDatabaseStore(WebResourceLoadStatisticsStore& store, WorkQueue& workQueue, ShouldIncludeLocalhost shouldIncludeLocalhost, const String& storageDirectoryPath, PAL::SessionID sessionID)
     : ResourceLoadStatisticsStore(store, workQueue, shouldIncludeLocalhost)
@@ -184,22 +194,10 @@ ResourceLoadStatisticsDatabaseStore::ResourceLoadStatisticsDatabaseStore(WebReso
     , m_insertObservedDomainStatement(m_database, insertObservedDomainQuery)
     , m_insertTopLevelDomainStatement(m_database, insertTopLevelDomainQuery)
     , m_domainIDFromStringStatement(m_database, domainIDFromStringQuery)
-    , m_storageAccessUnderTopFrameDomainsStatement(m_database, storageAccessUnderTopFrameDomainsQuery)
-    , m_storageAccessUnderTopFrameDomainsExistsStatement(m_database, storageAccessUnderTopFrameDomainsExistsQuery)
-    , m_topFrameUniqueRedirectsTo(m_database, topFrameUniqueRedirectsToQuery)
-    , m_topFrameUniqueRedirectsToExists(m_database, topFrameUniqueRedirectsToExistsQuery)
-    , m_topFrameUniqueRedirectsFrom(m_database, topFrameUniqueRedirectsFromQuery)
-    , m_topFrameUniqueRedirectsFromExists(m_database, topFrameUniqueRedirectsFromExistsQuery)
-    , m_topFrameLinkDecorationsFrom(m_database, topFrameLinkDecorationsFromQuery)
     , m_topFrameLinkDecorationsFromExists(m_database, topFrameLinkDecorationsFromExistsQuery)
-    , m_subframeUnderTopFrameDomains(m_database, subframeUnderTopFrameDomainsQuery)
     , m_subframeUnderTopFrameDomainExists(m_database, subframeUnderTopFrameDomainExistsQuery)
-    , m_subresourceUnderTopFrameDomains(m_database, subresourceUnderTopFrameDomainsQuery)
     , m_subresourceUnderTopFrameDomainExists(m_database, subresourceUnderTopFrameDomainExistsQuery)
-    , m_subresourceUniqueRedirectsTo(m_database, subresourceUniqueRedirectsToQuery)
     , m_subresourceUniqueRedirectsToExists(m_database, subresourceUniqueRedirectsToExistsQuery)
-    , m_subresourceUniqueRedirectsFrom(m_database, subresourceUniqueRedirectsFromQuery)
-    , m_subresourceUniqueRedirectsFromExists(m_database, subresourceUniqueRedirectsFromExistsQuery)
     , m_mostRecentUserInteractionStatement(m_database, mostRecentUserInteractionQuery)
     , m_updateLastSeenStatement(m_database, updateLastSeenQuery)
     , m_updateDataRecordsRemovedStatement(m_database, updateDataRecordsRemovedQuery)
@@ -262,6 +260,22 @@ bool ResourceLoadStatisticsDatabaseStore::isEmpty() const
     return result;
 }
 
+bool ResourceLoadStatisticsDatabaseStore::createUniqueIndices()
+{
+    if (!m_database.executeCommand(createUniqueIndexStorageAccessUnderTopFrameDomains)
+        || !m_database.executeCommand(createUniqueIndexTopFrameUniqueRedirectsTo)
+        || !m_database.executeCommand(createUniqueIndexTopFrameUniqueRedirectsFrom)
+        || !m_database.executeCommand(createUniqueIndexTopFrameLinkDecorationsFrom)
+        || !m_database.executeCommand(createUniqueIndexSubframeUnderTopFrameDomains)
+        || !m_database.executeCommand(createUniqueIndexSubresourceUnderTopFrameDomains)
+        || !m_database.executeCommand(createUniqueIndexSubresourceUniqueRedirectsTo)
+        || !m_database.executeCommand(createUniqueIndexSubresourceUnderTopFrameDomains)) {
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::createUniqueIndices failed to execute, error message: %{public}s", this, m_database.lastErrorMsg());
+        return false;
+    }
+    return true;
+}
+
 bool ResourceLoadStatisticsDatabaseStore::createSchema()
 {
     ASSERT(!RunLoop::isMain());
@@ -315,6 +329,9 @@ bool ResourceLoadStatisticsDatabaseStore::createSchema()
         LOG_ERROR("Could not create SubresourceUniqueRedirectsFrom table in database (%i) - %s", m_database.lastError(), m_database.lastErrorMsg());
         return false;
     }
+    
+    if (!createUniqueIndices())
+        return false;
 
     return true;
 }
@@ -327,20 +344,9 @@ bool ResourceLoadStatisticsDatabaseStore::prepareStatements()
         || m_insertObservedDomainStatement.prepare() != SQLITE_OK
         || m_insertTopLevelDomainStatement.prepare() != SQLITE_OK
         || m_domainIDFromStringStatement.prepare() != SQLITE_OK
-        || m_storageAccessUnderTopFrameDomainsStatement.prepare() != SQLITE_OK
-        || m_storageAccessUnderTopFrameDomainsExistsStatement.prepare() != SQLITE_OK
-        || m_topFrameUniqueRedirectsTo.prepare() != SQLITE_OK
-        || m_topFrameUniqueRedirectsToExists.prepare() != SQLITE_OK
-        || m_topFrameUniqueRedirectsFrom.prepare() != SQLITE_OK
-        || m_topFrameUniqueRedirectsFromExists.prepare() != SQLITE_OK
-        || m_subframeUnderTopFrameDomains.prepare() != SQLITE_OK
         || m_subframeUnderTopFrameDomainExists.prepare() != SQLITE_OK
-        || m_subresourceUnderTopFrameDomains.prepare() != SQLITE_OK
         || m_subresourceUnderTopFrameDomainExists.prepare() != SQLITE_OK
-        || m_subresourceUniqueRedirectsTo.prepare() != SQLITE_OK
         || m_subresourceUniqueRedirectsToExists.prepare() != SQLITE_OK
-        || m_subresourceUniqueRedirectsFrom.prepare() != SQLITE_OK
-        || m_subresourceUniqueRedirectsFromExists.prepare() != SQLITE_OK
         || m_updateLastSeenStatement.prepare() != SQLITE_OK
         || m_updateDataRecordsRemovedStatement.prepare() != SQLITE_OK
         || m_mostRecentUserInteractionStatement.prepare() != SQLITE_OK
@@ -353,7 +359,6 @@ bool ResourceLoadStatisticsDatabaseStore::prepareStatements()
         || m_updateGrandfatheredStatement.prepare() != SQLITE_OK
         || m_isGrandfatheredStatement.prepare() != SQLITE_OK
         || m_findExpiredUserInteractionStatement.prepare() != SQLITE_OK
-        || m_topFrameLinkDecorationsFrom.prepare() != SQLITE_OK
         || m_topFrameLinkDecorationsFromExists.prepare() != SQLITE_OK) {
         RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::prepareStatements failed to prepare, error message: %{public}s", this, m_database.lastErrorMsg());
         ASSERT_NOT_REACHED();
@@ -465,6 +470,37 @@ Optional<unsigned> ResourceLoadStatisticsDatabaseStore::domainID(const Registrab
     return domainID;
 }
 
+String ResourceLoadStatisticsDatabaseStore::ensureAndMakeDomainList(const HashSet<RegistrableDomain>& subframeUnderTopFrameDomains)
+{
+    StringBuilder builder;
+    
+    for (auto& topFrameResource : subframeUnderTopFrameDomains) {
+        
+        // Insert query will fail if top frame domain is not already in the database
+        ensureResourceStatisticsForRegistrableDomain(topFrameResource);
+        
+        if (!builder.isEmpty())
+            builder.appendLiteral(", ");
+        builder.append('"');
+        builder.append(topFrameResource.string());
+        builder.append('"');
+    }
+
+    return builder.toString();
+}
+
+void ResourceLoadStatisticsDatabaseStore::insertDomainRelationshipList(const String& statement, const HashSet<RegistrableDomain>& subframeUnderTopFrameDomains, unsigned domainID)
+{
+    SQLiteStatement insertRelationshipStatement(m_database, makeString(statement, ensureAndMakeDomainList(subframeUnderTopFrameDomains), " );"));
+    
+    if (insertRelationshipStatement.prepare() != SQLITE_OK
+        || insertRelationshipStatement.bindInt(1, domainID) != SQLITE_OK
+        || insertRelationshipStatement.step() != SQLITE_DONE) {
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::insertDomainRelationshipList failed, error message: %{private}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+    }
+}
+
 void ResourceLoadStatisticsDatabaseStore::insertDomainRelationships(const ResourceLoadStatistics& loadStatistics)
 {
     ASSERT(!RunLoop::isMain());
@@ -472,41 +508,14 @@ void ResourceLoadStatisticsDatabaseStore::insertDomainRelationships(const Resour
     
     if (!registrableDomainID)
         return;
-    
-    for (auto& topFrameDomain : loadStatistics.storageAccessUnderTopFrameDomains) {
-        if (!relationshipExists(m_storageAccessUnderTopFrameDomainsExistsStatement, registrableDomainID.value(), topFrameDomain))
-            insertDomainRelationship(m_storageAccessUnderTopFrameDomainsStatement, registrableDomainID.value(), topFrameDomain);
-    }
 
-    for (auto& toDomain : loadStatistics.topFrameUniqueRedirectsTo) {
-        if (!relationshipExists(m_topFrameUniqueRedirectsToExists, registrableDomainID.value(), toDomain))
-            insertDomainRelationship(m_topFrameUniqueRedirectsTo, registrableDomainID.value(), toDomain);
-    }
-    
-    for (auto& fromDomain : loadStatistics.topFrameUniqueRedirectsFrom) {
-        if (!relationshipExists(m_topFrameUniqueRedirectsFromExists, registrableDomainID.value(), fromDomain))
-            insertDomainRelationship(m_topFrameUniqueRedirectsFrom, registrableDomainID.value(), fromDomain);
-    }
-    
-    for (auto& topFrameDomain : loadStatistics.subframeUnderTopFrameDomains) {
-        if (!relationshipExists(m_subframeUnderTopFrameDomainExists, registrableDomainID.value(), topFrameDomain))
-            insertDomainRelationship(m_subframeUnderTopFrameDomains, registrableDomainID.value(), topFrameDomain);
-    }
-    
-    for (auto& topFrameDomain : loadStatistics.subresourceUnderTopFrameDomains) {
-        if (!relationshipExists(m_subresourceUnderTopFrameDomainExists, registrableDomainID.value(), topFrameDomain))
-            insertDomainRelationship(m_subresourceUnderTopFrameDomains, registrableDomainID.value(), topFrameDomain);
-    }
-    
-    for (auto& toDomain : loadStatistics.subresourceUniqueRedirectsTo) {
-        if (!relationshipExists(m_subresourceUniqueRedirectsToExists, registrableDomainID.value(), toDomain))
-            insertDomainRelationship(m_subresourceUniqueRedirectsTo, registrableDomainID.value(), toDomain);
-    }
-    
-    for (auto& fromDomain : loadStatistics.subresourceUniqueRedirectsFrom) {
-        if (!relationshipExists(m_subresourceUniqueRedirectsFromExists, registrableDomainID.value(), fromDomain))
-            insertDomainRelationship(m_subresourceUniqueRedirectsFrom, registrableDomainID.value(), fromDomain);
-    }
+    insertDomainRelationshipList(storageAccessUnderTopFrameDomainsQuery, loadStatistics.storageAccessUnderTopFrameDomains, registrableDomainID.value());
+    insertDomainRelationshipList(topFrameUniqueRedirectsToQuery, loadStatistics.topFrameUniqueRedirectsTo, registrableDomainID.value());
+    insertDomainRelationshipList(topFrameUniqueRedirectsFromQuery, loadStatistics.topFrameUniqueRedirectsFrom, registrableDomainID.value());
+    insertDomainRelationshipList(subframeUnderTopFrameDomainsQuery, loadStatistics.subframeUnderTopFrameDomains, registrableDomainID.value());
+    insertDomainRelationshipList(subresourceUnderTopFrameDomainsQuery, loadStatistics.subresourceUnderTopFrameDomains, registrableDomainID.value());
+    insertDomainRelationshipList(subresourceUniqueRedirectsToQuery, loadStatistics.subresourceUniqueRedirectsTo, registrableDomainID.value());
+    insertDomainRelationshipList(subresourceUniqueRedirectsFromQuery, loadStatistics.subresourceUniqueRedirectsFrom, registrableDomainID.value());
 }
 
 void ResourceLoadStatisticsDatabaseStore::populateFromMemoryStore(const ResourceLoadStatisticsMemoryStore& memoryStore)
@@ -917,7 +926,7 @@ void ResourceLoadStatisticsDatabaseStore::grantStorageAccess(SubFrameDomain&& su
         auto subFrameStatus = ensureResourceStatisticsForRegistrableDomain(subFrameDomain);
         ASSERT(subFrameStatus.first == AddedRecord::No);
         ASSERT(hasHadUserInteraction(subFrameDomain, OperatingDatesWindow::Long));
-        insertDomainRelationship(m_storageAccessUnderTopFrameDomainsStatement, subFrameStatus.second, topFrameDomain);
+        insertDomainRelationshipList(storageAccessUnderTopFrameDomainsQuery, HashSet<RegistrableDomain>({ topFrameDomain }), subFrameStatus.second);
     }
 
     grantStorageAccessInternal(WTFMove(subFrameDomain), WTFMove(topFrameDomain), frameID, pageID, promptWasShown, WTFMove(completionHandler));
@@ -1002,40 +1011,23 @@ void ResourceLoadStatisticsDatabaseStore::logFrameNavigation(const RegistrableDo
     if (!isMainFrame && !(areTargetAndTopFrameDomainsSameSite || areTargetAndSourceDomainsSameSite)) {
         auto targetResult = ensureResourceStatisticsForRegistrableDomain(targetDomain);
         updateLastSeen(targetDomain, ResourceLoadStatistics::reduceTimeResolution(WallTime::now()));
-        ensureResourceStatisticsForRegistrableDomain(topFrameDomain);
-        if (!relationshipExists(m_subframeUnderTopFrameDomainExists, targetResult.second, topFrameDomain)) {
-            insertDomainRelationship(m_subframeUnderTopFrameDomains, targetResult.second, topFrameDomain);
-            statisticsWereUpdated = true;
-        }
+        insertDomainRelationshipList(subframeUnderTopFrameDomainsQuery, HashSet<RegistrableDomain>({ topFrameDomain }), targetResult.second);
+        statisticsWereUpdated = true;
     }
 
     if (isRedirect && !areTargetAndSourceDomainsSameSite) {
         if (isMainFrame) {
             auto redirectingDomainResult = ensureResourceStatisticsForRegistrableDomain(sourceDomain);
             auto targetResult = ensureResourceStatisticsForRegistrableDomain(targetDomain);
-
-            if (!relationshipExists(m_topFrameUniqueRedirectsToExists, redirectingDomainResult.second, targetDomain)) {
-                insertDomainRelationship(m_topFrameUniqueRedirectsTo, redirectingDomainResult.second, targetDomain);
-                statisticsWereUpdated = true;
-            }
-
-            if (!relationshipExists(m_topFrameUniqueRedirectsFromExists, targetResult.second, sourceDomain)) {
-                insertDomainRelationship(m_topFrameUniqueRedirectsFrom, targetResult.second, sourceDomain);
-                statisticsWereUpdated = true;
-            }
+            insertDomainRelationshipList(topFrameUniqueRedirectsToQuery, HashSet<RegistrableDomain>({ targetDomain }), redirectingDomainResult.second);
+            insertDomainRelationshipList(topFrameUniqueRedirectsFromQuery, HashSet<RegistrableDomain>({ sourceDomain }), targetResult.second);
         } else {
             auto redirectingDomainResult = ensureResourceStatisticsForRegistrableDomain(sourceDomain);
             auto targetResult = ensureResourceStatisticsForRegistrableDomain(targetDomain);
-            if (!relationshipExists(m_subresourceUniqueRedirectsToExists, redirectingDomainResult.second, targetDomain)) {
-                insertDomainRelationship(m_subresourceUniqueRedirectsTo, redirectingDomainResult.second, targetDomain);
-                statisticsWereUpdated = true;
-            }
-
-            if (!relationshipExists(m_subresourceUniqueRedirectsFromExists, targetResult.second, sourceDomain)) {
-                insertDomainRelationship(m_subresourceUniqueRedirectsFrom, targetResult.second, sourceDomain);
-                statisticsWereUpdated = true;
-            }
+            insertDomainRelationshipList(subresourceUniqueRedirectsToQuery, HashSet<RegistrableDomain>({ targetDomain }), redirectingDomainResult.second);
+            insertDomainRelationshipList(subresourceUniqueRedirectsFromQuery, HashSet<RegistrableDomain>({ sourceDomain }), targetResult.second);
         }
+        statisticsWereUpdated = true;
     }
 
     if (statisticsWereUpdated)
@@ -1048,11 +1040,8 @@ void ResourceLoadStatisticsDatabaseStore::logCrossSiteLoadWithLinkDecoration(con
     ASSERT(fromDomain != toDomain);
 
     auto fromDomainResult = ensureResourceStatisticsForRegistrableDomain(fromDomain);
-
-    if (!relationshipExists(m_topFrameLinkDecorationsFromExists, fromDomainResult.second, toDomain)) {
-        insertDomainRelationship(m_topFrameLinkDecorationsFrom, fromDomainResult.second, toDomain);
-        scheduleStatisticsProcessingRequestIfNecessary();
-    }
+    insertDomainRelationshipList(topFrameLinkDecorationsFromQuery, HashSet<RegistrableDomain>({ toDomain }), fromDomainResult.second);
+    scheduleStatisticsProcessingRequestIfNecessary();
 }
 
 void ResourceLoadStatisticsDatabaseStore::setUserInteraction(const RegistrableDomain& domain, bool hadUserInteraction, WallTime mostRecentInteraction)
@@ -1299,9 +1288,7 @@ void ResourceLoadStatisticsDatabaseStore::setSubframeUnderTopFrameDomain(const S
     auto result = ensureResourceStatisticsForRegistrableDomain(subFrameDomain);
 
     // For consistency, make sure we also have a statistics entry for the top frame domain.
-    ensureResourceStatisticsForRegistrableDomain(topFrameDomain);
-
-    insertDomainRelationship(m_subframeUnderTopFrameDomains, result.second, topFrameDomain);
+    insertDomainRelationshipList(subframeUnderTopFrameDomainsQuery, HashSet<RegistrableDomain>({ topFrameDomain }), result.second);
 }
 
 void ResourceLoadStatisticsDatabaseStore::setSubresourceUnderTopFrameDomain(const SubResourceDomain& subresourceDomain, const TopFrameDomain& topFrameDomain)
@@ -1311,9 +1298,7 @@ void ResourceLoadStatisticsDatabaseStore::setSubresourceUnderTopFrameDomain(cons
     auto result = ensureResourceStatisticsForRegistrableDomain(subresourceDomain);
 
     // For consistency, make sure we also have a statistics entry for the top frame domain.
-    ensureResourceStatisticsForRegistrableDomain(topFrameDomain);
-
-    insertDomainRelationship(m_subresourceUnderTopFrameDomains, result.second, topFrameDomain);
+    insertDomainRelationshipList(subresourceUnderTopFrameDomainsQuery, HashSet<RegistrableDomain>({ topFrameDomain }), result.second);
 }
 
 void ResourceLoadStatisticsDatabaseStore::setSubresourceUniqueRedirectTo(const SubResourceDomain& subresourceDomain, const RedirectDomain& redirectDomain)
@@ -1323,9 +1308,7 @@ void ResourceLoadStatisticsDatabaseStore::setSubresourceUniqueRedirectTo(const S
     auto result = ensureResourceStatisticsForRegistrableDomain(subresourceDomain);
 
     // For consistency, make sure we also have a statistics entry for the redirect domain.
-    ensureResourceStatisticsForRegistrableDomain(redirectDomain);
-
-    insertDomainRelationship(m_subresourceUniqueRedirectsTo, result.second, redirectDomain);
+    insertDomainRelationshipList(subresourceUniqueRedirectsToQuery, HashSet<RegistrableDomain>({ redirectDomain }), result.second);
 }
 
 void ResourceLoadStatisticsDatabaseStore::setSubresourceUniqueRedirectFrom(const SubResourceDomain& subresourceDomain, const RedirectDomain& redirectDomain)
@@ -1335,9 +1318,7 @@ void ResourceLoadStatisticsDatabaseStore::setSubresourceUniqueRedirectFrom(const
     auto result = ensureResourceStatisticsForRegistrableDomain(subresourceDomain);
 
     // For consistency, make sure we also have a statistics entry for the redirect domain.
-    ensureResourceStatisticsForRegistrableDomain(redirectDomain);
-
-    insertDomainRelationship(m_subresourceUniqueRedirectsFrom, result.second, redirectDomain);
+    insertDomainRelationshipList(subresourceUniqueRedirectsFromQuery, HashSet<RegistrableDomain>({ redirectDomain }), result.second);
 }
 
 void ResourceLoadStatisticsDatabaseStore::setTopFrameUniqueRedirectTo(const TopFrameDomain& topFrameDomain, const RedirectDomain& redirectDomain)
@@ -1347,9 +1328,7 @@ void ResourceLoadStatisticsDatabaseStore::setTopFrameUniqueRedirectTo(const TopF
     auto result = ensureResourceStatisticsForRegistrableDomain(topFrameDomain);
 
     // For consistency, make sure we also have a statistics entry for the redirect domain.
-    ensureResourceStatisticsForRegistrableDomain(redirectDomain);
-
-    insertDomainRelationship(m_topFrameUniqueRedirectsTo, result.second, redirectDomain);
+    insertDomainRelationshipList(topFrameUniqueRedirectsToQuery, HashSet<RegistrableDomain>({ redirectDomain }), result.second);
 }
 
 void ResourceLoadStatisticsDatabaseStore::setTopFrameUniqueRedirectFrom(const TopFrameDomain& topFrameDomain, const RedirectDomain& redirectDomain)
@@ -1359,9 +1338,7 @@ void ResourceLoadStatisticsDatabaseStore::setTopFrameUniqueRedirectFrom(const To
     auto result = ensureResourceStatisticsForRegistrableDomain(topFrameDomain);
 
     // For consistency, make sure we also have a statistics entry for the redirect domain.
-    ensureResourceStatisticsForRegistrableDomain(redirectDomain);
-
-    insertDomainRelationship(m_topFrameUniqueRedirectsFrom, result.second, redirectDomain);
+    insertDomainRelationshipList(topFrameUniqueRedirectsFromQuery, HashSet<RegistrableDomain>({ redirectDomain }), result.second);
 }
 
 std::pair<ResourceLoadStatisticsDatabaseStore::AddedRecord, unsigned> ResourceLoadStatisticsDatabaseStore::ensureResourceStatisticsForRegistrableDomain(const RegistrableDomain& domain)
@@ -1758,6 +1735,40 @@ void ResourceLoadStatisticsDatabaseStore::updateDataRecordsRemoved(const Registr
     ASSERT_UNUSED(resetResult, resetResult == SQLITE_OK);
 }
 
+bool ResourceLoadStatisticsDatabaseStore::isCorrectSubStatisticsCount(const RegistrableDomain& subframeDomain, const TopFrameDomain& topFrameDomain)
+{
+    SQLiteStatement subFrameUnderTopFrameCount(m_database, countSubframeUnderTopFrameQuery);
+    SQLiteStatement subresourceUnderTopFrameCount(m_database, countSubresourceUnderTopFrameQuery);
+    SQLiteStatement subresourceUniqueRedirectsTo(m_database, countSubresourceUniqueRedirectsToQuery);
+    
+    if (subFrameUnderTopFrameCount.prepare() != SQLITE_OK
+        || subresourceUnderTopFrameCount.prepare() != SQLITE_OK
+        || subresourceUniqueRedirectsTo.prepare() != SQLITE_OK) {
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::countSubStatisticsTesting failed to prepare, error message: %{private}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+        return false;
+    }
+    
+    if (subFrameUnderTopFrameCount.bindInt(1, domainID(subframeDomain).value()) != SQLITE_OK
+        || subFrameUnderTopFrameCount.bindInt(2, domainID(topFrameDomain).value()) != SQLITE_OK
+        || subresourceUnderTopFrameCount.bindInt(1, domainID(subframeDomain).value()) != SQLITE_OK
+        || subresourceUnderTopFrameCount.bindInt(2, domainID(topFrameDomain).value()) != SQLITE_OK
+        || subresourceUniqueRedirectsTo.bindInt(1, domainID(subframeDomain).value()) != SQLITE_OK
+        || subresourceUniqueRedirectsTo.bindInt(2, domainID(topFrameDomain).value()) != SQLITE_OK) {
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::countSubStatisticsTesting failed to bind, error message: %{private}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+        return false;
+    }
+    
+    if (subFrameUnderTopFrameCount.step() != SQLITE_ROW
+        || subresourceUnderTopFrameCount.step() != SQLITE_ROW
+        || subresourceUniqueRedirectsTo.step() != SQLITE_ROW)
+        return false;
+    
+    return (subFrameUnderTopFrameCount.getColumnInt(0) == 1 && subresourceUnderTopFrameCount.getColumnInt(0) == 1 && subresourceUniqueRedirectsTo.getColumnInt(0) == 1);
+}
+
+
 } // namespace WebKit
 
 #endif
index ea7e649..e96716d 100644 (file)
@@ -103,6 +103,7 @@ public:
     bool hasHadUserInteraction(const RegistrableDomain&, OperatingDatesWindow) override;
 
     void setLastSeen(const RegistrableDomain&, Seconds) override;
+    bool isCorrectSubStatisticsCount(const RegistrableDomain&, const TopFrameDomain&);
 
 private:
     void mergeStatistic(const ResourceLoadStatistics&);
@@ -110,6 +111,7 @@ private:
     void clearDatabaseContents();
     bool insertObservedDomain(const ResourceLoadStatistics&);
     void insertDomainRelationships(const ResourceLoadStatistics&);
+    void insertDomainRelationshipList(const String&, const HashSet<RegistrableDomain>&, unsigned);
     bool insertDomainRelationship(WebCore::SQLiteStatement&, unsigned domainID, const RegistrableDomain& topFrameDomain);
     bool relationshipExists(WebCore::SQLiteStatement&, Optional<unsigned> firstDomainID, const RegistrableDomain& secondDomain) const;
     Optional<unsigned> domainID(const RegistrableDomain&) const;
@@ -166,8 +168,11 @@ private:
     Vector<std::pair<RegistrableDomain, WebsiteDataToRemove>> registrableDomainsToRemoveWebsiteDataFor() override;
     bool isDatabaseStore() const final { return true; }
 
+    bool createUniqueIndices();
     bool createSchema();
     bool prepareStatements();
+    String ensureAndMakeDomainList(const HashSet<RegistrableDomain>&);
+
     
     const String m_storageDirectoryPath;
     mutable WebCore::SQLiteDatabase m_database;
@@ -175,22 +180,10 @@ private:
     WebCore::SQLiteStatement m_insertObservedDomainStatement;
     WebCore::SQLiteStatement m_insertTopLevelDomainStatement;
     mutable WebCore::SQLiteStatement m_domainIDFromStringStatement;
-    WebCore::SQLiteStatement m_storageAccessUnderTopFrameDomainsStatement;
-    WebCore::SQLiteStatement m_storageAccessUnderTopFrameDomainsExistsStatement;
-    WebCore::SQLiteStatement m_topFrameUniqueRedirectsTo;
-    mutable WebCore::SQLiteStatement m_topFrameUniqueRedirectsToExists;
-    WebCore::SQLiteStatement m_topFrameUniqueRedirectsFrom;
-    mutable WebCore::SQLiteStatement m_topFrameUniqueRedirectsFromExists;
-    WebCore::SQLiteStatement m_topFrameLinkDecorationsFrom;
     mutable WebCore::SQLiteStatement m_topFrameLinkDecorationsFromExists;
-    WebCore::SQLiteStatement m_subframeUnderTopFrameDomains;
     mutable WebCore::SQLiteStatement m_subframeUnderTopFrameDomainExists;
-    WebCore::SQLiteStatement m_subresourceUnderTopFrameDomains;
     mutable WebCore::SQLiteStatement m_subresourceUnderTopFrameDomainExists;
-    WebCore::SQLiteStatement m_subresourceUniqueRedirectsTo;
     mutable WebCore::SQLiteStatement m_subresourceUniqueRedirectsToExists;
-    WebCore::SQLiteStatement m_subresourceUniqueRedirectsFrom;
-    mutable WebCore::SQLiteStatement m_subresourceUniqueRedirectsFromExists;
     WebCore::SQLiteStatement m_mostRecentUserInteractionStatement;
     WebCore::SQLiteStatement m_updateLastSeenStatement;
     mutable WebCore::SQLiteStatement m_updateDataRecordsRemovedStatement;
index 113d353..ea00bc1 100644 (file)
@@ -596,11 +596,11 @@ void WebResourceLoadStatisticsStore::setLastSeen(const RegistrableDomain& domain
     });
 }
 
-void WebResourceLoadStatisticsStore::mergeStatisticForTesting(const RegistrableDomain& domain, const RegistrableDomain& topFrameDomain, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&& completionHandler)
+void WebResourceLoadStatisticsStore::mergeStatisticForTesting(const RegistrableDomain& domain, const RegistrableDomain& topFrameDomain1, const RegistrableDomain& topFrameDomain2, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&& completionHandler)
 {
     ASSERT(RunLoop::isMain());
 
-    postTask([this, domain = domain.isolatedCopy(), topFrameDomain = topFrameDomain.isolatedCopy(), lastSeen, hadUserInteraction, mostRecentUserInteraction, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, completionHandler = WTFMove(completionHandler)]() mutable {
+    postTask([this, domain = domain.isolatedCopy(), topFrameDomain1 = topFrameDomain1.isolatedCopy(), topFrameDomain2 = topFrameDomain2.isolatedCopy(), lastSeen, hadUserInteraction, mostRecentUserInteraction, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, completionHandler = WTFMove(completionHandler)]() mutable {
         if (m_statisticsStore) {
             ResourceLoadStatistics statistic(domain);
             statistic.lastSeen = WallTime::fromRawSeconds(lastSeen.seconds());
@@ -611,11 +611,15 @@ void WebResourceLoadStatisticsStore::mergeStatisticForTesting(const RegistrableD
             statistic.isVeryPrevalentResource = isVeryPrevalent;
             statistic.dataRecordsRemoved = dataRecordsRemoved;
             
-            if (!topFrameDomain.isEmpty()) {
-                HashSet<RegistrableDomain> topFrameDomains;
-                topFrameDomains.add(topFrameDomain);
-                statistic.subframeUnderTopFrameDomains = WTFMove(topFrameDomains);
-            }
+            HashSet<RegistrableDomain> topFrameDomains;
+            
+            if (!topFrameDomain1.isEmpty())
+                topFrameDomains.add(topFrameDomain1);
+            
+            if (!topFrameDomain2.isEmpty())
+                topFrameDomains.add(topFrameDomain2);
+
+            statistic.subframeUnderTopFrameDomains = WTFMove(topFrameDomains);
 
             Vector<ResourceLoadStatistics> statistics;
             statistics.append(WTFMove(statistic));
@@ -624,6 +628,24 @@ void WebResourceLoadStatisticsStore::mergeStatisticForTesting(const RegistrableD
         postTaskReply(WTFMove(completionHandler));
     });
 }
+
+void WebResourceLoadStatisticsStore::isRelationshipOnlyInDatabaseOnce(const RegistrableDomain& subDomain, const RegistrableDomain& topDomain, CompletionHandler<void(bool)>&& completionHandler)
+{
+    ASSERT(RunLoop::isMain());
+
+    postTask([this, subDomain = subDomain.isolatedCopy(), topDomain = topDomain.isolatedCopy(), completionHandler = WTFMove(completionHandler)]() mutable {
+        if (!m_statisticsStore || !is<ResourceLoadStatisticsDatabaseStore>(*m_statisticsStore)) {
+            completionHandler(false);
+            return;
+        }
+        
+        bool isRelationshipOnlyInDatabaseOnce = downcast<ResourceLoadStatisticsDatabaseStore>(*m_statisticsStore).isCorrectSubStatisticsCount(subDomain, topDomain);
+        
+        postTaskReply([isRelationshipOnlyInDatabaseOnce, completionHandler = WTFMove(completionHandler)]() mutable {
+            completionHandler(isRelationshipOnlyInDatabaseOnce);
+        });
+    });
+}
     
 void WebResourceLoadStatisticsStore::setPrevalentResource(const RegistrableDomain& domain, CompletionHandler<void()>&& completionHandler)
 {
index fb18105..92fe157 100644 (file)
@@ -129,7 +129,8 @@ public:
     bool hasStorageAccessForFrame(const SubFrameDomain&, const TopFrameDomain&, WebCore::FrameIdentifier, WebCore::PageIdentifier);
     void requestStorageAccess(const SubFrameDomain&, const TopFrameDomain&, WebCore::FrameIdentifier, WebCore::PageIdentifier, WebPageProxyIdentifier, CompletionHandler<void(StorageAccessWasGranted, StorageAccessPromptWasShown)>&&);
     void setLastSeen(const RegistrableDomain&, Seconds, CompletionHandler<void()>&&);
-    void mergeStatisticForTesting(const RegistrableDomain&, const TopFrameDomain&, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&&);
+    void mergeStatisticForTesting(const RegistrableDomain&, const TopFrameDomain& topFrameDomain1, const TopFrameDomain& topFrameDomain2, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&&);
+    void isRelationshipOnlyInDatabaseOnce(const RegistrableDomain& subDomain, const RegistrableDomain& topDomain, CompletionHandler<void(bool)>&&);
     void setPrevalentResource(const RegistrableDomain&, CompletionHandler<void()>&&);
     void setVeryPrevalentResource(const RegistrableDomain&, CompletionHandler<void()>&&);
     void dumpResourceLoadStatistics(CompletionHandler<void(String)>&&);
index 3341aed..b580a07 100644 (file)
@@ -968,11 +968,11 @@ void NetworkProcess::setLastSeen(PAL::SessionID sessionID, const RegistrableDoma
     }
 }
 
-void NetworkProcess::mergeStatisticForTesting(PAL::SessionID sessionID, const RegistrableDomain& domain, const RegistrableDomain& topFrameDomain, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&& completionHandler)
+void NetworkProcess::mergeStatisticForTesting(PAL::SessionID sessionID, const RegistrableDomain& domain, const RegistrableDomain& topFrameDomain1, const RegistrableDomain& topFrameDomain2, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&& completionHandler)
 {
     if (auto* networkSession = this->networkSession(sessionID)) {
         if (auto* resourceLoadStatistics = networkSession->resourceLoadStatistics())
-            resourceLoadStatistics->mergeStatisticForTesting(domain, topFrameDomain, lastSeen, hadUserInteraction, mostRecentUserInteraction, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, WTFMove(completionHandler));
+            resourceLoadStatistics->mergeStatisticForTesting(domain, topFrameDomain1, topFrameDomain2, lastSeen, hadUserInteraction, mostRecentUserInteraction, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, WTFMove(completionHandler));
         else
             completionHandler();
     } else {
@@ -1026,6 +1026,19 @@ void NetworkProcess::hadUserInteraction(PAL::SessionID sessionID, const Registra
     }
 }
 
+void NetworkProcess::isRelationshipOnlyInDatabaseOnce(PAL::SessionID sessionID, const RegistrableDomain& subDomain, const RegistrableDomain& topDomain, CompletionHandler<void(bool)>&& completionHandler)
+{
+    if (auto* networkSession = this->networkSession(sessionID)) {
+        if (auto* resourceLoadStatistics = networkSession->resourceLoadStatistics())
+            resourceLoadStatistics->isRelationshipOnlyInDatabaseOnce(subDomain, topDomain, WTFMove(completionHandler));
+        else
+            completionHandler(false);
+    } else {
+        ASSERT_NOT_REACHED();
+        completionHandler(false);
+    }
+}
+
 void NetworkProcess::clearUserInteraction(PAL::SessionID sessionID, const RegistrableDomain& domain, CompletionHandler<void()>&& completionHandler)
 {
     if (auto* networkSession = this->networkSession(sessionID)) {
index 2f70430..1daea57 100644 (file)
@@ -228,6 +228,7 @@ public:
     void setVeryPrevalentResource(PAL::SessionID, const RegistrableDomain&, CompletionHandler<void()>&&);
     void setPruneEntriesDownTo(PAL::SessionID, uint64_t pruneTargetCount, CompletionHandler<void()>&&);
     void hadUserInteraction(PAL::SessionID, const RegistrableDomain&, CompletionHandler<void(bool)>&&);
+    void isRelationshipOnlyInDatabaseOnce(PAL::SessionID, const RegistrableDomain& subDomain, const RegistrableDomain& topDomain, CompletionHandler<void(bool)>&&);
     void hasLocalStorage(PAL::SessionID, const RegistrableDomain&, CompletionHandler<void(bool)>&&);
     void getAllStorageAccessEntries(PAL::SessionID, CompletionHandler<void(Vector<String> domains)>&&);
     void logFrameNavigation(PAL::SessionID, const NavigatedToDomain&, const TopFrameDomain&, const NavigatedFromDomain&, bool isRedirect, bool isMainFrame);
@@ -242,7 +243,7 @@ public:
     void setCacheMaxAgeCapForPrevalentResources(PAL::SessionID, Seconds, CompletionHandler<void()>&&);
     void setGrandfatheringTime(PAL::SessionID, Seconds, CompletionHandler<void()>&&);
     void setLastSeen(PAL::SessionID, const RegistrableDomain&, Seconds, CompletionHandler<void()>&&);
-    void mergeStatisticForTesting(PAL::SessionID, const RegistrableDomain&, const TopFrameDomain&, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&&);
+    void mergeStatisticForTesting(PAL::SessionID, const RegistrableDomain&, const TopFrameDomain& topFrameDomain1, const TopFrameDomain& topFrameDomain2, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&&);
     void setMinimumTimeBetweenDataRecordsRemoval(PAL::SessionID, Seconds, CompletionHandler<void()>&&);
     void setNotifyPagesWhenDataRecordsWereScanned(PAL::SessionID, bool value, CompletionHandler<void()>&&);
     void setIsRunningResourceLoadStatisticsTest(PAL::SessionID, bool value, CompletionHandler<void()>&&);
index 51371d6..bd3c24a 100644 (file)
@@ -94,10 +94,11 @@ messages -> NetworkProcess LegacyReceiver {
     IsVeryPrevalentResource(PAL::SessionID sessionID, WebCore::RegistrableDomain targetDomain) -> (bool isVeryPrevalent) Async
     SetAgeCapForClientSideCookies(PAL::SessionID sessionID, Optional<Seconds> seconds) -> () Async
     SetLastSeen(PAL::SessionID sessionID, WebCore::RegistrableDomain resourceDomain, Seconds seconds) -> () Async
-    MergeStatisticForTesting(PAL::SessionID sessionID, WebCore::RegistrableDomain resourceDomain, WebCore::RegistrableDomain topFrameDomain, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, uint64_t dataRecordsRemoved) -> () Async
+    MergeStatisticForTesting(PAL::SessionID sessionID, WebCore::RegistrableDomain resourceDomain, WebCore::RegistrableDomain topFrameDomain1, WebCore::RegistrableDomain topFrameDomain2, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, uint64_t dataRecordsRemoved) -> () Async
     SetPrevalentResource(PAL::SessionID sessionID, WebCore::RegistrableDomain resourceDomain) -> () Async
     SetPrevalentResourceForDebugMode(PAL::SessionID sessionID, WebCore::RegistrableDomain resourceDomain) -> () Async
     HadUserInteraction(PAL::SessionID sessionID, WebCore::RegistrableDomain resourceDomain) -> (bool hadUserInteraction) Async
+    IsRelationshipOnlyInDatabaseOnce(PAL::SessionID sessionID, WebCore::RegistrableDomain subDomain, WebCore::RegistrableDomain topDomain) -> (bool hadUserInteraction) Async
     HasLocalStorage(PAL::SessionID sessionID, WebCore::RegistrableDomain resourceDomain) -> (bool hadUserInteraction) Async
     GetAllStorageAccessEntries(PAL::SessionID sessionID) -> (Vector<String> domains) Async
     IsRegisteredAsRedirectingTo(PAL::SessionID sessionID, WebCore::RegistrableDomain redirectedFromDomain, WebCore::RegistrableDomain redirectedToDomain) -> (bool isRedirectingTo) Async
index 512e05c..9d7fa7d 100644 (file)
@@ -119,10 +119,10 @@ void WKWebsiteDataStoreSetStatisticsLastSeen(WKWebsiteDataStoreRef dataStoreRef,
 #endif
 }
 
-void WKWebsiteDataStoreSetStatisticsMergeStatistic(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, WKStringRef topFrameDomain, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, void* context, WKWebsiteDataStoreStatisticsMergeStatisticFunction completionHandler)
+void WKWebsiteDataStoreSetStatisticsMergeStatistic(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, WKStringRef topFrameDomain1, WKStringRef topFrameDomain2, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, void* context, WKWebsiteDataStoreStatisticsMergeStatisticFunction completionHandler)
 {
 #if ENABLE(RESOURCE_LOAD_STATISTICS)
-    WebKit::toImpl(dataStoreRef)->mergeStatisticForTesting(URL(URL(), WebKit::toImpl(host)->string()), URL(URL(), WebKit::toImpl(topFrameDomain)->string()), Seconds { lastSeen }, hadUserInteraction, Seconds { mostRecentUserInteraction }, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, [context, completionHandler] {
+    WebKit::toImpl(dataStoreRef)->mergeStatisticForTesting(URL(URL(), WebKit::toImpl(host)->string()), URL(URL(), WebKit::toImpl(topFrameDomain1)->string()), URL(URL(), WebKit::toImpl(topFrameDomain2)->string()), Seconds { lastSeen }, hadUserInteraction, Seconds { mostRecentUserInteraction }, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, [context, completionHandler] {
         completionHandler(context);
     });
 #else
@@ -261,6 +261,17 @@ void WKWebsiteDataStoreIsStatisticsHasHadUserInteraction(WKWebsiteDataStoreRef d
 #endif
 }
 
+void WKWebsiteDataStoreIsStatisticsOnlyInDatabaseOnce(WKWebsiteDataStoreRef dataStoreRef, WKStringRef subHost, WKStringRef topHost, void* context, WKWebsiteDataStoreIsStatisticsOnlyInDatabaseOnceFunction callback)
+{
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+    WebKit::toImpl(dataStoreRef)->isRelationshipOnlyInDatabaseOnce(URL(URL(), WebKit::toImpl(subHost)->string()), URL(URL(), WebKit::toImpl(topHost)->string()), [context, callback](bool onlyInDatabaseOnce) {
+        callback(onlyInDatabaseOnce, context);
+    });
+#else
+    callback(false, context);
+#endif
+}
+
 void WKWebsiteDataStoreSetStatisticsGrandfathered(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, bool value)
 {
 #if ENABLE(RESOURCE_LOAD_STATISTICS)
index f2a1782..c1f990b 100644 (file)
@@ -50,7 +50,7 @@ WK_EXPORT void WKWebsiteDataStoreSetResourceLoadStatisticsPrevalentResourceForDe
 typedef void (*WKWebsiteDataStoreStatisticsLastSeenFunction)(void* functionContext);
 WK_EXPORT void WKWebsiteDataStoreSetStatisticsLastSeen(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, double seconds, void* context, WKWebsiteDataStoreStatisticsLastSeenFunction completionHandler);
 typedef void (*WKWebsiteDataStoreStatisticsMergeStatisticFunction)(void* functionContext);
-WK_EXPORT void WKWebsiteDataStoreSetStatisticsMergeStatistic(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, WKStringRef topFrameDomain, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, void* context, WKWebsiteDataStoreStatisticsMergeStatisticFunction completionHandler);
+WK_EXPORT void WKWebsiteDataStoreSetStatisticsMergeStatistic(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, WKStringRef topFrameDomain1, WKStringRef topFrameDomain2, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, void* context, WKWebsiteDataStoreStatisticsMergeStatisticFunction completionHandler);
 typedef void (*WKWebsiteDataStoreStatisticsPrevalentResourceFunction)(void* functionContext);
 WK_EXPORT void WKWebsiteDataStoreSetStatisticsPrevalentResource(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, bool value, void* context, WKWebsiteDataStoreStatisticsPrevalentResourceFunction completionHandler);
 typedef void (*WKWebsiteDataStoreStatisticsVeryPrevalentResourceFunction)(void* functionContext);
@@ -70,6 +70,8 @@ typedef void (*WKWebsiteDataStoreStatisticsHasHadUserInteractionFunction)(void*
 WK_EXPORT void WKWebsiteDataStoreSetStatisticsHasHadUserInteraction(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, bool value, void* context, WKWebsiteDataStoreStatisticsHasHadUserInteractionFunction completionHandler);
 typedef void (*WKWebsiteDataStoreIsStatisticsHasHadUserInteractionFunction)(bool hasHadUserInteraction, void* functionContext);
 WK_EXPORT void WKWebsiteDataStoreIsStatisticsHasHadUserInteraction(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, void* context, WKWebsiteDataStoreIsStatisticsHasHadUserInteractionFunction callback);
+typedef void (*WKWebsiteDataStoreIsStatisticsOnlyInDatabaseOnceFunction)(bool onlyInDatabaseOnce, void* functionContext);
+WK_EXPORT void WKWebsiteDataStoreIsStatisticsOnlyInDatabaseOnce(WKWebsiteDataStoreRef dataStoreRef, WKStringRef subHost, WKStringRef topHost, void* context, WKWebsiteDataStoreIsStatisticsOnlyInDatabaseOnceFunction callback);
 WK_EXPORT void WKWebsiteDataStoreSetStatisticsGrandfathered(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, bool value);
 typedef void (*WKWebsiteDataStoreIsStatisticsGrandfatheredFunction)(bool isGrandfathered, void* functionContext);
 WK_EXPORT void WKWebsiteDataStoreIsStatisticsGrandfathered(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, void* context, WKWebsiteDataStoreIsStatisticsGrandfatheredFunction callback);
index 2773a06..50c40a5 100644 (file)
@@ -536,9 +536,9 @@ void NetworkProcessProxy::setLastSeen(PAL::SessionID sessionID, const Registrabl
     sendWithAsyncReply(Messages::NetworkProcess::SetLastSeen(sessionID, resourceDomain, lastSeen), WTFMove(completionHandler));
 }
 
-void NetworkProcessProxy::mergeStatisticForTesting(PAL::SessionID sessionID, const RegistrableDomain& resourceDomain, const RegistrableDomain& topFrameDomain, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&& completionHandler)
+void NetworkProcessProxy::mergeStatisticForTesting(PAL::SessionID sessionID, const RegistrableDomain& resourceDomain, const RegistrableDomain& topFrameDomain1, const RegistrableDomain& topFrameDomain2, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&& completionHandler)
 {
-    sendWithAsyncReply(Messages::NetworkProcess::MergeStatisticForTesting(sessionID, resourceDomain, topFrameDomain, lastSeen, hadUserInteraction, mostRecentUserInteraction, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved), WTFMove(completionHandler));
+    sendWithAsyncReply(Messages::NetworkProcess::MergeStatisticForTesting(sessionID, resourceDomain, topFrameDomain1, topFrameDomain2, lastSeen, hadUserInteraction, mostRecentUserInteraction, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved), WTFMove(completionHandler));
 }
 
 void NetworkProcessProxy::clearPrevalentResource(PAL::SessionID sessionID, const RegistrableDomain& resourceDomain, CompletionHandler<void()>&& completionHandler)
@@ -601,6 +601,11 @@ void NetworkProcessProxy::hasHadUserInteraction(PAL::SessionID sessionID, const
     sendWithAsyncReply(Messages::NetworkProcess::HadUserInteraction(sessionID, resourceDomain), WTFMove(completionHandler));
 }
 
+void NetworkProcessProxy::isRelationshipOnlyInDatabaseOnce(PAL::SessionID sessionID, const RegistrableDomain& subDomain, const RegistrableDomain& topDomain, CompletionHandler<void(bool)>&& completionHandler)
+{
+    sendWithAsyncReply(Messages::NetworkProcess::IsRelationshipOnlyInDatabaseOnce(sessionID, subDomain, topDomain), WTFMove(completionHandler));
+}
+
 void NetworkProcessProxy::clearUserInteraction(PAL::SessionID sessionID, const RegistrableDomain& resourceDomain, CompletionHandler<void()>&& completionHandler)
 {
     if (!canSendMessage()) {
index 2d6adbe..04cd511 100644 (file)
@@ -103,6 +103,7 @@ public:
     void dumpResourceLoadStatistics(PAL::SessionID, CompletionHandler<void(String)>&&);
     void updatePrevalentDomainsToBlockCookiesFor(PAL::SessionID, const Vector<RegistrableDomain>&, CompletionHandler<void()>&&);
     void hasHadUserInteraction(PAL::SessionID, const RegistrableDomain&, CompletionHandler<void(bool)>&&);
+    void isRelationshipOnlyInDatabaseOnce(PAL::SessionID, const RegistrableDomain& subDomain, const RegistrableDomain& topDomain, CompletionHandler<void(bool)>&&);
     void hasLocalStorage(PAL::SessionID, const RegistrableDomain&, CompletionHandler<void(bool)>&&);
     void isGrandfathered(PAL::SessionID, const RegistrableDomain&, CompletionHandler<void(bool)>&&);
     void isPrevalentResource(PAL::SessionID, const RegistrableDomain&, CompletionHandler<void(bool)>&&);
@@ -113,7 +114,7 @@ public:
     void logUserInteraction(PAL::SessionID, const RegistrableDomain&, CompletionHandler<void()>&&);
     void scheduleStatisticsAndDataRecordsProcessing(PAL::SessionID, CompletionHandler<void()>&&);
     void setLastSeen(PAL::SessionID, const RegistrableDomain&, Seconds, CompletionHandler<void()>&&);
-    void mergeStatisticForTesting(PAL::SessionID, const RegistrableDomain&, const TopFrameDomain&, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&&);
+    void mergeStatisticForTesting(PAL::SessionID, const RegistrableDomain&, const TopFrameDomain& topFrameDomain1, const TopFrameDomain& topFrameDomain2, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&&);
     void setAgeCapForClientSideCookies(PAL::SessionID, Optional<Seconds>, CompletionHandler<void()>&&);
     void setCacheMaxAgeCap(PAL::SessionID, Seconds, CompletionHandler<void()>&&);
     void setGrandfathered(PAL::SessionID, const RegistrableDomain&, bool isGrandfathered, CompletionHandler<void()>&&);
index 627d2d9..2619016 100644 (file)
@@ -1551,7 +1551,7 @@ void WebsiteDataStore::setLastSeen(const URL& url, Seconds seconds, CompletionHa
     }
 }
 
-void WebsiteDataStore::mergeStatisticForTesting(const URL& url , const URL& topFrameUrl, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&& completionHandler)
+void WebsiteDataStore::mergeStatisticForTesting(const URL& url , const URL& topFrameUrl1, const URL& topFrameUrl2, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&& completionHandler)
 {
     if (url.protocolIsAbout() || url.isEmpty()) {
         completionHandler();
@@ -1562,7 +1562,7 @@ void WebsiteDataStore::mergeStatisticForTesting(const URL& url , const URL& topF
 
     for (auto& processPool : processPools()) {
         if (auto* process = processPool->networkProcess())
-            process->mergeStatisticForTesting(m_sessionID, WebCore::RegistrableDomain { url }, WebCore::RegistrableDomain { topFrameUrl }, lastSeen, hadUserInteraction, mostRecentUserInteraction, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, [processPool, callbackAggregator = callbackAggregator.copyRef()] { });
+            process->mergeStatisticForTesting(m_sessionID, WebCore::RegistrableDomain { url }, WebCore::RegistrableDomain { topFrameUrl1 }, WebCore::RegistrableDomain { topFrameUrl2 }, lastSeen, hadUserInteraction, mostRecentUserInteraction, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, [processPool, callbackAggregator = callbackAggregator.copyRef()] { });
     }
 }
 
@@ -1646,6 +1646,24 @@ void WebsiteDataStore::hasHadUserInteraction(const URL& url, CompletionHandler<v
     }
 }
 
+void WebsiteDataStore::isRelationshipOnlyInDatabaseOnce(const URL& subUrl, const URL& topUrl, CompletionHandler<void(bool)>&& completionHandler)
+{
+    ASSERT(RunLoop::isMain());
+    
+    if (subUrl.protocolIsAbout() || subUrl.isEmpty() || topUrl.protocolIsAbout() || topUrl.isEmpty()) {
+        completionHandler(false);
+        return;
+    }
+    
+    for (auto& processPool : processPools()) {
+        if (auto* process = processPool->networkProcess()) {
+            process->isRelationshipOnlyInDatabaseOnce(m_sessionID, WebCore::RegistrableDomain { subUrl }, WebCore::RegistrableDomain { topUrl }, WTFMove(completionHandler));
+            ASSERT(processPools().size() == 1);
+            break;
+        }
+    }
+}
+
 void WebsiteDataStore::clearUserInteraction(const URL& url, CompletionHandler<void()>&& completionHandler)
 {
     ASSERT(RunLoop::isMain());
index 01837fe..4d49091 100644 (file)
@@ -147,6 +147,7 @@ public:
     void logUserInteraction(const URL&, CompletionHandler<void()>&&);
     void getAllStorageAccessEntries(WebPageProxyIdentifier, CompletionHandler<void(Vector<String>&& domains)>&&);
     void hasHadUserInteraction(const URL&, CompletionHandler<void(bool)>&&);
+    void isRelationshipOnlyInDatabaseOnce(const URL& subUrl, const URL& topUrl, CompletionHandler<void(bool)>&&);
     void isPrevalentResource(const URL&, CompletionHandler<void(bool)>&&);
     void isRegisteredAsRedirectingTo(const URL& hostRedirectedFrom, const URL& hostRedirectedTo, CompletionHandler<void(bool)>&&);
     void isRegisteredAsSubresourceUnder(const URL& subresource, const URL& topFrame, CompletionHandler<void(bool)>&&);
@@ -163,7 +164,7 @@ public:
     void setUseITPDatabase(bool);
     void setGrandfatheringTime(Seconds, CompletionHandler<void()>&&);
     void setLastSeen(const URL&, Seconds, CompletionHandler<void()>&&);
-    void mergeStatisticForTesting(const URL&, const URL& topFrameUrl, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&&);
+    void mergeStatisticForTesting(const URL&, const URL& topFrameUrl1, const URL& topFrameUrl2, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&&);
     void setNotifyPagesWhenDataRecordsWereScanned(bool, CompletionHandler<void()>&&);
     void setIsRunningResourceLoadStatisticsTest(bool, CompletionHandler<void()>&&);
     void setPruneEntriesDownTo(size_t, CompletionHandler<void()>&&);
index 894ee81..e7bb5e0 100644 (file)
@@ -1,3 +1,27 @@
+2019-10-07  Kate Cheney  <katherine_cheney@apple.com>
+
+        Domain relationships in the ITP Database should be inserted in a single query and ignore repeat insert attempts. (202604)
+        https://bugs.webkit.org/show_bug.cgi?id=202604
+        <rdar://problem/55995831>
+
+        Reviewed by Chris Dumez. 
+
+        Updates to testing infrastructure to test successful list-merging of
+        top domains into the ITP SQLite database and test against repeat
+        inserts.
+        * WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl:
+        * WebKitTestRunner/InjectedBundle/TestRunner.cpp:
+        (WTR::TestRunner::setStatisticsMergeStatistic):
+        (WTR::TestRunner::isStatisticsHasHadUserInteraction):
+        (WTR::TestRunner::isStatisticsOnlyInDatabaseOnce):
+        * WebKitTestRunner/InjectedBundle/TestRunner.h:
+        * WebKitTestRunner/TestController.cpp:
+        (WTR::TestController::setStatisticsMergeStatistic):
+        (WTR::TestController::isStatisticsOnlyInDatabaseOnce):
+        * WebKitTestRunner/TestController.h:
+        * WebKitTestRunner/TestInvocation.cpp:
+        (WTR::TestInvocation::didReceiveSynchronousMessageFromInjectedBundle):
+
 2019-10-07  Matt Lewis  <jlewis3@apple.com>
 
         Bring up queues for Catalina
index 42aad54..761812b 100644 (file)
@@ -298,7 +298,7 @@ interface TestRunner {
     void setStatisticsDebugMode(boolean value, object completionHandler);
     void setStatisticsPrevalentResourceForDebugMode(DOMString hostName, object completionHandler);
     void setStatisticsLastSeen(DOMString hostName, double seconds, object completionHandler);
-    void setStatisticsMergeStatistic(DOMString hostName, DOMString topFrameDomain, double lastSeen, boolean hadUserInteraction, double mostRecentUserInteraction, boolean isGrandfathered, boolean isPrevalent, boolean isVeryPrevalent, unsigned long dataRecordsRemoved, object completionHandler);
+    void setStatisticsMergeStatistic(DOMString hostName, DOMString topFrameDomain1, DOMString topFrameDomain2, double lastSeen, boolean hadUserInteraction, double mostRecentUserInteraction, boolean isGrandfathered, boolean isPrevalent, boolean isVeryPrevalent, unsigned long dataRecordsRemoved, object completionHandler);
     void setStatisticsPrevalentResource(DOMString hostName, boolean value, object completionHandler);
     void setStatisticsVeryPrevalentResource(DOMString hostName, boolean value, object completionHandler);
     boolean isStatisticsPrevalentResource(DOMString hostName);
@@ -308,6 +308,7 @@ interface TestRunner {
     boolean isStatisticsRegisteredAsRedirectingTo(DOMString hostRedirectedFrom, DOMString hostRedirectedTo);
     void setStatisticsHasHadUserInteraction(DOMString hostName, boolean value, object completionHandler);
     boolean isStatisticsHasHadUserInteraction(DOMString hostName);
+    boolean isStatisticsOnlyInDatabaseOnce(DOMString subHost, DOMString topHost);
     void setStatisticsGrandfathered(DOMString hostName, boolean value);
     boolean isStatisticsGrandfathered(DOMString hostName);
     void setUseITPDatabase(boolean value);
index 01104ac..6a3a31a 100644 (file)
@@ -1454,7 +1454,7 @@ void TestRunner::statisticsCallDidSetLastSeenCallback()
     callTestRunnerCallback(SetStatisticsLastSeenCallbackID);
 }
 
-void TestRunner::setStatisticsMergeStatistic(JSStringRef hostName, JSStringRef topFrameDomain, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, JSValueRef completionHandler)
+void TestRunner::setStatisticsMergeStatistic(JSStringRef hostName, JSStringRef topFrameDomain1, JSStringRef topFrameDomain2, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, JSValueRef completionHandler)
 {
     cacheTestRunnerCallback(SetStatisticsMergeStatisticCallbackID, completionHandler);
 
@@ -1464,8 +1464,11 @@ void TestRunner::setStatisticsMergeStatistic(JSStringRef hostName, JSStringRef t
     keys.append(adoptWK(WKStringCreateWithUTF8CString("HostName")));
     values.append(adoptWK(WKStringCreateWithJSString(hostName)));
     
-    keys.append(adoptWK(WKStringCreateWithUTF8CString("TopFrameDomain")));
-    values.append(adoptWK(WKStringCreateWithJSString(topFrameDomain)));
+    keys.append(adoptWK(WKStringCreateWithUTF8CString("TopFrameDomain1")));
+    values.append(adoptWK(WKStringCreateWithJSString(topFrameDomain1)));
+    
+    keys.append(adoptWK(WKStringCreateWithUTF8CString("TopFrameDomain2")));
+    values.append(adoptWK(WKStringCreateWithJSString(topFrameDomain2)));
 
     keys.append(adoptWK(WKStringCreateWithUTF8CString("LastSeen")));
     values.append(adoptWK(WKDoubleCreate(lastSeen)));
@@ -1724,6 +1727,34 @@ bool TestRunner::isStatisticsHasHadUserInteraction(JSStringRef hostName)
     return WKBooleanGetValue(adoptWK(static_cast<WKBooleanRef>(returnData)).get());
 }
 
+bool TestRunner::isStatisticsOnlyInDatabaseOnce(JSStringRef subHost, JSStringRef topHost)
+{
+    
+    Vector<WKRetainPtr<WKStringRef>> keys;
+    Vector<WKRetainPtr<WKTypeRef>> values;
+
+    keys.append(adoptWK(WKStringCreateWithUTF8CString("SubHost")));
+    values.append(adoptWK(WKStringCreateWithJSString(subHost)));
+    
+    keys.append(adoptWK(WKStringCreateWithUTF8CString("TopHost")));
+    values.append(adoptWK(WKStringCreateWithJSString(topHost)));
+    
+    Vector<WKStringRef> rawKeys(keys.size());
+    Vector<WKTypeRef> rawValues(values.size());
+
+    for (size_t i = 0; i < keys.size(); ++i) {
+        rawKeys[i] = keys[i].get();
+        rawValues[i] = values[i].get();
+    }
+
+    auto messageName = adoptWK(WKStringCreateWithUTF8CString("IsStatisticsOnlyInDatabaseOnce"));
+    auto messageBody = adoptWK(WKDictionaryCreate(rawKeys.data(), rawValues.data(), rawKeys.size()));
+    WKTypeRef returnData = nullptr;
+    WKBundlePagePostSynchronousMessageForTesting(InjectedBundle::singleton().page()->page(), messageName.get(), messageBody.get(), &returnData);
+    ASSERT(WKGetTypeID(returnData) == WKBooleanGetTypeID());
+    return WKBooleanGetValue(adoptWK(static_cast<WKBooleanRef>(returnData)).get());
+}
+
 void TestRunner::setStatisticsGrandfathered(JSStringRef hostName, bool value)
 {
     Vector<WKRetainPtr<WKStringRef>> keys;
index f170012..252643f 100644 (file)
@@ -396,7 +396,7 @@ public:
     void statisticsCallDidSetPrevalentResourceForDebugModeCallback();
     void setStatisticsLastSeen(JSStringRef hostName, double seconds, JSValueRef completionHandler);
     void statisticsCallDidSetLastSeenCallback();
-    void setStatisticsMergeStatistic(JSStringRef hostName, JSStringRef topFrameDomain, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, JSValueRef completionHandler);
+    void setStatisticsMergeStatistic(JSStringRef hostName, JSStringRef topFrameDomain1, JSStringRef topFrameDomain2, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, JSValueRef completionHandler);
     void statisticsCallDidSetMergeStatisticCallback();
     void setStatisticsPrevalentResource(JSStringRef hostName, bool value, JSValueRef completionHandler);
     void statisticsCallDidSetPrevalentResourceCallback();
@@ -410,6 +410,7 @@ public:
     void setStatisticsHasHadUserInteraction(JSStringRef hostName, bool value, JSValueRef completionHandler);
     void statisticsCallDidSetHasHadUserInteractionCallback();
     bool isStatisticsHasHadUserInteraction(JSStringRef hostName);
+    bool isStatisticsOnlyInDatabaseOnce(JSStringRef subHost, JSStringRef topHost);
     void setStatisticsGrandfathered(JSStringRef hostName, bool value);
     bool isStatisticsGrandfathered(JSStringRef hostName);
     void setUseITPDatabase(bool value);
index 9b8071d..7cd192a 100644 (file)
@@ -3267,10 +3267,10 @@ void TestController::setStatisticsLastSeen(WKStringRef host, double seconds)
     m_currentInvocation->didSetLastSeen();
 }
 
-void TestController::setStatisticsMergeStatistic(WKStringRef host, WKStringRef topFrameDomain, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, int dataRecordsRemoved)
+void TestController::setStatisticsMergeStatistic(WKStringRef host, WKStringRef topFrameDomain1, WKStringRef topFrameDomain2, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, int dataRecordsRemoved)
 {
     ResourceStatisticsCallbackContext context(*this);
-    WKWebsiteDataStoreSetStatisticsMergeStatistic(TestController::websiteDataStore(), host, topFrameDomain, lastSeen, hadUserInteraction, mostRecentUserInteraction, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, &context, resourceStatisticsVoidResultCallback);
+    WKWebsiteDataStoreSetStatisticsMergeStatistic(TestController::websiteDataStore(), host, topFrameDomain1, topFrameDomain2, lastSeen, hadUserInteraction, mostRecentUserInteraction, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, &context, resourceStatisticsVoidResultCallback);
     runUntil(context.done, noTimeout);
     m_currentInvocation->didMergeStatistic();
 }
@@ -3355,6 +3355,14 @@ bool TestController::isStatisticsHasHadUserInteraction(WKStringRef host)
     return context.result;
 }
 
+bool TestController::isStatisticsOnlyInDatabaseOnce(WKStringRef subHost, WKStringRef topHost)
+{
+    ResourceStatisticsCallbackContext context(*this);
+    WKWebsiteDataStoreIsStatisticsOnlyInDatabaseOnce(TestController::websiteDataStore(), subHost, topHost, &context, resourceStatisticsBooleanResultCallback);
+    runUntil(context.done, noTimeout);
+    return context.result;
+}
+
 void TestController::setStatisticsGrandfathered(WKStringRef host, bool value)
 {
     WKWebsiteDataStoreSetStatisticsGrandfathered(TestController::websiteDataStore(), host, value);
index 6c2d2d8..3c71519 100644 (file)
@@ -210,7 +210,7 @@ public:
     void setStatisticsDebugMode(bool value);
     void setStatisticsPrevalentResourceForDebugMode(WKStringRef hostName);
     void setStatisticsLastSeen(WKStringRef hostName, double seconds);
-    void setStatisticsMergeStatistic(WKStringRef host, WKStringRef topFrameDomain, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, int dataRecordsRemoved);
+    void setStatisticsMergeStatistic(WKStringRef host, WKStringRef topFrameDomain1, WKStringRef topFrameDomain2, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, int dataRecordsRemoved);
     void setStatisticsPrevalentResource(WKStringRef hostName, bool value);
     void setStatisticsVeryPrevalentResource(WKStringRef hostName, bool value);
     String dumpResourceLoadStatistics();
@@ -221,6 +221,7 @@ public:
     bool isStatisticsRegisteredAsRedirectingTo(WKStringRef hostRedirectedFrom, WKStringRef hostRedirectedTo);
     void setStatisticsHasHadUserInteraction(WKStringRef hostName, bool value);
     bool isStatisticsHasHadUserInteraction(WKStringRef hostName);
+    bool isStatisticsOnlyInDatabaseOnce(WKStringRef subHost, WKStringRef topHost);
     void setStatisticsGrandfathered(WKStringRef hostName, bool value);
     bool isStatisticsGrandfathered(WKStringRef hostName);
     void setUseITPDatabase(bool value);
index 2f8b306..0d12e70 100644 (file)
@@ -1105,7 +1105,8 @@ WKRetainPtr<WKTypeRef> TestInvocation::didReceiveSynchronousMessageFromInjectedB
 
         WKDictionaryRef messageBodyDictionary = static_cast<WKDictionaryRef>(messageBody);
         WKRetainPtr<WKStringRef> hostNameKey = adoptWK(WKStringCreateWithUTF8CString("HostName"));
-        WKRetainPtr<WKStringRef> topFrameDomainKey = adoptWK(WKStringCreateWithUTF8CString("TopFrameDomain"));
+        WKRetainPtr<WKStringRef> topFrameDomain1Key = adoptWK(WKStringCreateWithUTF8CString("TopFrameDomain1"));
+        WKRetainPtr<WKStringRef> topFrameDomain2Key = adoptWK(WKStringCreateWithUTF8CString("TopFrameDomain2"));
         WKRetainPtr<WKStringRef> lastSeenKey = adoptWK(WKStringCreateWithUTF8CString("LastSeen"));
         WKRetainPtr<WKStringRef> hadUserInteractionKey = adoptWK(WKStringCreateWithUTF8CString("HadUserInteraction"));
         WKRetainPtr<WKStringRef> mostRecentUserInteractionKey = adoptWK(WKStringCreateWithUTF8CString("MostRecentUserInteraction"));
@@ -1116,7 +1117,8 @@ WKRetainPtr<WKTypeRef> TestInvocation::didReceiveSynchronousMessageFromInjectedB
         WKRetainPtr<WKStringRef> timesAccessedFirstPartyInteractionKey = adoptWK(WKStringCreateWithUTF8CString("TimesAccessedFirstPartyInteraction"));
 
         WKStringRef hostName = static_cast<WKStringRef>(WKDictionaryGetItemForKey(messageBodyDictionary, hostNameKey.get()));
-        WKStringRef topFrameDomain = static_cast<WKStringRef>(WKDictionaryGetItemForKey(messageBodyDictionary, topFrameDomainKey.get()));
+        WKStringRef topFrameDomain1 = static_cast<WKStringRef>(WKDictionaryGetItemForKey(messageBodyDictionary, topFrameDomain1Key.get()));
+        WKStringRef topFrameDomain2 = static_cast<WKStringRef>(WKDictionaryGetItemForKey(messageBodyDictionary, topFrameDomain2Key.get()));
         WKDoubleRef lastSeen = static_cast<WKDoubleRef>(WKDictionaryGetItemForKey(messageBodyDictionary, lastSeenKey.get()));
         WKBooleanRef hadUserInteraction = static_cast<WKBooleanRef>(WKDictionaryGetItemForKey(messageBodyDictionary, hadUserInteractionKey.get()));
         WKDoubleRef mostRecentUserInteraction = static_cast<WKDoubleRef>(WKDictionaryGetItemForKey(messageBodyDictionary, mostRecentUserInteractionKey.get()));
@@ -1125,7 +1127,7 @@ WKRetainPtr<WKTypeRef> TestInvocation::didReceiveSynchronousMessageFromInjectedB
         WKBooleanRef isVeryPrevalent = static_cast<WKBooleanRef>(WKDictionaryGetItemForKey(messageBodyDictionary, isVeryPrevalentKey.get()));
         WKUInt64Ref dataRecordsRemoved = static_cast<WKUInt64Ref>(WKDictionaryGetItemForKey(messageBodyDictionary, dataRecordsRemovedKey.get()));
         
-        TestController::singleton().setStatisticsMergeStatistic(hostName, topFrameDomain, WKDoubleGetValue(lastSeen), WKBooleanGetValue(hadUserInteraction), WKDoubleGetValue(mostRecentUserInteraction), WKBooleanGetValue(isGrandfathered), WKBooleanGetValue(isPrevalent), WKBooleanGetValue(isVeryPrevalent), WKUInt64GetValue(dataRecordsRemoved));
+        TestController::singleton().setStatisticsMergeStatistic(hostName, topFrameDomain1, topFrameDomain2, WKDoubleGetValue(lastSeen), WKBooleanGetValue(hadUserInteraction), WKDoubleGetValue(mostRecentUserInteraction), WKBooleanGetValue(isGrandfathered), WKBooleanGetValue(isPrevalent), WKBooleanGetValue(isVeryPrevalent), WKUInt64GetValue(dataRecordsRemoved));
 
         return nullptr;
     }
@@ -1249,6 +1251,20 @@ WKRetainPtr<WKTypeRef> TestInvocation::didReceiveSynchronousMessageFromInjectedB
         return result;
     }
     
+    if (WKStringIsEqualToUTF8CString(messageName, "IsStatisticsOnlyInDatabaseOnce")) {
+        ASSERT(WKGetTypeID(messageBody) == WKDictionaryGetTypeID());
+        
+        WKDictionaryRef messageBodyDictionary = static_cast<WKDictionaryRef>(messageBody);
+        auto subHostKey = adoptWK(WKStringCreateWithUTF8CString("SubHost"));
+        auto topHostKey = adoptWK(WKStringCreateWithUTF8CString("TopHost"));
+        
+        WKStringRef subHost = static_cast<WKStringRef>(WKDictionaryGetItemForKey(messageBodyDictionary, subHostKey.get()));
+        WKStringRef topHost = static_cast<WKStringRef>(WKDictionaryGetItemForKey(messageBodyDictionary, topHostKey.get()));
+
+        bool statisticInDatabaseOnce = TestController::singleton().isStatisticsOnlyInDatabaseOnce(subHost, topHost);
+        return adoptWK(WKBooleanCreate(statisticInDatabaseOnce));
+    }
+    
     if (WKStringIsEqualToUTF8CString(messageName, "SetStatisticsGrandfathered")) {
         ASSERT(WKGetTypeID(messageBody) == WKDictionaryGetTypeID());