Updated resource load statistics are never merged into the SQLite Database backend...
authorcommit-queue@webkit.org <commit-queue@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 2 Oct 2019 19:21:54 +0000 (19:21 +0000)
committercommit-queue@webkit.org <commit-queue@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 2 Oct 2019 19:21:54 +0000 (19:21 +0000)
https://bugs.webkit.org/show_bug.cgi?id=202372
<rdar://problem/55854542>

Patch by Kate Cheney <katherine_cheney@apple.com> on 2019-10-02
Reviewed by Brent Fulgham.

Source/WebKit:

This patch has a lot of changes to the test infrastructure to be able
to test the mergeStatistics function. Merging functionality mimics
that of the ResourceLoadStatisticsMemoryStore and the merge() function in
ResourceLoadStatistics.cpp.

* NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:
(WebKit::ResourceLoadStatisticsDatabaseStore::ResourceLoadStatisticsDatabaseStore):
Added new SQLite queries that are needed to merge new statistics.

(WebKit::ResourceLoadStatisticsDatabaseStore::prepareStatements):
Added new prepare statements for new SQLite queries.

(WebKit::ResourceLoadStatisticsDatabaseStore::insertObservedDomain):
Updated to use domainID() instead of confirmDomainDoesNotExist which
was deleted. Changed the insert bind parameters
to utilize new enum of Observed Domain table indices which will help
reduce errors in the future if the database schema changes.

(WebKit::ResourceLoadStatisticsDatabaseStore::relationshipExists const):
Changed {public} to {private} in logging statement to avoid leaking
sensitive information when logging the query error.

(WebKit::ResourceLoadStatisticsDatabaseStore::confirmDomainDoesNotExist const): Deleted.
This function essentially does the exact same thing as domainID and
we can reduce code duplication by deleting it and transitioning all
other functions to use domainID instead.

(WebKit::ResourceLoadStatisticsDatabaseStore::domainID const):
Changed {public} to {private} in logging statement to avoid leaking
sensitive information when logging the query error.

(WebKit::ResourceLoadStatisticsDatabaseStore::insertDomainRelationships):
In order to reuse this code for the merge statistics function, I added
a check before each insert call to ensure the relationship does not
already exist in the database. This was not needed before because it
was only called on an empty database.

(WebKit::ResourceLoadStatisticsDatabaseStore::merge):
(WebKit::ResourceLoadStatisticsDatabaseStore::mergeStatistic):
(WebKit::ResourceLoadStatisticsDatabaseStore::mergeStatistics):
Merges new statistic.

(WebKit::ResourceLoadStatisticsDatabaseStore::incrementRecordsDeletedCountForDomains):
(WebKit::ResourceLoadStatisticsDatabaseStore::recursivelyFindNonPrevalentDomainsThatRedirectedToThisDomain):
(WebKit::ResourceLoadStatisticsDatabaseStore::markAsPrevalentIfHasRedirectedToPrevalent):
(WebKit::ResourceLoadStatisticsDatabaseStore::requestStorageAccess):
(WebKit::ResourceLoadStatisticsDatabaseStore::requestStorageAccessUnderOpener):
(WebKit::ResourceLoadStatisticsDatabaseStore::grandfatherDataForDomains):
(WebKit::ResourceLoadStatisticsDatabaseStore::ensurePrevalentResourcesForDebugMode):
(WebKit::ResourceLoadStatisticsDatabaseStore::setUserInteraction):
(WebKit::ResourceLoadStatisticsDatabaseStore::clearUserInteraction):
(WebKit::ResourceLoadStatisticsDatabaseStore::hasHadUserInteraction):
(WebKit::ResourceLoadStatisticsDatabaseStore::setPrevalentResource):
(WebKit::ResourceLoadStatisticsDatabaseStore::setDomainsAsPrevalent):
(WebKit::ResourceLoadStatisticsDatabaseStore::predicateValueForDomain const):
(WebKit::ResourceLoadStatisticsDatabaseStore::clearPrevalentResource):
Changed {public} to {private} in logging statement to avoid leaking
sensitive information when logging the query error.

(WebKit::ResourceLoadStatisticsDatabaseStore::setGrandfathered):
Fix a bug uncovered by the new test cases.

(WebKit::ResourceLoadStatisticsDatabaseStore::ensureResourceStatisticsForRegistrableDomain):
(WebKit::ResourceLoadStatisticsDatabaseStore::clearDatabaseContents):
(WebKit::ResourceLoadStatisticsDatabaseStore::cookieTreatmentForOrigin const):
(WebKit::ResourceLoadStatisticsDatabaseStore::clearExpiredUserInteractions):
(WebKit::ResourceLoadStatisticsDatabaseStore::clearGrandfathering):
(WebKit::ResourceLoadStatisticsDatabaseStore::pruneStatisticsIfNeeded):
(WebKit::ResourceLoadStatisticsDatabaseStore::updateLastSeen):
Changed {public} to {private} in logging statement to avoid leaking
sensitive information when logging the query error.

(WebKit::ResourceLoadStatisticsDatabaseStore::updateDataRecordsRemoved):
A new function that utilizes a query needed for the merging of two
statistics.

(WebKit::ResourceLoadStatisticsDatabaseStore::confirmDomainDoesNotExist const): Deleted.
* NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h:

* NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp:
(WebKit::ResourceLoadStatisticsMemoryStore::mergeStatistics):
(WebKit::ResourceLoadStatisticsMemoryStore::wasAccessedAsFirstPartyDueToUserInteraction const): Deleted.
Removed the check for updating times accessed due to first party
interaction, which is data no longer needed in updated ITP.

* NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h:
* NetworkProcess/Classifier/ResourceLoadStatisticsStore.h:
Added the mergeStatistics function (previously only in the memory
store) to the parent class now that it is used by both stores.

* NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:
(WebKit::WebResourceLoadStatisticsStore::resourceLoadStatisticsUpdated):
This function now calls mergeStatistics on the database store if
enabled.

(WebKit::WebResourceLoadStatisticsStore::mergeStatisticForTesting):
This function builds a vector from the sample data to test the
mergeStatistics() function of the database store.

* NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h:
* NetworkProcess/NetworkProcess.cpp:
(WebKit::NetworkProcess::mergeStatisticForTesting):
* NetworkProcess/NetworkProcess.h:
* NetworkProcess/NetworkProcess.messages.in:
* UIProcess/API/C/WKWebsiteDataStoreRef.cpp:
(WKWebsiteDataStoreSetStatisticsMergeStatistic):
Fixed bug where isGrandfathered function in WKWebsiteDataStoreRef
was calling hasHadUserInteraction by mistake.

* UIProcess/API/C/WKWebsiteDataStoreRef.h:
* UIProcess/Network/NetworkProcessProxy.cpp:
(WebKit::NetworkProcessProxy::mergeStatisticForTesting):
* UIProcess/Network/NetworkProcessProxy.h:
This code is for testing the mergeStatistics() function.

* UIProcess/WebsiteData/WebsiteDataStore.cpp:
(WebKit::WebsiteDataStore::isGrandfathered):
(WebKit::WebsiteDataStore::mergeStatisticForTesting):
* UIProcess/WebsiteData/WebsiteDataStore.h:
Added this function which contacts the networkProcess to retrieve the
grandfathered value for a domain. This was not included before because
of a bug in WKWebsiteDataStoreRef.

Tools:

Added testing functionality to the mergeStatistics() function of the
SQLite backend.

* WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl:
* WebKitTestRunner/InjectedBundle/InjectedBundle.cpp:
(WTR::InjectedBundle::didReceiveMessageToPage):
* WebKitTestRunner/InjectedBundle/TestRunner.cpp:
(WTR::TestRunner::setStatisticsMergeStatistic):
(WTR::TestRunner::statisticsCallDidSetMergeStatisticCallback):
* WebKitTestRunner/InjectedBundle/TestRunner.h:
* WebKitTestRunner/TestController.cpp:
(WTR::TestController::setStatisticsMergeStatistic):
* WebKitTestRunner/TestController.h:
* WebKitTestRunner/TestInvocation.cpp:
(WTR::TestInvocation::didReceiveSynchronousMessageFromInjectedBundle):
(WTR::TestInvocation::didMergeStatistic):
* WebKitTestRunner/TestInvocation.h:

LayoutTests:

Added layout tests to test succesful merging of resource load
statistics into SQLite backend.

* http/tests/resourceLoadStatistics/merge-statistic-does-not-overwrite-database-expected.txt: Added.
* http/tests/resourceLoadStatistics/merge-statistic-does-not-overwrite-database.html: Added.
* http/tests/resourceLoadStatistics/merge-statistic-does-overwrite-database-expected.txt: Added.
* http/tests/resourceLoadStatistics/merge-statistic-does-overwrite-database.html: Added.
* http/tests/resourceLoadStatistics/merge-statistic-does-partially-overwrite-database-expected.txt: Added.
* http/tests/resourceLoadStatistics/merge-statistic-does-partially-overwrite-database.html: Added.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@250621 268f45cc-cd09-0410-ab3c-d52691b4dbfc

33 files changed:
LayoutTests/ChangeLog
LayoutTests/http/tests/resourceLoadStatistics/merge-statistic-does-not-overwrite-database-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/resourceLoadStatistics/merge-statistic-does-not-overwrite-database.html [new file with mode: 0644]
LayoutTests/http/tests/resourceLoadStatistics/merge-statistic-does-overwrite-database-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/resourceLoadStatistics/merge-statistic-does-overwrite-database.html [new file with mode: 0644]
LayoutTests/http/tests/resourceLoadStatistics/merge-statistic-does-partially-overwrite-database-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/resourceLoadStatistics/merge-statistic-does-partially-overwrite-database.html [new file with mode: 0644]
Source/WebKit/ChangeLog
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsStore.h
Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp
Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h
Source/WebKit/NetworkProcess/NetworkProcess.cpp
Source/WebKit/NetworkProcess/NetworkProcess.h
Source/WebKit/NetworkProcess/NetworkProcess.messages.in
Source/WebKit/UIProcess/API/C/WKWebsiteDataStoreRef.cpp
Source/WebKit/UIProcess/API/C/WKWebsiteDataStoreRef.h
Source/WebKit/UIProcess/Network/NetworkProcessProxy.cpp
Source/WebKit/UIProcess/Network/NetworkProcessProxy.h
Source/WebKit/UIProcess/WebsiteData/WebsiteDataStore.cpp
Source/WebKit/UIProcess/WebsiteData/WebsiteDataStore.h
Tools/ChangeLog
Tools/WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl
Tools/WebKitTestRunner/InjectedBundle/InjectedBundle.cpp
Tools/WebKitTestRunner/InjectedBundle/TestRunner.cpp
Tools/WebKitTestRunner/InjectedBundle/TestRunner.h
Tools/WebKitTestRunner/TestController.cpp
Tools/WebKitTestRunner/TestController.h
Tools/WebKitTestRunner/TestInvocation.cpp
Tools/WebKitTestRunner/TestInvocation.h

index a84a92f..251866b 100644 (file)
@@ -1,3 +1,21 @@
+2019-10-02  Kate Cheney  <katherine_cheney@apple.com>
+
+        Updated resource load statistics are never merged into the SQLite Database backend (202372).
+        https://bugs.webkit.org/show_bug.cgi?id=202372
+        <rdar://problem/55854542>
+
+        Reviewed by Brent Fulgham. 
+
+        Added layout tests to test succesful merging of resource load
+        statistics into SQLite backend.
+
+        * http/tests/resourceLoadStatistics/merge-statistic-does-not-overwrite-database-expected.txt: Added.
+        * http/tests/resourceLoadStatistics/merge-statistic-does-not-overwrite-database.html: Added.
+        * http/tests/resourceLoadStatistics/merge-statistic-does-overwrite-database-expected.txt: Added.
+        * http/tests/resourceLoadStatistics/merge-statistic-does-overwrite-database.html: Added.
+        * http/tests/resourceLoadStatistics/merge-statistic-does-partially-overwrite-database-expected.txt: Added.
+        * http/tests/resourceLoadStatistics/merge-statistic-does-partially-overwrite-database.html: Added.
+
 2019-10-02  Miguel Gomez  <magomez@igalia.com>
 
         [WPE] Unreviewed gardening: add new expectations after r250602
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/merge-statistic-does-not-overwrite-database-expected.txt b/LayoutTests/http/tests/resourceLoadStatistics/merge-statistic-does-not-overwrite-database-expected.txt
new file mode 100644 (file)
index 0000000..bb401f9
--- /dev/null
@@ -0,0 +1,13 @@
+Tests that merged statistic does not overwrite old statistic
+
+On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
+
+
+PASS Host did not overwrite old prevalent resource value.
+PASS Host did not overwrite old very prevalent resource value.
+PASS Host did not overwrite old had user interaction value
+PASS Host did not overwrite old grandfathered value.
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/merge-statistic-does-not-overwrite-database.html b/LayoutTests/http/tests/resourceLoadStatistics/merge-statistic-does-not-overwrite-database.html
new file mode 100644 (file)
index 0000000..4249eec
--- /dev/null
@@ -0,0 +1,79 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <script src="/js-test-resources/js-test.js"></script>
+    <script src="resources/util.js"></script>
+    <script>description("Tests that merged statistic does not overwrite old statistic");</script>
+</head>
+<body onload="setTimeout('setUpAndRun()', 0)">
+<script>
+    jsTestIsAsync = true;
+
+    const url = "http://127.0.0.1:8000";
+    const sampleTopFrameURL = "http://localhost:8000";
+    const olderTimestamp = Math.round((new Date()).getTime() / 1000);
+    const newerTimestamp = olderTimestamp + 10;
+                                      
+    function insertSecondStatistic() {
+      testRunner.setStatisticsMergeStatistic(url, "", olderTimestamp, false, 99, false, false, false, 0, function() {
+          if (testRunner.isStatisticsPrevalentResource(url))
+              testPassed("Host did not overwrite old prevalent resource value.");
+          else
+              testFailed("Host did overwrite old prevalent resource value.")
+          
+          if (testRunner.isStatisticsVeryPrevalentResource(url))
+              testPassed("Host did not overwrite old very prevalent resource value.");
+          else
+              testFailed("Host did overwrite old very prevalent resource value.");
+                                             
+          if (testRunner.isStatisticsHasHadUserInteraction(url))
+              testPassed("Host did not overwrite old had user interaction value");
+          else
+              testFailed("Host did overwrite old had user interaction value");
+
+          if (testRunner.isStatisticsGrandfathered(url))
+              testPassed("Host did not overwrite old grandfathered value.");
+          else
+              testFailed("Host did overwrite old grandfathered value.");
+
+          testRunner.statisticsResetToConsistentState(function() {
+              finishJSTest();
+          });
+      });
+    }
+    
+    function runTestRunnerTest() {
+        testRunner.setStatisticsMergeStatistic(url, "", newerTimestamp, true, 100, true, true, true, 1, function() {
+            if (!testRunner.isStatisticsPrevalentResource(url))
+                testFailed("Host got set as prevalent resource.");
+
+            if (!testRunner.isStatisticsVeryPrevalentResource(url))
+                testFailed("Host got set as prevalent resource.");
+
+            if (!testRunner.isStatisticsHasHadUserInteraction(url))
+                testFailed("Host got logged for user interaction.");
+             
+            if (!testRunner.isStatisticsGrandfathered(url))
+                testFailed("Host got set as grandfathered.");
+
+            insertSecondStatistic();
+        });
+    }
+
+    function setUpAndRun() {
+        if (window.testRunner) {
+            testRunner.setUseITPDatabase(true);
+            setEnableFeature(true, function () {
+                runTestRunnerTest();
+            });
+        } else {
+            testFailed("no testRunner");
+            testRunner.statisticsResetToConsistentState(function() {
+                finishJSTest();
+            });
+        }
+    }
+</script>
+</body>
+</html>
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/merge-statistic-does-overwrite-database-expected.txt b/LayoutTests/http/tests/resourceLoadStatistics/merge-statistic-does-overwrite-database-expected.txt
new file mode 100644 (file)
index 0000000..adcf7e8
--- /dev/null
@@ -0,0 +1,14 @@
+Tests that merged statistic overwrites old statistic
+
+On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
+
+
+PASS Host overwrote old prevalent resource value.
+PASS Host overwrote old very prevalent resource value.
+PASS Host overwrote old user interaction value.
+PASS Host overwrote old grandfathered value.
+PASS Host set as subframe under top frame.
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/merge-statistic-does-overwrite-database.html b/LayoutTests/http/tests/resourceLoadStatistics/merge-statistic-does-overwrite-database.html
new file mode 100644 (file)
index 0000000..1412e15
--- /dev/null
@@ -0,0 +1,90 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <script src="/js-test-resources/js-test.js"></script>
+    <script src="resources/util.js"></script>
+    <script>description("Tests that merged statistic overwrites old statistic");</script>
+</head>
+<body onload="setTimeout('setUpAndRun()', 0)">
+<script>
+    jsTestIsAsync = true;
+
+    const url = "http://127.0.0.1:8000";
+    const sampleTopFrameURL = "http://topFrameDomain:8000";
+    const olderTimestamp = Math.round((new Date()).getTime() / 1000);
+    const newerTimestamp = olderTimestamp + 10;
+    const mostRecentUIToTriggerFirstPartyInteractionCount = newerTimestamp + 90000;
+                                      
+    function insertSecondStatistic() {
+    
+        // set this resource as prevalent so it is in the statistics store
+        testRunner.setStatisticsPrevalentResource(sampleTopFrameURL, true, function() {
+        
+              testRunner.setStatisticsMergeStatistic(url, sampleTopFrameURL, newerTimestamp, true, mostRecentUIToTriggerFirstPartyInteractionCount, true, true, true, 1, function() {
+                  if (testRunner.isStatisticsPrevalentResource(url))
+                      testPassed("Host overwrote old prevalent resource value.");
+                  else
+                      testFailed("Host not set as prevalent resource.");
+
+                  if (testRunner.isStatisticsVeryPrevalentResource(url))
+                      testPassed("Host overwrote old very prevalent resource value.");
+                  else
+                      testFailed("Host not set as very prevalent resource.");
+                                                     
+                  if (testRunner.isStatisticsHasHadUserInteraction(url))
+                      testPassed("Host overwrote old user interaction value.");
+                  else
+                      testFailed("Host not logged for user interaction.");
+                   
+                  if (testRunner.isStatisticsGrandfathered(url))
+                      testPassed("Host overwrote old grandfathered value.");
+                  else
+                      testFailed("Host not set as grandfathered.");
+
+                  if (testRunner.isStatisticsRegisteredAsSubFrameUnder(url, sampleTopFrameURL))
+                      testPassed("Host set as subframe under top frame.");
+                  else
+                      testFailed("Host not set as subframe under top frame.");
+
+                  testRunner.statisticsResetToConsistentState(function() {
+                      finishJSTest();
+                  });
+              });
+         });
+    }
+    
+    function runTestRunnerTest() {
+        testRunner.setStatisticsMergeStatistic(url, "", olderTimestamp, false, 0, false, false, false, 0, function() {
+            if (testRunner.isStatisticsPrevalentResource(url))
+                testFailed("Host got set as prevalent resource.");
+
+            if (testRunner.isStatisticsVeryPrevalentResource(url))
+                testFailed("Host got set as prevalent resource.");
+
+            if (testRunner.isStatisticsHasHadUserInteraction(url))
+                testFailed("Host got logged for user interaction.");
+             
+            if (testRunner.isStatisticsGrandfathered(url))
+                testFailed("Host got set as grandfathered.");
+
+            insertSecondStatistic();
+        });
+    }
+
+    function setUpAndRun() {
+        if (window.testRunner) {
+            testRunner.setUseITPDatabase(true);
+            setEnableFeature(true, function () {
+                runTestRunnerTest();
+            });
+        } else {
+            testFailed("no testRunner");
+            testRunner.statisticsResetToConsistentState(function() {
+                finishJSTest();
+            });
+        }
+    }
+</script>
+</body>
+</html>
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/merge-statistic-does-partially-overwrite-database-expected.txt b/LayoutTests/http/tests/resourceLoadStatistics/merge-statistic-does-partially-overwrite-database-expected.txt
new file mode 100644 (file)
index 0000000..c71e46b
--- /dev/null
@@ -0,0 +1,14 @@
+Tests that merged statistic overwrites old statistic
+
+On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
+
+
+PASS Host overwrote old prevalent resource value.
+PASS Host overwrote old very prevalent resource value.
+PASS Host did not overwrite old user interaction value.
+PASS Host overwrote old grandfathered value.
+PASS Host set as subframe under top frame.
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/merge-statistic-does-partially-overwrite-database.html b/LayoutTests/http/tests/resourceLoadStatistics/merge-statistic-does-partially-overwrite-database.html
new file mode 100644 (file)
index 0000000..44d16d6
--- /dev/null
@@ -0,0 +1,90 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <script src="/js-test-resources/js-test.js"></script>
+    <script src="resources/util.js"></script>
+    <script>description("Tests that merged statistic overwrites old statistic");</script>
+</head>
+<body onload="setTimeout('setUpAndRun()', 0)">
+<script>
+    jsTestIsAsync = true;
+
+    const url = "http://127.0.0.1:8000";
+    const sampleTopFrameURL = "http://topFrameDomain:8000";
+    const olderTimestamp = Math.round((new Date()).getTime() / 1000);
+    const newerTimestamp = olderTimestamp + 10;
+    const mostRecentUIToTriggerFirstPartyInteractionCount = newerTimestamp + 90000;
+                                      
+    function insertSecondStatistic() {
+    
+        // set this resource as prevalent so it is in the statistics store
+        testRunner.setStatisticsPrevalentResource(sampleTopFrameURL, true, function() {
+        
+              testRunner.setStatisticsMergeStatistic(url, sampleTopFrameURL, newerTimestamp, false, mostRecentUIToTriggerFirstPartyInteractionCount, true, true, true, 1, function() {
+                  if (testRunner.isStatisticsPrevalentResource(url))
+                      testPassed("Host overwrote old prevalent resource value.");
+                  else
+                      testFailed("Host not set as prevalent resource.");
+
+                  if (testRunner.isStatisticsVeryPrevalentResource(url))
+                      testPassed("Host overwrote old very prevalent resource value.");
+                  else
+                      testFailed("Host not set as very prevalent resource.");
+                                                     
+                  if (!testRunner.isStatisticsHasHadUserInteraction(url))
+                      testPassed("Host did not overwrite old user interaction value.");
+                  else
+                      testFailed("Host not logged for user interaction.");
+
+                  if (testRunner.isStatisticsGrandfathered(url))
+                      testPassed("Host overwrote old grandfathered value.");
+                  else
+                      testFailed("Host not set as grandfathered.");
+
+                  if (testRunner.isStatisticsRegisteredAsSubFrameUnder(url, sampleTopFrameURL))
+                      testPassed("Host set as subframe under top frame.");
+                  else
+                      testFailed("Host not set as subframe under top frame.");
+
+                  testRunner.statisticsResetToConsistentState(function() {
+                      finishJSTest();
+                  });
+              });
+         });
+    }
+    
+    function runTestRunnerTest() {
+        testRunner.setStatisticsMergeStatistic(url, "", olderTimestamp, false, 0, false, false, false, 0, function() {
+            if (testRunner.isStatisticsPrevalentResource(url))
+                testFailed("Host got set as prevalent resource.");
+
+            if (testRunner.isStatisticsVeryPrevalentResource(url))
+                testFailed("Host got set as prevalent resource.");
+
+            if (testRunner.isStatisticsHasHadUserInteraction(url))
+                testFailed("Host got logged for user interaction.");
+             
+            if (testRunner.isStatisticsGrandfathered(url))
+                testFailed("Host got set as grandfathered.");
+
+            insertSecondStatistic();
+        });
+    }
+
+    function setUpAndRun() {
+        if (window.testRunner) {
+            testRunner.setUseITPDatabase(true);
+            setEnableFeature(true, function () {
+                runTestRunnerTest();
+            });
+        } else {
+            testFailed("no testRunner");
+            testRunner.statisticsResetToConsistentState(function() {
+                finishJSTest();
+            });
+        }
+    }
+</script>
+</body>
+</html>
index c5aa1ee..3e1280f 100644 (file)
@@ -1,3 +1,134 @@
+2019-10-02  Kate Cheney  <katherine_cheney@apple.com>
+
+        Updated resource load statistics are never merged into the SQLite Database backend (202372).
+        https://bugs.webkit.org/show_bug.cgi?id=202372
+        <rdar://problem/55854542>
+
+        Reviewed by Brent Fulgham. 
+
+        This patch has a lot of changes to the test infrastructure to be able 
+        to test the mergeStatistics function. Merging functionality mimics
+        that of the ResourceLoadStatisticsMemoryStore and the merge() function in
+        ResourceLoadStatistics.cpp.
+
+        * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:
+        (WebKit::ResourceLoadStatisticsDatabaseStore::ResourceLoadStatisticsDatabaseStore):
+        Added new SQLite queries that are needed to merge new statistics.
+
+        (WebKit::ResourceLoadStatisticsDatabaseStore::prepareStatements):
+        Added new prepare statements for new SQLite queries.
+
+        (WebKit::ResourceLoadStatisticsDatabaseStore::insertObservedDomain):
+        Updated to use domainID() instead of confirmDomainDoesNotExist which 
+        was deleted. Changed the insert bind parameters
+        to utilize new enum of Observed Domain table indices which will help
+        reduce errors in the future if the database schema changes.
+
+        (WebKit::ResourceLoadStatisticsDatabaseStore::relationshipExists const):
+        Changed {public} to {private} in logging statement to avoid leaking
+        sensitive information when logging the query error.
+
+        (WebKit::ResourceLoadStatisticsDatabaseStore::confirmDomainDoesNotExist const): Deleted.
+        This function essentially does the exact same thing as domainID and
+        we can reduce code duplication by deleting it and transitioning all
+        other functions to use domainID instead.
+
+        (WebKit::ResourceLoadStatisticsDatabaseStore::domainID const):
+        Changed {public} to {private} in logging statement to avoid leaking
+        sensitive information when logging the query error.
+        
+        (WebKit::ResourceLoadStatisticsDatabaseStore::insertDomainRelationships):
+        In order to reuse this code for the merge statistics function, I added
+        a check before each insert call to ensure the relationship does not
+        already exist in the database. This was not needed before because it
+        was only called on an empty database.
+
+        (WebKit::ResourceLoadStatisticsDatabaseStore::merge):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::mergeStatistic):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::mergeStatistics):
+        Merges new statistic.
+
+        (WebKit::ResourceLoadStatisticsDatabaseStore::incrementRecordsDeletedCountForDomains):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::recursivelyFindNonPrevalentDomainsThatRedirectedToThisDomain):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::markAsPrevalentIfHasRedirectedToPrevalent):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::requestStorageAccess):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::requestStorageAccessUnderOpener):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::grandfatherDataForDomains):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::ensurePrevalentResourcesForDebugMode):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::setUserInteraction):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::clearUserInteraction):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::hasHadUserInteraction):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::setPrevalentResource):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::setDomainsAsPrevalent):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::predicateValueForDomain const):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::clearPrevalentResource):
+        Changed {public} to {private} in logging statement to avoid leaking
+        sensitive information when logging the query error.
+
+        (WebKit::ResourceLoadStatisticsDatabaseStore::setGrandfathered):
+        Fix a bug uncovered by the new test cases.
+        
+        (WebKit::ResourceLoadStatisticsDatabaseStore::ensureResourceStatisticsForRegistrableDomain):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::clearDatabaseContents):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::cookieTreatmentForOrigin const):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::clearExpiredUserInteractions):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::clearGrandfathering):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::pruneStatisticsIfNeeded):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::updateLastSeen):
+        Changed {public} to {private} in logging statement to avoid leaking
+        sensitive information when logging the query error.
+        
+        (WebKit::ResourceLoadStatisticsDatabaseStore::updateDataRecordsRemoved):
+        A new function that utilizes a query needed for the merging of two
+        statistics.
+        
+        (WebKit::ResourceLoadStatisticsDatabaseStore::confirmDomainDoesNotExist const): Deleted.
+        * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h:
+        
+        * NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp:
+        (WebKit::ResourceLoadStatisticsMemoryStore::mergeStatistics):
+        (WebKit::ResourceLoadStatisticsMemoryStore::wasAccessedAsFirstPartyDueToUserInteraction const): Deleted.
+        Removed the check for updating times accessed due to first party 
+        interaction, which is data no longer needed in updated ITP. 
+
+        * NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h:
+        * NetworkProcess/Classifier/ResourceLoadStatisticsStore.h:
+        Added the mergeStatistics function (previously only in the memory
+        store) to the parent class now that it is used by both stores.
+        
+        * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:
+        (WebKit::WebResourceLoadStatisticsStore::resourceLoadStatisticsUpdated):
+        This function now calls mergeStatistics on the database store if
+        enabled.
+
+        (WebKit::WebResourceLoadStatisticsStore::mergeStatisticForTesting):
+        This function builds a vector from the sample data to test the
+        mergeStatistics() function of the database store.
+
+        * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h:
+        * NetworkProcess/NetworkProcess.cpp:
+        (WebKit::NetworkProcess::mergeStatisticForTesting):
+        * NetworkProcess/NetworkProcess.h:
+        * NetworkProcess/NetworkProcess.messages.in:
+        * UIProcess/API/C/WKWebsiteDataStoreRef.cpp:
+        (WKWebsiteDataStoreSetStatisticsMergeStatistic):
+        Fixed bug where isGrandfathered function in WKWebsiteDataStoreRef
+        was calling hasHadUserInteraction by mistake.
+
+        * UIProcess/API/C/WKWebsiteDataStoreRef.h:
+        * UIProcess/Network/NetworkProcessProxy.cpp:
+        (WebKit::NetworkProcessProxy::mergeStatisticForTesting):
+        * UIProcess/Network/NetworkProcessProxy.h:
+        This code is for testing the mergeStatistics() function.
+
+        * UIProcess/WebsiteData/WebsiteDataStore.cpp:
+        (WebKit::WebsiteDataStore::isGrandfathered):
+        (WebKit::WebsiteDataStore::mergeStatisticForTesting):
+        * UIProcess/WebsiteData/WebsiteDataStore.h:
+        Added this function which contacts the networkProcess to retrieve the
+        grandfathered value for a domain. This was not included before because
+        of a bug in WKWebsiteDataStoreRef.
+
 2019-10-02  Dean Jackson  <dino@apple.com>
 
         Provide originating website URL to AR QuickLook
index 2c304e7..88fdb7e 100644 (file)
@@ -70,6 +70,8 @@ constexpr auto insertTopLevelDomainQuery = "INSERT INTO TopLevelDomains VALUES (
 constexpr auto domainIDFromStringQuery = "SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?"_s;
 constexpr auto storageAccessUnderTopFrameDomainsQuery = "INSERT INTO StorageAccessUnderTopFrameDomains (domainID, topLevelDomainID) "
     "SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain == ?"_s;
+constexpr auto storageAccessUnderTopFrameDomainsExistsQuery = "SELECT EXISTS (SELECT 1 FROM StorageAccessUnderTopFrameDomains WHERE domainID = ? "
+"AND topLevelDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
 constexpr auto topFrameUniqueRedirectsToQuery = "INSERT INTO TopFrameUniqueRedirectsTo (sourceDomainID, toDomainID) SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain == ?"_s;
 constexpr auto topFrameUniqueRedirectsToExistsQuery = "SELECT EXISTS (SELECT 1 FROM TopFrameUniqueRedirectsTo WHERE sourceDomainID = ? "
     "AND toDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
@@ -100,6 +102,7 @@ constexpr auto subresourceUniqueRedirectsFromExistsQuery = "SELECT EXISTS (SELEC
 constexpr auto mostRecentUserInteractionQuery = "UPDATE ObservedDomains SET hadUserInteraction = ?, mostRecentUserInteractionTime = ? "
     "WHERE registrableDomain = ?"_s;
 constexpr auto updateLastSeenQuery = "UPDATE ObservedDomains SET lastSeen = ? WHERE registrableDomain = ?"_s;
+constexpr auto updateDataRecordsRemovedQuery = "UPDATE ObservedDomains SET dataRecordsRemoved = ? WHERE registrableDomain = ?"_s;
 constexpr auto updatePrevalentResourceQuery = "UPDATE ObservedDomains SET isPrevalent = ? WHERE registrableDomain = ?"_s;
 constexpr auto isPrevalentResourceQuery = "SELECT isPrevalent FROM ObservedDomains WHERE registrableDomain = ?"_s;
 constexpr auto updateVeryPrevalentResourceQuery = "UPDATE ObservedDomains SET isVeryPrevalent = ? WHERE registrableDomain = ?"_s;
@@ -115,7 +118,21 @@ constexpr auto createObservedDomain = "CREATE TABLE ObservedDomains ("
     "hadUserInteraction INTEGER NOT NULL, mostRecentUserInteractionTime REAL NOT NULL, grandfathered INTEGER NOT NULL, "
     "isPrevalent INTEGER NOT NULL, isVeryPrevalent INTEGER NOT NULL, dataRecordsRemoved INTEGER NOT NULL,"
     "timesAccessedAsFirstPartyDueToUserInteraction INTEGER NOT NULL, timesAccessedAsFirstPartyDueToStorageAccessAPI INTEGER NOT NULL);"_s;
-    
+
+enum {
+    DomainIDIndex,
+    RegistrableDomainIndex,
+    LastSeenIndex,
+    HadUserInteractionIndex,
+    MostRecentUserInteractionTimeIndex,
+    GrandfatheredIndex,
+    IsPrevalentIndex,
+    IsVeryPrevalentIndex,
+    DataRecordsRemovedIndex,
+    TimesAccessedAsFirstPartyDueToUserInteractionIndex,
+    TimesAccessedAsFirstPartyDueToStorageAccessAPIIndex
+};
+
 constexpr auto createTopLevelDomains = "CREATE TABLE TopLevelDomains ("
     "topLevelDomainID INTEGER PRIMARY KEY, CONSTRAINT fkDomainID FOREIGN KEY(topLevelDomainID) "
     "REFERENCES ObservedDomains(domainID) ON DELETE CASCADE);"_s;
@@ -168,6 +185,7 @@ ResourceLoadStatisticsDatabaseStore::ResourceLoadStatisticsDatabaseStore(WebReso
     , m_insertTopLevelDomainStatement(m_database, insertTopLevelDomainQuery)
     , m_domainIDFromStringStatement(m_database, domainIDFromStringQuery)
     , m_storageAccessUnderTopFrameDomainsStatement(m_database, storageAccessUnderTopFrameDomainsQuery)
+    , m_storageAccessUnderTopFrameDomainsExistsStatement(m_database, storageAccessUnderTopFrameDomainsExistsQuery)
     , m_topFrameUniqueRedirectsTo(m_database, topFrameUniqueRedirectsToQuery)
     , m_topFrameUniqueRedirectsToExists(m_database, topFrameUniqueRedirectsToExistsQuery)
     , m_topFrameUniqueRedirectsFrom(m_database, topFrameUniqueRedirectsFromQuery)
@@ -184,6 +202,7 @@ ResourceLoadStatisticsDatabaseStore::ResourceLoadStatisticsDatabaseStore(WebReso
     , m_subresourceUniqueRedirectsFromExists(m_database, subresourceUniqueRedirectsFromExistsQuery)
     , m_mostRecentUserInteractionStatement(m_database, mostRecentUserInteractionQuery)
     , m_updateLastSeenStatement(m_database, updateLastSeenQuery)
+    , m_updateDataRecordsRemovedStatement(m_database, updateDataRecordsRemovedQuery)
     , m_updatePrevalentResourceStatement(m_database, updatePrevalentResourceQuery)
     , m_isPrevalentResourceStatement(m_database, isPrevalentResourceQuery)
     , m_updateVeryPrevalentResourceStatement(m_database, updateVeryPrevalentResourceQuery)
@@ -309,6 +328,7 @@ bool ResourceLoadStatisticsDatabaseStore::prepareStatements()
         || m_insertTopLevelDomainStatement.prepare() != SQLITE_OK
         || m_domainIDFromStringStatement.prepare() != SQLITE_OK
         || m_storageAccessUnderTopFrameDomainsStatement.prepare() != SQLITE_OK
+        || m_storageAccessUnderTopFrameDomainsExistsStatement.prepare() != SQLITE_OK
         || m_topFrameUniqueRedirectsTo.prepare() != SQLITE_OK
         || m_topFrameUniqueRedirectsToExists.prepare() != SQLITE_OK
         || m_topFrameUniqueRedirectsFrom.prepare() != SQLITE_OK
@@ -322,6 +342,7 @@ bool ResourceLoadStatisticsDatabaseStore::prepareStatements()
         || m_subresourceUniqueRedirectsFrom.prepare() != SQLITE_OK
         || m_subresourceUniqueRedirectsFromExists.prepare() != SQLITE_OK
         || m_updateLastSeenStatement.prepare() != SQLITE_OK
+        || m_updateDataRecordsRemovedStatement.prepare() != SQLITE_OK
         || m_mostRecentUserInteractionStatement.prepare() != SQLITE_OK
         || m_updatePrevalentResourceStatement.prepare() != SQLITE_OK
         || m_isPrevalentResourceStatement.prepare() != SQLITE_OK
@@ -346,27 +367,29 @@ bool ResourceLoadStatisticsDatabaseStore::insertObservedDomain(const ResourceLoa
 {
     ASSERT(!RunLoop::isMain());
 
-#ifndef NDEBUG
-    ASSERT(confirmDomainDoesNotExist(loadStatistics.registrableDomain));
-#endif
+    if (domainID(loadStatistics.registrableDomain)) {
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "ResourceLoadStatisticsDatabaseStore::insertObservedDomain can only be called on domains not in the database.");
+        ASSERT_NOT_REACHED();
+        return false;
+    }
 
-    if (m_insertObservedDomainStatement.bindText(1, loadStatistics.registrableDomain.string()) != SQLITE_OK
-        || m_insertObservedDomainStatement.bindDouble(2, loadStatistics.lastSeen.secondsSinceEpoch().value()) != SQLITE_OK
-        || m_insertObservedDomainStatement.bindInt(3, loadStatistics.hadUserInteraction) != SQLITE_OK
-        || m_insertObservedDomainStatement.bindDouble(4, loadStatistics.mostRecentUserInteractionTime.secondsSinceEpoch().value()) != SQLITE_OK
-        || m_insertObservedDomainStatement.bindInt(5, loadStatistics.grandfathered) != SQLITE_OK
-        || m_insertObservedDomainStatement.bindInt(6, loadStatistics.isPrevalentResource) != SQLITE_OK
-        || m_insertObservedDomainStatement.bindInt(7, loadStatistics.isVeryPrevalentResource) != SQLITE_OK
-        || m_insertObservedDomainStatement.bindInt(8, loadStatistics.dataRecordsRemoved) != SQLITE_OK
-        || m_insertObservedDomainStatement.bindInt(9, loadStatistics.timesAccessedAsFirstPartyDueToUserInteraction) != SQLITE_OK
-        || m_insertObservedDomainStatement.bindInt(10, loadStatistics.timesAccessedAsFirstPartyDueToStorageAccessAPI) != SQLITE_OK) {
-        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::insertObservedDomain failed to bind, error message: %{public}s", this, m_database.lastErrorMsg());
+    if (m_insertObservedDomainStatement.bindText(RegistrableDomainIndex, loadStatistics.registrableDomain.string()) != SQLITE_OK
+        || m_insertObservedDomainStatement.bindDouble(LastSeenIndex, loadStatistics.lastSeen.secondsSinceEpoch().value()) != SQLITE_OK
+        || m_insertObservedDomainStatement.bindInt(HadUserInteractionIndex, loadStatistics.hadUserInteraction) != SQLITE_OK
+        || m_insertObservedDomainStatement.bindDouble(MostRecentUserInteractionTimeIndex, loadStatistics.mostRecentUserInteractionTime.secondsSinceEpoch().value()) != SQLITE_OK
+        || m_insertObservedDomainStatement.bindInt(GrandfatheredIndex, loadStatistics.grandfathered) != SQLITE_OK
+        || m_insertObservedDomainStatement.bindInt(IsPrevalentIndex, loadStatistics.isPrevalentResource) != SQLITE_OK
+        || m_insertObservedDomainStatement.bindInt(IsVeryPrevalentIndex, loadStatistics.isVeryPrevalentResource) != SQLITE_OK
+        || m_insertObservedDomainStatement.bindInt(DataRecordsRemovedIndex, loadStatistics.dataRecordsRemoved) != SQLITE_OK
+        || m_insertObservedDomainStatement.bindInt(TimesAccessedAsFirstPartyDueToUserInteractionIndex, loadStatistics.timesAccessedAsFirstPartyDueToUserInteraction) != SQLITE_OK
+        || m_insertObservedDomainStatement.bindInt(TimesAccessedAsFirstPartyDueToStorageAccessAPIIndex, loadStatistics.timesAccessedAsFirstPartyDueToStorageAccessAPI) != SQLITE_OK) {
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::insertObservedDomain failed to bind, error message: %{private}s", this, m_database.lastErrorMsg());
         ASSERT_NOT_REACHED();
         return false;
     }
 
     if (m_insertObservedDomainStatement.step() != SQLITE_DONE) {
-        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::insertObservedDomain failed to commit, error message: %{public}s", this, m_database.lastErrorMsg());
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::insertObservedDomain failed to commit, error message: %{private}s", this, m_database.lastErrorMsg());
         ASSERT_NOT_REACHED();
         return false;
     }
@@ -387,7 +410,7 @@ bool ResourceLoadStatisticsDatabaseStore::relationshipExists(WebCore::SQLiteStat
     if (statement.bindInt(1, *firstDomainID) != SQLITE_OK
         || statement.bindText(2, secondDomain.string()) != SQLITE_OK
         || statement.step() != SQLITE_ROW) {
-        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::m_insertDomainRelationshipStatement failed to bind, error message: %{public}s", this, m_database.lastErrorMsg());
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::m_insertDomainRelationshipStatement failed to bind, error message: %{private}s", this, m_database.lastErrorMsg());
         ASSERT_NOT_REACHED();
         return false;
     }
@@ -406,7 +429,7 @@ bool ResourceLoadStatisticsDatabaseStore::insertDomainRelationship(WebCore::SQLi
     if (statement.bindInt(1, domainID) != SQLITE_OK
         || statement.bindText(2, topFrame.string()) != SQLITE_OK
         || statement.step() != SQLITE_DONE) {
-        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::m_insertDomainRelationshipStatement failed to bind, error message: %{public}s", this, m_database.lastErrorMsg());
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::m_insertDomainRelationshipStatement failed to bind, error message: %{private}s", this, m_database.lastErrorMsg());
         ASSERT_NOT_REACHED();
         return false;
     }
@@ -417,30 +440,6 @@ bool ResourceLoadStatisticsDatabaseStore::insertDomainRelationship(WebCore::SQLi
     return true;
 }
 
-#ifndef NDEBUG
-bool ResourceLoadStatisticsDatabaseStore::confirmDomainDoesNotExist(const RegistrableDomain& domain) const
-{
-    ASSERT(!RunLoop::isMain());
-
-    if (m_domainIDFromStringStatement.bindText(1, domain.string()) != SQLITE_OK) {
-        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::confirmDomainDoesNotExist failed to bind, error message: %{public}s", this, m_database.lastErrorMsg());
-        ASSERT_NOT_REACHED();
-        return false;
-    }
-        
-    if (m_domainIDFromStringStatement.step() == SQLITE_ROW) {
-        int resetResult = m_domainIDFromStringStatement.reset();
-        ASSERT_UNUSED(resetResult, resetResult == SQLITE_OK);
-        return false;
-    }
-
-    int resetResult = m_domainIDFromStringStatement.reset();
-    ASSERT_UNUSED(resetResult, resetResult == SQLITE_OK);
-    
-    return true;
-}
-#endif
-
 Optional<unsigned> ResourceLoadStatisticsDatabaseStore::domainID(const RegistrableDomain& domain) const
 {
     ASSERT(!RunLoop::isMain());
@@ -448,7 +447,7 @@ Optional<unsigned> ResourceLoadStatisticsDatabaseStore::domainID(const Registrab
     unsigned domainID = 0;
 
     if (m_domainIDFromStringStatement.bindText(1, domain.string()) != SQLITE_OK) {
-        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::domainIDFromString failed, error message: %{public}s", this, m_database.lastErrorMsg());
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::domainIDFromString failed, error message: %{private}s", this, m_database.lastErrorMsg());
         return WTF::nullopt;
     }
     
@@ -474,26 +473,40 @@ void ResourceLoadStatisticsDatabaseStore::insertDomainRelationships(const Resour
     if (!registrableDomainID)
         return;
     
-    for (auto& topFrameDomain : loadStatistics.storageAccessUnderTopFrameDomains)
-        insertDomainRelationship(m_storageAccessUnderTopFrameDomainsStatement, registrableDomainID.value(), topFrameDomain);
+    for (auto& topFrameDomain : loadStatistics.storageAccessUnderTopFrameDomains) {
+        if (!relationshipExists(m_storageAccessUnderTopFrameDomainsExistsStatement, registrableDomainID.value(), topFrameDomain))
+            insertDomainRelationship(m_storageAccessUnderTopFrameDomainsStatement, registrableDomainID.value(), topFrameDomain);
+    }
 
-    for (auto& toDomain : loadStatistics.topFrameUniqueRedirectsTo)
-        insertDomainRelationship(m_topFrameUniqueRedirectsTo, registrableDomainID.value(), toDomain);
+    for (auto& toDomain : loadStatistics.topFrameUniqueRedirectsTo) {
+        if (!relationshipExists(m_topFrameUniqueRedirectsToExists, registrableDomainID.value(), toDomain))
+            insertDomainRelationship(m_topFrameUniqueRedirectsTo, registrableDomainID.value(), toDomain);
+    }
     
-    for (auto& fromDomain : loadStatistics.topFrameUniqueRedirectsFrom)
-        insertDomainRelationship(m_topFrameUniqueRedirectsFrom, registrableDomainID.value(), fromDomain);
+    for (auto& fromDomain : loadStatistics.topFrameUniqueRedirectsFrom) {
+        if (!relationshipExists(m_topFrameUniqueRedirectsFromExists, registrableDomainID.value(), fromDomain))
+            insertDomainRelationship(m_topFrameUniqueRedirectsFrom, registrableDomainID.value(), fromDomain);
+    }
     
-    for (auto& topFrameDomain : loadStatistics.subframeUnderTopFrameDomains)
-        insertDomainRelationship(m_subframeUnderTopFrameDomains, registrableDomainID.value(), topFrameDomain);
+    for (auto& topFrameDomain : loadStatistics.subframeUnderTopFrameDomains) {
+        if (!relationshipExists(m_subframeUnderTopFrameDomainExists, registrableDomainID.value(), topFrameDomain))
+            insertDomainRelationship(m_subframeUnderTopFrameDomains, registrableDomainID.value(), topFrameDomain);
+    }
     
-    for (auto& topFrameDomain : loadStatistics.subresourceUnderTopFrameDomains)
-        insertDomainRelationship(m_subresourceUnderTopFrameDomains, registrableDomainID.value(), topFrameDomain);
+    for (auto& topFrameDomain : loadStatistics.subresourceUnderTopFrameDomains) {
+        if (!relationshipExists(m_subresourceUnderTopFrameDomainExists, registrableDomainID.value(), topFrameDomain))
+            insertDomainRelationship(m_subresourceUnderTopFrameDomains, registrableDomainID.value(), topFrameDomain);
+    }
     
-    for (auto& toDomain : loadStatistics.subresourceUniqueRedirectsTo)
-        insertDomainRelationship(m_subresourceUniqueRedirectsTo, registrableDomainID.value(), toDomain);
+    for (auto& toDomain : loadStatistics.subresourceUniqueRedirectsTo) {
+        if (!relationshipExists(m_subresourceUniqueRedirectsToExists, registrableDomainID.value(), toDomain))
+            insertDomainRelationship(m_subresourceUniqueRedirectsTo, registrableDomainID.value(), toDomain);
+    }
     
-    for (auto& fromDomain : loadStatistics.subresourceUniqueRedirectsFrom)
-        insertDomainRelationship(m_subresourceUniqueRedirectsFrom, registrableDomainID.value(), fromDomain);
+    for (auto& fromDomain : loadStatistics.subresourceUniqueRedirectsFrom) {
+        if (!relationshipExists(m_subresourceUniqueRedirectsFromExists, registrableDomainID.value(), fromDomain))
+            insertDomainRelationship(m_subresourceUniqueRedirectsFrom, registrableDomainID.value(), fromDomain);
+    }
 }
 
 void ResourceLoadStatisticsDatabaseStore::populateFromMemoryStore(const ResourceLoadStatisticsMemoryStore& memoryStore)
@@ -513,6 +526,73 @@ void ResourceLoadStatisticsDatabaseStore::populateFromMemoryStore(const Resource
         insertDomainRelationships(statistic.value);
 }
 
+void ResourceLoadStatisticsDatabaseStore::merge(WebCore::SQLiteStatement& current, const ResourceLoadStatistics& other)
+{
+    ASSERT(!RunLoop::isMain());
+
+    auto currentRegistrableDomain = current.getColumnText(RegistrableDomainIndex);
+    auto currentLastSeen = current.getColumnDouble(LastSeenIndex);
+    auto currentMostRecentUserInteraction = current.getColumnDouble(MostRecentUserInteractionTimeIndex);
+    bool currentGrandfathered = current.getColumnInt(GrandfatheredIndex);
+    bool currentIsPrevalent = current.getColumnInt(IsPrevalentIndex);
+    bool currentIsVeryPrevalent = current.getColumnInt(IsVeryPrevalentIndex);
+    unsigned currentDataRecordsRemoved = current.getColumnInt(DataRecordsRemovedIndex);
+
+    ASSERT(currentRegistrableDomain == other.registrableDomain.string());
+
+    if (WallTime::fromRawSeconds(currentLastSeen) < other.lastSeen)
+        updateLastSeen(other.registrableDomain, other.lastSeen);
+
+    if (!other.hadUserInteraction) {
+        // If most recent user interaction time has been reset do so here too.
+        if (!other.mostRecentUserInteractionTime)
+            setUserInteraction(other.registrableDomain, false, { });
+    } else
+        setUserInteraction(other.registrableDomain, true, std::max(WallTime::fromRawSeconds(currentMostRecentUserInteraction), other.mostRecentUserInteractionTime));
+
+    if (other.grandfathered && !currentGrandfathered)
+        setGrandfathered(other.registrableDomain, true);
+    if (other.isPrevalentResource && !currentIsPrevalent)
+        setPrevalentResource(other.registrableDomain);
+    if (other.isVeryPrevalentResource && !currentIsVeryPrevalent)
+        setVeryPrevalentResource(other.registrableDomain);
+    if (other.dataRecordsRemoved > currentDataRecordsRemoved)
+        updateDataRecordsRemoved(other.registrableDomain, other.dataRecordsRemoved);
+}
+
+void ResourceLoadStatisticsDatabaseStore::mergeStatistic(const ResourceLoadStatistics& statistic)
+{
+    ASSERT(!RunLoop::isMain());
+
+    SQLiteStatement fetchOldStatisticData(m_database, "SELECT * FROM ObservedDomains where registrableDomain = ?");
+    if (fetchOldStatisticData.prepare() != SQLITE_OK
+        || fetchOldStatisticData.bindText(1, statistic.registrableDomain.string()) != SQLITE_OK
+        || fetchOldStatisticData.step() != SQLITE_ROW) {
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::mergeStatistic. Statement failed to bind or domain was not found, error message: %{private}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+        return;
+    }
+
+    merge(fetchOldStatisticData, statistic);
+}
+
+void ResourceLoadStatisticsDatabaseStore::mergeStatistics(Vector<ResourceLoadStatistics>&& statistics)
+{
+    ASSERT(!RunLoop::isMain());
+
+    for (auto& statistic : statistics) {
+        if (!domainID(statistic.registrableDomain))
+            insertObservedDomain(statistic);
+        else
+            mergeStatistic(statistic);
+    }
+
+    // Make a separate pass for inter-domain relationships so we
+    // can refer to the ObservedDomain table entries.
+    for (auto& statistic : statistics)
+        insertDomainRelationships(statistic);
+}
+
 void ResourceLoadStatisticsDatabaseStore::calculateAndSubmitTelemetry() const
 {
     ASSERT(!RunLoop::isMain());
@@ -541,7 +621,7 @@ void ResourceLoadStatisticsDatabaseStore::incrementRecordsDeletedCountForDomains
     SQLiteStatement domainsToUpdateStatement(m_database, makeString("UPDATE ObservedDomains SET dataRecordsRemoved = dataRecordsRemoved + 1 WHERE registrableDomain IN (", domainsToString(domains), ")"));
     if (domainsToUpdateStatement.prepare() != SQLITE_OK
         || domainsToUpdateStatement.step() != SQLITE_DONE) {
-        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::incrementStatisticsForDomains failed, error message: %{public}s", this, m_database.lastErrorMsg());
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::incrementStatisticsForDomains failed, error message: %{private}s", this, m_database.lastErrorMsg());
         ASSERT_NOT_REACHED();
     }
 }
@@ -561,7 +641,7 @@ unsigned ResourceLoadStatisticsDatabaseStore::recursivelyFindNonPrevalentDomains
     SQLiteStatement findSubresources(m_database, "SELECT SubresourceUniqueRedirectsFrom.fromDomainID from SubresourceUniqueRedirectsFrom INNER JOIN ObservedDomains ON ObservedDomains.domainID = SubresourceUniqueRedirectsFrom.fromDomainID WHERE subresourceDomainID = ? AND ObservedDomains.isPrevalent = 0"_s);
     if (findSubresources.prepare() != SQLITE_OK
         || findSubresources.bindInt(1, primaryDomainID) != SQLITE_OK) {
-        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::recursivelyFindNonPrevalentDomainsThatRedirectedToThisDomain failed, error message: %{public}s", this, m_database.lastErrorMsg());
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::recursivelyFindNonPrevalentDomainsThatRedirectedToThisDomain failed, error message: %{private}s", this, m_database.lastErrorMsg());
         ASSERT_NOT_REACHED();
         return 0;
     }
@@ -576,7 +656,7 @@ unsigned ResourceLoadStatisticsDatabaseStore::recursivelyFindNonPrevalentDomains
     SQLiteStatement findTopFrames(m_database, "SELECT TopFrameUniqueRedirectsFrom.fromDomainID from TopFrameUniqueRedirectsFrom INNER JOIN ObservedDomains ON ObservedDomains.domainID = TopFrameUniqueRedirectsFrom.fromDomainID WHERE targetDomainID = ? AND ObservedDomains.isPrevalent = 0"_s);
     if (findTopFrames.prepare() != SQLITE_OK
         || findTopFrames.bindInt(1, primaryDomainID) != SQLITE_OK) {
-        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::recursivelyFindNonPrevalentDomainsThatRedirectedToThisDomain failed, error message: %{public}s", this, m_database.lastErrorMsg());
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::recursivelyFindNonPrevalentDomainsThatRedirectedToThisDomain failed, error message: %{private}s", this, m_database.lastErrorMsg());
         ASSERT_NOT_REACHED();
         return 0;
     }
@@ -629,7 +709,7 @@ void ResourceLoadStatisticsDatabaseStore::markAsPrevalentIfHasRedirectedToPreval
     SQLiteStatement markPrevalentStatement(m_database, makeString("UPDATE ObservedDomains SET isPrevalent = 1 WHERE domainID IN (", buildList(WTF::IteratorRange<StdSet<unsigned>::iterator>(prevalentDueToRedirect.begin(), prevalentDueToRedirect.end())), ")"));
     if (markPrevalentStatement.prepare() != SQLITE_OK
         || markPrevalentStatement.step() != SQLITE_DONE) {
-        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::markAsPrevalentIfHasRedirectedToPrevalent failed to execute, error message: %{public}s", this, m_database.lastErrorMsg());
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::markAsPrevalentIfHasRedirectedToPrevalent failed to execute, error message: %{private}s", this, m_database.lastErrorMsg());
         ASSERT_NOT_REACHED();
     }
 }
@@ -774,12 +854,12 @@ void ResourceLoadStatisticsDatabaseStore::requestStorageAccess(SubFrameDomain&&
 
     switch (cookieTreatmentForOrigin(subFrameDomain)) {
     case CookieTreatmentResult::BlockAndPurge: {
-        RELEASE_LOG_INFO_IF(debugLoggingEnabled(), ITPDebug, "Cannot grant storage access to %{public}s since its cookies are blocked in third-party contexts and it has not received user interaction as first-party.", subFrameDomain.string().utf8().data());
+        RELEASE_LOG_INFO_IF(debugLoggingEnabled(), ITPDebug, "Cannot grant storage access to %{private}s since its cookies are blocked in third-party contexts and it has not received user interaction as first-party.", subFrameDomain.string().utf8().data());
         completionHandler(StorageAccessStatus::CannotRequestAccess);
         }
         return;
     case CookieTreatmentResult::BlockAndKeep: {
-        RELEASE_LOG_INFO_IF(debugLoggingEnabled(), ITPDebug, "No need to grant storage access to %{public}s since its cookies are not blocked in third-party contexts.", subFrameDomain.string().utf8().data());
+        RELEASE_LOG_INFO_IF(debugLoggingEnabled(), ITPDebug, "No need to grant storage access to %{private}s since its cookies are not blocked in third-party contexts.", subFrameDomain.string().utf8().data());
         completionHandler(StorageAccessStatus::HasAccess);
         }
         return;
@@ -790,19 +870,19 @@ void ResourceLoadStatisticsDatabaseStore::requestStorageAccess(SubFrameDomain&&
 
     auto userWasPromptedEarlier = hasUserGrantedStorageAccessThroughPrompt(subFrameStatus.second, topFrameDomain);
     if (userWasPromptedEarlier == StorageAccessPromptWasShown::No) {
-        RELEASE_LOG_INFO_IF(debugLoggingEnabled(), ITPDebug, "About to ask the user whether they want to grant storage access to %{public}s under %{public}s or not.", subFrameDomain.string().utf8().data(), topFrameDomain.string().utf8().data());
+        RELEASE_LOG_INFO_IF(debugLoggingEnabled(), ITPDebug, "About to ask the user whether they want to grant storage access to %{private}s under %{private}s or not.", subFrameDomain.string().utf8().data(), topFrameDomain.string().utf8().data());
         completionHandler(StorageAccessStatus::RequiresUserPrompt);
         return;
     }
 
     if (userWasPromptedEarlier == StorageAccessPromptWasShown::Yes)
-        RELEASE_LOG_INFO_IF(debugLoggingEnabled(), ITPDebug, "Storage access was granted to %{public}s under %{public}s.", subFrameDomain.string().utf8().data(), topFrameDomain.string().utf8().data());
+        RELEASE_LOG_INFO_IF(debugLoggingEnabled(), ITPDebug, "Storage access was granted to %{private}s under %{private}s.", subFrameDomain.string().utf8().data(), topFrameDomain.string().utf8().data());
 
     SQLiteStatement incrementStorageAccess(m_database, "UPDATE ObservedDomains SET timesAccessedAsFirstPartyDueToStorageAccessAPI = timesAccessedAsFirstPartyDueToStorageAccessAPI + 1 WHERE domainID = ?");
     if (incrementStorageAccess.prepare() != SQLITE_OK
         || incrementStorageAccess.bindInt(1, subFrameStatus.second) != SQLITE_OK
         || incrementStorageAccess.step() != SQLITE_DONE) {
-        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::requestStorageAccess failed, error message: %{public}s", this, m_database.lastErrorMsg());
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::requestStorageAccess failed, error message: %{private}s", this, m_database.lastErrorMsg());
         ASSERT_NOT_REACHED();
         return;
     }
@@ -825,7 +905,7 @@ void ResourceLoadStatisticsDatabaseStore::requestStorageAccessUnderOpener(Domain
     if (cookieTreatmentForOrigin(domainInNeedOfStorageAccess) != CookieTreatmentResult::Allow)
         return;
 
-    RELEASE_LOG_INFO_IF(debugLoggingEnabled(), ITPDebug, "[Temporary combatibility fix] Storage access was granted for %{public}s under opener page from %{public}s, with user interaction in the opened window.", domainInNeedOfStorageAccess.string().utf8().data(), openerDomain.string().utf8().data());
+    RELEASE_LOG_INFO_IF(debugLoggingEnabled(), ITPDebug, "[Temporary combatibility fix] Storage access was granted for %{private}s under opener page from %{private}s, with user interaction in the opened window.", domainInNeedOfStorageAccess.string().utf8().data(), openerDomain.string().utf8().data());
     grantStorageAccessInternal(WTFMove(domainInNeedOfStorageAccess), WTFMove(openerDomain), WTF::nullopt, openerPageID, StorageAccessPromptWasShown::No, [](StorageAccessWasGranted) { });
 }
 
@@ -882,7 +962,7 @@ void ResourceLoadStatisticsDatabaseStore::grandfatherDataForDomains(const HashSe
     SQLiteStatement domainsToUpdateStatement(m_database, makeString("UPDATE ObservedDomains SET grandfathered = 1 WHERE registrableDomain IN (", domainsToString(domains), ")"));
     if (domainsToUpdateStatement.prepare() != SQLITE_OK
         || domainsToUpdateStatement.step() != SQLITE_DONE) {
-        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::grandfatherDataForDomains failed, error message: %{public}s", this, m_database.lastErrorMsg());
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::grandfatherDataForDomains failed, error message: %{private}s", this, m_database.lastErrorMsg());
         ASSERT_NOT_REACHED();
     }
 }
@@ -905,7 +985,7 @@ Vector<RegistrableDomain> ResourceLoadStatisticsDatabaseStore::ensurePrevalentRe
         ensureResourceStatisticsForRegistrableDomain(debugManualPrevalentResource());
         setPrevalentResource(debugManualPrevalentResource(), ResourceLoadPrevalence::High);
         primaryDomainsToBlock.uncheckedAppend(debugManualPrevalentResource());
-        RELEASE_LOG_INFO(ITPDebug, "Did set %{public}s as prevalent resource for the purposes of ITP Debug Mode.", debugManualPrevalentResource().string().utf8().data());
+        RELEASE_LOG_INFO(ITPDebug, "Did set %{private}s as prevalent resource for the purposes of ITP Debug Mode.", debugManualPrevalentResource().string().utf8().data());
     }
 
     return primaryDomainsToBlock;
@@ -983,7 +1063,7 @@ void ResourceLoadStatisticsDatabaseStore::setUserInteraction(const RegistrableDo
         || m_mostRecentUserInteractionStatement.bindDouble(2, mostRecentInteraction.secondsSinceEpoch().value()) != SQLITE_OK
         || m_mostRecentUserInteractionStatement.bindText(3, domain.string()) != SQLITE_OK
         || m_mostRecentUserInteractionStatement.step() != SQLITE_DONE) {
-        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::setUserInteraction, error message: %{public}s", this, m_database.lastErrorMsg());
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::setUserInteraction, error message: %{private}s", this, m_database.lastErrorMsg());
         ASSERT_NOT_REACHED();
         return;
     }
@@ -1011,7 +1091,7 @@ void ResourceLoadStatisticsDatabaseStore::clearUserInteraction(const Registrable
     if (removeStorageAccess.prepare() != SQLITE_OK
         || removeStorageAccess.bindInt(1, targetResult.second) != SQLITE_OK
         || removeStorageAccess.step() != SQLITE_DONE) {
-        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::logUserInteraction failed to bind, error message: %{public}s", this, m_database.lastErrorMsg());
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::logUserInteraction failed to bind, error message: %{private}s", this, m_database.lastErrorMsg());
         ASSERT_NOT_REACHED();
     }
 }
@@ -1022,7 +1102,7 @@ bool ResourceLoadStatisticsDatabaseStore::hasHadUserInteraction(const Registrabl
 
     if (m_hadUserInteractionStatement.bindText(1, domain.string()) != SQLITE_OK
         || m_hadUserInteractionStatement.step() != SQLITE_ROW) {
-        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::m_hadUserInteractionStatement failed, error message: %{public}s", this, m_database.lastErrorMsg());
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::m_hadUserInteractionStatement failed, error message: %{private}s", this, m_database.lastErrorMsg());
 
         int resetResult = m_hadUserInteractionStatement.reset();
         ASSERT_UNUSED(resetResult, resetResult == SQLITE_OK);
@@ -1061,7 +1141,7 @@ void ResourceLoadStatisticsDatabaseStore::setPrevalentResource(const Registrable
     if (m_updatePrevalentResourceStatement.bindInt(1, 1) != SQLITE_OK
         || m_updatePrevalentResourceStatement.bindText(2, domain.string()) != SQLITE_OK
         || m_updatePrevalentResourceStatement.step() != SQLITE_DONE) {
-        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::m_updatePrevalentResourceStatement failed, error message: %{public}s", this, m_database.lastErrorMsg());
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::m_updatePrevalentResourceStatement failed, error message: %{private}s", this, m_database.lastErrorMsg());
         ASSERT_NOT_REACHED();
         return;
     }
@@ -1073,7 +1153,7 @@ void ResourceLoadStatisticsDatabaseStore::setPrevalentResource(const Registrable
         if (m_updateVeryPrevalentResourceStatement.bindInt(1, 1) != SQLITE_OK
             || m_updateVeryPrevalentResourceStatement.bindText(2, domain.string()) != SQLITE_OK
             || m_updateVeryPrevalentResourceStatement.step() != SQLITE_DONE) {
-            RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::m_updateVeryPrevalentResourceStatement failed, error message: %{public}s", this, m_database.lastErrorMsg());
+            RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::m_updateVeryPrevalentResourceStatement failed, error message: %{private}s", this, m_database.lastErrorMsg());
             ASSERT_NOT_REACHED();
             return;
         }
@@ -1094,7 +1174,7 @@ void ResourceLoadStatisticsDatabaseStore::setDomainsAsPrevalent(StdSet<unsigned>
     SQLiteStatement domainsToUpdateStatement(m_database, makeString("UPDATE ObservedDomains SET isPrevalent = 1 WHERE domainID IN (", buildList(WTF::IteratorRange<StdSet<unsigned>::iterator>(domains.begin(), domains.end())), ")"));
     if (domainsToUpdateStatement.prepare() != SQLITE_OK
         || domainsToUpdateStatement.step() != SQLITE_DONE) {
-        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::setDomainsAsPrevalent failed, error message: %{public}s", this, m_database.lastErrorMsg());
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::setDomainsAsPrevalent failed, error message: %{private}s", this, m_database.lastErrorMsg());
         ASSERT_NOT_REACHED();
         return;
     }
@@ -1119,7 +1199,7 @@ bool ResourceLoadStatisticsDatabaseStore::predicateValueForDomain(WebCore::SQLit
         int resetResult = predicateStatement.reset();
         ASSERT_UNUSED(resetResult, resetResult == SQLITE_OK);
 
-        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::predicateValueForDomain failed to bind, error message: %{public}s", this, m_database.lastErrorMsg());
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::predicateValueForDomain failed to bind, error message: %{private}s", this, m_database.lastErrorMsg());
         return false;
     }
 
@@ -1178,7 +1258,7 @@ void ResourceLoadStatisticsDatabaseStore::clearPrevalentResource(const Registrab
     
     if (m_clearPrevalentResourceStatement.bindText(1, domain.string()) != SQLITE_OK
         || m_clearPrevalentResourceStatement.step() != SQLITE_DONE) {
-        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::clearPrevalentResource, error message: %{public}s", this, m_database.lastErrorMsg());
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::clearPrevalentResource, error message: %{private}s", this, m_database.lastErrorMsg());
         ASSERT_NOT_REACHED();
         return;
     }
@@ -1194,9 +1274,9 @@ void ResourceLoadStatisticsDatabaseStore::setGrandfathered(const RegistrableDoma
     ensureResourceStatisticsForRegistrableDomain(domain);
     
     if (m_updateGrandfatheredStatement.bindInt(1, value) != SQLITE_OK
-        || m_updateGrandfatheredStatement.bindText(1, domain.string()) != SQLITE_OK
+        || m_updateGrandfatheredStatement.bindText(2, domain.string()) != SQLITE_OK
         || m_updateGrandfatheredStatement.step() != SQLITE_DONE) {
-        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::setGrandfathered failed to bind, error message: %{public}s", this, m_database.lastErrorMsg());
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::setGrandfathered failed to bind, error message: %{private}s", this, m_database.lastErrorMsg());
         ASSERT_NOT_REACHED();
         return;
     }
@@ -1289,7 +1369,7 @@ std::pair<ResourceLoadStatisticsDatabaseStore::AddedRecord, unsigned> ResourceLo
     ASSERT(!RunLoop::isMain());
 
     if (m_domainIDFromStringStatement.bindText(1, domain.string()) != SQLITE_OK) {
-        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::ensureResourceStatisticsForRegistrableDomain failed, error message: %{public}s", this, m_database.lastErrorMsg());
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::ensureResourceStatisticsForRegistrableDomain failed, error message: %{private}s", this, m_database.lastErrorMsg());
         ASSERT_NOT_REACHED();
         return { AddedRecord::No, 0 };
     }
@@ -1316,7 +1396,7 @@ void ResourceLoadStatisticsDatabaseStore::clearDatabaseContents()
     m_database.clearAllTables();
 
     if (!createSchema()) {
-        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::clearDatabaseContents failed, error message: %{public}s, database path: %{public}s", this, m_database.lastErrorMsg(), m_storageDirectoryPath.utf8().data());
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::clearDatabaseContents failed, error message: %{private}s, database path: %{private}s", this, m_database.lastErrorMsg(), m_storageDirectoryPath.utf8().data());
         ASSERT_NOT_REACHED();
         return;
     }
@@ -1345,7 +1425,7 @@ ResourceLoadStatisticsDatabaseStore::CookieTreatmentResult ResourceLoadStatistic
     SQLiteStatement statement(m_database, "SELECT isPrevalent, hadUserInteraction FROM ObservedDomains WHERE registrableDomain = ?");
     if (statement.prepare() != SQLITE_OK
         || statement.bindText(1, domain.string()) != SQLITE_OK) {
-        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::cookieTreatmentForOrigin failed to bind, error message: %{public}s", this, m_database.lastErrorMsg());
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::cookieTreatmentForOrigin failed to bind, error message: %{private}s", this, m_database.lastErrorMsg());
         ASSERT_NOT_REACHED();
     }
     
@@ -1497,7 +1577,7 @@ void ResourceLoadStatisticsDatabaseStore::clearExpiredUserInteractions()
 
     if (clearExpiredInteraction.step() != SQLITE_DONE
         || removeStorageAccess.step() != SQLITE_DONE) {
-        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::clearExpiredUserInteractions statement(s) failed to step, error message: %{public}s", this, m_database.lastErrorMsg());
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::clearExpiredUserInteractions statement(s) failed to step, error message: %{private}s", this, m_database.lastErrorMsg());
         ASSERT_NOT_REACHED();
     }
 }
@@ -1516,7 +1596,7 @@ void ResourceLoadStatisticsDatabaseStore::clearGrandfathering(Vector<unsigned>&&
         return;
     
     if (clearGrandfatheringStatement.step() != SQLITE_DONE) {
-        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::clearGrandfathering failed to bind, error message: %{public}s", this, m_database.lastErrorMsg());
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::clearGrandfathering failed to bind, error message: %{private}s", this, m_database.lastErrorMsg());
         ASSERT_NOT_REACHED();
     }
 }
@@ -1596,7 +1676,7 @@ void ResourceLoadStatisticsDatabaseStore::pruneStatisticsIfNeeded()
     SQLiteStatement recordsToPrune(m_database, "SELECT domainID FROM ObservedDomains ORDER BY hadUserInteraction, isPrevalent, lastSeen LIMIT ?");
     if (recordsToPrune.prepare() != SQLITE_OK
         || recordsToPrune.bindInt(1, countLeftToPrune) != SQLITE_OK) {
-        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::pruneStatisticsIfNeeded failed, error message: %{public}s", this, m_database.lastErrorMsg());
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::pruneStatisticsIfNeeded failed, error message: %{private}s", this, m_database.lastErrorMsg());
         ASSERT_NOT_REACHED();
         return;
     }
@@ -1610,7 +1690,7 @@ void ResourceLoadStatisticsDatabaseStore::pruneStatisticsIfNeeded()
     SQLiteStatement pruneCommand(m_database, makeString("DELETE from ObservedDomains WHERE domainID IN (", listToPrune, ")"));
     if (pruneCommand.prepare() != SQLITE_OK
         || pruneCommand.step() != SQLITE_DONE) {
-        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::pruneStatisticsIfNeeded failed, error message: %{public}s", this, m_database.lastErrorMsg());
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::pruneStatisticsIfNeeded failed, error message: %{private}s", this, m_database.lastErrorMsg());
         ASSERT_NOT_REACHED();
         return;
     }
@@ -1623,7 +1703,7 @@ void ResourceLoadStatisticsDatabaseStore::updateLastSeen(const RegistrableDomain
     if (m_updateLastSeenStatement.bindDouble(1, lastSeen.secondsSinceEpoch().value()) != SQLITE_OK
         || m_updateLastSeenStatement.bindText(2, domain.string()) != SQLITE_OK
         || m_updateLastSeenStatement.step() != SQLITE_DONE) {
-        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::updateLastSeen failed to bind, error message: %{public}s", this, m_database.lastErrorMsg());
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::updateLastSeen failed to bind, error message: %{private}s", this, m_database.lastErrorMsg());
         ASSERT_NOT_REACHED();
         return;
     }
@@ -1662,6 +1742,22 @@ void ResourceLoadStatisticsDatabaseStore::setVeryPrevalentResource(const Registr
     setPrevalentResource(domain, ResourceLoadPrevalence::VeryHigh);
 }
 
+void ResourceLoadStatisticsDatabaseStore::updateDataRecordsRemoved(const RegistrableDomain& domain, int value)
+{
+    ASSERT(!RunLoop::isMain());
+
+    if (m_updateDataRecordsRemovedStatement.bindInt(1, value) != SQLITE_OK
+        || m_updateDataRecordsRemovedStatement.bindText(2, domain.string()) != SQLITE_OK
+        || m_updateDataRecordsRemovedStatement.step() != SQLITE_DONE) {
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::updateDataRecordsRemoved failed to bind, error message: %{private}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+        return;
+    }
+
+    int resetResult = m_updateDataRecordsRemovedStatement.reset();
+    ASSERT_UNUSED(resetResult, resetResult == SQLITE_OK);
+}
+
 } // namespace WebKit
 
 #endif
index f1d5655..ea7e649 100644 (file)
@@ -54,7 +54,7 @@ public:
     ResourceLoadStatisticsDatabaseStore(WebResourceLoadStatisticsStore&, WorkQueue&, ShouldIncludeLocalhost, const String& storageDirectoryPath, PAL::SessionID);
 
     void populateFromMemoryStore(const ResourceLoadStatisticsMemoryStore&);
-
+    void mergeStatistics(Vector<ResourceLoadStatistics>&&) override;
     void clear(CompletionHandler<void()>&&) override;
     bool isEmpty() const override;
 
@@ -105,16 +105,17 @@ public:
     void setLastSeen(const RegistrableDomain&, Seconds) override;
 
 private:
+    void mergeStatistic(const ResourceLoadStatistics&);
+    void merge(WebCore::SQLiteStatement&, const ResourceLoadStatistics&);
     void clearDatabaseContents();
     bool insertObservedDomain(const ResourceLoadStatistics&);
     void insertDomainRelationships(const ResourceLoadStatistics&);
     bool insertDomainRelationship(WebCore::SQLiteStatement&, unsigned domainID, const RegistrableDomain& topFrameDomain);
     bool relationshipExists(WebCore::SQLiteStatement&, Optional<unsigned> firstDomainID, const RegistrableDomain& secondDomain) const;
     Optional<unsigned> domainID(const RegistrableDomain&) const;
-#ifndef NDEBUG
-    bool confirmDomainDoesNotExist(const RegistrableDomain&) const;
-#endif
+    bool domainExists(const RegistrableDomain&) const;
     void updateLastSeen(const RegistrableDomain&, WallTime);
+    void updateDataRecordsRemoved(const RegistrableDomain&, int);
     void setUserInteraction(const RegistrableDomain&, bool hadUserInteraction, WallTime);
     Vector<RegistrableDomain> domainsToBlockAndDeleteCookiesFor() const;
     Vector<RegistrableDomain> domainsToBlockButKeepCookiesFor() const;
@@ -175,6 +176,7 @@ private:
     WebCore::SQLiteStatement m_insertTopLevelDomainStatement;
     mutable WebCore::SQLiteStatement m_domainIDFromStringStatement;
     WebCore::SQLiteStatement m_storageAccessUnderTopFrameDomainsStatement;
+    WebCore::SQLiteStatement m_storageAccessUnderTopFrameDomainsExistsStatement;
     WebCore::SQLiteStatement m_topFrameUniqueRedirectsTo;
     mutable WebCore::SQLiteStatement m_topFrameUniqueRedirectsToExists;
     WebCore::SQLiteStatement m_topFrameUniqueRedirectsFrom;
@@ -191,6 +193,7 @@ private:
     mutable WebCore::SQLiteStatement m_subresourceUniqueRedirectsFromExists;
     WebCore::SQLiteStatement m_mostRecentUserInteractionStatement;
     WebCore::SQLiteStatement m_updateLastSeenStatement;
+    mutable WebCore::SQLiteStatement m_updateDataRecordsRemovedStatement;
     WebCore::SQLiteStatement m_updatePrevalentResourceStatement;
     mutable WebCore::SQLiteStatement m_isPrevalentResourceStatement;
     WebCore::SQLiteStatement m_updateVeryPrevalentResourceStatement;
index f5e6663..6589f6b 100644 (file)
@@ -701,16 +701,6 @@ void ResourceLoadStatisticsMemoryStore::clear(CompletionHandler<void()>&& comple
     updateCookieBlockingForDomains(domainsToBlock, [callbackAggregator = callbackAggregator.copyRef()] { });
 }
 
-bool ResourceLoadStatisticsMemoryStore::wasAccessedAsFirstPartyDueToUserInteraction(const ResourceLoadStatistics& current, const ResourceLoadStatistics& updated) const
-{
-    if (!current.hadUserInteraction && !updated.hadUserInteraction)
-        return false;
-
-    auto mostRecentUserInteractionTime = std::max(current.mostRecentUserInteractionTime, updated.mostRecentUserInteractionTime);
-
-    return updated.lastSeen <= mostRecentUserInteractionTime + 24_h;
-}
-
 void ResourceLoadStatisticsMemoryStore::mergeStatistics(Vector<ResourceLoadStatistics>&& statistics)
 {
     ASSERT(!RunLoop::isMain());
@@ -719,11 +709,8 @@ void ResourceLoadStatisticsMemoryStore::mergeStatistics(Vector<ResourceLoadStati
         auto result = m_resourceStatisticsMap.ensure(statistic.registrableDomain, [&statistic] {
             return WTFMove(statistic);
         });
-        if (!result.isNewEntry) {
-            if (wasAccessedAsFirstPartyDueToUserInteraction(result.iterator->value, statistic))
-                result.iterator->value.timesAccessedAsFirstPartyDueToUserInteraction++;
+        if (!result.isNewEntry)
             result.iterator->value.merge(statistic);
-        }
     }
 }
 
index 879e1af..68054a5 100644 (file)
@@ -61,7 +61,7 @@ public:
     std::unique_ptr<WebCore::KeyedEncoder> createEncoderFromData() const;
     void mergeWithDataFromDecoder(WebCore::KeyedDecoder&);
 
-    void mergeStatistics(Vector<ResourceLoadStatistics>&&);
+    void mergeStatistics(Vector<ResourceLoadStatistics>&&) override;
     void processStatistics(const Function<void(const ResourceLoadStatistics&)>&) const;
 
     void updateCookieBlocking(CompletionHandler<void()>&&) override;
@@ -117,7 +117,6 @@ private:
     bool hasHadUnexpiredRecentUserInteraction(ResourceLoadStatistics&, OperatingDatesWindow) const;
     bool shouldRemoveAllWebsiteDataFor(ResourceLoadStatistics&, bool shouldCheckForGrandfathering) const;
     bool shouldRemoveAllButCookiesFor(ResourceLoadStatistics&, bool shouldCheckForGrandfathering) const;
-    bool wasAccessedAsFirstPartyDueToUserInteraction(const ResourceLoadStatistics& current, const ResourceLoadStatistics& updated) const;
     void incrementRecordsDeletedCountForDomains(HashSet<RegistrableDomain>&&) override;
     void setPrevalentResource(ResourceLoadStatistics&, ResourceLoadPrevalence);
     unsigned recursivelyGetAllDomainsThatHaveRedirectedToThisDomain(const ResourceLoadStatistics&, HashSet<RedirectedToDomain>&, unsigned numberOfRecursiveCalls) const;
index 6f61ed3..99a6710 100644 (file)
@@ -108,6 +108,7 @@ public:
     virtual void classifyPrevalentResources() = 0;
     virtual void syncStorageIfNeeded() = 0;
     virtual void syncStorageImmediately() = 0;
+    virtual void mergeStatistics(Vector<ResourceLoadStatistics>&&) = 0;
 
     virtual void requestStorageAccessUnderOpener(DomainInNeedOfStorageAccess&&, WebCore::PageIdentifier openerID, OpenerDomain&&) = 0;
     void removeAllStorageAccess(CompletionHandler<void()>&&);
index 94f0168..113d353 100644 (file)
@@ -264,19 +264,17 @@ void WebResourceLoadStatisticsStore::resourceLoadStatisticsUpdated(Vector<Resour
     // coming from IPC. ResourceLoadStatistics only contains strings which are safe to move to other threads as long
     // as nobody on this thread holds a reference to those strings.
     postTask([this, statistics = WTFMove(statistics)]() mutable {
-        if (!m_statisticsStore || !is<ResourceLoadStatisticsMemoryStore>(*m_statisticsStore))
+        if (!m_statisticsStore)
             return;
 
-        auto& memoryStore = downcast<ResourceLoadStatisticsMemoryStore>(*m_statisticsStore);
-    
-        memoryStore.mergeStatistics(WTFMove(statistics));
+        m_statisticsStore->mergeStatistics(WTFMove(statistics));
 
         // We can cancel any pending request to process statistics since we're doing it synchronously below.
-        memoryStore.cancelPendingStatisticsProcessingRequest();
+        m_statisticsStore->cancelPendingStatisticsProcessingRequest();
 
         // Fire before processing statistics to propagate user interaction as fast as possible to the network process.
-        memoryStore.updateCookieBlocking([]() { });
-        memoryStore.processStatisticsAndDataRecords();
+        m_statisticsStore->updateCookieBlocking([]() { });
+        m_statisticsStore->processStatisticsAndDataRecords();
     });
 }
 
@@ -597,6 +595,35 @@ void WebResourceLoadStatisticsStore::setLastSeen(const RegistrableDomain& domain
         postTaskReply(WTFMove(completionHandler));
     });
 }
+
+void WebResourceLoadStatisticsStore::mergeStatisticForTesting(const RegistrableDomain& domain, const RegistrableDomain& topFrameDomain, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&& completionHandler)
+{
+    ASSERT(RunLoop::isMain());
+
+    postTask([this, domain = domain.isolatedCopy(), topFrameDomain = topFrameDomain.isolatedCopy(), lastSeen, hadUserInteraction, mostRecentUserInteraction, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, completionHandler = WTFMove(completionHandler)]() mutable {
+        if (m_statisticsStore) {
+            ResourceLoadStatistics statistic(domain);
+            statistic.lastSeen = WallTime::fromRawSeconds(lastSeen.seconds());
+            statistic.hadUserInteraction = hadUserInteraction;
+            statistic.mostRecentUserInteractionTime = WallTime::fromRawSeconds(mostRecentUserInteraction.seconds());
+            statistic.grandfathered = isGrandfathered;
+            statistic.isPrevalentResource = isPrevalent;
+            statistic.isVeryPrevalentResource = isVeryPrevalent;
+            statistic.dataRecordsRemoved = dataRecordsRemoved;
+            
+            if (!topFrameDomain.isEmpty()) {
+                HashSet<RegistrableDomain> topFrameDomains;
+                topFrameDomains.add(topFrameDomain);
+                statistic.subframeUnderTopFrameDomains = WTFMove(topFrameDomains);
+            }
+
+            Vector<ResourceLoadStatistics> statistics;
+            statistics.append(WTFMove(statistic));
+            m_statisticsStore->mergeStatistics(WTFMove(statistics));
+        }
+        postTaskReply(WTFMove(completionHandler));
+    });
+}
     
 void WebResourceLoadStatisticsStore::setPrevalentResource(const RegistrableDomain& domain, CompletionHandler<void()>&& completionHandler)
 {
index 7eae1a7..fb18105 100644 (file)
@@ -129,6 +129,7 @@ public:
     bool hasStorageAccessForFrame(const SubFrameDomain&, const TopFrameDomain&, WebCore::FrameIdentifier, WebCore::PageIdentifier);
     void requestStorageAccess(const SubFrameDomain&, const TopFrameDomain&, WebCore::FrameIdentifier, WebCore::PageIdentifier, WebPageProxyIdentifier, CompletionHandler<void(StorageAccessWasGranted, StorageAccessPromptWasShown)>&&);
     void setLastSeen(const RegistrableDomain&, Seconds, CompletionHandler<void()>&&);
+    void mergeStatisticForTesting(const RegistrableDomain&, const TopFrameDomain&, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&&);
     void setPrevalentResource(const RegistrableDomain&, CompletionHandler<void()>&&);
     void setVeryPrevalentResource(const RegistrableDomain&, CompletionHandler<void()>&&);
     void dumpResourceLoadStatistics(CompletionHandler<void(String)>&&);
index dc25c6b..2fb7809 100644 (file)
@@ -974,6 +974,19 @@ void NetworkProcess::setLastSeen(PAL::SessionID sessionID, const RegistrableDoma
     }
 }
 
+void NetworkProcess::mergeStatisticForTesting(PAL::SessionID sessionID, const RegistrableDomain& domain, const RegistrableDomain& topFrameDomain, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&& completionHandler)
+{
+    if (auto* networkSession = this->networkSession(sessionID)) {
+        if (auto* resourceLoadStatistics = networkSession->resourceLoadStatistics())
+            resourceLoadStatistics->mergeStatisticForTesting(domain, topFrameDomain, lastSeen, hadUserInteraction, mostRecentUserInteraction, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, WTFMove(completionHandler));
+        else
+            completionHandler();
+    } else {
+        ASSERT_NOT_REACHED();
+        completionHandler();
+    }
+}
+
 void NetworkProcess::getAllStorageAccessEntries(PAL::SessionID sessionID, CompletionHandler<void(Vector<String> domains)>&& completionHandler)
 {
     if (auto* networkStorageSession = storageSession(sessionID))
index 90cf732..3a6256f 100644 (file)
@@ -242,6 +242,7 @@ public:
     void setCacheMaxAgeCapForPrevalentResources(PAL::SessionID, Seconds, CompletionHandler<void()>&&);
     void setGrandfatheringTime(PAL::SessionID, Seconds, CompletionHandler<void()>&&);
     void setLastSeen(PAL::SessionID, const RegistrableDomain&, Seconds, CompletionHandler<void()>&&);
+    void mergeStatisticForTesting(PAL::SessionID, const RegistrableDomain&, const TopFrameDomain&, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&&);
     void setMinimumTimeBetweenDataRecordsRemoval(PAL::SessionID, Seconds, CompletionHandler<void()>&&);
     void setNotifyPagesWhenDataRecordsWereScanned(PAL::SessionID, bool value, CompletionHandler<void()>&&);
     void setIsRunningResourceLoadStatisticsTest(PAL::SessionID, bool value, CompletionHandler<void()>&&);
index a77663e..1ef8fbc 100644 (file)
@@ -94,6 +94,7 @@ messages -> NetworkProcess LegacyReceiver {
     IsVeryPrevalentResource(PAL::SessionID sessionID, WebCore::RegistrableDomain targetDomain) -> (bool isVeryPrevalent) Async
     SetAgeCapForClientSideCookies(PAL::SessionID sessionID, Optional<Seconds> seconds) -> () Async
     SetLastSeen(PAL::SessionID sessionID, WebCore::RegistrableDomain resourceDomain, Seconds seconds) -> () Async
+    MergeStatisticForTesting(PAL::SessionID sessionID, WebCore::RegistrableDomain resourceDomain, WebCore::RegistrableDomain topFrameDomain, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, uint64_t dataRecordsRemoved) -> () Async
     SetPrevalentResource(PAL::SessionID sessionID, WebCore::RegistrableDomain resourceDomain) -> () Async
     SetPrevalentResourceForDebugMode(PAL::SessionID sessionID, WebCore::RegistrableDomain resourceDomain) -> () Async
     HadUserInteraction(PAL::SessionID sessionID, WebCore::RegistrableDomain resourceDomain) -> (bool hadUserInteraction) Async
index 960f9fb..512e05c 100644 (file)
@@ -119,6 +119,17 @@ void WKWebsiteDataStoreSetStatisticsLastSeen(WKWebsiteDataStoreRef dataStoreRef,
 #endif
 }
 
+void WKWebsiteDataStoreSetStatisticsMergeStatistic(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, WKStringRef topFrameDomain, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, void* context, WKWebsiteDataStoreStatisticsMergeStatisticFunction completionHandler)
+{
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+    WebKit::toImpl(dataStoreRef)->mergeStatisticForTesting(URL(URL(), WebKit::toImpl(host)->string()), URL(URL(), WebKit::toImpl(topFrameDomain)->string()), Seconds { lastSeen }, hadUserInteraction, Seconds { mostRecentUserInteraction }, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, [context, completionHandler] {
+        completionHandler(context);
+    });
+#else
+    completionHandler(context);
+#endif
+}
+
 void WKWebsiteDataStoreSetStatisticsPrevalentResource(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, bool value, void* context, WKWebsiteDataStoreStatisticsPrevalentResourceFunction completionHandler)
 {
 #if ENABLE(RESOURCE_LOAD_STATISTICS)
@@ -260,7 +271,7 @@ void WKWebsiteDataStoreSetStatisticsGrandfathered(WKWebsiteDataStoreRef dataStor
 void WKWebsiteDataStoreIsStatisticsGrandfathered(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, void* context, WKWebsiteDataStoreIsStatisticsGrandfatheredFunction callback)
 {
 #if ENABLE(RESOURCE_LOAD_STATISTICS)
-    WebKit::toImpl(dataStoreRef)->hasHadUserInteraction(URL(URL(), WebKit::toImpl(host)->string()), [context, callback](bool isGrandfathered) {
+    WebKit::toImpl(dataStoreRef)->isGrandfathered(URL(URL(), WebKit::toImpl(host)->string()), [context, callback](bool isGrandfathered) {
         callback(isGrandfathered, context);
     });
 #else
index ccf5913..f2a1782 100644 (file)
@@ -49,6 +49,8 @@ WK_EXPORT void WKWebsiteDataStoreSetResourceLoadStatisticsDebugModeWithCompletio
 WK_EXPORT void WKWebsiteDataStoreSetResourceLoadStatisticsPrevalentResourceForDebugMode(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, void* context, WKWebsiteDataStoreStatisticsDebugModeFunction completionHandler);    
 typedef void (*WKWebsiteDataStoreStatisticsLastSeenFunction)(void* functionContext);
 WK_EXPORT void WKWebsiteDataStoreSetStatisticsLastSeen(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, double seconds, void* context, WKWebsiteDataStoreStatisticsLastSeenFunction completionHandler);
+typedef void (*WKWebsiteDataStoreStatisticsMergeStatisticFunction)(void* functionContext);
+WK_EXPORT void WKWebsiteDataStoreSetStatisticsMergeStatistic(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, WKStringRef topFrameDomain, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, void* context, WKWebsiteDataStoreStatisticsMergeStatisticFunction completionHandler);
 typedef void (*WKWebsiteDataStoreStatisticsPrevalentResourceFunction)(void* functionContext);
 WK_EXPORT void WKWebsiteDataStoreSetStatisticsPrevalentResource(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, bool value, void* context, WKWebsiteDataStoreStatisticsPrevalentResourceFunction completionHandler);
 typedef void (*WKWebsiteDataStoreStatisticsVeryPrevalentResourceFunction)(void* functionContext);
index 31a7257..3830b25 100644 (file)
@@ -535,6 +535,11 @@ void NetworkProcessProxy::setLastSeen(PAL::SessionID sessionID, const Registrabl
     sendWithAsyncReply(Messages::NetworkProcess::SetLastSeen(sessionID, resourceDomain, lastSeen), WTFMove(completionHandler));
 }
 
+void NetworkProcessProxy::mergeStatisticForTesting(PAL::SessionID sessionID, const RegistrableDomain& resourceDomain, const RegistrableDomain& topFrameDomain, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&& completionHandler)
+{
+    sendWithAsyncReply(Messages::NetworkProcess::MergeStatisticForTesting(sessionID, resourceDomain, topFrameDomain, lastSeen, hadUserInteraction, mostRecentUserInteraction, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved), WTFMove(completionHandler));
+}
+
 void NetworkProcessProxy::clearPrevalentResource(PAL::SessionID sessionID, const RegistrableDomain& resourceDomain, CompletionHandler<void()>&& completionHandler)
 {
     if (!canSendMessage()) {
index 64713ab..2d6adbe 100644 (file)
@@ -113,6 +113,7 @@ public:
     void logUserInteraction(PAL::SessionID, const RegistrableDomain&, CompletionHandler<void()>&&);
     void scheduleStatisticsAndDataRecordsProcessing(PAL::SessionID, CompletionHandler<void()>&&);
     void setLastSeen(PAL::SessionID, const RegistrableDomain&, Seconds, CompletionHandler<void()>&&);
+    void mergeStatisticForTesting(PAL::SessionID, const RegistrableDomain&, const TopFrameDomain&, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&&);
     void setAgeCapForClientSideCookies(PAL::SessionID, Optional<Seconds>, CompletionHandler<void()>&&);
     void setCacheMaxAgeCap(PAL::SessionID, Seconds, CompletionHandler<void()>&&);
     void setGrandfathered(PAL::SessionID, const RegistrableDomain&, bool isGrandfathered, CompletionHandler<void()>&&);
index 69355d7..627d2d9 100644 (file)
@@ -1214,6 +1214,24 @@ void WebsiteDataStore::isPrevalentResource(const URL& url, CompletionHandler<voi
     }
 }
 
+void WebsiteDataStore::isGrandfathered(const URL& url, CompletionHandler<void(bool isPrevalent)>&& completionHandler)
+{
+    ASSERT(RunLoop::isMain());
+    
+    if (url.protocolIsAbout() || url.isEmpty()) {
+        completionHandler(false);
+        return;
+    }
+
+    for (auto& processPool : processPools()) {
+        if (auto* process = processPool->networkProcess()) {
+            process->isGrandfathered(m_sessionID, WebCore::RegistrableDomain { url }, WTFMove(completionHandler));
+            RELEASE_ASSERT(processPools().size() == 1);
+            break;
+        }
+    }
+}
+
 void WebsiteDataStore::setPrevalentResource(const URL& url, CompletionHandler<void()>&& completionHandler)
 {
     ASSERT(RunLoop::isMain());
@@ -1533,6 +1551,21 @@ void WebsiteDataStore::setLastSeen(const URL& url, Seconds seconds, CompletionHa
     }
 }
 
+void WebsiteDataStore::mergeStatisticForTesting(const URL& url , const URL& topFrameUrl, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&& completionHandler)
+{
+    if (url.protocolIsAbout() || url.isEmpty()) {
+        completionHandler();
+        return;
+    }
+
+    auto callbackAggregator = CallbackAggregator::create(WTFMove(completionHandler));
+
+    for (auto& processPool : processPools()) {
+        if (auto* process = processPool->networkProcess())
+            process->mergeStatisticForTesting(m_sessionID, WebCore::RegistrableDomain { url }, WebCore::RegistrableDomain { topFrameUrl }, lastSeen, hadUserInteraction, mostRecentUserInteraction, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, [processPool, callbackAggregator = callbackAggregator.copyRef()] { });
+    }
+}
+
 void WebsiteDataStore::setNotifyPagesWhenDataRecordsWereScanned(bool value, CompletionHandler<void()>&& completionHandler)
 {
     auto callbackAggregator = CallbackAggregator::create(WTFMove(completionHandler));
index 8a5feba..01837fe 100644 (file)
@@ -159,9 +159,11 @@ public:
     void scheduleStatisticsAndDataRecordsProcessing(CompletionHandler<void()>&&);
     void submitTelemetry();
     void setGrandfathered(const URL&, bool, CompletionHandler<void()>&&);
+    void isGrandfathered(const URL&, CompletionHandler<void(bool)>&&);
     void setUseITPDatabase(bool);
     void setGrandfatheringTime(Seconds, CompletionHandler<void()>&&);
     void setLastSeen(const URL&, Seconds, CompletionHandler<void()>&&);
+    void mergeStatisticForTesting(const URL&, const URL& topFrameUrl, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&&);
     void setNotifyPagesWhenDataRecordsWereScanned(bool, CompletionHandler<void()>&&);
     void setIsRunningResourceLoadStatisticsTest(bool, CompletionHandler<void()>&&);
     void setPruneEntriesDownTo(size_t, CompletionHandler<void()>&&);
index 43f253c..7f90ed8 100644 (file)
@@ -1,3 +1,29 @@
+2019-10-02  Kate Cheney  <katherine_cheney@apple.com>
+
+        Updated resource load statistics are never merged into the SQLite Database backend (202372).
+        https://bugs.webkit.org/show_bug.cgi?id=202372
+        <rdar://problem/55854542>
+
+        Reviewed by Brent Fulgham. 
+
+        Added testing functionality to the mergeStatistics() function of the 
+        SQLite backend.
+
+        * WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl:
+        * WebKitTestRunner/InjectedBundle/InjectedBundle.cpp:
+        (WTR::InjectedBundle::didReceiveMessageToPage):
+        * WebKitTestRunner/InjectedBundle/TestRunner.cpp:
+        (WTR::TestRunner::setStatisticsMergeStatistic):
+        (WTR::TestRunner::statisticsCallDidSetMergeStatisticCallback):
+        * WebKitTestRunner/InjectedBundle/TestRunner.h:
+        * WebKitTestRunner/TestController.cpp:
+        (WTR::TestController::setStatisticsMergeStatistic):
+        * WebKitTestRunner/TestController.h:
+        * WebKitTestRunner/TestInvocation.cpp:
+        (WTR::TestInvocation::didReceiveSynchronousMessageFromInjectedBundle):
+        (WTR::TestInvocation::didMergeStatistic):
+        * WebKitTestRunner/TestInvocation.h:
+
 2019-10-02  Aakash Jain  <aakash_jain@apple.com>
 
         [ews] windows or wincairo queue should use del instead of rm command
index a01d4fb..42aad54 100644 (file)
@@ -298,6 +298,7 @@ interface TestRunner {
     void setStatisticsDebugMode(boolean value, object completionHandler);
     void setStatisticsPrevalentResourceForDebugMode(DOMString hostName, object completionHandler);
     void setStatisticsLastSeen(DOMString hostName, double seconds, object completionHandler);
+    void setStatisticsMergeStatistic(DOMString hostName, DOMString topFrameDomain, double lastSeen, boolean hadUserInteraction, double mostRecentUserInteraction, boolean isGrandfathered, boolean isPrevalent, boolean isVeryPrevalent, unsigned long dataRecordsRemoved, object completionHandler);
     void setStatisticsPrevalentResource(DOMString hostName, boolean value, object completionHandler);
     void setStatisticsVeryPrevalentResource(DOMString hostName, boolean value, object completionHandler);
     boolean isStatisticsPrevalentResource(DOMString hostName);
index 9c10b40..01b7562 100644 (file)
@@ -356,6 +356,11 @@ void InjectedBundle::didReceiveMessageToPage(WKBundlePageRef page, WKStringRef m
         m_testRunner->statisticsCallDidSetLastSeenCallback();
         return;
     }
+    
+    if (WKStringIsEqualToUTF8CString(messageName, "CallDidMergeStatistic")) {
+        m_testRunner->statisticsCallDidSetMergeStatisticCallback();
+        return;
+    }
 
     if (WKStringIsEqualToUTF8CString(messageName, "CallDidSetPrevalentResource")) {
         m_testRunner->statisticsCallDidSetPrevalentResourceCallback();
index 82db674..01104ac 100644 (file)
@@ -732,6 +732,7 @@ enum {
     SetStatisticsDebugModeCallbackID,
     SetStatisticsPrevalentResourceForDebugModeCallbackID,
     SetStatisticsLastSeenCallbackID,
+    SetStatisticsMergeStatisticCallbackID,
     SetStatisticsPrevalentResourceCallbackID,
     SetStatisticsVeryPrevalentResourceCallbackID,
     SetStatisticsHasHadUserInteractionCallbackID,
@@ -1453,6 +1454,57 @@ void TestRunner::statisticsCallDidSetLastSeenCallback()
     callTestRunnerCallback(SetStatisticsLastSeenCallbackID);
 }
 
+void TestRunner::setStatisticsMergeStatistic(JSStringRef hostName, JSStringRef topFrameDomain, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, JSValueRef completionHandler)
+{
+    cacheTestRunnerCallback(SetStatisticsMergeStatisticCallbackID, completionHandler);
+
+    Vector<WKRetainPtr<WKStringRef>> keys;
+    Vector<WKRetainPtr<WKTypeRef>> values;
+
+    keys.append(adoptWK(WKStringCreateWithUTF8CString("HostName")));
+    values.append(adoptWK(WKStringCreateWithJSString(hostName)));
+    
+    keys.append(adoptWK(WKStringCreateWithUTF8CString("TopFrameDomain")));
+    values.append(adoptWK(WKStringCreateWithJSString(topFrameDomain)));
+
+    keys.append(adoptWK(WKStringCreateWithUTF8CString("LastSeen")));
+    values.append(adoptWK(WKDoubleCreate(lastSeen)));
+
+    keys.append(adoptWK(WKStringCreateWithUTF8CString("HadUserInteraction")));
+    values.append(adoptWK(WKBooleanCreate(hadUserInteraction)));
+
+    keys.append(adoptWK(WKStringCreateWithUTF8CString("MostRecentUserInteraction")));
+    values.append(adoptWK(WKDoubleCreate(mostRecentUserInteraction)));
+
+    keys.append(adoptWK(WKStringCreateWithUTF8CString("IsGrandfathered")));
+    values.append(adoptWK(WKBooleanCreate(isGrandfathered)));
+
+    keys.append(adoptWK(WKStringCreateWithUTF8CString("IsPrevalent")));
+    values.append(adoptWK(WKBooleanCreate(isPrevalent)));
+
+    keys.append(adoptWK(WKStringCreateWithUTF8CString("IsVeryPrevalent")));
+    values.append(adoptWK(WKBooleanCreate(isVeryPrevalent)));
+
+    keys.append(adoptWK(WKStringCreateWithUTF8CString("DataRecordsRemoved")));
+    values.append(adoptWK(WKUInt64Create(dataRecordsRemoved)));
+
+    Vector<WKStringRef> rawKeys(keys.size());
+    Vector<WKTypeRef> rawValues(values.size());
+
+    for (size_t i = 0; i < keys.size(); ++i) {
+        rawKeys[i] = keys[i].get();
+        rawValues[i] = values[i].get();
+    }
+    WKRetainPtr<WKStringRef> messageName = adoptWK(WKStringCreateWithUTF8CString("SetStatisticsMergeStatistic"));
+    WKRetainPtr<WKDictionaryRef> messageBody = adoptWK(WKDictionaryCreate(rawKeys.data(), rawValues.data(), rawKeys.size()));
+    WKBundlePostSynchronousMessage(InjectedBundle::singleton().bundle(), messageName.get(), messageBody.get(), nullptr);
+}
+
+void TestRunner::statisticsCallDidSetMergeStatisticCallback()
+{
+    callTestRunnerCallback(SetStatisticsMergeStatisticCallbackID);
+}
+
 void TestRunner::setStatisticsPrevalentResource(JSStringRef hostName, bool value, JSValueRef completionHandler)
 {
     cacheTestRunnerCallback(SetStatisticsPrevalentResourceCallbackID, completionHandler);
index 49e5ff0..f170012 100644 (file)
@@ -396,6 +396,8 @@ public:
     void statisticsCallDidSetPrevalentResourceForDebugModeCallback();
     void setStatisticsLastSeen(JSStringRef hostName, double seconds, JSValueRef completionHandler);
     void statisticsCallDidSetLastSeenCallback();
+    void setStatisticsMergeStatistic(JSStringRef hostName, JSStringRef topFrameDomain, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, JSValueRef completionHandler);
+    void statisticsCallDidSetMergeStatisticCallback();
     void setStatisticsPrevalentResource(JSStringRef hostName, bool value, JSValueRef completionHandler);
     void statisticsCallDidSetPrevalentResourceCallback();
     void setStatisticsVeryPrevalentResource(JSStringRef hostName, bool value, JSValueRef completionHandler);
index e7dbe38..0bd7761 100644 (file)
@@ -3261,6 +3261,14 @@ void TestController::setStatisticsLastSeen(WKStringRef host, double seconds)
     m_currentInvocation->didSetLastSeen();
 }
 
+void TestController::setStatisticsMergeStatistic(WKStringRef host, WKStringRef topFrameDomain, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, int dataRecordsRemoved)
+{
+    ResourceStatisticsCallbackContext context(*this);
+    WKWebsiteDataStoreSetStatisticsMergeStatistic(TestController::websiteDataStore(), host, topFrameDomain, lastSeen, hadUserInteraction, mostRecentUserInteraction, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, &context, resourceStatisticsVoidResultCallback);
+    runUntil(context.done, noTimeout);
+    m_currentInvocation->didMergeStatistic();
+}
+
 void TestController::setStatisticsPrevalentResource(WKStringRef host, bool value)
 {
     ResourceStatisticsCallbackContext context(*this);
index fd1c52a..6c2d2d8 100644 (file)
@@ -210,6 +210,7 @@ public:
     void setStatisticsDebugMode(bool value);
     void setStatisticsPrevalentResourceForDebugMode(WKStringRef hostName);
     void setStatisticsLastSeen(WKStringRef hostName, double seconds);
+    void setStatisticsMergeStatistic(WKStringRef host, WKStringRef topFrameDomain, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, int dataRecordsRemoved);
     void setStatisticsPrevalentResource(WKStringRef hostName, bool value);
     void setStatisticsVeryPrevalentResource(WKStringRef hostName, bool value);
     String dumpResourceLoadStatistics();
index c4d18ab..2f8b306 100644 (file)
@@ -1100,6 +1100,36 @@ WKRetainPtr<WKTypeRef> TestInvocation::didReceiveSynchronousMessageFromInjectedB
         return nullptr;
     }
     
+    if (WKStringIsEqualToUTF8CString(messageName, "SetStatisticsMergeStatistic")) {
+        ASSERT(WKGetTypeID(messageBody) == WKDictionaryGetTypeID());
+
+        WKDictionaryRef messageBodyDictionary = static_cast<WKDictionaryRef>(messageBody);
+        WKRetainPtr<WKStringRef> hostNameKey = adoptWK(WKStringCreateWithUTF8CString("HostName"));
+        WKRetainPtr<WKStringRef> topFrameDomainKey = adoptWK(WKStringCreateWithUTF8CString("TopFrameDomain"));
+        WKRetainPtr<WKStringRef> lastSeenKey = adoptWK(WKStringCreateWithUTF8CString("LastSeen"));
+        WKRetainPtr<WKStringRef> hadUserInteractionKey = adoptWK(WKStringCreateWithUTF8CString("HadUserInteraction"));
+        WKRetainPtr<WKStringRef> mostRecentUserInteractionKey = adoptWK(WKStringCreateWithUTF8CString("MostRecentUserInteraction"));
+        WKRetainPtr<WKStringRef> isGrandfatheredKey = adoptWK(WKStringCreateWithUTF8CString("IsGrandfathered"));
+        WKRetainPtr<WKStringRef> isPrevalentKey = adoptWK(WKStringCreateWithUTF8CString("IsPrevalent"));
+        WKRetainPtr<WKStringRef> isVeryPrevalentKey = adoptWK(WKStringCreateWithUTF8CString("IsVeryPrevalent"));
+        WKRetainPtr<WKStringRef> dataRecordsRemovedKey = adoptWK(WKStringCreateWithUTF8CString("DataRecordsRemoved"));
+        WKRetainPtr<WKStringRef> timesAccessedFirstPartyInteractionKey = adoptWK(WKStringCreateWithUTF8CString("TimesAccessedFirstPartyInteraction"));
+
+        WKStringRef hostName = static_cast<WKStringRef>(WKDictionaryGetItemForKey(messageBodyDictionary, hostNameKey.get()));
+        WKStringRef topFrameDomain = static_cast<WKStringRef>(WKDictionaryGetItemForKey(messageBodyDictionary, topFrameDomainKey.get()));
+        WKDoubleRef lastSeen = static_cast<WKDoubleRef>(WKDictionaryGetItemForKey(messageBodyDictionary, lastSeenKey.get()));
+        WKBooleanRef hadUserInteraction = static_cast<WKBooleanRef>(WKDictionaryGetItemForKey(messageBodyDictionary, hadUserInteractionKey.get()));
+        WKDoubleRef mostRecentUserInteraction = static_cast<WKDoubleRef>(WKDictionaryGetItemForKey(messageBodyDictionary, mostRecentUserInteractionKey.get()));
+        WKBooleanRef isGrandfathered = static_cast<WKBooleanRef>(WKDictionaryGetItemForKey(messageBodyDictionary, isGrandfatheredKey.get()));
+        WKBooleanRef isPrevalent = static_cast<WKBooleanRef>(WKDictionaryGetItemForKey(messageBodyDictionary, isPrevalentKey.get()));
+        WKBooleanRef isVeryPrevalent = static_cast<WKBooleanRef>(WKDictionaryGetItemForKey(messageBodyDictionary, isVeryPrevalentKey.get()));
+        WKUInt64Ref dataRecordsRemoved = static_cast<WKUInt64Ref>(WKDictionaryGetItemForKey(messageBodyDictionary, dataRecordsRemovedKey.get()));
+        
+        TestController::singleton().setStatisticsMergeStatistic(hostName, topFrameDomain, WKDoubleGetValue(lastSeen), WKBooleanGetValue(hadUserInteraction), WKDoubleGetValue(mostRecentUserInteraction), WKBooleanGetValue(isGrandfathered), WKBooleanGetValue(isPrevalent), WKBooleanGetValue(isVeryPrevalent), WKUInt64GetValue(dataRecordsRemoved));
+
+        return nullptr;
+    }
+    
     if (WKStringIsEqualToUTF8CString(messageName, "SetStatisticsPrevalentResource")) {
         ASSERT(WKGetTypeID(messageBody) == WKDictionaryGetTypeID());
 
@@ -1801,6 +1831,12 @@ void TestInvocation::didSetLastSeen()
     WKPagePostMessageToInjectedBundle(TestController::singleton().mainWebView()->page(), messageName.get(), 0);
 }
 
+void TestInvocation::didMergeStatistic()
+{
+    WKRetainPtr<WKStringRef> messageName = adoptWK(WKStringCreateWithUTF8CString("CallDidMergeStatistic"));
+    WKPagePostMessageToInjectedBundle(TestController::singleton().mainWebView()->page(), messageName.get(), 0);
+}
+
 void TestInvocation::didSetPrevalentResource()
 {
     WKRetainPtr<WKStringRef> messageName = adoptWK(WKStringCreateWithUTF8CString("CallDidSetPrevalentResource"));
index 76aa9b8..a265eb9 100644 (file)
@@ -79,6 +79,7 @@ public:
     void didSetStatisticsDebugMode();
     void didSetPrevalentResourceForDebugMode();
     void didSetLastSeen();
+    void didMergeStatistic();
     void didSetPrevalentResource();
     void didSetVeryPrevalentResource();
     void didSetHasHadUserInteraction();