Add timeStamp to ITP database
authorkatherine_cheney@apple.com <katherine_cheney@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Sat, 21 Dec 2019 14:58:01 +0000 (14:58 +0000)
committerkatherine_cheney@apple.com <katherine_cheney@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Sat, 21 Dec 2019 14:58:01 +0000 (14:58 +0000)
https://bugs.webkit.org/show_bug.cgi?id=205121
<rdar://problem/57633021>

Reviewed by John Wilander.

Source/WebCore:

* loader/ResourceLoadStatistics.h:

Source/WebKit:

This patch adds support for collecting most-recently-updated
timestamps for third-party/first-party domain pairs in the ITP database.
It updates the timestamp when new statistics are merged into the
database. It then exposes the timestamp via the
_getResourceLoadStatisticsDataSummary API.

* NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:
(WebKit::ResourceLoadStatisticsDatabaseStore::ResourceLoadStatisticsDatabaseStore):
(WebKit::ResourceLoadStatisticsDatabaseStore::prepareStatements):
(WebKit::ResourceLoadStatisticsDatabaseStore::insertDomainRelationshipList):
Changed INSERT OR IGNORE queries to be INSERT OR REPLACE so the timestamp
will be replaced upon a new attempted insert into one of the
third-party/first-party relationship tables.
ResourceLoadStatisticsDatabaseStore::insertDomainRelationshipList now
checks for the keyword "REPLACE" to know if another bind is needed
to update the timestamp.

(WebKit::ResourceLoadStatisticsDatabaseStore::getMostRecentlyUpdatedTimestamp):
Queries the most recent time that the third party has appeared as a
subframe or subresource under the first party or redirected to the first party.

* NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h:
* NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp:
* NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h:
(WebKit::ThirdPartyDataForSpecificFirstParty::toString const):
Updated the toString to check if the timestamp occured in the last
24 hours for testing purposes. It doesn't print the specific time
because it would change for every run and could not be tested.

* UIProcess/API/Cocoa/WKWebsiteDataStore.mm:
(-[WKWebsiteDataStore _setUseITPDatabase:completionHandler:]):
Added new function that enables the ITP Database backend so the
timestamp parameter can be tested in in TestWebKitAPI.

* UIProcess/API/APIResourceLoadStatisticsFirstParty.h:
* UIProcess/API/Cocoa/WKWebsiteDataStorePrivate.h:
* UIProcess/API/Cocoa/_WKResourceLoadStatisticsFirstParty.h:
* UIProcess/API/Cocoa/_WKResourceLoadStatisticsFirstParty.mm:
(-[_WKResourceLoadStatisticsFirstParty timeLastUpdated]):
Added the new timestamp parameter to the _WKResourceLoadStatisticsFirstParty.mm
class and its wrapper to be sent via API call.

Tools:

Added checks to test that the timestamp is properly exposed via API
in the ITP database backend. This also adds an API test case using the
ITP database store. It was previously only testing the ITP memory store.

* TestWebKitAPI/Tests/WebKitCocoa/ResourceLoadStatistics.mm:
(TEST):

LayoutTests:

This patch updates test expectations which call
dumpResourceLoadStatistics to reflect the new timestamp parameter that
is now printed with the ITP data summary.

* http/tests/resourceLoadStatistics/aggregate-sorted-data-no-storage-access-database-expected.txt:
* http/tests/resourceLoadStatistics/aggregate-sorted-data-no-storage-access-expected.txt:
* http/tests/storageAccess/aggregate-sorted-data-with-storage-access-database-expected.txt:
* http/tests/storageAccess/aggregate-sorted-data-with-storage-access-expected.txt:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@253863 268f45cc-cd09-0410-ab3c-d52691b4dbfc

19 files changed:
LayoutTests/ChangeLog
LayoutTests/http/tests/resourceLoadStatistics/aggregate-sorted-data-no-storage-access-database-expected.txt
LayoutTests/http/tests/resourceLoadStatistics/aggregate-sorted-data-no-storage-access-expected.txt
LayoutTests/http/tests/storageAccess/aggregate-sorted-data-with-storage-access-database-expected.txt
LayoutTests/http/tests/storageAccess/aggregate-sorted-data-with-storage-access-expected.txt
Source/WebCore/ChangeLog
Source/WebCore/loader/ResourceLoadStatistics.h
Source/WebKit/ChangeLog
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp
Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h
Source/WebKit/UIProcess/API/APIResourceLoadStatisticsFirstParty.h
Source/WebKit/UIProcess/API/Cocoa/WKWebsiteDataStore.mm
Source/WebKit/UIProcess/API/Cocoa/WKWebsiteDataStorePrivate.h
Source/WebKit/UIProcess/API/Cocoa/_WKResourceLoadStatisticsFirstParty.h
Source/WebKit/UIProcess/API/Cocoa/_WKResourceLoadStatisticsFirstParty.mm
Tools/ChangeLog
Tools/TestWebKitAPI/Tests/WebKitCocoa/ResourceLoadStatistics.mm

index 3ef7643..29d742b 100644 (file)
@@ -1,3 +1,20 @@
+2019-12-21  Kate Cheney  <katherine_cheney@apple.com>
+
+        Add timeStamp to ITP database
+        https://bugs.webkit.org/show_bug.cgi?id=205121
+        <rdar://problem/57633021>
+
+        Reviewed by John Wilander.
+
+        This patch updates test expectations which call
+        dumpResourceLoadStatistics to reflect the new timestamp parameter that
+        is now printed with the ITP data summary.
+
+        * http/tests/resourceLoadStatistics/aggregate-sorted-data-no-storage-access-database-expected.txt:
+        * http/tests/resourceLoadStatistics/aggregate-sorted-data-no-storage-access-expected.txt:
+        * http/tests/storageAccess/aggregate-sorted-data-with-storage-access-database-expected.txt:
+        * http/tests/storageAccess/aggregate-sorted-data-with-storage-access-expected.txt:
+
 2019-12-20  Chris Dumez  <cdumez@apple.com>
 
         [iOS Debug] imported/w3c/web-platform-tests/html/dom/usvstring-reflection.https.html is crashing
 
 2019-11-20  Kate Cheney  <katherine_cheney@apple.com>
 
-        [ Jazz ] http/tests/resourceLoadStatistics/cookie-deletion.html is timing out
+        [ MacOS ] http/tests/resourceLoadStatistics/cookie-deletion.html is timing out
         https://bugs.webkit.org/show_bug.cgi?id=203813
         <rdar://problem/54316765>
 
 
 2019-11-19  Kate Cheney  <katherine_cheney@apple.com>
 
-        [ Jazz ] http/tests/resourceLoadStatistics/cookie-deletion.html is timing out
+        [ MacOS ] http/tests/resourceLoadStatistics/cookie-deletion.html is timing out
         https://bugs.webkit.org/show_bug.cgi?id=203813
         <rdar://problem/54316765>
 
index 26dac94..1ebd42c 100644 (file)
@@ -76,8 +76,8 @@ Registrable domain: topframe4
 
 ITP Data:
 Third Party Registrable Domain: subframe3
-    {{ Has been granted storage access under topframe1: 0 },{ Has been granted storage access under topframe4: 0 },{ Has been granted storage access under topframe3: 0 },{ Has been granted storage access under topframe2: 0 },}
+    {{ Has been granted storage access under topframe1: 0; Has been seen under topframe1 in the last 24 hours: 1 },{ Has been granted storage access under topframe4: 0; Has been seen under topframe4 in the last 24 hours: 1 },{ Has been granted storage access under topframe3: 0; Has been seen under topframe3 in the last 24 hours: 1 },{ Has been granted storage access under topframe2: 0; Has been seen under topframe2 in the last 24 hours: 1 },}
 Third Party Registrable Domain: subframe1
-    {{ Has been granted storage access under topframe1: 0 },{ Has been granted storage access under topframe2: 0 },}
+    {{ Has been granted storage access under topframe1: 0; Has been seen under topframe1 in the last 24 hours: 1 },{ Has been granted storage access under topframe2: 0; Has been seen under topframe2 in the last 24 hours: 1 },}
 Third Party Registrable Domain: subframe2
-    {{ Has been granted storage access under topframe1: 0 },}
+    {{ Has been granted storage access under topframe1: 0; Has been seen under topframe1 in the last 24 hours: 1 },}
index 2d84376..65d75ea 100644 (file)
@@ -76,8 +76,8 @@ Registrable domain: subframe3
 
 ITP Data:
 Third Party Registrable Domain: subframe3
-    {{ Has been granted storage access under topframe1: 0 },{ Has been granted storage access under topframe4: 0 },{ Has been granted storage access under topframe3: 0 },{ Has been granted storage access under topframe2: 0 },}
+    {{ Has been granted storage access under topframe1: 0; Has been seen under topframe1 in the last 24 hours: 0 },{ Has been granted storage access under topframe4: 0; Has been seen under topframe4 in the last 24 hours: 0 },{ Has been granted storage access under topframe3: 0; Has been seen under topframe3 in the last 24 hours: 0 },{ Has been granted storage access under topframe2: 0; Has been seen under topframe2 in the last 24 hours: 0 },}
 Third Party Registrable Domain: subframe1
-    {{ Has been granted storage access under topframe1: 0 },{ Has been granted storage access under topframe2: 0 },}
+    {{ Has been granted storage access under topframe1: 0; Has been seen under topframe1 in the last 24 hours: 0 },{ Has been granted storage access under topframe2: 0; Has been seen under topframe2 in the last 24 hours: 0 },}
 Third Party Registrable Domain: subframe2
-    {{ Has been granted storage access under topframe1: 0 },}
+    {{ Has been granted storage access under topframe1: 0; Has been seen under topframe1 in the last 24 hours: 0 },}
index bbda8d9..2f96b28 100644 (file)
@@ -33,4 +33,4 @@ Registrable domain: localhost
 
 ITP Data:
 Third Party Registrable Domain: localhost
-    {{ Has been granted storage access under 127.0.0.1: 1 },}
+    {{ Has been granted storage access under 127.0.0.1: 1; Has been seen under 127.0.0.1 in the last 24 hours: 1 },}
index 9ae26dd..26a7c39 100644 (file)
@@ -33,4 +33,4 @@ Registrable domain: 127.0.0.1
 
 ITP Data:
 Third Party Registrable Domain: localhost
-    {{ Has been granted storage access under 127.0.0.1: 1 },}
+    {{ Has been granted storage access under 127.0.0.1: 1; Has been seen under 127.0.0.1 in the last 24 hours: 0 },}
index 9beaa81..7236a1e 100644 (file)
@@ -1,3 +1,13 @@
+2019-12-21  Kate Cheney  <katherine_cheney@apple.com>
+
+        Add timeStamp to ITP database
+        https://bugs.webkit.org/show_bug.cgi?id=205121
+        <rdar://problem/57633021>
+
+        Reviewed by John Wilander.
+
+        * loader/ResourceLoadStatistics.h:
+
 2019-12-20  Eric Carlson  <eric.carlson@apple.com>
 
         [Media in GPU process] Get audio playing
index 7e2653d..78e216a 100644 (file)
@@ -53,6 +53,8 @@ struct ResourceLoadStatistics {
     ResourceLoadStatistics(ResourceLoadStatistics&&) = default;
     ResourceLoadStatistics& operator=(ResourceLoadStatistics&&) = default;
 
+    static constexpr Seconds NoExistingTimestamp { -1 };
+    
     WEBCORE_EXPORT static WallTime reduceTimeResolution(WallTime);
 
     WEBCORE_EXPORT void encode(KeyedEncoder&) const;
index fe07805..27a9b02 100644 (file)
@@ -1,3 +1,53 @@
+2019-12-21  Kate Cheney  <katherine_cheney@apple.com>
+
+        Add timeStamp to ITP database
+        https://bugs.webkit.org/show_bug.cgi?id=205121
+        <rdar://problem/57633021>
+
+        Reviewed by John Wilander.
+
+        This patch adds support for collecting most-recently-updated
+        timestamps for third-party/first-party domain pairs in the ITP database.
+        It updates the timestamp when new statistics are merged into the
+        database. It then exposes the timestamp via the
+        _getResourceLoadStatisticsDataSummary API.
+
+        * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:
+        (WebKit::ResourceLoadStatisticsDatabaseStore::ResourceLoadStatisticsDatabaseStore):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::prepareStatements):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::insertDomainRelationshipList):
+        Changed INSERT OR IGNORE queries to be INSERT OR REPLACE so the timestamp
+        will be replaced upon a new attempted insert into one of the
+        third-party/first-party relationship tables.
+        ResourceLoadStatisticsDatabaseStore::insertDomainRelationshipList now
+        checks for the keyword "REPLACE" to know if another bind is needed
+        to update the timestamp.
+
+        (WebKit::ResourceLoadStatisticsDatabaseStore::getMostRecentlyUpdatedTimestamp):
+        Queries the most recent time that the third party has appeared as a
+        subframe or subresource under the first party or redirected to the first party.
+
+        * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h:
+        * NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp:
+        * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h:
+        (WebKit::ThirdPartyDataForSpecificFirstParty::toString const):
+        Updated the toString to check if the timestamp occured in the last
+        24 hours for testing purposes. It doesn't print the specific time
+        because it would change for every run and could not be tested.
+
+        * UIProcess/API/Cocoa/WKWebsiteDataStore.mm:
+        (-[WKWebsiteDataStore _setUseITPDatabase:completionHandler:]):
+        Added new function that enables the ITP Database backend so the
+        timestamp parameter can be tested in in TestWebKitAPI.
+
+        * UIProcess/API/APIResourceLoadStatisticsFirstParty.h:
+        * UIProcess/API/Cocoa/WKWebsiteDataStorePrivate.h:
+        * UIProcess/API/Cocoa/_WKResourceLoadStatisticsFirstParty.h:
+        * UIProcess/API/Cocoa/_WKResourceLoadStatisticsFirstParty.mm:
+        (-[_WKResourceLoadStatisticsFirstParty timeLastUpdated]):
+        Added the new timestamp parameter to the _WKResourceLoadStatisticsFirstParty.mm
+        class and its wrapper to be sent via API call.
+
 2019-12-20  Eric Carlson  <eric.carlson@apple.com>
 
         [Media in GPU process] Get audio playing
index 0bc3579..6541a4c 100644 (file)
@@ -73,21 +73,23 @@ constexpr auto countPrevalentResourcesWithUserInteractionQuery = "SELECT COUNT(D
 
 constexpr auto countPrevalentResourcesWithoutUserInteractionQuery = "SELECT COUNT(DISTINCT registrableDomain) FROM ObservedDomains WHERE isPrevalent = 1 AND hadUserInteraction = 0;"_s;
 
-// INSERT Queries
+// INSERT OR IGNORE Queries
 constexpr auto insertObservedDomainQuery = "INSERT INTO ObservedDomains (registrableDomain, lastSeen, hadUserInteraction,"
     "mostRecentUserInteractionTime, grandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, timesAccessedAsFirstPartyDueToUserInteraction,"
     "timesAccessedAsFirstPartyDueToStorageAccessAPI, isScheduledForAllButCookieDataRemoval) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"_s;
 constexpr auto insertTopLevelDomainQuery = "INSERT INTO TopLevelDomains VALUES (?)"_s;
 constexpr auto storageAccessUnderTopFrameDomainsQuery = "INSERT OR IGNORE INTO StorageAccessUnderTopFrameDomains (domainID, topLevelDomainID) SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain in ( "_s;
 constexpr auto topFrameUniqueRedirectsToQuery = "INSERT OR IGNORE into TopFrameUniqueRedirectsTo (sourceDomainID, toDomainID) SELECT ?, domainID FROM ObservedDomains where registrableDomain in ( "_s;
-constexpr auto subframeUnderTopFrameDomainsQuery = "INSERT OR IGNORE into SubframeUnderTopFrameDomains (subFrameDomainID, topFrameDomainID) SELECT ?, domainID FROM ObservedDomains where registrableDomain in ( "_s;
 constexpr auto topFrameUniqueRedirectsFromQuery = "INSERT OR IGNORE INTO TopFrameUniqueRedirectsFrom (targetDomainID, fromDomainID) SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain in ( "_s;
-constexpr auto topFrameLinkDecorationsFromQuery = "INSERT OR IGNORE INTO TopFrameLinkDecorationsFrom (toDomainID, fromDomainID) SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain in ( "_s;
 constexpr auto topFrameLoadedThirdPartyScriptsQuery = "INSERT OR IGNORE into TopFrameLoadedThirdPartyScripts (topFrameDomainID, subresourceDomainID) SELECT ?, domainID FROM ObservedDomains where registrableDomain in ( "_s;
-constexpr auto subresourceUnderTopFrameDomainsQuery = "INSERT OR IGNORE INTO SubresourceUnderTopFrameDomains (subresourceDomainID, topFrameDomainID) SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain in ( "_s;
-constexpr auto subresourceUniqueRedirectsToQuery = "INSERT OR IGNORE INTO SubresourceUniqueRedirectsTo (subresourceDomainID, toDomainID) SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain in ( "_s;
 constexpr auto subresourceUniqueRedirectsFromQuery = "INSERT OR IGNORE INTO SubresourceUniqueRedirectsFrom (subresourceDomainID, fromDomainID) SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain in ( "_s;
 
+// INSERT OR REPLACE Queries
+constexpr auto subframeUnderTopFrameDomainsQuery = "INSERT OR REPLACE into SubframeUnderTopFrameDomains (subFrameDomainID, lastUpdated, topFrameDomainID) SELECT ?, ?, domainID FROM ObservedDomains where registrableDomain in ( "_s;
+constexpr auto topFrameLinkDecorationsFromQuery = "INSERT OR REPLACE INTO TopFrameLinkDecorationsFrom (toDomainID, lastUpdated, fromDomainID) SELECT ?, ?, domainID FROM ObservedDomains WHERE registrableDomain in ( "_s;
+constexpr auto subresourceUnderTopFrameDomainsQuery = "INSERT OR REPLACE INTO SubresourceUnderTopFrameDomains (subresourceDomainID, lastUpdated, topFrameDomainID) SELECT ?, ?, domainID FROM ObservedDomains WHERE registrableDomain in ( "_s;
+constexpr auto subresourceUniqueRedirectsToQuery = "INSERT OR REPLACE INTO SubresourceUniqueRedirectsTo (subresourceDomainID, lastUpdated, toDomainID) SELECT ?, ?, domainID FROM ObservedDomains WHERE registrableDomain in ( "_s;
+
 // EXISTS Queries
 constexpr auto subframeUnderTopFrameDomainExistsQuery = "SELECT EXISTS (SELECT 1 FROM SubframeUnderTopFrameDomains WHERE subFrameDomainID = ? "
     "AND topFrameDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
@@ -121,6 +123,10 @@ constexpr auto hadUserInteractionQuery = "SELECT hadUserInteraction, mostRecentU
 constexpr auto isGrandfatheredQuery = "SELECT grandfathered FROM ObservedDomains WHERE registrableDomain = ?"_s;
 constexpr auto findExpiredUserInteractionQuery = "SELECT domainID FROM ObservedDomains WHERE hadUserInteraction = 1 AND mostRecentUserInteractionTime < ?"_s;
 constexpr auto getResourceDataByDomainNameQuery = "SELECT * FROM ObservedDomains WHERE registrableDomain = ?";
+constexpr auto getMostRecentlyUpdatedTimestampQuery = "SELECT MAX(lastUpdated) FROM (SELECT lastUpdated FROM SubframeUnderTopFrameDomains WHERE subFrameDomainID = ? and topFrameDomainID = ?"
+    "UNION ALL SELECT lastUpdated FROM TopFrameLinkDecorationsFrom WHERE toDomainID = ? and fromDomainID = ?"
+    "UNION ALL SELECT lastUpdated FROM SubresourceUnderTopFrameDomains WHERE subresourceDomainID = ? and topFrameDomainID = ?"
+    "UNION ALL SELECT lastUpdated FROM SubresourceUniqueRedirectsTo WHERE subresourceDomainID = ? and toDomainID = ?)"_s;
 constexpr auto getAllDomainsQuery = "SELECT registrableDomain FROM ObservedDomains"_s;
 constexpr auto getAllSubStatisticsUnderDomainQuery = "SELECT topFrameDomainID FROM SubframeUnderTopFrameDomains WHERE subFrameDomainID = ?"
     "UNION ALL SELECT topFrameDomainID FROM SubresourceUnderTopFrameDomains WHERE subresourceDomainID = ?"
@@ -183,7 +189,7 @@ constexpr auto createTopFrameUniqueRedirectsFrom = "CREATE TABLE TopFrameUniqueR
     "FOREIGN KEY(fromDomainID) REFERENCES TopLevelDomains(topLevelDomainID) ON DELETE CASCADE);"_s;
 
 constexpr auto createTopFrameLinkDecorationsFrom = "CREATE TABLE TopFrameLinkDecorationsFrom ("
-    "toDomainID INTEGER NOT NULL, fromDomainID INTEGER NOT NULL, "
+    "toDomainID INTEGER NOT NULL, lastUpdated REAL NOT NULL, fromDomainID INTEGER NOT NULL, "
     "FOREIGN KEY(toDomainID) REFERENCES TopLevelDomains(topLevelDomainID) ON DELETE CASCADE, "
     "FOREIGN KEY(fromDomainID) REFERENCES TopLevelDomains(topLevelDomainID) ON DELETE CASCADE);"_s;
 
@@ -193,17 +199,17 @@ constexpr auto createTopFrameLoadedThirdPartyScripts = "CREATE TABLE TopFrameLoa
     "FOREIGN KEY(subresourceDomainID) REFERENCES ObservedDomains(domainID) ON DELETE CASCADE);"_s;
 
 constexpr auto createSubframeUnderTopFrameDomains = "CREATE TABLE SubframeUnderTopFrameDomains ("
-    "subFrameDomainID INTEGER NOT NULL, topFrameDomainID INTEGER NOT NULL, "
+    "subFrameDomainID INTEGER NOT NULL, lastUpdated REAL NOT NULL, topFrameDomainID INTEGER NOT NULL, "
     "FOREIGN KEY(subFrameDomainID) REFERENCES ObservedDomains(domainID) ON DELETE CASCADE, "
     "FOREIGN KEY(topFrameDomainID) REFERENCES TopLevelDomains(topLevelDomainID) ON DELETE CASCADE);"_s;
     
 constexpr auto createSubresourceUnderTopFrameDomains = "CREATE TABLE SubresourceUnderTopFrameDomains ("
-    "subresourceDomainID INTEGER NOT NULL, topFrameDomainID INTEGER NOT NULL, "
+    "subresourceDomainID INTEGER NOT NULL, lastUpdated REAL NOT NULL, topFrameDomainID INTEGER NOT NULL, "
     "FOREIGN KEY(subresourceDomainID) REFERENCES ObservedDomains(domainID) ON DELETE CASCADE, "
     "FOREIGN KEY(topFrameDomainID) REFERENCES TopLevelDomains(topLevelDomainID) ON DELETE CASCADE);"_s;
     
 constexpr auto createSubresourceUniqueRedirectsTo = "CREATE TABLE SubresourceUniqueRedirectsTo ("
-    "subresourceDomainID INTEGER NOT NULL, toDomainID INTEGER NOT NULL, "
+    "subresourceDomainID INTEGER NOT NULL, lastUpdated REAL NOT NULL, toDomainID INTEGER NOT NULL, "
     "FOREIGN KEY(subresourceDomainID) REFERENCES ObservedDomains(domainID) ON DELETE CASCADE, "
     "FOREIGN KEY(toDomainID) REFERENCES ObservedDomains(domainID) ON DELETE CASCADE);"_s;
     
@@ -268,6 +274,7 @@ ResourceLoadStatisticsDatabaseStore::ResourceLoadStatisticsDatabaseStore(WebReso
     , m_domainStringFromDomainIDStatement(m_database, domainStringFromDomainIDQuery)
     , m_getAllSubStatisticsStatement(m_database, getAllSubStatisticsUnderDomainQuery)
     , m_storageAccessExistsStatement(m_database, storageAccessExistsQuery)
+    , m_getMostRecentlyUpdatedTimestampStatement(m_database, getMostRecentlyUpdatedTimestampQuery)
     , m_sessionID(sessionID)
 {
     ASSERT(!RunLoop::isMain());
@@ -511,6 +518,7 @@ bool ResourceLoadStatisticsDatabaseStore::prepareStatements()
         || m_domainStringFromDomainIDStatement.prepare() != SQLITE_OK
         || m_getAllSubStatisticsStatement.prepare() != SQLITE_OK
         || m_storageAccessExistsStatement.prepare() != SQLITE_OK
+        || m_getMostRecentlyUpdatedTimestampStatement.prepare() != SQLITE_OK
         ) {
         RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::prepareStatements failed to prepare, error message: %{public}s", this, m_database.lastErrorMsg());
         ASSERT_NOT_REACHED();
@@ -647,10 +655,23 @@ void ResourceLoadStatisticsDatabaseStore::insertDomainRelationshipList(const Str
     SQLiteStatement insertRelationshipStatement(m_database, makeString(statement, ensureAndMakeDomainList(domainList), " );"));
     
     if (insertRelationshipStatement.prepare() != SQLITE_OK
-        || insertRelationshipStatement.bindInt(1, domainID) != SQLITE_OK
-        || insertRelationshipStatement.step() != SQLITE_DONE) {
+        || insertRelationshipStatement.bindInt(1, domainID) != SQLITE_OK) {
+            RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::insertDomainRelationshipList failed, error message: %{private}s", this, m_database.lastErrorMsg());
+            ASSERT_NOT_REACHED();
+    }
+    
+    if (statement.contains("REPLACE")) {
+        if (insertRelationshipStatement.bindDouble(2, WallTime::now().secondsSinceEpoch().value()) != SQLITE_OK) {
+            RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::insertDomainRelationshipList failed, error message: %{private}s", this, m_database.lastErrorMsg());
+            ASSERT_NOT_REACHED();
+            return;
+        }
+    }
+
+    if (insertRelationshipStatement.step() != SQLITE_DONE) {
         RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::insertDomainRelationshipList failed, error message: %{private}s", this, m_database.lastErrorMsg());
         ASSERT_NOT_REACHED();
+        return;
     }
 }
 
@@ -797,7 +818,7 @@ Vector<WebResourceLoadStatisticsStore::ThirdPartyDataForSpecificFirstParty> Reso
     Vector<WebResourceLoadStatisticsStore::ThirdPartyDataForSpecificFirstParty> thirdPartyDataForSpecificFirstPartyDomains;
     while (m_getAllSubStatisticsStatement.step() == SQLITE_ROW) {
         RegistrableDomain firstPartyDomain = RegistrableDomain::uncheckedCreateFromRegistrableDomainString(getDomainStringFromDomainID(m_getAllSubStatisticsStatement.getColumnInt(0)));
-        thirdPartyDataForSpecificFirstPartyDomains.appendIfNotContains(WebResourceLoadStatisticsStore::ThirdPartyDataForSpecificFirstParty { firstPartyDomain, hasStorageAccess(firstPartyDomain, thirdPartyDomain) });
+        thirdPartyDataForSpecificFirstPartyDomains.appendIfNotContains(WebResourceLoadStatisticsStore::ThirdPartyDataForSpecificFirstParty { firstPartyDomain, hasStorageAccess(firstPartyDomain, thirdPartyDomain), getMostRecentlyUpdatedTimestamp(thirdPartyDomain, firstPartyDomain) });
     }
     resetStatement(m_getAllSubStatisticsStatement);
     return thirdPartyDataForSpecificFirstPartyDomains;
@@ -1808,6 +1829,36 @@ void ResourceLoadStatisticsDatabaseStore::setIsScheduledForAllButCookieDataRemov
 
     resetStatement(m_updateIsScheduledForAllButCookieDataRemovalStatement);
 }
+Seconds ResourceLoadStatisticsDatabaseStore::getMostRecentlyUpdatedTimestamp(const RegistrableDomain& subDomain, const TopFrameDomain& topFrameDomain) const
+{
+    ASSERT(!RunLoop::isMain());
+
+    Optional<unsigned> subFrameDomainID = domainID(subDomain);
+    Optional<unsigned> topFrameDomainID = domainID(topFrameDomain);
+
+    if (!subFrameDomainID || !topFrameDomainID)
+        return Seconds { ResourceLoadStatistics::NoExistingTimestamp };
+
+    if (m_getMostRecentlyUpdatedTimestampStatement.bindInt(1, *subFrameDomainID) != SQLITE_OK
+        || m_getMostRecentlyUpdatedTimestampStatement.bindInt(2, *topFrameDomainID) != SQLITE_OK
+        || m_getMostRecentlyUpdatedTimestampStatement.bindInt(3, *subFrameDomainID) != SQLITE_OK
+        || m_getMostRecentlyUpdatedTimestampStatement.bindInt(4, *topFrameDomainID) != SQLITE_OK
+        || m_getMostRecentlyUpdatedTimestampStatement.bindInt(5, *subFrameDomainID) != SQLITE_OK
+        || m_getMostRecentlyUpdatedTimestampStatement.bindInt(6, *topFrameDomainID) != SQLITE_OK
+        || m_getMostRecentlyUpdatedTimestampStatement.bindInt(7, *subFrameDomainID) != SQLITE_OK
+        || m_getMostRecentlyUpdatedTimestampStatement.bindInt(8, *topFrameDomainID) != SQLITE_OK) {
+        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::getMostRecentlyUpdatedTimestamp failed to bind, error message: %{private}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+        return Seconds { ResourceLoadStatistics::NoExistingTimestamp  };
+    }
+    if (m_getMostRecentlyUpdatedTimestampStatement.step() != SQLITE_ROW) {
+        resetStatement(m_getMostRecentlyUpdatedTimestampStatement);
+        return Seconds { ResourceLoadStatistics::NoExistingTimestamp  };
+    }
+    double mostRecentlyUpdatedTimestamp = m_getMostRecentlyUpdatedTimestampStatement.getColumnDouble(0);
+    resetStatement(m_getMostRecentlyUpdatedTimestampStatement);
+    return Seconds { mostRecentlyUpdatedTimestamp };
+}
 
 bool ResourceLoadStatisticsDatabaseStore::isGrandfathered(const RegistrableDomain& domain) const
 {
index 77e9f8f..9b310ec 100644 (file)
@@ -134,6 +134,7 @@ public:
     void setLastSeen(const RegistrableDomain&, Seconds) override;
     bool isCorrectSubStatisticsCount(const RegistrableDomain&, const TopFrameDomain&);
     void resourceToString(StringBuilder&, const String&) const;
+    Seconds getMostRecentlyUpdatedTimestamp(const RegistrableDomain&, const TopFrameDomain&) const;
 
 private:
     void openITPDatabase();
@@ -253,6 +254,7 @@ private:
     mutable WebCore::SQLiteStatement m_domainStringFromDomainIDStatement;
     mutable WebCore::SQLiteStatement m_getAllSubStatisticsStatement;
     mutable WebCore::SQLiteStatement m_storageAccessExistsStatement;
+    mutable WebCore::SQLiteStatement m_getMostRecentlyUpdatedTimestampStatement;
     PAL::SessionID m_sessionID;
 };
 
index 08f699e..89385a9 100644 (file)
@@ -105,7 +105,7 @@ void ResourceLoadStatisticsMemoryStore::calculateAndSubmitTelemetry() const
 
 static void ensureThirdPartyDataForSpecificFirstPartyDomain(Vector<WebResourceLoadStatisticsStore::ThirdPartyDataForSpecificFirstParty>& thirdPartyDataForSpecificFirstPartyDomain, const RegistrableDomain& firstPartyDomain, bool thirdPartyHasStorageAccess)
 {
-    WebResourceLoadStatisticsStore::ThirdPartyDataForSpecificFirstParty thirdPartyDataForSpecificFirstParty { firstPartyDomain, thirdPartyHasStorageAccess };
+    WebResourceLoadStatisticsStore::ThirdPartyDataForSpecificFirstParty thirdPartyDataForSpecificFirstParty { firstPartyDomain, thirdPartyHasStorageAccess, Seconds { ResourceLoadStatistics::NoExistingTimestamp }};
     thirdPartyDataForSpecificFirstPartyDomain.appendIfNotContains(thirdPartyDataForSpecificFirstParty);
 }
 
index 0de9594..2e6a93f 100644 (file)
@@ -108,16 +108,18 @@ public:
 struct ThirdPartyDataForSpecificFirstParty {
     WebCore::RegistrableDomain firstPartyDomain;
     bool storageAccessGranted;
+    Seconds timeLastUpdated;
 
     String toString() const
     {
-        return makeString("Has been granted storage access under ", firstPartyDomain.string(), ": ", storageAccessGranted ? '1' : '0');
+        return makeString("Has been granted storage access under ", firstPartyDomain.string(), ": ", storageAccessGranted ? '1' : '0', "; Has been seen under ", firstPartyDomain.string(), " in the last 24 hours: ", WallTime::now().secondsSinceEpoch() - timeLastUpdated < 24_h ? '1' : '0');
     }
 
     void encode(IPC::Encoder& encoder) const
     {
         encoder << firstPartyDomain;
         encoder << storageAccessGranted;
+        encoder << timeLastUpdated;
     }
 
     static Optional<ThirdPartyDataForSpecificFirstParty> decode(IPC::Decoder& decoder)
@@ -131,8 +133,13 @@ struct ThirdPartyDataForSpecificFirstParty {
         decoder >> decodedStorageAccess;
         if (!decodedStorageAccess)
             return WTF::nullopt;
-
-        return {{ WTFMove(*decodedDomain), WTFMove(*decodedStorageAccess) }};
+        
+        Optional<Seconds> decodedTimeLastUpdated;
+        decoder >> decodedTimeLastUpdated;
+        if (!decodedTimeLastUpdated)
+            return WTF::nullopt;
+        
+        return {{ WTFMove(*decodedDomain), WTFMove(*decodedStorageAccess), WTFMove(*decodedTimeLastUpdated) }};
     }
 
     bool operator==(ThirdPartyDataForSpecificFirstParty const other) const
index da11c23..c8e2858 100644 (file)
@@ -45,6 +45,7 @@ public:
 
     const WTF::String& firstPartyDomain() const { return m_firstPartyData.firstPartyDomain.string(); }
     bool storageAccess() const { return m_firstPartyData.storageAccessGranted; }
+    double timeLastUpdated() const { return m_firstPartyData.timeLastUpdated.value(); }
 
 private:
     const WebKit::WebResourceLoadStatisticsStore::ThirdPartyDataForSpecificFirstParty m_firstPartyData;
index 538373d..6cb574e 100644 (file)
@@ -281,6 +281,17 @@ static Vector<WebKit::WebsiteDataRecord> toWebsiteDataRecords(NSArray *dataRecor
     _websiteDataStore->setResourceLoadStatisticsEnabled(enabled);
 }
 
+- (void)_setUseITPDatabase:(BOOL)enabled completionHandler:(void (^)(void))completionHandler
+{
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+    _websiteDataStore->setUseITPDatabase(enabled, [completionHandler = makeBlockPtr(completionHandler)]() {
+        completionHandler();
+    });
+#else
+    completionHandler();
+#endif
+}
+
 - (BOOL)_resourceLoadStatisticsDebugMode
 {
 #if ENABLE(RESOURCE_LOAD_STATISTICS)
index bf95125..1828510 100644 (file)
@@ -73,6 +73,7 @@ typedef NS_OPTIONS(NSUInteger, _WKWebsiteDataStoreFetchOptions) {
 - (void)_setPrevalentDomain:(NSURL *)domain completionHandler:(void (^)(void))completionHandler WK_API_AVAILABLE(macos(10.15), ios(13.0));
 - (void)_getIsPrevalentDomain:(NSURL *)domain completionHandler:(void (^)(BOOL))completionHandler WK_API_AVAILABLE(macos(10.15), ios(13.0));
 - (void)_getResourceLoadStatisticsDataSummary:(void (^)(NSArray<_WKResourceLoadStatisticsThirdParty *> *))completionHandler WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
+- (void)_setUseITPDatabase:(BOOL)enabled completionHandler:(void (^)(void))completionHandler WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
 - (void)_clearPrevalentDomain:(NSURL *)domain completionHandler:(void (^)(void))completionHandler WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
 - (void)_clearResourceLoadStatistics:(void (^)(void))completionHandler WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
 - (void)_isRegisteredAsSubresourceUnderFirstParty:(NSURL *)firstPartyURL thirdParty:(NSURL *)thirdPartyURL completionHandler:(void (^)(BOOL))completionHandler WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
index cd6f477..e49024e 100644 (file)
@@ -32,6 +32,7 @@ NS_ASSUME_NONNULL_BEGIN
 
 @property (nonatomic, readonly) NSString *firstPartyDomain;
 @property (nonatomic, readonly) BOOL thirdPartyStorageAccessGranted;
+@property (nonatomic, readonly) NSTimeInterval timeLastUpdated;
 
 @end
 
index 1cfaac8..e463562 100644 (file)
     return _firstParty->storageAccess();
 }
 
+- (NSTimeInterval)timeLastUpdated
+{
+    return _firstParty->timeLastUpdated();
+}
+
 - (API::Object&)_apiObject
 {
     return *_firstParty;
index 7720660..6a53afc 100644 (file)
@@ -1,3 +1,18 @@
+2019-12-21  Kate Cheney  <katherine_cheney@apple.com>
+
+        Add timeStamp to ITP database
+        https://bugs.webkit.org/show_bug.cgi?id=205121
+        <rdar://problem/57633021>
+
+        Reviewed by John Wilander.
+
+        Added checks to test that the timestamp is properly exposed via API
+        in the ITP database backend. This also adds an API test case using the
+        ITP database store. It was previously only testing the ITP memory store.
+
+        * TestWebKitAPI/Tests/WebKitCocoa/ResourceLoadStatistics.mm:
+        (TEST):
+
 2019-12-20  Ryosuke Niwa  <rniwa@webkit.org>
 
         TextManipulationController should respect new token orders
index daf6109..2d279fb 100644 (file)
@@ -43,6 +43,7 @@ static bool finishedNavigation = false;
 @interface _WKResourceLoadStatisticsFirstParty : NSObject
 @property (nonatomic, readonly) NSString *firstPartyDomain;
 @property (nonatomic, readonly) BOOL thirdPartyStorageAccessGranted;
+@property (nonatomic, readonly) NSTimeInterval timeLastUpdated;
 @end
 
 @interface _WKResourceLoadStatisticsThirdParty : NSObject
@@ -866,3 +867,265 @@ TEST(ResourceLoadStatistics, GetResourceLoadStatisticsDataSummary)
 
     TestWebKitAPI::Util::run(&doneFlag);
 }
+
+TEST(ResourceLoadStatistics, GetResourceLoadStatisticsDataSummaryDatabase)
+{
+    auto *sharedProcessPool = [WKProcessPool _sharedProcessPool];
+    auto *dataStore = [WKWebsiteDataStore defaultDataStore];
+
+    auto configuration = adoptNS([[WKWebViewConfiguration alloc] init]);
+    [configuration setProcessPool: sharedProcessPool];
+    configuration.get().websiteDataStore = dataStore;
+
+    auto webView1 = adoptNS([[WKWebView alloc] initWithFrame:NSMakeRect(0, 0, 800, 600) configuration:configuration.get()]);
+    auto webView2 = adoptNS([[WKWebView alloc] initWithFrame:NSMakeRect(0, 0, 800, 600) configuration:configuration.get()]);
+    auto webView3 = adoptNS([[WKWebView alloc] initWithFrame:NSMakeRect(0, 0, 800, 600) configuration:configuration.get()]);
+
+    [webView1 loadHTMLString:@"WebKit Test" baseURL:[NSURL URLWithString:@"http://webkit.org"]];
+    [webView1 _test_waitForDidFinishNavigation];
+    [webView2 loadHTMLString:@"WebKit Test" baseURL:[NSURL URLWithString:@"http://webkit2.org"]];
+    [webView2 _test_waitForDidFinishNavigation];
+    [webView3 loadHTMLString:@"WebKit Test" baseURL:[NSURL URLWithString:@"http://webkit3.org"]];
+    [webView3 _test_waitForDidFinishNavigation];
+
+    [dataStore _setResourceLoadStatisticsEnabled:YES];
+
+    static bool doneFlag = false;
+    [dataStore _setUseITPDatabase:true completionHandler: ^(void) {
+        doneFlag = true;
+    }];
+
+    TestWebKitAPI::Util::run(&doneFlag);
+
+    doneFlag = false;
+    [dataStore _clearResourceLoadStatistics:^(void) {
+        doneFlag = true;
+    }];
+
+    static bool statisticsUpdated = false;
+    [dataStore _setResourceLoadStatisticsTestingCallback:^(WKWebsiteDataStore *, NSString *message) {
+        if (![message isEqualToString:@"Statistics Updated"])
+            return;
+        statisticsUpdated = true;
+    }];
+
+    TestWebKitAPI::Util::run(&doneFlag);
+
+    // Teach ITP about bad origins.
+    doneFlag = false;
+    [dataStore _setPrevalentDomain:[NSURL URLWithString:@"http://evil1.com"] completionHandler: ^(void) {
+        doneFlag = true;
+    }];
+
+    TestWebKitAPI::Util::run(&doneFlag);
+
+    doneFlag = false;
+    [dataStore _setPrevalentDomain:[NSURL URLWithString:@"http://evil2.com"] completionHandler: ^(void) {
+        doneFlag = true;
+    }];
+
+    TestWebKitAPI::Util::run(&doneFlag);
+
+    doneFlag = false;
+    [dataStore _setPrevalentDomain:[NSURL URLWithString:@"http://evil3.com"] completionHandler: ^(void) {
+        doneFlag = true;
+    }];
+
+    TestWebKitAPI::Util::run(&doneFlag);
+
+    // Capture time for comparison later.
+    NSTimeInterval beforeUpdatingTime = [[NSDate date] timeIntervalSince1970];
+
+    // Seed test data in the web process' observer.
+
+    // evil1
+    doneFlag = false;
+    [sharedProcessPool _seedResourceLoadStatisticsForTestingWithFirstParty:[NSURL URLWithString:@"http://webkit.org"] thirdParty:[NSURL URLWithString:@"http://evil1.com"] shouldScheduleNotification:NO completionHandler: ^() {
+        doneFlag = true;
+    }];
+    TestWebKitAPI::Util::run(&doneFlag);
+
+    doneFlag = false;
+    [sharedProcessPool _seedResourceLoadStatisticsForTestingWithFirstParty:[NSURL URLWithString:@"http://webkit2.org"] thirdParty:[NSURL URLWithString:@"http://evil1.com"] shouldScheduleNotification:NO completionHandler: ^() {
+        doneFlag = true;
+    }];
+    TestWebKitAPI::Util::run(&doneFlag);
+
+    // evil2
+    doneFlag = false;
+    [sharedProcessPool _seedResourceLoadStatisticsForTestingWithFirstParty:[NSURL URLWithString:@"http://webkit.org"] thirdParty:[NSURL URLWithString:@"http://evil2.com"] shouldScheduleNotification:NO completionHandler: ^() {
+        doneFlag = true;
+    }];
+    TestWebKitAPI::Util::run(&doneFlag);
+
+    // evil3
+    doneFlag = false;
+    [sharedProcessPool _seedResourceLoadStatisticsForTestingWithFirstParty:[NSURL URLWithString:@"http://webkit.org"] thirdParty:[NSURL URLWithString:@"http://evil3.com"] shouldScheduleNotification:NO completionHandler: ^() {
+        doneFlag = true;
+    }];
+    TestWebKitAPI::Util::run(&doneFlag);
+    doneFlag = false;
+    [sharedProcessPool _seedResourceLoadStatisticsForTestingWithFirstParty:[NSURL URLWithString:@"http://webkit2.org"] thirdParty:[NSURL URLWithString:@"http://evil3.com"] shouldScheduleNotification:NO completionHandler: ^() {
+        doneFlag = true;
+    }];
+    TestWebKitAPI::Util::run(&doneFlag);
+    doneFlag = false;
+    [sharedProcessPool _seedResourceLoadStatisticsForTestingWithFirstParty:[NSURL URLWithString:@"http://webkit3.org"] thirdParty:[NSURL URLWithString:@"http://evil3.com"] shouldScheduleNotification:NO completionHandler: ^() {
+        doneFlag = true;
+    }];
+    TestWebKitAPI::Util::run(&doneFlag);
+
+    statisticsUpdated = false;
+    [webView1 loadHTMLString:@"<body><script>close();</script></body>" baseURL:[NSURL URLWithString:@"http://webkit.org"]];
+    [webView2 loadHTMLString:@"<body><script>close();</script></body>" baseURL:[NSURL URLWithString:@"http://webkit2.org"]];
+    [webView3 loadHTMLString:@"<body><script>close();</script></body>" baseURL:[NSURL URLWithString:@"http://webkit3.org"]];
+
+    // Wait for the statistics to be updated in the network process.
+    TestWebKitAPI::Util::run(&statisticsUpdated);
+
+    // Check that the third-party evil1 is now registered as subresource.
+    doneFlag = false;
+    [dataStore _isRegisteredAsSubresourceUnderFirstParty:[NSURL URLWithString:@"http://webkit.org"] thirdParty:[NSURL URLWithString:@"http://evil1.com"] completionHandler: ^(BOOL isRegistered) {
+        EXPECT_TRUE(isRegistered);
+        doneFlag = true;
+    }];
+    TestWebKitAPI::Util::run(&doneFlag);
+    doneFlag = false;
+    [dataStore _isRegisteredAsSubresourceUnderFirstParty:[NSURL URLWithString:@"http://webkit2.org"] thirdParty:[NSURL URLWithString:@"http://evil1.com"] completionHandler: ^(BOOL isRegistered) {
+        EXPECT_TRUE(isRegistered);
+        doneFlag = true;
+    }];
+    TestWebKitAPI::Util::run(&doneFlag);
+
+    // Check that the third-party evil2 is now registered as subresource.
+    doneFlag = false;
+    [dataStore _isRegisteredAsSubresourceUnderFirstParty:[NSURL URLWithString:@"http://webkit.org"] thirdParty:[NSURL URLWithString:@"http://evil2.com"] completionHandler: ^(BOOL isRegistered) {
+        EXPECT_TRUE(isRegistered);
+        doneFlag = true;
+    }];
+    TestWebKitAPI::Util::run(&doneFlag);
+
+    // Check that the third-party evil3 is now registered as subresource.
+    doneFlag = false;
+    [dataStore _isRegisteredAsSubresourceUnderFirstParty:[NSURL URLWithString:@"http://webkit.org"] thirdParty:[NSURL URLWithString:@"http://evil3.com"] completionHandler: ^(BOOL isRegistered) {
+        EXPECT_TRUE(isRegistered);
+        doneFlag = true;
+    }];
+    TestWebKitAPI::Util::run(&doneFlag);
+    doneFlag = false;
+    [dataStore _isRegisteredAsSubresourceUnderFirstParty:[NSURL URLWithString:@"http://webkit2.org"] thirdParty:[NSURL URLWithString:@"http://evil3.com"] completionHandler: ^(BOOL isRegistered) {
+        EXPECT_TRUE(isRegistered);
+        doneFlag = true;
+    }];
+    TestWebKitAPI::Util::run(&doneFlag);
+    doneFlag = false;
+    [dataStore _isRegisteredAsSubresourceUnderFirstParty:[NSURL URLWithString:@"http://webkit3.org"] thirdParty:[NSURL URLWithString:@"http://evil3.com"] completionHandler: ^(BOOL isRegistered) {
+        EXPECT_TRUE(isRegistered);
+        doneFlag = true;
+    }];
+    TestWebKitAPI::Util::run(&doneFlag);
+
+    // Collect the ITP data summary which should include all third parties in the
+    // order: [evil3.com, evil1.com, evil2.com] sorted by number of first parties
+    // it appears under or redirects to.
+    doneFlag = false;
+    [dataStore _getResourceLoadStatisticsDataSummary:^void(NSArray<_WKResourceLoadStatisticsThirdParty *> *thirdPartyData)
+    {
+        NSEnumerator *thirdPartyDomains = [thirdPartyData objectEnumerator];
+
+        // evil3
+        _WKResourceLoadStatisticsThirdParty *evil3ThirdParty = [thirdPartyDomains nextObject];
+        EXPECT_WK_STREQ(evil3ThirdParty.thirdPartyDomain, @"evil3.com");
+
+        NSEnumerator *evil3Enumerator = [evil3ThirdParty.underFirstParties objectEnumerator];
+        _WKResourceLoadStatisticsFirstParty *evil3FirstParty1 = [evil3Enumerator nextObject];
+        _WKResourceLoadStatisticsFirstParty *evil3FirstParty2 = [evil3Enumerator nextObject];
+        _WKResourceLoadStatisticsFirstParty *evil3FirstParty3 = [evil3Enumerator nextObject];
+
+        ASSERT_TRUE(evil3FirstParty1 != nil);
+        ASSERT_TRUE(evil3FirstParty2 != nil);
+        ASSERT_TRUE(evil3FirstParty3 != nil);
+
+        EXPECT_WK_STREQ(evil3FirstParty1.firstPartyDomain, @"webkit2.org");
+        EXPECT_WK_STREQ(evil3FirstParty2.firstPartyDomain, @"webkit3.org");
+        EXPECT_WK_STREQ(evil3FirstParty3.firstPartyDomain, @"webkit.org");
+
+        EXPECT_FALSE(evil3FirstParty1.thirdPartyStorageAccessGranted);
+        EXPECT_FALSE(evil3FirstParty2.thirdPartyStorageAccessGranted);
+        EXPECT_FALSE(evil3FirstParty3.thirdPartyStorageAccessGranted);
+
+        NSTimeInterval rightNow = [[NSDate date] timeIntervalSince1970];
+
+        // Check timestamp for evil3 is reported as being within the correct range.
+        NSTimeInterval evil1TimeLastUpdated = evil3FirstParty1.timeLastUpdated;
+        NSTimeInterval evil2TimeLastUpdated = evil3FirstParty2.timeLastUpdated;
+        NSTimeInterval evil3TimeLastUpdated = evil3FirstParty3.timeLastUpdated;
+
+        EXPECT_TRUE(beforeUpdatingTime < evil1TimeLastUpdated);
+        EXPECT_TRUE(beforeUpdatingTime < evil2TimeLastUpdated);
+        EXPECT_TRUE(beforeUpdatingTime < evil3TimeLastUpdated);
+
+        NSTimeInterval evil1HourDifference = (rightNow - evil1TimeLastUpdated) / 3600;
+        NSTimeInterval evil2HourDifference = (rightNow - evil2TimeLastUpdated) / 3600;
+        NSTimeInterval evil3HourDifference = (rightNow - evil3TimeLastUpdated) / 3600;
+
+        EXPECT_TRUE(evil1HourDifference < 24*3600);
+        EXPECT_TRUE(evil2HourDifference < 24*3600);
+        EXPECT_TRUE(evil3HourDifference < 24*3600);
+
+        // evil1
+        _WKResourceLoadStatisticsThirdParty *evil1ThirdParty = [thirdPartyDomains nextObject];
+        EXPECT_WK_STREQ(evil1ThirdParty.thirdPartyDomain, @"evil1.com");
+
+        NSEnumerator *evil1Enumerator = [evil1ThirdParty.underFirstParties objectEnumerator];
+        _WKResourceLoadStatisticsFirstParty *evil1FirstParty1= [evil1Enumerator nextObject];
+        _WKResourceLoadStatisticsFirstParty *evil1FirstParty2 = [evil1Enumerator nextObject];
+
+        ASSERT_TRUE(evil1FirstParty1 != nil);
+        ASSERT_TRUE(evil1FirstParty2 != nil);
+
+        EXPECT_WK_STREQ(evil1FirstParty1.firstPartyDomain, @"webkit2.org");
+        EXPECT_WK_STREQ(evil1FirstParty2.firstPartyDomain, @"webkit.org");
+
+        EXPECT_FALSE(evil1FirstParty1.thirdPartyStorageAccessGranted);
+        EXPECT_FALSE(evil1FirstParty2.thirdPartyStorageAccessGranted);
+
+        // Check timestamp for evil1 is reported as being within the correct range.
+        evil1TimeLastUpdated = evil1FirstParty1.timeLastUpdated;
+        evil2TimeLastUpdated = evil1FirstParty2.timeLastUpdated;
+
+        EXPECT_TRUE(beforeUpdatingTime < evil1TimeLastUpdated);
+        EXPECT_TRUE(beforeUpdatingTime < evil2TimeLastUpdated);
+
+        evil1HourDifference = (rightNow - evil1TimeLastUpdated) / 3600;
+        evil2HourDifference = (rightNow - evil2TimeLastUpdated) / 3600;
+
+        EXPECT_TRUE(evil1HourDifference < 24*3600);
+        EXPECT_TRUE(evil2HourDifference < 24*3600);
+
+        // evil2
+        _WKResourceLoadStatisticsThirdParty *evil2ThirdParty = [thirdPartyDomains nextObject];
+        EXPECT_WK_STREQ(evil2ThirdParty.thirdPartyDomain, @"evil2.com");
+
+        NSEnumerator *evil2Enumerator = [evil2ThirdParty.underFirstParties objectEnumerator];
+        _WKResourceLoadStatisticsFirstParty *evil2FirstParty1 = [evil2Enumerator nextObject];
+
+        ASSERT_TRUE(evil2FirstParty1 != nil);
+
+        EXPECT_WK_STREQ(evil2FirstParty1.firstPartyDomain, @"webkit.org");
+        EXPECT_FALSE(evil2FirstParty1.thirdPartyStorageAccessGranted);
+
+        // Check timestamp for evil2 is reported as being within the correct range.
+        evil1TimeLastUpdated = evil1FirstParty1.timeLastUpdated;
+
+        EXPECT_TRUE(beforeUpdatingTime < evil1TimeLastUpdated);
+
+        evil1HourDifference = (rightNow - evil1TimeLastUpdated) / 3600;
+
+        EXPECT_TRUE(evil1HourDifference < 24*3600);
+
+        doneFlag = true;
+    }];
+
+    TestWebKitAPI::Util::run(&doneFlag);
+}