Use a SQLite database to hold the ResourceLoadStatistics data
authorbfulgham@apple.com <bfulgham@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 5 Mar 2019 03:00:29 +0000 (03:00 +0000)
committerbfulgham@apple.com <bfulgham@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 5 Mar 2019 03:00:29 +0000 (03:00 +0000)
https://bugs.webkit.org/show_bug.cgi?id=194867
<rdar://problem/24240854>

Reviewed by Chris Dumez.

Source/WebCore:

Add a new runtime feature flag to support use of an experimental database
back-end. Also expose some SQLite function calls for use outside of WebCore.

No change in functionality, so no new tests.

* page/RuntimeEnabledFeatures.h:
(WebCore::RuntimeEnabledFeatures::setItpDatabaseModeEnabled):
(WebCore::RuntimeEnabledFeatures::itpDatabaseModeEnabled const):
* platform/sql/SQLiteDatabase.h:
* platform/sql/SQLiteStatement.h:

Source/WebKit:

The ResourceLoadStatistics database plist is inefficient. It requires more memory use
than it needs, and forces lots of looping and string comparisons to find information.

This problem has already been solved in the form of relational databases. We use them
elsewhere in WebKit, and should do so for this storage as well.

This patch creates an optional SQLite database to handle ITP operations.

1. It adds a new internal experimental feature flag to activate it. It requires the user
   exit and restart the process to take effect.
2. It populates itself from the existing plist file (if it exists).
3. It stops using the plist in favor of the database.
4. It does queries and other operations using the database instead of the old hash table
   implementation.

This patch refactors the exisiting ResourceLoadStatisticsMemoryStore class into a base
ResourceLoadStatisticsStore class, which ResourceLoadStatisticsMemoryStore is based on.
It adds a new ResourceLoadStatisticsDatabaseStore that is implemented in terms of SQL
operations.

These code changes should not have any impact on existing operations, and when enabled
should produce the same test results.

* NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp: Added.
(WebKit::ResourceLoadStatisticsDatabaseStore::ResourceLoadStatisticsDatabaseStore):
(WebKit::ResourceLoadStatisticsDatabaseStore::createSchema):
(WebKit::ResourceLoadStatisticsDatabaseStore::prepareStatements):
(WebKit::ResourceLoadStatisticsDatabaseStore::insertObservedDomain):
(WebKit::ResourceLoadStatisticsDatabaseStore::relationshipExists const):
(WebKit::ResourceLoadStatisticsDatabaseStore::insertDomainRelationship):
(WebKit::ResourceLoadStatisticsDatabaseStore::confirmDomainDoesNotExist const):
(WebKit::ResourceLoadStatisticsDatabaseStore::domainIDFromString const):
(WebKit::ResourceLoadStatisticsDatabaseStore::insertDomainRelationships):
(WebKit::ResourceLoadStatisticsDatabaseStore::databaseIsEmpty const):
(WebKit::ResourceLoadStatisticsDatabaseStore::populateFromMemoryStore):
(WebKit::ResourceLoadStatisticsDatabaseStore::calculateAndSubmitTelemetry const):
(WebKit::ResourceLoadStatisticsDatabaseStore::incrementRecordsDeletedCountForDomains):
(WebKit::ResourceLoadStatisticsDatabaseStore::recursivelyFindNonPrevalentDomainsThatRedirectedToThisDomain):
(WebKit::buildList):
(WebKit::ResourceLoadStatisticsDatabaseStore::markAsPrevalentIfHasRedirectedToPrevalent):
(WebKit::listToString):
(WebKit::ResourceLoadStatisticsDatabaseStore::findNotVeryPrevalentResources):
(WebKit::ResourceLoadStatisticsDatabaseStore::reclassifyResources):
(WebKit::ResourceLoadStatisticsDatabaseStore::classifyPrevalentResources):
(WebKit::ResourceLoadStatisticsDatabaseStore::syncStorageIfNeeded):
(WebKit::ResourceLoadStatisticsDatabaseStore::syncStorageImmediately):
(WebKit::ResourceLoadStatisticsDatabaseStore::hasStorageAccess):
(WebKit::ResourceLoadStatisticsDatabaseStore::requestStorageAccess):
(WebKit::ResourceLoadStatisticsDatabaseStore::requestStorageAccessUnderOpener):
(WebKit::ResourceLoadStatisticsDatabaseStore::grantStorageAccess):
(WebKit::ResourceLoadStatisticsDatabaseStore::grantStorageAccessInternal):
(WebKit::ResourceLoadStatisticsDatabaseStore::grandfatherDataForDomains):
(WebKit::ResourceLoadStatisticsDatabaseStore::ensurePrevalentResourcesForDebugMode):
(WebKit::ResourceLoadStatisticsDatabaseStore::logFrameNavigation):
(WebKit::ResourceLoadStatisticsDatabaseStore::logSubresourceLoading):
(WebKit::ResourceLoadStatisticsDatabaseStore::logSubresourceRedirect):
(WebKit::ResourceLoadStatisticsDatabaseStore::setUserInteraction):
(WebKit::ResourceLoadStatisticsDatabaseStore::logUserInteraction):
(WebKit::ResourceLoadStatisticsDatabaseStore::clearUserInteraction):
(WebKit::ResourceLoadStatisticsDatabaseStore::hasHadUserInteraction):
(WebKit::ResourceLoadStatisticsDatabaseStore::setPrevalentResource):
(WebKit::ResourceLoadStatisticsDatabaseStore::setDomainsAsPrevalent):
(WebKit::ResourceLoadStatisticsDatabaseStore::dumpResourceLoadStatistics const):
(WebKit::ResourceLoadStatisticsDatabaseStore::predicateValueForDomain const):
(WebKit::ResourceLoadStatisticsDatabaseStore::isPrevalentResource const):
(WebKit::ResourceLoadStatisticsDatabaseStore::isVeryPrevalentResource const):
(WebKit::ResourceLoadStatisticsDatabaseStore::isRegisteredAsSubresourceUnder const):
(WebKit::ResourceLoadStatisticsDatabaseStore::isRegisteredAsSubFrameUnder const):
(WebKit::ResourceLoadStatisticsDatabaseStore::isRegisteredAsRedirectingTo const):
(WebKit::ResourceLoadStatisticsDatabaseStore::clearPrevalentResource):
(WebKit::ResourceLoadStatisticsDatabaseStore::setGrandfathered):
(WebKit::ResourceLoadStatisticsDatabaseStore::isGrandfathered const):
(WebKit::ResourceLoadStatisticsDatabaseStore::setSubframeUnderTopFrameOrigin):
(WebKit::ResourceLoadStatisticsDatabaseStore::setSubresourceUnderTopFrameOrigin):
(WebKit::ResourceLoadStatisticsDatabaseStore::setSubresourceUniqueRedirectTo):
(WebKit::ResourceLoadStatisticsDatabaseStore::setSubresourceUniqueRedirectFrom):
(WebKit::ResourceLoadStatisticsDatabaseStore::setTopFrameUniqueRedirectTo):
(WebKit::ResourceLoadStatisticsDatabaseStore::setTopFrameUniqueRedirectFrom):
(WebKit::ResourceLoadStatisticsDatabaseStore::ensureResourceStatisticsForPrimaryDomain):
(WebKit::ResourceLoadStatisticsDatabaseStore::clear):
(WebKit::ResourceLoadStatisticsDatabaseStore::cookieTreatmentForOrigin const):
(WebKit::ResourceLoadStatisticsDatabaseStore::hasUserGrantedStorageAccessThroughPrompt):
(WebKit::ResourceLoadStatisticsDatabaseStore::domainsToBlock const):
(WebKit::ResourceLoadStatisticsDatabaseStore::updateCookieBlocking):
(WebKit::ResourceLoadStatisticsDatabaseStore::updateCookieBlockingForDomains):
(WebKit::ResourceLoadStatisticsDatabaseStore::clearBlockingStateForDomains):
(WebKit::ResourceLoadStatisticsDatabaseStore::processStatistics const):
(WebKit::ResourceLoadStatisticsDatabaseStore::prevalentDomains const):
(WebKit::ResourceLoadStatisticsDatabaseStore::findExpiredUserInteractions const):
(WebKit::ResourceLoadStatisticsDatabaseStore::clearExpiredUserInteractions):
(WebKit::ResourceLoadStatisticsDatabaseStore::clearGrandfathering):
(WebKit::ResourceLoadStatisticsDatabaseStore::topPrivatelyControlledDomainsToRemoveWebsiteDataFor):
(WebKit::ResourceLoadStatisticsDatabaseStore::pruneStatisticsIfNeeded):
(WebKit::ResourceLoadStatisticsDatabaseStore::updateLastSeen):
(WebKit::ResourceLoadStatisticsDatabaseStore::setLastSeen):
(WebKit::ResourceLoadStatisticsDatabaseStore::setVeryPrevalentResource):
(WebKit::ResourceLoadStatisticsDatabaseStore::removeAllStorageAccess):
* NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h: Added.
(WebKit::ResourceLoadStatisticsDatabaseStore::isEmpty const):
* NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp:
(WebKit::ResourceLoadStatisticsMemoryStore::ResourceLoadStatisticsMemoryStore):
(WebKit::ResourceLoadStatisticsMemoryStore::incrementRecordsDeletedCountForDomains):
(WebKit::ResourceLoadStatisticsMemoryStore::classifyPrevalentResources):
(WebKit::ResourceLoadStatisticsMemoryStore::syncStorageIfNeeded):
(WebKit::ResourceLoadStatisticsMemoryStore::syncStorageImmediately):
(WebKit::ResourceLoadStatisticsMemoryStore::grandfatherDataForDomains):
(WebKit::domainsToString): Deleted.
(WebKit::OperatingDate::fromWallTime): Deleted.
(WebKit::OperatingDate::today): Deleted.
(WebKit::OperatingDate::secondsSinceEpoch const): Deleted.
(WebKit::OperatingDate::operator== const): Deleted.
(WebKit::OperatingDate::operator< const): Deleted.
(WebKit::OperatingDate::operator<= const): Deleted.
(WebKit::OperatingDate::OperatingDate): Deleted.
(WebKit::mergeOperatingDates): Deleted.
(WebKit::computeImportance): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::~ResourceLoadStatisticsMemoryStore): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::setNotifyPagesWhenDataRecordsWereScanned): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::setShouldClassifyResourcesBeforeDataRecordsRemoval): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::setShouldSubmitTelemetry): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::removeDataRecords): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::processStatisticsAndDataRecords): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::grandfatherExistingWebsiteData): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::setResourceLoadStatisticsDebugMode): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::setPrevalentResourceForDebugMode): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::scheduleStatisticsProcessingRequestIfNecessary): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::cancelPendingStatisticsProcessingRequest): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::setTimeToLiveUserInteraction): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::setMinimumTimeBetweenDataRecordsRemoval): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::setGrandfatheringTime): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::setCacheMaxAgeCap): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::updateCacheMaxAgeCap): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::setAgeCapForClientSideCookies): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::updateClientSideCookiesAgeCap): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::shouldRemoveDataRecords const): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::setDataRecordsBeingRemoved): Deleted.
(WebKit::debugLogDomainsInBatches): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::includeTodayAsOperatingDateIfNecessary): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::hasStatisticsExpired const): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::setMaxStatisticsEntries): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::setPruneEntriesDownTo): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::resetParametersToDefaultValues): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::logTestingEvent): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::didCreateNetworkProcess): Deleted.
* NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h:
(WebKit::ResourceLoadStatisticsMemoryStore::data const):
(WebKit::ResourceLoadStatisticsMemoryStore::isDebugModeEnabled const): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::store const): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::setStorageAccessPromptsEnabled): Deleted.
(WebKit::ResourceLoadStatisticsMemoryStore::setDebugLogggingEnabled): Deleted.
(): Deleted.
* NetworkProcess/Classifier/ResourceLoadStatisticsStore.cpp: Added.
(WebKit::domainsToString):
(WebKit::OperatingDate::fromWallTime):
(WebKit::OperatingDate::today):
(WebKit::OperatingDate::secondsSinceEpoch const):
(WebKit::OperatingDate::operator== const):
(WebKit::OperatingDate::operator< const):
(WebKit::OperatingDate::operator<= const):
(WebKit::ResourceLoadStatisticsStore::ResourceLoadStatisticsStore):
(WebKit::ResourceLoadStatisticsStore::~ResourceLoadStatisticsStore):
(WebKit::ResourceLoadStatisticsStore::calculateAndSubmitTelemetry const):
(WebKit::ResourceLoadStatisticsStore::computeImportance):
(WebKit::ResourceLoadStatisticsStore::setNotifyPagesWhenDataRecordsWereScanned):
(WebKit::ResourceLoadStatisticsStore::setShouldClassifyResourcesBeforeDataRecordsRemoval):
(WebKit::ResourceLoadStatisticsStore::setShouldSubmitTelemetry):
(WebKit::ResourceLoadStatisticsStore::removeDataRecords):
(WebKit::ResourceLoadStatisticsStore::processStatisticsAndDataRecords):
(WebKit::ResourceLoadStatisticsStore::grandfatherExistingWebsiteData):
(WebKit::ResourceLoadStatisticsStore::setResourceLoadStatisticsDebugMode):
(WebKit::ResourceLoadStatisticsStore::setPrevalentResourceForDebugMode):
(WebKit::ResourceLoadStatisticsStore::scheduleStatisticsProcessingRequestIfNecessary):
(WebKit::ResourceLoadStatisticsStore::cancelPendingStatisticsProcessingRequest):
(WebKit::ResourceLoadStatisticsStore::setTimeToLiveUserInteraction):
(WebKit::ResourceLoadStatisticsStore::setMinimumTimeBetweenDataRecordsRemoval):
(WebKit::ResourceLoadStatisticsStore::setGrandfatheringTime):
(WebKit::ResourceLoadStatisticsStore::setCacheMaxAgeCap):
(WebKit::ResourceLoadStatisticsStore::updateCacheMaxAgeCap):
(WebKit::ResourceLoadStatisticsStore::setAgeCapForClientSideCookies):
(WebKit::ResourceLoadStatisticsStore::updateClientSideCookiesAgeCap):
(WebKit::ResourceLoadStatisticsStore::shouldRemoveDataRecords const):
(WebKit::ResourceLoadStatisticsStore::setDataRecordsBeingRemoved):
(WebKit::ResourceLoadStatisticsStore::updateCookieBlockingForDomains):
(WebKit::ResourceLoadStatisticsStore::clearBlockingStateForDomains):
(WebKit::ResourceLoadStatisticsStore::processStatistics const):
(WebKit::ResourceLoadStatisticsStore::statisticsEpirationTime const):
(WebKit::ResourceLoadStatisticsStore::mergeOperatingDates):
(WebKit::ResourceLoadStatisticsStore::includeTodayAsOperatingDateIfNecessary):
(WebKit::ResourceLoadStatisticsStore::hasStatisticsExpired const):
(WebKit::ResourceLoadStatisticsStore::setMaxStatisticsEntries):
(WebKit::ResourceLoadStatisticsStore::setPruneEntriesDownTo):
(WebKit::ResourceLoadStatisticsStore::resetParametersToDefaultValues):
(WebKit::ResourceLoadStatisticsStore::logTestingEvent):
(WebKit::ResourceLoadStatisticsStore::removeAllStorageAccess):
(WebKit::ResourceLoadStatisticsStore::didCreateNetworkProcess):
(WebKit::ResourceLoadStatisticsStore::debugLogDomainsInBatches):
* NetworkProcess/Classifier/ResourceLoadStatisticsStore.h: Copied from Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h.
(WebKit::OperatingDate::OperatingDate):
(WebKit::ResourceLoadStatisticsStore::isEmpty const):
(WebKit::ResourceLoadStatisticsStore::data const):
(WebKit::ResourceLoadStatisticsStore::isDebugModeEnabled const):
(WebKit::ResourceLoadStatisticsStore::store const):
(WebKit::ResourceLoadStatisticsStore::setStorageAccessPromptsEnabled):
(WebKit::ResourceLoadStatisticsStore::setDebugLogggingEnabled):
* NetworkProcess/Classifier/ResourceLoadStatisticsStoreCocoa.mm: Renamed from Source/WebKit/UIProcess/Cocoa/ResourceLoadStatisticsMemoryStoreCocoa.mm.
(WebKit::ResourceLoadStatisticsStore::registerUserDefaultsIfNeeded):
* NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:
(WebKit::WebResourceLoadStatisticsStore::setNotifyPagesWhenDataRecordsWereScanned):
(WebKit::WebResourceLoadStatisticsStore::setShouldClassifyResourcesBeforeDataRecordsRemoval):
(WebKit::WebResourceLoadStatisticsStore::setShouldSubmitTelemetry):
(WebKit::WebResourceLoadStatisticsStore::WebResourceLoadStatisticsStore):
(WebKit::WebResourceLoadStatisticsStore::flushAndDestroyPersistentStore):
(WebKit::WebResourceLoadStatisticsStore::setResourceLoadStatisticsDebugMode):
(WebKit::WebResourceLoadStatisticsStore::setPrevalentResourceForDebugMode):
(WebKit::WebResourceLoadStatisticsStore::scheduleStatisticsAndDataRecordsProcessing):
(WebKit::WebResourceLoadStatisticsStore::hasStorageAccess):
(WebKit::WebResourceLoadStatisticsStore::requestStorageAccess):
(WebKit::WebResourceLoadStatisticsStore::requestStorageAccessUnderOpener):
(WebKit::WebResourceLoadStatisticsStore::grantStorageAccess):
(WebKit::WebResourceLoadStatisticsStore::performDailyTasks):
(WebKit::WebResourceLoadStatisticsStore::logFrameNavigation):
(WebKit::WebResourceLoadStatisticsStore::logWebSocketLoading):
(WebKit::WebResourceLoadStatisticsStore::logSubresourceLoading):
(WebKit::WebResourceLoadStatisticsStore::logSubresourceRedirect):
(WebKit::WebResourceLoadStatisticsStore::logUserInteraction):
(WebKit::WebResourceLoadStatisticsStore::clearUserInteraction):
(WebKit::WebResourceLoadStatisticsStore::hasHadUserInteraction):
(WebKit::WebResourceLoadStatisticsStore::setLastSeen):
(WebKit::WebResourceLoadStatisticsStore::setPrevalentResource):
(WebKit::WebResourceLoadStatisticsStore::setVeryPrevalentResource):
(WebKit::WebResourceLoadStatisticsStore::isPrevalentResource):
(WebKit::WebResourceLoadStatisticsStore::isVeryPrevalentResource):
(WebKit::WebResourceLoadStatisticsStore::isRegisteredAsSubresourceUnder):
(WebKit::WebResourceLoadStatisticsStore::isRegisteredAsSubFrameUnder):
(WebKit::WebResourceLoadStatisticsStore::isRegisteredAsRedirectingTo):
(WebKit::WebResourceLoadStatisticsStore::clearPrevalentResource):
(WebKit::WebResourceLoadStatisticsStore::setGrandfathered):
(WebKit::WebResourceLoadStatisticsStore::isGrandfathered):
(WebKit::WebResourceLoadStatisticsStore::setSubframeUnderTopFrameOrigin):
(WebKit::WebResourceLoadStatisticsStore::setSubresourceUnderTopFrameOrigin):
(WebKit::WebResourceLoadStatisticsStore::setSubresourceUniqueRedirectTo):
(WebKit::WebResourceLoadStatisticsStore::setSubresourceUniqueRedirectFrom):
(WebKit::WebResourceLoadStatisticsStore::setTopFrameUniqueRedirectTo):
(WebKit::WebResourceLoadStatisticsStore::setTopFrameUniqueRedirectFrom):
(WebKit::WebResourceLoadStatisticsStore::scheduleCookieBlockingUpdate):
(WebKit::WebResourceLoadStatisticsStore::scheduleCookieBlockingUpdateForDomains):
(WebKit::WebResourceLoadStatisticsStore::scheduleClearBlockingStateForDomains):
(WebKit::WebResourceLoadStatisticsStore::scheduleClearInMemoryAndPersistent):
(WebKit::WebResourceLoadStatisticsStore::setTimeToLiveUserInteraction):
(WebKit::WebResourceLoadStatisticsStore::setMinimumTimeBetweenDataRecordsRemoval):
(WebKit::WebResourceLoadStatisticsStore::setGrandfatheringTime):
(WebKit::WebResourceLoadStatisticsStore::setMaxStatisticsEntries):
(WebKit::WebResourceLoadStatisticsStore::setPruneEntriesDownTo):
(WebKit::WebResourceLoadStatisticsStore::resetParametersToDefaultValues):
* NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h:
* NetworkProcess/NetworkProcess.cpp:
(WebKit::NetworkProcess::initializeNetworkProcess):
* NetworkProcess/NetworkProcessCreationParameters.cpp:
(WebKit::NetworkProcessCreationParameters::encode const):
(WebKit::NetworkProcessCreationParameters::decode):
* NetworkProcess/NetworkProcessCreationParameters.h:
* Platform/classifier/ResourceLoadStatisticsClassifier.cpp:
(WebKit::ResourceLoadStatisticsClassifier::calculateResourcePrevalence):
* Platform/classifier/ResourceLoadStatisticsClassifier.h:
* Shared/WebPreferences.yaml:
* Sources.txt:
* SourcesCocoa.txt:
* UIProcess/Cocoa/WebProcessPoolCocoa.mm:
(WebKit::WebProcessPool::platformInitializeNetworkProcess):
* UIProcess/WebProcessPool.cpp:
* WebKit.xcodeproj/project.pbxproj:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@242406 268f45cc-cd09-0410-ab3c-d52691b4dbfc

27 files changed:
Source/WebCore/ChangeLog
Source/WebCore/page/RuntimeEnabledFeatures.h
Source/WebCore/platform/sql/SQLiteDatabase.h
Source/WebCore/platform/sql/SQLiteStatement.h
Source/WebKit/ChangeLog
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp [new file with mode: 0644]
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h [new file with mode: 0644]
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsStore.cpp [new file with mode: 0644]
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsStore.h [new file with mode: 0644]
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsStoreCocoa.mm [moved from Source/WebKit/UIProcess/Cocoa/ResourceLoadStatisticsMemoryStoreCocoa.mm with 95% similarity]
Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp
Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h
Source/WebKit/NetworkProcess/NetworkProcess.cpp
Source/WebKit/NetworkProcess/NetworkProcessCreationParameters.cpp
Source/WebKit/NetworkProcess/NetworkProcessCreationParameters.h
Source/WebKit/Platform/classifier/ResourceLoadStatisticsClassifier.cpp
Source/WebKit/Platform/classifier/ResourceLoadStatisticsClassifier.h
Source/WebKit/Platform/classifier/cocoa/ResourceLoadStatisticsClassifierCocoa.cpp
Source/WebKit/Shared/WebPreferences.yaml
Source/WebKit/Sources.txt
Source/WebKit/SourcesCocoa.txt
Source/WebKit/UIProcess/Cocoa/WebProcessPoolCocoa.mm
Source/WebKit/UIProcess/ProvisionalPageProxy.h
Source/WebKit/UIProcess/WebProcessPool.cpp
Source/WebKit/WebKit.xcodeproj/project.pbxproj

index c1a41bc..9702053 100644 (file)
@@ -1,3 +1,22 @@
+2019-03-04  Brent Fulgham  <bfulgham@apple.com>
+
+        Use a SQLite database to hold the ResourceLoadStatistics data
+        https://bugs.webkit.org/show_bug.cgi?id=194867
+        <rdar://problem/24240854>
+
+        Reviewed by Chris Dumez.
+
+        Add a new runtime feature flag to support use of an experimental database
+        back-end. Also expose some SQLite function calls for use outside of WebCore.
+
+        No change in functionality, so no new tests.
+
+        * page/RuntimeEnabledFeatures.h:
+        (WebCore::RuntimeEnabledFeatures::setItpDatabaseModeEnabled):
+        (WebCore::RuntimeEnabledFeatures::itpDatabaseModeEnabled const):
+        * platform/sql/SQLiteDatabase.h:
+        * platform/sql/SQLiteStatement.h:
+
 2019-03-04  Justin Fan  <justin_fan@apple.com>
 
         [Web GPU] Blitting function prototypes
index 54fb6bb..9ea2698 100644 (file)
@@ -148,6 +148,9 @@ public:
     void setItpDebugModeEnabled(bool isEnabled) { m_itpDebugMode = isEnabled; }
     bool itpDebugModeEnabled() const { return m_itpDebugMode; }
 
+    void setIsITPDatabaseEnabled(bool isEnabled) { m_isITPDatabaseEnabled = isEnabled; }
+    bool isITPDatabaseEnabled() const { return m_isITPDatabaseEnabled; }
+
     void setRestrictedHTTPResponseAccess(bool isEnabled) { m_isRestrictedHTTPResponseAccess = isEnabled; }
     bool restrictedHTTPResponseAccess() const { return m_isRestrictedHTTPResponseAccess; }
 
@@ -534,6 +537,8 @@ private:
     bool m_mousemoveEventHandlingPreventsDefaultEnabled { false };
 #endif
 
+    bool m_isITPDatabaseEnabled { false };
+
     friend class WTF::NeverDestroyed<RuntimeEnabledFeatures>;
 };
 
index 40fce06..643d242 100644 (file)
@@ -65,7 +65,7 @@ public:
     
     WEBCORE_EXPORT bool tableExists(const String&);
     void clearAllTables();
-    int runVacuumCommand();
+    WEBCORE_EXPORT int runVacuumCommand();
     int runIncrementalVacuumCommand();
     
     bool transactionInProgress() const { return m_transactionInProgress; }
index a193d8c..f366c73 100644 (file)
@@ -76,7 +76,7 @@ public:
     String getColumnName(int col);
     SQLValue getColumnValue(int col);
     WEBCORE_EXPORT String getColumnText(int col);
-    double getColumnDouble(int col);
+    WEBCORE_EXPORT double getColumnDouble(int col);
     WEBCORE_EXPORT int getColumnInt(int col);
     WEBCORE_EXPORT int64_t getColumnInt64(int col);
     WEBCORE_EXPORT String getColumnBlobAsString(int col);
index 51635c2..5b243e0 100644 (file)
@@ -1,3 +1,283 @@
+2019-03-04  Brent Fulgham  <bfulgham@apple.com>
+
+        Use a SQLite database to hold the ResourceLoadStatistics data
+        https://bugs.webkit.org/show_bug.cgi?id=194867
+        <rdar://problem/24240854>
+
+        Reviewed by Chris Dumez.
+
+        The ResourceLoadStatistics database plist is inefficient. It requires more memory use
+        than it needs, and forces lots of looping and string comparisons to find information.
+
+        This problem has already been solved in the form of relational databases. We use them
+        elsewhere in WebKit, and should do so for this storage as well.
+
+        This patch creates an optional SQLite database to handle ITP operations.
+
+        1. It adds a new internal experimental feature flag to activate it. It requires the user
+           exit and restart the process to take effect.
+        2. It populates itself from the existing plist file (if it exists).
+        3. It stops using the plist in favor of the database.
+        4. It does queries and other operations using the database instead of the old hash table
+           implementation.
+
+        This patch refactors the exisiting ResourceLoadStatisticsMemoryStore class into a base
+        ResourceLoadStatisticsStore class, which ResourceLoadStatisticsMemoryStore is based on.
+        It adds a new ResourceLoadStatisticsDatabaseStore that is implemented in terms of SQL
+        operations.
+
+        These code changes should not have any impact on existing operations, and when enabled
+        should produce the same test results.
+
+        * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp: Added.
+        (WebKit::ResourceLoadStatisticsDatabaseStore::ResourceLoadStatisticsDatabaseStore):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::createSchema):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::prepareStatements):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::insertObservedDomain):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::relationshipExists const):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::insertDomainRelationship):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::confirmDomainDoesNotExist const):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::domainIDFromString const):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::insertDomainRelationships):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::databaseIsEmpty const):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::populateFromMemoryStore):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::calculateAndSubmitTelemetry const):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::incrementRecordsDeletedCountForDomains):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::recursivelyFindNonPrevalentDomainsThatRedirectedToThisDomain):
+        (WebKit::buildList):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::markAsPrevalentIfHasRedirectedToPrevalent):
+        (WebKit::listToString):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::findNotVeryPrevalentResources):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::reclassifyResources):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::classifyPrevalentResources):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::syncStorageIfNeeded):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::syncStorageImmediately):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::hasStorageAccess):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::requestStorageAccess):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::requestStorageAccessUnderOpener):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::grantStorageAccess):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::grantStorageAccessInternal):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::grandfatherDataForDomains):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::ensurePrevalentResourcesForDebugMode):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::logFrameNavigation):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::logSubresourceLoading):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::logSubresourceRedirect):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::setUserInteraction):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::logUserInteraction):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::clearUserInteraction):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::hasHadUserInteraction):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::setPrevalentResource):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::setDomainsAsPrevalent):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::dumpResourceLoadStatistics const):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::predicateValueForDomain const):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::isPrevalentResource const):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::isVeryPrevalentResource const):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::isRegisteredAsSubresourceUnder const):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::isRegisteredAsSubFrameUnder const):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::isRegisteredAsRedirectingTo const):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::clearPrevalentResource):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::setGrandfathered):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::isGrandfathered const):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::setSubframeUnderTopFrameOrigin):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::setSubresourceUnderTopFrameOrigin):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::setSubresourceUniqueRedirectTo):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::setSubresourceUniqueRedirectFrom):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::setTopFrameUniqueRedirectTo):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::setTopFrameUniqueRedirectFrom):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::ensureResourceStatisticsForPrimaryDomain):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::clear):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::cookieTreatmentForOrigin const):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::hasUserGrantedStorageAccessThroughPrompt):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::domainsToBlock const):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::updateCookieBlocking):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::updateCookieBlockingForDomains):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::clearBlockingStateForDomains):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::processStatistics const):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::prevalentDomains const):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::findExpiredUserInteractions const):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::clearExpiredUserInteractions):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::clearGrandfathering):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::topPrivatelyControlledDomainsToRemoveWebsiteDataFor):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::pruneStatisticsIfNeeded):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::updateLastSeen):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::setLastSeen):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::setVeryPrevalentResource):
+        (WebKit::ResourceLoadStatisticsDatabaseStore::removeAllStorageAccess):
+        * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h: Added.
+        (WebKit::ResourceLoadStatisticsDatabaseStore::isEmpty const):
+        * NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp:
+        (WebKit::ResourceLoadStatisticsMemoryStore::ResourceLoadStatisticsMemoryStore):
+        (WebKit::ResourceLoadStatisticsMemoryStore::incrementRecordsDeletedCountForDomains):
+        (WebKit::ResourceLoadStatisticsMemoryStore::classifyPrevalentResources):
+        (WebKit::ResourceLoadStatisticsMemoryStore::syncStorageIfNeeded):
+        (WebKit::ResourceLoadStatisticsMemoryStore::syncStorageImmediately):
+        (WebKit::ResourceLoadStatisticsMemoryStore::grandfatherDataForDomains):
+        (WebKit::domainsToString): Deleted.
+        (WebKit::OperatingDate::fromWallTime): Deleted.
+        (WebKit::OperatingDate::today): Deleted.
+        (WebKit::OperatingDate::secondsSinceEpoch const): Deleted.
+        (WebKit::OperatingDate::operator== const): Deleted.
+        (WebKit::OperatingDate::operator< const): Deleted.
+        (WebKit::OperatingDate::operator<= const): Deleted.
+        (WebKit::OperatingDate::OperatingDate): Deleted.
+        (WebKit::mergeOperatingDates): Deleted.
+        (WebKit::computeImportance): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::~ResourceLoadStatisticsMemoryStore): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::setNotifyPagesWhenDataRecordsWereScanned): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::setShouldClassifyResourcesBeforeDataRecordsRemoval): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::setShouldSubmitTelemetry): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::removeDataRecords): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::processStatisticsAndDataRecords): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::grandfatherExistingWebsiteData): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::setResourceLoadStatisticsDebugMode): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::setPrevalentResourceForDebugMode): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::scheduleStatisticsProcessingRequestIfNecessary): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::cancelPendingStatisticsProcessingRequest): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::setTimeToLiveUserInteraction): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::setMinimumTimeBetweenDataRecordsRemoval): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::setGrandfatheringTime): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::setCacheMaxAgeCap): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::updateCacheMaxAgeCap): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::setAgeCapForClientSideCookies): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::updateClientSideCookiesAgeCap): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::shouldRemoveDataRecords const): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::setDataRecordsBeingRemoved): Deleted.
+        (WebKit::debugLogDomainsInBatches): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::includeTodayAsOperatingDateIfNecessary): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::hasStatisticsExpired const): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::setMaxStatisticsEntries): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::setPruneEntriesDownTo): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::resetParametersToDefaultValues): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::logTestingEvent): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::didCreateNetworkProcess): Deleted.
+        * NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h:
+        (WebKit::ResourceLoadStatisticsMemoryStore::data const):
+        (WebKit::ResourceLoadStatisticsMemoryStore::isDebugModeEnabled const): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::store const): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::setStorageAccessPromptsEnabled): Deleted.
+        (WebKit::ResourceLoadStatisticsMemoryStore::setDebugLogggingEnabled): Deleted.
+        (): Deleted.
+        * NetworkProcess/Classifier/ResourceLoadStatisticsStore.cpp: Added.
+        (WebKit::domainsToString):
+        (WebKit::OperatingDate::fromWallTime):
+        (WebKit::OperatingDate::today):
+        (WebKit::OperatingDate::secondsSinceEpoch const):
+        (WebKit::OperatingDate::operator== const):
+        (WebKit::OperatingDate::operator< const):
+        (WebKit::OperatingDate::operator<= const):
+        (WebKit::ResourceLoadStatisticsStore::ResourceLoadStatisticsStore):
+        (WebKit::ResourceLoadStatisticsStore::~ResourceLoadStatisticsStore):
+        (WebKit::ResourceLoadStatisticsStore::calculateAndSubmitTelemetry const):
+        (WebKit::ResourceLoadStatisticsStore::computeImportance):
+        (WebKit::ResourceLoadStatisticsStore::setNotifyPagesWhenDataRecordsWereScanned):
+        (WebKit::ResourceLoadStatisticsStore::setShouldClassifyResourcesBeforeDataRecordsRemoval):
+        (WebKit::ResourceLoadStatisticsStore::setShouldSubmitTelemetry):
+        (WebKit::ResourceLoadStatisticsStore::removeDataRecords):
+        (WebKit::ResourceLoadStatisticsStore::processStatisticsAndDataRecords):
+        (WebKit::ResourceLoadStatisticsStore::grandfatherExistingWebsiteData):
+        (WebKit::ResourceLoadStatisticsStore::setResourceLoadStatisticsDebugMode):
+        (WebKit::ResourceLoadStatisticsStore::setPrevalentResourceForDebugMode):
+        (WebKit::ResourceLoadStatisticsStore::scheduleStatisticsProcessingRequestIfNecessary):
+        (WebKit::ResourceLoadStatisticsStore::cancelPendingStatisticsProcessingRequest):
+        (WebKit::ResourceLoadStatisticsStore::setTimeToLiveUserInteraction):
+        (WebKit::ResourceLoadStatisticsStore::setMinimumTimeBetweenDataRecordsRemoval):
+        (WebKit::ResourceLoadStatisticsStore::setGrandfatheringTime):
+        (WebKit::ResourceLoadStatisticsStore::setCacheMaxAgeCap):
+        (WebKit::ResourceLoadStatisticsStore::updateCacheMaxAgeCap):
+        (WebKit::ResourceLoadStatisticsStore::setAgeCapForClientSideCookies):
+        (WebKit::ResourceLoadStatisticsStore::updateClientSideCookiesAgeCap):
+        (WebKit::ResourceLoadStatisticsStore::shouldRemoveDataRecords const):
+        (WebKit::ResourceLoadStatisticsStore::setDataRecordsBeingRemoved):
+        (WebKit::ResourceLoadStatisticsStore::updateCookieBlockingForDomains):
+        (WebKit::ResourceLoadStatisticsStore::clearBlockingStateForDomains):
+        (WebKit::ResourceLoadStatisticsStore::processStatistics const):
+        (WebKit::ResourceLoadStatisticsStore::statisticsEpirationTime const):
+        (WebKit::ResourceLoadStatisticsStore::mergeOperatingDates):
+        (WebKit::ResourceLoadStatisticsStore::includeTodayAsOperatingDateIfNecessary):
+        (WebKit::ResourceLoadStatisticsStore::hasStatisticsExpired const):
+        (WebKit::ResourceLoadStatisticsStore::setMaxStatisticsEntries):
+        (WebKit::ResourceLoadStatisticsStore::setPruneEntriesDownTo):
+        (WebKit::ResourceLoadStatisticsStore::resetParametersToDefaultValues):
+        (WebKit::ResourceLoadStatisticsStore::logTestingEvent):
+        (WebKit::ResourceLoadStatisticsStore::removeAllStorageAccess):
+        (WebKit::ResourceLoadStatisticsStore::didCreateNetworkProcess):
+        (WebKit::ResourceLoadStatisticsStore::debugLogDomainsInBatches):
+        * NetworkProcess/Classifier/ResourceLoadStatisticsStore.h: Copied from Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h.
+        (WebKit::OperatingDate::OperatingDate):
+        (WebKit::ResourceLoadStatisticsStore::isEmpty const):
+        (WebKit::ResourceLoadStatisticsStore::data const):
+        (WebKit::ResourceLoadStatisticsStore::isDebugModeEnabled const):
+        (WebKit::ResourceLoadStatisticsStore::store const):
+        (WebKit::ResourceLoadStatisticsStore::setStorageAccessPromptsEnabled):
+        (WebKit::ResourceLoadStatisticsStore::setDebugLogggingEnabled):
+        * NetworkProcess/Classifier/ResourceLoadStatisticsStoreCocoa.mm: Renamed from Source/WebKit/UIProcess/Cocoa/ResourceLoadStatisticsMemoryStoreCocoa.mm.
+        (WebKit::ResourceLoadStatisticsStore::registerUserDefaultsIfNeeded):
+        * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:
+        (WebKit::WebResourceLoadStatisticsStore::setNotifyPagesWhenDataRecordsWereScanned):
+        (WebKit::WebResourceLoadStatisticsStore::setShouldClassifyResourcesBeforeDataRecordsRemoval):
+        (WebKit::WebResourceLoadStatisticsStore::setShouldSubmitTelemetry):
+        (WebKit::WebResourceLoadStatisticsStore::WebResourceLoadStatisticsStore):
+        (WebKit::WebResourceLoadStatisticsStore::flushAndDestroyPersistentStore):
+        (WebKit::WebResourceLoadStatisticsStore::setResourceLoadStatisticsDebugMode):
+        (WebKit::WebResourceLoadStatisticsStore::setPrevalentResourceForDebugMode):
+        (WebKit::WebResourceLoadStatisticsStore::scheduleStatisticsAndDataRecordsProcessing):
+        (WebKit::WebResourceLoadStatisticsStore::hasStorageAccess):
+        (WebKit::WebResourceLoadStatisticsStore::requestStorageAccess):
+        (WebKit::WebResourceLoadStatisticsStore::requestStorageAccessUnderOpener):
+        (WebKit::WebResourceLoadStatisticsStore::grantStorageAccess):
+        (WebKit::WebResourceLoadStatisticsStore::performDailyTasks):
+        (WebKit::WebResourceLoadStatisticsStore::logFrameNavigation):
+        (WebKit::WebResourceLoadStatisticsStore::logWebSocketLoading):
+        (WebKit::WebResourceLoadStatisticsStore::logSubresourceLoading):
+        (WebKit::WebResourceLoadStatisticsStore::logSubresourceRedirect):
+        (WebKit::WebResourceLoadStatisticsStore::logUserInteraction):
+        (WebKit::WebResourceLoadStatisticsStore::clearUserInteraction):
+        (WebKit::WebResourceLoadStatisticsStore::hasHadUserInteraction):
+        (WebKit::WebResourceLoadStatisticsStore::setLastSeen):
+        (WebKit::WebResourceLoadStatisticsStore::setPrevalentResource):
+        (WebKit::WebResourceLoadStatisticsStore::setVeryPrevalentResource):
+        (WebKit::WebResourceLoadStatisticsStore::isPrevalentResource):
+        (WebKit::WebResourceLoadStatisticsStore::isVeryPrevalentResource):
+        (WebKit::WebResourceLoadStatisticsStore::isRegisteredAsSubresourceUnder):
+        (WebKit::WebResourceLoadStatisticsStore::isRegisteredAsSubFrameUnder):
+        (WebKit::WebResourceLoadStatisticsStore::isRegisteredAsRedirectingTo):
+        (WebKit::WebResourceLoadStatisticsStore::clearPrevalentResource):
+        (WebKit::WebResourceLoadStatisticsStore::setGrandfathered):
+        (WebKit::WebResourceLoadStatisticsStore::isGrandfathered):
+        (WebKit::WebResourceLoadStatisticsStore::setSubframeUnderTopFrameOrigin):
+        (WebKit::WebResourceLoadStatisticsStore::setSubresourceUnderTopFrameOrigin):
+        (WebKit::WebResourceLoadStatisticsStore::setSubresourceUniqueRedirectTo):
+        (WebKit::WebResourceLoadStatisticsStore::setSubresourceUniqueRedirectFrom):
+        (WebKit::WebResourceLoadStatisticsStore::setTopFrameUniqueRedirectTo):
+        (WebKit::WebResourceLoadStatisticsStore::setTopFrameUniqueRedirectFrom):
+        (WebKit::WebResourceLoadStatisticsStore::scheduleCookieBlockingUpdate):
+        (WebKit::WebResourceLoadStatisticsStore::scheduleCookieBlockingUpdateForDomains):
+        (WebKit::WebResourceLoadStatisticsStore::scheduleClearBlockingStateForDomains):
+        (WebKit::WebResourceLoadStatisticsStore::scheduleClearInMemoryAndPersistent):
+        (WebKit::WebResourceLoadStatisticsStore::setTimeToLiveUserInteraction):
+        (WebKit::WebResourceLoadStatisticsStore::setMinimumTimeBetweenDataRecordsRemoval):
+        (WebKit::WebResourceLoadStatisticsStore::setGrandfatheringTime):
+        (WebKit::WebResourceLoadStatisticsStore::setMaxStatisticsEntries):
+        (WebKit::WebResourceLoadStatisticsStore::setPruneEntriesDownTo):
+        (WebKit::WebResourceLoadStatisticsStore::resetParametersToDefaultValues):
+        * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h:
+        * NetworkProcess/NetworkProcess.cpp:
+        (WebKit::NetworkProcess::initializeNetworkProcess):
+        * NetworkProcess/NetworkProcessCreationParameters.cpp:
+        (WebKit::NetworkProcessCreationParameters::encode const):
+        (WebKit::NetworkProcessCreationParameters::decode):
+        * NetworkProcess/NetworkProcessCreationParameters.h:
+        * Platform/classifier/ResourceLoadStatisticsClassifier.cpp:
+        (WebKit::ResourceLoadStatisticsClassifier::calculateResourcePrevalence):
+        * Platform/classifier/ResourceLoadStatisticsClassifier.h:
+        * Shared/WebPreferences.yaml:
+        * Sources.txt:
+        * SourcesCocoa.txt:
+        * UIProcess/Cocoa/WebProcessPoolCocoa.mm:
+        (WebKit::WebProcessPool::platformInitializeNetworkProcess):
+        * UIProcess/WebProcessPool.cpp:
+        * WebKit.xcodeproj/project.pbxproj:
+
 2019-03-04  Zalan Bujtas  <zalan@apple.com>
 
         [ContentChangeObserver] Introduce fixed duration content observation
diff --git a/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp b/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp
new file mode 100644 (file)
index 0000000..82c8e94
--- /dev/null
@@ -0,0 +1,1558 @@
+/*
+ * Copyright (C) 2019 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "ResourceLoadStatisticsDatabaseStore.h"
+
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+
+#include "Logging.h"
+#include "NetworkSession.h"
+#include "PluginProcessManager.h"
+#include "PluginProcessProxy.h"
+#include "ResourceLoadStatisticsMemoryStore.h"
+#include "StorageAccessStatus.h"
+#include "WebProcessProxy.h"
+#include "WebResourceLoadStatisticsTelemetry.h"
+#include "WebsiteDataStore.h"
+#include <WebCore/KeyedCoding.h>
+#include <WebCore/NetworkStorageSession.h>
+#include <WebCore/ResourceLoadStatistics.h>
+#include <WebCore/SQLiteDatabase.h>
+#include <WebCore/SQLiteStatement.h>
+#include <wtf/CallbackAggregator.h>
+#include <wtf/DateMath.h>
+#include <wtf/HashMap.h>
+#include <wtf/MathExtras.h>
+#include <wtf/StdSet.h>
+#include <wtf/text/StringBuilder.h>
+
+namespace WebKit {
+using namespace WebCore;
+
+#if PLATFORM(COCOA)
+#define RELEASE_LOG_IF_ALLOWED(sessionID, fmt, ...) RELEASE_LOG_IF(sessionID.isAlwaysOnLoggingAllowed(), Network, "%p - ResourceLoadStatisticsDatabaseStore::" fmt, this, ##__VA_ARGS__)
+#define RELEASE_LOG_ERROR_IF_ALLOWED(sessionID, fmt, ...) RELEASE_LOG_ERROR_IF(sessionID.isAlwaysOnLoggingAllowed(), Network, "%p - ResourceLoadStatisticsDatabaseStore::" fmt, this, ##__VA_ARGS__)
+#else
+#define RELEASE_LOG_IF_ALLOWED(sessionID, fmt, ...)  ((void)0)
+#define RELEASE_LOG_ERROR_IF_ALLOWED(sessionID, fmt, ...)  ((void)0)
+#endif
+
+constexpr auto observedDomainCountQuery = "SELECT COUNT(*) FROM ObservedDomains"_s;
+constexpr auto insertObservedDomainQuery = "INSERT INTO ObservedDomains (registrableDomain, lastSeen, hadUserInteraction,"
+    "mostRecentUserInteractionTime, grandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, timesAccessedAsFirstPartyDueToUserInteraction,"
+    "timesAccessedAsFirstPartyDueToStorageAccessAPI) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"_s;
+constexpr auto insertTopLevelDomainQuery = "INSERT INTO TopLevelDomains VALUES (?)"_s;
+constexpr auto domainIDFromStringQuery = "SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?"_s;
+constexpr auto storageAccessUnderTopFrameDomainsQuery = "INSERT INTO StorageAccessUnderTopFrameDomains (domainID, topLevelDomainID) "
+    "SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain == ?"_s;
+constexpr auto topFrameUniqueRedirectsToQuery = "INSERT INTO TopFrameUniqueRedirectsTo (sourceDomainID, toDomainID) SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain == ?"_s;
+constexpr auto topFrameUniqueRedirectsToExistsQuery = "SELECT EXISTS (SELECT 1 FROM TopFrameUniqueRedirectsTo WHERE sourceDomainID = ? "
+    "AND toDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
+constexpr auto topFrameUniqueRedirectsFromQuery = "INSERT INTO TopFrameUniqueRedirectsFrom (targetDomainID, fromDomainID) "
+    "SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain = ?"_s;
+constexpr auto topFrameUniqueRedirectsFromExistsQuery = "SELECT EXISTS (SELECT 1 FROM TopFrameUniqueRedirectsFrom WHERE targetDomainID = ? "
+    "AND fromDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
+constexpr auto subframeUnderTopFrameDomainsQuery = "INSERT INTO SubframeUnderTopFrameDomains (subFrameDomainID, topFrameDomainID) "
+    "SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain = ?"_s;
+constexpr auto subframeUnderTopFrameDomainExistsQuery = "SELECT EXISTS (SELECT 1 FROM SubframeUnderTopFrameDomains WHERE subFrameDomainID = ? "
+    "AND topFrameDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
+constexpr auto subresourceUnderTopFrameDomainsQuery = "INSERT INTO SubresourceUnderTopFrameDomains (subresourceDomainID, topFrameDomainID) "
+    "SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain = ?"_s;
+constexpr auto subresourceUnderTopFrameDomainExistsQuery = "SELECT EXISTS (SELECT 1 FROM SubresourceUnderTopFrameDomains "
+    "WHERE subresourceDomainID = ? AND topFrameDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
+constexpr auto subresourceUniqueRedirectsToQuery = "INSERT INTO SubresourceUniqueRedirectsTo (subresourceDomainID, toDomainID) "
+    "SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain == ?"_s;
+constexpr auto subresourceUniqueRedirectsToExistsQuery = "SELECT EXISTS (SELECT 1 FROM SubresourceUniqueRedirectsTo WHERE subresourceDomainID = ? "
+    "AND toDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
+constexpr auto subresourceUniqueRedirectsFromQuery = "INSERT INTO SubresourceUniqueRedirectsFrom (subresourceDomainID, fromDomainID) "
+    "SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain == ?"_s;
+constexpr auto subresourceUniqueRedirectsFromExistsQuery = "SELECT EXISTS (SELECT 1 FROM SubresourceUniqueRedirectsFrom WHERE subresourceDomainID = ? "
+    "AND fromDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
+constexpr auto mostRecentUserInteractionQuery = "UPDATE ObservedDomains SET hadUserInteraction = ?, mostRecentUserInteractionTime = ? "
+    "WHERE registrableDomain = ?"_s;
+constexpr auto updateLastSeenQuery = "UPDATE ObservedDomains SET lastSeen = ? WHERE registrableDomain = ?"_s;
+constexpr auto updatePrevalentResourceQuery = "UPDATE ObservedDomains SET isPrevalent = ? WHERE registrableDomain = ?"_s;
+constexpr auto isPrevalentResourceQuery = "SELECT isPrevalent FROM ObservedDomains WHERE registrableDomain = ?"_s;
+constexpr auto updateVeryPrevalentResourceQuery = "UPDATE ObservedDomains SET isVeryPrevalent = ? WHERE registrableDomain = ?"_s;
+constexpr auto isVeryPrevalentResourceQuery = "SELECT isVeryPrevalent FROM ObservedDomains WHERE registrableDomain = ?"_s;
+constexpr auto clearPrevalentResourceQuery = "UPDATE ObservedDomains SET isPrevalent = 0, isVeryPrevalent = 0 WHERE registrableDomain = ?"_s;
+constexpr auto hadUserInteractionQuery = "SELECT hadUserInteraction, mostRecentUserInteractionTime FROM ObservedDomains WHERE registrableDomain = ?"_s;
+constexpr auto updateGrandfatheredQuery = "UPDATE ObservedDomains SET grandfathered = ? WHERE registrableDomain = ?"_s;
+constexpr auto isGrandfatheredQuery = "SELECT grandfathered FROM ObservedDomains WHERE registrableDomain = ?"_s;
+constexpr auto findExpiredUserInteractionQuery = "SELECT domainID FROM ObservedDomains WHERE hadUserInteraction = 1 AND mostRecentUserInteractionTime < ?"_s;
+
+constexpr auto createObservedDomain = "CREATE TABLE ObservedDomains ("
+    "domainID INTEGER PRIMARY KEY, registrableDomain TEXT NOT NULL UNIQUE ON CONFLICT FAIL, lastSeen REAL NOT NULL, "
+    "hadUserInteraction INTEGER NOT NULL, mostRecentUserInteractionTime REAL NOT NULL, grandfathered INTEGER NOT NULL, "
+    "isPrevalent INTEGER NOT NULL, isVeryPrevalent INTEGER NOT NULL, dataRecordsRemoved INTEGER NOT NULL,"
+    "timesAccessedAsFirstPartyDueToUserInteraction INTEGER NOT NULL, timesAccessedAsFirstPartyDueToStorageAccessAPI INTEGER NOT NULL);"_s;
+    
+constexpr auto createTopLevelDomains = "CREATE TABLE TopLevelDomains ("
+    "topLevelDomainID INTEGER PRIMARY KEY, CONSTRAINT fkDomainID FOREIGN KEY(topLevelDomainID) "
+    "REFERENCES ObservedDomains(domainID) ON DELETE CASCADE);"_s;
+    
+constexpr auto createStorageAccessUnderTopFrameDomains = "CREATE TABLE StorageAccessUnderTopFrameDomains ("
+    "domainID INTEGER NOT NULL, topLevelDomainID INTEGER NOT NULL ON CONFLICT FAIL, "
+    "CONSTRAINT fkDomainID FOREIGN KEY(domainID) REFERENCES ObservedDomains(domainID) ON DELETE CASCADE, "
+    "FOREIGN KEY(topLevelDomainID) REFERENCES TopLevelDomains(topLevelDomainID) ON DELETE CASCADE);"_s;
+    
+constexpr auto createTopFrameUniqueRedirectsTo = "CREATE TABLE TopFrameUniqueRedirectsTo ("
+    "sourceDomainID INTEGER NOT NULL, toDomainID INTEGER NOT NULL, "
+    "FOREIGN KEY(sourceDomainID) REFERENCES TopLevelDomains(topLevelDomainID) ON DELETE CASCADE, "
+    "FOREIGN KEY(toDomainID) REFERENCES TopLevelDomains(topLevelDomainID) ON DELETE CASCADE);"_s;
+    
+constexpr auto createTopFrameUniqueRedirectsFrom = "CREATE TABLE TopFrameUniqueRedirectsFrom ("
+    "targetDomainID INTEGER NOT NULL, fromDomainID INTEGER NOT NULL, "
+    "FOREIGN KEY(targetDomainID) REFERENCES TopLevelDomains(topLevelDomainID) ON DELETE CASCADE, "
+    "FOREIGN KEY(fromDomainID) REFERENCES TopLevelDomains(topLevelDomainID) ON DELETE CASCADE);"_s;
+    
+constexpr auto createSubframeUnderTopFrameDomains = "CREATE TABLE SubframeUnderTopFrameDomains ("
+    "subFrameDomainID INTEGER NOT NULL, topFrameDomainID INTEGER NOT NULL, "
+    "FOREIGN KEY(subFrameDomainID) REFERENCES ObservedDomains(domainID) ON DELETE CASCADE, "
+    "FOREIGN KEY(topFrameDomainID) REFERENCES TopLevelDomains(topLevelDomainID) ON DELETE CASCADE);"_s;
+    
+constexpr auto createSubresourceUnderTopFrameDomains = "CREATE TABLE SubresourceUnderTopFrameDomains ("
+    "subresourceDomainID INTEGER NOT NULL, topFrameDomainID INTEGER NOT NULL, "
+    "FOREIGN KEY(subresourceDomainID) REFERENCES ObservedDomains(domainID) ON DELETE CASCADE, "
+    "FOREIGN KEY(topFrameDomainID) REFERENCES TopLevelDomains(topLevelDomainID) ON DELETE CASCADE);"_s;
+    
+constexpr auto createSubresourceUniqueRedirectsTo = "CREATE TABLE SubresourceUniqueRedirectsTo ("
+    "subresourceDomainID INTEGER NOT NULL, toDomainID INTEGER NOT NULL, "
+    "FOREIGN KEY(subresourceDomainID) REFERENCES ObservedDomains(domainID) ON DELETE CASCADE, "
+    "FOREIGN KEY(toDomainID) REFERENCES ObservedDomains(domainID) ON DELETE CASCADE);"_s;
+    
+constexpr auto createSubresourceUniqueRedirectsFrom = "CREATE TABLE SubresourceUniqueRedirectsFrom ("
+    "subresourceDomainID INTEGER NOT NULL, fromDomainID INTEGER NOT NULL, "
+    "FOREIGN KEY(subresourceDomainID) REFERENCES ObservedDomains(domainID) ON DELETE CASCADE, "
+    "FOREIGN KEY(fromDomainID) REFERENCES ObservedDomains(domainID) ON DELETE CASCADE);"_s;
+    
+ResourceLoadStatisticsDatabaseStore::ResourceLoadStatisticsDatabaseStore(WebResourceLoadStatisticsStore& store, WorkQueue& workQueue, const String& storageDirectoryPath)
+    : ResourceLoadStatisticsStore(store, workQueue)
+    , m_storageDirectoryPath(storageDirectoryPath + "/observations.db")
+    , m_observedDomainCount(m_database, observedDomainCountQuery)
+    , m_insertObservedDomainStatement(m_database, insertObservedDomainQuery)
+    , m_insertTopLevelDomainStatement(m_database, insertTopLevelDomainQuery)
+    , m_domainIDFromStringStatement(m_database, domainIDFromStringQuery)
+    , m_storageAccessUnderTopFrameDomainsStatement(m_database, storageAccessUnderTopFrameDomainsQuery)
+    , m_topFrameUniqueRedirectsTo(m_database, topFrameUniqueRedirectsToQuery)
+    , m_topFrameUniqueRedirectsToExists(m_database, topFrameUniqueRedirectsToExistsQuery)
+    , m_topFrameUniqueRedirectsFrom(m_database, topFrameUniqueRedirectsFromQuery)
+    , m_topFrameUniqueRedirectsFromExists(m_database, topFrameUniqueRedirectsFromExistsQuery)
+    , m_subframeUnderTopFrameDomains(m_database, subframeUnderTopFrameDomainsQuery)
+    , m_subframeUnderTopFrameDomainExists(m_database, subframeUnderTopFrameDomainExistsQuery)
+    , m_subresourceUnderTopFrameDomains(m_database, subresourceUnderTopFrameDomainsQuery)
+    , m_subresourceUnderTopFrameDomainExists(m_database, subresourceUnderTopFrameDomainExistsQuery)
+    , m_subresourceUniqueRedirectsTo(m_database, subresourceUniqueRedirectsToQuery)
+    , m_subresourceUniqueRedirectsToExists(m_database, subresourceUniqueRedirectsToExistsQuery)
+    , m_subresourceUniqueRedirectsFrom(m_database, subresourceUniqueRedirectsFromQuery)
+    , m_subresourceUniqueRedirectsFromExists(m_database, subresourceUniqueRedirectsFromExistsQuery)
+    , m_mostRecentUserInteractionStatement(m_database, mostRecentUserInteractionQuery)
+    , m_updateLastSeenStatement(m_database, updateLastSeenQuery)
+    , m_updatePrevalentResourceStatement(m_database, updatePrevalentResourceQuery)
+    , m_isPrevalentResourceStatement(m_database, isPrevalentResourceQuery)
+    , m_updateVeryPrevalentResourceStatement(m_database, updateVeryPrevalentResourceQuery)
+    , m_isVeryPrevalentResourceStatement(m_database, isVeryPrevalentResourceQuery)
+    , m_clearPrevalentResourceStatement(m_database, clearPrevalentResourceQuery)
+    , m_hadUserInteractionStatement(m_database, hadUserInteractionQuery)
+    , m_updateGrandfatheredStatement(m_database, updateGrandfatheredQuery)
+    , m_isGrandfatheredStatement(m_database, isGrandfatheredQuery)
+    , m_findExpiredUserInteractionStatement(m_database, findExpiredUserInteractionQuery)
+{
+    ASSERT(!RunLoop::isMain());
+
+#if PLATFORM(COCOA)
+    registerUserDefaultsIfNeeded();
+#endif
+
+    if (!m_database.open(m_storageDirectoryPath)) {
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::open failed, error message: %{public}s, database path: %{public}s", this, m_database.lastErrorMsg(), m_storageDirectoryPath.utf8().data());
+        ASSERT_NOT_REACHED();
+        return;
+    }
+    
+    // Since we are using a workerQueue, the sequential dispatch blocks may be called by different threads.
+    m_database.disableThreadingChecks();
+
+    if (!m_database.tableExists("ObservedDomains"_s)) {
+        if (!createSchema()) {
+            RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::createSchema failed, error message: %{public}s, database path: %{public}s", this, m_database.lastErrorMsg(), m_storageDirectoryPath.utf8().data());
+            ASSERT_NOT_REACHED();
+            return;
+        }
+    }
+
+    if (!prepareStatements()) {
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::prepareStatements failed, error message: %{public}s, database path: %{public}s", this, m_database.lastErrorMsg(), m_storageDirectoryPath.utf8().data());
+        ASSERT_NOT_REACHED();
+        return;
+    }
+
+    workQueue.dispatchAfter(5_s, [weakThis = makeWeakPtr(*this)] {
+        if (weakThis)
+            weakThis->calculateAndSubmitTelemetry();
+    });
+}
+
+bool ResourceLoadStatisticsDatabaseStore::isEmpty() const
+{
+    ASSERT(!RunLoop::isMain());
+    
+    bool result = false;
+    if (m_observedDomainCount.step() == SQLITE_ROW)
+        result = !m_observedDomainCount.getColumnInt(0);
+    
+    int resetResult = m_observedDomainCount.reset();
+    ASSERT_UNUSED(resetResult, resetResult == SQLITE_OK);
+    
+    return result;
+}
+
+bool ResourceLoadStatisticsDatabaseStore::createSchema()
+{
+    ASSERT(!RunLoop::isMain());
+
+    if (!m_database.executeCommand(createObservedDomain)) {
+        LOG_ERROR("Could not create ObservedDomains table in database (%i) - %s", m_database.lastError(), m_database.lastErrorMsg());
+        return false;
+    }
+
+    if (!m_database.executeCommand(createTopLevelDomains)) {
+        LOG_ERROR("Could not create TopLevelDomains table in database (%i) - %s", m_database.lastError(), m_database.lastErrorMsg());
+        return false;
+    }
+
+    if (!m_database.executeCommand(createStorageAccessUnderTopFrameDomains)) {
+        LOG_ERROR("Could not create StorageAccessUnderTopFrameDomains table in database (%i) - %s", m_database.lastError(), m_database.lastErrorMsg());
+        return false;
+    }
+
+    if (!m_database.executeCommand(createTopFrameUniqueRedirectsTo)) {
+        LOG_ERROR("Could not create TopFrameUniqueRedirectsTo table in database (%i) - %s", m_database.lastError(), m_database.lastErrorMsg());
+        return false;
+    }
+
+    if (!m_database.executeCommand(createTopFrameUniqueRedirectsFrom)) {
+        LOG_ERROR("Could not create TopFrameUniqueRedirectsFrom table in database (%i) - %s", m_database.lastError(), m_database.lastErrorMsg());
+        return false;
+    }
+
+    if (!m_database.executeCommand(createSubframeUnderTopFrameDomains)) {
+        LOG_ERROR("Could not create SubframeUnderTopFrameDomains table in database (%i) - %s", m_database.lastError(), m_database.lastErrorMsg());
+        return false;
+    }
+
+    if (!m_database.executeCommand(createSubresourceUnderTopFrameDomains)) {
+        LOG_ERROR("Could not create SubresourceUnderTopFrameDomains table in database (%i) - %s", m_database.lastError(), m_database.lastErrorMsg());
+        return false;
+    }
+
+    if (!m_database.executeCommand(createSubresourceUniqueRedirectsTo)) {
+        LOG_ERROR("Could not create SubresourceUniqueRedirectsTo table in database (%i) - %s", m_database.lastError(), m_database.lastErrorMsg());
+        return false;
+    }
+
+    if (!m_database.executeCommand(createSubresourceUniqueRedirectsFrom)) {
+        LOG_ERROR("Could not create SubresourceUniqueRedirectsFrom table in database (%i) - %s", m_database.lastError(), m_database.lastErrorMsg());
+        return false;
+    }
+
+    return true;
+}
+
+bool ResourceLoadStatisticsDatabaseStore::prepareStatements()
+{
+    ASSERT(!RunLoop::isMain());
+        
+    if (m_observedDomainCount.prepare() != SQLITE_OK
+        || m_insertObservedDomainStatement.prepare() != SQLITE_OK
+        || m_insertTopLevelDomainStatement.prepare() != SQLITE_OK
+        || m_domainIDFromStringStatement.prepare() != SQLITE_OK
+        || m_storageAccessUnderTopFrameDomainsStatement.prepare() != SQLITE_OK
+        || m_topFrameUniqueRedirectsTo.prepare() != SQLITE_OK
+        || m_topFrameUniqueRedirectsToExists.prepare() != SQLITE_OK
+        || m_topFrameUniqueRedirectsFrom.prepare() != SQLITE_OK
+        || m_topFrameUniqueRedirectsFromExists.prepare() != SQLITE_OK
+        || m_subframeUnderTopFrameDomains.prepare() != SQLITE_OK
+        || m_subframeUnderTopFrameDomainExists.prepare() != SQLITE_OK
+        || m_subresourceUnderTopFrameDomains.prepare() != SQLITE_OK
+        || m_subresourceUnderTopFrameDomainExists.prepare() != SQLITE_OK
+        || m_subresourceUniqueRedirectsTo.prepare() != SQLITE_OK
+        || m_subresourceUniqueRedirectsToExists.prepare() != SQLITE_OK
+        || m_subresourceUniqueRedirectsFrom.prepare() != SQLITE_OK
+        || m_subresourceUniqueRedirectsFromExists.prepare() != SQLITE_OK
+        || m_updateLastSeenStatement.prepare() != SQLITE_OK
+        || m_mostRecentUserInteractionStatement.prepare() != SQLITE_OK
+        || m_updatePrevalentResourceStatement.prepare() != SQLITE_OK
+        || m_isPrevalentResourceStatement.prepare() != SQLITE_OK
+        || m_updateVeryPrevalentResourceStatement.prepare() != SQLITE_OK
+        || m_isVeryPrevalentResourceStatement.prepare() != SQLITE_OK
+        || m_clearPrevalentResourceStatement.prepare() != SQLITE_OK
+        || m_hadUserInteractionStatement.prepare() != SQLITE_OK
+        || m_updateGrandfatheredStatement.prepare() != SQLITE_OK
+        || m_isGrandfatheredStatement.prepare() != SQLITE_OK
+        || m_findExpiredUserInteractionStatement.prepare() != SQLITE_OK
+        ) {
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::prepareStatements failed to prepare, error message: %{public}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+        return false;
+    }
+
+    return true;
+}
+
+bool ResourceLoadStatisticsDatabaseStore::insertObservedDomain(const ResourceLoadStatistics& loadStatistics)
+{
+    ASSERT(!RunLoop::isMain());
+
+#ifndef NDEBUG
+    ASSERT(confirmDomainDoesNotExist(loadStatistics.registrableDomain));
+#endif
+
+    if (m_insertObservedDomainStatement.bindText(1, loadStatistics.registrableDomain.string()) != SQLITE_OK
+        || m_insertObservedDomainStatement.bindDouble(2, loadStatistics.lastSeen.secondsSinceEpoch().value()) != SQLITE_OK
+        || m_insertObservedDomainStatement.bindInt(3, loadStatistics.hadUserInteraction) != SQLITE_OK
+        || m_insertObservedDomainStatement.bindDouble(4, loadStatistics.mostRecentUserInteractionTime.secondsSinceEpoch().value()) != SQLITE_OK
+        || m_insertObservedDomainStatement.bindInt(5, loadStatistics.grandfathered) != SQLITE_OK
+        || m_insertObservedDomainStatement.bindInt(6, loadStatistics.isPrevalentResource) != SQLITE_OK
+        || m_insertObservedDomainStatement.bindInt(7, loadStatistics.isVeryPrevalentResource) != SQLITE_OK
+        || m_insertObservedDomainStatement.bindInt(8, loadStatistics.dataRecordsRemoved) != SQLITE_OK
+        || m_insertObservedDomainStatement.bindInt(9, loadStatistics.timesAccessedAsFirstPartyDueToUserInteraction) != SQLITE_OK
+        || m_insertObservedDomainStatement.bindInt(10, loadStatistics.timesAccessedAsFirstPartyDueToStorageAccessAPI) != SQLITE_OK) {
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::insertObservedDomain failed to bind, error message: %{public}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+        return false;
+    }
+
+    if (m_insertObservedDomainStatement.step() != SQLITE_DONE) {
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::insertObservedDomain failed to commit, error message: %{public}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+        return false;
+    }
+
+    int resetResult = m_insertObservedDomainStatement.reset();
+    ASSERT_UNUSED(resetResult, resetResult == SQLITE_OK);
+
+    return true;
+}
+
+bool ResourceLoadStatisticsDatabaseStore::relationshipExists(WebCore::SQLiteStatement& statement, unsigned firstDomainID, const RegistrableDomain& secondDomain) const
+{
+    ASSERT(!RunLoop::isMain());
+
+    if (statement.bindInt(1, firstDomainID) != SQLITE_OK
+        || statement.bindText(2, secondDomain.string()) != SQLITE_OK
+        || statement.step() != SQLITE_ROW) {
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::m_insertDomainRelationshipStatement failed to bind, error message: %{public}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+        return false;
+    }
+    
+    bool relationShipExists = !!statement.getColumnInt(0);
+
+    int resetResult = statement.reset();
+    ASSERT_UNUSED(resetResult, resetResult == SQLITE_OK);
+    
+    return relationShipExists;
+}
+
+bool ResourceLoadStatisticsDatabaseStore::insertDomainRelationship(WebCore::SQLiteStatement& statement, unsigned domainID, const RegistrableDomain& topFrame)
+{
+    ASSERT(!RunLoop::isMain());
+
+    if (statement.bindInt(1, domainID) != SQLITE_OK
+        || statement.bindText(2, topFrame.string()) != SQLITE_OK
+        || statement.step() != SQLITE_DONE) {
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::m_insertDomainRelationshipStatement failed to bind, error message: %{public}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+        return false;
+    }
+
+    int resetResult = statement.reset();
+    ASSERT_UNUSED(resetResult, resetResult == SQLITE_OK);
+    
+    return true;
+}
+
+#ifndef NDEBUG
+bool ResourceLoadStatisticsDatabaseStore::confirmDomainDoesNotExist(const RegistrableDomain& domain) const
+{
+    ASSERT(!RunLoop::isMain());
+
+    if (m_domainIDFromStringStatement.bindText(1, domain.string()) != SQLITE_OK) {
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::confirmDomainDoesNotExist failed to bind, error message: %{public}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+        return false;
+    }
+        
+    if (m_domainIDFromStringStatement.step() == SQLITE_ROW) {
+        int resetResult = m_domainIDFromStringStatement.reset();
+        ASSERT_UNUSED(resetResult, resetResult == SQLITE_OK);
+        return false;
+    }
+
+    int resetResult = m_domainIDFromStringStatement.reset();
+    ASSERT_UNUSED(resetResult, resetResult == SQLITE_OK);
+    
+    return true;
+}
+#endif
+
+unsigned ResourceLoadStatisticsDatabaseStore::domainID(const RegistrableDomain& domain) const
+{
+    ASSERT(!RunLoop::isMain());
+
+    unsigned domainID = 0;
+
+    if (m_domainIDFromStringStatement.bindText(1, domain.string()) != SQLITE_OK
+        || m_domainIDFromStringStatement.step() != SQLITE_ROW) {
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::domainIDFromString failed, error message: %{public}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+        return domainID;
+    }
+
+    domainID = m_domainIDFromStringStatement.getColumnInt(0);
+
+    int resetResult = m_domainIDFromStringStatement.reset();
+    ASSERT_UNUSED(resetResult, resetResult == SQLITE_OK);
+    
+    return domainID;
+}
+
+void ResourceLoadStatisticsDatabaseStore::insertDomainRelationships(const WebCore::ResourceLoadStatistics& loadStatistics)
+{
+    ASSERT(!RunLoop::isMain());
+
+    auto registrableDomainID = domainID(loadStatistics.registrableDomain);
+    for (auto& topFrameDomain : loadStatistics.storageAccessUnderTopFrameDomains)
+        insertDomainRelationship(m_storageAccessUnderTopFrameDomainsStatement, registrableDomainID, topFrameDomain);
+
+    for (auto& toDomain : loadStatistics.topFrameUniqueRedirectsTo)
+        insertDomainRelationship(m_topFrameUniqueRedirectsTo, registrableDomainID, toDomain);
+    
+    for (auto& fromDomain : loadStatistics.topFrameUniqueRedirectsFrom)
+        insertDomainRelationship(m_topFrameUniqueRedirectsFrom, registrableDomainID, fromDomain);
+    
+    for (auto& topFrameDomain : loadStatistics.subframeUnderTopFrameDomains)
+        insertDomainRelationship(m_subframeUnderTopFrameDomains, registrableDomainID, topFrameDomain);
+    
+    for (auto& topFrameDomain : loadStatistics.subresourceUnderTopFrameDomains)
+        insertDomainRelationship(m_subresourceUnderTopFrameDomains, registrableDomainID, topFrameDomain);
+    
+    for (auto& toDomain : loadStatistics.subresourceUniqueRedirectsTo)
+        insertDomainRelationship(m_subresourceUniqueRedirectsTo, registrableDomainID, toDomain);
+    
+    for (auto& fromDomain : loadStatistics.subresourceUniqueRedirectsFrom)
+        insertDomainRelationship(m_subresourceUniqueRedirectsFrom, registrableDomainID, fromDomain);
+}
+
+void ResourceLoadStatisticsDatabaseStore::populateFromMemoryStore(const ResourceLoadStatisticsMemoryStore& memoryStore)
+{
+    ASSERT(!RunLoop::isMain());
+
+    if (!isEmpty())
+        return;
+
+    auto& statisticsMap = memoryStore.data();
+    for (const auto& statistic : statisticsMap)
+        insertObservedDomain(statistic.value);
+
+    // Make a separate pass for inter-domain relationships so we
+    // can refer to the ObservedDomain table entries
+    for (auto& statistic : statisticsMap)
+        insertDomainRelationships(statistic.value);
+
+    m_database.runVacuumCommand();
+}
+
+void ResourceLoadStatisticsDatabaseStore::calculateAndSubmitTelemetry() const
+{
+    ASSERT(!RunLoop::isMain());
+
+    // FIXME(195088): Implement for Database version.
+}
+
+static String domainsToString(const HashSet<RegistrableDomain>& domains)
+{
+    StringBuilder builder;
+    for (auto domainName : domains) {
+        if (!builder.isEmpty())
+            builder.appendLiteral(", ");
+        builder.append('"');
+        builder.append(domainName.string());
+        builder.append('"');
+    }
+
+    return builder.toString();
+}
+
+void ResourceLoadStatisticsDatabaseStore::incrementRecordsDeletedCountForDomains(HashSet<RegistrableDomain>&& domains)
+{
+    ASSERT(!RunLoop::isMain());
+
+    SQLiteStatement domainsToUpdateStatement(m_database, makeString("UPDATE ObservedDomains SET dataRecordsRemoved = dataRecordsRemoved + 1 WHERE registrableDomain IN (", domainsToString(domains), ")"));
+    if (domainsToUpdateStatement.prepare() != SQLITE_OK
+        || domainsToUpdateStatement.step() != SQLITE_DONE) {
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::incrementStatisticsForDomains failed, error message: %{public}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+    }
+}
+
+unsigned ResourceLoadStatisticsDatabaseStore::recursivelyFindNonPrevalentDomainsThatRedirectedToThisDomain(unsigned primaryDomainID, StdSet<unsigned>& nonPrevalentRedirectionSources, unsigned numberOfRecursiveCalls)
+{
+    ASSERT(!RunLoop::isMain());
+
+    if (numberOfRecursiveCalls >= maxNumberOfRecursiveCallsInRedirectTraceBack) {
+        RELEASE_LOG(ResourceLoadStatistics, "Hit %u recursive calls in redirect backtrace. Returning early.", maxNumberOfRecursiveCallsInRedirectTraceBack);
+        return numberOfRecursiveCalls;
+    }
+    
+    ++numberOfRecursiveCalls;
+
+    StdSet<unsigned> newlyIdentifiedDomains;
+    SQLiteStatement findSubresources(m_database, makeString("SELECT SubresourceUniqueRedirectsFrom.fromDomainID from SubresourceUniqueRedirectsFrom INNER JOIN ObservedDomains ON ObservedDomains.domainID = SubresourceUniqueRedirectsFrom.fromDomainID WHERE subresourceDomainID = ", String::number(primaryDomainID), "AND ObservedDomains.isPrevalent = 0"));
+    if (findSubresources.prepare() == SQLITE_OK) {
+        while (findSubresources.step() == SQLITE_ROW) {
+            auto newDomainID = findSubresources.getColumnInt(0);
+            auto insertResult = nonPrevalentRedirectionSources.insert(newDomainID);
+            if (insertResult.second)
+                newlyIdentifiedDomains.insert(newDomainID);
+        }
+    }
+
+    SQLiteStatement findTopFrames(m_database, makeString("SELECT TopFrameUniqueRedirectsFrom.fromDomainID from TopFrameUniqueRedirectsFrom INNER JOIN ObservedDomains ON ObservedDomains.domainID = TopFrameUniqueRedirectsFrom.fromDomainID WHERE targetDomainID = ", String::number(primaryDomainID), "AND ObservedDomains.isPrevalent = 0"));
+    if (findTopFrames.prepare() == SQLITE_OK) {
+        while (findTopFrames.step() == SQLITE_ROW) {
+            auto newDomainID = findTopFrames.getColumnInt(0);
+            auto insertResult = nonPrevalentRedirectionSources.insert(newDomainID);
+            if (insertResult.second)
+                newlyIdentifiedDomains.insert(newDomainID);
+        }
+    }
+
+    if (newlyIdentifiedDomains.empty())
+        return numberOfRecursiveCalls;
+
+    for (auto domainID : newlyIdentifiedDomains)
+        numberOfRecursiveCalls = recursivelyFindNonPrevalentDomainsThatRedirectedToThisDomain(domainID, nonPrevalentRedirectionSources, numberOfRecursiveCalls);
+
+    return numberOfRecursiveCalls;
+}
+
+template <typename IteratorType>
+static String buildList(const WTF::IteratorRange<IteratorType>& values)
+{
+    StringBuilder builder;
+    for (auto domainID : values) {
+        if (!builder.isEmpty())
+            builder.appendLiteral(", ");
+        builder.appendNumber(domainID);
+    }
+    return builder.toString();
+}
+
+void ResourceLoadStatisticsDatabaseStore::markAsPrevalentIfHasRedirectedToPrevalent()
+{
+    ASSERT(!RunLoop::isMain());
+
+    StdSet<unsigned> prevalentDueToRedirect;
+    SQLiteStatement subresourceRedirectStatement(m_database, "SELECT DISTINCT SubresourceUniqueRedirectsTo.subresourceDomainID FROM SubresourceUniqueRedirectsTo JOIN ObservedDomains ON ObservedDomains.domainID = SubresourceUniqueRedirectsTo.toDomainID AND ObservedDomains.isPrevalent = 1"_s);
+    if (subresourceRedirectStatement.prepare() == SQLITE_OK) {
+        while (subresourceRedirectStatement.step() == SQLITE_ROW)
+            prevalentDueToRedirect.insert(subresourceRedirectStatement.getColumnInt(0));
+    }
+
+    SQLiteStatement topFrameRedirectStatement(m_database, "SELECT DISTINCT TopFrameUniqueRedirectsTo.sourceDomainID FROM TopFrameUniqueRedirectsTo JOIN ObservedDomains ON ObservedDomains.domainID = TopFrameUniqueRedirectsTo.toDomainID AND ObservedDomains.isPrevalent = 1"_s);
+    if (topFrameRedirectStatement.prepare() == SQLITE_OK) {
+        while (topFrameRedirectStatement.step() == SQLITE_ROW)
+            prevalentDueToRedirect.insert(topFrameRedirectStatement.getColumnInt(0));
+    }
+
+    SQLiteStatement markPrevalentStatement(m_database, makeString("UPDATE ObservedDomains SET isPrevalent = 1 WHERE domainID IN (", buildList(WTF::IteratorRange<StdSet<unsigned>::iterator>(prevalentDueToRedirect.begin(), prevalentDueToRedirect.end())), ")"));
+    if (markPrevalentStatement.prepare() != SQLITE_OK
+        || markPrevalentStatement.step() != SQLITE_DONE) {
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::markAsPrevalentIfHasRedirectedToPrevalent failed to execute, error message: %{public}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+    }
+}
+
+HashMap<unsigned, ResourceLoadStatisticsDatabaseStore::NotVeryPrevalentResources> ResourceLoadStatisticsDatabaseStore::findNotVeryPrevalentResources()
+{
+    ASSERT(!RunLoop::isMain());
+
+    HashMap<unsigned, NotVeryPrevalentResources> results;
+    SQLiteStatement notVeryPrevalentResourcesStatement(m_database, "SELECT domainID, registrableDomain, isPrevalent FROM ObservedDomains WHERE isVeryPrevalent = 0"_s);
+    if (notVeryPrevalentResourcesStatement.prepare() == SQLITE_OK) {
+        while (notVeryPrevalentResourcesStatement.step() == SQLITE_ROW) {
+            unsigned key = static_cast<unsigned>(notVeryPrevalentResourcesStatement.getColumnInt(0));
+            NotVeryPrevalentResources value({ RegistrableDomain::uncheckedCreateFromRegistrableDomainString(notVeryPrevalentResourcesStatement.getColumnText(1))
+                , notVeryPrevalentResourcesStatement.getColumnInt(2) ? ResourceLoadPrevalence::High : ResourceLoadPrevalence::Low
+                , 0, 0, 0, 0 });
+            results.add(key, value);
+        }
+    }
+
+    StringBuilder builder;
+    for (auto value : results.keys()) {
+        if (!builder.isEmpty())
+            builder.appendLiteral(", ");
+        builder.appendNumber(value);
+    }
+
+    auto domainIDsOfInterest = builder.toString();
+
+    SQLiteStatement subresourceUnderTopFrameDomainsStatement(m_database, makeString("SELECT subresourceDomainID, COUNT(topFrameDomainID) FROM SubresourceUnderTopFrameDomains WHERE subresourceDomainID IN (", domainIDsOfInterest, ") GROUP BY subresourceDomainID"));
+    if (subresourceUnderTopFrameDomainsStatement.prepare() == SQLITE_OK) {
+        while (subresourceUnderTopFrameDomainsStatement.step() == SQLITE_ROW) {
+            unsigned domainID = static_cast<unsigned>(subresourceUnderTopFrameDomainsStatement.getColumnInt(0));
+            auto result = results.find(domainID);
+            if (result != results.end())
+                result->value.subresourceUnderTopFrameDomainsCount = static_cast<unsigned>(subresourceUnderTopFrameDomainsStatement.getColumnInt(1));
+        }
+    }
+
+    SQLiteStatement subresourceUniqueRedirectsToCountStatement(m_database, makeString("SELECT subresourceDomainID, COUNT(toDomainID) FROM SubresourceUniqueRedirectsTo WHERE subresourceDomainID IN (", domainIDsOfInterest, ") GROUP BY subresourceDomainID"));
+    if (subresourceUniqueRedirectsToCountStatement.prepare() == SQLITE_OK) {
+        while (subresourceUniqueRedirectsToCountStatement.step() == SQLITE_ROW) {
+            unsigned domainID = static_cast<unsigned>(subresourceUniqueRedirectsToCountStatement.getColumnInt(0));
+            auto result = results.find(domainID);
+            if (result != results.end())
+                result->value.subresourceUniqueRedirectsToCount = static_cast<unsigned>(subresourceUniqueRedirectsToCountStatement.getColumnInt(1));
+        }
+    }
+
+    SQLiteStatement subframeUnderTopFrameDomainsCountStatement(m_database, makeString("SELECT subresourceDomainID, COUNT(topFrameDomainID) FROM SubresourceUnderTopFrameDomains WHERE subresourceDomainID IN (", domainIDsOfInterest, ") GROUP BY subresourceDomainID"));
+    if (subframeUnderTopFrameDomainsCountStatement.prepare() == SQLITE_OK) {
+        while (subframeUnderTopFrameDomainsCountStatement.step() == SQLITE_ROW) {
+            unsigned domainID = static_cast<unsigned>(subframeUnderTopFrameDomainsCountStatement.getColumnInt(0));
+            auto result = results.find(domainID);
+            if (result != results.end())
+                result->value.subframeUnderTopFrameDomainsCount = static_cast<unsigned>(subframeUnderTopFrameDomainsCountStatement.getColumnInt(1));
+        }
+    }
+
+    SQLiteStatement topFrameUniqueRedirectsToCountStatement(m_database, makeString("SELECT sourceDomainID, COUNT(toDomainID) FROM TopFrameUniqueRedirectsTo WHERE sourceDomainID IN (", domainIDsOfInterest, ") GROUP BY sourceDomainID"));
+    if (topFrameUniqueRedirectsToCountStatement.prepare() == SQLITE_OK) {
+        while (topFrameUniqueRedirectsToCountStatement.step() == SQLITE_ROW) {
+            unsigned domainID = static_cast<unsigned>(topFrameUniqueRedirectsToCountStatement.getColumnInt(0));
+            auto result = results.find(domainID);
+            if (result != results.end())
+                result->value.topFrameUniqueRedirectsToCount = static_cast<unsigned>(topFrameUniqueRedirectsToCountStatement.getColumnInt(1));
+        }
+    }
+
+    return results;
+}
+
+void ResourceLoadStatisticsDatabaseStore::reclassifyResources()
+{
+    ASSERT(!RunLoop::isMain());
+
+    auto notVeryPrevalentResources = findNotVeryPrevalentResources();
+
+    for (auto& resourceStatistic : notVeryPrevalentResources.values()) {
+        auto newPrevalence = classifier().calculateResourcePrevalence(resourceStatistic.subresourceUnderTopFrameDomainsCount, resourceStatistic.subresourceUniqueRedirectsToCount, resourceStatistic.subframeUnderTopFrameDomainsCount, resourceStatistic.topFrameUniqueRedirectsToCount, resourceStatistic.prevalence);
+        if (newPrevalence != resourceStatistic.prevalence)
+            setPrevalentResource(resourceStatistic.registerableDomain, newPrevalence);
+    }
+}
+
+void ResourceLoadStatisticsDatabaseStore::classifyPrevalentResources()
+{
+    ASSERT(!RunLoop::isMain());
+    ensurePrevalentResourcesForDebugMode();
+    markAsPrevalentIfHasRedirectedToPrevalent();
+    reclassifyResources();
+}
+
+void ResourceLoadStatisticsDatabaseStore::syncStorageIfNeeded()
+{
+    ASSERT(!RunLoop::isMain());
+    m_database.runVacuumCommand();
+}
+
+void ResourceLoadStatisticsDatabaseStore::syncStorageImmediately()
+{
+    ASSERT(!RunLoop::isMain());
+    m_database.runVacuumCommand();
+}
+
+void ResourceLoadStatisticsDatabaseStore::hasStorageAccess(const SubFrameDomain& subFrameDomain, const TopFrameDomain& topFrameDomain, Optional<FrameID> frameID, PageID pageID, CompletionHandler<void(bool)>&& completionHandler)
+{
+    ASSERT(!RunLoop::isMain());
+
+    ensureResourceStatisticsForRegistrableDomain(subFrameDomain);
+
+    switch (cookieTreatmentForOrigin(subFrameDomain)) {
+    case CookieTreatmentResult::BlockAndPurge:
+        completionHandler(false);
+        return;
+    case CookieTreatmentResult::BlockAndKeep:
+        completionHandler(true);
+        return;
+    case CookieTreatmentResult::Allow:
+        // Do nothing
+        break;
+    };
+
+    RunLoop::main().dispatch([store = makeRef(store()), subFrameDomain = subFrameDomain.isolatedCopy(), topFrameDomain = topFrameDomain.isolatedCopy(), frameID, pageID, completionHandler = WTFMove(completionHandler)]() mutable {
+        store->callHasStorageAccessForFrameHandler(subFrameDomain, topFrameDomain, frameID.value(), pageID, [store = store.copyRef(), completionHandler = WTFMove(completionHandler)](bool result) mutable {
+            store->statisticsQueue().dispatch([completionHandler = WTFMove(completionHandler), result] () mutable {
+                completionHandler(result);
+            });
+        });
+    });
+}
+
+void ResourceLoadStatisticsDatabaseStore::requestStorageAccess(SubFrameDomain&& subFrameDomain, TopFrameDomain&& topFrameDomain, FrameID frameID, PageID pageID, bool promptEnabled, CompletionHandler<void(StorageAccessStatus)>&& completionHandler)
+{
+    ASSERT(!RunLoop::isMain());
+
+    auto subFrameStatus = ensureResourceStatisticsForRegistrableDomain(subFrameDomain);
+
+    switch (cookieTreatmentForOrigin(subFrameDomain)) {
+    case CookieTreatmentResult::BlockAndPurge: {
+        RELEASE_LOG_INFO_IF(debugLoggingEnabled(), ResourceLoadStatisticsDebug, "Cannot grant storage access to %{public}s since its cookies are blocked in third-party contexts and it has not received user interaction as first-party.", subFrameDomain.string().utf8().data());
+        completionHandler(StorageAccessStatus::CannotRequestAccess);
+        }
+        return;
+    case CookieTreatmentResult::BlockAndKeep: {
+        RELEASE_LOG_INFO_IF(debugLoggingEnabled(), ResourceLoadStatisticsDebug, "No need to grant storage access to %{public}s since its cookies are not blocked in third-party contexts.", subFrameDomain.string().utf8().data());
+        completionHandler(StorageAccessStatus::HasAccess);
+        }
+        return;
+    case CookieTreatmentResult::Allow:
+        // Do nothing
+        break;
+    };
+
+    auto userWasPromptedEarlier = promptEnabled && hasUserGrantedStorageAccessThroughPrompt(subFrameStatus.second, topFrameDomain);
+    if (promptEnabled && !userWasPromptedEarlier) {
+        RELEASE_LOG_INFO_IF(debugLoggingEnabled(), ResourceLoadStatisticsDebug, "About to ask the user whether they want to grant storage access to %{public}s under %{public}s or not.", subFrameDomain.string().utf8().data(), topFrameDomain.string().utf8().data());
+        completionHandler(StorageAccessStatus::RequiresUserPrompt);
+        return;
+    }
+
+    if (userWasPromptedEarlier)
+        RELEASE_LOG_INFO_IF(debugLoggingEnabled(), ResourceLoadStatisticsDebug, "Storage access was granted to %{public}s under %{public}s.", subFrameDomain.string().utf8().data(), topFrameDomain.string().utf8().data());
+
+    SQLiteStatement incrementStorageAccess(m_database, makeString("UPDATE ObservedDomains SET timesAccessedAsFirstPartyDueToStorageAccessAPI = timesAccessedAsFirstPartyDueToStorageAccessAPI + 1 WHERE domainID = ", String::number(subFrameStatus.second)));
+    if (incrementStorageAccess.prepare() != SQLITE_OK
+        || incrementStorageAccess.step() != SQLITE_DONE) {
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::requestStorageAccess failed, error message: %{public}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+    }
+    
+    grantStorageAccessInternal(WTFMove(subFrameDomain), WTFMove(topFrameDomain), frameID, pageID, userWasPromptedEarlier, [completionHandler = WTFMove(completionHandler)] (bool wasGrantedAccess) mutable {
+        completionHandler(wasGrantedAccess ? StorageAccessStatus::HasAccess : StorageAccessStatus::CannotRequestAccess);
+    });
+}
+
+void ResourceLoadStatisticsDatabaseStore::requestStorageAccessUnderOpener(DomainInNeedOfStorageAccess&& domainInNeedOfStorageAccess, OpenerPageID openerPageID, OpenerDomain&& openerDomain)
+{
+    ASSERT(domainInNeedOfStorageAccess != openerDomain);
+    ASSERT(!RunLoop::isMain());
+
+    if (domainInNeedOfStorageAccess == openerDomain)
+        return;
+
+    ensureResourceStatisticsForRegistrableDomain(domainInNeedOfStorageAccess);
+
+    if (cookieTreatmentForOrigin(domainInNeedOfStorageAccess) != CookieTreatmentResult::Allow)
+        return;
+
+    RELEASE_LOG_INFO_IF(debugLoggingEnabled(), ResourceLoadStatisticsDebug, "[Temporary combatibility fix] Storage access was granted for %{public}s under opener page from %{public}s, with user interaction in the opened window.", domainInNeedOfStorageAccess.string().utf8().data(), openerDomain.string().utf8().data());
+    grantStorageAccessInternal(WTFMove(domainInNeedOfStorageAccess), WTFMove(openerDomain), WTF::nullopt, openerPageID, false, [](bool) { });
+}
+
+void ResourceLoadStatisticsDatabaseStore::grantStorageAccess(SubFrameDomain&& subFrameDomain, TopFrameDomain&& topFrameDomain, uint64_t frameID, uint64_t pageID, bool userWasPromptedNow, CompletionHandler<void(bool)>&& completionHandler)
+{
+    ASSERT(!RunLoop::isMain());
+
+    if (userWasPromptedNow) {
+        auto subFrameStatus = ensureResourceStatisticsForRegistrableDomain(subFrameDomain);
+        ASSERT(subFrameStatus.first == AddedRecord::No);
+        ASSERT(hasHadUserInteraction(subFrameDomain));
+        insertDomainRelationship(m_storageAccessUnderTopFrameDomainsStatement, subFrameStatus.second, topFrameDomain);
+    }
+
+    grantStorageAccessInternal(WTFMove(subFrameDomain), WTFMove(topFrameDomain), frameID, pageID, userWasPromptedNow, WTFMove(completionHandler));
+}
+
+void ResourceLoadStatisticsDatabaseStore::grantStorageAccessInternal(SubFrameDomain&& subFrameDomain, TopFrameDomain&& topFrameDomain, Optional<FrameID> frameID, PageID pageID, bool userWasPromptedNowOrEarlier, CompletionHandler<void(bool)>&& callback)
+{
+    ASSERT(!RunLoop::isMain());
+
+    if (subFrameDomain == topFrameDomain) {
+        callback(true);
+        return;
+    }
+
+    // FIXME: Remove m_storageAccessPromptsEnabled check if prompting is no longer experimental.
+    if (userWasPromptedNowOrEarlier && storageAccessPromptsEnabled()) {
+#ifndef NDEBUG
+        auto subFrameStatus = ensureResourceStatisticsForRegistrableDomain(subFrameDomain);
+        ASSERT(subFrameStatus.first == AddedRecord::No);
+        ASSERT(hasHadUserInteraction(subFrameDomain));
+        ASSERT(hasUserGrantedStorageAccessThroughPrompt(subFrameStatus.second, topFrameDomain));
+#endif
+        setUserInteraction(subFrameDomain, true, WallTime::now());
+    }
+
+    RunLoop::main().dispatch([subFrameDomain = subFrameDomain.isolatedCopy(), topFrameDomain = topFrameDomain.isolatedCopy(), frameID, pageID, store = makeRef(store()), callback = WTFMove(callback)]() mutable {
+        store->callGrantStorageAccessHandler(subFrameDomain, topFrameDomain, frameID, pageID, [callback = WTFMove(callback), store = store.copyRef()](bool value) mutable {
+            store->statisticsQueue().dispatch([callback = WTFMove(callback), value] () mutable {
+                callback(value);
+            });
+        });
+    });
+
+}
+
+void ResourceLoadStatisticsDatabaseStore::grandfatherDataForDomains(const HashSet<RegistrableDomain>& domains)
+{
+    ASSERT(!RunLoop::isMain());
+
+    SQLiteStatement domainsToUpdateStatement(m_database, makeString("UPDATE ObservedDomains SET grandfathered = 1 WHERE registrableDomain IN (", domainsToString(domains), ")"));
+    if (domainsToUpdateStatement.prepare() != SQLITE_OK
+        || domainsToUpdateStatement.step() != SQLITE_DONE) {
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::grandfatherDataForDomains failed, error message: %{public}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+    }
+}
+
+Vector<RegistrableDomain> ResourceLoadStatisticsDatabaseStore::ensurePrevalentResourcesForDebugMode()
+{
+    ASSERT(!RunLoop::isMain());
+
+    if (!debugModeEnabled())
+        return { };
+
+    Vector<RegistrableDomain> primaryDomainsToBlock;
+    primaryDomainsToBlock.reserveInitialCapacity(2);
+
+    ensureResourceStatisticsForRegistrableDomain(debugStaticPrevalentResource());
+    setPrevalentResource(debugStaticPrevalentResource(), ResourceLoadPrevalence::High);
+    primaryDomainsToBlock.uncheckedAppend(debugStaticPrevalentResource());
+
+    if (!debugManualPrevalentResource().isEmpty()) {
+        ensureResourceStatisticsForRegistrableDomain(debugManualPrevalentResource());
+        setPrevalentResource(debugManualPrevalentResource(), ResourceLoadPrevalence::High);
+        primaryDomainsToBlock.uncheckedAppend(debugManualPrevalentResource());
+        RELEASE_LOG_INFO(ResourceLoadStatisticsDebug, "Did set %{public}s as prevalent resource for the purposes of ITP Debug Mode.", debugManualPrevalentResource().string().utf8().data());
+    }
+
+    return primaryDomainsToBlock;
+}
+
+void ResourceLoadStatisticsDatabaseStore::logFrameNavigation(const RegistrableDomain& targetDomain, const RegistrableDomain& topFrameDomain, const RegistrableDomain& sourceDomain, bool isRedirect, bool isMainFrame)
+{
+    ASSERT(!RunLoop::isMain());
+
+    bool areTargetAndTopFrameDomainsSameSite = targetDomain == topFrameDomain;
+    bool areTargetAndSourceDomainsSameSite = targetDomain == sourceDomain;
+
+    bool statisticsWereUpdated = false;
+    if (!isMainFrame && !(areTargetAndTopFrameDomainsSameSite || areTargetAndSourceDomainsSameSite)) {
+        auto targetResult = ensureResourceStatisticsForRegistrableDomain(targetDomain);
+        updateLastSeen(targetDomain, ResourceLoadStatistics::reduceTimeResolution(WallTime::now()));
+
+        if (!relationshipExists(m_subframeUnderTopFrameDomainExists, targetResult.second, topFrameDomain)) {
+            insertDomainRelationship(m_subframeUnderTopFrameDomains, targetResult.second, topFrameDomain);
+            statisticsWereUpdated = true;
+        }
+    }
+
+    if (isRedirect && !areTargetAndSourceDomainsSameSite) {
+        if (isMainFrame) {
+            auto redirectingDomainResult = ensureResourceStatisticsForRegistrableDomain(sourceDomain);
+            if (!relationshipExists(m_topFrameUniqueRedirectsToExists, redirectingDomainResult.second, targetDomain)) {
+                insertDomainRelationship(m_topFrameUniqueRedirectsTo, redirectingDomainResult.second, targetDomain);
+                statisticsWereUpdated = true;
+            }
+
+            auto targetResult = ensureResourceStatisticsForRegistrableDomain(targetDomain);
+            if (!relationshipExists(m_topFrameUniqueRedirectsFromExists, targetResult.second, sourceDomain)) {
+                insertDomainRelationship(m_topFrameUniqueRedirectsFrom, targetResult.second, sourceDomain);
+                statisticsWereUpdated = true;
+            }
+        } else {
+            auto redirectingDomainResult = ensureResourceStatisticsForRegistrableDomain(sourceDomain);
+            if (!relationshipExists(m_subresourceUniqueRedirectsToExists, redirectingDomainResult.second, targetDomain)) {
+                insertDomainRelationship(m_subresourceUniqueRedirectsTo, redirectingDomainResult.second, targetDomain);
+                statisticsWereUpdated = true;
+            }
+
+            auto targetResult = ensureResourceStatisticsForRegistrableDomain(targetDomain);
+            if (!relationshipExists(m_subresourceUniqueRedirectsFromExists, targetResult.second, sourceDomain)) {
+                insertDomainRelationship(m_subresourceUniqueRedirectsFrom, targetResult.second, sourceDomain);
+                statisticsWereUpdated = true;
+            }
+        }
+    }
+
+    if (statisticsWereUpdated)
+        scheduleStatisticsProcessingRequestIfNecessary();
+}
+
+void ResourceLoadStatisticsDatabaseStore::logSubresourceLoading(const SubResourceDomain& targetDomain, const TopFrameDomain& topFrameDomain, WallTime lastSeen)
+{
+    ASSERT(!RunLoop::isMain());
+
+    auto result = ensureResourceStatisticsForRegistrableDomain(targetDomain);
+    updateLastSeen(targetDomain, lastSeen);
+
+    auto targetDomainID = result.second;
+    if (!relationshipExists(m_subresourceUnderTopFrameDomainExists, targetDomainID, topFrameDomain)) {
+        insertDomainRelationship(m_subresourceUnderTopFrameDomains, targetDomainID, topFrameDomain);
+        scheduleStatisticsProcessingRequestIfNecessary();
+    }
+}
+
+void ResourceLoadStatisticsDatabaseStore::logSubresourceRedirect(const RedirectedFromDomain& sourceDomain, const RedirectedToDomain& targetDomain)
+{
+    ASSERT(!RunLoop::isMain());
+
+    auto sourceDomainResult = ensureResourceStatisticsForRegistrableDomain(sourceDomain);
+    auto targetDomainResult = ensureResourceStatisticsForRegistrableDomain(targetDomain);
+
+    bool isNewRedirectToEntry = false;
+    if (!relationshipExists(m_subresourceUniqueRedirectsToExists, sourceDomainResult.second, targetDomain)) {
+        insertDomainRelationship(m_subresourceUniqueRedirectsTo, sourceDomainResult.second, targetDomain);
+        isNewRedirectToEntry = true;
+    }
+
+    bool isNewRedirectFromEntry = false;
+    if (!relationshipExists(m_subresourceUniqueRedirectsFromExists, targetDomainResult.second, sourceDomain)) {
+        insertDomainRelationship(m_subresourceUniqueRedirectsFrom, targetDomainResult.second, sourceDomain);
+        isNewRedirectFromEntry = true;
+    }
+
+    if (isNewRedirectToEntry || isNewRedirectFromEntry)
+        scheduleStatisticsProcessingRequestIfNecessary();
+}
+
+void ResourceLoadStatisticsDatabaseStore::setUserInteraction(const RegistrableDomain& domain, bool hadUserInteraction, WallTime mostRecentInteraction)
+{
+    ASSERT(!RunLoop::isMain());
+
+    if (m_mostRecentUserInteractionStatement.bindInt(1, hadUserInteraction)
+        || m_mostRecentUserInteractionStatement.bindDouble(2, mostRecentInteraction.secondsSinceEpoch().value()) != SQLITE_OK
+        || m_mostRecentUserInteractionStatement.bindText(3, domain.string()) != SQLITE_OK
+        || m_mostRecentUserInteractionStatement.step() != SQLITE_DONE) {
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::setUserInteraction, error message: %{public}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+        return;
+    }
+
+    int resetResult = m_mostRecentUserInteractionStatement.reset();
+    ASSERT_UNUSED(resetResult, resetResult == SQLITE_OK);
+}
+
+void ResourceLoadStatisticsDatabaseStore::logUserInteraction(const TopFrameDomain& domain)
+{
+    ASSERT(!RunLoop::isMain());
+
+    ensureResourceStatisticsForRegistrableDomain(domain);
+    setUserInteraction(domain, true, WallTime::now());
+}
+
+void ResourceLoadStatisticsDatabaseStore::clearUserInteraction(const RegistrableDomain& domain)
+{
+    ASSERT(!RunLoop::isMain());
+
+    auto targetResult = ensureResourceStatisticsForRegistrableDomain(domain);
+    setUserInteraction(domain, false, { });
+
+    SQLiteStatement removeStorageAccess(m_database, makeString("DELETE FROM StorageAccessUnderTopFrameDomains WHERE domainID = ", String::number(targetResult.second)));
+    if (removeStorageAccess.prepare() != SQLITE_OK
+        || removeStorageAccess.step() != SQLITE_DONE) {
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::logUserInteraction failed to bind, error message: %{public}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+    }
+}
+
+bool ResourceLoadStatisticsDatabaseStore::hasHadUserInteraction(const RegistrableDomain& domain)
+{
+    ASSERT(!RunLoop::isMain());
+
+    if (m_hadUserInteractionStatement.bindText(1, domain.string()) != SQLITE_OK
+        || m_hadUserInteractionStatement.step() != SQLITE_DONE) {
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::m_updatePrevalentResourceStatement failed, error message: %{public}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+        return false;
+    }
+
+    bool hadUserInteraction = !!m_hadUserInteractionStatement.getColumnInt(0);
+    if (!hadUserInteraction)
+        return false;
+    
+    WallTime mostRecentUserInteractionTime = WallTime::fromRawSeconds(m_hadUserInteractionStatement.getColumnDouble(1));
+
+    if (hasStatisticsExpired(mostRecentUserInteractionTime)) {
+        // Drop privacy sensitive data because we no longer need it.
+        // Set timestamp to 0 so that statistics merge will know
+        // it has been reset as opposed to its default -1.
+        clearUserInteraction(domain);
+        hadUserInteraction = false;
+    }
+    
+    return hadUserInteraction;
+}
+
+void ResourceLoadStatisticsDatabaseStore::setPrevalentResource(const RegistrableDomain& domain, ResourceLoadPrevalence newPrevalence)
+{
+    ASSERT(!RunLoop::isMain());
+
+    if (m_updatePrevalentResourceStatement.bindInt(1, 1) != SQLITE_OK
+        || m_updatePrevalentResourceStatement.bindText(2, domain.string()) != SQLITE_OK
+        || m_updatePrevalentResourceStatement.step() != SQLITE_DONE) {
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::m_updatePrevalentResourceStatement failed, error message: %{public}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+        return;
+    }
+    
+    int resetResult = m_updatePrevalentResourceStatement.reset();
+    ASSERT_UNUSED(resetResult, resetResult == SQLITE_OK);
+
+    if (newPrevalence == ResourceLoadPrevalence::VeryHigh) {
+        if (m_updateVeryPrevalentResourceStatement.bindInt(1, 1) != SQLITE_OK
+            || m_updateVeryPrevalentResourceStatement.bindText(2, domain.string()) != SQLITE_OK
+            || m_updateVeryPrevalentResourceStatement.step() != SQLITE_DONE) {
+            RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::m_updateVeryPrevalentResourceStatement failed, error message: %{public}s", this, m_database.lastErrorMsg());
+            ASSERT_NOT_REACHED();
+            return;
+        }
+
+        int resetResult = m_updateVeryPrevalentResourceStatement.reset();
+        ASSERT_UNUSED(resetResult, resetResult == SQLITE_OK);
+    }
+
+    StdSet<unsigned> nonPrevalentRedirectionSources;
+    recursivelyFindNonPrevalentDomainsThatRedirectedToThisDomain(domainID(domain), nonPrevalentRedirectionSources, 0);
+    setDomainsAsPrevalent(WTFMove(nonPrevalentRedirectionSources));
+}
+
+void ResourceLoadStatisticsDatabaseStore::setDomainsAsPrevalent(StdSet<unsigned>&& domains)
+{
+    ASSERT(!RunLoop::isMain());
+
+    SQLiteStatement domainsToUpdateStatement(m_database, makeString("UPDATE ObservedDomains SET isPrevalent = 1 WHERE domainID IN (", buildList(WTF::IteratorRange<StdSet<unsigned>::iterator>(domains.begin(), domains.end())), ")"));
+    if (domainsToUpdateStatement.prepare() != SQLITE_OK
+        || domainsToUpdateStatement.step() != SQLITE_DONE) {
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::setDomainsAsPrevalent failed, error message: %{public}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+        return;
+    }
+}
+
+String ResourceLoadStatisticsDatabaseStore::dumpResourceLoadStatistics() const
+{
+    ASSERT(!RunLoop::isMain());
+
+    // FIXME(195088): Implement SQLite-based dumping routines.
+    ASSERT_NOT_REACHED();
+    return { };
+}
+
+bool ResourceLoadStatisticsDatabaseStore::predicateValueForDomain(WebCore::SQLiteStatement& predicateStatement, const RegistrableDomain& domain) const
+{
+    ASSERT(!RunLoop::isMain());
+    
+    if (predicateStatement.bindText(1, domain.string()) != SQLITE_OK
+        || predicateStatement.step() != SQLITE_DONE) {
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::predicateValueForDomain failed to bind, error message: %{public}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+        return false;
+    }
+
+    bool result = !!predicateStatement.getColumnInt(0);
+    int resetResult = predicateStatement.reset();
+    ASSERT_UNUSED(resetResult, resetResult == SQLITE_OK);
+    return result;
+}
+
+bool ResourceLoadStatisticsDatabaseStore::isPrevalentResource(const RegistrableDomain& domain) const
+{
+    ASSERT(!RunLoop::isMain());
+
+    return predicateValueForDomain(m_isPrevalentResourceStatement, domain);
+}
+
+bool ResourceLoadStatisticsDatabaseStore::isVeryPrevalentResource(const RegistrableDomain& domain) const
+{
+    ASSERT(!RunLoop::isMain());
+
+    return predicateValueForDomain(m_isVeryPrevalentResourceStatement, domain);
+}
+
+bool ResourceLoadStatisticsDatabaseStore::isRegisteredAsSubresourceUnder(const SubResourceDomain& subresourceDomain, const TopFrameDomain& topFrameDomain) const
+{
+    ASSERT(!RunLoop::isMain());
+
+    return relationshipExists(m_subresourceUnderTopFrameDomainExists, domainID(subresourceDomain), topFrameDomain);
+}
+
+bool ResourceLoadStatisticsDatabaseStore::isRegisteredAsSubFrameUnder(const SubFrameDomain& subFrameDomain, const TopFrameDomain& topFrameDomain) const
+{
+    ASSERT(!RunLoop::isMain());
+
+    return relationshipExists(m_subframeUnderTopFrameDomainExists, domainID(subFrameDomain), topFrameDomain);
+}
+
+bool ResourceLoadStatisticsDatabaseStore::isRegisteredAsRedirectingTo(const RedirectedFromDomain& redirectedFromDomain, const RedirectedToDomain& redirectedToDomain) const
+{
+    ASSERT(!RunLoop::isMain());
+
+    return relationshipExists(m_subresourceUniqueRedirectsToExists, domainID(redirectedFromDomain), redirectedToDomain);
+}
+
+void ResourceLoadStatisticsDatabaseStore::clearPrevalentResource(const RegistrableDomain& domain)
+{
+    ASSERT(!RunLoop::isMain());
+
+    ensureResourceStatisticsForRegistrableDomain(domain);
+    
+    if (m_clearPrevalentResourceStatement.bindText(1, domain.string()) != SQLITE_OK
+        || m_clearPrevalentResourceStatement.step() != SQLITE_DONE) {
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::clearPrevalentResource, error message: %{public}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+        return;
+    }
+    
+    int resetResult = m_clearPrevalentResourceStatement.reset();
+    ASSERT_UNUSED(resetResult, resetResult == SQLITE_OK);
+}
+
+void ResourceLoadStatisticsDatabaseStore::setGrandfathered(const RegistrableDomain& domain, bool value)
+{
+    ASSERT(!RunLoop::isMain());
+
+    ensureResourceStatisticsForRegistrableDomain(domain);
+    
+    if (m_updateGrandfatheredStatement.bindInt(1, value) != SQLITE_OK
+        || m_updateGrandfatheredStatement.bindText(1, domain.string()) != SQLITE_OK
+        || m_updateGrandfatheredStatement.step() != SQLITE_DONE) {
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::setGrandfathered failed to bind, error message: %{public}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+        return;
+    }
+    
+    int resetResult = m_updateGrandfatheredStatement.reset();
+    ASSERT_UNUSED(resetResult, resetResult == SQLITE_OK);
+}
+
+bool ResourceLoadStatisticsDatabaseStore::isGrandfathered(const RegistrableDomain& domain) const
+{
+    ASSERT(!RunLoop::isMain());
+
+    return predicateValueForDomain(m_isGrandfatheredStatement, domain);
+}
+
+void ResourceLoadStatisticsDatabaseStore::setSubframeUnderTopFrameDomain(const SubFrameDomain& subFrameDomain, const TopFrameDomain& topFrameDomain)
+{
+    ASSERT(!RunLoop::isMain());
+
+    auto result = ensureResourceStatisticsForRegistrableDomain(subFrameDomain);
+
+    // For consistency, make sure we also have a statistics entry for the top frame domain.
+    ensureResourceStatisticsForRegistrableDomain(topFrameDomain);
+
+    insertDomainRelationship(m_subframeUnderTopFrameDomains, result.second, topFrameDomain);
+}
+
+void ResourceLoadStatisticsDatabaseStore::setSubresourceUnderTopFrameDomain(const SubResourceDomain& subresourceDomain, const TopFrameDomain& topFrameDomain)
+{
+    ASSERT(!RunLoop::isMain());
+
+    auto result = ensureResourceStatisticsForRegistrableDomain(subresourceDomain);
+
+    // For consistency, make sure we also have a statistics entry for the top frame domain.
+    ensureResourceStatisticsForRegistrableDomain(topFrameDomain);
+
+    insertDomainRelationship(m_subresourceUnderTopFrameDomains, result.second, topFrameDomain);
+}
+
+void ResourceLoadStatisticsDatabaseStore::setSubresourceUniqueRedirectTo(const SubResourceDomain& subresourceDomain, const RedirectDomain& redirectDomain)
+{
+    ASSERT(!RunLoop::isMain());
+
+    auto result = ensureResourceStatisticsForRegistrableDomain(subresourceDomain);
+
+    // For consistency, make sure we also have a statistics entry for the redirect domain.
+    ensureResourceStatisticsForRegistrableDomain(redirectDomain);
+
+    insertDomainRelationship(m_subresourceUniqueRedirectsTo, result.second, redirectDomain);
+}
+
+void ResourceLoadStatisticsDatabaseStore::setSubresourceUniqueRedirectFrom(const SubResourceDomain& subresourceDomain, const RedirectDomain& redirectDomain)
+{
+    ASSERT(!RunLoop::isMain());
+
+    auto result = ensureResourceStatisticsForRegistrableDomain(subresourceDomain);
+
+    // For consistency, make sure we also have a statistics entry for the redirect domain.
+    ensureResourceStatisticsForRegistrableDomain(redirectDomain);
+
+    insertDomainRelationship(m_subresourceUniqueRedirectsFrom, result.second, redirectDomain);
+}
+
+void ResourceLoadStatisticsDatabaseStore::setTopFrameUniqueRedirectTo(const TopFrameDomain& topFrameDomain, const RedirectDomain& redirectDomain)
+{
+    ASSERT(!RunLoop::isMain());
+
+    auto result = ensureResourceStatisticsForRegistrableDomain(topFrameDomain);
+
+    // For consistency, make sure we also have a statistics entry for the redirect domain.
+    ensureResourceStatisticsForRegistrableDomain(redirectDomain);
+
+    insertDomainRelationship(m_topFrameUniqueRedirectsTo, result.second, redirectDomain);
+}
+
+void ResourceLoadStatisticsDatabaseStore::setTopFrameUniqueRedirectFrom(const TopFrameDomain& topFrameDomain, const RedirectDomain& redirectDomain)
+{
+    ASSERT(!RunLoop::isMain());
+
+    auto result = ensureResourceStatisticsForRegistrableDomain(topFrameDomain);
+
+    // For consistency, make sure we also have a statistics entry for the redirect domain.
+    ensureResourceStatisticsForRegistrableDomain(redirectDomain);
+
+    insertDomainRelationship(m_topFrameUniqueRedirectsFrom, result.second, redirectDomain);
+}
+
+std::pair<ResourceLoadStatisticsDatabaseStore::AddedRecord, unsigned> ResourceLoadStatisticsDatabaseStore::ensureResourceStatisticsForRegistrableDomain(const RegistrableDomain& domain)
+{
+    ASSERT(!RunLoop::isMain());
+
+    if (m_domainIDFromStringStatement.bindText(1, domain.string()) != SQLITE_OK) {
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::createSchema failed, error message: %{public}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+        return { AddedRecord::No, 0 };
+    }
+    
+    if (m_domainIDFromStringStatement.step() == SQLITE_ROW) {
+        unsigned domainID = m_domainIDFromStringStatement.getColumnInt(0);
+
+        int resetResult = m_domainIDFromStringStatement.reset();
+        ASSERT_UNUSED(resetResult, resetResult == SQLITE_OK);
+        return { AddedRecord::No, domainID };
+    }
+
+    int resetResult = m_domainIDFromStringStatement.reset();
+    ASSERT_UNUSED(resetResult, resetResult == SQLITE_OK);
+
+    ResourceLoadStatistics newObservation(domain);
+    insertObservedDomain(newObservation);
+
+    return { AddedRecord::Yes, domainID(domain) };
+}
+
+void ResourceLoadStatisticsDatabaseStore::clear(CompletionHandler<void()>&& completionHandler)
+{
+    ASSERT(!RunLoop::isMain());
+
+    clearOperatingDates();
+
+    auto callbackAggregator = CallbackAggregator::create(WTFMove(completionHandler));
+
+    removeAllStorageAccess([callbackAggregator = callbackAggregator.copyRef()] { });
+
+    auto primaryDomainsToBlock = ensurePrevalentResourcesForDebugMode();
+    updateCookieBlockingForDomains(primaryDomainsToBlock, [callbackAggregator = callbackAggregator.copyRef()] { });
+}
+
+ResourceLoadStatisticsDatabaseStore::CookieTreatmentResult ResourceLoadStatisticsDatabaseStore::cookieTreatmentForOrigin(const RegistrableDomain& domain) const
+{
+    ASSERT(!RunLoop::isMain());
+
+    SQLiteStatement statement(m_database, makeString("SELECT isPrevalent, hadUserInteraction FROM ObservedDomains WHERE registrableDomain = ", domain.string()));
+    if (statement.prepare() != SQLITE_OK
+        || statement.step() != SQLITE_ROW) {
+        return CookieTreatmentResult::Allow;
+    }
+    
+    bool isPrevalent = !!statement.getColumnInt(0);
+    if (!isPrevalent)
+        return CookieTreatmentResult::Allow;
+
+    bool hadUserInteraction = statement.getColumnInt(1) ? true : false;
+    return hadUserInteraction ? CookieTreatmentResult::BlockAndKeep : CookieTreatmentResult::BlockAndPurge;
+}
+    
+bool ResourceLoadStatisticsDatabaseStore::hasUserGrantedStorageAccessThroughPrompt(unsigned requestingDomainID, const RegistrableDomain& firstPartyDomain) const
+{
+    ASSERT(!RunLoop::isMain());
+
+    auto firstPartyPrimaryDomainID = domainID(firstPartyDomain);
+
+    SQLiteStatement statement(m_database, makeString("SELECT COUNT(*) FROM StorageAccessUnderTopFrameDomains WHERE domainID = ", String::number(requestingDomainID), " AND topLevelDomainID = ", String::number(firstPartyPrimaryDomainID)));
+    if (statement.prepare() != SQLITE_OK
+        || statement.step() != SQLITE_ROW)
+        return false;
+
+    return !statement.getColumnInt(0);
+}
+
+Vector<RegistrableDomain> ResourceLoadStatisticsDatabaseStore::domainsToBlock() const
+{
+    ASSERT(!RunLoop::isMain());
+
+    Vector<RegistrableDomain> results;
+    SQLiteStatement statement(m_database, "SELECT registrableDomain FROM ObservedDomains WHERE isPrevalent = 1"_s);
+    if (statement.prepare() != SQLITE_OK)
+        return results;
+    
+    while (statement.step() == SQLITE_ROW)
+        results.append(RegistrableDomain::uncheckedCreateFromRegistrableDomainString(statement.getColumnText(0)));
+
+    return results;
+}
+
+void ResourceLoadStatisticsDatabaseStore::updateCookieBlocking(CompletionHandler<void()>&& completionHandler)
+{
+    ASSERT(!RunLoop::isMain());
+
+    auto domainsToBlock = this->domainsToBlock();
+
+    if (domainsToBlock.isEmpty()) {
+        completionHandler();
+        return;
+    }
+
+    if (debugLoggingEnabled() && !domainsToBlock.isEmpty())
+        debugLogDomainsInBatches("block", domainsToBlock);
+
+    RunLoop::main().dispatch([weakThis = makeWeakPtr(*this), store = makeRef(store()), domainsToBlock = crossThreadCopy(domainsToBlock), completionHandler = WTFMove(completionHandler)] () mutable {
+        store->callUpdatePrevalentDomainsToBlockCookiesForHandler(domainsToBlock, [weakThis = WTFMove(weakThis), store = store.copyRef(), completionHandler = WTFMove(completionHandler)]() mutable {
+            store->statisticsQueue().dispatch([weakThis = WTFMove(weakThis), completionHandler = WTFMove(completionHandler)]() mutable {
+                completionHandler();
+                if (!weakThis)
+                    return;
+                RELEASE_LOG_INFO_IF(weakThis->debugLoggingEnabled(), ResourceLoadStatisticsDebug, "Done updating cookie blocking.");
+            });
+        });
+    });
+}
+
+Vector<ResourceLoadStatisticsDatabaseStore::PrevalentDomainData> ResourceLoadStatisticsDatabaseStore::prevalentDomains() const
+{
+    ASSERT(!RunLoop::isMain());
+    
+    Vector<PrevalentDomainData> results;
+    SQLiteStatement statement(m_database, "SELECT domainID, registrableDomain, mostRecentUserInteractionTime, hadUserInteraction, grandfathered FROM ObservedDomains WHERE isPrevalent = 1"_s);
+    if (statement.prepare() != SQLITE_OK)
+        return results;
+    
+    while (statement.step() == SQLITE_ROW) {
+        results.append({ static_cast<unsigned>(statement.getColumnInt(0))
+            , RegistrableDomain::uncheckedCreateFromRegistrableDomainString(statement.getColumnText(1))
+            , WallTime::fromRawSeconds(statement.getColumnDouble(2))
+            , statement.getColumnInt(3) ? true : false
+            , statement.getColumnInt(4) ? true : false
+        });
+    }
+    
+    return results;
+}
+
+Vector<unsigned> ResourceLoadStatisticsDatabaseStore::findExpiredUserInteractions() const
+{
+    ASSERT(!RunLoop::isMain());
+
+    Vector<unsigned> results;
+    Optional<Seconds> expirationDateTime = statisticsEpirationTime();
+    if (!expirationDateTime)
+        return results;
+
+    if (m_findExpiredUserInteractionStatement.bindDouble(1, expirationDateTime.value().value()) != SQLITE_OK)
+        return results;
+
+    while (m_findExpiredUserInteractionStatement.step() == SQLITE_ROW)
+        results.append(m_findExpiredUserInteractionStatement.getColumnInt(0));
+
+    int resetResult = m_findExpiredUserInteractionStatement.reset();
+    ASSERT_UNUSED(resetResult, resetResult == SQLITE_OK);
+
+    return results;
+}
+
+void ResourceLoadStatisticsDatabaseStore::clearExpiredUserInteractions()
+{
+    ASSERT(!RunLoop::isMain());
+
+    auto expiredRecords = findExpiredUserInteractions();
+    if (expiredRecords.isEmpty())
+        return;
+
+    auto expiredRecordIDs = buildList(WTF::IteratorRange<Vector<unsigned>::iterator>(expiredRecords.begin(), expiredRecords.end()));
+
+    SQLiteStatement clearExpiredInteraction(m_database, makeString("UPDATE ObservedDomains SET mostRecentUserInteractionTime = 0, hadUserInteraction = 1 WHERE domainID IN (", expiredRecordIDs, ")"));
+    if (clearExpiredInteraction.prepare() != SQLITE_OK)
+        return;
+
+    SQLiteStatement removeStorageAccess(m_database, makeString("DELETE FROM StorageAccessUnderTopFrameDomains ", expiredRecordIDs, ")"));
+    if (removeStorageAccess.prepare() != SQLITE_OK)
+        return;
+
+    if (clearExpiredInteraction.step() != SQLITE_DONE
+        || removeStorageAccess.step() != SQLITE_DONE) {
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::logUserInteraction failed to bind, error message: %{public}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+    }
+}
+
+void ResourceLoadStatisticsDatabaseStore::clearGrandfathering(Vector<unsigned>&& domainIDsToClear)
+{
+    ASSERT(!RunLoop::isMain());
+
+    if (domainIDsToClear.isEmpty())
+        return;
+
+    auto listToClear = buildList(WTF::IteratorRange<Vector<unsigned>::iterator>(domainIDsToClear.begin(), domainIDsToClear.end()));
+
+    SQLiteStatement clearGrandfatheringStatement(m_database, makeString("UPDATE ObservedDomains SET grandfathered = 0 WHERE domainID IN (", listToClear, ")"));
+    if (clearGrandfatheringStatement.prepare() != SQLITE_OK)
+        return;
+    
+    if (clearGrandfatheringStatement.step() != SQLITE_DONE) {
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::clearGrandfathering failed to bind, error message: %{public}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+    }
+}
+
+Vector<RegistrableDomain> ResourceLoadStatisticsDatabaseStore::registrableDomainsToRemoveWebsiteDataFor()
+{
+    ASSERT(!RunLoop::isMain());
+
+    bool shouldCheckForGrandfathering = endOfGrandfatheringTimestamp() > WallTime::now();
+    bool shouldClearGrandfathering = !shouldCheckForGrandfathering && endOfGrandfatheringTimestamp();
+
+    if (shouldClearGrandfathering)
+        clearEndOfGrandfatheringTimeStamp();
+
+    clearExpiredUserInteractions();
+    
+    Vector<RegistrableDomain> prevalentResources;
+
+    Vector<PrevalentDomainData> prevalentDomains = this->prevalentDomains();
+    Vector<unsigned> domainIDsToClearGrandfathering;
+    for (auto& statistic : prevalentDomains) {
+        if (!statistic.hadUserInteraction && (!shouldCheckForGrandfathering || !statistic.grandfathered))
+            prevalentResources.append(statistic.registerableDomain);
+
+        if (shouldClearGrandfathering && statistic.grandfathered)
+            domainIDsToClearGrandfathering.append(statistic.domainID);
+    }
+
+    clearGrandfathering(WTFMove(domainIDsToClearGrandfathering));
+    
+    return prevalentResources;
+}
+
+void ResourceLoadStatisticsDatabaseStore::pruneStatisticsIfNeeded()
+{
+    ASSERT(!RunLoop::isMain());
+
+    unsigned count = 0;
+    if (m_observedDomainCount.step() == SQLITE_ROW)
+        count = m_observedDomainCount.getColumnInt(0);
+
+    int resetResult = m_observedDomainCount.reset();
+    ASSERT_UNUSED(resetResult, resetResult == SQLITE_OK);
+
+    if (count <= parameters().maxStatisticsEntries)
+        return;
+
+    ASSERT(parameters().pruneEntriesDownTo <= parameters().maxStatisticsEntries);
+
+    size_t countLeftToPrune = count - parameters().pruneEntriesDownTo;
+    ASSERT(countLeftToPrune);
+
+    SQLiteStatement recordsToPrune(m_database, makeString("SELECT domainID FROM ObservedDomains ORDER BY hadUserInteraction, isPrevalent, lastSeen DESC LIMIT ", String::number(countLeftToPrune)));
+    if (recordsToPrune.prepare() != SQLITE_OK) {
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::pruneStatisticsIfNeeded failed, error message: %{public}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+        return;
+    }
+
+    Vector<unsigned> entriesToPrune;
+    while (recordsToPrune.step() == SQLITE_ROW)
+        entriesToPrune.append(recordsToPrune.getColumnInt(0));
+
+    auto listToPrune = buildList(WTF::IteratorRange<Vector<unsigned>::iterator>(entriesToPrune.begin(), entriesToPrune.end()));
+
+    SQLiteStatement pruneCommand(m_database, makeString("DELETE from ObservedDomains WHERE domainID IN (", listToPrune, ")"));
+    if (pruneCommand.prepare() != SQLITE_OK
+        || pruneCommand.step() != SQLITE_DONE) {
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::pruneStatisticsIfNeeded failed, error message: %{public}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+        return;
+    }
+}
+
+void ResourceLoadStatisticsDatabaseStore::updateLastSeen(const RegistrableDomain& domain, WallTime lastSeen)
+{
+    ASSERT(!RunLoop::isMain());
+
+    if (m_updateLastSeenStatement.bindDouble(1, lastSeen.secondsSinceEpoch().value()) != SQLITE_OK
+        || m_updateLastSeenStatement.bindText(2, domain.string()) != SQLITE_OK
+        || m_updateLastSeenStatement.step() != SQLITE_DONE) {
+        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::updateLastSeen failed to bind, error message: %{public}s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+        return;
+    }
+    
+    int resetResult = m_updateLastSeenStatement.reset();
+    ASSERT_UNUSED(resetResult, resetResult == SQLITE_OK);
+}
+
+void ResourceLoadStatisticsDatabaseStore::setLastSeen(const RegistrableDomain& domain, Seconds seconds)
+{
+    ASSERT(!RunLoop::isMain());
+
+    ensureResourceStatisticsForRegistrableDomain(domain);
+    updateLastSeen(domain, WallTime::fromRawSeconds(seconds.seconds()));
+}
+
+void ResourceLoadStatisticsDatabaseStore::setPrevalentResource(const RegistrableDomain& domain)
+{
+    ASSERT(!RunLoop::isMain());
+
+    ensureResourceStatisticsForRegistrableDomain(domain);
+    setPrevalentResource(domain, ResourceLoadPrevalence::High);
+}
+
+void ResourceLoadStatisticsDatabaseStore::setVeryPrevalentResource(const RegistrableDomain& domain)
+{
+    ASSERT(!RunLoop::isMain());
+
+    ensureResourceStatisticsForRegistrableDomain(domain);
+    setPrevalentResource(domain, ResourceLoadPrevalence::VeryHigh);
+}
+
+} // namespace WebKit
+
+#endif
diff --git a/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h b/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h
new file mode 100644 (file)
index 0000000..d20bc32
--- /dev/null
@@ -0,0 +1,204 @@
+/*
+ * Copyright (C) 2019 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+
+#include "ResourceLoadStatisticsStore.h"
+#include "WebResourceLoadStatisticsStore.h"
+#include <WebCore/SQLiteDatabase.h>
+#include <WebCore/SQLiteStatement.h>
+#include <wtf/CompletionHandler.h>
+#include <wtf/UniqueRef.h>
+#include <wtf/Vector.h>
+#include <wtf/WeakPtr.h>
+#include <wtf/WorkQueue.h>
+
+namespace WebCore {
+class SQLiteDatabase;
+class SQLiteStatement;
+struct ResourceLoadStatistics;
+}
+
+namespace WebKit {
+
+class ResourceLoadStatisticsMemoryStore;
+
+// This is always constructed / used / destroyed on the WebResourceLoadStatisticsStore's statistics queue.
+class ResourceLoadStatisticsDatabaseStore final : public ResourceLoadStatisticsStore {
+public:
+    ResourceLoadStatisticsDatabaseStore(WebResourceLoadStatisticsStore&, WorkQueue&, const String& storageDirectoryPath);
+
+    void populateFromMemoryStore(const ResourceLoadStatisticsMemoryStore&);
+
+    void clear(CompletionHandler<void()>&&) override;
+    bool isEmpty() const override;
+
+    void updateCookieBlocking(CompletionHandler<void()>&&) override;
+
+    void classifyPrevalentResources() override;
+    void syncStorageIfNeeded() override;
+    void syncStorageImmediately() override;
+
+    void requestStorageAccessUnderOpener(DomainInNeedOfStorageAccess&&, OpenerPageID, OpenerDomain&&) override;
+
+    void grandfatherDataForDomains(const HashSet<RegistrableDomain>&) override;
+
+    bool isRegisteredAsSubresourceUnder(const SubResourceDomain&, const TopFrameDomain&) const override;
+    bool isRegisteredAsSubFrameUnder(const SubFrameDomain&, const TopFrameDomain&) const override;
+    bool isRegisteredAsRedirectingTo(const RedirectedFromDomain&, const RedirectedToDomain&) const override;
+
+    void clearPrevalentResource(const RegistrableDomain&) override;
+    String dumpResourceLoadStatistics() const override;
+    bool isPrevalentResource(const RegistrableDomain&) const override;
+    bool isVeryPrevalentResource(const RegistrableDomain&) const override;
+    void setPrevalentResource(const RegistrableDomain&) override;
+    void setVeryPrevalentResource(const RegistrableDomain&) override;
+
+    void setGrandfathered(const RegistrableDomain&, bool value) override;
+    bool isGrandfathered(const RegistrableDomain&) const override;
+
+    void setSubframeUnderTopFrameDomain(const SubFrameDomain&, const TopFrameDomain&) override;
+    void setSubresourceUnderTopFrameDomain(const SubResourceDomain&, const TopFrameDomain&) override;
+    void setSubresourceUniqueRedirectTo(const SubResourceDomain&, const RedirectDomain&) override;
+    void setSubresourceUniqueRedirectFrom(const SubResourceDomain&, const RedirectDomain&) override;
+    void setTopFrameUniqueRedirectTo(const TopFrameDomain&, const RedirectDomain&) override;
+    void setTopFrameUniqueRedirectFrom(const TopFrameDomain&, const RedirectDomain&) override;
+
+    void calculateAndSubmitTelemetry() const override;
+
+    void hasStorageAccess(const SubFrameDomain&, const TopFrameDomain&, Optional<FrameID>, PageID, CompletionHandler<void(bool)>&&) override;
+    void requestStorageAccess(SubFrameDomain&&, TopFrameDomain&&, FrameID, PageID, bool promptEnabled, CompletionHandler<void(StorageAccessStatus)>&&) override;
+    void grantStorageAccess(SubFrameDomain&&, TopFrameDomain&&, FrameID, PageID, bool userWasPromptedNow, CompletionHandler<void(bool)>&&) override;
+
+    void logFrameNavigation(const NavigatedToDomain&, const TopFrameDomain&, const NavigatedFromDomain&, bool isRedirect, bool isMainFrame) override;
+    void logUserInteraction(const TopFrameDomain&) override;
+    void logSubresourceLoading(const SubResourceDomain&, const TopFrameDomain&, WallTime lastSeen) override;
+    void logSubresourceRedirect(const RedirectedFromDomain&, const RedirectedToDomain&) override;
+
+    void clearUserInteraction(const RegistrableDomain&) override;
+    bool hasHadUserInteraction(const RegistrableDomain&) override;
+
+    void setLastSeen(const RegistrableDomain&, Seconds) override;
+
+private:
+    bool insertObservedDomain(const ResourceLoadStatistics&);
+    void insertDomainRelationships(const ResourceLoadStatistics&);
+    bool insertDomainRelationship(WebCore::SQLiteStatement&, unsigned domainID, const RegistrableDomain& topFrameDomain);
+    bool relationshipExists(WebCore::SQLiteStatement&, unsigned firstDomainID, const RegistrableDomain& secondDomain) const;
+    unsigned domainID(const RegistrableDomain&) const;
+#ifndef NDEBUG
+    bool confirmDomainDoesNotExist(const RegistrableDomain&) const;
+#endif
+    void updateLastSeen(const RegistrableDomain&, WallTime);
+    void setUserInteraction(const RegistrableDomain&, bool hadUserInteraction, WallTime);
+    Vector<RegistrableDomain> domainsToBlock() const;
+
+    struct PrevalentDomainData {
+        unsigned domainID;
+        RegistrableDomain registerableDomain;
+        WallTime mostRecentUserInteractionTime;
+        bool hadUserInteraction;
+        bool grandfathered;
+    };
+    Vector<PrevalentDomainData> prevalentDomains() const;
+    Vector<unsigned> findExpiredUserInteractions() const;
+    void clearExpiredUserInteractions();
+    void clearGrandfathering(Vector<unsigned>&&);
+    bool hasUserGrantedStorageAccessThroughPrompt(unsigned domainID, const RegistrableDomain&) const;
+    void incrementRecordsDeletedCountForDomains(HashSet<RegistrableDomain>&&) override;
+
+    void reclassifyResources();
+    struct NotVeryPrevalentResources {
+        RegistrableDomain registerableDomain;
+        ResourceLoadPrevalence prevalence;
+        unsigned subresourceUnderTopFrameDomainsCount;
+        unsigned subresourceUniqueRedirectsToCount;
+        unsigned subframeUnderTopFrameDomainsCount;
+        unsigned topFrameUniqueRedirectsToCount;
+    };
+    HashMap<unsigned, NotVeryPrevalentResources> findNotVeryPrevalentResources();
+
+    bool predicateValueForDomain(WebCore::SQLiteStatement&, const RegistrableDomain&) const;
+
+    enum class CookieTreatmentResult { Allow, BlockAndKeep, BlockAndPurge };
+    CookieTreatmentResult cookieTreatmentForOrigin(const RegistrableDomain&) const;
+    
+    void setPrevalentResource(const RegistrableDomain&, ResourceLoadPrevalence);
+    unsigned recursivelyFindNonPrevalentDomainsThatRedirectedToThisDomain(unsigned primaryDomainID, StdSet<unsigned>& nonPrevalentRedirectionSources, unsigned numberOfRecursiveCalls);
+    void setDomainsAsPrevalent(StdSet<unsigned>&&);
+    void grantStorageAccessInternal(SubFrameDomain&&, TopFrameDomain&&, Optional<FrameID>, PageID, bool userWasPromptedNowOrEarlier, CompletionHandler<void(bool)>&&);
+    void markAsPrevalentIfHasRedirectedToPrevalent();
+    Vector<RegistrableDomain> ensurePrevalentResourcesForDebugMode() override;
+    void removeDataRecords(CompletionHandler<void()>&&);
+    void pruneStatisticsIfNeeded() override;
+    enum class AddedRecord { No, Yes };
+    std::pair<AddedRecord, unsigned> ensureResourceStatisticsForRegistrableDomain(const RegistrableDomain&);
+    Vector<RegistrableDomain> registrableDomainsToRemoveWebsiteDataFor() override;
+    bool isDatabaseStore() const final { return true; }
+
+    bool createSchema();
+    bool prepareStatements();
+    
+    const String m_storageDirectoryPath;
+    mutable WebCore::SQLiteDatabase m_database;
+    mutable WebCore::SQLiteStatement m_observedDomainCount;
+    WebCore::SQLiteStatement m_insertObservedDomainStatement;
+    WebCore::SQLiteStatement m_insertTopLevelDomainStatement;
+    mutable WebCore::SQLiteStatement m_domainIDFromStringStatement;
+    WebCore::SQLiteStatement m_storageAccessUnderTopFrameDomainsStatement;
+    WebCore::SQLiteStatement m_topFrameUniqueRedirectsTo;
+    mutable WebCore::SQLiteStatement m_topFrameUniqueRedirectsToExists;
+    WebCore::SQLiteStatement m_topFrameUniqueRedirectsFrom;
+    mutable WebCore::SQLiteStatement m_topFrameUniqueRedirectsFromExists;
+    WebCore::SQLiteStatement m_subframeUnderTopFrameDomains;
+    mutable WebCore::SQLiteStatement m_subframeUnderTopFrameDomainExists;
+    WebCore::SQLiteStatement m_subresourceUnderTopFrameDomains;
+    mutable WebCore::SQLiteStatement m_subresourceUnderTopFrameDomainExists;
+    WebCore::SQLiteStatement m_subresourceUniqueRedirectsTo;
+    mutable WebCore::SQLiteStatement m_subresourceUniqueRedirectsToExists;
+    WebCore::SQLiteStatement m_subresourceUniqueRedirectsFrom;
+    mutable WebCore::SQLiteStatement m_subresourceUniqueRedirectsFromExists;
+    WebCore::SQLiteStatement m_mostRecentUserInteractionStatement;
+    WebCore::SQLiteStatement m_updateLastSeenStatement;
+    WebCore::SQLiteStatement m_updatePrevalentResourceStatement;
+    mutable WebCore::SQLiteStatement m_isPrevalentResourceStatement;
+    WebCore::SQLiteStatement m_updateVeryPrevalentResourceStatement;
+    mutable WebCore::SQLiteStatement m_isVeryPrevalentResourceStatement;
+    WebCore::SQLiteStatement m_clearPrevalentResourceStatement;
+    mutable WebCore::SQLiteStatement m_hadUserInteractionStatement;
+    WebCore::SQLiteStatement m_updateGrandfatheredStatement;
+    mutable WebCore::SQLiteStatement m_isGrandfatheredStatement;
+    mutable WebCore::SQLiteStatement m_findExpiredUserInteractionStatement;
+};
+
+} // namespace WebKit
+
+SPECIALIZE_TYPE_TRAITS_BEGIN(WebKit::ResourceLoadStatisticsDatabaseStore)
+    static bool isType(const WebKit::ResourceLoadStatisticsStore& store) { return store.isDatabaseStore(); }
+SPECIALIZE_TYPE_TRAITS_END()
+
+#endif
index 17ca57f..5861dba 100644 (file)
@@ -49,94 +49,6 @@ namespace WebKit {
 using namespace WebCore;
 
 constexpr unsigned statisticsModelVersion { 15 };
-constexpr unsigned maxNumberOfRecursiveCallsInRedirectTraceBack { 50 };
-constexpr Seconds minimumStatisticsProcessingInterval { 5_s };
-constexpr unsigned operatingDatesWindow { 30 };
-constexpr unsigned maxImportance { 3 };
-
-#if !RELEASE_LOG_DISABLED
-static String domainsToString(const Vector<RegistrableDomain>& domains)
-{
-    StringBuilder builder;
-    for (auto& domain : domains) {
-        if (!builder.isEmpty())
-            builder.appendLiteral(", ");
-        builder.append(domain.string());
-    }
-    return builder.toString();
-}
-#endif
-
-class OperatingDate {
-public:
-    OperatingDate() = default;
-
-    static OperatingDate fromWallTime(WallTime time)
-    {
-        double ms = time.secondsSinceEpoch().milliseconds();
-        int year = msToYear(ms);
-        int yearDay = dayInYear(ms, year);
-        int month = monthFromDayInYear(yearDay, isLeapYear(year));
-        int monthDay = dayInMonthFromDayInYear(yearDay, isLeapYear(year));
-
-        return OperatingDate { year, month, monthDay };
-    }
-
-    static OperatingDate today()
-    {
-        return OperatingDate::fromWallTime(WallTime::now());
-    }
-
-    Seconds secondsSinceEpoch() const
-    {
-        return Seconds { dateToDaysFrom1970(m_year, m_month, m_monthDay) * secondsPerDay };
-    }
-
-    bool operator==(const OperatingDate& other) const
-    {
-        return m_monthDay == other.m_monthDay && m_month == other.m_month && m_year == other.m_year;
-    }
-
-    bool operator<(const OperatingDate& other) const
-    {
-        return secondsSinceEpoch() < other.secondsSinceEpoch();
-    }
-
-    bool operator<=(const OperatingDate& other) const
-    {
-        return secondsSinceEpoch() <= other.secondsSinceEpoch();
-    }
-
-private:
-    OperatingDate(int year, int month, int monthDay)
-        : m_year(year)
-        , m_month(month)
-        , m_monthDay(monthDay)
-    { }
-
-    int m_year { 0 };
-    int m_month { 0 }; // [0, 11].
-    int m_monthDay { 0 }; // [1, 31].
-};
-
-static Vector<OperatingDate> mergeOperatingDates(const Vector<OperatingDate>& existingDates, Vector<OperatingDate>&& newDates)
-{
-    if (existingDates.isEmpty())
-        return WTFMove(newDates);
-
-    Vector<OperatingDate> mergedDates(existingDates.size() + newDates.size());
-
-    // Merge the two sorted vectors of dates.
-    std::merge(existingDates.begin(), existingDates.end(), newDates.begin(), newDates.end(), mergedDates.begin());
-    // Remove duplicate dates.
-    removeRepeatedElements(mergedDates);
-
-    // Drop old dates until the Vector size reaches operatingDatesWindow.
-    while (mergedDates.size() > operatingDatesWindow)
-        mergedDates.remove(0);
-
-    return mergedDates;
-}
 
 struct StatisticsLastSeen {
     RegistrableDomain domain;
@@ -155,36 +67,24 @@ static void pruneResources(HashMap<RegistrableDomain, ResourceLoadStatistics>& s
         statisticsMap.remove(statisticsToPrune[i].domain);
 }
 
-static unsigned computeImportance(const ResourceLoadStatistics& resourceStatistic)
-{
-    unsigned importance = maxImportance;
-    if (!resourceStatistic.isPrevalentResource)
-        importance -= 1;
-    if (!resourceStatistic.hadUserInteraction)
-        importance -= 2;
-    return importance;
-}
-
 ResourceLoadStatisticsMemoryStore::ResourceLoadStatisticsMemoryStore(WebResourceLoadStatisticsStore& store, WorkQueue& workQueue)
-    : m_store(store)
-    , m_workQueue(workQueue)
+    : ResourceLoadStatisticsStore(store, workQueue)
 {
     ASSERT(!RunLoop::isMain());
 
 #if PLATFORM(COCOA)
     registerUserDefaultsIfNeeded();
 #endif
-    includeTodayAsOperatingDateIfNecessary();
 
-    m_workQueue->dispatchAfter(5_s, [weakThis = makeWeakPtr(*this)] {
+    workQueue.dispatchAfter(5_s, [weakThis = makeWeakPtr(*this)] {
         if (weakThis)
             weakThis->calculateAndSubmitTelemetry();
     });
 }
 
-ResourceLoadStatisticsMemoryStore::~ResourceLoadStatisticsMemoryStore()
+bool ResourceLoadStatisticsMemoryStore::isEmpty() const
 {
-    ASSERT(!RunLoop::isMain());
+    return m_resourceStatisticsMap.isEmpty();
 }
 
 void ResourceLoadStatisticsMemoryStore::setPersistentStorage(ResourceLoadStatisticsPersistentStorage& persistentStorage)
@@ -196,79 +96,16 @@ void ResourceLoadStatisticsMemoryStore::calculateAndSubmitTelemetry() const
 {
     ASSERT(!RunLoop::isMain());
 
-    if (m_parameters.shouldSubmitTelemetry)
+    if (parameters().shouldSubmitTelemetry)
         WebResourceLoadStatisticsTelemetry::calculateAndSubmit(*this);
 }
 
-void ResourceLoadStatisticsMemoryStore::setNotifyPagesWhenDataRecordsWereScanned(bool value)
-{
-    ASSERT(!RunLoop::isMain());
-    m_parameters.shouldNotifyPagesWhenDataRecordsWereScanned = value;
-}
-
-void ResourceLoadStatisticsMemoryStore::setShouldClassifyResourcesBeforeDataRecordsRemoval(bool value)
-{
-    ASSERT(!RunLoop::isMain());
-    m_parameters.shouldClassifyResourcesBeforeDataRecordsRemoval = value;
-}
-
-void ResourceLoadStatisticsMemoryStore::setShouldSubmitTelemetry(bool value)
-{
-    ASSERT(!RunLoop::isMain());
-    m_parameters.shouldSubmitTelemetry = value;
-}
-
-void ResourceLoadStatisticsMemoryStore::removeDataRecords(CompletionHandler<void()>&& callback)
+void ResourceLoadStatisticsMemoryStore::incrementRecordsDeletedCountForDomains(HashSet<RegistrableDomain>&& domainsWithDeletedWebsiteData)
 {
-    ASSERT(!RunLoop::isMain());
-
-    if (!shouldRemoveDataRecords()) {
-        callback();
-        return;
-    }
-
-#if ENABLE(NETSCAPE_PLUGIN_API)
-    m_activePluginTokens.clear();
-    for (const auto& plugin : PluginProcessManager::singleton().pluginProcesses())
-        m_activePluginTokens.add(plugin->pluginProcessToken());
-#endif
-
-    auto prevalentResourceDomains = registrableDomainsToRemoveWebsiteDataFor();
-    if (prevalentResourceDomains.isEmpty()) {
-        callback();
-        return;
+    for (auto& domain : domainsWithDeletedWebsiteData) {
+        auto& statistic = ensureResourceStatisticsForRegistrableDomain(domain);
+        ++statistic.dataRecordsRemoved;
     }
-
-#if !RELEASE_LOG_DISABLED
-    RELEASE_LOG_INFO_IF(m_debugLoggingEnabled, ResourceLoadStatisticsDebug, "About to remove data records for %{public}s.", domainsToString(prevalentResourceDomains).utf8().data());
-#endif
-
-    setDataRecordsBeingRemoved(true);
-
-    RunLoop::main().dispatch([prevalentResourceDomains = crossThreadCopy(prevalentResourceDomains), callback = WTFMove(callback), weakThis = makeWeakPtr(*this), shouldNotifyPagesWhenDataRecordsWereScanned = m_parameters.shouldNotifyPagesWhenDataRecordsWereScanned, workQueue = m_workQueue.copyRef()] () mutable {
-        if (!weakThis) {
-            callback();
-            return;
-        }
-
-        weakThis->m_store.deleteWebsiteDataForRegistrableDomainsInAllPersistentDataStores(WebResourceLoadStatisticsStore::monitoredDataTypes(), WTFMove(prevalentResourceDomains), shouldNotifyPagesWhenDataRecordsWereScanned, [callback = WTFMove(callback), weakThis = WTFMove(weakThis), workQueue = workQueue.copyRef()](const HashSet<RegistrableDomain>& domainsWithDeletedWebsiteData) mutable {
-            workQueue->dispatch([domainsWithDeletedWebsiteData = crossThreadCopy(domainsWithDeletedWebsiteData), callback = WTFMove(callback), weakThis = WTFMove(weakThis)] () mutable {
-                if (!weakThis) {
-                    callback();
-                    return;
-                }
-                for (auto& domain : domainsWithDeletedWebsiteData) {
-                    auto& statistic = weakThis->ensureResourceStatisticsForRegistrableDomain(domain);
-                    ++statistic.dataRecordsRemoved;
-                }
-                weakThis->setDataRecordsBeingRemoved(false);
-                callback();
-#if !RELEASE_LOG_DISABLED
-                RELEASE_LOG_INFO_IF(weakThis->m_debugLoggingEnabled, ResourceLoadStatisticsDebug, "Done removing data records.");
-#endif
-            });
-        });
-    });
 }
 
 unsigned ResourceLoadStatisticsMemoryStore::recursivelyGetAllDomainsThatHaveRedirectedToThisDomain(const ResourceLoadStatistics& resourceStatistic, HashSet<RegistrableDomain>& domainsThatHaveRedirectedTo, unsigned numberOfRecursiveCalls) const
@@ -329,53 +166,40 @@ void ResourceLoadStatisticsMemoryStore::markAsPrevalentIfHasRedirectedToPrevalen
 
 bool ResourceLoadStatisticsMemoryStore::isPrevalentDueToDebugMode(ResourceLoadStatistics& resourceStatistic)
 {
-    if (!m_debugModeEnabled)
+    if (!debugModeEnabled())
         return false;
 
-    return resourceStatistic.registrableDomain == m_debugStaticPrevalentResource || resourceStatistic.registrableDomain == m_debugManualPrevalentResource;
+    return resourceStatistic.registrableDomain == debugStaticPrevalentResource() || resourceStatistic.registrableDomain == debugManualPrevalentResource();
 }
 
-void ResourceLoadStatisticsMemoryStore::processStatisticsAndDataRecords()
+void ResourceLoadStatisticsMemoryStore::classifyPrevalentResources()
 {
-    ASSERT(!RunLoop::isMain());
-
-    if (m_parameters.shouldClassifyResourcesBeforeDataRecordsRemoval) {
-        for (auto& resourceStatistic : m_resourceStatisticsMap.values()) {
-            if (isPrevalentDueToDebugMode(resourceStatistic))
-                setPrevalentResource(resourceStatistic, ResourceLoadPrevalence::High);
-            else if (!resourceStatistic.isVeryPrevalentResource) {
-                markAsPrevalentIfHasRedirectedToPrevalent(resourceStatistic);
-                auto currentPrevalence = resourceStatistic.isPrevalentResource ? ResourceLoadPrevalence::High : ResourceLoadPrevalence::Low;
-                auto newPrevalence = m_resourceLoadStatisticsClassifier.calculateResourcePrevalence(resourceStatistic, currentPrevalence);
-                if (newPrevalence != currentPrevalence)
-                    setPrevalentResource(resourceStatistic, newPrevalence);
-            }
+    for (auto& resourceStatistic : m_resourceStatisticsMap.values()) {
+        if (isPrevalentDueToDebugMode(resourceStatistic))
+            setPrevalentResource(resourceStatistic, ResourceLoadPrevalence::High);
+        else if (!resourceStatistic.isVeryPrevalentResource) {
+            markAsPrevalentIfHasRedirectedToPrevalent(resourceStatistic);
+            auto currentPrevalence = resourceStatistic.isPrevalentResource ? ResourceLoadPrevalence::High : ResourceLoadPrevalence::Low;
+            auto newPrevalence = classifier().calculateResourcePrevalence(resourceStatistic, currentPrevalence);
+            if (newPrevalence != currentPrevalence)
+                setPrevalentResource(resourceStatistic, newPrevalence);
         }
     }
+}
 
-    removeDataRecords([this, weakThis = makeWeakPtr(*this)]() mutable {
-        ASSERT(!RunLoop::isMain());
-        if (!weakThis)
-            return;
-
-        pruneStatisticsIfNeeded();
-        if (m_persistentStorage)
-            m_persistentStorage->scheduleOrWriteMemoryStore(ResourceLoadStatisticsPersistentStorage::ForceImmediateWrite::No);
-
-        if (!m_parameters.shouldNotifyPagesWhenDataRecordsWereScanned)
-            return;
-
-        RunLoop::main().dispatch([this, weakThis = WTFMove(weakThis)]() {
-            ASSERT(RunLoop::isMain());
-            if (!weakThis)
-                return;
+void ResourceLoadStatisticsMemoryStore::syncStorageIfNeeded()
+{
+    if (m_persistentStorage)
+        m_persistentStorage->scheduleOrWriteMemoryStore(ResourceLoadStatisticsPersistentStorage::ForceImmediateWrite::No);
+}
 
-            m_store.notifyResourceLoadStatisticsProcessed();
-        });
-    });
+void ResourceLoadStatisticsMemoryStore::syncStorageImmediately()
+{
+    if (m_persistentStorage)
+        m_persistentStorage->scheduleOrWriteMemoryStore(ResourceLoadStatisticsPersistentStorage::ForceImmediateWrite::Yes);
 }
 
-void ResourceLoadStatisticsMemoryStore::hasStorageAccess(const RegistrableDomain& subFrameDomain, const RegistrableDomain& topFrameDomain, Optional<uint64_t> frameID, uint64_t pageID, CompletionHandler<void(bool)>&& completionHandler)
+void ResourceLoadStatisticsMemoryStore::hasStorageAccess(const SubFrameDomain& subFrameDomain, const TopFrameDomain& topFrameDomain, Optional<FrameID> frameID, PageID pageID, CompletionHandler<void(bool)>&& completionHandler)
 {
     ASSERT(!RunLoop::isMain());
 
@@ -390,7 +214,7 @@ void ResourceLoadStatisticsMemoryStore::hasStorageAccess(const RegistrableDomain
         return;
     }
 
-    RunLoop::main().dispatch([store = makeRef(m_store), subFrameDomain = subFrameDomain.isolatedCopy(), topFrameDomain = topFrameDomain.isolatedCopy(), frameID, pageID, completionHandler = WTFMove(completionHandler)]() mutable {
+    RunLoop::main().dispatch([store = makeRef(store()), subFrameDomain = subFrameDomain.isolatedCopy(), topFrameDomain = topFrameDomain.isolatedCopy(), frameID, pageID, completionHandler = WTFMove(completionHandler)]() mutable {
         store->callHasStorageAccessForFrameHandler(subFrameDomain, topFrameDomain, frameID.value(), pageID, [store = store.copyRef(), completionHandler = WTFMove(completionHandler)](bool result) mutable {
             store->statisticsQueue().dispatch([completionHandler = WTFMove(completionHandler), result] () mutable {
                 completionHandler(result);
@@ -399,14 +223,14 @@ void ResourceLoadStatisticsMemoryStore::hasStorageAccess(const RegistrableDomain
     });
 }
 
-void ResourceLoadStatisticsMemoryStore::requestStorageAccess(RegistrableDomain&& subFrameDomain, RegistrableDomain&& topFrameDomain, uint64_t frameID, uint64_t pageID, bool promptEnabled, CompletionHandler<void(StorageAccessStatus)>&& completionHandler)
+void ResourceLoadStatisticsMemoryStore::requestStorageAccess(SubFrameDomain&& subFrameDomain, TopFrameDomain&& topFrameDomain, FrameID frameID, uint64_t pageID, bool promptEnabled, CompletionHandler<void(StorageAccessStatus)>&& completionHandler)
 {
     ASSERT(!RunLoop::isMain());
 
     auto& subFrameStatistic = ensureResourceStatisticsForRegistrableDomain(subFrameDomain);
     if (shouldBlockAndPurgeCookies(subFrameStatistic)) {
 #if !RELEASE_LOG_DISABLED
-        RELEASE_LOG_INFO_IF(m_debugLoggingEnabled, ResourceLoadStatisticsDebug, "Cannot grant storage access to %{public}s since its cookies are blocked in third-party contexts and it has not received user interaction as first-party.", subFrameDomain.string().utf8().data());
+        RELEASE_LOG_INFO_IF(debugLoggingEnabled(), ResourceLoadStatisticsDebug, "Cannot grant storage access to %{public}s since its cookies are blocked in third-party contexts and it has not received user interaction as first-party.", subFrameDomain.string().utf8().data());
 #endif
         completionHandler(StorageAccessStatus::CannotRequestAccess);
         return;
@@ -414,7 +238,7 @@ void ResourceLoadStatisticsMemoryStore::requestStorageAccess(RegistrableDomain&&
 
     if (!shouldBlockAndKeepCookies(subFrameStatistic)) {
 #if !RELEASE_LOG_DISABLED
-        RELEASE_LOG_INFO_IF(m_debugLoggingEnabled, ResourceLoadStatisticsDebug, "No need to grant storage access to %{public}s since its cookies are not blocked in third-party contexts.", subFrameDomain.string().utf8().data());
+        RELEASE_LOG_INFO_IF(debugLoggingEnabled(), ResourceLoadStatisticsDebug, "No need to grant storage access to %{public}s since its cookies are not blocked in third-party contexts.", subFrameDomain.string().utf8().data());
 #endif
         completionHandler(StorageAccessStatus::HasAccess);
         return;
@@ -423,13 +247,13 @@ void ResourceLoadStatisticsMemoryStore::requestStorageAccess(RegistrableDomain&&
     auto userWasPromptedEarlier = promptEnabled && hasUserGrantedStorageAccessThroughPrompt(subFrameStatistic, topFrameDomain);
     if (promptEnabled && !userWasPromptedEarlier) {
 #if !RELEASE_LOG_DISABLED
-        RELEASE_LOG_INFO_IF(m_debugLoggingEnabled, ResourceLoadStatisticsDebug, "About to ask the user whether they want to grant storage access to %{public}s under %{public}s or not.", subFrameDomain.string().utf8().data(), topFrameDomain.string().utf8().data());
+        RELEASE_LOG_INFO_IF(debugLoggingEnabled(), ResourceLoadStatisticsDebug, "About to ask the user whether they want to grant storage access to %{public}s under %{public}s or not.", subFrameDomain.string().utf8().data(), topFrameDomain.string().utf8().data());
 #endif
         completionHandler(StorageAccessStatus::RequiresUserPrompt);
         return;
     } else if (userWasPromptedEarlier) {
 #if !RELEASE_LOG_DISABLED
-        RELEASE_LOG_INFO_IF(m_debugLoggingEnabled, ResourceLoadStatisticsDebug, "Storage access was granted to %{public}s under %{public}s.", subFrameDomain.string().utf8().data(), topFrameDomain.string().utf8().data());
+        RELEASE_LOG_INFO_IF(debugLoggingEnabled(), ResourceLoadStatisticsDebug, "Storage access was granted to %{public}s under %{public}s.", subFrameDomain.string().utf8().data(), topFrameDomain.string().utf8().data());
 #endif
     }
 
@@ -440,7 +264,7 @@ void ResourceLoadStatisticsMemoryStore::requestStorageAccess(RegistrableDomain&&
     });
 }
 
-void ResourceLoadStatisticsMemoryStore::requestStorageAccessUnderOpener(RegistrableDomain&& domainInNeedOfStorageAccess, uint64_t openerPageID, RegistrableDomain&& openerDomain)
+void ResourceLoadStatisticsMemoryStore::requestStorageAccessUnderOpener(DomainInNeedOfStorageAccess&& domainInNeedOfStorageAccess, OpenerPageID openerPageID, OpenerDomain&& openerDomain)
 {
     ASSERT(domainInNeedOfStorageAccess != openerDomain);
     ASSERT(!RunLoop::isMain());
@@ -456,12 +280,12 @@ void ResourceLoadStatisticsMemoryStore::requestStorageAccessUnderOpener(Registra
         return;
 
 #if !RELEASE_LOG_DISABLED
-    RELEASE_LOG_INFO_IF(m_debugLoggingEnabled, ResourceLoadStatisticsDebug, "[Temporary combatibility fix] Storage access was granted for %{public}s under opener page from %{public}s, with user interaction in the opened window.", domainInNeedOfStorageAccess.string().utf8().data(), openerDomain.string().utf8().data());
+    RELEASE_LOG_INFO_IF(debugLoggingEnabled(), ResourceLoadStatisticsDebug, "[Temporary combatibility fix] Storage access was granted for %{public}s under opener page from %{public}s, with user interaction in the opened window.", domainInNeedOfStorageAccess.string().utf8().data(), openerDomain.string().utf8().data());
 #endif
     grantStorageAccessInternal(WTFMove(domainInNeedOfStorageAccess), WTFMove(openerDomain), WTF::nullopt, openerPageID, false, [](bool) { });
 }
 
-void ResourceLoadStatisticsMemoryStore::grantStorageAccess(RegistrableDomain&& subFrameDomain, RegistrableDomain&& topFrameDomain, uint64_t frameID, uint64_t pageID, bool userWasPromptedNow, CompletionHandler<void(bool)>&& completionHandler)
+void ResourceLoadStatisticsMemoryStore::grantStorageAccess(SubFrameDomain&& subFrameDomain, TopFrameDomain&& topFrameDomain, uint64_t frameID, uint64_t pageID, bool userWasPromptedNow, CompletionHandler<void(bool)>&& completionHandler)
 {
     ASSERT(!RunLoop::isMain());
 
@@ -473,7 +297,7 @@ void ResourceLoadStatisticsMemoryStore::grantStorageAccess(RegistrableDomain&& s
     grantStorageAccessInternal(WTFMove(subFrameDomain), WTFMove(topFrameDomain), frameID, pageID, userWasPromptedNow, WTFMove(completionHandler));
 }
 
-void ResourceLoadStatisticsMemoryStore::grantStorageAccessInternal(RegistrableDomain&& subFrameDomain, RegistrableDomain&& topFrameDomain, Optional<uint64_t> frameID, uint64_t pageID, bool userWasPromptedNowOrEarlier, CompletionHandler<void(bool)>&& callback)
+void ResourceLoadStatisticsMemoryStore::grantStorageAccessInternal(SubFrameDomain&& subFrameDomain, TopFrameDomain&& topFrameDomain, Optional<FrameID> frameID, PageID pageID, bool userWasPromptedNowOrEarlier, CompletionHandler<void(bool)>&& callback)
 {
     ASSERT(!RunLoop::isMain());
 
@@ -483,14 +307,14 @@ void ResourceLoadStatisticsMemoryStore::grantStorageAccessInternal(RegistrableDo
     }
 
     // FIXME: Remove m_storageAccessPromptsEnabled check if prompting is no longer experimental.
-    if (userWasPromptedNowOrEarlier && m_storageAccessPromptsEnabled) {
+    if (userWasPromptedNowOrEarlier && storageAccessPromptsEnabled()) {
         auto& subFrameStatistic = ensureResourceStatisticsForRegistrableDomain(subFrameDomain);
         ASSERT(subFrameStatistic.hadUserInteraction);
         ASSERT(subFrameStatistic.storageAccessUnderTopFrameDomains.contains(topFrameDomain));
         subFrameStatistic.mostRecentUserInteractionTime = WallTime::now();
     }
 
-    RunLoop::main().dispatch([subFrameDomain = subFrameDomain.isolatedCopy(), topFrameDomain = topFrameDomain.isolatedCopy(), frameID, pageID, store = makeRef(m_store), callback = WTFMove(callback)]() mutable {
+    RunLoop::main().dispatch([subFrameDomain = subFrameDomain.isolatedCopy(), topFrameDomain = topFrameDomain.isolatedCopy(), frameID, pageID, store = makeRef(store()), callback = WTFMove(callback)]() mutable {
         store->callGrantStorageAccessHandler(subFrameDomain, topFrameDomain, frameID, pageID, [callback = WTFMove(callback), store = store.copyRef()](bool value) mutable {
             store->statisticsQueue().dispatch([callback = WTFMove(callback), value] () mutable {
                 callback(value);
@@ -499,105 +323,38 @@ void ResourceLoadStatisticsMemoryStore::grantStorageAccessInternal(RegistrableDo
     });
 }
 
-void ResourceLoadStatisticsMemoryStore::grandfatherExistingWebsiteData(CompletionHandler<void()>&& callback)
+void ResourceLoadStatisticsMemoryStore::grandfatherDataForDomains(const HashSet<RegistrableDomain>& domains)
 {
-    ASSERT(!RunLoop::isMain());
-
-    RunLoop::main().dispatch([weakThis = makeWeakPtr(*this), callback = WTFMove(callback), shouldNotifyPagesWhenDataRecordsWereScanned = m_parameters.shouldNotifyPagesWhenDataRecordsWereScanned, workQueue = m_workQueue.copyRef(), store = makeRef(m_store)] () mutable {
-        store->registrableDomainsWithWebsiteData(WebResourceLoadStatisticsStore::monitoredDataTypes(), shouldNotifyPagesWhenDataRecordsWereScanned, [weakThis = WTFMove(weakThis), callback = WTFMove(callback), workQueue = workQueue.copyRef()] (HashSet<RegistrableDomain>&& domainsWithWebsiteData) mutable {
-            workQueue->dispatch([weakThis = WTFMove(weakThis), domainsWithWebsiteData = crossThreadCopy(domainsWithWebsiteData), callback = WTFMove(callback)] () mutable {
-                if (!weakThis) {
-                    callback();
-                    return;
-                }
-
-                for (auto& domain : domainsWithWebsiteData) {
-                    auto& statistic = weakThis->ensureResourceStatisticsForRegistrableDomain(domain);
-                    statistic.grandfathered = true;
-                }
-                weakThis->m_endOfGrandfatheringTimestamp = WallTime::now() + weakThis->m_parameters.grandfatheringTime;
-                if (weakThis->m_persistentStorage)
-                    weakThis->m_persistentStorage->scheduleOrWriteMemoryStore(ResourceLoadStatisticsPersistentStorage::ForceImmediateWrite::Yes);
-                callback();
-                weakThis->logTestingEvent("Grandfathered"_s);
-            });
-        });
-    });
+    for (auto& domain : domains) {
+        auto& statistic = ensureResourceStatisticsForRegistrableDomain(domain);
+        statistic.grandfathered = true;
+    }
 }
 
 Vector<RegistrableDomain> ResourceLoadStatisticsMemoryStore::ensurePrevalentResourcesForDebugMode()
 {
-    if (!m_debugModeEnabled)
+    if (!debugModeEnabled())
         return { };
 
     Vector<RegistrableDomain> domainsToBlock;
     domainsToBlock.reserveInitialCapacity(2);
 
-    auto& staticSesourceStatistic = ensureResourceStatisticsForRegistrableDomain(m_debugStaticPrevalentResource);
+    auto& staticSesourceStatistic = ensureResourceStatisticsForRegistrableDomain(debugStaticPrevalentResource());
     setPrevalentResource(staticSesourceStatistic, ResourceLoadPrevalence::High);
-    domainsToBlock.uncheckedAppend(m_debugStaticPrevalentResource);
+    domainsToBlock.uncheckedAppend(debugStaticPrevalentResource());
 
-    if (!m_debugManualPrevalentResource.isEmpty()) {
-        auto& manualResourceStatistic = ensureResourceStatisticsForRegistrableDomain(m_debugManualPrevalentResource);
+    if (!debugManualPrevalentResource().isEmpty()) {
+        auto& manualResourceStatistic = ensureResourceStatisticsForRegistrableDomain(debugManualPrevalentResource());
         setPrevalentResource(manualResourceStatistic, ResourceLoadPrevalence::High);
-        domainsToBlock.uncheckedAppend(m_debugManualPrevalentResource);
+        domainsToBlock.uncheckedAppend(debugManualPrevalentResource());
 #if !RELEASE_LOG_DISABLED
-        RELEASE_LOG_INFO(ResourceLoadStatisticsDebug, "Did set %{public}s as prevalent resource for the purposes of ITP Debug Mode.", m_debugManualPrevalentResource.string().utf8().data());
+        RELEASE_LOG_INFO(ResourceLoadStatisticsDebug, "Did set %{public}s as prevalent resource for the purposes of ITP Debug Mode.", debugManualPrevalentResource().string().utf8().data());
 #endif
     }
     
     return domainsToBlock;
 }
 
-void ResourceLoadStatisticsMemoryStore::setResourceLoadStatisticsDebugMode(bool enable)
-{
-    ASSERT(!RunLoop::isMain());
-
-#if !RELEASE_LOG_DISABLED
-    if (enable)
-        RELEASE_LOG_INFO(ResourceLoadStatisticsDebug, "Turned ITP Debug Mode on.");
-#endif
-
-    m_debugModeEnabled = enable;
-    m_debugLoggingEnabled = enable;
-
-    ensurePrevalentResourcesForDebugMode();
-    // This will log the current cookie blocking state.
-    if (enable)
-        updateCookieBlocking([]() { });
-}
-
-void ResourceLoadStatisticsMemoryStore::setPrevalentResourceForDebugMode(const RegistrableDomain& domain)
-{
-    m_debugManualPrevalentResource = domain;
-}
-
-void ResourceLoadStatisticsMemoryStore::scheduleStatisticsProcessingRequestIfNecessary()
-{
-    ASSERT(!RunLoop::isMain());
-
-    m_pendingStatisticsProcessingRequestIdentifier = ++m_lastStatisticsProcessingRequestIdentifier;
-    m_workQueue->dispatchAfter(minimumStatisticsProcessingInterval, [this, weakThis = makeWeakPtr(*this), statisticsProcessingRequestIdentifier = *m_pendingStatisticsProcessingRequestIdentifier] {
-        if (!weakThis)
-            return;
-
-        if (!m_pendingStatisticsProcessingRequestIdentifier || *m_pendingStatisticsProcessingRequestIdentifier != statisticsProcessingRequestIdentifier) {
-            // This request has been canceled.
-            return;
-        }
-
-        updateCookieBlocking([]() { });
-        processStatisticsAndDataRecords();
-    });
-}
-
-void ResourceLoadStatisticsMemoryStore::cancelPendingStatisticsProcessingRequest()
-{
-    ASSERT(!RunLoop::isMain());
-
-    m_pendingStatisticsProcessingRequestIdentifier = WTF::nullopt;
-}
-
 void ResourceLoadStatisticsMemoryStore::logFrameNavigation(const RegistrableDomain& targetDomain, const RegistrableDomain& topFrameDomain, const RegistrableDomain& sourceDomain, bool isRedirect, bool isMainFrame)
 {
     ASSERT(!RunLoop::isMain());
@@ -635,7 +392,7 @@ void ResourceLoadStatisticsMemoryStore::logFrameNavigation(const RegistrableDoma
         scheduleStatisticsProcessingRequestIfNecessary();
 }
 
-void ResourceLoadStatisticsMemoryStore::logSubresourceLoading(const RegistrableDomain& targetDomain, const RegistrableDomain& topFrameDomain, WallTime lastSeen)
+void ResourceLoadStatisticsMemoryStore::logSubresourceLoading(const SubResourceDomain& targetDomain, const TopFrameDomain& topFrameDomain, WallTime lastSeen)
 {
     ASSERT(!RunLoop::isMain());
 
@@ -645,7 +402,7 @@ void ResourceLoadStatisticsMemoryStore::logSubresourceLoading(const RegistrableD
         scheduleStatisticsProcessingRequestIfNecessary();
 }
 
-void ResourceLoadStatisticsMemoryStore::logSubresourceRedirect(const RegistrableDomain& sourceDomain, const RegistrableDomain& targetDomain)
+void ResourceLoadStatisticsMemoryStore::logSubresourceRedirect(const RedirectedFromDomain& sourceDomain, const RedirectedToDomain& targetDomain)
 {
     ASSERT(!RunLoop::isMain());
 
@@ -658,7 +415,7 @@ void ResourceLoadStatisticsMemoryStore::logSubresourceRedirect(const Registrable
         scheduleStatisticsProcessingRequestIfNecessary();
 }
 
-void ResourceLoadStatisticsMemoryStore::logUserInteraction(const RegistrableDomain& domain)
+void ResourceLoadStatisticsMemoryStore::logUserInteraction(const TopFrameDomain& domain)
 {
     ASSERT(!RunLoop::isMain());
 
@@ -728,7 +485,7 @@ bool ResourceLoadStatisticsMemoryStore::isVeryPrevalentResource(const Registrabl
     return mapEntry == m_resourceStatisticsMap.end() ? false : mapEntry->value.isPrevalentResource && mapEntry->value.isVeryPrevalentResource;
 }
 
-bool ResourceLoadStatisticsMemoryStore::isRegisteredAsSubresourceUnder(const RegistrableDomain& subresourceDomain, const RegistrableDomain& topFrameDomain) const
+bool ResourceLoadStatisticsMemoryStore::isRegisteredAsSubresourceUnder(const SubResourceDomain& subresourceDomain, const TopFrameDomain& topFrameDomain) const
 {
     ASSERT(!RunLoop::isMain());
 
@@ -736,7 +493,7 @@ bool ResourceLoadStatisticsMemoryStore::isRegisteredAsSubresourceUnder(const Reg
     return mapEntry == m_resourceStatisticsMap.end() ? false : mapEntry->value.subresourceUnderTopFrameDomains.contains(topFrameDomain);
 }
 
-bool ResourceLoadStatisticsMemoryStore::isRegisteredAsSubFrameUnder(const RegistrableDomain& subFrameDomain, const RegistrableDomain& topFrameDomain) const
+bool ResourceLoadStatisticsMemoryStore::isRegisteredAsSubFrameUnder(const SubFrameDomain& subFrameDomain, const TopFrameDomain& topFrameDomain) const
 {
     ASSERT(!RunLoop::isMain());
 
@@ -744,7 +501,7 @@ bool ResourceLoadStatisticsMemoryStore::isRegisteredAsSubFrameUnder(const Regist
     return mapEntry == m_resourceStatisticsMap.end() ? false : mapEntry->value.subframeUnderTopFrameDomains.contains(topFrameDomain);
 }
 
-bool ResourceLoadStatisticsMemoryStore::isRegisteredAsRedirectingTo(const RegistrableDomain& redirectedFromDomain, const RegistrableDomain& redirectedToDomain) const
+bool ResourceLoadStatisticsMemoryStore::isRegisteredAsRedirectingTo(const RedirectedFromDomain& redirectedFromDomain, const RedirectedToDomain& redirectedToDomain) const
 {
     ASSERT(!RunLoop::isMain());
 
@@ -777,7 +534,7 @@ bool ResourceLoadStatisticsMemoryStore::isGrandfathered(const RegistrableDomain&
     return mapEntry == m_resourceStatisticsMap.end() ? false : mapEntry->value.grandfathered;
 }
 
-void ResourceLoadStatisticsMemoryStore::setSubframeUnderTopFrameDomain(const RegistrableDomain& subFrameDomain, const RegistrableDomain& topFrameDomain)
+void ResourceLoadStatisticsMemoryStore::setSubframeUnderTopFrameDomain(const SubFrameDomain& subFrameDomain, const TopFrameDomain& topFrameDomain)
 {
     ASSERT(!RunLoop::isMain());
 
@@ -787,7 +544,7 @@ void ResourceLoadStatisticsMemoryStore::setSubframeUnderTopFrameDomain(const Reg
     ensureResourceStatisticsForRegistrableDomain(topFrameDomain);
 }
 
-void ResourceLoadStatisticsMemoryStore::setSubresourceUnderTopFrameDomain(const RegistrableDomain& subresourceDomain, const RegistrableDomain& topFrameDomain)
+void ResourceLoadStatisticsMemoryStore::setSubresourceUnderTopFrameDomain(const SubResourceDomain& subresourceDomain, const RegistrableDomain& topFrameDomain)
 {
     ASSERT(!RunLoop::isMain());
 
@@ -797,7 +554,7 @@ void ResourceLoadStatisticsMemoryStore::setSubresourceUnderTopFrameDomain(const
     ensureResourceStatisticsForRegistrableDomain(topFrameDomain);
 }
 
-void ResourceLoadStatisticsMemoryStore::setSubresourceUniqueRedirectTo(const RegistrableDomain& subresourceDomain, const RegistrableDomain& redirectDomain)
+void ResourceLoadStatisticsMemoryStore::setSubresourceUniqueRedirectTo(const SubResourceDomain& subresourceDomain, const RedirectDomain& redirectDomain)
 {
     ASSERT(!RunLoop::isMain());
 
@@ -807,7 +564,7 @@ void ResourceLoadStatisticsMemoryStore::setSubresourceUniqueRedirectTo(const Reg
     ensureResourceStatisticsForRegistrableDomain(redirectDomain);
 }
 
-void ResourceLoadStatisticsMemoryStore::setSubresourceUniqueRedirectFrom(const RegistrableDomain& subresourceDomain, const RegistrableDomain& redirectDomain)
+void ResourceLoadStatisticsMemoryStore::setSubresourceUniqueRedirectFrom(const SubResourceDomain& subresourceDomain, const RedirectDomain& redirectDomain)
 {
     ASSERT(!RunLoop::isMain());
 
@@ -817,7 +574,7 @@ void ResourceLoadStatisticsMemoryStore::setSubresourceUniqueRedirectFrom(const R
     ensureResourceStatisticsForRegistrableDomain(redirectDomain);
 }
 
-void ResourceLoadStatisticsMemoryStore::setTopFrameUniqueRedirectTo(const RegistrableDomain& topFrameDomain, const RegistrableDomain& redirectDomain)
+void ResourceLoadStatisticsMemoryStore::setTopFrameUniqueRedirectTo(const TopFrameDomain& topFrameDomain, const RedirectDomain& redirectDomain)
 {
     ASSERT(!RunLoop::isMain());
 
@@ -827,7 +584,7 @@ void ResourceLoadStatisticsMemoryStore::setTopFrameUniqueRedirectTo(const Regist
     ensureResourceStatisticsForRegistrableDomain(redirectDomain);
 }
 
-void ResourceLoadStatisticsMemoryStore::setTopFrameUniqueRedirectFrom(const RegistrableDomain& topFrameDomain, const RegistrableDomain& redirectDomain)
+void ResourceLoadStatisticsMemoryStore::setTopFrameUniqueRedirectFrom(const TopFrameDomain& topFrameDomain, const RedirectDomain& redirectDomain)
 {
     ASSERT(!RunLoop::isMain());
 
@@ -837,95 +594,6 @@ void ResourceLoadStatisticsMemoryStore::setTopFrameUniqueRedirectFrom(const Regi
     ensureResourceStatisticsForRegistrableDomain(redirectDomain);
 }
 
-void ResourceLoadStatisticsMemoryStore::setTimeToLiveUserInteraction(Seconds seconds)
-{
-    ASSERT(!RunLoop::isMain());
-    ASSERT(seconds >= 0_s);
-
-    m_parameters.timeToLiveUserInteraction = seconds;
-}
-
-void ResourceLoadStatisticsMemoryStore::setMinimumTimeBetweenDataRecordsRemoval(Seconds seconds)
-{
-    ASSERT(!RunLoop::isMain());
-    ASSERT(seconds >= 0_s);
-
-    m_parameters.minimumTimeBetweenDataRecordsRemoval = seconds;
-}
-
-void ResourceLoadStatisticsMemoryStore::setGrandfatheringTime(Seconds seconds)
-{
-    ASSERT(!RunLoop::isMain());
-    ASSERT(seconds >= 0_s);
-
-    m_parameters.grandfatheringTime = seconds;
-}
-
-void ResourceLoadStatisticsMemoryStore::setCacheMaxAgeCap(Seconds seconds)
-{
-    ASSERT(!RunLoop::isMain());
-    ASSERT(seconds >= 0_s);
-
-    m_parameters.cacheMaxAgeCapTime = seconds;
-    updateCacheMaxAgeCap();
-}
-
-void ResourceLoadStatisticsMemoryStore::updateCacheMaxAgeCap()
-{
-    ASSERT(!RunLoop::isMain());
-    
-    RunLoop::main().dispatch([store = makeRef(m_store), seconds = m_parameters.cacheMaxAgeCapTime] () {
-        store->setCacheMaxAgeCap(seconds, [] { });
-    });
-}
-
-void ResourceLoadStatisticsMemoryStore::setAgeCapForClientSideCookies(Seconds seconds)
-{
-    ASSERT(!RunLoop::isMain());
-    ASSERT(seconds >= 0_s);
-    
-    m_parameters.clientSideCookiesAgeCapTime = seconds;
-    updateClientSideCookiesAgeCap();
-}
-
-void ResourceLoadStatisticsMemoryStore::updateClientSideCookiesAgeCap()
-{
-    ASSERT(!RunLoop::isMain());
-
-#if ENABLE(RESOURCE_LOAD_STATISTICS)
-    RunLoop::main().dispatch([store = makeRef(m_store), seconds = m_parameters.clientSideCookiesAgeCapTime] () {
-        if (auto* networkSession = store->networkSession())
-            networkSession->networkStorageSession().setAgeCapForClientSideCookies(seconds);
-    });
-#endif
-}
-
-bool ResourceLoadStatisticsMemoryStore::shouldRemoveDataRecords() const
-{
-    ASSERT(!RunLoop::isMain());
-
-    if (m_dataRecordsBeingRemoved)
-        return false;
-
-#if ENABLE(NETSCAPE_PLUGIN_API)
-    for (const auto& plugin : PluginProcessManager::singleton().pluginProcesses()) {
-        if (!m_activePluginTokens.contains(plugin->pluginProcessToken()))
-            return true;
-    }
-#endif
-
-    return !m_lastTimeDataRecordsWereRemoved || MonotonicTime::now() >= (m_lastTimeDataRecordsWereRemoved + m_parameters.minimumTimeBetweenDataRecordsRemoval);
-}
-
-void ResourceLoadStatisticsMemoryStore::setDataRecordsBeingRemoved(bool value)
-{
-    ASSERT(!RunLoop::isMain());
-
-    m_dataRecordsBeingRemoved = value;
-    if (m_dataRecordsBeingRemoved)
-        m_lastTimeDataRecordsWereRemoved = MonotonicTime::now();
-}
-
 ResourceLoadStatistics& ResourceLoadStatisticsMemoryStore::ensureResourceStatisticsForRegistrableDomain(const RegistrableDomain& domain)
 {
     ASSERT(!RunLoop::isMain());
@@ -941,13 +609,14 @@ std::unique_ptr<KeyedEncoder> ResourceLoadStatisticsMemoryStore::createEncoderFr
 
     auto encoder = KeyedEncoder::encoder();
     encoder->encodeUInt32("version", statisticsModelVersion);
-    encoder->encodeDouble("endOfGrandfatheringTimestamp", m_endOfGrandfatheringTimestamp.secondsSinceEpoch().value());
+    encoder->encodeDouble("endOfGrandfatheringTimestamp", endOfGrandfatheringTimestamp().secondsSinceEpoch().value());
 
     encoder->encodeObjects("browsingStatistics", m_resourceStatisticsMap.begin(), m_resourceStatisticsMap.end(), [](KeyedEncoder& encoderInner, const auto& domain) {
         domain.value.encode(encoderInner);
     });
 
-    encoder->encodeObjects("operatingDates", m_operatingDates.begin(), m_operatingDates.end(), [](KeyedEncoder& encoderInner, OperatingDate date) {
+    auto& operatingDates = this->operatingDates();
+    encoder->encodeObjects("operatingDates", operatingDates.begin(), operatingDates.end(), [](KeyedEncoder& encoderInner, OperatingDate date) {
         encoderInner.encodeDouble("date", date.secondsSinceEpoch().value());
     });
 
@@ -969,9 +638,9 @@ void ResourceLoadStatisticsMemoryStore::mergeWithDataFromDecoder(KeyedDecoder& d
 
     double endOfGrandfatheringTimestamp;
     if (decoder.decodeDouble("endOfGrandfatheringTimestamp", endOfGrandfatheringTimestamp))
-        m_endOfGrandfatheringTimestamp = WallTime::fromRawSeconds(endOfGrandfatheringTimestamp);
+        setEndOfGrandfatheringTimestamp(WallTime::fromRawSeconds(endOfGrandfatheringTimestamp));
     else
-        m_endOfGrandfatheringTimestamp = { };
+        clearEndOfGrandfatheringTimeStamp();
 
     Vector<ResourceLoadStatistics> loadedStatistics;
     bool succeeded = decoder.decodeObjects("browsingStatistics", loadedStatistics, [versionOnDisk](KeyedDecoder& decoderInner, ResourceLoadStatistics& statistics) {
@@ -997,7 +666,7 @@ void ResourceLoadStatisticsMemoryStore::mergeWithDataFromDecoder(KeyedDecoder& d
     if (!succeeded)
         return;
 
-    m_operatingDates = mergeOperatingDates(m_operatingDates, WTFMove(operatingDates));
+    mergeOperatingDates(WTFMove(operatingDates));
 }
 
 void ResourceLoadStatisticsMemoryStore::clear(CompletionHandler<void()>&& completionHandler)
@@ -1005,7 +674,7 @@ void ResourceLoadStatisticsMemoryStore::clear(CompletionHandler<void()>&& comple
     ASSERT(!RunLoop::isMain());
 
     m_resourceStatisticsMap.clear();
-    m_operatingDates.clear();
+    clearOperatingDates();
 
     auto callbackAggregator = CallbackAggregator::create(WTFMove(completionHandler));
 
@@ -1056,39 +725,6 @@ bool ResourceLoadStatisticsMemoryStore::hasUserGrantedStorageAccessThroughPrompt
     return statistic.storageAccessUnderTopFrameDomains.contains(firstPartyDomain);
 }
 
-static void debugLogDomainsInBatches(const char* action, const Vector<RegistrableDomain>& domains)
-{
-#if !RELEASE_LOG_DISABLED
-    static const auto maxNumberOfDomainsInOneLogStatement = 50;
-    if (domains.isEmpty())
-        return;
-
-    if (domains.size() <= maxNumberOfDomainsInOneLogStatement) {
-        RELEASE_LOG_INFO(ResourceLoadStatisticsDebug, "About to %{public}s cookies in third-party contexts for: %{public}s.", action, domainsToString(domains).utf8().data());
-        return;
-    }
-    
-    Vector<RegistrableDomain> batch;
-    batch.reserveInitialCapacity(maxNumberOfDomainsInOneLogStatement);
-    auto batchNumber = 1;
-    unsigned numberOfBatches = std::ceil(domains.size() / static_cast<float>(maxNumberOfDomainsInOneLogStatement));
-
-    for (auto& domain : domains) {
-        if (batch.size() == maxNumberOfDomainsInOneLogStatement) {
-            RELEASE_LOG_INFO(ResourceLoadStatisticsDebug, "About to %{public}s cookies in third-party contexts for (%{public}d of %u): %{public}s.", action, batchNumber, numberOfBatches, domainsToString(batch).utf8().data());
-            batch.shrink(0);
-            ++batchNumber;
-        }
-        batch.append(domain);
-    }
-    if (!batch.isEmpty())
-        RELEASE_LOG_INFO(ResourceLoadStatisticsDebug, "About to %{public}s cookies in third-party contexts for (%{public}d of %u): %{public}s.", action, batchNumber, numberOfBatches, domainsToString(batch).utf8().data());
-#else
-    UNUSED_PARAM(action);
-    UNUSED_PARAM(domains);
-#endif
-}
-
 void ResourceLoadStatisticsMemoryStore::updateCookieBlocking(CompletionHandler<void()>&& completionHandler)
 {
     ASSERT(!RunLoop::isMain());
@@ -1104,52 +740,23 @@ void ResourceLoadStatisticsMemoryStore::updateCookieBlocking(CompletionHandler<v
         return;
     }
 
-    if (m_debugLoggingEnabled && !domainsToBlock.isEmpty())
+    if (debugLoggingEnabled() && !domainsToBlock.isEmpty())
         debugLogDomainsInBatches("block", domainsToBlock);
 
-    RunLoop::main().dispatch([weakThis = makeWeakPtr(*this), store = makeRef(m_store), domainsToBlock = crossThreadCopy(domainsToBlock), completionHandler = WTFMove(completionHandler)] () mutable {
+    RunLoop::main().dispatch([weakThis = makeWeakPtr(*this), store = makeRef(store()), domainsToBlock = crossThreadCopy(domainsToBlock), completionHandler = WTFMove(completionHandler)] () mutable {
         store->callUpdatePrevalentDomainsToBlockCookiesForHandler(domainsToBlock, [weakThis = WTFMove(weakThis), store = store.copyRef(), completionHandler = WTFMove(completionHandler)]() mutable {
             store->statisticsQueue().dispatch([weakThis = WTFMove(weakThis), completionHandler = WTFMove(completionHandler)]() mutable {
                 completionHandler();
                 if (!weakThis)
                     return;
 #if !RELEASE_LOG_DISABLED
-                RELEASE_LOG_INFO_IF(weakThis->m_debugLoggingEnabled, ResourceLoadStatisticsDebug, "Done updating cookie blocking.");
+                RELEASE_LOG_INFO_IF(weakThis->debugLoggingEnabled(), ResourceLoadStatisticsDebug, "Done updating cookie blocking.");
 #endif
             });
         });
     });
 }
 
-void ResourceLoadStatisticsMemoryStore::updateCookieBlockingForDomains(const Vector<RegistrableDomain>& domainsToBlock, CompletionHandler<void()>&& completionHandler)
-{
-    ASSERT(!RunLoop::isMain());
-
-    RunLoop::main().dispatch([store = makeRef(m_store), domainsToBlock = crossThreadCopy(domainsToBlock), completionHandler = WTFMove(completionHandler)] () mutable {
-        store->callUpdatePrevalentDomainsToBlockCookiesForHandler(domainsToBlock, [store = store.copyRef(), completionHandler = WTFMove(completionHandler)]() mutable {
-            store->statisticsQueue().dispatch([completionHandler = WTFMove(completionHandler)]() mutable {
-                completionHandler();
-            });
-        });
-    });
-}
-
-void ResourceLoadStatisticsMemoryStore::clearBlockingStateForDomains(const Vector<RegistrableDomain>& domains, CompletionHandler<void()>&& completionHandler)
-{
-    ASSERT(!RunLoop::isMain());
-
-    if (domains.isEmpty()) {
-        completionHandler();
-        return;
-    }
-
-    RunLoop::main().dispatch([store = makeRef(m_store), domains = crossThreadCopy(domains)] {
-        store->callRemoveDomainsHandler(domains);
-    });
-
-    completionHandler();
-}
-
 void ResourceLoadStatisticsMemoryStore::processStatistics(const Function<void(const ResourceLoadStatistics&)>& processFunction) const
 {
     ASSERT(!RunLoop::isMain());
@@ -1178,11 +785,11 @@ Vector<RegistrableDomain> ResourceLoadStatisticsMemoryStore::registrableDomainsT
 {
     ASSERT(!RunLoop::isMain());
 
-    bool shouldCheckForGrandfathering = m_endOfGrandfatheringTimestamp > WallTime::now();
-    bool shouldClearGrandfathering = !shouldCheckForGrandfathering && m_endOfGrandfatheringTimestamp;
+    bool shouldCheckForGrandfathering = endOfGrandfatheringTimestamp() > WallTime::now();
+    bool shouldClearGrandfathering = !shouldCheckForGrandfathering && endOfGrandfatheringTimestamp();
 
     if (shouldClearGrandfathering)
-        m_endOfGrandfatheringTimestamp = { };
+        clearEndOfGrandfatheringTimeStamp();
 
     Vector<RegistrableDomain> prevalentResources;
     for (auto& statistic : m_resourceStatisticsMap.values()) {
@@ -1196,62 +803,16 @@ Vector<RegistrableDomain> ResourceLoadStatisticsMemoryStore::registrableDomainsT
     return prevalentResources;
 }
 
-void ResourceLoadStatisticsMemoryStore::includeTodayAsOperatingDateIfNecessary()
-{
-    ASSERT(!RunLoop::isMain());
-
-    auto today = OperatingDate::today();
-    if (!m_operatingDates.isEmpty() && today <= m_operatingDates.last())
-        return;
-
-    while (m_operatingDates.size() >= operatingDatesWindow)
-        m_operatingDates.remove(0);
-
-    m_operatingDates.append(today);
-}
-
-bool ResourceLoadStatisticsMemoryStore::hasStatisticsExpired(const ResourceLoadStatistics& resourceStatistic) const
-{
-    ASSERT(!RunLoop::isMain());
-
-    if (m_operatingDates.size() >= operatingDatesWindow) {
-        if (OperatingDate::fromWallTime(resourceStatistic.mostRecentUserInteractionTime) < m_operatingDates.first())
-            return true;
-    }
-
-    // If we don't meet the real criteria for an expired statistic, check the user setting for a tighter restriction (mainly for testing).
-    if (m_parameters.timeToLiveUserInteraction) {
-        if (WallTime::now() > resourceStatistic.mostRecentUserInteractionTime + m_parameters.timeToLiveUserInteraction.value())
-            return true;
-    }
-
-    return false;
-}
-
-void ResourceLoadStatisticsMemoryStore::setMaxStatisticsEntries(size_t maximumEntryCount)
-{
-    ASSERT(!RunLoop::isMain());
-
-    m_parameters.maxStatisticsEntries = maximumEntryCount;
-}
-
-void ResourceLoadStatisticsMemoryStore::setPruneEntriesDownTo(size_t pruneTargetCount)
-{
-    ASSERT(!RunLoop::isMain());
-
-    m_parameters.pruneEntriesDownTo = pruneTargetCount;
-}
-
 void ResourceLoadStatisticsMemoryStore::pruneStatisticsIfNeeded()
 {
     ASSERT(!RunLoop::isMain());
 
-    if (m_resourceStatisticsMap.size() <= m_parameters.maxStatisticsEntries)
+    if (m_resourceStatisticsMap.size() <= parameters().maxStatisticsEntries)
         return;
 
-    ASSERT(m_parameters.pruneEntriesDownTo <= m_parameters.maxStatisticsEntries);
+    ASSERT(parameters().pruneEntriesDownTo <= parameters().maxStatisticsEntries);
 
-    size_t numberOfEntriesLeftToPrune = m_resourceStatisticsMap.size() - m_parameters.pruneEntriesDownTo;
+    size_t numberOfEntriesLeftToPrune = m_resourceStatisticsMap.size() - parameters().pruneEntriesDownTo;
     ASSERT(numberOfEntriesLeftToPrune);
 
     Vector<StatisticsLastSeen> resourcesToPrunePerImportance[maxImportance + 1];
@@ -1264,22 +825,6 @@ void ResourceLoadStatisticsMemoryStore::pruneStatisticsIfNeeded()
     ASSERT(!numberOfEntriesLeftToPrune);
 }
 
-void ResourceLoadStatisticsMemoryStore::resetParametersToDefaultValues()
-{
-    ASSERT(!RunLoop::isMain());
-
-    m_parameters = { };
-}
-
-void ResourceLoadStatisticsMemoryStore::logTestingEvent(const String& event)
-{
-    ASSERT(!RunLoop::isMain());
-
-    RunLoop::main().dispatch([store = makeRef(m_store), event = event.isolatedCopy()] {
-        store->logTestingEvent(event);
-    });
-}
-
 void ResourceLoadStatisticsMemoryStore::setLastSeen(const RegistrableDomain& domain, Seconds seconds)
 {
     ASSERT(!RunLoop::isMain());
@@ -1304,27 +849,6 @@ void ResourceLoadStatisticsMemoryStore::setVeryPrevalentResource(const Registrab
     setPrevalentResource(resourceStatistic, ResourceLoadPrevalence::VeryHigh);
 }
 
-void ResourceLoadStatisticsMemoryStore::removeAllStorageAccess(CompletionHandler<void()>&& completionHandler)
-{
-    ASSERT(!RunLoop::isMain());
-    RunLoop::main().dispatch([store = makeRef(m_store), completionHandler = WTFMove(completionHandler)]() mutable {
-        store->removeAllStorageAccess([store = store.copyRef(), completionHandler = WTFMove(completionHandler)]() mutable {
-            store->statisticsQueue().dispatch([completionHandler = WTFMove(completionHandler)]() mutable {
-                completionHandler();
-            });
-        });
-    });
-}
-
-void ResourceLoadStatisticsMemoryStore::didCreateNetworkProcess()
-{
-    ASSERT(!RunLoop::isMain());
-
-    updateCookieBlocking([]() { });
-    updateCacheMaxAgeCap();
-    updateClientSideCookiesAgeCap();
-}
-
 } // namespace WebKit
 
 #endif
index e31177a..ca1d92a 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2017-2018 Apple Inc. All rights reserved.
+ * Copyright (C) 2017-2019 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
 
 #if ENABLE(RESOURCE_LOAD_STATISTICS)
 
-#include "ResourceLoadStatisticsClassifier.h"
+#include "ResourceLoadStatisticsStore.h"
 #include "WebResourceLoadStatisticsStore.h"
 #include <wtf/CompletionHandler.h>
 #include <wtf/Vector.h>
 #include <wtf/WeakPtr.h>
 #include <wtf/WorkQueue.h>
 
-#if HAVE(CORE_PREDICTION)
-#include "ResourceLoadStatisticsClassifierCocoa.h"
-#endif
-
 namespace WebCore {
 class KeyedDecoder;
 class KeyedEncoder;
@@ -46,35 +42,19 @@ struct ResourceLoadStatistics;
 
 namespace WebKit {
 
-class OperatingDate;
 class ResourceLoadStatisticsPersistentStorage;
 
 // This is always constructed / used / destroyed on the WebResourceLoadStatisticsStore's statistics queue.
-class ResourceLoadStatisticsMemoryStore : public CanMakeWeakPtr<ResourceLoadStatisticsMemoryStore> {
+class ResourceLoadStatisticsMemoryStore final : public ResourceLoadStatisticsStore {
 public:
-    using ResourceLoadStatistics = WebCore::ResourceLoadStatistics;
-    using RegistrableDomain = WebCore::RegistrableDomain;
-    using TopFrameDomain = WebCore::RegistrableDomain;
-    using SubFrameDomain = WebCore::RegistrableDomain;
-    using SubResourceDomain = WebCore::RegistrableDomain;
-    using RedirectDomain = WebCore::RegistrableDomain;
-    using RedirectedFromDomain = WebCore::RegistrableDomain;
-    using RedirectedToDomain = WebCore::RegistrableDomain;
-    using NavigatedFromDomain = WebCore::RegistrableDomain;
-    using NavigatedToDomain = WebCore::RegistrableDomain;
-    using DomainInNeedOfStorageAccess = WebCore::RegistrableDomain;
-    using OpenerDomain = WebCore::RegistrableDomain;
-    using OpenerPageID = uint64_t;
-    using PageID = uint64_t;
-    using FrameID = uint64_t;
-
     ResourceLoadStatisticsMemoryStore(WebResourceLoadStatisticsStore&, WorkQueue&);
-    ~ResourceLoadStatisticsMemoryStore();
 
     void setPersistentStorage(ResourceLoadStatisticsPersistentStorage&);
 
-    void clear(CompletionHandler<void()>&&);
-    bool isEmpty() const { return m_resourceStatisticsMap.isEmpty(); }
+    void clear(CompletionHandler<void()>&&) override;
+    bool isEmpty() const override;
+
+    const HashMap<RegistrableDomain, WebCore::ResourceLoadStatistics>& data() const { return m_resourceStatisticsMap; }
 
     std::unique_ptr<WebCore::KeyedEncoder> createEncoderFromData() const;
     void mergeWithDataFromDecoder(WebCore::KeyedDecoder&);
@@ -82,147 +62,81 @@ public:
     void mergeStatistics(Vector<ResourceLoadStatistics>&&);
     void processStatistics(const Function<void(const ResourceLoadStatistics&)>&) const;
 
-    void updateCookieBlocking(CompletionHandler<void()>&&);
-    void updateCookieBlockingForDomains(const Vector<RegistrableDomain>&, CompletionHandler<void()>&&);
-    void clearBlockingStateForDomains(const Vector<RegistrableDomain>&, CompletionHandler<void()>&&);
-
-    void includeTodayAsOperatingDateIfNecessary();
-    void processStatisticsAndDataRecords();
-
-    void requestStorageAccessUnderOpener(DomainInNeedOfStorageAccess&&, OpenerPageID, OpenerDomain&&);
-    void removeAllStorageAccess(CompletionHandler<void()>&&);
-
-    void grandfatherExistingWebsiteData(CompletionHandler<void()>&&);
-    void cancelPendingStatisticsProcessingRequest();
+    void updateCookieBlocking(CompletionHandler<void()>&&) override;
 
-    bool isRegisteredAsSubresourceUnder(const SubResourceDomain&, const TopFrameDomain&) const;
-    bool isRegisteredAsSubFrameUnder(const SubFrameDomain&, const TopFrameDomain&) const;
-    bool isRegisteredAsRedirectingTo(const RedirectedFromDomain&, const RedirectedToDomain&) const;
+    void classifyPrevalentResources() override;
+    void syncStorageIfNeeded() override;
+    void syncStorageImmediately() override;
 
-    void clearPrevalentResource(const RegistrableDomain&);
-    String dumpResourceLoadStatistics() const;
-    bool isPrevalentResource(const RegistrableDomain&) const;
-    bool isVeryPrevalentResource(const RegistrableDomain&) const;
-    void setPrevalentResource(const RegistrableDomain&);
-    void setVeryPrevalentResource(const RegistrableDomain&);
+    void requestStorageAccessUnderOpener(DomainInNeedOfStorageAccess&&, OpenerPageID, OpenerDomain&&) override;
 
-    void setGrandfathered(const RegistrableDomain&, bool value);
-    bool isGrandfathered(const RegistrableDomain&) const;
+    void grandfatherDataForDomains(const HashSet<RegistrableDomain>&) override;
 
-    void setSubframeUnderTopFrameDomain(const SubFrameDomain&, const TopFrameDomain&);
-    void setSubresourceUnderTopFrameDomain(const SubResourceDomain&, const TopFrameDomain&);
-    void setSubresourceUniqueRedirectTo(const SubResourceDomain&, const RedirectDomain&);
-    void setSubresourceUniqueRedirectFrom(const SubResourceDomain&, const RedirectDomain&);
-    void setTopFrameUniqueRedirectTo(const TopFrameDomain&, const RedirectDomain&);
-    void setTopFrameUniqueRedirectFrom(const TopFrameDomain&, const RedirectDomain&);
+    bool isRegisteredAsSubresourceUnder(const SubResourceDomain&, const TopFrameDomain&) const override;
+    bool isRegisteredAsSubFrameUnder(const SubFrameDomain&, const TopFrameDomain&) const override;
+    bool isRegisteredAsRedirectingTo(const RedirectedFromDomain&, const RedirectedToDomain&) const override;
 
-    void logTestingEvent(const String&);
+    void clearPrevalentResource(const RegistrableDomain&) override;
+    String dumpResourceLoadStatistics() const override;
+    bool isPrevalentResource(const RegistrableDomain&) const override;
+    bool isVeryPrevalentResource(const RegistrableDomain&) const override;
+    void setPrevalentResource(const RegistrableDomain&) override;
+    void setVeryPrevalentResource(const RegistrableDomain&) override;
 
-    void setMaxStatisticsEntries(size_t maximumEntryCount);
-    void setPruneEntriesDownTo(size_t pruneTargetCount);
-    void resetParametersToDefaultValues();
+    void setGrandfathered(const RegistrableDomain&, bool value) override;
+    bool isGrandfathered(const RegistrableDomain&) const override;
 
-    void calculateAndSubmitTelemetry() const;
+    void setSubframeUnderTopFrameDomain(const SubFrameDomain&, const TopFrameDomain&) override;
+    void setSubresourceUnderTopFrameDomain(const SubResourceDomain&, const TopFrameDomain&) override;
+    void setSubresourceUniqueRedirectTo(const SubResourceDomain&, const RedirectDomain&) override;
+    void setSubresourceUniqueRedirectFrom(const SubResourceDomain&, const RedirectDomain&) override;
+    void setTopFrameUniqueRedirectTo(const TopFrameDomain&, const RedirectDomain&) override;
+    void setTopFrameUniqueRedirectFrom(const TopFrameDomain&, const RedirectDomain&) override;
 
-    void setNotifyPagesWhenDataRecordsWereScanned(bool);
-    void setShouldClassifyResourcesBeforeDataRecordsRemoval(bool);
-    void setShouldSubmitTelemetry(bool);
-    void setTimeToLiveUserInteraction(Seconds);
-    void setMinimumTimeBetweenDataRecordsRemoval(Seconds);
-    void setGrandfatheringTime(Seconds);
-    void setResourceLoadStatisticsDebugMode(bool);
-    bool isDebugModeEnabled() const { return m_debugModeEnabled; };
-    void setPrevalentResourceForDebugMode(const RegistrableDomain&);
+    void calculateAndSubmitTelemetry() const override;
 
-    void hasStorageAccess(const SubFrameDomain&, const TopFrameDomain&, Optional<FrameID>, PageID, CompletionHandler<void(bool)>&&);
-    void requestStorageAccess(SubFrameDomain&&, TopFrameDomain&&, FrameID, PageID, bool promptEnabled, CompletionHandler<void(StorageAccessStatus)>&&);
-    void grantStorageAccess(SubFrameDomain&&, TopFrameDomain&&, FrameID, PageID, bool userWasPromptedNow, CompletionHandler<void(bool)>&&);
+    void hasStorageAccess(const SubFrameDomain&, const TopFrameDomain&, Optional<FrameID>, PageID, CompletionHandler<void(bool)>&&) override;
+    void requestStorageAccess(SubFrameDomain&&, TopFrameDomain&&, FrameID, PageID, bool promptEnabled, CompletionHandler<void(StorageAccessStatus)>&&) override;
+    void grantStorageAccess(SubFrameDomain&&, TopFrameDomain&&, FrameID, PageID, bool userWasPromptedNow, CompletionHandler<void(bool)>&&) override;
 
-    void logFrameNavigation(const NavigatedToDomain&, const TopFrameDomain&, const NavigatedFromDomain&, bool isRedirect, bool isMainFrame);
-    void logUserInteraction(const TopFrameDomain&);
-    void logSubresourceLoading(const SubResourceDomain&, const TopFrameDomain&, WallTime lastSeen);
-    void logSubresourceRedirect(const RedirectedFromDomain&, const RedirectedToDomain&);
+    void logFrameNavigation(const NavigatedToDomain&, const TopFrameDomain&, const NavigatedFromDomain&, bool isRedirect, bool isMainFrame) override;
+    void logUserInteraction(const TopFrameDomain&) override;
+    void logSubresourceLoading(const SubResourceDomain&, const TopFrameDomain&, WallTime lastSeen) override;
+    void logSubresourceRedirect(const RedirectedFromDomain&, const RedirectedToDomain&) override;
 
-    void clearUserInteraction(const RegistrableDomain&);
-    bool hasHadUserInteraction(const RegistrableDomain&);
+    void clearUserInteraction(const RegistrableDomain&) override;
+    bool hasHadUserInteraction(const RegistrableDomain&) override;
 
-    void setLastSeen(const RegistrableDomain&, Seconds);
-
-    void didCreateNetworkProcess();
-
-    const WebResourceLoadStatisticsStore& store() const { return m_store; }
+    void setLastSeen(const RegistrableDomain&, Seconds) override;
 
 private:
     static bool shouldBlockAndKeepCookies(const ResourceLoadStatistics&);
     static bool shouldBlockAndPurgeCookies(const ResourceLoadStatistics&);
     static bool hasUserGrantedStorageAccessThroughPrompt(const ResourceLoadStatistics&, const RegistrableDomain&);
     bool hasHadUnexpiredRecentUserInteraction(ResourceLoadStatistics&) const;
-    bool hasStatisticsExpired(const ResourceLoadStatistics&) const;
     bool wasAccessedAsFirstPartyDueToUserInteraction(const ResourceLoadStatistics& current, const ResourceLoadStatistics& updated) const;
+    void incrementRecordsDeletedCountForDomains(HashSet<RegistrableDomain>&&) override;
     void setPrevalentResource(ResourceLoadStatistics&, ResourceLoadPrevalence);
     unsigned recursivelyGetAllDomainsThatHaveRedirectedToThisDomain(const ResourceLoadStatistics&, HashSet<RedirectedToDomain>&, unsigned numberOfRecursiveCalls) const;
-    void setStorageAccessPromptsEnabled(bool enabled) { m_storageAccessPromptsEnabled  = enabled; }
-    bool shouldRemoveDataRecords() const;
-    void setDebugLogggingEnabled(bool enabled) { m_debugLoggingEnabled  = enabled; }
-    void setDataRecordsBeingRemoved(bool);
-    void scheduleStatisticsProcessingRequestIfNecessary();
     void grantStorageAccessInternal(SubFrameDomain&&, TopFrameDomain&&, Optional<FrameID>, PageID, bool userWasPromptedNowOrEarlier, CompletionHandler<void(bool)>&&);
     void markAsPrevalentIfHasRedirectedToPrevalent(ResourceLoadStatistics&);
     bool isPrevalentDueToDebugMode(ResourceLoadStatistics&);
-    Vector<RegistrableDomain> ensurePrevalentResourcesForDebugMode();
+    Vector<RegistrableDomain> ensurePrevalentResourcesForDebugMode() override;
     void removeDataRecords(CompletionHandler<void()>&&);
-    void pruneStatisticsIfNeeded();
+    void pruneStatisticsIfNeeded() override;
     ResourceLoadStatistics& ensureResourceStatisticsForRegistrableDomain(const RegistrableDomain&);
-    Vector<RegistrableDomain> registrableDomainsToRemoveWebsiteDataFor();
-    void setCacheMaxAgeCap(Seconds);
-    void updateCacheMaxAgeCap();
-    void setAgeCapForClientSideCookies(Seconds);
-    void updateClientSideCookiesAgeCap();
-
-#if PLATFORM(COCOA)
-    void registerUserDefaultsIfNeeded();
-#endif
+    Vector<RegistrableDomain> registrableDomainsToRemoveWebsiteDataFor() override;
+    bool isMemoryStore() const final { return true; }
+
 
-    struct Parameters {
-        size_t pruneEntriesDownTo { 800 };
-        size_t maxStatisticsEntries { 1000 };
-        Optional<Seconds> timeToLiveUserInteraction;
-        Seconds minimumTimeBetweenDataRecordsRemoval { 1_h };
-        Seconds grandfatheringTime { 24_h * 7 };
-        Seconds cacheMaxAgeCapTime { 24_h * 7 };
-        Seconds clientSideCookiesAgeCapTime { 24_h * 7 };
-        bool shouldNotifyPagesWhenDataRecordsWereScanned { false };
-        bool shouldClassifyResourcesBeforeDataRecordsRemoval { true };
-        bool shouldSubmitTelemetry { true };
-    };
-
-    WebResourceLoadStatisticsStore& m_store;
-    Ref<WorkQueue> m_workQueue;
     WeakPtr<ResourceLoadStatisticsPersistentStorage> m_persistentStorage;
     HashMap<RegistrableDomain, ResourceLoadStatistics> m_resourceStatisticsMap;
-#if HAVE(CORE_PREDICTION)
-    ResourceLoadStatisticsClassifierCocoa m_resourceLoadStatisticsClassifier;
-#else
-    ResourceLoadStatisticsClassifier m_resourceLoadStatisticsClassifier;
-#endif
-#if ENABLE(NETSCAPE_PLUGIN_API)
-    HashSet<uint64_t> m_activePluginTokens;
-#endif
-    Parameters m_parameters;
-    Vector<OperatingDate> m_operatingDates;
-    WallTime m_endOfGrandfatheringTimestamp;
-    bool m_debugLoggingEnabled { false };
-    bool m_debugModeEnabled { false };
-    const RegistrableDomain m_debugStaticPrevalentResource { URL { URL(), "https://3rdpartytestwebkit.org"_s } };
-    RegistrableDomain m_debugManualPrevalentResource;
-    bool m_storageAccessPromptsEnabled { false };
-    bool m_dataRecordsBeingRemoved { false };
-    MonotonicTime m_lastTimeDataRecordsWereRemoved;
-
-    uint64_t m_lastStatisticsProcessingRequestIdentifier { 0 };
-    Optional<uint64_t> m_pendingStatisticsProcessingRequestIdentifier;
 };
 
 } // namespace WebKit
 
+SPECIALIZE_TYPE_TRAITS_BEGIN(WebKit::ResourceLoadStatisticsMemoryStore)
+    static bool isType(const WebKit::ResourceLoadStatisticsStore& store) { return store.isMemoryStore(); }
+SPECIALIZE_TYPE_TRAITS_END()
+
 #endif
diff --git a/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsStore.cpp b/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsStore.cpp
new file mode 100644 (file)
index 0000000..1f92324
--- /dev/null
@@ -0,0 +1,571 @@
+/*
+ * Copyright (C) 2019 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "ResourceLoadStatisticsStore.h"
+
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+
+#include "Logging.h"
+#include "NetworkSession.h"
+#include "PluginProcessManager.h"
+#include "PluginProcessProxy.h"
+#include "ResourceLoadStatisticsPersistentStorage.h"
+#include "StorageAccessStatus.h"
+#include "WebProcessProxy.h"
+#include "WebResourceLoadStatisticsTelemetry.h"
+#include "WebsiteDataStore.h"
+#include <WebCore/KeyedCoding.h>
+#include <WebCore/NetworkStorageSession.h>
+#include <WebCore/ResourceLoadStatistics.h>
+#include <wtf/CallbackAggregator.h>
+#include <wtf/DateMath.h>
+#include <wtf/MathExtras.h>
+#include <wtf/text/StringBuilder.h>
+
+namespace WebKit {
+using namespace WebCore;
+
+constexpr Seconds minimumStatisticsProcessingInterval { 5_s };
+constexpr unsigned operatingDatesWindow { 30 };
+
+#if !RELEASE_LOG_DISABLED
+static String domainsToString(const Vector<RegistrableDomain>& domains)
+{
+    StringBuilder builder;
+    for (auto& domain : domains) {
+        if (!builder.isEmpty())
+            builder.appendLiteral(", ");
+        builder.append(domain.string());
+    }
+    return builder.toString();
+}
+#endif
+
+OperatingDate OperatingDate::fromWallTime(WallTime time)
+{
+    double ms = time.secondsSinceEpoch().milliseconds();
+    int year = msToYear(ms);
+    int yearDay = dayInYear(ms, year);
+    int month = monthFromDayInYear(yearDay, isLeapYear(year));
+    int monthDay = dayInMonthFromDayInYear(yearDay, isLeapYear(year));
+
+    return OperatingDate { year, month, monthDay };
+}
+
+OperatingDate OperatingDate::today()
+{
+    return OperatingDate::fromWallTime(WallTime::now());
+}
+
+Seconds OperatingDate::secondsSinceEpoch() const
+{
+    return Seconds { dateToDaysFrom1970(m_year, m_month, m_monthDay) * secondsPerDay };
+}
+
+bool OperatingDate::operator==(const OperatingDate& other) const
+{
+    return m_monthDay == other.m_monthDay && m_month == other.m_month && m_year == other.m_year;
+}
+
+bool OperatingDate::operator<(const OperatingDate& other) const
+{
+    return secondsSinceEpoch() < other.secondsSinceEpoch();
+}
+
+bool OperatingDate::operator<=(const OperatingDate& other) const
+{
+    return secondsSinceEpoch() <= other.secondsSinceEpoch();
+}
+
+ResourceLoadStatisticsStore::ResourceLoadStatisticsStore(WebResourceLoadStatisticsStore& store, WorkQueue& workQueue)
+    : m_store(store)
+    , m_workQueue(workQueue)
+{
+    ASSERT(!RunLoop::isMain());
+
+    includeTodayAsOperatingDateIfNecessary();
+}
+
+ResourceLoadStatisticsStore::~ResourceLoadStatisticsStore()
+{
+    ASSERT(!RunLoop::isMain());
+}
+
+unsigned ResourceLoadStatisticsStore::computeImportance(const ResourceLoadStatistics& resourceStatistic)
+{
+    unsigned importance = ResourceLoadStatisticsStore::maxImportance;
+    if (!resourceStatistic.isPrevalentResource)
+        importance -= 1;
+    if (!resourceStatistic.hadUserInteraction)
+        importance -= 2;
+    return importance;
+}
+
+void ResourceLoadStatisticsStore::setNotifyPagesWhenDataRecordsWereScanned(bool value)
+{
+    ASSERT(!RunLoop::isMain());
+    m_parameters.shouldNotifyPagesWhenDataRecordsWereScanned = value;
+}
+
+void ResourceLoadStatisticsStore::setShouldClassifyResourcesBeforeDataRecordsRemoval(bool value)
+{
+    ASSERT(!RunLoop::isMain());
+    m_parameters.shouldClassifyResourcesBeforeDataRecordsRemoval = value;
+}
+
+void ResourceLoadStatisticsStore::setShouldSubmitTelemetry(bool value)
+{
+    ASSERT(!RunLoop::isMain());
+    m_parameters.shouldSubmitTelemetry = value;
+}
+
+void ResourceLoadStatisticsStore::removeDataRecords(CompletionHandler<void()>&& callback)
+{
+    ASSERT(!RunLoop::isMain());
+
+    if (!shouldRemoveDataRecords()) {
+        callback();
+        return;
+    }
+
+#if ENABLE(NETSCAPE_PLUGIN_API)
+    m_activePluginTokens.clear();
+    for (const auto& plugin : PluginProcessManager::singleton().pluginProcesses())
+        m_activePluginTokens.add(plugin->pluginProcessToken());
+#endif
+
+    auto prevalentResourceDomains = registrableDomainsToRemoveWebsiteDataFor();
+    if (prevalentResourceDomains.isEmpty()) {
+        callback();
+        return;
+    }
+
+#if !RELEASE_LOG_DISABLED
+    RELEASE_LOG_INFO_IF(m_debugLoggingEnabled, ResourceLoadStatisticsDebug, "About to remove data records for %{public}s.", domainsToString(prevalentResourceDomains).utf8().data());
+#endif
+
+    setDataRecordsBeingRemoved(true);
+
+    RunLoop::main().dispatch([prevalentResourceDomains = crossThreadCopy(prevalentResourceDomains), callback = WTFMove(callback), weakThis = makeWeakPtr(*this), shouldNotifyPagesWhenDataRecordsWereScanned = m_parameters.shouldNotifyPagesWhenDataRecordsWereScanned, workQueue = m_workQueue.copyRef()] () mutable {
+        if (!weakThis) {
+            callback();
+            return;
+        }
+
+        weakThis->m_store.deleteWebsiteDataForRegistrableDomainsInAllPersistentDataStores(WebResourceLoadStatisticsStore::monitoredDataTypes(), WTFMove(prevalentResourceDomains), shouldNotifyPagesWhenDataRecordsWereScanned, [callback = WTFMove(callback), weakThis = WTFMove(weakThis), workQueue = workQueue.copyRef()](const HashSet<RegistrableDomain>& domainsWithDeletedWebsiteData) mutable {
+            workQueue->dispatch([domainsWithDeletedWebsiteData = crossThreadCopy(domainsWithDeletedWebsiteData), callback = WTFMove(callback), weakThis = WTFMove(weakThis)] () mutable {
+                if (!weakThis) {
+                    callback();
+                    return;
+                }
+                weakThis->incrementRecordsDeletedCountForDomains(WTFMove(domainsWithDeletedWebsiteData));
+                weakThis->setDataRecordsBeingRemoved(false);
+                callback();
+#if !RELEASE_LOG_DISABLED
+                RELEASE_LOG_INFO_IF(weakThis->m_debugLoggingEnabled, ResourceLoadStatisticsDebug, "Done removing data records.");
+#endif
+            });
+        });
+    });
+}
+
+void ResourceLoadStatisticsStore::processStatisticsAndDataRecords()
+{
+    ASSERT(!RunLoop::isMain());
+
+    if (m_parameters.shouldClassifyResourcesBeforeDataRecordsRemoval)
+        classifyPrevalentResources();
+    
+    removeDataRecords([this, weakThis = makeWeakPtr(*this)] () mutable {
+        ASSERT(!RunLoop::isMain());
+        if (!weakThis)
+            return;
+
+        pruneStatisticsIfNeeded();
+        syncStorageIfNeeded();
+
+        if (!m_parameters.shouldNotifyPagesWhenDataRecordsWereScanned)
+            return;
+
+        RunLoop::main().dispatch([this, weakThis = WTFMove(weakThis)] {
+            ASSERT(RunLoop::isMain());
+            if (!weakThis)
+                return;
+
+            m_store.notifyResourceLoadStatisticsProcessed();
+        });
+    });
+}
+
+void ResourceLoadStatisticsStore::grandfatherExistingWebsiteData(CompletionHandler<void()>&& callback)
+{
+    ASSERT(!RunLoop::isMain());
+
+    RunLoop::main().dispatch([weakThis = makeWeakPtr(*this), callback = WTFMove(callback), shouldNotifyPagesWhenDataRecordsWereScanned = m_parameters.shouldNotifyPagesWhenDataRecordsWereScanned, workQueue = m_workQueue.copyRef(), store = makeRef(m_store)] () mutable {
+        store->registrableDomainsWithWebsiteData(WebResourceLoadStatisticsStore::monitoredDataTypes(), shouldNotifyPagesWhenDataRecordsWereScanned, [weakThis = WTFMove(weakThis), callback = WTFMove(callback), workQueue = workQueue.copyRef()] (HashSet<RegistrableDomain>&& domainsWithWebsiteData) mutable {
+            workQueue->dispatch([weakThis = WTFMove(weakThis), domainsWithWebsiteData = crossThreadCopy(domainsWithWebsiteData), callback = WTFMove(callback)] () mutable {
+                if (!weakThis) {
+                    callback();
+                    return;
+                }
+
+                weakThis->grandfatherDataForDomains(domainsWithWebsiteData);
+                weakThis->m_endOfGrandfatheringTimestamp = WallTime::now() + weakThis->m_parameters.grandfatheringTime;
+                weakThis->syncStorageImmediately();
+                callback();
+                weakThis->logTestingEvent("Grandfathered"_s);
+            });
+        });
+    });
+}
+
+void ResourceLoadStatisticsStore::setResourceLoadStatisticsDebugMode(bool enable)
+{
+    ASSERT(!RunLoop::isMain());
+
+#if !RELEASE_LOG_DISABLED
+    if (enable)
+        RELEASE_LOG_INFO(ResourceLoadStatisticsDebug, "Turned ITP Debug Mode on.");
+#endif
+
+    m_debugModeEnabled = enable;
+    m_debugLoggingEnabled = enable;
+
+    ensurePrevalentResourcesForDebugMode();
+    // This will log the current cookie blocking state.
+    if (enable)
+        updateCookieBlocking([]() { });
+}
+
+void ResourceLoadStatisticsStore::setPrevalentResourceForDebugMode(const RegistrableDomain& domain)
+{
+    m_debugManualPrevalentResource = domain;
+}
+
+void ResourceLoadStatisticsStore::scheduleStatisticsProcessingRequestIfNecessary()
+{
+    ASSERT(!RunLoop::isMain());
+
+    m_pendingStatisticsProcessingRequestIdentifier = ++m_lastStatisticsProcessingRequestIdentifier;
+    m_workQueue->dispatchAfter(minimumStatisticsProcessingInterval, [this, weakThis = makeWeakPtr(*this), statisticsProcessingRequestIdentifier = *m_pendingStatisticsProcessingRequestIdentifier] {
+        if (!weakThis)
+            return;
+
+        if (!m_pendingStatisticsProcessingRequestIdentifier || *m_pendingStatisticsProcessingRequestIdentifier != statisticsProcessingRequestIdentifier) {
+            // This request has been canceled.
+            return;
+        }
+
+        updateCookieBlocking([]() { });
+        processStatisticsAndDataRecords();
+    });
+}
+
+void ResourceLoadStatisticsStore::cancelPendingStatisticsProcessingRequest()
+{
+    ASSERT(!RunLoop::isMain());
+
+    m_pendingStatisticsProcessingRequestIdentifier = WTF::nullopt;
+}
+
+void ResourceLoadStatisticsStore::setTimeToLiveUserInteraction(Seconds seconds)
+{
+    ASSERT(!RunLoop::isMain());
+    ASSERT(seconds >= 0_s);
+
+    m_parameters.timeToLiveUserInteraction = seconds;
+}
+
+void ResourceLoadStatisticsStore::setMinimumTimeBetweenDataRecordsRemoval(Seconds seconds)
+{
+    ASSERT(!RunLoop::isMain());
+    ASSERT(seconds >= 0_s);
+
+    m_parameters.minimumTimeBetweenDataRecordsRemoval = seconds;
+}
+
+void ResourceLoadStatisticsStore::setGrandfatheringTime(Seconds seconds)
+{
+    ASSERT(!RunLoop::isMain());
+    ASSERT(seconds >= 0_s);
+
+    m_parameters.grandfatheringTime = seconds;
+}
+
+void ResourceLoadStatisticsStore::setCacheMaxAgeCap(Seconds seconds)
+{
+    ASSERT(!RunLoop::isMain());
+    ASSERT(seconds >= 0_s);
+
+    m_parameters.cacheMaxAgeCapTime = seconds;
+    updateCacheMaxAgeCap();
+}
+
+void ResourceLoadStatisticsStore::updateCacheMaxAgeCap()
+{
+    ASSERT(!RunLoop::isMain());
+    
+    RunLoop::main().dispatch([store = makeRef(m_store), seconds = m_parameters.cacheMaxAgeCapTime] () {
+        store->setCacheMaxAgeCap(seconds, [] { });
+    });
+}
+
+void ResourceLoadStatisticsStore::setAgeCapForClientSideCookies(Seconds seconds)
+{
+    ASSERT(!RunLoop::isMain());
+    ASSERT(seconds >= 0_s);
+    
+    m_parameters.clientSideCookiesAgeCapTime = seconds;
+    updateClientSideCookiesAgeCap();
+}
+
+void ResourceLoadStatisticsStore::updateClientSideCookiesAgeCap()
+{
+    ASSERT(!RunLoop::isMain());
+
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+    RunLoop::main().dispatch([store = makeRef(m_store), seconds = m_parameters.clientSideCookiesAgeCapTime] () {
+        if (auto* networkSession = store->networkSession())
+            networkSession->networkStorageSession().setAgeCapForClientSideCookies(seconds);
+    });
+#endif
+}
+
+bool ResourceLoadStatisticsStore::shouldRemoveDataRecords() const
+{
+    ASSERT(!RunLoop::isMain());
+
+    if (m_dataRecordsBeingRemoved)
+        return false;
+
+#if ENABLE(NETSCAPE_PLUGIN_API)
+    for (const auto& plugin : PluginProcessManager::singleton().pluginProcesses()) {
+        if (!m_activePluginTokens.contains(plugin->pluginProcessToken()))
+            return true;
+    }
+#endif
+
+    return !m_lastTimeDataRecordsWereRemoved || MonotonicTime::now() >= (m_lastTimeDataRecordsWereRemoved + m_parameters.minimumTimeBetweenDataRecordsRemoval);
+}
+
+void ResourceLoadStatisticsStore::setDataRecordsBeingRemoved(bool value)
+{
+    ASSERT(!RunLoop::isMain());
+
+    m_dataRecordsBeingRemoved = value;
+    if (m_dataRecordsBeingRemoved)
+        m_lastTimeDataRecordsWereRemoved = MonotonicTime::now();
+}
+
+void ResourceLoadStatisticsStore::updateCookieBlockingForDomains(const Vector<RegistrableDomain>& domainsToBlock, CompletionHandler<void()>&& completionHandler)
+{
+    ASSERT(!RunLoop::isMain());
+    
+    RunLoop::main().dispatch([store = makeRef(m_store), domainsToBlock = crossThreadCopy(domainsToBlock), completionHandler = WTFMove(completionHandler)] () mutable {
+        store->callUpdatePrevalentDomainsToBlockCookiesForHandler(domainsToBlock, [store = store.copyRef(), completionHandler = WTFMove(completionHandler)]() mutable {
+            store->statisticsQueue().dispatch([completionHandler = WTFMove(completionHandler)]() mutable {
+                completionHandler();
+            });
+        });
+    });
+}
+    
+
+void ResourceLoadStatisticsStore::clearBlockingStateForDomains(const Vector<RegistrableDomain>& domains, CompletionHandler<void()>&& completionHandler)
+{
+    ASSERT(!RunLoop::isMain());
+
+    if (domains.isEmpty()) {
+        completionHandler();
+        return;
+    }
+
+    RunLoop::main().dispatch([store = makeRef(m_store), domains = crossThreadCopy(domains)] {
+        store->callRemoveDomainsHandler(domains);
+    });
+
+    completionHandler();
+}
+
+Optional<Seconds> ResourceLoadStatisticsStore::statisticsEpirationTime() const
+{
+    if (m_parameters.timeToLiveUserInteraction)
+        return WallTime::now().secondsSinceEpoch() - m_parameters.timeToLiveUserInteraction.value();
+    
+    if (m_operatingDates.size() >= operatingDatesWindow)
+        return m_operatingDates.first().secondsSinceEpoch();
+    
+    return WTF::nullopt;
+}
+
+Vector<OperatingDate> ResourceLoadStatisticsStore::mergeOperatingDates(const Vector<OperatingDate>& existingDates, Vector<OperatingDate>&& newDates)
+{
+    if (existingDates.isEmpty())
+        return WTFMove(newDates);
+    
+    Vector<OperatingDate> mergedDates(existingDates.size() + newDates.size());
+    
+    // Merge the two sorted vectors of dates.
+    std::merge(existingDates.begin(), existingDates.end(), newDates.begin(), newDates.end(), mergedDates.begin());
+    // Remove duplicate dates.
+    removeRepeatedElements(mergedDates);
+    
+    // Drop old dates until the Vector size reaches operatingDatesWindow.
+    while (mergedDates.size() > operatingDatesWindow)
+        mergedDates.remove(0);
+    
+    return mergedDates;
+}
+
+void ResourceLoadStatisticsStore::mergeOperatingDates(Vector<OperatingDate>&& newDates)
+{
+    m_operatingDates = mergeOperatingDates(m_operatingDates, WTFMove(newDates));
+}
+
+void ResourceLoadStatisticsStore::includeTodayAsOperatingDateIfNecessary()
+{
+    ASSERT(!RunLoop::isMain());
+
+    auto today = OperatingDate::today();
+    if (!m_operatingDates.isEmpty() && today <= m_operatingDates.last())
+        return;
+
+    while (m_operatingDates.size() >= operatingDatesWindow)
+        m_operatingDates.remove(0);
+
+    m_operatingDates.append(today);
+}
+
+bool ResourceLoadStatisticsStore::hasStatisticsExpired(WallTime mostRecentUserInteractionTime) const
+{
+    ASSERT(!RunLoop::isMain());
+    
+    if (m_operatingDates.size() >= operatingDatesWindow) {
+        if (OperatingDate::fromWallTime(mostRecentUserInteractionTime) < m_operatingDates.first())
+            return true;
+    }
+    
+    // If we don't meet the real criteria for an expired statistic, check the user setting for a tighter restriction (mainly for testing).
+    if (m_parameters.timeToLiveUserInteraction) {
+        if (WallTime::now() > mostRecentUserInteractionTime + m_parameters.timeToLiveUserInteraction.value())
+            return true;
+    }
+    
+    return false;
+}
+
+bool ResourceLoadStatisticsStore::hasStatisticsExpired(const ResourceLoadStatistics& resourceStatistic) const
+{
+    return hasStatisticsExpired(resourceStatistic.mostRecentUserInteractionTime);
+}
+
+void ResourceLoadStatisticsStore::setMaxStatisticsEntries(size_t maximumEntryCount)
+{
+    ASSERT(!RunLoop::isMain());
+
+    m_parameters.maxStatisticsEntries = maximumEntryCount;
+}
+
+void ResourceLoadStatisticsStore::setPruneEntriesDownTo(size_t pruneTargetCount)
+{
+    ASSERT(!RunLoop::isMain());
+
+    m_parameters.pruneEntriesDownTo = pruneTargetCount;
+}
+
+void ResourceLoadStatisticsStore::resetParametersToDefaultValues()
+{
+    ASSERT(!RunLoop::isMain());
+
+    m_parameters = { };
+}
+
+void ResourceLoadStatisticsStore::logTestingEvent(const String& event)
+{
+    ASSERT(!RunLoop::isMain());
+
+    RunLoop::main().dispatch([store = makeRef(m_store), event = event.isolatedCopy()] {
+        store->logTestingEvent(event);
+    });
+}
+
+void ResourceLoadStatisticsStore::removeAllStorageAccess(CompletionHandler<void()>&& completionHandler)
+{
+    ASSERT(!RunLoop::isMain());
+    RunLoop::main().dispatch([store = makeRef(m_store), completionHandler = WTFMove(completionHandler)]() mutable {
+        store->removeAllStorageAccess([store = store.copyRef(), completionHandler = WTFMove(completionHandler)]() mutable {
+            store->statisticsQueue().dispatch([completionHandler = WTFMove(completionHandler)]() mutable {
+                completionHandler();
+            });
+        });
+    });
+}
+
+void ResourceLoadStatisticsStore::didCreateNetworkProcess()
+{
+    ASSERT(!RunLoop::isMain());
+
+    updateCookieBlocking([]() { });
+    updateCacheMaxAgeCap();
+    updateClientSideCookiesAgeCap();
+}
+
+void ResourceLoadStatisticsStore::debugLogDomainsInBatches(const char* action, const Vector<RegistrableDomain>& domains)
+{
+#if !RELEASE_LOG_DISABLED
+    static const auto maxNumberOfDomainsInOneLogStatement = 50;
+    if (domains.isEmpty())
+        return;
+    
+    if (domains.size() <= maxNumberOfDomainsInOneLogStatement) {
+        RELEASE_LOG_INFO(ResourceLoadStatisticsDebug, "About to %{public}s cookies in third-party contexts for: %{public}s.", action, domainsToString(domains).utf8().data());
+        return;
+    }
+    
+    Vector<RegistrableDomain> batch;
+    batch.reserveInitialCapacity(maxNumberOfDomainsInOneLogStatement);
+    auto batchNumber = 1;
+    unsigned numberOfBatches = std::ceil(domains.size() / static_cast<float>(maxNumberOfDomainsInOneLogStatement));
+    
+    for (auto& domain : domains) {
+        if (batch.size() == maxNumberOfDomainsInOneLogStatement) {
+            RELEASE_LOG_INFO(ResourceLoadStatisticsDebug, "About to %{public}s cookies in third-party contexts for (%{public}d of %u): %{public}s.", action, batchNumber, numberOfBatches, domainsToString(batch).utf8().data());
+            batch.shrink(0);
+            ++batchNumber;
+        }
+        batch.append(domain);
+    }
+    if (!batch.isEmpty())
+        RELEASE_LOG_INFO(ResourceLoadStatisticsDebug, "About to %{public}s cookies in third-party contexts for (%{public}d of %u): %{public}s.", action, batchNumber, numberOfBatches, domainsToString(batch).utf8().data());
+#else
+    UNUSED_PARAM(action);
+    UNUSED_PARAM(domains);
+#endif
+}
+
+} // namespace WebKit
+
+#endif
diff --git a/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsStore.h b/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsStore.h
new file mode 100644 (file)
index 0000000..f2e7b63
--- /dev/null
@@ -0,0 +1,271 @@
+/*
+ * Copyright (C) 2019 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+
+#include "ResourceLoadStatisticsClassifier.h"
+#include "WebResourceLoadStatisticsStore.h"
+#include <wtf/CompletionHandler.h>
+#include <wtf/Vector.h>
+#include <wtf/WeakPtr.h>
+#include <wtf/WorkQueue.h>
+
+#if HAVE(CORE_PREDICTION)
+#include "ResourceLoadStatisticsClassifierCocoa.h"
+#endif
+
+namespace WebCore {
+class KeyedDecoder;
+class KeyedEncoder;
+struct ResourceLoadStatistics;
+}
+
+namespace WebKit {
+
+class ResourceLoadStatisticsPersistentStorage;
+
+class OperatingDate {
+public:
+    OperatingDate() = default;
+    
+    static OperatingDate fromWallTime(WallTime);
+    static OperatingDate today();
+    Seconds secondsSinceEpoch() const;
+    bool operator==(const OperatingDate& other) const;
+    bool operator<(const OperatingDate& other) const;
+    bool operator<=(const OperatingDate& other) const;
+    
+private:
+    OperatingDate(int year, int month, int monthDay)
+        : m_year(year)
+        , m_month(month)
+        , m_monthDay(monthDay)
+    { }
+
+    int m_year { 0 };
+    int m_month { 0 }; // [0, 11].
+    int m_monthDay { 0 }; // [1, 31].
+};
+
+// This is always constructed / used / destroyed on the WebResourceLoadStatisticsStore's statistics queue.
+class ResourceLoadStatisticsStore : public CanMakeWeakPtr<ResourceLoadStatisticsStore> {
+public:
+    using ResourceLoadStatistics = WebCore::ResourceLoadStatistics;
+    using RegistrableDomain = WebCore::RegistrableDomain;
+    using TopFrameDomain = WebCore::RegistrableDomain;
+    using SubFrameDomain = WebCore::RegistrableDomain;
+    using SubResourceDomain = WebCore::RegistrableDomain;
+    using RedirectDomain = WebCore::RegistrableDomain;
+    using RedirectedFromDomain = WebCore::RegistrableDomain;
+    using RedirectedToDomain = WebCore::RegistrableDomain;
+    using NavigatedFromDomain = WebCore::RegistrableDomain;
+    using NavigatedToDomain = WebCore::RegistrableDomain;
+    using DomainInNeedOfStorageAccess = WebCore::RegistrableDomain;
+    using OpenerDomain = WebCore::RegistrableDomain;
+    using OpenerPageID = uint64_t;
+    using PageID = uint64_t;
+    using FrameID = uint64_t;
+    
+    virtual ~ResourceLoadStatisticsStore();
+
+    virtual void clear(CompletionHandler<void()>&&) = 0;
+    virtual bool isEmpty() const = 0;
+
+    virtual void updateCookieBlocking(CompletionHandler<void()>&&) = 0;
+    void updateCookieBlockingForDomains(const Vector<RegistrableDomain>& domainsToBlock, CompletionHandler<void()>&&);
+    void clearBlockingStateForDomains(const Vector<RegistrableDomain>& domains, CompletionHandler<void()>&&);
+
+    void includeTodayAsOperatingDateIfNecessary();
+    void processStatisticsAndDataRecords();
+
+    virtual void classifyPrevalentResources() = 0;
+    virtual void syncStorageIfNeeded() = 0;
+    virtual void syncStorageImmediately() = 0;
+
+    virtual void requestStorageAccessUnderOpener(DomainInNeedOfStorageAccess&&, OpenerPageID, OpenerDomain&&) = 0;
+    void removeAllStorageAccess(CompletionHandler<void()>&&);
+
+    void grandfatherExistingWebsiteData(CompletionHandler<void()>&&);
+    void cancelPendingStatisticsProcessingRequest();
+
+    virtual bool isRegisteredAsSubresourceUnder(const SubResourceDomain&, const TopFrameDomain&) const = 0;
+    virtual bool isRegisteredAsSubFrameUnder(const SubFrameDomain&, const TopFrameDomain&) const = 0;
+    virtual bool isRegisteredAsRedirectingTo(const RedirectedFromDomain&, const RedirectedToDomain&) const = 0;
+
+    virtual void clearPrevalentResource(const RegistrableDomain&) = 0;
+    virtual String dumpResourceLoadStatistics() const = 0;
+    virtual bool isPrevalentResource(const RegistrableDomain&) const = 0;
+    virtual bool isVeryPrevalentResource(const RegistrableDomain&) const = 0;
+    virtual void setPrevalentResource(const RegistrableDomain&) = 0;
+    virtual void setVeryPrevalentResource(const RegistrableDomain&) = 0;
+
+    virtual void setGrandfathered(const RegistrableDomain&, bool value) = 0;
+    virtual bool isGrandfathered(const RegistrableDomain&) const = 0;
+
+    virtual void incrementRecordsDeletedCountForDomains(HashSet<RegistrableDomain>&&) = 0;
+    virtual void grandfatherDataForDomains(const HashSet<RegistrableDomain>&) = 0;
+
+    virtual void setSubframeUnderTopFrameDomain(const SubFrameDomain&, const TopFrameDomain&) = 0;
+    virtual void setSubresourceUnderTopFrameDomain(const SubResourceDomain&, const TopFrameDomain&) = 0;
+    virtual void setSubresourceUniqueRedirectTo(const SubResourceDomain&, const RedirectDomain&) = 0;
+    virtual void setSubresourceUniqueRedirectFrom(const SubResourceDomain&, const RedirectDomain&) = 0;
+    virtual void setTopFrameUniqueRedirectTo(const TopFrameDomain&, const RedirectDomain&) = 0;
+    virtual void setTopFrameUniqueRedirectFrom(const TopFrameDomain&, const RedirectDomain&) = 0;
+
+    void logTestingEvent(const String&);
+
+    void setMaxStatisticsEntries(size_t maximumEntryCount);
+    void setPruneEntriesDownTo(size_t pruneTargetCount);
+    void resetParametersToDefaultValues();
+    Optional<Seconds> statisticsEpirationTime() const;
+
+    virtual void calculateAndSubmitTelemetry() const = 0;
+
+    void setNotifyPagesWhenDataRecordsWereScanned(bool);
+    void setShouldClassifyResourcesBeforeDataRecordsRemoval(bool);
+    void setShouldSubmitTelemetry(bool);
+    void setTimeToLiveUserInteraction(Seconds);
+    void setMinimumTimeBetweenDataRecordsRemoval(Seconds);
+    void setGrandfatheringTime(Seconds);
+    void setResourceLoadStatisticsDebugMode(bool);
+    bool isDebugModeEnabled() const { return m_debugModeEnabled; };
+    void setPrevalentResourceForDebugMode(const RegistrableDomain&);
+
+    virtual void hasStorageAccess(const SubFrameDomain&, const TopFrameDomain&, Optional<FrameID>, PageID, CompletionHandler<void(bool)>&&) = 0;
+    virtual void requestStorageAccess(SubFrameDomain&&, TopFrameDomain&&, FrameID, PageID, bool promptEnabled, CompletionHandler<void(StorageAccessStatus)>&&) = 0;
+    virtual void grantStorageAccess(SubFrameDomain&&, TopFrameDomain&&, FrameID, PageID, bool userWasPromptedNow, CompletionHandler<void(bool)>&&) = 0;
+
+    virtual void logFrameNavigation(const NavigatedToDomain&, const TopFrameDomain&, const NavigatedFromDomain&, bool isRedirect, bool isMainFrame) = 0;
+    virtual void logUserInteraction(const TopFrameDomain&) = 0;
+    virtual void logSubresourceLoading(const SubResourceDomain&, const TopFrameDomain&, WallTime lastSeen) = 0;
+    virtual void logSubresourceRedirect(const RedirectedFromDomain&, const RedirectedToDomain&) = 0;
+
+    virtual void clearUserInteraction(const RegistrableDomain&) = 0;
+    virtual bool hasHadUserInteraction(const RegistrableDomain&) = 0;
+
+    virtual void setLastSeen(const RegistrableDomain& primaryDomain, Seconds) = 0;
+
+    void didCreateNetworkProcess();
+
+    const WebResourceLoadStatisticsStore& store() const { return m_store; }
+
+    static constexpr unsigned maxImportance { 3 };
+
+    virtual bool isMemoryStore() const { return false; }
+    virtual bool isDatabaseStore()const { return false; }
+
+protected:
+    static unsigned computeImportance(const WebCore::ResourceLoadStatistics&);
+    static Vector<OperatingDate> mergeOperatingDates(const Vector<OperatingDate>& existingDates, Vector<OperatingDate>&& newDates);
+    static void debugLogDomainsInBatches(const char* action, const Vector<RegistrableDomain>& domains);
+
+    ResourceLoadStatisticsStore(WebResourceLoadStatisticsStore&, WorkQueue&);
+
+    bool hasStatisticsExpired(const ResourceLoadStatistics&) const;
+    bool hasStatisticsExpired(WallTime mostRecentUserInteractionTime) const;
+#if PLATFORM(COCOA)
+    void registerUserDefaultsIfNeeded();
+#endif
+    void scheduleStatisticsProcessingRequestIfNecessary();
+    void mergeOperatingDates(Vector<OperatingDate>&&);
+    virtual Vector<RegistrableDomain> ensurePrevalentResourcesForDebugMode() = 0;
+    virtual Vector<RegistrableDomain> registrableDomainsToRemoveWebsiteDataFor() = 0;
+    virtual void pruneStatisticsIfNeeded() = 0;
+
+    WebResourceLoadStatisticsStore& store() { return m_store; }
+    Ref<WorkQueue>& workQueue() { return m_workQueue; }
+#if HAVE(CORE_PREDICTION)
+    ResourceLoadStatisticsClassifierCocoa& classifier() { return m_resourceLoadStatisticsClassifier; }
+#else
+    ResourceLoadStatisticsClassifier& classifier() { return m_resourceLoadStatisticsClassifier; }
+#endif
+
+    struct Parameters {
+        size_t pruneEntriesDownTo { 800 };
+        size_t maxStatisticsEntries { 1000 };
+        Optional<Seconds> timeToLiveUserInteraction;
+        Seconds minimumTimeBetweenDataRecordsRemoval { 1_h };
+        Seconds grandfatheringTime { 24_h * 7 };
+        Seconds cacheMaxAgeCapTime { 24_h * 7 };
+        Seconds clientSideCookiesAgeCapTime { 24_h * 7 };
+        bool shouldNotifyPagesWhenDataRecordsWereScanned { false };
+        bool shouldClassifyResourcesBeforeDataRecordsRemoval { true };
+        bool shouldSubmitTelemetry { true };
+    };
+    const Parameters& parameters() const { return m_parameters; }
+    const Vector<OperatingDate>& operatingDates() const { return m_operatingDates; }
+    void clearOperatingDates() { m_operatingDates.clear(); }
+    WallTime& endOfGrandfatheringTimestamp() { return m_endOfGrandfatheringTimestamp; }
+    const WallTime& endOfGrandfatheringTimestamp() const { return m_endOfGrandfatheringTimestamp; }
+    void setEndOfGrandfatheringTimestamp(const WallTime& grandfatheringTime) { m_endOfGrandfatheringTimestamp = grandfatheringTime; }
+    void clearEndOfGrandfatheringTimeStamp() { m_endOfGrandfatheringTimestamp = { }; }
+    const RegistrableDomain& debugManualPrevalentResource() const { return m_debugManualPrevalentResource; }
+    const RegistrableDomain& debugStaticPrevalentResource() const { return m_debugStaticPrevalentResource; }
+    bool debugLoggingEnabled() const { return m_debugLoggingEnabled; };
+    bool debugModeEnabled() const { return m_debugModeEnabled; }
+    bool storageAccessPromptsEnabled() const { return m_storageAccessPromptsEnabled; }
+
+    static constexpr unsigned maxNumberOfRecursiveCallsInRedirectTraceBack { 50 };
+
+private:
+    void setStorageAccessPromptsEnabled(bool enabled) { m_storageAccessPromptsEnabled  = enabled; }
+    bool shouldRemoveDataRecords() const;
+    void setDebugLogggingEnabled(bool enabled) { m_debugLoggingEnabled  = enabled; }
+    void setDataRecordsBeingRemoved(bool);
+    void removeDataRecords(CompletionHandler<void()>&&);
+    void setCacheMaxAgeCap(Seconds);
+    void updateCacheMaxAgeCap();
+    void setAgeCapForClientSideCookies(Seconds);
+    void updateClientSideCookiesAgeCap();
+
+    WebResourceLoadStatisticsStore& m_store;
+    Ref<WorkQueue> m_workQueue;
+#if HAVE(CORE_PREDICTION)
+    ResourceLoadStatisticsClassifierCocoa m_resourceLoadStatisticsClassifier;
+#else
+    ResourceLoadStatisticsClassifier m_resourceLoadStatisticsClassifier;
+#endif
+#if ENABLE(NETSCAPE_PLUGIN_API)
+    HashSet<uint64_t> m_activePluginTokens;
+#endif
+    Parameters m_parameters;
+    Vector<OperatingDate> m_operatingDates;
+    WallTime m_endOfGrandfatheringTimestamp;
+    RegistrableDomain m_debugManualPrevalentResource;
+    MonotonicTime m_lastTimeDataRecordsWereRemoved;
+    uint64_t m_lastStatisticsProcessingRequestIdentifier { 0 };
+    Optional<uint64_t> m_pendingStatisticsProcessingRequestIdentifier;
+    const RegistrableDomain m_debugStaticPrevalentResource { URL { URL(), "https://3rdpartytestwebkit.org"_s } };
+    bool m_debugLoggingEnabled { false };
+    bool m_debugModeEnabled { false };
+    bool m_storageAccessPromptsEnabled { false };
+    bool m_dataRecordsBeingRemoved { false };
+};
+
+} // namespace WebKit
+
+#endif
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2017-2018 Apple Inc. All rights reserved.
+ * Copyright (C) 2017-2019 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -24,7 +24,7 @@
  */
 
 #import "config.h"
-#import "ResourceLoadStatisticsMemoryStore.h"
+#import "ResourceLoadStatisticsStore.h"
 
 #if ENABLE(RESOURCE_LOAD_STATISTICS)
 
@@ -34,7 +34,7 @@
 
 namespace WebKit {
 
-void ResourceLoadStatisticsMemoryStore::registerUserDefaultsIfNeeded()
+void ResourceLoadStatisticsStore::registerUserDefaultsIfNeeded()
 {
     static dispatch_once_t initOnce;
 
index 655d11e..3713f9b 100644 (file)
@@ -32,6 +32,7 @@
 #include "Logging.h"
 #include "NetworkProcess.h"
 #include "NetworkSession.h"
+#include "ResourceLoadStatisticsDatabaseStore.h"
 #include "ResourceLoadStatisticsMemoryStore.h"
 #include "ResourceLoadStatisticsPersistentStorage.h"
 #include "ShouldGrandfatherStatistics.h"
@@ -48,6 +49,9 @@
 #include <WebCore/DiagnosticLoggingKeys.h>
 #include <WebCore/NetworkStorageSession.h>
 #include <WebCore/ResourceLoadStatistics.h>
+#include <WebCore/RuntimeEnabledFeatures.h>
+#include <WebCore/SQLiteDatabase.h>
+#include <WebCore/SQLiteStatement.h>
 #include <wtf/CallbackAggregator.h>
 #include <wtf/CrossThreadCopier.h>
 #include <wtf/NeverDestroyed.h>
@@ -86,8 +90,8 @@ void WebResourceLoadStatisticsStore::setNotifyPagesWhenDataRecordsWereScanned(bo
     ASSERT(RunLoop::isMain());
 
     postTask([this, value] {
-        if (m_memoryStore)
-            m_memoryStore->setNotifyPagesWhenDataRecordsWereScanned(value);
+        if (m_statisticsStore)
+            m_statisticsStore->setNotifyPagesWhenDataRecordsWereScanned(value);
     });
 }
 
@@ -96,9 +100,9 @@ void WebResourceLoadStatisticsStore::setNotifyPagesWhenDataRecordsWereScanned(bo
     ASSERT(RunLoop::isMain());
     
     postTask([this, value, completionHandler = WTFMove(completionHandler)]() mutable {
-        if (m_memoryStore)
-            m_memoryStore->setNotifyPagesWhenDataRecordsWereScanned(value);
-        
+        if (m_statisticsStore)
+            m_statisticsStore->setNotifyPagesWhenDataRecordsWereScanned(value);
+
         postTaskReply(WTFMove(completionHandler));
     });
 }
@@ -108,8 +112,8 @@ void WebResourceLoadStatisticsStore::setShouldClassifyResourcesBeforeDataRecords
     ASSERT(RunLoop::isMain());
 
     postTask([this, value, completionHandler = WTFMove(completionHandler)]() mutable {
-        if (m_memoryStore)
-            m_memoryStore->setShouldClassifyResourcesBeforeDataRecordsRemoval(value);
+        if (m_statisticsStore)
+            m_statisticsStore->setShouldClassifyResourcesBeforeDataRecordsRemoval(value);
 
         postTaskReply(WTFMove(completionHandler));
     });
@@ -120,8 +124,8 @@ void WebResourceLoadStatisticsStore::setShouldSubmitTelemetry(bool value)
     ASSERT(RunLoop::isMain());
 
     postTask([this, value] {
-        if (m_memoryStore)
-            m_memoryStore->setShouldSubmitTelemetry(value);
+        if (m_statisticsStore)
+            m_statisticsStore->setShouldSubmitTelemetry(value);
     });
 }
 
@@ -141,13 +145,22 @@ WebResourceLoadStatisticsStore::WebResourceLoadStatisticsStore(NetworkSession& n
     ASSERT(RunLoop::isMain());
     
     postTask([this, resourceLoadStatisticsDirectory = resourceLoadStatisticsDirectory.isolatedCopy()] {
-        m_memoryStore = std::make_unique<ResourceLoadStatisticsMemoryStore>(*this, m_statisticsQueue);
-        m_persistentStorage = std::make_unique<ResourceLoadStatisticsPersistentStorage>(*m_memoryStore, m_statisticsQueue, resourceLoadStatisticsDirectory);
+        if (RuntimeEnabledFeatures::sharedFeatures().isITPDatabaseEnabled()) {
+            m_statisticsStore = std::make_unique<ResourceLoadStatisticsDatabaseStore>(*this, m_statisticsQueue, resourceLoadStatisticsDirectory);
+
+            auto memoryStore = std::make_unique<ResourceLoadStatisticsMemoryStore>(*this, m_statisticsQueue);
+            auto persistentStore = std::make_unique<ResourceLoadStatisticsPersistentStorage>(*memoryStore, m_statisticsQueue, resourceLoadStatisticsDirectory);
+
+            downcast<ResourceLoadStatisticsDatabaseStore>(*m_statisticsStore.get()).populateFromMemoryStore(*memoryStore);
+        } else {
+            m_statisticsStore = std::make_unique<ResourceLoadStatisticsMemoryStore>(*this, m_statisticsQueue);
+            m_persistentStorage = std::make_unique<ResourceLoadStatisticsPersistentStorage>(downcast<ResourceLoadStatisticsMemoryStore>(*m_statisticsStore), m_statisticsQueue, resourceLoadStatisticsDirectory);
+        }
 
         // FIXME(193297): This should be revised after the UIProcess version goes away.
-        m_memoryStore->didCreateNetworkProcess();
+        m_statisticsStore->didCreateNetworkProcess();
     });
-    
+
     m_dailyTasksTimer.startRepeating(24_h);
 }
 
@@ -176,7 +189,7 @@ void WebResourceLoadStatisticsStore::flushAndDestroyPersistentStore()
 {
     ASSERT(RunLoop::isMain());
 
-    if (!m_persistentStorage && !m_memoryStore)
+    if (!m_persistentStorage && !m_statisticsStore)
         return;
 
     // Make sure we destroy the persistent store on the background queue and wait for it to die
@@ -186,7 +199,7 @@ void WebResourceLoadStatisticsStore::flushAndDestroyPersistentStore()
     BinarySemaphore semaphore;
     m_statisticsQueue->dispatch([&semaphore, this] {
         m_persistentStorage = nullptr;
-        m_memoryStore = nullptr;
+        m_statisticsStore = nullptr;
         semaphore.signal();
     });
     semaphore.wait();
@@ -197,8 +210,8 @@ void WebResourceLoadStatisticsStore::setResourceLoadStatisticsDebugMode(bool val
     ASSERT(RunLoop::isMain());
 
     postTask([this, value, completionHandler = WTFMove(completionHandler)]() mutable {
-        if (m_memoryStore)
-            m_memoryStore->setResourceLoadStatisticsDebugMode(value);
+        if (m_statisticsStore)
+            m_statisticsStore->setResourceLoadStatisticsDebugMode(value);
         postTaskReply(WTFMove(completionHandler));
     });
 }
@@ -208,8 +221,8 @@ void WebResourceLoadStatisticsStore::setPrevalentResourceForDebugMode(const Regi
     ASSERT(RunLoop::isMain());
     
     postTask([this, domain = domain.isolatedCopy(), completionHandler = WTFMove(completionHandler)]() mutable {
-        if (m_memoryStore)
-            m_memoryStore->setPrevalentResourceForDebugMode(domain);
+        if (m_statisticsStore)
+            m_statisticsStore->setPrevalentResourceForDebugMode(domain);
         postTaskReply(WTFMove(completionHandler));
     });
 }
@@ -219,8 +232,8 @@ void WebResourceLoadStatisticsStore::scheduleStatisticsAndDataRecordsProcessing(
     ASSERT(RunLoop::isMain());
     
     postTask([this, completionHandler = WTFMove(completionHandler)]() mutable {
-        if (m_memoryStore)
-            m_memoryStore->processStatisticsAndDataRecords();
+        if (m_statisticsStore)
+            m_statisticsStore->processStatisticsAndDataRecords();
         postTaskReply(WTFMove(completionHandler));
     });
 }
@@ -238,17 +251,19 @@ void WebResourceLoadStatisticsStore::resourceLoadStatisticsUpdated(Vector<WebCor
     // coming from IPC. ResourceLoadStatistics only contains strings which are safe to move to other threads as long
     // as nobody on this thread holds a reference to those strings.
     postTask([this, origins = WTFMove(origins)]() mutable {
-        if (!m_memoryStore)
+        if (!m_statisticsStore || !is<ResourceLoadStatisticsMemoryStore>(*m_statisticsStore))
             return;
 
-        m_memoryStore->mergeStatistics(WTFMove(origins));
+        auto& memoryStore = downcast<ResourceLoadStatisticsMemoryStore>(*m_statisticsStore);
+    
+        memoryStore.mergeStatistics(WTFMove(origins));
 
         // We can cancel any pending request to process statistics since we're doing it synchronously below.
-        m_memoryStore->cancelPendingStatisticsProcessingRequest();
+        memoryStore.cancelPendingStatisticsProcessingRequest();
 
         // Fire before processing statistics to propagate user interaction as fast as possible to the network process.
-        m_memoryStore->updateCookieBlocking([]() { });
-        m_memoryStore->processStatisticsAndDataRecords();
+        memoryStore.updateCookieBlocking([]() { });
+        memoryStore.processStatisticsAndDataRecords();
     });
 }
 
@@ -258,17 +273,20 @@ void WebResourceLoadStatisticsStore::hasStorageAccess(const RegistrableDomain& s
     ASSERT(RunLoop::isMain());
 
     postTask([this, subFrameDomain = subFrameDomain.isolatedCopy(), topFrameDomain = topFrameDomain.isolatedCopy(), frameID, pageID, completionHandler = WTFMove(completionHandler)]() mutable {
-        if (!m_memoryStore) {
+        if (!m_statisticsStore) {
             postTaskReply([completionHandler = WTFMove(completionHandler)]() mutable {
                 completionHandler(false);
             });
             return;
         }
-        m_memoryStore->hasStorageAccess(subFrameDomain, topFrameDomain, frameID, pageID, [completionHandler = WTFMove(completionHandler)](bool hasStorageAccess) mutable {
-            postTaskReply([completionHandler = WTFMove(completionHandler), hasStorageAccess]() mutable {
-                completionHandler(hasStorageAccess);
+
+        if (m_statisticsStore) {
+            m_statisticsStore->hasStorageAccess(subFrameDomain, topFrameDomain, frameID, pageID, [completionHandler = WTFMove(completionHandler)](bool hasStorageAccess) mutable {
+                postTaskReply([completionHandler = WTFMove(completionHandler), hasStorageAccess]() mutable {
+                    completionHandler(hasStorageAccess);
+                });
             });
-        });
+        }
     });
 }
 
@@ -325,18 +343,20 @@ void WebResourceLoadStatisticsStore::requestStorageAccess(const RegistrableDomai
     }
 
     postTask([this, subFrameDomain = subFrameDomain.isolatedCopy(), topFrameDomain = topFrameDomain.isolatedCopy(), frameID, pageID, promptEnabled, completionHandler = WTFMove(completionHandler)]() mutable {
-        if (!m_memoryStore) {
+        if (!m_statisticsStore) {
             postTaskReply([completionHandler = WTFMove(completionHandler)]() mutable {
                 completionHandler(StorageAccessStatus::CannotRequestAccess);
             });
             return;
         }
 
-        m_memoryStore->requestStorageAccess(WTFMove(subFrameDomain), WTFMove(topFrameDomain), frameID.value(), pageID, promptEnabled, [completionHandler = WTFMove(completionHandler)](StorageAccessStatus status) mutable {
-            postTaskReply([completionHandler = WTFMove(completionHandler), status]() mutable {
-                completionHandler(status);
+        if (m_statisticsStore) {
+            m_statisticsStore->requestStorageAccess(WTFMove(subFrameDomain), WTFMove(topFrameDomain), frameID.value(), pageID, promptEnabled, [completionHandler = WTFMove(completionHandler)](StorageAccessStatus status) mutable {
+                postTaskReply([completionHandler = WTFMove(completionHandler), status]() mutable {
+                    completionHandler(status);
+                });
             });
-        });
+        }
     });
 }
 
@@ -348,8 +368,8 @@ void WebResourceLoadStatisticsStore::requestStorageAccessUnderOpener(Registrable
     // coming from IPC. Strings which are safe to move to other threads as long as nobody on this thread holds a reference
     // to those strings.
     postTask([this, domainInNeedOfStorageAccess = WTFMove(domainInNeedOfStorageAccess), openerPageID, openerDomain = WTFMove(openerDomain)]() mutable {
-        if (m_memoryStore)
-            m_memoryStore->requestStorageAccessUnderOpener(WTFMove(domainInNeedOfStorageAccess), openerPageID, WTFMove(openerDomain));
+        if (m_statisticsStore)
+            m_statisticsStore->requestStorageAccessUnderOpener(WTFMove(domainInNeedOfStorageAccess), openerPageID, WTFMove(openerDomain));
     });
 }
 
@@ -357,18 +377,20 @@ void WebResourceLoadStatisticsStore::grantStorageAccess(const RegistrableDomain&
 {
     ASSERT(RunLoop::isMain());
     postTask([this, subFrameDomain = subFrameDomain.isolatedCopy(), topFrameDomain = topFrameDomain.isolatedCopy(), frameID, pageID, userWasPromptedNow, completionHandler = WTFMove(completionHandler)]() mutable {
-        if (!m_memoryStore) {
+        if (!m_statisticsStore) {
             postTaskReply([completionHandler = WTFMove(completionHandler)]() mutable {
                 completionHandler(false);
             });
             return;
         }
 
-        m_memoryStore->grantStorageAccess(WTFMove(subFrameDomain), WTFMove(topFrameDomain), frameID, pageID, userWasPromptedNow, [completionHandler = WTFMove(completionHandler)](bool wasGrantedAccess) mutable {
-            postTaskReply([completionHandler = WTFMove(completionHandler), wasGrantedAccess]() mutable {
-                completionHandler(wasGrantedAccess);
+        if (m_statisticsStore) {
+            m_statisticsStore->grantStorageAccess(WTFMove(subFrameDomain), WTFMove(topFrameDomain), frameID, pageID, userWasPromptedNow, [completionHandler = WTFMove(completionHandler)](bool wasGrantedAccess) mutable {
+                postTaskReply([completionHandler = WTFMove(completionHandler), wasGrantedAccess]() mutable {
+                    completionHandler(wasGrantedAccess);
+                });
             });
-        });
+        }
     });
 }
 
@@ -397,9 +419,9 @@ void WebResourceLoadStatisticsStore::didCreateNetworkProcess()
     ASSERT(RunLoop::isMain());
 
     postTask([this] {
-        if (!m_memoryStore)
+        if (!m_statisticsStore)
             return;
-        m_memoryStore->didCreateNetworkProcess();
+        m_statisticsStore->didCreateNetworkProcess();
     });
 }
 
@@ -423,11 +445,10 @@ void WebResourceLoadStatisticsStore::performDailyTasks()
     ASSERT(RunLoop::isMain());
 
     postTask([this] {
-        if (!m_memoryStore)
-            return;
-
-        m_memoryStore->includeTodayAsOperatingDateIfNecessary();
-        m_memoryStore->calculateAndSubmitTelemetry();
+        if (m_statisticsStore) {
+            m_statisticsStore->includeTodayAsOperatingDateIfNecessary();
+            m_statisticsStore->calculateAndSubmitTelemetry();
+        }
     });
 }
 
@@ -436,8 +457,8 @@ void WebResourceLoadStatisticsStore::submitTelemetry(CompletionHandler<void()>&&
     ASSERT(RunLoop::isMain());
 
     postTask([this, completionHandler = WTFMove(completionHandler)]() mutable  {
-        if (m_memoryStore)
-            WebResourceLoadStatisticsTelemetry::calculateAndSubmit(*m_memoryStore);
+        if (m_statisticsStore && is<ResourceLoadStatisticsMemoryStore>(*m_statisticsStore))
+            WebResourceLoadStatisticsTelemetry::calculateAndSubmit(downcast<ResourceLoadStatisticsMemoryStore>(*m_statisticsStore));
 
         postTaskReply(WTFMove(completionHandler));
     });
@@ -476,27 +497,26 @@ void WebResourceLoadStatisticsStore::logFrameNavigation(const WebFrameProxy& fra
 void WebResourceLoadStatisticsStore::logFrameNavigation(const RegistrableDomain& targetDomain, const RegistrableDomain& topFrameDomain, const RegistrableDomain& sourceDomain, bool isRedirect, bool isMainFrame)
 {
     postTask([this, targetDomain = targetDomain.isolatedCopy(), topFrameDomain = topFrameDomain.isolatedCopy(), sourceDomain = sourceDomain.isolatedCopy(), isRedirect, isMainFrame] {
-
-        if (m_memoryStore)
-            m_memoryStore->logFrameNavigation(targetDomain, topFrameDomain, sourceDomain, isRedirect, isMainFrame);
+        if (m_statisticsStore)
+            m_statisticsStore->logFrameNavigation(targetDomain, topFrameDomain, sourceDomain, isRedirect, isMainFrame);
     });
 }
 
 void WebResourceLoadStatisticsStore::logWebSocketLoading(const RegistrableDomain& targetDomain, const RegistrableDomain& topFrameDomain, WallTime lastSeen, CompletionHandler<void()>&& completionHandler)
 {
     postTask([this, targetDomain = targetDomain.isolatedCopy(), topFrameDomain = topFrameDomain.isolatedCopy(), lastSeen, completionHandler = WTFMove(completionHandler)]() mutable {
-        if (m_memoryStore)
-            m_memoryStore->logSubresourceLoading(targetDomain, topFrameDomain, lastSeen);
+        if (m_statisticsStore)
+            m_statisticsStore->logSubresourceLoading(targetDomain, topFrameDomain, lastSeen);
 
         postTaskReply(WTFMove(completionHandler));
     });
 }
 
-void WebResourceLoadStatisticsStore::logSubresourceLoading(const RegistrableDomain& targetDomain, const RegistrableDomain& topFrameDomain, WallTime lastSeen, CompletionHandler<void()>&& completionHandler)
+void WebResourceLoadStatisticsStore::logSubresourceLoading(const SubResourceDomain& targetDomain, const TopFrameDomain& topFrameDomain, WallTime lastSeen, CompletionHandler<void()>&& completionHandler)
 {
     postTask([this, targetDomain = targetDomain.isolatedCopy(), topFrameDomain = topFrameDomain.isolatedCopy(), lastSeen, completionHandler = WTFMove(completionHandler)]() mutable {
-        if (m_memoryStore)
-            m_memoryStore->logSubresourceLoading(targetDomain, topFrameDomain, lastSeen);
+        if (m_statisticsStore)
+            m_statisticsStore->logSubresourceLoading(targetDomain, topFrameDomain, lastSeen);
 
         postTaskReply(WTFMove(completionHandler));
     });
@@ -505,9 +525,8 @@ void WebResourceLoadStatisticsStore::logSubresourceLoading(const RegistrableDoma
 void WebResourceLoadStatisticsStore::logSubresourceRedirect(const RegistrableDomain& sourceDomain, const RegistrableDomain& targetDomain, CompletionHandler<void()>&& completionHandler)
 {
     postTask([this, sourceDomain = sourceDomain.isolatedCopy(), targetDomain = targetDomain.isolatedCopy(), completionHandler = WTFMove(completionHandler)]() mutable {
-        if (m_memoryStore)
-            m_memoryStore->logSubresourceRedirect(sourceDomain, targetDomain);
-
+        if (m_statisticsStore)
+            m_statisticsStore->logSubresourceRedirect(sourceDomain, targetDomain);
         postTaskReply(WTFMove(completionHandler));
     });
 }
@@ -517,8 +536,8 @@ void WebResourceLoadStatisticsStore::logUserInteraction(const RegistrableDomain&
     ASSERT(RunLoop::isMain());
 
     postTask([this, domain = domain.isolatedCopy(), completionHandler = WTFMove(completionHandler)]() mutable {
-        if (m_memoryStore)
-            m_memoryStore->logUserInteraction(domain);
+        if (m_statisticsStore)
+            m_statisticsStore->logUserInteraction(domain);
         postTaskReply(WTFMove(completionHandler));
     });
 }
@@ -528,8 +547,8 @@ void WebResourceLoadStatisticsStore::clearUserInteraction(const RegistrableDomai
     ASSERT(RunLoop::isMain());
     
     postTask([this, domain = domain.isolatedCopy(), completionHandler = WTFMove(completionHandler)]() mutable {
-        if (m_memoryStore)
-            m_memoryStore->clearUserInteraction(domain);
+        if (m_statisticsStore)
+            m_statisticsStore->clearUserInteraction(domain);
         postTaskReply(WTFMove(completionHandler));
     });
 }
@@ -537,7 +556,7 @@ void WebResourceLoadStatisticsStore::clearUserInteraction(const RegistrableDomai
 void WebResourceLoadStatisticsStore::hasHadUserInteraction(const RegistrableDomain& domain, CompletionHandler<void(bool)>&& completionHandler)
 {
     postTask([this, domain, completionHandler = WTFMove(completionHandler)]() mutable {
-        bool hadUserInteraction = m_memoryStore ? m_memoryStore->hasHadUserInteraction(domain) : false;
+        bool hadUserInteraction = m_statisticsStore ? m_statisticsStore->hasHadUserInteraction(domain) : false;
         postTaskReply([hadUserInteraction, completionHandler = WTFMove(completionHandler)]() mutable {
             completionHandler(hadUserInteraction);
         });
@@ -549,8 +568,8 @@ void WebResourceLoadStatisticsStore::setLastSeen(const RegistrableDomain& domain
     ASSERT(RunLoop::isMain());
     
     postTask([this, domain = domain.isolatedCopy(), seconds, completionHandler = WTFMove(completionHandler)]() mutable {
-        if (m_memoryStore)
-            m_memoryStore->setLastSeen(domain, seconds);
+        if (m_statisticsStore)
+            m_statisticsStore->setLastSeen(domain, seconds);
         postTaskReply(WTFMove(completionHandler));
     });
 }
@@ -560,8 +579,8 @@ void WebResourceLoadStatisticsStore::setPrevalentResource(const RegistrableDomai
     ASSERT(RunLoop::isMain());
 
     postTask([this, domain = domain.isolatedCopy(), completionHandler = WTFMove(completionHandler)]() mutable {
-        if (m_memoryStore)
-            m_memoryStore->setPrevalentResource(domain);
+        if (m_statisticsStore)
+            m_statisticsStore->setPrevalentResource(domain);
         postTaskReply(WTFMove(completionHandler));
     });
 }
@@ -571,8 +590,8 @@ void WebResourceLoadStatisticsStore::setVeryPrevalentResource(const RegistrableD
     ASSERT(RunLoop::isMain());
 
     postTask([this, domain = domain.isolatedCopy(), completionHandler = WTFMove(completionHandler)]() mutable {
-        if (m_memoryStore)
-            m_memoryStore->setVeryPrevalentResource(domain);
+        if (m_statisticsStore)
+            m_statisticsStore->setVeryPrevalentResource(domain);
         postTaskReply(WTFMove(completionHandler));
     });
 }
@@ -582,7 +601,7 @@ void WebResourceLoadStatisticsStore::dumpResourceLoadStatistics(CompletionHandle
     ASSERT(RunLoop::isMain());
 
     postTask([this, completionHandler = WTFMove(completionHandler)]() mutable {
-        String result = m_memoryStore ? m_memoryStore->dumpResourceLoadStatistics() : emptyString();
+        String result = m_statisticsStore ? m_statisticsStore->dumpResourceLoadStatistics() : emptyString();
         postTaskReply([result = result.isolatedCopy(), completionHandler = WTFMove(completionHandler)]() mutable {
             completionHandler(result);
         });
@@ -594,7 +613,7 @@ void WebResourceLoadStatisticsStore::isPrevalentResource(const RegistrableDomain
     ASSERT(RunLoop::isMain());
     
     postTask([this, domain = domain.isolatedCopy(), completionHandler = WTFMove(completionHandler)]() mutable {
-        bool isPrevalentResource = m_memoryStore ? m_memoryStore->isPrevalentResource(domain) : false;
+        bool isPrevalentResource = m_statisticsStore ? m_statisticsStore->isPrevalentResource(domain) : false;
         postTaskReply([isPrevalentResource, completionHandler = WTFMove(completionHandler)]() mutable {
             completionHandler(isPrevalentResource);
         });
@@ -606,7 +625,7 @@ void WebResourceLoadStatisticsStore::isVeryPrevalentResource(const RegistrableDo
     ASSERT(RunLoop::isMain());
     
     postTask([this, domain = domain.isolatedCopy(), completionHandler = WTFMove(completionHandler)]() mutable {
-        bool isVeryPrevalentResource = m_memoryStore ? m_memoryStore->isVeryPrevalentResource(domain) : false;
+        bool isVeryPrevalentResource = m_statisticsStore ? m_statisticsStore->isVeryPrevalentResource(domain) : false;
         postTaskReply([isVeryPrevalentResource, completionHandler = WTFMove(completionHandler)]() mutable {
             completionHandler(isVeryPrevalentResource);
         });
@@ -618,7 +637,8 @@ void WebResourceLoadStatisticsStore::isRegisteredAsSubresourceUnder(const Regist
     ASSERT(RunLoop::isMain());
 
     postTask([this, subresourceDomain = subresourceDomain.isolatedCopy(), topFrameDomain = topFrameDomain.isolatedCopy(), completionHandler = WTFMove(completionHandler)]() mutable {
-        bool isRegisteredAsSubresourceUnder = m_memoryStore ? m_memoryStore->isRegisteredAsSubresourceUnder(subresourceDomain, topFrameDomain) : false;
+        bool isRegisteredAsSubresourceUnder = m_statisticsStore ? m_statisticsStore->isRegisteredAsSubresourceUnder(subresourceDomain, topFrameDomain)
+            : false;
         postTaskReply([isRegisteredAsSubresourceUnder, completionHandler = WTFMove(completionHandler)]() mutable {
             completionHandler(isRegisteredAsSubresourceUnder);
         });
@@ -630,7 +650,8 @@ void WebResourceLoadStatisticsStore::isRegisteredAsSubFrameUnder(const Registrab
     ASSERT(RunLoop::isMain());
 
     postTask([this, subFrameDomain = subFrameDomain.isolatedCopy(), topFrameDomain = topFrameDomain.isolatedCopy(), completionHandler = WTFMove(completionHandler)]() mutable {
-        bool isRegisteredAsSubFrameUnder = m_memoryStore ? m_memoryStore->isRegisteredAsSubFrameUnder(subFrameDomain, topFrameDomain) : false;
+        bool isRegisteredAsSubFrameUnder = m_statisticsStore ? m_statisticsStore->isRegisteredAsSubFrameUnder(subFrameDomain, topFrameDomain)
+            : false;
         postTaskReply([isRegisteredAsSubFrameUnder, completionHandler = WTFMove(completionHandler)]() mutable {
             completionHandler(isRegisteredAsSubFrameUnder);
         });
@@ -642,7 +663,8 @@ void WebResourceLoadStatisticsStore::isRegisteredAsRedirectingTo(const Registrab
     ASSERT(RunLoop::isMain());
 
     postTask([this, domainRedirectedFrom = domainRedirectedFrom.isolatedCopy(), domainRedirectedTo = domainRedirectedTo.isolatedCopy(), completionHandler = WTFMove(completionHandler)]() mutable {
-        bool isRegisteredAsRedirectingTo = m_memoryStore ? m_memoryStore->isRegisteredAsRedirectingTo(domainRedirectedFrom, domainRedirectedTo) : false;
+        bool isRegisteredAsRedirectingTo = m_statisticsStore ? m_statisticsStore->isRegisteredAsRedirectingTo(domainRedirectedFrom, domainRedirectedTo)
+            : false;
         postTaskReply([isRegisteredAsRedirectingTo, completionHandler = WTFMove(completionHandler)]() mutable {
             completionHandler(isRegisteredAsRedirectingTo);
         });
@@ -654,8 +676,8 @@ void WebResourceLoadStatisticsStore::clearPrevalentResource(const RegistrableDom
     ASSERT(RunLoop::isMain());
     
     postTask([this, domain = domain.isolatedCopy(), completionHandler = WTFMove(completionHandler)]() mutable {
-        if (m_memoryStore)
-            m_memoryStore->clearPrevalentResource(domain);
+        if (m_statisticsStore)
+            m_statisticsStore->clearPrevalentResource(domain);
         postTaskReply(WTFMove(completionHandler));
     });
 }
@@ -665,8 +687,8 @@ void WebResourceLoadStatisticsStore::setGrandfathered(const RegistrableDomain& d
     ASSERT(RunLoop::isMain());
 
     postTask([this, domain = domain.isolatedCopy(), value, completionHandler = WTFMove(completionHandler)]() mutable {
-        if (m_memoryStore)
-            m_memoryStore->setGrandfathered(domain, value);
+        if (m_statisticsStore)
+            m_statisticsStore->setGrandfathered(domain, value);
         postTaskReply(WTFMove(completionHandler));
     });
 }
@@ -676,7 +698,8 @@ void WebResourceLoadStatisticsStore::isGrandfathered(const RegistrableDomain& do
     ASSERT(RunLoop::isMain());
 
     postTask([this, completionHandler = WTFMove(completionHandler), domain = domain.isolatedCopy()]() mutable {
-        bool isGrandFathered = m_memoryStore ? m_memoryStore->isGrandfathered(domain) : false;
+        bool isGrandFathered = m_statisticsStore ? m_statisticsStore->isGrandfathered(domain)
+            : false;
         postTaskReply([isGrandFathered, completionHandler = WTFMove(completionHandler)]() mutable {
             completionHandler(isGrandFathered);
         });
@@ -688,8 +711,8 @@ void WebResourceLoadStatisticsStore::setSubframeUnderTopFrameDomain(const Regist
     ASSERT(RunLoop::isMain());
     
     postTask([this, completionHandler = WTFMove(completionHandler), subFrameDomain = subFrameDomain.isolatedCopy(), topFrameDomain = topFrameDomain.isolatedCopy()]() mutable {
-        if (m_memoryStore)
-            m_memoryStore->setSubframeUnderTopFrameDomain(subFrameDomain, topFrameDomain);
+        if (m_statisticsStore)
+            m_statisticsStore->setSubframeUnderTopFrameDomain(subFrameDomain, topFrameDomain);
         postTaskReply(WTFMove(completionHandler));
     });
 }
@@ -699,8 +722,8 @@ void WebResourceLoadStatisticsStore::setSubresourceUnderTopFrameDomain(const Reg
     ASSERT(RunLoop::isMain());
     
     postTask([this, completionHandler = WTFMove(completionHandler), subresourceDomain = subresourceDomain.isolatedCopy(), topFrameDomain = topFrameDomain.isolatedCopy()]() mutable {
-        if (m_memoryStore)
-            m_memoryStore->setSubresourceUnderTopFrameDomain(subresourceDomain, topFrameDomain);
+        if (m_statisticsStore)
+            m_statisticsStore->setSubresourceUnderTopFrameDomain(subresourceDomain, topFrameDomain);
         postTaskReply(WTFMove(completionHandler));
     });
 }
@@ -710,8 +733,8 @@ void WebResourceLoadStatisticsStore::setSubresourceUniqueRedirectTo(const Regist
     ASSERT(RunLoop::isMain());
     
     postTask([this, completionHandler = WTFMove(completionHandler), subresourceDomain = subresourceDomain.isolatedCopy(), domainRedirectedTo = domainRedirectedTo.isolatedCopy()]() mutable {
-        if (m_memoryStore)
-            m_memoryStore->setSubresourceUniqueRedirectTo(subresourceDomain, domainRedirectedTo);
+        if (m_statisticsStore)
+            m_statisticsStore->setSubresourceUniqueRedirectTo(subresourceDomain, domainRedirectedTo);
         postTaskReply(WTFMove(completionHandler));
     });
 }
@@ -721,8 +744,8 @@ void WebResourceLoadStatisticsStore::setSubresourceUniqueRedirectFrom(const Regi
     ASSERT(RunLoop::isMain());
     
     postTask([this, completionHandler = WTFMove(completionHandler), subresourceDomain = subresourceDomain.isolatedCopy(), domainRedirectedFrom = domainRedirectedFrom.isolatedCopy()]() mutable {
-        if (m_memoryStore)
-            m_memoryStore->setSubresourceUniqueRedirectFrom(subresourceDomain, domainRedirectedFrom);
+        if (m_statisticsStore)
+            m_statisticsStore->setSubresourceUniqueRedirectFrom(subresourceDomain, domainRedirectedFrom);
         postTaskReply(WTFMove(completionHandler));
     });
 }
@@ -732,8 +755,8 @@ void WebResourceLoadStatisticsStore::setTopFrameUniqueRedirectTo(const Registrab
     ASSERT(RunLoop::isMain());
     
     postTask([this, completionHandler = WTFMove(completionHandler), topFrameDomain = topFrameDomain.isolatedCopy(), domainRedirectedTo = domainRedirectedTo.isolatedCopy()]() mutable {
-        if (m_memoryStore)
-            m_memoryStore->setTopFrameUniqueRedirectTo(topFrameDomain, domainRedirectedTo);
+        if (m_statisticsStore)
+            m_statisticsStore->setTopFrameUniqueRedirectTo(topFrameDomain, domainRedirectedTo);
         postTaskReply(WTFMove(completionHandler));
     });
 }
@@ -743,8 +766,8 @@ void WebResourceLoadStatisticsStore::setTopFrameUniqueRedirectFrom(const Registr
     ASSERT(RunLoop::isMain());
     
     postTask([this, completionHandler = WTFMove(completionHandler), topFrameDomain = topFrameDomain.isolatedCopy(), domainRedirectedFrom = domainRedirectedFrom.isolatedCopy()]() mutable {
-        if (m_memoryStore)
-            m_memoryStore->setTopFrameUniqueRedirectFrom(topFrameDomain, domainRedirectedFrom);
+        if (m_statisticsStore)
+            m_statisticsStore->setTopFrameUniqueRedirectFrom(topFrameDomain, domainRedirectedFrom);
         postTaskReply(WTFMove(completionHandler));
     });
 }
@@ -755,13 +778,17 @@ void WebResourceLoadStatisticsStore::scheduleCookieBlockingUpdate(CompletionHand
     ASSERT(RunLoop::isMain());
 
     postTask([this, completionHandler = WTFMove(completionHandler)]() mutable {
-        if (!m_memoryStore) {
+        if (!m_statisticsStore) {
             postTaskReply(WTFMove(completionHandler));
             return;
         }
-        m_memoryStore->updateCookieBlocking([completionHandler = WTFMove(completionHandler)]() mutable {
+
+        auto callbackAggregator = CallbackAggregator::create([completionHandler = WTFMove(completionHandler)] () mutable {
             postTaskReply(WTFMove(completionHandler));
         });
+        
+        if (m_statisticsStore)
+            m_statisticsStore->updateCookieBlocking([callbackAggregator = callbackAggregator.copyRef()]() { });
     });
 }
 
@@ -770,14 +797,17 @@ void WebResourceLoadStatisticsStore::scheduleCookieBlockingUpdateForDomains(cons
     // Helper function used by testing system. Should only be called from the main thread.
     ASSERT(RunLoop::isMain());
     postTask([this, domainsToBlock = crossThreadCopy(domainsToBlock), completionHandler = WTFMove(completionHandler)]() mutable {
-        if (!m_memoryStore) {
+        if (!m_statisticsStore) {
             postTaskReply(WTFMove(completionHandler));
             return;
         }
 
-        m_memoryStore->updateCookieBlockingForDomains(domainsToBlock, [completionHandler = WTFMove(completionHandler)]() mutable {
+        auto callbackAggregator = CallbackAggregator::create([completionHandler = WTFMove(completionHandler)] () mutable {
             postTaskReply(WTFMove(completionHandler));
         });
+
+        if (m_statisticsStore)
+            m_statisticsStore->updateCookieBlockingForDomains(domainsToBlock, [callbackAggregator = callbackAggregator.copyRef()]() { });
     });
 }
 
@@ -786,14 +816,17 @@ void WebResourceLoadStatisticsStore::scheduleClearBlockingStateForDomains(const
     // Helper function used by testing system. Should only be called from the main thread.
     ASSERT(RunLoop::isMain());
     postTask([this, domains = crossThreadCopy(domains), completionHandler = WTFMove(completionHandler)]() mutable {
-        if (!m_memoryStore) {
+        if (!m_statisticsStore) {
             postTaskReply(WTFMove(completionHandler));
             return;
         }
 
-        m_memoryStore->clearBlockingStateForDomains(domains, [completionHandler = WTFMove(completionHandler)]() mutable {
+        auto callbackAggregator = CallbackAggregator::create([completionHandler = WTFMove(completionHandler)] () mutable {
             postTaskReply(WTFMove(completionHandler));
         });
+
+        if (m_statisticsStore)
+            m_statisticsStore->clearBlockingStateForDomains(domains, [callbackAggregator = callbackAggregator.copyRef()]() { });
     });
 }
 
@@ -804,22 +837,27 @@ void WebResourceLoadStatisticsStore::scheduleClearInMemoryAndPersistent(ShouldGr
         if (m_persistentStorage)
             m_persistentStorage->clear();
 
-        CompletionHandlerCallingScope completionHandlerCaller([completionHandler = WTFMove(completionHandler)]() mutable {
+        if (!m_statisticsStore) {
+            if (shouldGrandfather == ShouldGrandfatherStatistics::Yes)
+                RELEASE_LOG(ResourceLoadStatistics, "WebResourceLoadStatisticsStore::scheduleClearInMemoryAndPersistent Before being cleared, m_statisticsStore is null when trying to grandfather data.");
+
+            postTaskReply(WTFMove(completionHandler));
+            return;
+        }
+
+        auto callbackAggregator = CallbackAggregator::create([completionHandler = WTFMove(completionHandler)] () mutable {
             postTaskReply(WTFMove(completionHandler));
         });
 
-        if (m_memoryStore) {
-            m_memoryStore->clear([this, protectedThis = protectedThis.copyRef(), shouldGrandfather, completionHandlerCaller = WTFMove(completionHandlerCaller)] () mutable {
+        if (m_statisticsStore) {
+            m_statisticsStore->clear([this, protectedThis = protectedThis.copyRef(), shouldGrandfather, callbackAggregator = callbackAggregator.copyRef()] () mutable {
                 if (shouldGrandfather == ShouldGrandfatherStatistics::Yes) {
-                    if (m_memoryStore)
-                        m_memoryStore->grandfatherExistingWebsiteData(completionHandlerCaller.release());
+                    if (m_statisticsStore)
+                        m_statisticsStore->grandfatherExistingWebsiteData([callbackAggregator = WTFMove(callbackAggregator)]() mutable { });
                     else
-                        RELEASE_LOG(ResourceLoadStatistics, "WebResourceLoadStatisticsStore::scheduleClearInMemoryAndPersistent After being cleared, m_memoryStore is null when trying to grandfather data.");
+                        RELEASE_LOG(ResourceLoadStatistics, "WebResourceLoadStatisticsStore::scheduleClearInMemoryAndPersistent After being cleared, m_statisticsStore is null when trying to grandfather data.");
                 }
             });
-        } else {
-            if (shouldGrandfather == ShouldGrandfatherStatistics::Yes)
-                RELEASE_LOG(ResourceLoadStatistics, "WebResourceLoadStatisticsStore::scheduleClearInMemoryAndPersistent Before being cleared, m_memoryStore is null when trying to grandfather data.");
         }
     });
 }
@@ -837,8 +875,8 @@ void WebResourceLoadStatisticsStore::setTimeToLiveUserInteraction(Seconds second
 {
     ASSERT(RunLoop::isMain());
     postTask([this, seconds, completionHandler = WTFMove(completionHandler)]() mutable {
-        if (m_memoryStore)
-            m_memoryStore->setTimeToLiveUserInteraction(seconds);
+        if (m_statisticsStore)
+            m_statisticsStore->setTimeToLiveUserInteraction(seconds);
         postTaskReply(WTFMove(completionHandler));
     });
 }
@@ -847,8 +885,8 @@ void WebResourceLoadStatisticsStore::setMinimumTimeBetweenDataRecordsRemoval(Sec
 {
     ASSERT(RunLoop::isMain());
     postTask([this, seconds, completionHandler = WTFMove(completionHandler)]() mutable  {
-        if (m_memoryStore)
-            m_memoryStore->setMinimumTimeBetweenDataRecordsRemoval(seconds);
+        if (m_statisticsStore)
+            m_statisticsStore->setMinimumTimeBetweenDataRecordsRemoval(seconds);
 
         postTaskReply(WTFMove(completionHandler));
     });
@@ -858,8 +896,8 @@ void WebResourceLoadStatisticsStore::setGrandfatheringTime(Seconds seconds, Comp
 {
     ASSERT(RunLoop::isMain());
     postTask([this, seconds, completionHandler = WTFMove(completionHandler)]() mutable  {
-        if (m_memoryStore)
-            m_memoryStore->setGrandfatheringTime(seconds);
+        if (m_statisticsStore)
+            m_statisticsStore->setGrandfatheringTime(seconds);
 
         postTaskReply(WTFMove(completionHandler));
     });
@@ -903,8 +941,8 @@ void WebResourceLoadStatisticsStore::setMaxStatisticsEntries(size_t maximumEntry
 {
     ASSERT(RunLoop::isMain());
     postTask([this, maximumEntryCount, completionHandler = WTFMove(completionHandler)]() mutable  {
-        if (m_memoryStore)
-            m_memoryStore->setMaxStatisticsEntries(maximumEntryCount);
+        if (m_statisticsStore)
+            m_statisticsStore->setMaxStatisticsEntries(maximumEntryCount);
 
         postTaskReply(WTFMove(completionHandler));
     });
@@ -915,8 +953,8 @@ void WebResourceLoadStatisticsStore::setPruneEntriesDownTo(size_t pruneTargetCou
     ASSERT(RunLoop::isMain());
 
     postTask([this, pruneTargetCount, completionHandler = WTFMove(completionHandler)]() mutable  {
-        if (m_memoryStore)
-            m_memoryStore->setPruneEntriesDownTo(pruneTargetCount);
+        if (m_statisticsStore)
+            m_statisticsStore->setPruneEntriesDownTo(pruneTargetCount);
 
         postTaskReply(WTFMove(completionHandler));
     });
@@ -927,8 +965,8 @@ void WebResourceLoadStatisticsStore::resetParametersToDefaultValues(CompletionHa
     ASSERT(RunLoop::isMain());
 
     postTask([this, completionHandler = WTFMove(completionHandler)]() mutable {
-        if (m_memoryStore)
-            m_memoryStore->resetParametersToDefaultValues();
+        if (m_statisticsStore)
+            m_statisticsStore->resetParametersToDefaultValues();
 
         postTaskReply(WTFMove(completionHandler));
     });
index 0634385..0a1a303 100644 (file)
@@ -52,7 +52,7 @@ enum class ShouldSample : bool;
 namespace WebKit {
 
 class NetworkSession;
-class ResourceLoadStatisticsMemoryStore;
+class ResourceLoadStatisticsStore;
 class ResourceLoadStatisticsPersistentStorage;
 class WebFrameProxy;
 class WebProcessProxy;
@@ -189,7 +189,7 @@ private:
 
     WeakPtr<NetworkSession> m_networkSession;
     Ref<WorkQueue> m_statisticsQueue;
-    std::unique_ptr<ResourceLoadStatisticsMemoryStore> m_memoryStore;
+    std::unique_ptr<ResourceLoadStatisticsStore> m_statisticsStore;
     std::unique_ptr<ResourceLoadStatisticsPersistentStorage> m_persistentStorage;
 
     RunLoop::Timer<WebResourceLoadStatisticsStore> m_dailyTasksTimer;
index 55cbab8..facdfd5 100644 (file)
@@ -72,6 +72,7 @@
 #include <WebCore/NetworkStorageSession.h>
 #include <WebCore/ResourceRequest.h>
 #include <WebCore/RuntimeApplicationChecks.h>
+#include <WebCore/RuntimeEnabledFeatures.h>
 #include <WebCore/SchemeRegistry.h>
 #include <WebCore/SecurityOriginData.h>
 #include <wtf/Algorithms.h>
@@ -309,6 +310,8 @@ void NetworkProcess::initializeNetworkProcess(NetworkProcessCreationParameters&&
     if (parameters.shouldUseTestingNetworkSession)
         switchToNewTestingSession();
 
+    WebCore::RuntimeEnabledFeatures::sharedFeatures().setIsITPDatabaseEnabled(parameters.shouldEnableITPDatabase);
+
     SandboxExtension::consumePermanently(parameters.defaultDataStoreParameters.networkSessionParameters.resourceLoadStatisticsDirectoryExtensionHandle);
 
     auto sessionID = parameters.defaultDataStoreParameters.networkSessionParameters.sessionID;
index c6df08e..087150a 100644 (file)
@@ -93,6 +93,7 @@ void NetworkProcessCreationParameters::encode(IPC::Encoder& encoder) const
 #if ENABLE(SERVICE_WORKER)
     encoder << serviceWorkerRegistrationDirectory << serviceWorkerRegistrationDirectoryExtensionHandle << urlSchemesServiceWorkersCanHandle << shouldDisableServiceWorkerProcessTerminationDelay;
 #endif
+    encoder << shouldEnableITPDatabase;
 }
 
 bool NetworkProcessCreationParameters::decode(IPC::Decoder& decoder, NetworkProcessCreationParameters& result)
@@ -217,6 +218,9 @@ bool NetworkProcessCreationParameters::decode(IPC::Decoder& decoder, NetworkProc
         return false;
 #endif
 
+    if (!decoder.decode(result.shouldEnableITPDatabase))
+        return false;
+
     return true;
 }
 
index 2fd3be9..fc900c2 100644 (file)
@@ -111,6 +111,7 @@ struct NetworkProcessCreationParameters {
     Vector<String> urlSchemesServiceWorkersCanHandle;
     bool shouldDisableServiceWorkerProcessTerminationDelay { false };
 #endif
+    bool shouldEnableITPDatabase { false };
 };
 
 } // namespace WebKit
index f1ea792..b4492a4 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2017 Apple Inc. All rights reserved.
+ * Copyright (C) 2017-2019 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -47,7 +47,12 @@ ResourceLoadPrevalence ResourceLoadStatisticsClassifier::calculateResourcePreval
     auto subresourceUniqueRedirectsToCount = resourceStatistic.subresourceUniqueRedirectsTo.size();
     auto subframeUnderTopFrameDomainsCount = resourceStatistic.subframeUnderTopFrameDomains.size();
     auto topFrameUniqueRedirectsToCount = resourceStatistic.topFrameUniqueRedirectsTo.size();
-    
+
+    return calculateResourcePrevalence(subresourceUnderTopFrameDomainsCount, subresourceUniqueRedirectsToCount, subframeUnderTopFrameDomainsCount, topFrameUniqueRedirectsToCount, currentPrevalence);
+}
+
+ResourceLoadPrevalence ResourceLoadStatisticsClassifier::calculateResourcePrevalence(unsigned subresourceUnderTopFrameDomainsCount, unsigned subresourceUniqueRedirectsToCount, unsigned subframeUnderTopFrameDomainsCount, unsigned topFrameUniqueRedirectsToCount, ResourceLoadPrevalence currentPrevalence)
+{
     if (!subresourceUnderTopFrameDomainsCount
         && !subresourceUniqueRedirectsToCount
         && !subframeUnderTopFrameDomainsCount
index acf1f38..58f565a 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2017 Apple Inc. All rights reserved.
+ * Copyright (C) 2017-2019 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -42,10 +42,11 @@ public:
     ResourceLoadStatisticsClassifier() = default;
     virtual ~ResourceLoadStatisticsClassifier() = default;
     ResourceLoadPrevalence calculateResourcePrevalence(const WebCore::ResourceLoadStatistics& resourceStatistic, ResourceLoadPrevalence currentPrevalence);
+    ResourceLoadPrevalence calculateResourcePrevalence(unsigned subresourceUnderTopFrameDomainsCount, unsigned subresourceUniqueRedirectsToCount, unsigned subframeUnderTopFrameOriginsCount, unsigned topFrameUniqueRedirectsToCount, ResourceLoadPrevalence currentPrevalence);
 protected:
-    virtual bool classify(unsigned subresourceUnderTopFrameOriginsCount, unsigned subresourceUniqueRedirectsToCount, unsigned subframeUnderTopFrameOriginsCount)
+    virtual bool classify(unsigned subresourceUnderTopFrameDomainsCount, unsigned subresourceUniqueRedirectsToCount, unsigned subframeUnderTopFrameOriginsCount)
     {
-        return classifyWithVectorThreshold(subresourceUnderTopFrameOriginsCount, subresourceUniqueRedirectsToCount, subframeUnderTopFrameOriginsCount);
+        return classifyWithVectorThreshold(subresourceUnderTopFrameDomainsCount, subresourceUniqueRedirectsToCount, subframeUnderTopFrameOriginsCount);
     }
     bool classifyWithVectorThreshold(unsigned, unsigned, unsigned);
 };
index acfbe3c..e1f5304 100644 (file)
 
 namespace WebKit {
 
-bool ResourceLoadStatisticsClassifierCocoa::classify(unsigned subresourceUnderTopFrameOriginsCount, unsigned subresourceUniqueRedirectsToCount, unsigned subframeUnderTopFrameOriginsCount)
+bool ResourceLoadStatisticsClassifierCocoa::classify(unsigned subresourceUnderTopFrameDomainsCount, unsigned subresourceUniqueRedirectsToCount, unsigned subframeUnderTopFrameOriginsCount)
 {
     if (!canUseCorePrediction())
-        return classifyWithVectorThreshold(subresourceUnderTopFrameOriginsCount, subresourceUniqueRedirectsToCount, subframeUnderTopFrameOriginsCount);
+        return classifyWithVectorThreshold(subresourceUnderTopFrameDomainsCount, subresourceUniqueRedirectsToCount, subframeUnderTopFrameOriginsCount);
 
     Vector<svm_node> features;
 
-    if (subresourceUnderTopFrameOriginsCount)
-        features.append({1, static_cast<double>(subresourceUnderTopFrameOriginsCount)});
+    if (subresourceUnderTopFrameDomainsCount)
+        features.append({1, static_cast<double>(subresourceUnderTopFrameDomainsCount)});
     if (subresourceUniqueRedirectsToCount)
         features.append({2, static_cast<double>(subresourceUniqueRedirectsToCount)});
     if (subframeUnderTopFrameOriginsCount)
index f558432..e2ae6a8 100644 (file)
@@ -1379,6 +1379,14 @@ ItpDebugModeEnabled:
   humanReadableDescription: "Intelligent Tracking Prevention Debug Mode"
   webcoreBinding: RuntimeEnabledFeatures
 
+IsITPDatabaseEnabled:
+  type: bool
+  defaultValue: false
+  humanReadableName: "ITP Database Backend"
+  humanReadableDescription: "Enable Intelligent Tracking Prevention Database Backend"
+  webcoreBinding: RuntimeEnabledFeatures
+  category: internal
+
 ServiceWorkersEnabled:
   type: bool
   defaultValue: DEFAULT_SERVICE_WORKERS_ENABLED
index 7407cc2..2efb8e5 100644 (file)
@@ -42,8 +42,10 @@ NetworkProcess/NetworkSocketStream.cpp
 NetworkProcess/PingLoad.cpp
 NetworkProcess/PreconnectTask.cpp
 
+NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp
 NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp
 NetworkProcess/Classifier/ResourceLoadStatisticsPersistentStorage.cpp
+NetworkProcess/Classifier/ResourceLoadStatisticsStore.cpp
 
 NetworkProcess/Cookies/WebCookieManager.cpp
 
index 929c50e..f97d740 100644 (file)
@@ -29,6 +29,7 @@ NetworkProcess/cocoa/NetworkDataTaskCocoa.mm
 NetworkProcess/cocoa/NetworkProcessCocoa.mm
 NetworkProcess/cocoa/NetworkSessionCocoa.mm
 
+NetworkProcess/Classifier/ResourceLoadStatisticsStoreCocoa.mm
 NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp
 NetworkProcess/Classifier/WebResourceLoadStatisticsTelemetry.cpp
 
@@ -329,7 +330,6 @@ UIProcess/Cocoa/MediaCaptureUtilities.mm
 UIProcess/Cocoa/NavigationState.mm
 UIProcess/Cocoa/PageClientImplCocoa.mm
 UIProcess/Cocoa/PlaybackSessionManagerProxy.mm
-UIProcess/Cocoa/ResourceLoadStatisticsMemoryStoreCocoa.mm
 UIProcess/Cocoa/SafeBrowsingWarningCocoa.mm
 UIProcess/Cocoa/SessionStateCoding.mm
 UIProcess/Cocoa/SystemPreviewControllerCocoa.mm
index 98746c3..b207cd4 100644 (file)
@@ -296,6 +296,8 @@ void WebProcessPool::platformInitializeNetworkProcess(NetworkProcessCreationPara
 #if ENABLE(PROXIMITY_NETWORKING)
     parameters.wirelessContextIdentifier = m_configuration->wirelessContextIdentifier();
 #endif
+
+    parameters.shouldEnableITPDatabase = [defaults boolForKey:[NSString stringWithFormat:@"InternalDebug%@", WebPreferencesKey::isITPDatabaseEnabledKey().createCFString().get()]];
 }
 
 void WebProcessPool::platformInvalidateContext()
index 8883f73..16bd5cd 100644 (file)
@@ -28,6 +28,7 @@
 #include "MessageReceiver.h"
 #include "SandboxExtension.h"
 #include "WebFramePolicyListenerProxy.h"
+#include "WebPageProxyMessages.h"
 #include "WebsitePoliciesData.h"
 #include <wtf/WeakPtr.h>
 
index 87e1b7c..9c6b926 100644 (file)
@@ -86,6 +86,7 @@
 #include <WebCore/ProcessWarming.h>
 #include <WebCore/ResourceRequest.h>
 #include <WebCore/RuntimeApplicationChecks.h>
+#include <WebCore/RuntimeEnabledFeatures.h>
 #include <pal/SessionID.h>
 #include <wtf/Language.h>
 #include <wtf/MainThread.h>
index a9fb2d0..d01a97a 100644 (file)
                75A8D2C5187CCF9F00C39C9E /* WKWebsiteDataStore.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = WKWebsiteDataStore.mm; sourceTree = "<group>"; };
                75A8D2D4187D1C0100C39C9E /* WKWebsiteDataStoreInternal.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WKWebsiteDataStoreInternal.h; sourceTree = "<group>"; };
                762B7484120BBA2D00819339 /* WKPreferencesRefPrivate.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WKPreferencesRefPrivate.h; sourceTree = "<group>"; };
+               7A0D7861220791EC00EBCF54 /* ResourceLoadStatisticsDatabaseStore.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = ResourceLoadStatisticsDatabaseStore.h; path = Classifier/ResourceLoadStatisticsDatabaseStore.h; sourceTree = "<group>"; };
+               7A0D7862220791ED00EBCF54 /* ResourceLoadStatisticsDatabaseStore.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = ResourceLoadStatisticsDatabaseStore.cpp; path = Classifier/ResourceLoadStatisticsDatabaseStore.cpp; sourceTree = "<group>"; };
                7A1506721DD56298001F4B58 /* com.apple.WebKit.plugin-common.sb */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; name = "com.apple.WebKit.plugin-common.sb"; path = "DerivedSources/WebKit2/com.apple.WebKit.plugin-common.sb"; sourceTree = BUILT_PRODUCTS_DIR; };
                7A1E2A841EEFE88A0037A0E0 /* APINotificationProvider.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = APINotificationProvider.h; sourceTree = "<group>"; };
                7A3ACE1A1EEEF78C00A864A4 /* APIInjectedBundlePageLoaderClient.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = APIInjectedBundlePageLoaderClient.h; sourceTree = "<group>"; };
                7A9CD8C21C779AD600D9F6C7 /* WebResourceLoadStatisticsStore.messages.in */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = WebResourceLoadStatisticsStore.messages.in; sourceTree = "<group>"; };
                7AB6EA441EEAAE2300037B2B /* APIIconDatabaseClient.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = APIIconDatabaseClient.h; sourceTree = "<group>"; };
                7AB6EA461EEAB6B000037B2B /* APIGeolocationProvider.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = APIGeolocationProvider.h; sourceTree = "<group>"; };
+               7ACE82E7221CAE06000DA94C /* ResourceLoadStatisticsStore.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = ResourceLoadStatisticsStore.h; path = Classifier/ResourceLoadStatisticsStore.h; sourceTree = "<group>"; };
+               7ACE82E8221CAE07000DA94C /* ResourceLoadStatisticsStore.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = ResourceLoadStatisticsStore.cpp; path = Classifier/ResourceLoadStatisticsStore.cpp; sourceTree = "<group>"; };
+               7ACE82E9221DE722000DA94C /* ResourceLoadStatisticsStoreCocoa.mm */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.objcpp; name = ResourceLoadStatisticsStoreCocoa.mm; path = Classifier/ResourceLoadStatisticsStoreCocoa.mm; sourceTree = "<group>"; };
                7AF2361E1E79A3B400438A05 /* WebErrors.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = WebErrors.cpp; sourceTree = "<group>"; };
                7AF2361F1E79A3D800438A05 /* WebErrors.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WebErrors.h; sourceTree = "<group>"; };
                7AF236221E79A43100438A05 /* WebErrorsCocoa.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = WebErrorsCocoa.mm; sourceTree = "<group>"; };
                8372DB261A67562800C697C5 /* WebPageDiagnosticLoggingClient.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = WebPageDiagnosticLoggingClient.cpp; sourceTree = "<group>"; };
                8372DB271A67562800C697C5 /* WebPageDiagnosticLoggingClient.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WebPageDiagnosticLoggingClient.h; sourceTree = "<group>"; };
                8372DB2E1A677D4A00C697C5 /* WKDiagnosticLoggingResultType.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WKDiagnosticLoggingResultType.h; sourceTree = "<group>"; };
-               837A660020E2AD8400A9DBD8 /* ResourceLoadStatisticsMemoryStoreCocoa.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = ResourceLoadStatisticsMemoryStoreCocoa.mm; sourceTree = "<group>"; };
                83891B621A68B3420030F386 /* APIDiagnosticLoggingClient.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = APIDiagnosticLoggingClient.h; sourceTree = "<group>"; };
                83891B681A68BEBC0030F386 /* _WKDiagnosticLoggingDelegate.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = _WKDiagnosticLoggingDelegate.h; sourceTree = "<group>"; };
                83891B6A1A68C30B0030F386 /* DiagnosticLoggingClient.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = DiagnosticLoggingClient.h; sourceTree = "<group>"; };
                                CDA29A1E1CBEB5FB00901CCF /* PlaybackSessionManagerProxy.h */,
                                CDA29A221CBEB61A00901CCF /* PlaybackSessionManagerProxy.messages.in */,
                                CDA29A1F1CBEB5FB00901CCF /* PlaybackSessionManagerProxy.mm */,
-                               837A660020E2AD8400A9DBD8 /* ResourceLoadStatisticsMemoryStoreCocoa.mm */,
                                5CA9854B210BEB730057EB6B /* SafeBrowsingWarningCocoa.mm */,
                                1A002D47196B345D00B9AD44 /* SessionStateCoding.h */,
                                1A002D46196B345D00B9AD44 /* SessionStateCoding.mm */,
                7A843A1D21E41FD900DEF663 /* Classifier */ = {
                        isa = PBXGroup;
                        children = (
+                               7A0D7862220791ED00EBCF54 /* ResourceLoadStatisticsDatabaseStore.cpp */,
+                               7A0D7861220791EC00EBCF54 /* ResourceLoadStatisticsDatabaseStore.h */,
                                7AFBD36421E51BAB005DBACB /* ResourceLoadStatisticsMemoryStore.cpp */,
                                7AFBD36521E51BAB005DBACB /* ResourceLoadStatisticsMemoryStore.h */,
                                7AFBD36921E542E8005DBACB /* ResourceLoadStatisticsPersistentStorage.cpp */,
                                7AFBD36821E542E8005DBACB /* ResourceLoadStatisticsPersistentStorage.h */,
+                               7ACE82E8221CAE07000DA94C /* ResourceLoadStatisticsStore.cpp */,
+                               7ACE82E7221CAE06000DA94C /* ResourceLoadStatisticsStore.h */,
+                               7ACE82E9221DE722000DA94C /* ResourceLoadStatisticsStoreCocoa.mm */,
                                7A41E9FA21F81DAC00B88CDB /* ShouldGrandfatherStatistics.h */,
                                7A3FECA121F7C09700F267CD /* StorageAccessStatus.h */,
                                7A843A1A21E41FB200DEF663 /* WebResourceLoadStatisticsStore.cpp */,
                                7C361D79192803BD0036A59D /* WebUserContentControllerProxyMessages.h in Headers */,
                                15739BBD1B42042D00D258C1 /* WebUserMediaClient.h in Headers */,
                                83EE575C1DB7D61100C74C50 /* WebValidationMessageClient.h in Headers */,
+                               572FD44322265CE200A1ECC3 /* WebViewDidMoveToWindowObserver.h in Headers */,
                                2DFC7DBB1BCCC19500C1548C /* WebViewImpl.h in Headers */,
                                29CD55AA128E294F00133C85 /* WKAccessibilityWebPageObjectBase.h in Headers */,
                                29232DF418B29D6800D0596F /* WKAccessibilityWebPageObjectMac.h in Headers */,
                                C54256BA18BEC18C00DE4179 /* WKFormSelectControl.h in Headers */,
                                0F08CF521D63C13A00B48DF1 /* WKFormSelectPicker.h in Headers */,
                                0F08CF541D63C14000B48DF1 /* WKFormSelectPopover.h in Headers */,
-                               572FD44322265CE200A1ECC3 /* WebViewDidMoveToWindowObserver.h in Headers */,
                                BCE4695A1214EDF4000B98EB /* WKFormSubmissionListener.h in Headers */,
                                37DFA7001810BB92001F4A9F /* WKFoundation.h in Headers */,
                                BCD0139B110FA420003B8A67 /* WKFrame.h in Headers */,