Updates to Resource Load Statistics: Get the right website data store and introduce...
authorwilander@apple.com <wilander@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Sat, 11 Feb 2017 07:34:40 +0000 (07:34 +0000)
committerwilander@apple.com <wilander@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Sat, 11 Feb 2017 07:34:40 +0000 (07:34 +0000)
https://bugs.webkit.org/show_bug.cgi?id=167474
<rdar://problem/24681808>
<rdar://problem/24703286>
<rdar://problem/30290270>

Source/WebCore:

This patch does the following:
1. Gets the right website data store. API::WebsiteDataStore::defaultDataStore()
    does not provide the right data store.
2. Introduces timeout for user interaction. A domain needs interaction every 30
    days to stay in that category.
3. Adds grandfathered to the statistics model in preparation for grandfathering of
    existing data records.
4. Adds test infrastructure to allow testing of the various rules in place for
    data records removal.
5. Fixes various smaller bugs that were found as part of setting up the tests.
6. Regresses the data records removal counting. We need to come up with a thread
    safe way of gathering removal statistics from more than one data store now
    that we potentially interact with multiple stores.
7. Adds a first set of layout tests for resource load statistics.

Reviewed by Andy Estes.

Tests: http/tests/loading/resourceLoadStatistics/non-prevalent-resource-with-user-interaction.html
       http/tests/loading/resourceLoadStatistics/non-prevalent-resource-without-user-interaction.html
       http/tests/loading/resourceLoadStatistics/prevalent-resource-with-user-interaction-timeout.html
       http/tests/loading/resourceLoadStatistics/prevalent-resource-with-user-interaction.html
       http/tests/loading/resourceLoadStatistics/prevalent-resource-without-user-interaction.html

* dom/Document.cpp:
(WebCore::Document::updateLastHandledUserGestureTimestamp):
    This now calls ResourceLoadObserver::logUserInteraction() every time since
    we want to keep track of the most recent user interaction.
* loader/ResourceLoadObserver.cpp:
(WebCore::ResourceLoadObserver::statisticsStore):
    New getter used by WebResourceLoadStatisticsManager::resetToConsistentState().
(WebCore::reduceTimeResolutionToOneDay):
    Convenience function.
(WebCore::ResourceLoadObserver::logUserInteractionWithReducedTimeResolution):
    Reduces time resolution for privacy reasons.
(WebCore::ResourceLoadObserver::logUserInteraction):
(WebCore::ResourceLoadObserver::clearUserInteraction):
(WebCore::ResourceLoadObserver::hasHadUserInteraction):
(WebCore::ResourceLoadObserver::setPrevalentResource):
(WebCore::ResourceLoadObserver::isPrevalentResource):
(WebCore::ResourceLoadObserver::clearPrevalentResource):
(WebCore::ResourceLoadObserver::setTimeToLiveUserInteraction):
(WebCore::ResourceLoadObserver::fireDataModificationHandler):
    New functions that allow WebKitTestRunner to stage exact
    statistics, fire the handler, and test the outcome.
* loader/ResourceLoadObserver.h:
* loader/ResourceLoadStatistics.cpp:
(WebCore::ResourceLoadStatistics::encode):
(WebCore::ResourceLoadStatistics::decode):
(WebCore::ResourceLoadStatistics::toString):
(WebCore::ResourceLoadStatistics::merge):
    Support for statistics mostRecentUserInteraction, grandfathered, and
    dataRecordsRemoved.
* loader/ResourceLoadStatistics.h:
* loader/ResourceLoadStatisticsStore.cpp:
(WebCore::ResourceLoadStatisticsStore::setTimeToLiveUserInteraction):
    New function that allows WebKitTestRunner to test
    aging out of user interaction.
(WebCore::ResourceLoadStatisticsStore::hasHadRecentUserInteraction):
    Now takes into account the timestamp and ages
    out user interaction.
(WebCore::ResourceLoadStatisticsStore::prevalentResourceDomainsWithoutUserInteraction):
    Now makes use of ResourceLoadStatisticsStore::hasHadRecentUserInteraction().
* loader/ResourceLoadStatisticsStore.h:

Source/WebKit2:

This patch does the following:
1. Gets the right website data store. API::WebsiteDataStore::defaultDataStore()
    does not provide the right data store.
2. Introduces timeout for user interaction. A domain needs interaction every 30
    days to stay in that category.
3. Adds grandfathered to the statistics model in preparation for grandfathering of
    existing data records.
4. Adds test infrastructure to allow testing of the various rules in place for
    data records removal.
5. Fixes various smaller bugs that were found as part of setting up the tests.
6. Regresses the data records removal counting. We need to come up with a thread
    safe way of gathering removal statistics from more than one data store now
    that we potentially interact with multiple stores.
7. Adds a first set of layout tests for resource load statistics.

Reviewed by Andy Estes.

* PlatformEfl.cmake:
    Added UIProcess/WebResourceLoadStatisticsManager.cpp and
    UIProcess/API/C/WKResourceLoadStatisticsManager.cpp.
* PlatformGTK.cmake:
    Added UIProcess/WebResourceLoadStatisticsManager.cpp and
    UIProcess/API/C/WKResourceLoadStatisticsManager.cpp.
* PlatformMac.cmake:
    Added UIProcess/WebResourceLoadStatisticsManager.cpp and
    UIProcess/API/C/WKResourceLoadStatisticsManager.cpp.
* Shared/API/APIObject.h:
    Adds WebResourceLoadStatisticsManager to the Type enum.
* Shared/API/c/WKBase.h:
    Typedef of WKResourceLoadStatisticsManagerRef.
* Shared/WebCoreArgumentCoders.cpp:
(IPC::ArgumentCoder<ResourceLoadStatistics>::encode):
(IPC::ArgumentCoder<ResourceLoadStatistics>::decode):
    Adds support for statistics mostRecentUserInteraction, grandfathered, and
    dataRecordsRemoved.
* UIProcess/API/C/WKAPICast.h:
    Adds API mapping between WKResourceLoadStatisticsManagerRef and
    WebResourceLoadStatisticsManager.
* UIProcess/API/C/WKResourceLoadStatisticsManager.cpp: Added.
(WKResourceLoadStatisticsManagerGetTypeID):
(WKResourceLoadStatisticsManagerSetPrevalentResource):
(WKResourceLoadStatisticsManagerIsPrevalentResource):
(WKResourceLoadStatisticsManagerSetHasHadUserInteraction):
(WKResourceLoadStatisticsManagerIsHasHadUserInteraction):
(WKResourceLoadStatisticsManagerSetTimeToLiveUserInteraction):
(WKResourceLoadStatisticsManagerFireDataModificationHandler):
(WKResourceLoadStatisticsManagerSetNotifyPagesWhenDataRecordsWereScanned):
(WKResourceLoadStatisticsManagerSetShouldClassifyResourcesBeforeDataRecordsRemoval):
(WKResourceLoadStatisticsManagerSetMinimumTimeBetweeenDataRecordsRemoval):
(WKResourceLoadStatisticsManagerResetToConsistentState):
    API level bridge to WebResourceLoadStatisticsManager functions.
* UIProcess/API/C/WKResourceLoadStatisticsManager.h: Added.
* UIProcess/WebProcessProxy.cpp:
(WebKit::WebProcessProxy::deleteWebsiteDataForTopPrivatelyOwnedDomainsInAllPersistentDataStores):
    A way to delete website data in persistent store(s) configured by the
    embedding client. API::WebsiteDataStore::defaultDataStore() does not
    do this which caused us to skip session storage.
* UIProcess/WebProcessProxy.h:
* UIProcess/WebResourceLoadStatisticsManager.cpp: Added.
(WebKit::WebResourceLoadStatisticsManager::setPrevalentResource):
(WebKit::WebResourceLoadStatisticsManager::isPrevalentResource):
(WebKit::WebResourceLoadStatisticsManager::setHasHadUserInteraction):
(WebKit::WebResourceLoadStatisticsManager::hasHadUserInteraction):
(WebKit::WebResourceLoadStatisticsManager::setTimeToLiveUserInteraction):
(WebKit::WebResourceLoadStatisticsManager::fireDataModificationHandler):
(WebKit::WebResourceLoadStatisticsManager::setNotifyPagesWhenDataRecordsWereScanned):
(WebKit::WebResourceLoadStatisticsManager::setShouldClassifyResourcesBeforeDataRecordsRemoval):
(WebKit::WebResourceLoadStatisticsManager::setMinimumTimeBetweeenDataRecordsRemoval):
(WebKit::WebResourceLoadStatisticsManager::resetToConsistentState):
    Static functions to manage the state and behavior of the UI process'
    resource load statistics store.
* UIProcess/WebResourceLoadStatisticsManager.h: Added.
(WebKit::WebResourceLoadStatisticsManager::create):
* UIProcess/WebResourceLoadStatisticsStore.cpp:
(WebKit::WebResourceLoadStatisticsStore::WebResourceLoadStatisticsStore):
    An effect of a member variable rename. See header file comment below.
(WebKit::WebResourceLoadStatisticsStore::setNotifyPagesWhenDataRecordsWereScanned):
(WebKit::WebResourceLoadStatisticsStore::setShouldClassifyResourcesBeforeDataRecordsRemoval):
(WebKit::WebResourceLoadStatisticsStore::setMinimumTimeBetweeenDataRecordsRemoval):
    Three static functions for controlling the behavior of the store.
(WebKit::WebResourceLoadStatisticsStore::removeDataRecords):
    Changes:
    1. Covers all data record types.
    2. Allows data removal when m_lastTimeDataRecordsWereRemoved is not set.
    3. Moves the heavy lifting to WebsiteDataStore (see below).
    4. No longer counts removed data regressions.
(WebKit::WebResourceLoadStatisticsStore::processStatisticsAndDataRecords):
    See next comment.
(WebKit::WebResourceLoadStatisticsStore::resourceLoadStatisticsUpdated):
    WebResourceLoadStatisticsStore::processStatisticsAndDataRecords() is
    broken out of WebResourceLoadStatisticsStore::resourceLoadStatisticsUpdated
    to allow for statistics processing without prior classification.
    Tests need to set statistics and a run of the classifier may
    reset the data.
(WebKit::WebResourceLoadStatisticsStore::registerSharedResourceLoadObserver):
    Sets the notification callback which calls
    WebResourceLoadStatisticsStore::processStatisticsAndDataRecords().
* UIProcess/WebResourceLoadStatisticsStore.h:
(WebKit::WebResourceLoadStatisticsStore::coreStore):
    Renamed m_resourceStatisticsStore to m_resourceLoadStatisticsStore to
    harmonize with other naming.
* UIProcess/WebsiteData/WebsiteDataStore.cpp:
(WebKit::WebsiteDataStore::fetchDataForTopPrivatelyOwnedDomains):
    This is now where matching is made between prevalent resources
    and data records. This function is used by
    WebsiteDataStore::removeDataForTopPrivatelyOwnedDomains().
(WebKit::WebsiteDataStore::removeDataForTopPrivatelyOwnedDomains):
    The function WebResourceLoadStatisticsStore::removeDataRecords()
    calls to get data records removed in all persistent stores.
* UIProcess/WebsiteData/WebsiteDataStore.h:
* WebKit2.xcodeproj/project.pbxproj:
* WebProcess/WebProcess.cpp:
(WebKit::m_resourceLoadStatisticsStore):
(WebKit::WebProcess::statisticsChangedTimerFired):
(WebKit::m_resourceLoadStatisticsStorage): Deleted.
    Renamed m_resourceLoadStatisticsStorage to m_resourceLoadStatisticsStore
    to harmonize with other naming.
* WebProcess/WebProcess.h:

Tools:

This patch adds test infrastructure to allow testing
of the various rules in place for data records removal.

Reviewed by Andy Estes.

* WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl:
* WebKitTestRunner/InjectedBundle/InjectedBundle.cpp:
(WTR::InjectedBundle::didReceiveMessageToPage):
    Callback mechanism to tell pages that a website data store
    scan has happened.
* WebKitTestRunner/InjectedBundle/TestRunner.cpp:
(WTR::TestRunner::setStatisticsPrevalentResource):
(WTR::TestRunner::isStatisticsPrevalentResource):
(WTR::TestRunner::setStatisticsHasHadUserInteraction):
(WTR::TestRunner::isStatisticsHasHadUserInteraction):
(WTR::TestRunner::setStatisticsTimeToLiveUserInteraction):
(WTR::TestRunner::installStatisticsDidModifyDataRecordsCallback):
(WTR::TestRunner::statisticsDidModifyDataRecordsCallback):
(WTR::TestRunner::statisticsFireDataModificationHandler):
(WTR::TestRunner::setStatisticsNotifyPagesWhenDataRecordsWereScanned):
(WTR::TestRunner::setStatisticsShouldClassifyResourcesBeforeDataRecordsRemoval):
(WTR::TestRunner::setStatisticsMinimumTimeBetweeenDataRecordsRemoval):
(WTR::TestRunner::statisticsResetToConsistentState):
* WebKitTestRunner/InjectedBundle/TestRunner.h:
* WebKitTestRunner/TestController.cpp:
(WTR::TestController::setStatisticsPrevalentResource):
(WTR::TestController::isStatisticsPrevalentResource):
(WTR::TestController::setStatisticsHasHadUserInteraction):
(WTR::TestController::isStatisticsHasHadUserInteraction):
(WTR::TestController::setStatisticsTimeToLiveUserInteraction):
(WTR::TestController::statisticsFireDataModificationHandler):
(WTR::TestController::setStatisticsNotifyPagesWhenDataRecordsWereScanned):
(WTR::TestController::setStatisticsShouldClassifyResourcesBeforeDataRecordsRemoval):
(WTR::TestController::setStatisticsMinimumTimeBetweeenDataRecordsRemoval):
(WTR::TestController::statisticsResetToConsistentState):
* WebKitTestRunner/TestController.h:
    These are all configuration and test functions.
* WebKitTestRunner/TestInvocation.cpp:
(WTR::TestInvocation::didReceiveSynchronousMessageFromInjectedBundle):

LayoutTests:

Reviewed by Andy Estes.

* TestExpectations:
    Marked as general skip (see WK2 expectations below).
* http/tests/loading/resourceLoadStatistics: Added.
* http/tests/loading/resourceLoadStatistics/non-prevalent-resource-with-user-interaction-expected.txt: Added.
* http/tests/loading/resourceLoadStatistics/non-prevalent-resource-with-user-interaction.html: Added.
* http/tests/loading/resourceLoadStatistics/non-prevalent-resource-without-user-interaction-expected.txt: Added.
* http/tests/loading/resourceLoadStatistics/non-prevalent-resource-without-user-interaction.html: Added.
* http/tests/loading/resourceLoadStatistics/prevalent-resource-with-user-interaction-expected.txt: Added.
* http/tests/loading/resourceLoadStatistics/prevalent-resource-with-user-interaction-timeout-expected.txt: Added.
* http/tests/loading/resourceLoadStatistics/prevalent-resource-with-user-interaction-timeout.html: Added.
* http/tests/loading/resourceLoadStatistics/prevalent-resource-with-user-interaction.html: Added.
* http/tests/loading/resourceLoadStatistics/prevalent-resource-without-user-interaction-expected.txt: Added.
* http/tests/loading/resourceLoadStatistics/prevalent-resource-without-user-interaction.html: Added.
* platform/wk2/TestExpectations:
    Marked as valid for WebKit2 only.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@212183 268f45cc-cd09-0410-ab3c-d52691b4dbfc

50 files changed:
LayoutTests/ChangeLog
LayoutTests/TestExpectations
LayoutTests/http/tests/loading/resourceLoadStatistics/non-prevalent-resource-with-user-interaction-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/loading/resourceLoadStatistics/non-prevalent-resource-with-user-interaction.html [new file with mode: 0644]
LayoutTests/http/tests/loading/resourceLoadStatistics/non-prevalent-resource-without-user-interaction-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/loading/resourceLoadStatistics/non-prevalent-resource-without-user-interaction.html [new file with mode: 0644]
LayoutTests/http/tests/loading/resourceLoadStatistics/prevalent-resource-with-user-interaction-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/loading/resourceLoadStatistics/prevalent-resource-with-user-interaction-timeout-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/loading/resourceLoadStatistics/prevalent-resource-with-user-interaction-timeout.html [new file with mode: 0644]
LayoutTests/http/tests/loading/resourceLoadStatistics/prevalent-resource-with-user-interaction.html [new file with mode: 0644]
LayoutTests/http/tests/loading/resourceLoadStatistics/prevalent-resource-without-user-interaction-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/loading/resourceLoadStatistics/prevalent-resource-without-user-interaction.html [new file with mode: 0644]
LayoutTests/platform/wk2/TestExpectations
Source/WebCore/ChangeLog
Source/WebCore/dom/Document.cpp
Source/WebCore/loader/ResourceLoadObserver.cpp
Source/WebCore/loader/ResourceLoadObserver.h
Source/WebCore/loader/ResourceLoadStatistics.cpp
Source/WebCore/loader/ResourceLoadStatistics.h
Source/WebCore/loader/ResourceLoadStatisticsStore.cpp
Source/WebCore/loader/ResourceLoadStatisticsStore.h
Source/WebKit2/ChangeLog
Source/WebKit2/PlatformEfl.cmake
Source/WebKit2/PlatformGTK.cmake
Source/WebKit2/PlatformMac.cmake
Source/WebKit2/Shared/API/APIObject.h
Source/WebKit2/Shared/API/c/WKBase.h
Source/WebKit2/Shared/WebCoreArgumentCoders.cpp
Source/WebKit2/UIProcess/API/C/WKAPICast.h
Source/WebKit2/UIProcess/API/C/WKResourceLoadStatisticsManager.cpp [new file with mode: 0644]
Source/WebKit2/UIProcess/API/C/WKResourceLoadStatisticsManager.h [new file with mode: 0644]
Source/WebKit2/UIProcess/WebProcessProxy.cpp
Source/WebKit2/UIProcess/WebProcessProxy.h
Source/WebKit2/UIProcess/WebResourceLoadStatisticsManager.cpp [new file with mode: 0644]
Source/WebKit2/UIProcess/WebResourceLoadStatisticsManager.h [new file with mode: 0644]
Source/WebKit2/UIProcess/WebResourceLoadStatisticsStore.cpp
Source/WebKit2/UIProcess/WebResourceLoadStatisticsStore.h
Source/WebKit2/UIProcess/WebsiteData/WebsiteDataStore.cpp
Source/WebKit2/UIProcess/WebsiteData/WebsiteDataStore.h
Source/WebKit2/WebKit2.xcodeproj/project.pbxproj
Source/WebKit2/WebProcess/WebProcess.cpp
Source/WebKit2/WebProcess/WebProcess.h
Tools/ChangeLog
Tools/WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl
Tools/WebKitTestRunner/InjectedBundle/InjectedBundle.cpp
Tools/WebKitTestRunner/InjectedBundle/TestRunner.cpp
Tools/WebKitTestRunner/InjectedBundle/TestRunner.h
Tools/WebKitTestRunner/TestController.cpp
Tools/WebKitTestRunner/TestController.h
Tools/WebKitTestRunner/TestInvocation.cpp

index 445d411..704a593 100644 (file)
@@ -1,3 +1,29 @@
+2017-02-10  John Wilander  <wilander@apple.com>
+
+        Updates to Resource Load Statistics: Get the right website data store and introduce timeout for user interaction
+        https://bugs.webkit.org/show_bug.cgi?id=167474
+        <rdar://problem/24681808>
+        <rdar://problem/24703286>
+        <rdar://problem/30290270>
+
+        Reviewed by Andy Estes.
+
+        * TestExpectations:
+            Marked as general skip (see WK2 expectations below).
+        * http/tests/loading/resourceLoadStatistics: Added.
+        * http/tests/loading/resourceLoadStatistics/non-prevalent-resource-with-user-interaction-expected.txt: Added.
+        * http/tests/loading/resourceLoadStatistics/non-prevalent-resource-with-user-interaction.html: Added.
+        * http/tests/loading/resourceLoadStatistics/non-prevalent-resource-without-user-interaction-expected.txt: Added.
+        * http/tests/loading/resourceLoadStatistics/non-prevalent-resource-without-user-interaction.html: Added.
+        * http/tests/loading/resourceLoadStatistics/prevalent-resource-with-user-interaction-expected.txt: Added.
+        * http/tests/loading/resourceLoadStatistics/prevalent-resource-with-user-interaction-timeout-expected.txt: Added.
+        * http/tests/loading/resourceLoadStatistics/prevalent-resource-with-user-interaction-timeout.html: Added.
+        * http/tests/loading/resourceLoadStatistics/prevalent-resource-with-user-interaction.html: Added.
+        * http/tests/loading/resourceLoadStatistics/prevalent-resource-without-user-interaction-expected.txt: Added.
+        * http/tests/loading/resourceLoadStatistics/prevalent-resource-without-user-interaction.html: Added.
+        * platform/wk2/TestExpectations:
+            Marked as valid for WebKit2 only.
+
 2017-02-10  Sam Weinig  <sam@webkit.org>
 
         [WebIDL] Cleanup XMLHttpRequest's bindings
index 8ff7527..ec698d2 100644 (file)
@@ -64,6 +64,7 @@ http/tests/quicklook [ Skip ]
 # These tests are WebKit2-only
 http/tests/appcache/decide-navigation-policy-after-delay.html [ Skip ]
 http/tests/misc/will-send-request-with-client-provided-http-body.html [ Skip ]
+http/tests/loading/resourceLoadStatistics/ [ Skip ]
 
 # Only Mac and iOS have an implementation of UIScriptController::doAsyncTask().
 fast/harness/uiscriptcontroller [ Skip ]
diff --git a/LayoutTests/http/tests/loading/resourceLoadStatistics/non-prevalent-resource-with-user-interaction-expected.txt b/LayoutTests/http/tests/loading/resourceLoadStatistics/non-prevalent-resource-with-user-interaction-expected.txt
new file mode 100644 (file)
index 0000000..5eb75dd
--- /dev/null
@@ -0,0 +1,8 @@
+main frame - didStartProvisionalLoadForFrame
+main frame - didCommitLoadForFrame
+main frame - didReceiveTitle: Tests for Non-Prevalent Resource With User Interaction
+main frame - didFinishDocumentLoadForFrame
+main frame - didHandleOnloadEventsForFrame
+main frame - didFinishLoadForFrame
+PASS Cookie not deleted.
+
diff --git a/LayoutTests/http/tests/loading/resourceLoadStatistics/non-prevalent-resource-with-user-interaction.html b/LayoutTests/http/tests/loading/resourceLoadStatistics/non-prevalent-resource-with-user-interaction.html
new file mode 100644 (file)
index 0000000..bdc9ed6
--- /dev/null
@@ -0,0 +1,56 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <title>Tests for Non-Prevalent Resource With User Interaction</title>
+    <script src="../../resources/js-test-pre.js"></script>
+</head>
+<body>
+<script>
+    const hostUnderTest = "127.0.0.1:8000";
+    const statisticsUrl = "http://" + hostUnderTest + "/temp";
+    const otherPrevalentUrl = "http://localhost:8000/temp";
+    function runTestRunnerTest() {
+        if (document.cookie !== "")
+            testFailed("document.cookie not empty.");
+        const cookie = "testCookie=testValue";
+        document.cookie = cookie + "; max-age=100;";
+        if (document.cookie !== cookie)
+            testFailed("document.cookie did not get set.");
+
+        testRunner.setStatisticsPrevalentResource(statisticsUrl, false);
+        if (testRunner.isStatisticsPrevalentResource(statisticsUrl))
+            testFailed("Host did not get set as non-prevalent resource.");
+
+        // This is done to not have an empty set of prevalent resources.
+        // Otherwise data records are never scanned.
+        testRunner.setStatisticsPrevalentResource(otherPrevalentUrl, true);
+        if (!testRunner.isStatisticsPrevalentResource(otherPrevalentUrl))
+            testFailed("Other host did not get set as prevalent resource.");
+
+        testRunner.setStatisticsHasHadUserInteraction(statisticsUrl, true);
+        if (!testRunner.isStatisticsHasHadUserInteraction(statisticsUrl))
+            testFailed("Host did not get logged for user interaction.");
+
+        testRunner.installStatisticsDidModifyDataRecordsCallback(function() {
+            if (document.cookie === cookie)
+                testPassed("Cookie not deleted.");
+            else
+                testFailed("Cookie deleted or document.cookie contains other cookies: " + document.cookie);
+            testRunner.statisticsResetToConsistentState();
+            testRunner.notifyDone();
+        });
+        testRunner.setStatisticsShouldClassifyResourcesBeforeDataRecordsRemoval(false);
+        testRunner.setStatisticsMinimumTimeBetweeenDataRecordsRemoval(0);
+        testRunner.statisticsFireDataModificationHandler();
+    }
+
+    if (document.location.host === hostUnderTest && window.testRunner && window.internals) {
+        testRunner.waitUntilDone();
+        internals.setResourceLoadStatisticsEnabled(true);
+        testRunner.setStatisticsNotifyPagesWhenDataRecordsWereScanned(true);
+        runTestRunnerTest();
+    }
+</script>
+</body>
+</html>
\ No newline at end of file
diff --git a/LayoutTests/http/tests/loading/resourceLoadStatistics/non-prevalent-resource-without-user-interaction-expected.txt b/LayoutTests/http/tests/loading/resourceLoadStatistics/non-prevalent-resource-without-user-interaction-expected.txt
new file mode 100644 (file)
index 0000000..54b4059
--- /dev/null
@@ -0,0 +1,8 @@
+main frame - didStartProvisionalLoadForFrame
+main frame - didCommitLoadForFrame
+main frame - didReceiveTitle: Tests for Non-Prevalent Resource Without User Interaction
+main frame - didFinishDocumentLoadForFrame
+main frame - didHandleOnloadEventsForFrame
+main frame - didFinishLoadForFrame
+PASS Cookie not deleted.
+
diff --git a/LayoutTests/http/tests/loading/resourceLoadStatistics/non-prevalent-resource-without-user-interaction.html b/LayoutTests/http/tests/loading/resourceLoadStatistics/non-prevalent-resource-without-user-interaction.html
new file mode 100644 (file)
index 0000000..955244f
--- /dev/null
@@ -0,0 +1,56 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <title>Tests for Non-Prevalent Resource Without User Interaction</title>
+    <script src="../../resources/js-test-pre.js"></script>
+</head>
+<body>
+<script>
+    const hostUnderTest = "127.0.0.1:8000";
+    const statisticsUrl = "http://" + hostUnderTest + "/temp";
+    const otherPrevalentUrl = "http://localhost:8000/temp";
+    function runTestRunnerTest() {
+        if (document.cookie !== "")
+            testFailed("document.cookie not empty.");
+        const cookie = "testCookie=testValue";
+        document.cookie = cookie + "; max-age=100;";
+        if (document.cookie !== cookie)
+            testFailed("document.cookie did not get set.");
+
+        testRunner.setStatisticsPrevalentResource(statisticsUrl, false);
+        if (testRunner.isStatisticsPrevalentResource(statisticsUrl))
+            testFailed("Host did not get set as non-prevalent resource.");
+
+        // This is done to not have an empty set of prevalent resources.
+        // Otherwise data records are never scanned.
+        testRunner.setStatisticsPrevalentResource(otherPrevalentUrl, true);
+        if (!testRunner.isStatisticsPrevalentResource(otherPrevalentUrl))
+            testFailed("Other host did not get set as prevalent resource.");
+
+        testRunner.setStatisticsHasHadUserInteraction(statisticsUrl, false);
+        if (testRunner.isStatisticsHasHadUserInteraction(statisticsUrl))
+            testFailed("Host did not get cleared of user interaction.");
+
+        testRunner.installStatisticsDidModifyDataRecordsCallback(function() {
+            if (document.cookie === cookie)
+                testPassed("Cookie not deleted.");
+            else
+                testFailed("Cookie deleted or document.cookie contains other cookies: " + document.cookie);
+            testRunner.statisticsResetToConsistentState();
+            testRunner.notifyDone();
+        });
+        testRunner.setStatisticsShouldClassifyResourcesBeforeDataRecordsRemoval(false);
+        testRunner.setStatisticsMinimumTimeBetweeenDataRecordsRemoval(0);
+        testRunner.statisticsFireDataModificationHandler();
+    }
+
+    if (document.location.host === hostUnderTest && window.testRunner && window.internals) {
+        testRunner.waitUntilDone();
+        internals.setResourceLoadStatisticsEnabled(true);
+        testRunner.setStatisticsNotifyPagesWhenDataRecordsWereScanned(true);
+        runTestRunnerTest();
+    }
+</script>
+</body>
+</html>
\ No newline at end of file
diff --git a/LayoutTests/http/tests/loading/resourceLoadStatistics/prevalent-resource-with-user-interaction-expected.txt b/LayoutTests/http/tests/loading/resourceLoadStatistics/prevalent-resource-with-user-interaction-expected.txt
new file mode 100644 (file)
index 0000000..9801557
--- /dev/null
@@ -0,0 +1,8 @@
+main frame - didStartProvisionalLoadForFrame
+main frame - didCommitLoadForFrame
+main frame - didReceiveTitle: Tests for Prevalent Resource With User Interaction
+main frame - didFinishDocumentLoadForFrame
+main frame - didHandleOnloadEventsForFrame
+main frame - didFinishLoadForFrame
+PASS Cookie not deleted.
+
diff --git a/LayoutTests/http/tests/loading/resourceLoadStatistics/prevalent-resource-with-user-interaction-timeout-expected.txt b/LayoutTests/http/tests/loading/resourceLoadStatistics/prevalent-resource-with-user-interaction-timeout-expected.txt
new file mode 100644 (file)
index 0000000..bc66e65
--- /dev/null
@@ -0,0 +1,8 @@
+main frame - didStartProvisionalLoadForFrame
+main frame - didCommitLoadForFrame
+main frame - didReceiveTitle: Tests for Prevalent Resource With User Interaction Timeout
+main frame - didFinishDocumentLoadForFrame
+main frame - didHandleOnloadEventsForFrame
+main frame - didFinishLoadForFrame
+PASS Cookie deleted.
+
diff --git a/LayoutTests/http/tests/loading/resourceLoadStatistics/prevalent-resource-with-user-interaction-timeout.html b/LayoutTests/http/tests/loading/resourceLoadStatistics/prevalent-resource-with-user-interaction-timeout.html
new file mode 100644 (file)
index 0000000..2dda79d
--- /dev/null
@@ -0,0 +1,50 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <title>Tests for Prevalent Resource With User Interaction Timeout</title>
+    <script src="../../resources/js-test-pre.js"></script>
+</head>
+<body>
+<script>
+    const hostUnderTest = "127.0.0.1:8000";
+    const statisticsUrl = "http://" + hostUnderTest + "/temp";
+    function runTestRunnerTest() {
+        if (document.cookie !== "")
+            testFailed("document.cookie not empty.");
+        const cookie = "testCookie=testValue";
+        document.cookie = cookie + "; max-age=100;";
+        if (document.cookie !== cookie)
+            testFailed("document.cookie did not get set.");
+
+        testRunner.setStatisticsPrevalentResource(statisticsUrl, true);
+        if (!testRunner.isStatisticsPrevalentResource(statisticsUrl))
+            testFailed("Host did not get set as prevalent resource.");
+
+        testRunner.setStatisticsHasHadUserInteraction(statisticsUrl, true);
+        if (!testRunner.isStatisticsHasHadUserInteraction(statisticsUrl))
+            testFailed("Host did not get logged for user interaction.");
+
+        testRunner.installStatisticsDidModifyDataRecordsCallback(function() {
+            if (document.cookie !== "")
+                testFailed("Cookie not deleted: " + document.cookie);
+            else
+                testPassed("Cookie deleted.");
+            testRunner.statisticsResetToConsistentState();
+            testRunner.notifyDone();
+        });
+        testRunner.setStatisticsShouldClassifyResourcesBeforeDataRecordsRemoval(false);
+        testRunner.setStatisticsMinimumTimeBetweeenDataRecordsRemoval(0);
+        testRunner.setStatisticsTimeToLiveUserInteraction(0);
+        setTimeout("testRunner.statisticsFireDataModificationHandler()", 1000);
+    }
+
+    if (document.location.host === hostUnderTest && window.testRunner && window.internals) {
+        testRunner.waitUntilDone();
+        internals.setResourceLoadStatisticsEnabled(true);
+        testRunner.setStatisticsNotifyPagesWhenDataRecordsWereScanned(true);
+        runTestRunnerTest();
+    }
+</script>
+</body>
+</html>
\ No newline at end of file
diff --git a/LayoutTests/http/tests/loading/resourceLoadStatistics/prevalent-resource-with-user-interaction.html b/LayoutTests/http/tests/loading/resourceLoadStatistics/prevalent-resource-with-user-interaction.html
new file mode 100644 (file)
index 0000000..a65151c
--- /dev/null
@@ -0,0 +1,56 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <title>Tests for Prevalent Resource With User Interaction</title>
+    <script src="../../resources/js-test-pre.js"></script>
+</head>
+<body>
+<script>
+    const hostUnderTest = "127.0.0.1:8000";
+    const statisticsUrl = "http://" + hostUnderTest + "/temp";
+    const otherPrevalentUrl = "http://localhost:8000/temp";
+    function runTestRunnerTest() {
+        if (document.cookie !== "")
+            testFailed("document.cookie not empty.");
+        const cookie = "testCookie=testValue";
+        document.cookie = cookie + "; max-age=100;";
+        if (document.cookie !== cookie)
+            testFailed("document.cookie did not get set.");
+
+        testRunner.setStatisticsPrevalentResource(statisticsUrl, true);
+        if (!testRunner.isStatisticsPrevalentResource(statisticsUrl))
+            testFailed("Host did not get set as prevalent resource.");
+
+        // This is done to not have an empty set of prevalent resources.
+        // Otherwise data records are never scanned.
+        testRunner.setStatisticsPrevalentResource(otherPrevalentUrl, true);
+        if (!testRunner.isStatisticsPrevalentResource(otherPrevalentUrl))
+            testFailed("Other host did not get set as prevalent resource.");
+
+        testRunner.setStatisticsHasHadUserInteraction(statisticsUrl, true);
+        if (!testRunner.isStatisticsHasHadUserInteraction(statisticsUrl))
+            testFailed("Host did not get logged for user interaction.");
+
+        testRunner.installStatisticsDidModifyDataRecordsCallback(function() {
+            if (document.cookie === cookie)
+                testPassed("Cookie not deleted.");
+            else
+                testFailed("Cookie deleted or document.cookie contains other cookies: " + document.cookie);
+            testRunner.statisticsResetToConsistentState();
+            testRunner.notifyDone();
+        });
+        testRunner.setStatisticsShouldClassifyResourcesBeforeDataRecordsRemoval(false);
+        testRunner.setStatisticsMinimumTimeBetweeenDataRecordsRemoval(0);
+        testRunner.statisticsFireDataModificationHandler();
+    }
+
+    if (document.location.host === hostUnderTest && window.testRunner && window.internals) {
+        testRunner.waitUntilDone();
+        internals.setResourceLoadStatisticsEnabled(true);
+        testRunner.setStatisticsNotifyPagesWhenDataRecordsWereScanned(true);
+        runTestRunnerTest();
+    }
+</script>
+</body>
+</html>
\ No newline at end of file
diff --git a/LayoutTests/http/tests/loading/resourceLoadStatistics/prevalent-resource-without-user-interaction-expected.txt b/LayoutTests/http/tests/loading/resourceLoadStatistics/prevalent-resource-without-user-interaction-expected.txt
new file mode 100644 (file)
index 0000000..34a0741
--- /dev/null
@@ -0,0 +1,8 @@
+main frame - didStartProvisionalLoadForFrame
+main frame - didCommitLoadForFrame
+main frame - didReceiveTitle: Tests for Prevalent Resource Without User Interaction
+main frame - didFinishDocumentLoadForFrame
+main frame - didHandleOnloadEventsForFrame
+main frame - didFinishLoadForFrame
+PASS Cookie deleted.
+
diff --git a/LayoutTests/http/tests/loading/resourceLoadStatistics/prevalent-resource-without-user-interaction.html b/LayoutTests/http/tests/loading/resourceLoadStatistics/prevalent-resource-without-user-interaction.html
new file mode 100644 (file)
index 0000000..7cef910
--- /dev/null
@@ -0,0 +1,49 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <title>Tests for Prevalent Resource Without User Interaction</title>
+    <script src="../../resources/js-test-pre.js"></script>
+</head>
+<body>
+<script>
+    const hostUnderTest = "127.0.0.1:8000";
+    const statisticsUrl = "http://" + hostUnderTest + "/temp";
+    function runTestRunnerTest() {
+        if (document.cookie !== "")
+            testFailed("document.cookie not empty.");
+        const cookie = "testCookie=testValue";
+        document.cookie = cookie + "; max-age=100;";
+        if (document.cookie !== cookie)
+            testFailed("document.cookie did not get set.");
+
+        testRunner.setStatisticsPrevalentResource(statisticsUrl, true);
+        if (!testRunner.isStatisticsPrevalentResource(statisticsUrl))
+            testFailed("Host did not get set as prevalent resource.");
+
+        testRunner.setStatisticsHasHadUserInteraction(statisticsUrl, false);
+        if (testRunner.isStatisticsHasHadUserInteraction(statisticsUrl))
+            testFailed("Host did not get cleared of user interaction.");
+
+        testRunner.installStatisticsDidModifyDataRecordsCallback(function() {
+            if (document.cookie !== "")
+                testFailed("Cookie not deleted: " + document.cookie);
+            else
+                testPassed("Cookie deleted.");
+            testRunner.statisticsResetToConsistentState();
+            testRunner.notifyDone();
+        });
+        testRunner.setStatisticsShouldClassifyResourcesBeforeDataRecordsRemoval(false);
+        testRunner.setStatisticsMinimumTimeBetweeenDataRecordsRemoval(0);
+        testRunner.statisticsFireDataModificationHandler();
+    }
+
+    if (document.location.host === hostUnderTest && window.testRunner && window.internals) {
+        testRunner.waitUntilDone();
+        internals.setResourceLoadStatisticsEnabled(true);
+        testRunner.setStatisticsNotifyPagesWhenDataRecordsWereScanned(true);
+        runTestRunnerTest();
+    }
+</script>
+</body>
+</html>
\ No newline at end of file
index bb7616e..2261341 100644 (file)
@@ -698,6 +698,9 @@ http/tests/misc/will-send-request-with-client-provided-http-body.html [ Pass ]
 # bug 162281
 imported/w3c/web-platform-tests/fetch/api/request/request-cache-only-if-cached.html [ Failure ]
 
+# Resource Load Statistics are only supported in WebKit2
+http/tests/loading/resourceLoadStatistics/ [ Pass ]
+
 ### END OF (5) Progressions, expected successes that are expected failures in WebKit1.
 ########################################
 
index 61a0c36..b557879 100644 (file)
@@ -1,3 +1,75 @@
+2017-02-10  John Wilander  <wilander@apple.com>
+
+        Updates to Resource Load Statistics: Get the right website data store and introduce timeout for user interaction
+        https://bugs.webkit.org/show_bug.cgi?id=167474
+        <rdar://problem/24681808>
+        <rdar://problem/24703286>
+        <rdar://problem/30290270>
+
+        This patch does the following:
+        1. Gets the right website data store. API::WebsiteDataStore::defaultDataStore()
+            does not provide the right data store.
+        2. Introduces timeout for user interaction. A domain needs interaction every 30
+            days to stay in that category.
+        3. Adds grandfathered to the statistics model in preparation for grandfathering of
+            existing data records.
+        4. Adds test infrastructure to allow testing of the various rules in place for
+            data records removal.
+        5. Fixes various smaller bugs that were found as part of setting up the tests.
+        6. Regresses the data records removal counting. We need to come up with a thread
+            safe way of gathering removal statistics from more than one data store now
+            that we potentially interact with multiple stores.
+        7. Adds a first set of layout tests for resource load statistics.
+
+        Reviewed by Andy Estes.
+
+        Tests: http/tests/loading/resourceLoadStatistics/non-prevalent-resource-with-user-interaction.html
+               http/tests/loading/resourceLoadStatistics/non-prevalent-resource-without-user-interaction.html
+               http/tests/loading/resourceLoadStatistics/prevalent-resource-with-user-interaction-timeout.html
+               http/tests/loading/resourceLoadStatistics/prevalent-resource-with-user-interaction.html
+               http/tests/loading/resourceLoadStatistics/prevalent-resource-without-user-interaction.html
+
+        * dom/Document.cpp:
+        (WebCore::Document::updateLastHandledUserGestureTimestamp):
+            This now calls ResourceLoadObserver::logUserInteraction() every time since
+            we want to keep track of the most recent user interaction.
+        * loader/ResourceLoadObserver.cpp:
+        (WebCore::ResourceLoadObserver::statisticsStore):
+            New getter used by WebResourceLoadStatisticsManager::resetToConsistentState().
+        (WebCore::reduceTimeResolutionToOneDay):
+            Convenience function.
+        (WebCore::ResourceLoadObserver::logUserInteractionWithReducedTimeResolution):
+            Reduces time resolution for privacy reasons.
+        (WebCore::ResourceLoadObserver::logUserInteraction):
+        (WebCore::ResourceLoadObserver::clearUserInteraction):
+        (WebCore::ResourceLoadObserver::hasHadUserInteraction):
+        (WebCore::ResourceLoadObserver::setPrevalentResource):
+        (WebCore::ResourceLoadObserver::isPrevalentResource):
+        (WebCore::ResourceLoadObserver::clearPrevalentResource):
+        (WebCore::ResourceLoadObserver::setTimeToLiveUserInteraction):
+        (WebCore::ResourceLoadObserver::fireDataModificationHandler):
+            New functions that allow WebKitTestRunner to stage exact
+            statistics, fire the handler, and test the outcome.
+        * loader/ResourceLoadObserver.h:
+        * loader/ResourceLoadStatistics.cpp:
+        (WebCore::ResourceLoadStatistics::encode):
+        (WebCore::ResourceLoadStatistics::decode):
+        (WebCore::ResourceLoadStatistics::toString):
+        (WebCore::ResourceLoadStatistics::merge):
+            Support for statistics mostRecentUserInteraction, grandfathered, and
+            dataRecordsRemoved.
+        * loader/ResourceLoadStatistics.h:
+        * loader/ResourceLoadStatisticsStore.cpp:
+        (WebCore::ResourceLoadStatisticsStore::setTimeToLiveUserInteraction):
+            New function that allows WebKitTestRunner to test 
+            aging out of user interaction.
+        (WebCore::ResourceLoadStatisticsStore::hasHadRecentUserInteraction):
+            Now takes into account the timestamp and ages
+            out user interaction.
+        (WebCore::ResourceLoadStatisticsStore::prevalentResourceDomainsWithoutUserInteraction):
+            Now makes use of ResourceLoadStatisticsStore::hasHadRecentUserInteraction().
+        * loader/ResourceLoadStatisticsStore.h:
+
 2017-02-10  Sam Weinig  <sam@webkit.org>
 
         [WebIDL] Cleanup XMLHttpRequest's bindings
index 2f90f25..5b08411 100644 (file)
@@ -6356,10 +6356,8 @@ Document::RegionFixedPair Document::absoluteRegionForEventTargets(const EventTar
 
 void Document::updateLastHandledUserGestureTimestamp()
 {
-    if (!m_lastHandledUserGestureTimestamp)
-        ResourceLoadObserver::sharedObserver().logUserInteraction(*this);
-
     m_lastHandledUserGestureTimestamp = monotonicallyIncreasingTime();
+    ResourceLoadObserver::sharedObserver().logUserInteractionWithReducedTimeResolution(*this);
 }
 
 void Document::startTrackingStyleRecalcs()
index 5310fe9..8d3da3b 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2016 Apple Inc. All rights reserved.
+ * Copyright (C) 2016-2017 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
 #include "Settings.h"
 #include "SharedBuffer.h"
 #include "URL.h"
+#include <wtf/CurrentTime.h>
 #include <wtf/NeverDestroyed.h>
 #include <wtf/text/StringBuilder.h>
 
 namespace WebCore {
 
+// One day in seconds.
+static auto timestampResolution = 86400;
+
 ResourceLoadObserver& ResourceLoadObserver::sharedObserver()
 {
     static NeverDestroyed<ResourceLoadObserver> resourceLoadObserver;
     return resourceLoadObserver;
 }
 
+RefPtr<ResourceLoadStatisticsStore> ResourceLoadObserver::statisticsStore()
+{
+    ASSERT(m_store);
+    return m_store;
+}
+
 void ResourceLoadObserver::setStatisticsStore(Ref<ResourceLoadStatisticsStore>&& store)
 {
     m_store = WTFMove(store);
@@ -278,7 +288,12 @@ void ResourceLoadObserver::logWebSocketLoading(const Frame* frame, const URL& ta
         m_store->fireDataModificationHandler();
 }
 
-void ResourceLoadObserver::logUserInteraction(const Document& document)
+static double reduceTimeResolutionToOneDay(double seconds)
+{
+    return std::floor(seconds / timestampResolution) * timestampResolution;
+}
+
+void ResourceLoadObserver::logUserInteractionWithReducedTimeResolution(const Document& document)
 {
     ASSERT(document.page());
 
@@ -286,15 +301,90 @@ void ResourceLoadObserver::logUserInteraction(const Document& document)
         return;
 
     auto& url = document.url();
-
     if (url.isBlankURL() || url.isEmpty())
         return;
 
     auto& statistics = m_store->ensureResourceStatisticsForPrimaryDomain(primaryDomain(url));
+    double newTimestamp = reduceTimeResolutionToOneDay(WTF::currentTime());
+    if (newTimestamp == statistics.mostRecentUserInteraction)
+        return;
+
     statistics.hadUserInteraction = true;
+    statistics.mostRecentUserInteraction = newTimestamp;
     m_store->fireDataModificationHandler();
 }
+
+void ResourceLoadObserver::logUserInteraction(const URL& url)
+{
+    if (url.isBlankURL() || url.isEmpty())
+        return;
+
+    auto& statistics = m_store->ensureResourceStatisticsForPrimaryDomain(primaryDomain(url));
+    statistics.hadUserInteraction = true;
+    statistics.mostRecentUserInteraction = WTF::currentTime();
+}
+
+void ResourceLoadObserver::clearUserInteraction(const URL& url)
+{
+    if (url.isBlankURL() || url.isEmpty())
+        return;
+
+    auto& statistics = m_store->ensureResourceStatisticsForPrimaryDomain(primaryDomain(url));
+    
+    statistics.hadUserInteraction = false;
+    statistics.mostRecentUserInteraction = 0;
+}
+
+bool ResourceLoadObserver::hasHadUserInteraction(const URL& url)
+{
+    if (url.isBlankURL() || url.isEmpty())
+        return false;
+
+    auto& statistics = m_store->ensureResourceStatisticsForPrimaryDomain(primaryDomain(url));
     
+    return m_store->hasHadRecentUserInteraction(statistics);
+}
+
+void ResourceLoadObserver::setPrevalentResource(const URL& url)
+{
+    if (url.isBlankURL() || url.isEmpty())
+        return;
+
+    auto& statistics = m_store->ensureResourceStatisticsForPrimaryDomain(primaryDomain(url));
+    
+    statistics.isPrevalentResource = true;
+}
+
+bool ResourceLoadObserver::isPrevalentResource(const URL& url)
+{
+    if (url.isBlankURL() || url.isEmpty())
+        return false;
+
+    auto& statistics = m_store->ensureResourceStatisticsForPrimaryDomain(primaryDomain(url));
+    
+    return statistics.isPrevalentResource;
+}
+    
+void ResourceLoadObserver::clearPrevalentResource(const URL& url)
+{
+    if (url.isBlankURL() || url.isEmpty())
+        return;
+
+    auto& statistics = m_store->ensureResourceStatisticsForPrimaryDomain(primaryDomain(url));
+    
+    statistics.isPrevalentResource = false;
+}
+
+void ResourceLoadObserver::setTimeToLiveUserInteraction(double seconds)
+{
+    m_store->setTimeToLiveUserInteraction(seconds);
+}
+
+void ResourceLoadObserver::fireDataModificationHandler()
+{
+    m_store->fireDataModificationHandler();
+}
+
 String ResourceLoadObserver::primaryDomain(const URL& url)
 {
     String primaryDomain;
index 33124e8..c8cce7b 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2016 Apple Inc.  All rights reserved.
+ * Copyright (C) 2016-2017 Apple Inc.  All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -48,9 +48,22 @@ public:
     void logFrameNavigation(const Frame& frame, const Frame& topFrame, const ResourceRequest& newRequest, const ResourceResponse& redirectResponse);
     void logSubresourceLoading(const Frame*, const ResourceRequest& newRequest, const ResourceResponse& redirectResponse);
     void logWebSocketLoading(const Frame*, const URL&);
+    void logUserInteractionWithReducedTimeResolution(const Document&);
 
-    void logUserInteraction(const Document&);
-    
+    WEBCORE_EXPORT void logUserInteraction(const URL&);
+    WEBCORE_EXPORT bool hasHadUserInteraction(const URL&);
+    WEBCORE_EXPORT void clearUserInteraction(const URL&);
+
+    WEBCORE_EXPORT void setPrevalentResource(const URL&);
+    WEBCORE_EXPORT bool isPrevalentResource(const URL&);
+    WEBCORE_EXPORT void clearPrevalentResource(const URL&);
+
+    WEBCORE_EXPORT void setTimeToLiveUserInteraction(double seconds);
+    WEBCORE_EXPORT void setReducedTimestampResolution(double seconds);
+
+    WEBCORE_EXPORT void fireDataModificationHandler();
+
+    WEBCORE_EXPORT RefPtr<ResourceLoadStatisticsStore> statisticsStore();
     WEBCORE_EXPORT void setStatisticsStore(Ref<ResourceLoadStatisticsStore>&&);
 
     WEBCORE_EXPORT String statisticsForOrigin(const String&);
index f728a78..5aa1814 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2016 Apple Inc.  All rights reserved.
+ * Copyright (C) 2016-2017 Apple Inc.  All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -51,6 +51,8 @@ void ResourceLoadStatistics::encode(KeyedEncoder& encoder) const
     
     // User interaction
     encoder.encodeBool("hadUserInteraction", hadUserInteraction);
+    encoder.encodeDouble("mostRecentUserInteraction", mostRecentUserInteraction);
+    encoder.encodeBool("grandfathered", grandfathered);
     
     // Top frame stats
     encoder.encodeBool("topFrameHasBeenNavigatedToBefore", topFrameHasBeenNavigatedToBefore);
@@ -177,7 +179,16 @@ bool ResourceLoadStatistics::decode(KeyedDecoder& decoder, unsigned version)
 
     if (!decoder.decodeUInt32("dataRecordsRemoved", dataRecordsRemoved))
         return false;
-    
+
+    if (version < 3)
+        return true;
+
+    if (!decoder.decodeDouble("mostRecentUserInteraction", mostRecentUserInteraction))
+        return false;
+
+    if (!decoder.decodeBool("grandfathered", grandfathered))
+        return false;
+
     return true;
 }
 
@@ -214,6 +225,11 @@ String ResourceLoadStatistics::toString() const
     // User interaction
     appendBoolean(builder, "hadUserInteraction", hadUserInteraction);
     builder.append('\n');
+    builder.appendLiteral("    mostRecentUserInteraction: ");
+    builder.appendNumber(mostRecentUserInteraction);
+    builder.append('\n');
+    appendBoolean(builder, "    grandfathered", grandfathered);
+    builder.append('\n');
     
     // Top frame stats
     appendBoolean(builder, "topFrameHasBeenNavigatedToBefore", topFrameHasBeenNavigatedToBefore);
@@ -294,7 +310,19 @@ void ResourceLoadStatistics::merge(const ResourceLoadStatistics& other)
 {
     ASSERT(other.highLevelDomain == highLevelDomain);
 
-    hadUserInteraction |= other.hadUserInteraction;
+    if (!other.hadUserInteraction) {
+        // If user interaction has been reset do so here too.
+        // Else, do nothing.
+        if (!other.mostRecentUserInteraction) {
+            hadUserInteraction = false;
+            mostRecentUserInteraction = 0;
+        }
+    } else {
+        hadUserInteraction = true;
+        if (mostRecentUserInteraction < other.mostRecentUserInteraction)
+            mostRecentUserInteraction = other.mostRecentUserInteraction;
+    }
+    grandfathered |= other.grandfathered;
     
     // Top frame stats
     topFrameHasBeenRedirectedTo += other.topFrameHasBeenRedirectedTo;
index c0c81f7..1ec25e2 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2016 Apple Inc.  All rights reserved.
+ * Copyright (C) 2016-2017 Apple Inc.  All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -53,7 +53,10 @@ struct ResourceLoadStatistics {
 
     // User interaction
     bool hadUserInteraction { false };
-    
+    // Timestamp. Default value is negative, 0 means it was reset.
+    double mostRecentUserInteraction { -1 };
+    bool grandfathered { false };
+
     // Top frame stats
     unsigned topFrameHasBeenRedirectedTo { 0 };
     unsigned topFrameHasBeenRedirectedFrom { 0 };
index 4650dc1..16f4567 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2016 Apple Inc. All rights reserved.
+ * Copyright (C) 2016-2017 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
 #include "ResourceLoadStatistics.h"
 #include "SharedBuffer.h"
 #include "URL.h"
+#include <wtf/CurrentTime.h>
 #include <wtf/NeverDestroyed.h>
 
 namespace WebCore {
 
-static const auto statisticsModelVersion = 2;
+static const auto statisticsModelVersion = 3;
+// 30 days in seconds
+static auto timeToLiveUserInteraction = 2592000;
 
 Ref<ResourceLoadStatisticsStore> ResourceLoadStatisticsStore::create()
 {
@@ -144,17 +147,40 @@ void ResourceLoadStatisticsStore::fireDataModificationHandler()
         m_dataAddedHandler();
 }
 
+void ResourceLoadStatisticsStore::setTimeToLiveUserInteraction(double seconds)
+{
+    if (seconds >= 0)
+        timeToLiveUserInteraction = seconds;
+}
+
 void ResourceLoadStatisticsStore::processStatistics(std::function<void(ResourceLoadStatistics&)>&& processFunction)
 {
     for (auto& resourceStatistic : m_resourceStatisticsMap.values())
         processFunction(resourceStatistic);
 }
 
+bool ResourceLoadStatisticsStore::hasHadRecentUserInteraction(ResourceLoadStatistics& resourceStatistic)
+{
+    if (!resourceStatistic.hadUserInteraction)
+        return false;
+
+    if (currentTime() > resourceStatistic.mostRecentUserInteraction + timeToLiveUserInteraction) {
+        // Drop privacy sensitive data because we no longer need it.
+        // Set timestamp to 0.0 so that statistics merge will know
+        // it has been reset as opposed to its default -1.
+        resourceStatistic.mostRecentUserInteraction = 0;
+        resourceStatistic.hadUserInteraction = false;
+        return false;
+    }
+
+    return true;
+}
+
 Vector<String> ResourceLoadStatisticsStore::prevalentResourceDomainsWithoutUserInteraction()
 {
     Vector<String> prevalentResources;
     for (auto& resourceStatistic : m_resourceStatisticsMap.values()) {
-        if (resourceStatistic.isPrevalentResource && !resourceStatistic.hadUserInteraction)
+        if (resourceStatistic.isPrevalentResource && !hasHadRecentUserInteraction(resourceStatistic))
             prevalentResources.append(resourceStatistic.highLevelDomain);
     }
     return prevalentResources;
index 5b9e9f6..fcebdf5 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2016 Apple Inc.  All rights reserved.
+ * Copyright (C) 2016-2017 Apple Inc.  All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -59,9 +59,11 @@ public:
     WEBCORE_EXPORT void setNotificationCallback(std::function<void()> handler);
 
     void fireDataModificationHandler();
+    void setTimeToLiveUserInteraction(double seconds);
 
     WEBCORE_EXPORT void processStatistics(std::function<void(ResourceLoadStatistics&)>&&);
 
+    WEBCORE_EXPORT bool hasHadRecentUserInteraction(ResourceLoadStatistics&);
     WEBCORE_EXPORT Vector<String> prevalentResourceDomainsWithoutUserInteraction();
     WEBCORE_EXPORT void updateStatisticsForRemovedDataRecords(const Vector<String>& prevalentResourceDomains);
 private:
index 31c5052..0099076 100644 (file)
@@ -1,3 +1,130 @@
+2017-02-10  John Wilander  <wilander@apple.com>
+
+        Updates to Resource Load Statistics: Get the right website data store and introduce timeout for user interaction
+        https://bugs.webkit.org/show_bug.cgi?id=167474
+        <rdar://problem/24681808>
+        <rdar://problem/24703286>
+        <rdar://problem/30290270>
+
+        This patch does the following:
+        1. Gets the right website data store. API::WebsiteDataStore::defaultDataStore()
+            does not provide the right data store.
+        2. Introduces timeout for user interaction. A domain needs interaction every 30
+            days to stay in that category.
+        3. Adds grandfathered to the statistics model in preparation for grandfathering of
+            existing data records.
+        4. Adds test infrastructure to allow testing of the various rules in place for
+            data records removal.
+        5. Fixes various smaller bugs that were found as part of setting up the tests.
+        6. Regresses the data records removal counting. We need to come up with a thread
+            safe way of gathering removal statistics from more than one data store now
+            that we potentially interact with multiple stores.
+        7. Adds a first set of layout tests for resource load statistics.
+
+        Reviewed by Andy Estes.
+
+        * PlatformEfl.cmake:
+            Added UIProcess/WebResourceLoadStatisticsManager.cpp and
+            UIProcess/API/C/WKResourceLoadStatisticsManager.cpp.
+        * PlatformGTK.cmake:
+            Added UIProcess/WebResourceLoadStatisticsManager.cpp and
+            UIProcess/API/C/WKResourceLoadStatisticsManager.cpp.
+        * PlatformMac.cmake:
+            Added UIProcess/WebResourceLoadStatisticsManager.cpp and
+            UIProcess/API/C/WKResourceLoadStatisticsManager.cpp.
+        * Shared/API/APIObject.h:
+            Adds WebResourceLoadStatisticsManager to the Type enum.
+        * Shared/API/c/WKBase.h:
+            Typedef of WKResourceLoadStatisticsManagerRef.
+        * Shared/WebCoreArgumentCoders.cpp:
+        (IPC::ArgumentCoder<ResourceLoadStatistics>::encode):
+        (IPC::ArgumentCoder<ResourceLoadStatistics>::decode):
+            Adds support for statistics mostRecentUserInteraction, grandfathered, and
+            dataRecordsRemoved.
+        * UIProcess/API/C/WKAPICast.h:
+            Adds API mapping between WKResourceLoadStatisticsManagerRef and 
+            WebResourceLoadStatisticsManager.
+        * UIProcess/API/C/WKResourceLoadStatisticsManager.cpp: Added.
+        (WKResourceLoadStatisticsManagerGetTypeID):
+        (WKResourceLoadStatisticsManagerSetPrevalentResource):
+        (WKResourceLoadStatisticsManagerIsPrevalentResource):
+        (WKResourceLoadStatisticsManagerSetHasHadUserInteraction):
+        (WKResourceLoadStatisticsManagerIsHasHadUserInteraction):
+        (WKResourceLoadStatisticsManagerSetTimeToLiveUserInteraction):
+        (WKResourceLoadStatisticsManagerFireDataModificationHandler):
+        (WKResourceLoadStatisticsManagerSetNotifyPagesWhenDataRecordsWereScanned):
+        (WKResourceLoadStatisticsManagerSetShouldClassifyResourcesBeforeDataRecordsRemoval):
+        (WKResourceLoadStatisticsManagerSetMinimumTimeBetweeenDataRecordsRemoval):
+        (WKResourceLoadStatisticsManagerResetToConsistentState):
+            API level bridge to WebResourceLoadStatisticsManager functions.
+        * UIProcess/API/C/WKResourceLoadStatisticsManager.h: Added.
+        * UIProcess/WebProcessProxy.cpp:
+        (WebKit::WebProcessProxy::deleteWebsiteDataForTopPrivatelyOwnedDomainsInAllPersistentDataStores):
+            A way to delete website data in persistent store(s) configured by the 
+            embedding client. API::WebsiteDataStore::defaultDataStore() does not 
+            do this which caused us to skip session storage.
+        * UIProcess/WebProcessProxy.h:
+        * UIProcess/WebResourceLoadStatisticsManager.cpp: Added.
+        (WebKit::WebResourceLoadStatisticsManager::setPrevalentResource):
+        (WebKit::WebResourceLoadStatisticsManager::isPrevalentResource):
+        (WebKit::WebResourceLoadStatisticsManager::setHasHadUserInteraction):
+        (WebKit::WebResourceLoadStatisticsManager::hasHadUserInteraction):
+        (WebKit::WebResourceLoadStatisticsManager::setTimeToLiveUserInteraction):
+        (WebKit::WebResourceLoadStatisticsManager::fireDataModificationHandler):
+        (WebKit::WebResourceLoadStatisticsManager::setNotifyPagesWhenDataRecordsWereScanned):
+        (WebKit::WebResourceLoadStatisticsManager::setShouldClassifyResourcesBeforeDataRecordsRemoval):
+        (WebKit::WebResourceLoadStatisticsManager::setMinimumTimeBetweeenDataRecordsRemoval):
+        (WebKit::WebResourceLoadStatisticsManager::resetToConsistentState):
+            Static functions to manage the state and behavior of the UI process' 
+            resource load statistics store.
+        * UIProcess/WebResourceLoadStatisticsManager.h: Added.
+        (WebKit::WebResourceLoadStatisticsManager::create):
+        * UIProcess/WebResourceLoadStatisticsStore.cpp:
+        (WebKit::WebResourceLoadStatisticsStore::WebResourceLoadStatisticsStore):
+            An effect of a member variable rename. See header file comment below.
+        (WebKit::WebResourceLoadStatisticsStore::setNotifyPagesWhenDataRecordsWereScanned):
+        (WebKit::WebResourceLoadStatisticsStore::setShouldClassifyResourcesBeforeDataRecordsRemoval):
+        (WebKit::WebResourceLoadStatisticsStore::setMinimumTimeBetweeenDataRecordsRemoval):
+            Three static functions for controlling the behavior of the store.
+        (WebKit::WebResourceLoadStatisticsStore::removeDataRecords):
+            Changes:
+            1. Covers all data record types.
+            2. Allows data removal when m_lastTimeDataRecordsWereRemoved is not set.
+            3. Moves the heavy lifting to WebsiteDataStore (see below).
+            4. No longer counts removed data regressions.
+        (WebKit::WebResourceLoadStatisticsStore::processStatisticsAndDataRecords):
+            See next comment.
+        (WebKit::WebResourceLoadStatisticsStore::resourceLoadStatisticsUpdated):
+            WebResourceLoadStatisticsStore::processStatisticsAndDataRecords() is 
+            broken out of WebResourceLoadStatisticsStore::resourceLoadStatisticsUpdated
+            to allow for statistics processing without prior classification.
+            Tests need to set statistics and a run of the classifier may
+            reset the data.
+        (WebKit::WebResourceLoadStatisticsStore::registerSharedResourceLoadObserver):
+            Sets the notification callback which calls 
+            WebResourceLoadStatisticsStore::processStatisticsAndDataRecords().
+        * UIProcess/WebResourceLoadStatisticsStore.h:
+        (WebKit::WebResourceLoadStatisticsStore::coreStore):
+            Renamed m_resourceStatisticsStore to m_resourceLoadStatisticsStore to
+            harmonize with other naming.
+        * UIProcess/WebsiteData/WebsiteDataStore.cpp:
+        (WebKit::WebsiteDataStore::fetchDataForTopPrivatelyOwnedDomains):
+            This is now where matching is made between prevalent resources
+            and data records. This function is used by 
+            WebsiteDataStore::removeDataForTopPrivatelyOwnedDomains().
+        (WebKit::WebsiteDataStore::removeDataForTopPrivatelyOwnedDomains):
+            The function WebResourceLoadStatisticsStore::removeDataRecords()
+            calls to get data records removed in all persistent stores.
+        * UIProcess/WebsiteData/WebsiteDataStore.h:
+        * WebKit2.xcodeproj/project.pbxproj:
+        * WebProcess/WebProcess.cpp:
+        (WebKit::m_resourceLoadStatisticsStore):
+        (WebKit::WebProcess::statisticsChangedTimerFired):
+        (WebKit::m_resourceLoadStatisticsStorage): Deleted.
+            Renamed m_resourceLoadStatisticsStorage to m_resourceLoadStatisticsStore 
+            to harmonize with other naming.
+        * WebProcess/WebProcess.h:
+
 2017-02-10  Simon Fraser  <simon.fraser@apple.com>
 
         Zero out some WKWebView state in the case of a web content crash
index e748545..f7d36bb 100644 (file)
@@ -62,10 +62,13 @@ list(APPEND WebKit2_SOURCES
     UIProcess/BackingStore.cpp
     UIProcess/DefaultUndoController.cpp
     UIProcess/LegacySessionStateCodingNone.cpp
+    UIProcess/WebResourceLoadStatisticsManager.cpp
     UIProcess/WebResourceLoadStatisticsStore.cpp
     UIProcess/WebTextChecker.cpp
     UIProcess/WebTextCheckerClient.cpp
 
+    UIProcess/API/C/WKResourceLoadStatisticsManager.cpp
+
     UIProcess/API/C/CoordinatedGraphics/WKView.cpp
 
     UIProcess/API/C/cairo/WKIconDatabaseCairo.cpp
index 9bef066..2e9067e 100644 (file)
@@ -88,10 +88,13 @@ list(APPEND WebKit2_SOURCES
     UIProcess/DefaultUndoController.cpp
     UIProcess/DrawingAreaProxyImpl.cpp
     UIProcess/LegacySessionStateCodingNone.cpp
+    UIProcess/WebResourceLoadStatisticsManager.cpp
     UIProcess/WebResourceLoadStatisticsStore.cpp
     UIProcess/WebTextChecker.cpp
     UIProcess/WebTextCheckerClient.cpp
 
+    UIProcess/API/C/WKResourceLoadStatisticsManager.cpp
+
     UIProcess/API/C/cairo/WKIconDatabaseCairo.cpp
 
     UIProcess/API/C/gtk/WKFullScreenClientGtk.cpp
index 202055c..e80b8d6 100644 (file)
@@ -160,6 +160,7 @@ list(APPEND WebKit2_SOURCES
     UIProcess/HighPerformanceGraphicsUsageSampler.cpp
     UIProcess/PerActivityStateCPUUsageSampler.cpp
     UIProcess/WebContextMenuListenerProxy.cpp
+    UIProcess/WebResourceLoadStatisticsManager.cpp
     UIProcess/WebResourceLoadStatisticsStore.cpp
 
     UIProcess/Automation/WebAutomationSession.cpp
@@ -172,6 +173,8 @@ list(APPEND WebKit2_SOURCES
     UIProcess/API/APIUserStyleSheet.cpp
     UIProcess/API/APIWebsiteDataRecord.cpp
 
+    UIProcess/API/C/WKResourceLoadStatisticsManager.cpp
+
     UIProcess/API/C/mac/WKContextPrivateMac.mm
     UIProcess/API/C/mac/WKPagePrivateMac.mm
 
index 2bd4849..a0ad04f 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2010 Apple Inc. All rights reserved.
+ * Copyright (C) 2010-2017 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -154,6 +154,7 @@ public:
         Vibration,
         ViewportAttributes,
         VisitedLinkStore,
+        WebResourceLoadStatisticsManager,
         WebsiteDataRecord,
         WebsiteDataStore,
         WebsitePolicies,
index 795d900..e9df787 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2010, 2011 Apple Inc. All rights reserved.
+ * Copyright (C) 2010-2017 Apple Inc. All rights reserved.
  * Portions Copyright (c) 2010 Motorola Mobility, Inc.  All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
@@ -127,6 +127,7 @@ typedef const struct OpaqueWKPageRunBeforeUnloadConfirmPanelResultListener* WKPa
 typedef const struct OpaqueWKPageRunJavaScriptAlertResultListener* WKPageRunJavaScriptAlertResultListenerRef;
 typedef const struct OpaqueWKPageRunJavaScriptConfirmResultListener* WKPageRunJavaScriptConfirmResultListenerRef;
 typedef const struct OpaqueWKPageRunJavaScriptPromptResultListener* WKPageRunJavaScriptPromptResultListenerRef;
+typedef const struct OpaqueWKResourceLoadStatisticsManager* WKResourceLoadStatisticsManagerRef;
 typedef const struct OpaqueWKTextChecker* WKTextCheckerRef;
 typedef const struct OpaqueWKSession* WKSessionRef;
 typedef const struct OpaqueWKSessionState* WKSessionStateRef;
index 3542e72..74cc5bc 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2011-2016 Apple Inc. All rights reserved.
+ * Copyright (C) 2011-2017 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -2247,6 +2247,8 @@ void ArgumentCoder<ResourceLoadStatistics>::encode(Encoder& encoder, const WebCo
     
     // User interaction
     encoder << statistics.hadUserInteraction;
+    encoder << statistics.mostRecentUserInteraction;
+    encoder << statistics.grandfathered;
     
     // Top frame stats
     encoder << statistics.topFrameHasBeenNavigatedToBefore;
@@ -2277,6 +2279,7 @@ void ArgumentCoder<ResourceLoadStatistics>::encode(Encoder& encoder, const WebCo
     // Prevalent Resource
     encoder << statistics.redirectedToOtherPrevalentResourceOrigins;
     encoder << statistics.isPrevalentResource;
+    encoder << statistics.dataRecordsRemoved;
 }
 
 bool ArgumentCoder<ResourceLoadStatistics>::decode(Decoder& decoder, WebCore::ResourceLoadStatistics& statistics)
@@ -2287,6 +2290,12 @@ bool ArgumentCoder<ResourceLoadStatistics>::decode(Decoder& decoder, WebCore::Re
     // User interaction
     if (!decoder.decode(statistics.hadUserInteraction))
         return false;
+
+    if (!decoder.decode(statistics.mostRecentUserInteraction))
+        return false;
+
+    if (!decoder.decode(statistics.grandfathered))
+        return false;
     
     // Top frame stats
     if (!decoder.decode(statistics.topFrameHasBeenNavigatedToBefore))
@@ -2357,7 +2366,10 @@ bool ArgumentCoder<ResourceLoadStatistics>::decode(Decoder& decoder, WebCore::Re
     
     if (!decoder.decode(statistics.isPrevalentResource))
         return false;
-    
+
+    if (!decoder.decode(statistics.dataRecordsRemoved))
+        return false;
+
     return true;
 }
 
index 5e251e7..7946cac 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2010, 2011 Apple Inc. All rights reserved.
+ * Copyright (C) 2010-2017 Apple Inc. All rights reserved.
  * Portions Copyright (c) 2010 Motorola Mobility, Inc.  All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
@@ -107,6 +107,7 @@ class WebProcessPool;
 class WebProtectionSpace;
 class WebRenderLayer;
 class WebRenderObject;
+class WebResourceLoadStatisticsManager;
 class WebTextChecker;
 class WebUserContentControllerProxy;
 class WebVibrationProxy;
@@ -156,6 +157,7 @@ WK_ADD_API_MAPPING(WKPreferencesRef, WebPreferences)
 WK_ADD_API_MAPPING(WKProtectionSpaceRef, WebProtectionSpace)
 WK_ADD_API_MAPPING(WKRenderLayerRef, WebRenderLayer)
 WK_ADD_API_MAPPING(WKRenderObjectRef, WebRenderObject)
+WK_ADD_API_MAPPING(WKResourceLoadStatisticsManagerRef, WebResourceLoadStatisticsManager)
 WK_ADD_API_MAPPING(WKSessionStateRef, API::SessionState)
 WK_ADD_API_MAPPING(WKTextCheckerRef, WebTextChecker)
 WK_ADD_API_MAPPING(WKUserContentControllerRef, WebUserContentControllerProxy)
diff --git a/Source/WebKit2/UIProcess/API/C/WKResourceLoadStatisticsManager.cpp b/Source/WebKit2/UIProcess/API/C/WKResourceLoadStatisticsManager.cpp
new file mode 100644 (file)
index 0000000..879abeb
--- /dev/null
@@ -0,0 +1,87 @@
+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "WKResourceLoadStatisticsManager.h"
+
+#include "WKAPICast.h"
+#include "WebResourceLoadStatisticsManager.h"
+
+using namespace WebKit;
+
+WKTypeID WKResourceLoadStatisticsManagerGetTypeID()
+{
+    return toAPI(WebResourceLoadStatisticsManager::APIType);
+}
+
+void WKResourceLoadStatisticsManagerSetPrevalentResource(WKStringRef hostName, bool value)
+{
+    WebResourceLoadStatisticsManager::setPrevalentResource(toWTFString(hostName), value);
+}
+
+bool WKResourceLoadStatisticsManagerIsPrevalentResource(WKStringRef hostName)
+{
+    return WebResourceLoadStatisticsManager::isPrevalentResource(toWTFString(hostName));
+}
+
+void WKResourceLoadStatisticsManagerSetHasHadUserInteraction(WKStringRef hostName, bool value)
+{
+    WebResourceLoadStatisticsManager::setHasHadUserInteraction(toWTFString(hostName), value);
+}
+
+bool WKResourceLoadStatisticsManagerIsHasHadUserInteraction(WKStringRef hostName)
+{
+    return WebResourceLoadStatisticsManager::hasHadUserInteraction(toWTFString(hostName));
+}
+
+void WKResourceLoadStatisticsManagerSetTimeToLiveUserInteraction(double seconds)
+{
+    WebResourceLoadStatisticsManager::setTimeToLiveUserInteraction(seconds);
+}
+
+void WKResourceLoadStatisticsManagerFireDataModificationHandler()
+{
+    WebResourceLoadStatisticsManager::fireDataModificationHandler();
+}
+
+void WKResourceLoadStatisticsManagerSetNotifyPagesWhenDataRecordsWereScanned(bool value)
+{
+    WebResourceLoadStatisticsManager::setNotifyPagesWhenDataRecordsWereScanned(value);
+}
+
+void WKResourceLoadStatisticsManagerSetShouldClassifyResourcesBeforeDataRecordsRemoval(bool value)
+{
+    WebResourceLoadStatisticsManager::setShouldClassifyResourcesBeforeDataRecordsRemoval(value);
+}
+
+void WKResourceLoadStatisticsManagerSetMinimumTimeBetweeenDataRecordsRemoval(double seconds)
+{
+    WebResourceLoadStatisticsManager::setMinimumTimeBetweeenDataRecordsRemoval(seconds);
+}
+
+void WKResourceLoadStatisticsManagerResetToConsistentState()
+{
+    WebResourceLoadStatisticsManager::resetToConsistentState();
+}
diff --git a/Source/WebKit2/UIProcess/API/C/WKResourceLoadStatisticsManager.h b/Source/WebKit2/UIProcess/API/C/WKResourceLoadStatisticsManager.h
new file mode 100644 (file)
index 0000000..2b83f5c
--- /dev/null
@@ -0,0 +1,51 @@
+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#include <WebKit/WKBase.h>
+#include <wtf/text/WTFString.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+    
+    WK_EXPORT WKTypeID WKResourceLoadStatisticsManagerGetTypeID();
+    
+    WK_EXPORT void WKResourceLoadStatisticsManagerSetPrevalentResource(WKStringRef hostName, bool value);
+    WK_EXPORT bool WKResourceLoadStatisticsManagerIsPrevalentResource(WKStringRef hostName);
+    WK_EXPORT void WKResourceLoadStatisticsManagerSetHasHadUserInteraction(WKStringRef hostName, bool value);
+    WK_EXPORT bool WKResourceLoadStatisticsManagerIsHasHadUserInteraction(WKStringRef hostName);
+    WK_EXPORT void WKResourceLoadStatisticsManagerSetTimeToLiveUserInteraction(double seconds);
+    WK_EXPORT void WKResourceLoadStatisticsManagerSetReducedTimestampResolution(double seconds);
+    WK_EXPORT void WKResourceLoadStatisticsManagerFireDataModificationHandler();
+    WK_EXPORT void WKResourceLoadStatisticsManagerSetNotifyPagesWhenDataRecordsWereScanned(bool value);
+    WK_EXPORT void WKResourceLoadStatisticsManagerSetShouldClassifyResourcesBeforeDataRecordsRemoval(bool value);
+    WK_EXPORT void WKResourceLoadStatisticsManagerSetMinimumTimeBetweeenDataRecordsRemoval(double seconds);
+    WK_EXPORT void WKResourceLoadStatisticsManagerResetToConsistentState();
+
+#ifdef __cplusplus
+}
+#endif
index 75ddec6..e194cda 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2010-2016 Apple Inc. All rights reserved.
+ * Copyright (C) 2010-2017 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -200,6 +200,53 @@ WebPageProxy* WebProcessProxy::webPage(uint64_t pageID)
     return globalPageMap().get(pageID);
 }
 
+void WebProcessProxy::deleteWebsiteDataForTopPrivatelyOwnedDomainsInAllPersistentDataStores(OptionSet<WebsiteDataType> dataTypes, Vector<String>& topPrivatelyOwnedDomains, bool shouldNotifyPage, std::function<void()> completionHandler)
+{
+    struct CallbackAggregator : ThreadSafeRefCounted<CallbackAggregator> {
+        explicit CallbackAggregator(std::function<void()> completionHandler)
+            : completionHandler(WTFMove(completionHandler))
+        {
+        }
+        
+        void addPendingCallback()
+        {
+            ++pendingCallbacks;
+        }
+        
+        void removePendingCallback()
+        {
+            ASSERT(pendingCallbacks);
+            --pendingCallbacks;
+            
+            callIfNeeded();
+        }
+        
+        void callIfNeeded()
+        {
+            if (!pendingCallbacks)
+                completionHandler();
+        }
+        
+        unsigned pendingCallbacks = 0;
+        std::function<void()> completionHandler;
+    };
+    
+    RefPtr<CallbackAggregator> callbackAggregator = adoptRef(new CallbackAggregator(WTFMove(completionHandler)));
+
+    for (auto& page : globalPageMap()) {
+        if (!page.value->websiteDataStore().isPersistent())
+            continue;
+        callbackAggregator->addPendingCallback();
+        page.value->websiteDataStore().removeDataForTopPrivatelyOwnedDomains(dataTypes, { }, topPrivatelyOwnedDomains, [callbackAggregator, shouldNotifyPage, page]() {
+            if (shouldNotifyPage)
+                page.value->postMessageToInjectedBundle("WebsiteDataDeletionForTopPrivatelyOwnedDomainsFinished", nullptr);
+            WTF::RunLoop::main().dispatch([callbackAggregator] {
+                callbackAggregator->removePendingCallback();
+            });
+        });
+    }
+}
+
 Ref<WebPageProxy> WebProcessProxy::createWebPage(PageClient& pageClient, Ref<API::PageConfiguration>&& pageConfiguration)
 {
     uint64_t pageID = generatePageID();
index 6dcd65f..158467a 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2010-2016 Apple Inc. All rights reserved.
+ * Copyright (C) 2010-2017 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -120,6 +120,7 @@ public:
     void fetchWebsiteData(WebCore::SessionID, OptionSet<WebsiteDataType>, Function<void(WebsiteData)> completionHandler);
     void deleteWebsiteData(WebCore::SessionID, OptionSet<WebsiteDataType>, std::chrono::system_clock::time_point modifiedSince, Function<void()> completionHandler);
     void deleteWebsiteDataForOrigins(WebCore::SessionID, OptionSet<WebsiteDataType>, const Vector<WebCore::SecurityOriginData>&, Function<void()> completionHandler);
+    static void deleteWebsiteDataForTopPrivatelyOwnedDomainsInAllPersistentDataStores(OptionSet<WebsiteDataType>, Vector<String>& topPrivatelyOwnedDomains, bool shouldNotifyPages, std::function<void()> completionHandler);
 
     void enableSuddenTermination();
     void disableSuddenTermination();
diff --git a/Source/WebKit2/UIProcess/WebResourceLoadStatisticsManager.cpp b/Source/WebKit2/UIProcess/WebResourceLoadStatisticsManager.cpp
new file mode 100644 (file)
index 0000000..ddc8cc3
--- /dev/null
@@ -0,0 +1,100 @@
+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "WebResourceLoadStatisticsManager.h"
+
+#include "Logging.h"
+#include "WebResourceLoadStatisticsStore.h"
+#include <WebCore/ResourceLoadObserver.h>
+#include <WebCore/URL.h>
+
+using namespace WebCore;
+
+namespace WebKit {
+
+void WebResourceLoadStatisticsManager::setPrevalentResource(const String& hostName, bool value)
+{
+    if (value)
+        WebCore::ResourceLoadObserver::sharedObserver().setPrevalentResource(URL(URL(), hostName));
+    else
+        WebCore::ResourceLoadObserver::sharedObserver().clearPrevalentResource(URL(URL(), hostName));
+}
+
+bool WebResourceLoadStatisticsManager::isPrevalentResource(const String& hostName)
+{
+    return WebCore::ResourceLoadObserver::sharedObserver().isPrevalentResource(URL(URL(), hostName));
+}
+    
+void WebResourceLoadStatisticsManager::setHasHadUserInteraction(const String& hostName, bool value)
+{
+    if (value)
+        WebCore::ResourceLoadObserver::sharedObserver().logUserInteraction(URL(URL(), hostName));
+    else
+        WebCore::ResourceLoadObserver::sharedObserver().clearUserInteraction(URL(URL(), hostName));
+}
+
+bool WebResourceLoadStatisticsManager::hasHadUserInteraction(const String& hostName)
+{
+    return WebCore::ResourceLoadObserver::sharedObserver().hasHadUserInteraction(URL(URL(), hostName));
+}
+
+void WebResourceLoadStatisticsManager::setTimeToLiveUserInteraction(double seconds)
+{
+    WebCore::ResourceLoadObserver::sharedObserver().setTimeToLiveUserInteraction(seconds);
+}
+
+void WebResourceLoadStatisticsManager::fireDataModificationHandler()
+{
+    WebCore::ResourceLoadObserver::sharedObserver().fireDataModificationHandler();
+}
+
+void WebResourceLoadStatisticsManager::setNotifyPagesWhenDataRecordsWereScanned(bool value)
+{
+    WebResourceLoadStatisticsStore::setNotifyPagesWhenDataRecordsWereScanned(value);
+}
+
+void WebResourceLoadStatisticsManager::setShouldClassifyResourcesBeforeDataRecordsRemoval(bool value)
+{
+    WebResourceLoadStatisticsStore::setShouldClassifyResourcesBeforeDataRecordsRemoval(value);
+}
+
+void WebResourceLoadStatisticsManager::setMinimumTimeBetweeenDataRecordsRemoval(double seconds)
+{
+    WebResourceLoadStatisticsStore::setMinimumTimeBetweeenDataRecordsRemoval(seconds);
+}
+
+void WebResourceLoadStatisticsManager::resetToConsistentState()
+{
+    WebCore::ResourceLoadObserver::sharedObserver().setTimeToLiveUserInteraction(2592000);
+    WebResourceLoadStatisticsStore::setNotifyPagesWhenDataRecordsWereScanned(false);
+    WebResourceLoadStatisticsStore::setShouldClassifyResourcesBeforeDataRecordsRemoval(true);
+    WebResourceLoadStatisticsStore::setMinimumTimeBetweeenDataRecordsRemoval(60);
+    auto store = WebCore::ResourceLoadObserver::sharedObserver().statisticsStore();
+    if (store)
+        store->clear();
+}
+    
+} // namespace WebKit
diff --git a/Source/WebKit2/UIProcess/WebResourceLoadStatisticsManager.h b/Source/WebKit2/UIProcess/WebResourceLoadStatisticsManager.h
new file mode 100644 (file)
index 0000000..0e8102f
--- /dev/null
@@ -0,0 +1,56 @@
+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#include "APIObject.h"
+
+#include <wtf/RefPtr.h>
+#include <wtf/text/WTFString.h>
+
+namespace WebKit {
+
+class WebResourceLoadStatisticsManager : public API::ObjectImpl<API::Object::Type::WebResourceLoadStatisticsManager> {
+public:
+    static Ref<WebResourceLoadStatisticsManager> create()
+    {
+        return adoptRef(*new WebResourceLoadStatisticsManager());
+    }
+    static void setPrevalentResource(const String& hostName, bool value);
+    static bool isPrevalentResource(const String& hostName);
+    static void setHasHadUserInteraction(const String& hostName, bool value);
+    static bool hasHadUserInteraction(const String& hostName);
+    static void setTimeToLiveUserInteraction(double seconds);
+    static void setReducedTimestampResolution(double seconds);
+    static void fireDataModificationHandler();
+    static void setNotifyPagesWhenDataRecordsWereScanned(bool);
+    static void setShouldClassifyResourcesBeforeDataRecordsRemoval(bool value);
+    static void setMinimumTimeBetweeenDataRecordsRemoval(double seconds);
+    static void resetToConsistentState();
+
+private:
+};
+
+} // namespace WebKit
index 1ca282a..7871acf 100644 (file)
@@ -26,9 +26,9 @@
 #include "config.h"
 #include "WebResourceLoadStatisticsStore.h"
 
-#include "APIWebsiteDataStore.h"
 #include "WebProcessMessages.h"
 #include "WebProcessPool.h"
+#include "WebProcessProxy.h"
 #include "WebResourceLoadStatisticsStoreMessages.h"
 #include "WebsiteDataFetchOption.h"
 #include "WebsiteDataType.h"
@@ -45,9 +45,11 @@ using namespace WebCore;
 
 namespace WebKit {
 
-static const auto numberOfSecondsBetweenRemovingDataRecords = 60;
 static const auto featureVectorLengthThreshold = 3;
+static auto minimumTimeBetweeenDataRecordsRemoval = 60;
 static OptionSet<WebKit::WebsiteDataType> dataTypesToRemove;
+static auto notifyPages = false;
+static auto shouldClassifyResourcesBeforeDataRecordsRemoval = true;
 
 Ref<WebResourceLoadStatisticsStore> WebResourceLoadStatisticsStore::create(const String& resourceLoadStatisticsDirectory)
 {
@@ -55,7 +57,7 @@ Ref<WebResourceLoadStatisticsStore> WebResourceLoadStatisticsStore::create(const
 }
 
 WebResourceLoadStatisticsStore::WebResourceLoadStatisticsStore(const String& resourceLoadStatisticsDirectory)
-    : m_resourceStatisticsStore(ResourceLoadStatisticsStore::create())
+    : m_resourceLoadStatisticsStore(ResourceLoadStatisticsStore::create())
     , m_statisticsQueue(WorkQueue::create("WebResourceLoadStatisticsStore Process Data Queue"))
     , m_storagePath(resourceLoadStatisticsDirectory)
 {
@@ -65,6 +67,22 @@ WebResourceLoadStatisticsStore::~WebResourceLoadStatisticsStore()
 {
 }
 
+void WebResourceLoadStatisticsStore::setNotifyPagesWhenDataRecordsWereScanned(bool always)
+{
+    notifyPages = always;
+}
+
+void WebResourceLoadStatisticsStore::setShouldClassifyResourcesBeforeDataRecordsRemoval(bool value)
+{
+    shouldClassifyResourcesBeforeDataRecordsRemoval = value;
+}
+
+void WebResourceLoadStatisticsStore::setMinimumTimeBetweeenDataRecordsRemoval(double seconds)
+{
+    if (seconds >= 0)
+        minimumTimeBetweeenDataRecordsRemoval = seconds;
+}
+
 bool WebResourceLoadStatisticsStore::hasPrevalentResourceCharacteristics(const ResourceLoadStatistics& resourceStatistic)
 {
     auto subresourceUnderTopFrameOriginsCount = resourceStatistic.subresourceUnderTopFrameOrigins.size();
@@ -111,12 +129,8 @@ void WebResourceLoadStatisticsStore::removeDataRecords()
         return;
 
     double now = currentTime();
-    if (!m_lastTimeDataRecordsWereRemoved) {
-        m_lastTimeDataRecordsWereRemoved = now;
-        return;
-    }
-
-    if (now < (m_lastTimeDataRecordsWereRemoved + numberOfSecondsBetweenRemovingDataRecords))
+    if (m_lastTimeDataRecordsWereRemoved
+        && now < m_lastTimeDataRecordsWereRemoved + minimumTimeBetweeenDataRecordsRemoval)
         return;
 
     m_dataRecordsRemovalPending = true;
@@ -124,59 +138,52 @@ void WebResourceLoadStatisticsStore::removeDataRecords()
 
     if (dataTypesToRemove.isEmpty()) {
         dataTypesToRemove |= WebsiteDataType::Cookies;
-        dataTypesToRemove |= WebsiteDataType::LocalStorage;
-        dataTypesToRemove |= WebsiteDataType::IndexedDBDatabases;
         dataTypesToRemove |= WebsiteDataType::DiskCache;
         dataTypesToRemove |= WebsiteDataType::MemoryCache;
+        dataTypesToRemove |= WebsiteDataType::OfflineWebApplicationCache;
+        dataTypesToRemove |= WebsiteDataType::SessionStorage;
+        dataTypesToRemove |= WebsiteDataType::LocalStorage;
+        dataTypesToRemove |= WebsiteDataType::WebSQLDatabases;
+        dataTypesToRemove |= WebsiteDataType::IndexedDBDatabases;
+        dataTypesToRemove |= WebsiteDataType::MediaKeys;
+        dataTypesToRemove |= WebsiteDataType::HSTSCache;
+        dataTypesToRemove |= WebsiteDataType::SearchFieldRecentSearches;
+#if ENABLE(NETSCAPE_PLUGIN_API)
+        dataTypesToRemove |= WebsiteDataType::PlugInData;
+#endif
+#if ENABLE(MEDIA_STREAM)
+        dataTypesToRemove |= WebsiteDataType::MediaDeviceIdentifier;
+#endif
     }
 
     // Switch to the main thread to get the default website data store
     RunLoop::main().dispatch([prevalentResourceDomains = WTFMove(prevalentResourceDomains), this] () mutable {
-        auto& websiteDataStore = API::WebsiteDataStore::defaultDataStore()->websiteDataStore();
-
-        websiteDataStore.fetchData(dataTypesToRemove, { }, [prevalentResourceDomains = WTFMove(prevalentResourceDomains), this](auto websiteDataRecords) {
-            Vector<WebsiteDataRecord> dataRecords;
-            Vector<String> prevalentResourceDomainsWithDataRecords;
-            for (auto& websiteDataRecord : websiteDataRecords) {
-                for (auto& prevalentResourceDomain : prevalentResourceDomains) {
-                    if (websiteDataRecord.displayName.endsWithIgnoringASCIICase(prevalentResourceDomain)) {
-                        auto suffixStart = websiteDataRecord.displayName.length() - prevalentResourceDomain.length();
-                        if (!suffixStart || websiteDataRecord.displayName[suffixStart - 1] == '.') {
-                            dataRecords.append(websiteDataRecord);
-                            prevalentResourceDomainsWithDataRecords.append(prevalentResourceDomain);
-                        }
-                    }
-                }
-            }
-
-            if (!dataRecords.size()) {
-                m_dataRecordsRemovalPending = false;
-                return;
-            }
-
-            auto& websiteDataStore = API::WebsiteDataStore::defaultDataStore()->websiteDataStore();
-            websiteDataStore.removeData(dataTypesToRemove, dataRecords, [prevalentResourceDomainsWithDataRecords = WTFMove(prevalentResourceDomainsWithDataRecords), this] {
-                this->coreStore().updateStatisticsForRemovedDataRecords(prevalentResourceDomainsWithDataRecords);
-                m_dataRecordsRemovalPending = false;
-            });
+        WebProcessProxy::deleteWebsiteDataForTopPrivatelyOwnedDomainsInAllPersistentDataStores(dataTypesToRemove, prevalentResourceDomains, notifyPages, [this]() mutable {
+            m_dataRecordsRemovalPending = false;
         });
     });
 }
 
-void WebResourceLoadStatisticsStore::resourceLoadStatisticsUpdated(const Vector<WebCore::ResourceLoadStatistics>& origins)
+void WebResourceLoadStatisticsStore::processStatisticsAndDataRecords()
 {
-    coreStore().mergeStatistics(origins);
-
-    coreStore().processStatistics([this] (ResourceLoadStatistics& resourceStatistic) {
-        classifyResource(resourceStatistic);
-        removeDataRecords();
-    });
+    if (shouldClassifyResourcesBeforeDataRecordsRemoval) {
+        coreStore().processStatistics([this] (ResourceLoadStatistics& resourceStatistic) {
+            classifyResource(resourceStatistic);
+        });
+    }
+    removeDataRecords();
     
     auto encoder = coreStore().createEncoderFromData();
     
     writeEncoderToDisk(*encoder.get(), "full_browsing_session");
 }
 
+void WebResourceLoadStatisticsStore::resourceLoadStatisticsUpdated(const Vector<WebCore::ResourceLoadStatistics>& origins)
+{
+    coreStore().mergeStatistics(origins);
+    processStatisticsAndDataRecords();
+}
+
 void WebResourceLoadStatisticsStore::setResourceLoadStatisticsEnabled(bool enabled)
 {
     if (enabled == m_resourceLoadStatisticsEnabled)
@@ -192,9 +199,15 @@ bool WebResourceLoadStatisticsStore::resourceLoadStatisticsEnabled() const
     return m_resourceLoadStatisticsEnabled;
 }
 
+
 void WebResourceLoadStatisticsStore::registerSharedResourceLoadObserver()
 {
-    ResourceLoadObserver::sharedObserver().setStatisticsStore(m_resourceStatisticsStore.copyRef());
+    ResourceLoadObserver::sharedObserver().setStatisticsStore(m_resourceLoadStatisticsStore.copyRef());
+    m_resourceLoadStatisticsStore->setNotificationCallback([this] {
+        if (m_resourceLoadStatisticsStore->isEmpty())
+            return;
+        processStatisticsAndDataRecords();
+    });
 }
 
 void WebResourceLoadStatisticsStore::readDataFromDiskIfNeeded()
index 060a887..5e49007 100644 (file)
@@ -51,6 +51,9 @@ class WebProcessProxy;
 class WebResourceLoadStatisticsStore : public IPC::Connection::WorkQueueMessageReceiver {
 public:
     static Ref<WebResourceLoadStatisticsStore> create(const String&);
+    static void setNotifyPagesWhenDataRecordsWereScanned(bool);
+    static void setShouldClassifyResourcesBeforeDataRecordsRemoval(bool);
+    static void setMinimumTimeBetweeenDataRecordsRemoval(double);
     virtual ~WebResourceLoadStatisticsStore();
     
     void setResourceLoadStatisticsEnabled(bool);
@@ -65,14 +68,14 @@ public:
 
     void readDataFromDiskIfNeeded();
 
-    void mergeStatistics(const Vector<WebCore::ResourceLoadStatistics>&);
-
-    WebCore::ResourceLoadStatisticsStore& coreStore() { return m_resourceStatisticsStore.get(); }
-    const WebCore::ResourceLoadStatisticsStore& coreStore() const { return m_resourceStatisticsStore.get(); }
+    WebCore::ResourceLoadStatisticsStore& coreStore() { return m_resourceLoadStatisticsStore.get(); }
+    const WebCore::ResourceLoadStatisticsStore& coreStore() const { return m_resourceLoadStatisticsStore.get(); }
 
 private:
     explicit WebResourceLoadStatisticsStore(const String&);
 
+    void processStatisticsAndDataRecords();
+
     bool hasPrevalentResourceCharacteristics(const WebCore::ResourceLoadStatistics&);
     void classifyResource(WebCore::ResourceLoadStatistics&);
     void removeDataRecords();
@@ -85,7 +88,7 @@ private:
     void writeEncoderToDisk(WebCore::KeyedEncoder&, const String& label) const;
     std::unique_ptr<WebCore::KeyedDecoder> createDecoderFromDisk(const String& label) const;
 
-    Ref<WebCore::ResourceLoadStatisticsStore> m_resourceStatisticsStore;
+    Ref<WebCore::ResourceLoadStatisticsStore> m_resourceLoadStatisticsStore;
     Ref<WTF::WorkQueue> m_statisticsQueue;
     String m_storagePath;
     bool m_resourceLoadStatisticsEnabled { false };
index 4a01fdd..8fec3e4 100644 (file)
@@ -473,6 +473,35 @@ void WebsiteDataStore::fetchData(OptionSet<WebsiteDataType> dataTypes, OptionSet
     callbackAggregator->callIfNeeded();
 }
 
+void WebsiteDataStore::fetchDataForTopPrivatelyOwnedDomains(OptionSet<WebsiteDataType> dataTypes, OptionSet<WebsiteDataFetchOption> fetchOptions, const Vector<String>& topPrivatelyOwnedDomains, std::function<void(Vector<WebsiteDataRecord>)> completionHandler)
+{
+    fetchData(dataTypes, fetchOptions, [topPrivatelyOwnedDomains, completionHandler, this](auto existingDataRecords) {
+        Vector<WebsiteDataRecord> matchingDataRecords;
+        Vector<String> domainsWithDataRecords;
+        for (auto& dataRecord : existingDataRecords) {
+            bool dataRecordAdded;
+            for (auto& dataRecordOriginData : dataRecord.origins) {
+                dataRecordAdded = false;
+                String dataRecordHost = dataRecordOriginData.securityOrigin().get().host();
+                for (auto& topPrivatelyOwnedDomain : topPrivatelyOwnedDomains) {
+                    if (dataRecordHost.endsWithIgnoringASCIICase(topPrivatelyOwnedDomain)) {
+                        auto suffixStart = dataRecordHost.length() - topPrivatelyOwnedDomain.length();
+                        if (!suffixStart || dataRecordHost[suffixStart - 1] == '.') {
+                            matchingDataRecords.append(dataRecord);
+                            domainsWithDataRecords.append(topPrivatelyOwnedDomain);
+                            dataRecordAdded = true;
+                            break;
+                        }
+                    }
+                }
+                if (dataRecordAdded)
+                    break;
+            }
+        }
+        completionHandler(matchingDataRecords);
+    });
+}
+    
 static ProcessAccessType computeNetworkProcessAccessTypeForDataRemoval(OptionSet<WebsiteDataType> dataTypes, bool isNonPersistentStore)
 {
     ProcessAccessType processAccessType = ProcessAccessType::None;
@@ -998,6 +1027,15 @@ void WebsiteDataStore::removeData(OptionSet<WebsiteDataType> dataTypes, const Ve
     callbackAggregator->callIfNeeded();
 }
 
+void WebsiteDataStore::removeDataForTopPrivatelyOwnedDomains(OptionSet<WebsiteDataType> dataTypes, OptionSet<WebsiteDataFetchOption> fetchOptions, const Vector<String>& topPrivatelyOwnedDomains, std::function<void()> completionHandler)
+{
+    fetchDataForTopPrivatelyOwnedDomains(dataTypes, fetchOptions, topPrivatelyOwnedDomains, [dataTypes, completionHandler, this](auto websiteDataRecords) {
+        this->removeData(dataTypes, websiteDataRecords, [completionHandler]() {
+            completionHandler();
+        });
+    });
+}
+
 void WebsiteDataStore::webPageWasAdded(WebPageProxy& webPageProxy)
 {
     if (m_storageManager)
index abe5f52..cba299d 100644 (file)
@@ -84,8 +84,10 @@ public:
     static void cloneSessionData(WebPageProxy& sourcePage, WebPageProxy& newPage);
 
     void fetchData(OptionSet<WebsiteDataType>, OptionSet<WebsiteDataFetchOption>, std::function<void (Vector<WebsiteDataRecord>)> completionHandler);
+    void fetchDataForTopPrivatelyOwnedDomains(OptionSet<WebsiteDataType>, OptionSet<WebsiteDataFetchOption>, const Vector<String>& topPrivatelyOwnedDomains, std::function<void(Vector<WebsiteDataRecord>)> completionHandler);
     void removeData(OptionSet<WebsiteDataType>, std::chrono::system_clock::time_point modifiedSince, std::function<void ()> completionHandler);
     void removeData(OptionSet<WebsiteDataType>, const Vector<WebsiteDataRecord>&, std::function<void ()> completionHandler);
+    void removeDataForTopPrivatelyOwnedDomains(OptionSet<WebsiteDataType>, OptionSet<WebsiteDataFetchOption>, const Vector<String>& topPrivatelyOwnedDomains, std::function<void()> completionHandler);
 
     StorageManager* storageManager() { return m_storageManager.get(); }
 
index 57aa614..2f5ce94 100644 (file)
                6501BD1A12F1243400E9F248 /* WKBundleInspector.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 65B86F1712F11D7B00B7DD8A /* WKBundleInspector.cpp */; };
                659C551E130006410025C0C2 /* InjectedBundlePageResourceLoadClient.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 6546A82913000164000CEB1C /* InjectedBundlePageResourceLoadClient.cpp */; };
                65B86F1E12F11DE300B7DD8A /* WKBundleInspector.h in Headers */ = {isa = PBXBuildFile; fileRef = 65B86F1812F11D7B00B7DD8A /* WKBundleInspector.h */; settings = {ATTRIBUTES = (Private, ); }; };
+               6BE9699C1E43B3FF008B7483 /* WKResourceLoadStatisticsManager.h in Headers */ = {isa = PBXBuildFile; fileRef = 6BE9699B1E43B3FF008B7483 /* WKResourceLoadStatisticsManager.h */; settings = {ATTRIBUTES = (Private, ); }; };
+               6BE9699E1E43B41D008B7483 /* WKResourceLoadStatisticsManager.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 6BE9699D1E43B41D008B7483 /* WKResourceLoadStatisticsManager.cpp */; };
+               6BE969A01E43B86E008B7483 /* WebResourceLoadStatisticsManager.h in Headers */ = {isa = PBXBuildFile; fileRef = 6BE9699F1E43B86E008B7483 /* WebResourceLoadStatisticsManager.h */; };
+               6BE969A21E43B8A4008B7483 /* WebResourceLoadStatisticsManager.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 6BE969A11E43B8A4008B7483 /* WebResourceLoadStatisticsManager.cpp */; };
                6EE849C81368D9390038D481 /* WKInspectorPrivateMac.h in Headers */ = {isa = PBXBuildFile; fileRef = 6EE849C61368D92D0038D481 /* WKInspectorPrivateMac.h */; settings = {ATTRIBUTES = (Private, ); }; };
                728E86F11795188C0087879E /* WebColorPickerMac.h in Headers */ = {isa = PBXBuildFile; fileRef = 728E86EF1795188C0087879E /* WebColorPickerMac.h */; };
                728E86F21795188C0087879E /* WebColorPickerMac.mm in Sources */ = {isa = PBXBuildFile; fileRef = 728E86F01795188C0087879E /* WebColorPickerMac.mm */; };
                6546A82A13000164000CEB1C /* InjectedBundlePageResourceLoadClient.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = InjectedBundlePageResourceLoadClient.h; sourceTree = "<group>"; };
                65B86F1712F11D7B00B7DD8A /* WKBundleInspector.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = WKBundleInspector.cpp; sourceTree = "<group>"; };
                65B86F1812F11D7B00B7DD8A /* WKBundleInspector.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WKBundleInspector.h; sourceTree = "<group>"; };
+               6BE9699B1E43B3FF008B7483 /* WKResourceLoadStatisticsManager.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WKResourceLoadStatisticsManager.h; sourceTree = "<group>"; };
+               6BE9699D1E43B41D008B7483 /* WKResourceLoadStatisticsManager.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = WKResourceLoadStatisticsManager.cpp; sourceTree = "<group>"; };
+               6BE9699F1E43B86E008B7483 /* WebResourceLoadStatisticsManager.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WebResourceLoadStatisticsManager.h; sourceTree = "<group>"; };
+               6BE969A11E43B8A4008B7483 /* WebResourceLoadStatisticsManager.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = WebResourceLoadStatisticsManager.cpp; sourceTree = "<group>"; };
                6D8A91A511F0EFD100DD01FE /* com.apple.WebProcess.sb.in */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; name = com.apple.WebProcess.sb.in; path = WebProcess/com.apple.WebProcess.sb.in; sourceTree = "<group>"; };
                6EE849C61368D92D0038D481 /* WKInspectorPrivateMac.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = WKInspectorPrivateMac.h; path = mac/WKInspectorPrivateMac.h; sourceTree = "<group>"; };
                728E86EF1795188C0087879E /* WebColorPickerMac.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WebColorPickerMac.h; sourceTree = "<group>"; };
                                BC111B0D112F5E4F00337BAB /* WebProcessProxy.cpp */,
                                BC032DCF10F4389F0058C15A /* WebProcessProxy.h */,
                                BCEE7AB312817095009827DA /* WebProcessProxy.messages.in */,
+                               6BE9699F1E43B86E008B7483 /* WebResourceLoadStatisticsManager.h */,
+                               6BE969A11E43B8A4008B7483 /* WebResourceLoadStatisticsManager.cpp */,
                                7A9CD8C01C77984900D9F6C7 /* WebResourceLoadStatisticsStore.cpp */,
                                7A9CD8C11C77984900D9F6C7 /* WebResourceLoadStatisticsStore.h */,
                                7A9CD8C21C779AD600D9F6C7 /* WebResourceLoadStatisticsStore.messages.in */,
                                518ACAE912AEE6BB00B04B83 /* WKProtectionSpaceTypes.h */,
                                33367638130C99DC006C9DE2 /* WKResourceCacheManager.cpp */,
                                33367639130C99DC006C9DE2 /* WKResourceCacheManager.h */,
+                               6BE9699D1E43B41D008B7483 /* WKResourceLoadStatisticsManager.cpp */,
+                               6BE9699B1E43B3FF008B7483 /* WKResourceLoadStatisticsManager.h */,
                                1ADE46B01954EC61000F7985 /* WKSessionStateRef.cpp */,
                                1ADE46B11954EC61000F7985 /* WKSessionStateRef.h */,
                                314888FE1D91B11D00377042 /* WKTextChecker.cpp */,
                                9321D58A1A38F196008052BE /* WKImmediateActionTypes.h in Headers */,
                                1C8E293912761E5B00BC7BD0 /* WKInspector.h in Headers */,
                                0F3C725B196F604E00AEDD0C /* WKInspectorHighlightView.h in Headers */,
+                               6BE969A01E43B86E008B7483 /* WebResourceLoadStatisticsManager.h in Headers */,
                                A54293A4195A43DA002782C7 /* WKInspectorNodeSearchGestureRecognizer.h in Headers */,
                                6EE849C81368D9390038D481 /* WKInspectorPrivateMac.h in Headers */,
                                51A9E10B1315CD18009E7031 /* WKKeyValueStorageManager.h in Headers */,
                                1A3CC16718906ACF001E6ED8 /* WKWebView.h in Headers */,
                                1ADF591B1890528E0043C145 /* WKWebViewConfiguration.h in Headers */,
                                2D7AAFD618C956AF00A7ACD4 /* WKWebViewConfigurationInternal.h in Headers */,
+                               6BE9699C1E43B3FF008B7483 /* WKResourceLoadStatisticsManager.h in Headers */,
                                1AC1415118AC47EE006C602C /* WKWebViewConfigurationPrivate.h in Headers */,
                                41DC459C1E3DBB2800B11F51 /* LibWebRTCSocketClient.h in Headers */,
                                2D7AAFD318C8640600A7ACD4 /* WKWebViewContentProvider.h in Headers */,
                                3760881E150413E900FC82C7 /* WebRenderObject.cpp in Sources */,
                                510AFFB916542048001BA05E /* WebResourceLoader.cpp in Sources */,
                                51F060E11654318500F3281B /* WebResourceLoaderMessageReceiver.cpp in Sources */,
+                                6BE969A21E43B8A4008B7483 /* WebResourceLoadStatisticsManager.cpp in Sources */,
                                51F060E11654318500F3281C /* WebRTCSocketMessageReceiver.cpp in Sources */,
                                51F060E11654318500F3282C /* WebRTCResolverMessageReceiver.cpp in Sources */,
                                51F060E11654318500F3281D /* NetworkRTCSocketMessageReceiver.cpp in Sources */,
                                37948408150C4B9700E52CE9 /* WKRenderLayer.cpp in Sources */,
                                37608822150414F700FC82C7 /* WKRenderObject.cpp in Sources */,
                                3336763A130C99DC006C9DE2 /* WKResourceCacheManager.cpp in Sources */,
+                                6BE9699E1E43B41D008B7483 /* WKResourceLoadStatisticsManager.cpp in Sources */,
                                1A7E377818E4A4FE003D0FFF /* WKScriptMessage.mm in Sources */,
                                0FCB4E5518BBE044000FCFC9 /* WKScrollView.mm in Sources */,
                                51CD1C661B34B9DC00142CA5 /* WKSecurityOrigin.mm in Sources */,
index e91ce10..c97d488 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2009-2016 Apple Inc. All rights reserved.
+ * Copyright (C) 2009-2017 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -168,7 +168,7 @@ WebProcess::WebProcess()
 #if PLATFORM(IOS)
     , m_webSQLiteDatabaseTracker(*this)
 #endif
-    , m_resourceLoadStatisticsStorage(WebCore::ResourceLoadStatisticsStore::create())
+    , m_resourceLoadStatisticsStore(WebCore::ResourceLoadStatisticsStore::create())
 {
     // Initialize our platform strategies.
     WebPlatformStrategies::initialize();
@@ -190,8 +190,8 @@ WebProcess::WebProcess()
 
     m_plugInAutoStartOriginHashes.add(SessionID::defaultSessionID(), HashMap<unsigned, double>());
 
-    ResourceLoadObserver::sharedObserver().setStatisticsStore(m_resourceLoadStatisticsStorage.copyRef());
-    m_resourceLoadStatisticsStorage->setNotificationCallback([this] {
+    ResourceLoadObserver::sharedObserver().setStatisticsStore(m_resourceLoadStatisticsStore.copyRef());
+    m_resourceLoadStatisticsStore->setNotificationCallback([this] {
         if (m_statisticsChangedTimer.isActive())
             return;
         m_statisticsChangedTimer.startOneShot(std::chrono::seconds(5));
@@ -1389,10 +1389,10 @@ void WebProcess::nonVisibleProcessCleanupTimerFired()
 
 void WebProcess::statisticsChangedTimerFired()
 {
-    if (m_resourceLoadStatisticsStorage->isEmpty())
+    if (m_resourceLoadStatisticsStore->isEmpty())
         return;
 
-    parentProcessConnection()->send(Messages::WebResourceLoadStatisticsStore::ResourceLoadStatisticsUpdated(m_resourceLoadStatisticsStorage->takeStatistics()), 0);
+    parentProcessConnection()->send(Messages::WebResourceLoadStatisticsStore::ResourceLoadStatisticsUpdated(m_resourceLoadStatisticsStore->takeStatistics()), 0);
 }
 
 void WebProcess::setResourceLoadStatisticsEnabled(bool enabled)
index 85b0cb2..b0ce003 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2010-2016 Apple Inc. All rights reserved.
+ * Copyright (C) 2010-2017 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -412,7 +412,7 @@ private:
     WebSQLiteDatabaseTracker m_webSQLiteDatabaseTracker;
 #endif
 
-    Ref<WebCore::ResourceLoadStatisticsStore> m_resourceLoadStatisticsStorage;
+    Ref<WebCore::ResourceLoadStatisticsStore> m_resourceLoadStatisticsStore;
 
     unsigned m_pagesMarkingLayersAsVolatile { 0 };
     bool m_suppressMemoryPressureHandler { false };
index 48857c5..8882354 100644 (file)
@@ -1,3 +1,51 @@
+2017-02-10  John Wilander  <wilander@apple.com>
+
+        Updates to Resource Load Statistics: Get the right website data store and introduce timeout for user interaction
+        https://bugs.webkit.org/show_bug.cgi?id=167474
+        <rdar://problem/24681808>
+        <rdar://problem/24703286>
+        <rdar://problem/30290270>
+
+        This patch adds test infrastructure to allow testing 
+        of the various rules in place for data records removal.
+
+        Reviewed by Andy Estes.
+
+        * WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl:
+        * WebKitTestRunner/InjectedBundle/InjectedBundle.cpp:
+        (WTR::InjectedBundle::didReceiveMessageToPage):
+            Callback mechanism to tell pages that a website data store
+            scan has happened.
+        * WebKitTestRunner/InjectedBundle/TestRunner.cpp:
+        (WTR::TestRunner::setStatisticsPrevalentResource):
+        (WTR::TestRunner::isStatisticsPrevalentResource):
+        (WTR::TestRunner::setStatisticsHasHadUserInteraction):
+        (WTR::TestRunner::isStatisticsHasHadUserInteraction):
+        (WTR::TestRunner::setStatisticsTimeToLiveUserInteraction):
+        (WTR::TestRunner::installStatisticsDidModifyDataRecordsCallback):
+        (WTR::TestRunner::statisticsDidModifyDataRecordsCallback):
+        (WTR::TestRunner::statisticsFireDataModificationHandler):
+        (WTR::TestRunner::setStatisticsNotifyPagesWhenDataRecordsWereScanned):
+        (WTR::TestRunner::setStatisticsShouldClassifyResourcesBeforeDataRecordsRemoval):
+        (WTR::TestRunner::setStatisticsMinimumTimeBetweeenDataRecordsRemoval):
+        (WTR::TestRunner::statisticsResetToConsistentState):
+        * WebKitTestRunner/InjectedBundle/TestRunner.h:
+        * WebKitTestRunner/TestController.cpp:
+        (WTR::TestController::setStatisticsPrevalentResource):
+        (WTR::TestController::isStatisticsPrevalentResource):
+        (WTR::TestController::setStatisticsHasHadUserInteraction):
+        (WTR::TestController::isStatisticsHasHadUserInteraction):
+        (WTR::TestController::setStatisticsTimeToLiveUserInteraction):
+        (WTR::TestController::statisticsFireDataModificationHandler):
+        (WTR::TestController::setStatisticsNotifyPagesWhenDataRecordsWereScanned):
+        (WTR::TestController::setStatisticsShouldClassifyResourcesBeforeDataRecordsRemoval):
+        (WTR::TestController::setStatisticsMinimumTimeBetweeenDataRecordsRemoval):
+        (WTR::TestController::statisticsResetToConsistentState):
+        * WebKitTestRunner/TestController.h:
+            These are all configuration and test functions.
+        * WebKitTestRunner/TestInvocation.cpp:
+        (WTR::TestInvocation::didReceiveSynchronousMessageFromInjectedBundle):
+
 2017-02-10  Carlos Alberto Lopez Perez  <clopez@igalia.com>
 
         REGRESSION(r182916) run-perf-tests never timeouts
index b528cd5..352fa0a 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2010-2016 Apple Inc. All rights reserved.
+ * Copyright (C) 2010-2017 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -246,4 +246,17 @@ interface TestRunner {
     void setMockGamepadButtonValue(unsigned long index, unsigned long buttonIndex, double value);
     void connectMockGamepad(unsigned long index);
     void disconnectMockGamepad(unsigned long index);
+
+    // Resource Load Statistics
+    void installStatisticsDidModifyDataRecordsCallback(object callback);
+    void setStatisticsPrevalentResource(DOMString hostName, boolean value);
+    boolean isStatisticsPrevalentResource(DOMString hostName);
+    void setStatisticsHasHadUserInteraction(DOMString hostName, boolean value);
+    boolean isStatisticsHasHadUserInteraction(DOMString hostName);
+    void setStatisticsTimeToLiveUserInteraction(double seconds);
+    void statisticsFireDataModificationHandler();
+    void setStatisticsNotifyPagesWhenDataRecordsWereScanned(boolean value);
+    void setStatisticsShouldClassifyResourcesBeforeDataRecordsRemoval(boolean value);
+    void setStatisticsMinimumTimeBetweeenDataRecordsRemoval(double seconds);
+    void statisticsResetToConsistentState();
 };
index 8d3545a..f247b36 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2010-2016 Apple Inc. All rights reserved.
+ * Copyright (C) 2010-2017 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -266,6 +266,11 @@ void InjectedBundle::didReceiveMessageToPage(WKBundlePageRef page, WKStringRef m
         return;
     }
 
+    if (WKStringIsEqualToUTF8CString(messageName, "WebsiteDataDeletionForTopPrivatelyOwnedDomainsFinished")) {
+        m_testRunner->statisticsDidModifyDataRecordsCallback();
+        return;
+    }
+
     WKRetainPtr<WKStringRef> errorMessageName(AdoptWK, WKStringCreateWithUTF8CString("Error"));
     WKRetainPtr<WKStringRef> errorMessageBody(AdoptWK, WKStringCreateWithUTF8CString("Unknown"));
     WKBundlePagePostMessage(page, errorMessageName.get(), errorMessageBody.get());
index 8000f98..6f7eb0c 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2010-2016 Apple Inc. All rights reserved.
+ * Copyright (C) 2010-2017 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -33,6 +33,7 @@
 #include "StringFunctions.h"
 #include "TestController.h"
 #include <JavaScriptCore/JSCTestRunnerUtils.h>
+#include <WebCore/ResourceLoadObserver.h>
 #include <WebKit/WKBundle.h>
 #include <WebKit/WKBundleBackForwardList.h>
 #include <WebKit/WKBundleFrame.h>
@@ -635,6 +636,7 @@ enum {
     WillEndSwipeCallbackID,
     DidEndSwipeCallbackID,
     DidRemoveSwipeSnapshotCallbackID,
+    StatisticsDidModifyDataRecordsCallbackID,
     FirstUIScriptCallbackID = 100
 };
 
@@ -1152,6 +1154,128 @@ void TestRunner::callDidRemoveSwipeSnapshotCallback()
     callTestRunnerCallback(DidRemoveSwipeSnapshotCallbackID);
 }
 
+void TestRunner::setStatisticsPrevalentResource(JSStringRef hostName, bool value)
+{
+    Vector<WKRetainPtr<WKStringRef>> keys;
+    Vector<WKRetainPtr<WKTypeRef>> values;
+
+    keys.append({ AdoptWK, WKStringCreateWithUTF8CString("HostName") });
+    values.append({ AdoptWK, WKStringCreateWithJSString(hostName) });
+    
+    keys.append({ AdoptWK, WKStringCreateWithUTF8CString("Value") });
+    values.append({ AdoptWK, WKBooleanCreate(value) });
+    
+    Vector<WKStringRef> rawKeys;
+    Vector<WKTypeRef> rawValues;
+    rawKeys.resize(keys.size());
+    rawValues.resize(values.size());
+    
+    for (size_t i = 0; i < keys.size(); ++i) {
+        rawKeys[i] = keys[i].get();
+        rawValues[i] = values[i].get();
+    }
+    
+    WKRetainPtr<WKStringRef> messageName(AdoptWK, WKStringCreateWithUTF8CString("SetStatisticsPrevalentResource"));
+    WKRetainPtr<WKDictionaryRef> messageBody(AdoptWK, WKDictionaryCreate(rawKeys.data(), rawValues.data(), rawKeys.size()));
+
+    WKBundlePostSynchronousMessage(InjectedBundle::singleton().bundle(), messageName.get(), messageBody.get(), nullptr);
+}
+
+bool TestRunner::isStatisticsPrevalentResource(JSStringRef hostName)
+{
+    WKRetainPtr<WKStringRef> messageName(AdoptWK, WKStringCreateWithUTF8CString("IsStatisticsPrevalentResource"));
+    WKRetainPtr<WKStringRef> messageBody(AdoptWK, WKStringCreateWithJSString(hostName));
+    WKTypeRef returnData = 0;
+    WKBundlePagePostSynchronousMessageForTesting(InjectedBundle::singleton().page()->page(), messageName.get(), messageBody.get(), &returnData);
+    return WKBooleanGetValue(static_cast<WKBooleanRef>(returnData));
+}
+
+void TestRunner::setStatisticsHasHadUserInteraction(JSStringRef hostName, bool value)
+{
+    Vector<WKRetainPtr<WKStringRef>> keys;
+    Vector<WKRetainPtr<WKTypeRef>> values;
+    
+    keys.append({ AdoptWK, WKStringCreateWithUTF8CString("HostName") });
+    values.append({ AdoptWK, WKStringCreateWithJSString(hostName) });
+    
+    keys.append({ AdoptWK, WKStringCreateWithUTF8CString("Value") });
+    values.append({ AdoptWK, WKBooleanCreate(value) });
+    
+    Vector<WKStringRef> rawKeys;
+    Vector<WKTypeRef> rawValues;
+    rawKeys.resize(keys.size());
+    rawValues.resize(values.size());
+    
+    for (size_t i = 0; i < keys.size(); ++i) {
+        rawKeys[i] = keys[i].get();
+        rawValues[i] = values[i].get();
+    }
+    
+    WKRetainPtr<WKStringRef> messageName(AdoptWK, WKStringCreateWithUTF8CString("SetStatisticsHasHadUserInteraction"));
+    WKRetainPtr<WKDictionaryRef> messageBody(AdoptWK, WKDictionaryCreate(rawKeys.data(), rawValues.data(), rawKeys.size()));
+    
+    WKBundlePostSynchronousMessage(InjectedBundle::singleton().bundle(), messageName.get(), messageBody.get(), nullptr);
+}
+
+bool TestRunner::isStatisticsHasHadUserInteraction(JSStringRef hostName)
+{
+    WKRetainPtr<WKStringRef> messageName(AdoptWK, WKStringCreateWithUTF8CString("IsStatisticsHasHadUserInteraction"));
+    WKRetainPtr<WKStringRef> messageBody(AdoptWK, WKStringCreateWithJSString(hostName));
+    WKTypeRef returnData = 0;
+    WKBundlePagePostSynchronousMessageForTesting(InjectedBundle::singleton().page()->page(), messageName.get(), messageBody.get(), &returnData);
+    return WKBooleanGetValue(static_cast<WKBooleanRef>(returnData));
+}
+
+void TestRunner::setStatisticsTimeToLiveUserInteraction(double seconds)
+{
+    WKRetainPtr<WKStringRef> messageName(AdoptWK, WKStringCreateWithUTF8CString("SetStatisticsTimeToLiveUserInteraction"));
+    WKRetainPtr<WKDoubleRef> messageBody(AdoptWK, WKDoubleCreate(seconds));
+    WKBundlePostSynchronousMessage(InjectedBundle::singleton().bundle(), messageName.get(), messageBody.get(), nullptr);
+}
+
+void TestRunner::installStatisticsDidModifyDataRecordsCallback(JSValueRef callback)
+{
+    cacheTestRunnerCallback(StatisticsDidModifyDataRecordsCallbackID, callback);
+}
+
+void TestRunner::statisticsDidModifyDataRecordsCallback()
+{
+    callTestRunnerCallback(StatisticsDidModifyDataRecordsCallbackID);
+}
+
+void TestRunner::statisticsFireDataModificationHandler()
+{
+    WKRetainPtr<WKStringRef> messageName(AdoptWK, WKStringCreateWithUTF8CString("StatisticsFireDataModificationHandler"));
+    WKBundlePostSynchronousMessage(InjectedBundle::singleton().bundle(), messageName.get(), 0, nullptr);
+}
+
+void TestRunner::setStatisticsNotifyPagesWhenDataRecordsWereScanned(bool value)
+{
+    WKRetainPtr<WKStringRef> messageName(AdoptWK, WKStringCreateWithUTF8CString("StatisticsNotifyPagesWhenDataRecordsWereScanned"));
+    WKRetainPtr<WKBooleanRef> messageBody(AdoptWK, WKBooleanCreate(value));
+    WKBundlePostSynchronousMessage(InjectedBundle::singleton().bundle(), messageName.get(), messageBody.get(), nullptr);
+}
+
+void TestRunner::setStatisticsShouldClassifyResourcesBeforeDataRecordsRemoval(bool value)
+{
+    WKRetainPtr<WKStringRef> messageName(AdoptWK, WKStringCreateWithUTF8CString("StatisticsShouldClassifyResourcesBeforeDataRecordsRemoval"));
+    WKRetainPtr<WKBooleanRef> messageBody(AdoptWK, WKBooleanCreate(value));
+    WKBundlePostSynchronousMessage(InjectedBundle::singleton().bundle(), messageName.get(), messageBody.get(), nullptr);
+}
+
+void TestRunner::setStatisticsMinimumTimeBetweeenDataRecordsRemoval(double seconds)
+{
+    WKRetainPtr<WKStringRef> messageName(AdoptWK, WKStringCreateWithUTF8CString("SetStatisticsMinimumTimeBetweeenDataRecordsRemoval"));
+    WKRetainPtr<WKDoubleRef> messageBody(AdoptWK, WKDoubleCreate(seconds));
+    WKBundlePostSynchronousMessage(InjectedBundle::singleton().bundle(), messageName.get(), messageBody.get(), nullptr);
+}
+    
+void TestRunner::statisticsResetToConsistentState()
+{
+    WKRetainPtr<WKStringRef> messageName(AdoptWK, WKStringCreateWithUTF8CString("StatisticsResetToConsistentState"));
+    WKBundlePostSynchronousMessage(InjectedBundle::singleton().bundle(), messageName.get(), 0, nullptr);
+}
+
 #if PLATFORM(MAC)
 void TestRunner::connectMockGamepad(unsigned index)
 {
index e009414..764c9e4 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2010-2016 Apple Inc. All rights reserved.
+ * Copyright (C) 2010-2017 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -345,6 +345,20 @@ public:
     void setMockGamepadDetails(unsigned index, JSStringRef gamepadID, unsigned axisCount, unsigned buttonCount);
     void setMockGamepadAxisValue(unsigned index, unsigned axisIndex, double value);
     void setMockGamepadButtonValue(unsigned index, unsigned buttonIndex, double value);
+    
+    // Resource Load Statistics
+    void installStatisticsDidModifyDataRecordsCallback(JSValueRef callback);
+    void statisticsDidModifyDataRecordsCallback();
+    void statisticsFireDataModificationHandler();
+    void setStatisticsPrevalentResource(JSStringRef hostName, bool value);
+    bool isStatisticsPrevalentResource(JSStringRef hostName);
+    void setStatisticsHasHadUserInteraction(JSStringRef hostName, bool value);
+    bool isStatisticsHasHadUserInteraction(JSStringRef hostName);
+    void setStatisticsTimeToLiveUserInteraction(double seconds);
+    void setStatisticsNotifyPagesWhenDataRecordsWereScanned(bool);
+    void setStatisticsShouldClassifyResourcesBeforeDataRecordsRemoval(bool);
+    void setStatisticsMinimumTimeBetweeenDataRecordsRemoval(double);
+    void statisticsResetToConsistentState();
 
 private:
     TestRunner();
index 8d8d91a..c38a9c7 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2010, 2014-2016 Apple Inc. All rights reserved.
+ * Copyright (C) 2010, 2014-2017 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -55,6 +55,7 @@
 #include <WebKit/WKPluginInformation.h>
 #include <WebKit/WKPreferencesRefPrivate.h>
 #include <WebKit/WKProtectionSpace.h>
+#include <WebKit/WKResourceLoadStatisticsManager.h>
 #include <WebKit/WKRetainPtr.h>
 #include <WebKit/WKSecurityOriginRef.h>
 #include <WebKit/WKTextChecker.h>
@@ -2207,6 +2208,55 @@ void TestController::setIgnoresViewportScaleLimits(bool ignoresViewportScaleLimi
     WKPageSetIgnoresViewportScaleLimits(m_mainWebView->page(), ignoresViewportScaleLimits);
 }
 
+void TestController::setStatisticsPrevalentResource(WKStringRef hostName, bool value)
+{
+    WKResourceLoadStatisticsManagerSetPrevalentResource(hostName, value);
+}
+
+bool TestController::isStatisticsPrevalentResource(WKStringRef hostName)
+{
+    return WKResourceLoadStatisticsManagerIsPrevalentResource(hostName);
+}
+
+void TestController::setStatisticsHasHadUserInteraction(WKStringRef hostName, bool value)
+{
+    WKResourceLoadStatisticsManagerSetHasHadUserInteraction(hostName, value);
+}
+
+bool TestController::isStatisticsHasHadUserInteraction(WKStringRef hostName)
+{
+    return WKResourceLoadStatisticsManagerIsHasHadUserInteraction(hostName);
+}
+
+void TestController::setStatisticsTimeToLiveUserInteraction(double seconds)
+{
+    WKResourceLoadStatisticsManagerSetTimeToLiveUserInteraction(seconds);
+}
+
+void TestController::statisticsFireDataModificationHandler()
+{
+    WKResourceLoadStatisticsManagerFireDataModificationHandler();
+}
+    
+void TestController::setStatisticsNotifyPagesWhenDataRecordsWereScanned(bool value)
+{
+    WKResourceLoadStatisticsManagerSetNotifyPagesWhenDataRecordsWereScanned(value);
+}
+    
+void TestController::setStatisticsShouldClassifyResourcesBeforeDataRecordsRemoval(bool value)
+{
+    WKResourceLoadStatisticsManagerSetShouldClassifyResourcesBeforeDataRecordsRemoval(value);
+}
+
+void TestController::setStatisticsMinimumTimeBetweeenDataRecordsRemoval(double seconds)
+{
+    WKResourceLoadStatisticsManagerSetMinimumTimeBetweeenDataRecordsRemoval(seconds);
+}
+void TestController::statisticsResetToConsistentState()
+{
+    WKResourceLoadStatisticsManagerResetToConsistentState();
+}
+    
 #if !PLATFORM(COCOA)
 void TestController::platformWillRunTest(const TestInvocation&)
 {
index df22e2a..cc753c2 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2010, 2015-2016 Apple Inc. All rights reserved.
+ * Copyright (C) 2010, 2015-2017 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -148,6 +148,17 @@ public:
 
     void setShouldDownloadUndisplayableMIMETypes(bool value) { m_shouldDownloadUndisplayableMIMETypes = value; }
 
+    void setStatisticsPrevalentResource(WKStringRef hostName, bool value);
+    bool isStatisticsPrevalentResource(WKStringRef hostName);
+    void setStatisticsHasHadUserInteraction(WKStringRef hostName, bool value);
+    bool isStatisticsHasHadUserInteraction(WKStringRef hostName);
+    void setStatisticsTimeToLiveUserInteraction(double seconds);
+    void statisticsFireDataModificationHandler();
+    void setStatisticsNotifyPagesWhenDataRecordsWereScanned(bool);
+    void setStatisticsShouldClassifyResourcesBeforeDataRecordsRemoval(bool);
+    void setStatisticsMinimumTimeBetweeenDataRecordsRemoval(double);
+    void statisticsResetToConsistentState();
+
 private:
     WKRetainPtr<WKPageConfigurationRef> generatePageConfiguration(WKContextConfigurationRef);
     WKRetainPtr<WKContextConfigurationRef> generateContextConfiguration() const;
index 684f9ed..b0a0442 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2010-2016 Apple Inc. All rights reserved.
+ * Copyright (C) 2010-2017 Apple Inc. All rights reserved.
  * Copyright (C) 2012 Intel Corporation. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
@@ -886,6 +886,89 @@ WKRetainPtr<WKTypeRef> TestInvocation::didReceiveSynchronousMessageFromInjectedB
         return result;
     }
 
+    if (WKStringIsEqualToUTF8CString(messageName, "SetStatisticsPrevalentResource")) {
+        ASSERT(WKGetTypeID(messageBody) == WKDictionaryGetTypeID());
+
+        WKDictionaryRef messageBodyDictionary = static_cast<WKDictionaryRef>(messageBody);
+        WKRetainPtr<WKStringRef> hostNameKey(AdoptWK, WKStringCreateWithUTF8CString("HostName"));
+        WKRetainPtr<WKStringRef> valueKey(AdoptWK, WKStringCreateWithUTF8CString("Value"));
+
+        WKStringRef hostName = static_cast<WKStringRef>(WKDictionaryGetItemForKey(messageBodyDictionary, hostNameKey.get()));
+        WKBooleanRef value = static_cast<WKBooleanRef>(WKDictionaryGetItemForKey(messageBodyDictionary, valueKey.get()));
+
+        TestController::singleton().setStatisticsPrevalentResource(hostName, WKBooleanGetValue(value));
+        return nullptr;
+    }
+
+    if (WKStringIsEqualToUTF8CString(messageName, "IsStatisticsPrevalentResource")) {
+        ASSERT(WKGetTypeID(messageBody) == WKStringGetTypeID());
+
+        WKStringRef hostName = static_cast<WKStringRef>(messageBody);
+        bool isPrevalent = TestController::singleton().isStatisticsPrevalentResource(hostName);
+        WKRetainPtr<WKTypeRef> result(AdoptWK, WKBooleanCreate(isPrevalent));
+        return result;
+    }
+
+    if (WKStringIsEqualToUTF8CString(messageName, "SetStatisticsHasHadUserInteraction")) {
+        ASSERT(WKGetTypeID(messageBody) == WKDictionaryGetTypeID());
+        
+        WKDictionaryRef messageBodyDictionary = static_cast<WKDictionaryRef>(messageBody);
+        WKRetainPtr<WKStringRef> hostNameKey(AdoptWK, WKStringCreateWithUTF8CString("HostName"));
+        WKRetainPtr<WKStringRef> valueKey(AdoptWK, WKStringCreateWithUTF8CString("Value"));
+        
+        WKStringRef hostName = static_cast<WKStringRef>(WKDictionaryGetItemForKey(messageBodyDictionary, hostNameKey.get()));
+        WKBooleanRef value = static_cast<WKBooleanRef>(WKDictionaryGetItemForKey(messageBodyDictionary, valueKey.get()));
+        
+        TestController::singleton().setStatisticsHasHadUserInteraction(hostName, WKBooleanGetValue(value));
+        return nullptr;
+    }
+
+    if (WKStringIsEqualToUTF8CString(messageName, "IsStatisticsHasHadUserInteraction")) {
+        ASSERT(WKGetTypeID(messageBody) == WKStringGetTypeID());
+        
+        WKStringRef hostName = static_cast<WKStringRef>(messageBody);
+        bool hasHadUserInteraction = TestController::singleton().isStatisticsHasHadUserInteraction(hostName);
+        WKRetainPtr<WKTypeRef> result(AdoptWK, WKBooleanCreate(hasHadUserInteraction));
+        return result;
+    }
+    
+    if (WKStringIsEqualToUTF8CString(messageName, "SetStatisticsTimeToLiveUserInteraction")) {
+        ASSERT(WKGetTypeID(messageBody) == WKDoubleGetTypeID());
+        WKDoubleRef seconds = static_cast<WKDoubleRef>(messageBody);
+        TestController::singleton().setStatisticsTimeToLiveUserInteraction(WKDoubleGetValue(seconds));
+        return nullptr;
+    }
+    
+    if (WKStringIsEqualToUTF8CString(messageName, "StatisticsFireDataModificationHandler")) {
+        TestController::singleton().statisticsFireDataModificationHandler();
+        return nullptr;
+    }
+    
+    if (WKStringIsEqualToUTF8CString(messageName, "StatisticsNotifyPagesWhenDataRecordsWereScanned")) {
+        ASSERT(WKGetTypeID(messageBody) == WKBooleanGetTypeID());
+        WKBooleanRef value = static_cast<WKBooleanRef>(messageBody);
+        TestController::singleton().setStatisticsNotifyPagesWhenDataRecordsWereScanned(WKBooleanGetValue(value));
+        return nullptr;
+    }
+
+    if (WKStringIsEqualToUTF8CString(messageName, "StatisticsShouldClassifyResourcesBeforeDataRecordsRemoval")) {
+        ASSERT(WKGetTypeID(messageBody) == WKBooleanGetTypeID());
+        WKBooleanRef value = static_cast<WKBooleanRef>(messageBody);
+        TestController::singleton().setStatisticsShouldClassifyResourcesBeforeDataRecordsRemoval(WKBooleanGetValue(value));
+        return nullptr;
+    }
+
+    if (WKStringIsEqualToUTF8CString(messageName, "SetStatisticsMinimumTimeBetweeenDataRecordsRemoval")) {
+        ASSERT(WKGetTypeID(messageBody) == WKDoubleGetTypeID());
+        WKDoubleRef seconds = static_cast<WKDoubleRef>(messageBody);
+        TestController::singleton().setStatisticsMinimumTimeBetweeenDataRecordsRemoval(WKDoubleGetValue(seconds));
+        return nullptr;
+    }
+    
+    if (WKStringIsEqualToUTF8CString(messageName, "StatisticsResetToConsistentState")) {
+        TestController::singleton().statisticsResetToConsistentState();
+        return nullptr;
+    }
 
     ASSERT_NOT_REACHED();
     return nullptr;