Resource Load Statistics (experimental): Add fast mode for non-cookie website data...
authorwilander@apple.com <wilander@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 6 Dec 2019 00:26:09 +0000 (00:26 +0000)
committerwilander@apple.com <wilander@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 6 Dec 2019 00:26:09 +0000 (00:26 +0000)
https://bugs.webkit.org/show_bug.cgi?id=204858
<rdar://problem/57639851>

Reviewed by Alex Christensen.

Source/WebCore:

This change adds two internal flags:
- "Live-On Testing" with a one hour timeout instead of seven days.
- "Repro Testing" with an instant timeout (bar ITP's regular delays) instead of seven days.

These internal flags should be removed once testing is complete: <rdar://problem/57673418>

No new tests. This change just adds new opt-in settings for manual testing.

* page/Settings.yaml:
* platform/network/NetworkStorageSession.h:
    The FirstPartyWebsiteDataRemovalMode enum now has two new values:
    - AllButCookiesLiveOnTestingTimeout
    - AllButCookiesReproTestingTimeout

Source/WebKit:

The purpose of this change is to allow for dedicated testing of the change in
https://trac.webkit.org/changeset/253082/webkit. Waiting seven days just isn't a good
starting point.

This change adds two internal flags:
- "Live-On Testing" with a one hour timeout instead of seven days.
- "Repro Testing" with an instant timeout (bar ITP's regular delays) instead of seven days.

The change also makes sure that hasHadUnexpiredRecentUserInteraction() in
ResourceLoadStatisticsDatabaseStore and ResourceLoadStatisticsMemoryStore only
age out the user interaction timestamp if the OperatingDatesWindow is Long so
that we don't age out timestamps early with the shorter OperatingDatesWindows.

This change changes the default value of IsFirstPartyWebsiteDataRemovalEnabled to true.

These internal flags should be removed once testing is complete: <rdar://problem/57673418>

* NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:
* NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp:
(WebKit::ResourceLoadStatisticsMemoryStore::hasHadUnexpiredRecentUserInteraction const):
(WebKit::ResourceLoadStatisticsMemoryStore::shouldRemoveAllButCookiesFor const):
* NetworkProcess/Classifier/ResourceLoadStatisticsStore.cpp:
(WebKit::ResourceLoadStatisticsStore::hasStatisticsExpired const):
* NetworkProcess/Classifier/ResourceLoadStatisticsStore.h:
* NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:
(WebKit::WebResourceLoadStatisticsStore::setFirstPartyWebsiteDataRemovalMode):
* NetworkProcess/NetworkProcess.messages.in:
* Shared/WebPreferences.yaml:
* UIProcess/WebsiteData/Cocoa/WebsiteDataStoreCocoa.mm:
(WebKit::WebsiteDataStore::parameters):

git-svn-id: http://svn.webkit.org/repository/webkit/trunk@253185 268f45cc-cd09-0410-ab3c-d52691b4dbfc

12 files changed:
Source/WebCore/ChangeLog
Source/WebCore/page/Settings.yaml
Source/WebCore/platform/network/NetworkStorageSession.h
Source/WebKit/ChangeLog
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsStore.cpp
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsStore.h
Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp
Source/WebKit/NetworkProcess/NetworkProcess.messages.in
Source/WebKit/Shared/WebPreferences.yaml
Source/WebKit/UIProcess/WebsiteData/Cocoa/WebsiteDataStoreCocoa.mm

index f7ff228..c475a12 100644 (file)
@@ -1,3 +1,25 @@
+2019-12-05  John Wilander  <wilander@apple.com>
+
+        Resource Load Statistics (experimental): Add fast mode for non-cookie website data deletion
+        https://bugs.webkit.org/show_bug.cgi?id=204858
+        <rdar://problem/57639851>
+
+        Reviewed by Alex Christensen.
+
+        This change adds two internal flags:
+        - "Live-On Testing" with a one hour timeout instead of seven days.
+        - "Repro Testing" with an instant timeout (bar ITP's regular delays) instead of seven days.
+
+        These internal flags should be removed once testing is complete: <rdar://problem/57673418>
+
+        No new tests. This change just adds new opt-in settings for manual testing.
+
+        * page/Settings.yaml:
+        * platform/network/NetworkStorageSession.h:
+            The FirstPartyWebsiteDataRemovalMode enum now has two new values:
+            - AllButCookiesLiveOnTestingTimeout
+            - AllButCookiesReproTestingTimeout
+
 2019-12-05  Chris Dumez  <cdumez@apple.com>
 
         PageConfiguration::dragClient should use a smart pointer
index 6122882..a3e3526 100644 (file)
@@ -887,6 +887,12 @@ isThirdPartyCookieBlockingEnabled:
   initial: true
 
 isFirstPartyWebsiteDataRemovalEnabled:
+  initial: true
+
+isFirstPartyWebsiteDataRemovalLiveOnTestingEnabled:
+  initial: false
+
+isFirstPartyWebsiteDataRemovalReproTestingEnabled:
   initial: false
 
 isLoggedInAPIEnabled:
index 1ba7bf9..eefb870 100644 (file)
@@ -75,7 +75,7 @@ struct SameSiteInfo;
 enum class IncludeSecureCookies : bool;
 enum class IncludeHttpOnlyCookies : bool;
 enum class ThirdPartyCookieBlockingMode : uint8_t { All, AllOnSitesWithoutUserInteraction, OnlyAccordingToPerDomainPolicy };
-enum class FirstPartyWebsiteDataRemovalMode : bool { AllButCookies, None };
+enum class FirstPartyWebsiteDataRemovalMode : uint8_t { AllButCookies, None, AllButCookiesLiveOnTestingTimeout, AllButCookiesReproTestingTimeout };
 
 class NetworkStorageSession {
     WTF_MAKE_NONCOPYABLE(NetworkStorageSession); WTF_MAKE_FAST_ALLOCATED;
@@ -233,4 +233,14 @@ template<> struct EnumTraits<WebCore::ThirdPartyCookieBlockingMode> {
     >;
 };
 
+template<> struct EnumTraits<WebCore::FirstPartyWebsiteDataRemovalMode> {
+    using values = EnumValues<
+        WebCore::FirstPartyWebsiteDataRemovalMode,
+        WebCore::FirstPartyWebsiteDataRemovalMode::AllButCookies,
+        WebCore::FirstPartyWebsiteDataRemovalMode::None,
+        WebCore::FirstPartyWebsiteDataRemovalMode::AllButCookiesLiveOnTestingTimeout,
+        WebCore::FirstPartyWebsiteDataRemovalMode::AllButCookiesReproTestingTimeout
+    >;
+};
+
 }
index 5d086a6..6366c6b 100644 (file)
@@ -1,3 +1,42 @@
+2019-12-05  John Wilander  <wilander@apple.com>
+
+        Resource Load Statistics (experimental): Add fast mode for non-cookie website data deletion
+        https://bugs.webkit.org/show_bug.cgi?id=204858
+        <rdar://problem/57639851>
+
+        Reviewed by Alex Christensen.
+
+        The purpose of this change is to allow for dedicated testing of the change in
+        https://trac.webkit.org/changeset/253082/webkit. Waiting seven days just isn't a good
+        starting point.
+
+        This change adds two internal flags:
+        - "Live-On Testing" with a one hour timeout instead of seven days.
+        - "Repro Testing" with an instant timeout (bar ITP's regular delays) instead of seven days.
+
+        The change also makes sure that hasHadUnexpiredRecentUserInteraction() in
+        ResourceLoadStatisticsDatabaseStore and ResourceLoadStatisticsMemoryStore only
+        age out the user interaction timestamp if the OperatingDatesWindow is Long so
+        that we don't age out timestamps early with the shorter OperatingDatesWindows.
+
+        This change changes the default value of IsFirstPartyWebsiteDataRemovalEnabled to true.
+
+        These internal flags should be removed once testing is complete: <rdar://problem/57673418>
+
+        * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:
+        * NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp:
+        (WebKit::ResourceLoadStatisticsMemoryStore::hasHadUnexpiredRecentUserInteraction const):
+        (WebKit::ResourceLoadStatisticsMemoryStore::shouldRemoveAllButCookiesFor const):
+        * NetworkProcess/Classifier/ResourceLoadStatisticsStore.cpp:
+        (WebKit::ResourceLoadStatisticsStore::hasStatisticsExpired const):
+        * NetworkProcess/Classifier/ResourceLoadStatisticsStore.h:
+        * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:
+        (WebKit::WebResourceLoadStatisticsStore::setFirstPartyWebsiteDataRemovalMode):
+        * NetworkProcess/NetworkProcess.messages.in:
+        * Shared/WebPreferences.yaml:
+        * UIProcess/WebsiteData/Cocoa/WebsiteDataStoreCocoa.mm:
+        (WebKit::WebsiteDataStore::parameters):
+
 2019-12-05  Chris Dumez  <cdumez@apple.com>
 
         Optimize IPC::Connection::SyncMessageState methods
index 8f89531..dbd0058 100644 (file)
@@ -2143,7 +2143,11 @@ void ResourceLoadStatisticsDatabaseStore::clearGrandfathering(Vector<unsigned>&&
 bool ResourceLoadStatisticsDatabaseStore::hasHadUnexpiredRecentUserInteraction(const DomainData& resourceStatistic, OperatingDatesWindow operatingDatesWindow)
 {
     if (resourceStatistic.hadUserInteraction && hasStatisticsExpired(resourceStatistic.mostRecentUserInteractionTime, operatingDatesWindow)) {
-        clearUserInteraction(resourceStatistic.registrableDomain, [] { });
+
+        // Drop privacy sensitive data if we no longer need it.
+        if (operatingDatesWindow == OperatingDatesWindow::Long)
+            clearUserInteraction(resourceStatistic.registrableDomain, [] { });
+
         return false;
     }
 
@@ -2160,7 +2164,21 @@ bool ResourceLoadStatisticsDatabaseStore::shouldRemoveAllButCookiesFor(const Dom
     bool isRemovalEnabled = firstPartyWebsiteDataRemovalMode() != FirstPartyWebsiteDataRemovalMode::None || resourceStatistic.isScheduledForAllButCookieDataRemoval;
     bool isResourceGrandfathered = shouldCheckForGrandfathering && resourceStatistic.grandfathered;
 
-    return isRemovalEnabled && !isResourceGrandfathered && !hasHadUnexpiredRecentUserInteraction(resourceStatistic, OperatingDatesWindow::Short);
+    OperatingDatesWindow window;
+    switch (firstPartyWebsiteDataRemovalMode()) {
+    case FirstPartyWebsiteDataRemovalMode::AllButCookies:
+        FALLTHROUGH;
+    case FirstPartyWebsiteDataRemovalMode::None:
+        window = OperatingDatesWindow::Short;
+        break;
+    case FirstPartyWebsiteDataRemovalMode::AllButCookiesLiveOnTestingTimeout:
+        window = OperatingDatesWindow::ForLiveOnTesting;
+        break;
+    case FirstPartyWebsiteDataRemovalMode::AllButCookiesReproTestingTimeout:
+        window = OperatingDatesWindow::ForReproTesting;
+    }
+
+    return isRemovalEnabled && !isResourceGrandfathered && !hasHadUnexpiredRecentUserInteraction(resourceStatistic, window);
 }
 
 Vector<std::pair<RegistrableDomain, WebsiteDataToRemove>> ResourceLoadStatisticsDatabaseStore::registrableDomainsToRemoveWebsiteDataFor()
index 340df93..8192853 100644 (file)
@@ -863,12 +863,16 @@ bool ResourceLoadStatisticsMemoryStore::hasHadUnexpiredRecentUserInteraction(Res
     ASSERT(!RunLoop::isMain());
 
     if (resourceStatistic.hadUserInteraction && hasStatisticsExpired(resourceStatistic, operatingDatesWindow)) {
-        // Drop privacy sensitive data because we no longer need it.
-        // Set timestamp to 0 so that statistics merge will know
-        // it has been reset as opposed to its default -1.
-        resourceStatistic.mostRecentUserInteractionTime = { };
-        resourceStatistic.storageAccessUnderTopFrameDomains.clear();
-        resourceStatistic.hadUserInteraction = false;
+        if (operatingDatesWindow == OperatingDatesWindow::Long) {
+            // Drop privacy sensitive data because we no longer need it.
+            // Set timestamp to 0 so that statistics merge will know
+            // it has been reset as opposed to its default -1.
+            resourceStatistic.mostRecentUserInteractionTime = { };
+            resourceStatistic.storageAccessUnderTopFrameDomains.clear();
+            resourceStatistic.hadUserInteraction = false;
+        }
+        
+        return false;
     }
 
     return resourceStatistic.hadUserInteraction;
@@ -883,8 +887,22 @@ bool ResourceLoadStatisticsMemoryStore::shouldRemoveAllButCookiesFor(ResourceLoa
 {
     bool isRemovalEnabled = firstPartyWebsiteDataRemovalMode() != FirstPartyWebsiteDataRemovalMode::None || resourceStatistic.gotLinkDecorationFromPrevalentResource;
     bool isResourceGrandfathered = shouldCheckForGrandfathering && resourceStatistic.grandfathered;
+    
+    OperatingDatesWindow window;
+    switch (firstPartyWebsiteDataRemovalMode()) {
+    case FirstPartyWebsiteDataRemovalMode::AllButCookies:
+        FALLTHROUGH;
+    case FirstPartyWebsiteDataRemovalMode::None:
+        window = OperatingDatesWindow::Short;
+        break;
+    case FirstPartyWebsiteDataRemovalMode::AllButCookiesLiveOnTestingTimeout:
+        window = OperatingDatesWindow::ForLiveOnTesting;
+        break;
+    case FirstPartyWebsiteDataRemovalMode::AllButCookiesReproTestingTimeout:
+        window = OperatingDatesWindow::ForReproTesting;
+    }
 
-    return isRemovalEnabled && !isResourceGrandfathered && !hasHadUnexpiredRecentUserInteraction(resourceStatistic, OperatingDatesWindow::Short);
+    return isRemovalEnabled && !isResourceGrandfathered && !hasHadUnexpiredRecentUserInteraction(resourceStatistic, window);
 }
 
 Vector<std::pair<RegistrableDomain, WebsiteDataToRemove>> ResourceLoadStatisticsMemoryStore::registrableDomainsToRemoveWebsiteDataFor()
index d93545c..7846b54 100644 (file)
@@ -53,6 +53,7 @@ using namespace WebCore;
 constexpr Seconds minimumStatisticsProcessingInterval { 5_s };
 constexpr unsigned operatingDatesWindowLong { 30 };
 constexpr unsigned operatingDatesWindowShort { 7 };
+constexpr Seconds operatingTimeWindowForLiveOnTesting { 1_h };
 
 #if !RELEASE_LOG_DISABLED
 static String domainsToString(const Vector<RegistrableDomain>& domains)
@@ -503,7 +504,20 @@ bool ResourceLoadStatisticsStore::hasStatisticsExpired(WallTime mostRecentUserIn
 {
     ASSERT(!RunLoop::isMain());
 
-    unsigned operatingDatesWindowInDays = (operatingDatesWindow == OperatingDatesWindow::Long ? operatingDatesWindowLong : operatingDatesWindowShort);
+    unsigned operatingDatesWindowInDays;
+    switch (operatingDatesWindow) {
+    case OperatingDatesWindow::Long:
+        operatingDatesWindowInDays = operatingDatesWindowLong;
+        break;
+    case OperatingDatesWindow::Short:
+        operatingDatesWindowInDays = operatingDatesWindowShort;
+        break;
+    case OperatingDatesWindow::ForLiveOnTesting:
+        return WallTime::now() > mostRecentUserInteractionTime + operatingTimeWindowForLiveOnTesting;
+    case OperatingDatesWindow::ForReproTesting:
+        return true;
+    }
+
     if (m_operatingDates.size() >= operatingDatesWindowInDays) {
         if (OperatingDate::fromWallTime(mostRecentUserInteractionTime) < m_operatingDates.first())
             return true;
index 9a5f89a..ae3c8ee 100644 (file)
@@ -74,7 +74,7 @@ private:
     int m_monthDay { 0 }; // [1, 31].
 };
 
-enum class OperatingDatesWindow : bool { Long, Short };
+enum class OperatingDatesWindow : uint8_t { Long, Short, ForLiveOnTesting, ForReproTesting };
 enum class CookieAccess : uint8_t { CannotRequest, BasedOnCookiePolicy, OnlyIfGranted };
 
 // This is always constructed / used / destroyed on the WebResourceLoadStatisticsStore's statistics queue.
index 3ff0a6b..d4586d5 100644 (file)
@@ -486,8 +486,11 @@ void WebResourceLoadStatisticsStore::setFirstPartyWebsiteDataRemovalMode(FirstPa
     ASSERT(RunLoop::isMain());
 
     postTask([this, mode, completionHandler = WTFMove(completionHandler)]() mutable {
-        if (m_statisticsStore)
+        if (m_statisticsStore) {
             m_statisticsStore->setFirstPartyWebsiteDataRemovalMode(mode);
+            if (mode == FirstPartyWebsiteDataRemovalMode::AllButCookiesReproTestingTimeout)
+                m_statisticsStore->setIsRunningTest(true);
+        }
         postTaskReply([completionHandler = WTFMove(completionHandler)]() mutable {
             completionHandler();
         });
index 0257d8d..ee086c0 100644 (file)
@@ -138,7 +138,7 @@ messages -> NetworkProcess LegacyReceiver {
     HasIsolatedSession(PAL::SessionID sessionID, WebCore::RegistrableDomain domain) -> (bool hasIsolatedSession) Async
     SetShouldDowngradeReferrerForTesting(bool enabled) -> () Async
     SetShouldBlockThirdPartyCookiesForTesting(PAL::SessionID sessionID, enum:uint8_t WebCore::ThirdPartyCookieBlockingMode blockingMode) -> () Async
-    SetFirstPartyWebsiteDataRemovalModeForTesting(PAL::SessionID sessionID, enum:bool WebCore::FirstPartyWebsiteDataRemovalMode mode) -> () Async
+    SetFirstPartyWebsiteDataRemovalModeForTesting(PAL::SessionID sessionID, enum:uint8_t WebCore::FirstPartyWebsiteDataRemovalMode mode) -> () Async
 #endif
 
     SetSessionIsControlledByAutomation(PAL::SessionID sessionID, bool controlled);
index c53d664..835fdc8 100644 (file)
@@ -1819,11 +1819,25 @@ IsThirdPartyCookieBlockingEnabled:
 
 IsFirstPartyWebsiteDataRemovalEnabled:
   type: bool
-  defaultValue: false
+  defaultValue: true
   humanReadableName: "Remove Non-Cookie Data After 7 Days of No User Interaction (ITP)"
   humanReadableDescription: "Remove all non-cookie website data after seven days of no user interaction when Intelligent Tracking Prevention is enabled"
   category: experimental
 
+IsFirstPartyWebsiteDataRemovalLiveOnTestingEnabled:
+  type: bool
+  defaultValue: false
+  humanReadableName: "[ITP Live-On] 1 Hour Timeout For Non-Cookie Data Removal"
+  humanReadableDescription: "Remove all non-cookie website data after just one hour of no user interaction when Intelligent Tracking Prevention is enabled"
+  category: internal
+
+IsFirstPartyWebsiteDataRemovalReproTestingEnabled:
+  type: bool
+  defaultValue: false
+  humanReadableName: "[ITP Repro] 30 Second Timeout For Non-Cookie Data Removal"
+  humanReadableDescription: "Remove all non-cookie website data after just 30 seconds of no user interaction when Intelligent Tracking Prevention is enabled"
+  category: internal
+
 IsLoggedInAPIEnabled:
     type: bool
     defaultValue: false
index 3e4b154..1888c8d 100644 (file)
@@ -82,8 +82,14 @@ WebsiteDataStoreParameters WebsiteDataStore::parameters()
         thirdPartyCookieBlockingMode = WebCore::ThirdPartyCookieBlockingMode::All;
     else
         thirdPartyCookieBlockingMode = WebCore::ThirdPartyCookieBlockingMode::AllOnSitesWithoutUserInteraction;
-    if ([defaults boolForKey:[NSString stringWithFormat:@"Experimental%@", WebPreferencesKey::isFirstPartyWebsiteDataRemovalEnabledKey().createCFString().get()]])
-        firstPartyWebsiteDataRemovalMode = WebCore::FirstPartyWebsiteDataRemovalMode::AllButCookies;
+    if ([defaults boolForKey:[NSString stringWithFormat:@"Experimental%@", WebPreferencesKey::isFirstPartyWebsiteDataRemovalEnabledKey().createCFString().get()]]) {
+        if ([defaults boolForKey:[NSString stringWithFormat:@"InternalDebug%@", WebPreferencesKey::isFirstPartyWebsiteDataRemovalReproTestingEnabledKey().createCFString().get()]])
+            firstPartyWebsiteDataRemovalMode = WebCore::FirstPartyWebsiteDataRemovalMode::AllButCookiesReproTestingTimeout;
+        else if ([defaults boolForKey:[NSString stringWithFormat:@"InternalDebug%@", WebPreferencesKey::isFirstPartyWebsiteDataRemovalLiveOnTestingEnabledKey().createCFString().get()]])
+            firstPartyWebsiteDataRemovalMode = WebCore::FirstPartyWebsiteDataRemovalMode::AllButCookiesLiveOnTestingTimeout;
+        else
+            firstPartyWebsiteDataRemovalMode = WebCore::FirstPartyWebsiteDataRemovalMode::AllButCookies;
+    }
     auto* manualPrevalentResource = [defaults stringForKey:@"ITPManualPrevalentResource"];
     if (manualPrevalentResource) {
         URL url { URL(), manualPrevalentResource };