Add telemetry to track storage access API adoption
authorbfulgham@apple.com <bfulgham@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 29 Jan 2018 22:00:58 +0000 (22:00 +0000)
committerbfulgham@apple.com <bfulgham@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 29 Jan 2018 22:00:58 +0000 (22:00 +0000)
https://bugs.webkit.org/show_bug.cgi?id=182197
<rdar://problem/35803309>

Reviewed by Chris Dumez.
Source/WebCore:

Part 1: Add telemetry for the user interaction case

This patch adds telemetry to track how frequently third-party cookies are
used in a first party context due to user interaction. This will help
understand cases where the new Storage Access API can help, and to help
us understand if we have considered relevant use cases in its design.

* loader/ResourceLoadObserver.cpp:
(WebCore::ResourceLoadObserver::setTimeToLivePartitionFree): Let the observer
know the first party interaction duration.
(WebCore::ResourceLoadObserver::wasAccessedWithinInteractionWindow const): Added.
(WebCore::ResourceLoadObserver::logFrameNavigation): Note when a third party
resource is accessed as a first party due to user interaction.
(WebCore::ResourceLoadObserver::logSubresourceLoading): Ditto.
* loader/ResourceLoadObserver.h:
* loader/ResourceLoadStatistics.cpp:
(WebCore::ResourceLoadStatistics::encode const): Handle new fields.
(WebCore::ResourceLoadStatistics::decode): Ditto.
* loader/ResourceLoadStatistics.h:

Source/WebKit:

Part 1: Add telemetry for the user interaction case

This patch adds telemetry to track how frequently third-party cookies are
used in a first party context due to user interaction. This will help
understand cases where the new Storage Access API can help, and to help
us understand if we have considered relevant use cases in its design.

* Shared/WebProcessCreationParameters.cpp:
(WebKit::WebProcessCreationParameters::encode const):
(WebKit::WebProcessCreationParameters::decode):
* Shared/WebProcessCreationParameters.h:
* UIProcess/Cocoa/WebProcessPoolCocoa.mm:
(WebKit::WebProcessPool::platformInitializeWebProcess):
* UIProcess/WebResourceLoadStatisticsTelemetry.cpp:
(WebKit::sortedPrevalentResourceTelemetry): Update for new telemetry.
(WebKit::submitTopList): Update for new data types.
* WebProcess/WebProcess.cpp:
(WebKit::WebProcess::initializeWebProcess): Handle the partitioning time
passed from the UIProcess.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@227755 268f45cc-cd09-0410-ab3c-d52691b4dbfc

Source/WebCore/ChangeLog
Source/WebCore/loader/ResourceLoadObserver.cpp
Source/WebCore/loader/ResourceLoadObserver.h
Source/WebCore/loader/ResourceLoadStatistics.cpp
Source/WebCore/loader/ResourceLoadStatistics.h
Source/WebKit/ChangeLog
Source/WebKit/Shared/WebProcessCreationParameters.cpp
Source/WebKit/Shared/WebProcessCreationParameters.h
Source/WebKit/UIProcess/Cocoa/WebProcessPoolCocoa.mm
Source/WebKit/UIProcess/WebResourceLoadStatisticsTelemetry.cpp
Source/WebKit/WebProcess/WebProcess.cpp

index cecf22f..6b28d69 100644 (file)
@@ -1,3 +1,31 @@
+2018-01-29  Brent Fulgham  <bfulgham@apple.com>
+
+        Add telemetry to track storage access API adoption
+        https://bugs.webkit.org/show_bug.cgi?id=182197
+        <rdar://problem/35803309>
+
+        Reviewed by Chris Dumez.
+        
+        Part 1: Add telemetry for the user interaction case
+        
+        This patch adds telemetry to track how frequently third-party cookies are
+        used in a first party context due to user interaction. This will help
+        understand cases where the new Storage Access API can help, and to help
+        us understand if we have considered relevant use cases in its design.
+
+        * loader/ResourceLoadObserver.cpp:
+        (WebCore::ResourceLoadObserver::setTimeToLivePartitionFree): Let the observer
+        know the first party interaction duration.
+        (WebCore::ResourceLoadObserver::wasAccessedWithinInteractionWindow const): Added.
+        (WebCore::ResourceLoadObserver::logFrameNavigation): Note when a third party 
+        resource is accessed as a first party due to user interaction. 
+        (WebCore::ResourceLoadObserver::logSubresourceLoading): Ditto.
+        * loader/ResourceLoadObserver.h:
+        * loader/ResourceLoadStatistics.cpp:
+        (WebCore::ResourceLoadStatistics::encode const): Handle new fields.
+        (WebCore::ResourceLoadStatistics::decode): Ditto.
+        * loader/ResourceLoadStatistics.h:
+
 2018-01-29  Antti Koivisto  <antti@apple.com>
 
         CalcExpressionBlendLength::evaluate hits stack limit
index 8d0fa4b..44ad4a7 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2016-2017 Apple Inc. All rights reserved.
+ * Copyright (C) 2016-2018 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -132,6 +132,16 @@ static WallTime reduceToHourlyTimeResolution(WallTime time)
     return WallTime::fromRawSeconds(std::floor(time.secondsSinceEpoch() / timestampResolution) * timestampResolution.seconds());
 }
 
+void ResourceLoadObserver::setTimeToLivePartitionFree(Seconds value)
+{
+    m_timeToLiveCookiePartitionFree = value;
+}
+
+bool ResourceLoadObserver::wasAccessedWithinInteractionWindow(const ResourceLoadStatistics& statistic) const
+{
+    return WallTime::now() <= statistic.mostRecentUserInteractionTime + m_timeToLiveCookiePartitionFree;
+}
+
 void ResourceLoadObserver::logFrameNavigation(const Frame& frame, const Frame& topFrame, const ResourceRequest& newRequest, const URL& redirectUrl)
 {
     ASSERT(frame.document());
@@ -171,6 +181,8 @@ void ResourceLoadObserver::logFrameNavigation(const Frame& frame, const Frame& t
         && !(areDomainsAssociated(page, targetPrimaryDomain, mainFramePrimaryDomain) || areDomainsAssociated(page, targetPrimaryDomain, sourcePrimaryDomain))) {
         auto& targetStatistics = ensureResourceStatisticsForPrimaryDomain(targetPrimaryDomain);
         targetStatistics.lastSeen = reduceToHourlyTimeResolution(WallTime::now());
+        if (targetStatistics.hadUserInteraction && wasAccessedWithinInteractionWindow(targetStatistics))
+            targetStatistics.timesAccessedAsFirstPartyDueToUserInteraction++;
         if (targetStatistics.subframeUnderTopFrameOrigins.add(mainFramePrimaryDomain).isNewEntry)
             shouldCallNotificationCallback = true;
     }
@@ -228,6 +240,8 @@ void ResourceLoadObserver::logSubresourceLoading(const Frame* frame, const Resou
     {
         auto& targetStatistics = ensureResourceStatisticsForPrimaryDomain(targetPrimaryDomain);
         targetStatistics.lastSeen = reduceToHourlyTimeResolution(WallTime::now());
+        if (targetStatistics.hadUserInteraction && wasAccessedWithinInteractionWindow(targetStatistics))
+            targetStatistics.timesAccessedAsFirstPartyDueToUserInteraction++;
         if (targetStatistics.subresourceUnderTopFrameOrigins.add(mainFramePrimaryDomain).isNewEntry)
             shouldCallNotificationCallback = true;
     }
index 59f3293..2a4de57 100644 (file)
@@ -69,12 +69,14 @@ public:
     bool shouldLogUserInteraction() const { return m_shouldLogUserInteraction; }
     void setShouldLogUserInteraction(bool shouldLogUserInteraction) { m_shouldLogUserInteraction = shouldLogUserInteraction; }
 #endif
+    WEBCORE_EXPORT void setTimeToLivePartitionFree(Seconds);
 
 private:
     ResourceLoadObserver();
 
     bool shouldLog(Page*) const;
     ResourceLoadStatistics& ensureResourceStatisticsForPrimaryDomain(const String&);
+    bool wasAccessedWithinInteractionWindow(const ResourceLoadStatistics&) const;
 
     void scheduleNotificationIfNeeded();
     Vector<ResourceLoadStatistics> takeStatistics();
@@ -83,6 +85,7 @@ private:
     HashMap<String, WTF::WallTime> m_lastReportedUserInteractionMap;
     WTF::Function<void (Vector<ResourceLoadStatistics>&&)> m_notificationCallback;
     Timer m_notificationTimer;
+    Seconds m_timeToLiveCookiePartitionFree { 24_h };
 #if HAVE(CFNETWORK_STORAGE_PARTITIONING) && !RELEASE_LOG_DISABLED
     uint64_t m_loggingCounter { 0 };
     bool m_shouldLogUserInteraction { false };
index 92cae4b..46d13ac 100644 (file)
@@ -80,6 +80,9 @@ void ResourceLoadStatistics::encode(KeyedEncoder& encoder) const
     // Prevalent Resource
     encoder.encodeBool("isPrevalentResource", isPrevalentResource);
     encoder.encodeUInt32("dataRecordsRemoved", dataRecordsRemoved);
+
+    encoder.encodeUInt32("timesAccessedAsFirstPartyDueToUserInteraction", timesAccessedAsFirstPartyDueToUserInteraction);
+    encoder.encodeUInt32("timesAccessedAsFirstPartyDueToStorageAccessAPI", timesAccessedAsFirstPartyDueToStorageAccessAPI);
 }
 
 static void decodeHashCountedSet(KeyedDecoder& decoder, const String& label, HashCountedSet<String>& hashCountedSet)
@@ -148,7 +151,12 @@ bool ResourceLoadStatistics::decode(KeyedDecoder& decoder)
     if (!decoder.decodeDouble("lastSeen", lastSeenTimeAsDouble))
         return false;
     lastSeen = WallTime::fromRawSeconds(lastSeenTimeAsDouble);
-    
+
+    if (!decoder.decodeUInt32("timesAccessedAsFirstPartyDueToUserInteraction", timesAccessedAsFirstPartyDueToUserInteraction))
+        timesAccessedAsFirstPartyDueToUserInteraction = 0;
+    if (!decoder.decodeUInt32("timesAccessedAsFirstPartyDueToStorageAccessAPI", timesAccessedAsFirstPartyDueToStorageAccessAPI))
+        timesAccessedAsFirstPartyDueToStorageAccessAPI = 0;
+
     return true;
 }
 
index 48084f7..1ea7afd 100644 (file)
@@ -83,6 +83,8 @@ struct ResourceLoadStatistics {
     // Prevalent resource stats
     bool isPrevalentResource { false };
     unsigned dataRecordsRemoved { 0 };
+    unsigned timesAccessedAsFirstPartyDueToUserInteraction { 0 };
+    unsigned timesAccessedAsFirstPartyDueToStorageAccessAPI { 0 };
 
     // In-memory only
     bool isMarkedForCookiePartitioning { false };
index 280f1fb..03f7c07 100644 (file)
@@ -1,3 +1,31 @@
+2018-01-29  Brent Fulgham  <bfulgham@apple.com>
+
+        Add telemetry to track storage access API adoption
+        https://bugs.webkit.org/show_bug.cgi?id=182197
+        <rdar://problem/35803309>
+
+        Reviewed by Chris Dumez.
+
+        Part 1: Add telemetry for the user interaction case
+        
+        This patch adds telemetry to track how frequently third-party cookies are
+        used in a first party context due to user interaction. This will help
+        understand cases where the new Storage Access API can help, and to help
+        us understand if we have considered relevant use cases in its design.
+
+        * Shared/WebProcessCreationParameters.cpp:
+        (WebKit::WebProcessCreationParameters::encode const):
+        (WebKit::WebProcessCreationParameters::decode):
+        * Shared/WebProcessCreationParameters.h:
+        * UIProcess/Cocoa/WebProcessPoolCocoa.mm:
+        (WebKit::WebProcessPool::platformInitializeWebProcess):
+        * UIProcess/WebResourceLoadStatisticsTelemetry.cpp:
+        (WebKit::sortedPrevalentResourceTelemetry): Update for new telemetry.
+        (WebKit::submitTopList): Update for new data types.
+        * WebProcess/WebProcess.cpp:
+        (WebKit::WebProcess::initializeWebProcess): Handle the partitioning time
+        passed from the UIProcess.
+
 2018-01-29  Alex Christensen  <achristensen@webkit.org>
 
         Fix crash when during canAuthenticateAgainstProtectionSpace
index bab8ecc..2ff207d 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2010-2017 Apple Inc. All rights reserved.
+ * Copyright (C) 2010-2018 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -154,6 +154,8 @@ void WebProcessCreationParameters::encode(IPC::Encoder& encoder) const
 #if HAVE(CFNETWORK_STORAGE_PARTITIONING) && !RELEASE_LOG_DISABLED
     encoder << shouldLogUserInteraction;
 #endif
+
+    encoder << cookiePartitionTimeToLive;
 }
 
 bool WebProcessCreationParameters::decode(IPC::Decoder& decoder, WebProcessCreationParameters& parameters)
@@ -400,6 +402,9 @@ bool WebProcessCreationParameters::decode(IPC::Decoder& decoder, WebProcessCreat
         return false;
 #endif
 
+    if (!decoder.decode(parameters.cookiePartitionTimeToLive))
+        return false;
+
     return true;
 }
 
index bc5d9c3..8d2e1fd 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2010-2017 Apple Inc. All rights reserved.
+ * Copyright (C) 2010-2018 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -188,6 +188,7 @@ struct WebProcessCreationParameters {
 #if HAVE(CFNETWORK_STORAGE_PARTITIONING) && !RELEASE_LOG_DISABLED
     bool shouldLogUserInteraction { false };
 #endif
+    Seconds cookiePartitionTimeToLive;
 };
 
 } // namespace WebKit
index fb6e245..0fc44a1 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2010-2017 Apple Inc. All rights reserved.
+ * Copyright (C) 2010-2018 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -279,6 +279,11 @@ void WebProcessPool::platformInitializeWebProcess(WebProcessCreationParameters&
 #if HAVE(CFNETWORK_STORAGE_PARTITIONING) && !RELEASE_LOG_DISABLED
     parameters.shouldLogUserInteraction = [defaults boolForKey:WebKitLogCookieInformationDefaultsKey];
 #endif
+
+    Seconds timeToLiveUserInteraction([[NSUserDefaults standardUserDefaults] doubleForKey:@"ResourceLoadStatisticsTimeToLiveUserInteraction"]);
+    if (timeToLiveUserInteraction < 0_s || timeToLiveUserInteraction > 24_h * 30)
+        timeToLiveUserInteraction = 24_h;
+    parameters.cookiePartitionTimeToLive = timeToLiveUserInteraction;
 }
 
 void WebProcessPool::platformInitializeNetworkProcess(NetworkProcessCreationParameters& parameters)
index 3e8ec41..85a5e16 100644 (file)
@@ -51,6 +51,8 @@ struct PrevalentResourceTelemetry {
     unsigned subframeUnderTopFrameOrigins;
     unsigned subresourceUnderTopFrameOrigins;
     unsigned subresourceUniqueRedirectsTo;
+    unsigned timesAccessedAsFirstPartyDueToUserInteraction;
+    unsigned timesAccessedAsFirstPartyDueToStorageAccessAPI;
 };
 
 static Vector<PrevalentResourceTelemetry> sortedPrevalentResourceTelemetry(const WebResourceLoadStatisticsStore& store)
@@ -68,7 +70,9 @@ static Vector<PrevalentResourceTelemetry> sortedPrevalentResourceTelemetry(const
             daysSinceUserInteraction,
             statistic.subframeUnderTopFrameOrigins.size(),
             statistic.subresourceUnderTopFrameOrigins.size(),
-            statistic.subresourceUniqueRedirectsTo.size()
+            statistic.subresourceUniqueRedirectsTo.size(),
+            statistic.timesAccessedAsFirstPartyDueToUserInteraction,
+            statistic.timesAccessedAsFirstPartyDueToStorageAccessAPI
         });
     });
 
@@ -146,25 +150,33 @@ static WebPageProxy* nonEphemeralWebPageProxy()
     
 static void submitTopList(unsigned numberOfResourcesFromTheTop, const Vector<PrevalentResourceTelemetry>& sortedPrevalentResources, const Vector<PrevalentResourceTelemetry>& sortedPrevalentResourcesWithoutUserInteraction, WebPageProxy& webPageProxy)
 {
-    WTF::Function<unsigned(const PrevalentResourceTelemetry& telemetry)> subframeUnderTopFrameOriginsGetter = [] (const PrevalentResourceTelemetry& t) {
+    WTF::Function<unsigned(const PrevalentResourceTelemetry& telemetry)> subframeUnderTopFrameOriginsGetter = [] (auto& t) {
         return t.subframeUnderTopFrameOrigins;
     };
-    WTF::Function<unsigned(const PrevalentResourceTelemetry& telemetry)> subresourceUnderTopFrameOriginsGetter = [] (const PrevalentResourceTelemetry& t) {
+    WTF::Function<unsigned(const PrevalentResourceTelemetry& telemetry)> subresourceUnderTopFrameOriginsGetter = [] (auto& t) {
         return t.subresourceUnderTopFrameOrigins;
     };
-    WTF::Function<unsigned(const PrevalentResourceTelemetry& telemetry)> subresourceUniqueRedirectsToGetter = [] (const PrevalentResourceTelemetry& t) {
+    WTF::Function<unsigned(const PrevalentResourceTelemetry& telemetry)> subresourceUniqueRedirectsToGetter = [] (auto& t) {
         return t.subresourceUniqueRedirectsTo;
     };
-    WTF::Function<unsigned(const PrevalentResourceTelemetry& telemetry)> numberOfTimesDataRecordsRemovedGetter = [] (const PrevalentResourceTelemetry& t) {
+    WTF::Function<unsigned(const PrevalentResourceTelemetry& telemetry)> numberOfTimesDataRecordsRemovedGetter = [] (auto& t) {
         return t.numberOfTimesDataRecordsRemoved;
     };
-    
+    WTF::Function<unsigned(const PrevalentResourceTelemetry& telemetry)> numberOfTimesAccessedAsFirstPartyDueToUserInteractionGetter = [] (auto& t) {
+        return t.timesAccessedAsFirstPartyDueToUserInteraction;
+    };
+    WTF::Function<unsigned(const PrevalentResourceTelemetry& telemetry)> numberOfTimesAccessedAsFirstPartyDueToStorageAccessAPIGetter = [] (auto& t) {
+        return t.timesAccessedAsFirstPartyDueToStorageAccessAPI;
+    };
+
     unsigned topPrevalentResourcesWithUserInteraction = numberOfResourcesWithUserInteraction(sortedPrevalentResources, 0, numberOfResourcesFromTheTop - 1);
     unsigned topSubframeUnderTopFrameOrigins = median(sortedPrevalentResourcesWithoutUserInteraction, 0, numberOfResourcesFromTheTop - 1, subframeUnderTopFrameOriginsGetter);
     unsigned topSubresourceUnderTopFrameOrigins = median(sortedPrevalentResourcesWithoutUserInteraction, 0, numberOfResourcesFromTheTop - 1, subresourceUnderTopFrameOriginsGetter);
     unsigned topSubresourceUniqueRedirectsTo = median(sortedPrevalentResourcesWithoutUserInteraction, 0, numberOfResourcesFromTheTop - 1, subresourceUniqueRedirectsToGetter);
     unsigned topNumberOfTimesDataRecordsRemoved = median(sortedPrevalentResourcesWithoutUserInteraction, 0, numberOfResourcesFromTheTop - 1, numberOfTimesDataRecordsRemovedGetter);
-    
+    unsigned topNumberOfTimesAccessedAsFirstPartyDueToUserInteraction = median(sortedPrevalentResourcesWithoutUserInteraction, 0, numberOfResourcesFromTheTop - 1, numberOfTimesAccessedAsFirstPartyDueToUserInteractionGetter);
+    unsigned topNumberOfTimesAccessedAsFirstPartyDueToStorageAccessAPI = median(sortedPrevalentResourcesWithoutUserInteraction, 0, numberOfResourcesFromTheTop - 1, numberOfTimesAccessedAsFirstPartyDueToStorageAccessAPIGetter);
+
     StringBuilder preambleBuilder;
     preambleBuilder.appendLiteral("top");
     preambleBuilder.appendNumber(numberOfResourcesFromTheTop);
@@ -180,6 +192,10 @@ static void submitTopList(unsigned numberOfResourcesFromTheTop, const Vector<Pre
         topSubresourceUniqueRedirectsTo, significantFiguresForLoggedValues, ShouldSample::No);
     webPageProxy.logDiagnosticMessageWithValue(DiagnosticLoggingKeys::resourceLoadStatisticsTelemetryKey(), descriptionPreamble + "NumberOfTimesDataRecordsRemoved",
         topNumberOfTimesDataRecordsRemoved, significantFiguresForLoggedValues, ShouldSample::No);
+    webPageProxy.logDiagnosticMessageWithValue(DiagnosticLoggingKeys::resourceLoadStatisticsTelemetryKey(), descriptionPreamble + "NumberOfTimesAccessedAsFirstPartyDueToUserInteraction",
+        topNumberOfTimesAccessedAsFirstPartyDueToUserInteraction, significantFiguresForLoggedValues, ShouldSample::No);
+    webPageProxy.logDiagnosticMessageWithValue(DiagnosticLoggingKeys::resourceLoadStatisticsTelemetryKey(), descriptionPreamble + "NumberOfTimesAccessedAsFirstPartyDueToStorageAccessAPI",
+        topNumberOfTimesAccessedAsFirstPartyDueToStorageAccessAPI, significantFiguresForLoggedValues, ShouldSample::No);
 }
     
 static void submitTopLists(const Vector<PrevalentResourceTelemetry>& sortedPrevalentResources, const Vector<PrevalentResourceTelemetry>& sortedPrevalentResourcesWithoutUserInteraction, WebPageProxy& webPageProxy)
index e7caddc..4232e29 100644 (file)
@@ -424,6 +424,8 @@ void WebProcess::initializeWebProcess(WebProcessCreationParameters&& parameters)
     ResourceLoadObserver::shared().setShouldLogUserInteraction(parameters.shouldLogUserInteraction);
 #endif
 
+    ResourceLoadObserver::shared().setTimeToLivePartitionFree(parameters.cookiePartitionTimeToLive);
+
     RELEASE_LOG(Process, "%p - WebProcess::initializeWebProcess: Presenting process = %d", this, WebCore::presentingApplicationPID());
 }