Resource Load Statistics: Introduce debug mode as experimental feature
authorwilander@apple.com <wilander@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 30 Jan 2018 00:25:58 +0000 (00:25 +0000)
committerwilander@apple.com <wilander@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 30 Jan 2018 00:25:58 +0000 (00:25 +0000)
https://bugs.webkit.org/show_bug.cgi?id=182199
<rdar://problem/36930364>

Reviewed by Alex Christensen.

Source/WebCore:

No new tests. This adds an experimental feature.

The only changes to default behavior are:
- Increased resolution on timestamps which is needed to be able to set shorter
  timeouts in debug mode.
- Only update partitioning and blocking table when needed. This is an optimization
  which pays off in less XPC with shorter timeouts.

* loader/ResourceLoadObserver.cpp:
(WebCore::reduceTimeResolution):
(WebCore::ResourceLoadObserver::logFrameNavigation):
(WebCore::ResourceLoadObserver::logSubresourceLoading):
(WebCore::ResourceLoadObserver::logWebSocketLoading):
(WebCore::ResourceLoadObserver::logUserInteractionWithReducedTimeResolution):
(WebCore::reduceToHourlyTimeResolution): Deleted.
* page/RuntimeEnabledFeatures.h:
(WebCore::RuntimeEnabledFeatures::setResourceLoadStatisticsDebugMode):
(WebCore::RuntimeEnabledFeatures::resourceLoadStatisticsDebugMode const):
* page/Settings.yaml:

Source/WebKit:

The only changes to default behavior are:
- Increased resolution on timestamps which is needed to be able to set shorter
  timeouts in debug mode.
- Only update partitioning and blocking table when needed. This is an optimization
  which pays off in less XPC with shorter timeouts.

* Shared/WebPreferences.yaml:
* UIProcess/API/APIWebsiteDataStore.cpp:
(API::WebsiteDataStore::resourceLoadStatisticsDebugMode const):
(API::WebsiteDataStore::setResourceLoadStatisticsDebugMode):
* UIProcess/API/APIWebsiteDataStore.h:
* UIProcess/API/C/WKWebsiteDataStoreRef.cpp:
(WKWebsiteDataStoreSetResourceLoadStatisticsDebugMode):
* UIProcess/API/C/WKWebsiteDataStoreRef.h:
* UIProcess/API/Cocoa/WKWebsiteDataStore.mm:
(-[WKWebsiteDataStore _resourceLoadStatisticsDebugMode]):
(-[WKWebsiteDataStore _setResourceLoadStatisticsDebugMode:]):
* UIProcess/API/Cocoa/WKWebsiteDataStorePrivate.h:
* UIProcess/WebResourceLoadStatisticsStore.cpp:
(WebKit::WebResourceLoadStatisticsStore::setResourceLoadStatisticsDebugMode):
(WebKit::WebResourceLoadStatisticsStore::logUserInteraction):
* UIProcess/WebResourceLoadStatisticsStore.h:
* UIProcess/WebsiteData/WebsiteDataStore.cpp:
(WebKit::WebsiteDataStore::resourceLoadStatisticsDebugMode const):
(WebKit::WebsiteDataStore::setResourceLoadStatisticsDebugMode):
* UIProcess/WebsiteData/WebsiteDataStore.h:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@227762 268f45cc-cd09-0410-ab3c-d52691b4dbfc

16 files changed:
Source/WebCore/ChangeLog
Source/WebCore/loader/ResourceLoadObserver.cpp
Source/WebCore/page/RuntimeEnabledFeatures.h
Source/WebCore/page/Settings.yaml
Source/WebKit/ChangeLog
Source/WebKit/Shared/WebPreferences.yaml
Source/WebKit/UIProcess/API/APIWebsiteDataStore.cpp
Source/WebKit/UIProcess/API/APIWebsiteDataStore.h
Source/WebKit/UIProcess/API/C/WKWebsiteDataStoreRef.cpp
Source/WebKit/UIProcess/API/C/WKWebsiteDataStoreRef.h
Source/WebKit/UIProcess/API/Cocoa/WKWebsiteDataStore.mm
Source/WebKit/UIProcess/API/Cocoa/WKWebsiteDataStorePrivate.h
Source/WebKit/UIProcess/WebResourceLoadStatisticsStore.cpp
Source/WebKit/UIProcess/WebResourceLoadStatisticsStore.h
Source/WebKit/UIProcess/WebsiteData/WebsiteDataStore.cpp
Source/WebKit/UIProcess/WebsiteData/WebsiteDataStore.h

index 4967f71..569d990 100644 (file)
@@ -1,3 +1,31 @@
+2018-01-29  John Wilander  <wilander@apple.com>
+
+        Resource Load Statistics: Introduce debug mode as experimental feature
+        https://bugs.webkit.org/show_bug.cgi?id=182199
+        <rdar://problem/36930364>
+
+        Reviewed by Alex Christensen.
+
+        No new tests. This adds an experimental feature.
+
+        The only changes to default behavior are:
+        - Increased resolution on timestamps which is needed to be able to set shorter
+          timeouts in debug mode.
+        - Only update partitioning and blocking table when needed. This is an optimization
+          which pays off in less XPC with shorter timeouts.
+
+        * loader/ResourceLoadObserver.cpp:
+        (WebCore::reduceTimeResolution):
+        (WebCore::ResourceLoadObserver::logFrameNavigation):
+        (WebCore::ResourceLoadObserver::logSubresourceLoading):
+        (WebCore::ResourceLoadObserver::logWebSocketLoading):
+        (WebCore::ResourceLoadObserver::logUserInteractionWithReducedTimeResolution):
+        (WebCore::reduceToHourlyTimeResolution): Deleted.
+        * page/RuntimeEnabledFeatures.h:
+        (WebCore::RuntimeEnabledFeatures::setResourceLoadStatisticsDebugMode):
+        (WebCore::RuntimeEnabledFeatures::resourceLoadStatisticsDebugMode const):
+        * page/Settings.yaml:
+
 2018-01-29  Oleksandr Skachkov  <gskachkov@gmail.com>
 
         FetchResponse should support ConsumeData callback on chunk data is received: handling ReadableStream bodies
index 44ad4a7..74dd134 100644 (file)
@@ -48,7 +48,7 @@ template<typename T> static inline String primaryDomain(const T& value)
     return ResourceLoadStatistics::primaryDomain(value);
 }
 
-static Seconds timestampResolution { 1_h };
+static Seconds timestampResolution { 5_s };
 static const Seconds minimumNotificationInterval { 5_s };
 
 ResourceLoadObserver& ResourceLoadObserver::shared()
@@ -127,7 +127,7 @@ bool ResourceLoadObserver::shouldLog(Page* page) const
     return DeprecatedGlobalSettings::resourceLoadStatisticsEnabled() && !page->usesEphemeralSession() && m_notificationCallback;
 }
 
-static WallTime reduceToHourlyTimeResolution(WallTime time)
+static WallTime reduceTimeResolution(WallTime time)
 {
     return WallTime::fromRawSeconds(std::floor(time.secondsSinceEpoch() / timestampResolution) * timestampResolution.seconds());
 }
@@ -180,7 +180,7 @@ void ResourceLoadObserver::logFrameNavigation(const Frame& frame, const Frame& t
     if (targetHost != mainFrameHost
         && !(areDomainsAssociated(page, targetPrimaryDomain, mainFramePrimaryDomain) || areDomainsAssociated(page, targetPrimaryDomain, sourcePrimaryDomain))) {
         auto& targetStatistics = ensureResourceStatisticsForPrimaryDomain(targetPrimaryDomain);
-        targetStatistics.lastSeen = reduceToHourlyTimeResolution(WallTime::now());
+        targetStatistics.lastSeen = reduceTimeResolution(WallTime::now());
         if (targetStatistics.hadUserInteraction && wasAccessedWithinInteractionWindow(targetStatistics))
             targetStatistics.timesAccessedAsFirstPartyDueToUserInteraction++;
         if (targetStatistics.subframeUnderTopFrameOrigins.add(mainFramePrimaryDomain).isNewEntry)
@@ -239,7 +239,7 @@ void ResourceLoadObserver::logSubresourceLoading(const Frame* frame, const Resou
     bool shouldCallNotificationCallback = false;
     {
         auto& targetStatistics = ensureResourceStatisticsForPrimaryDomain(targetPrimaryDomain);
-        targetStatistics.lastSeen = reduceToHourlyTimeResolution(WallTime::now());
+        targetStatistics.lastSeen = reduceTimeResolution(WallTime::now());
         if (targetStatistics.hadUserInteraction && wasAccessedWithinInteractionWindow(targetStatistics))
             targetStatistics.timesAccessedAsFirstPartyDueToUserInteraction++;
         if (targetStatistics.subresourceUnderTopFrameOrigins.add(mainFramePrimaryDomain).isNewEntry)
@@ -282,7 +282,7 @@ void ResourceLoadObserver::logWebSocketLoading(const Frame* frame, const URL& ta
         return;
 
     auto& targetStatistics = ensureResourceStatisticsForPrimaryDomain(targetPrimaryDomain);
-    targetStatistics.lastSeen = reduceToHourlyTimeResolution(WallTime::now());
+    targetStatistics.lastSeen = reduceTimeResolution(WallTime::now());
     if (targetStatistics.subresourceUnderTopFrameOrigins.add(mainFramePrimaryDomain).isNewEntry)
         scheduleNotificationIfNeeded();
 }
@@ -299,7 +299,7 @@ void ResourceLoadObserver::logUserInteractionWithReducedTimeResolution(const Doc
         return;
 
     auto domain = primaryDomain(url);
-    auto newTime = reduceToHourlyTimeResolution(WallTime::now());
+    auto newTime = reduceTimeResolution(WallTime::now());
     auto lastReportedUserInteraction = m_lastReportedUserInteractionMap.get(domain);
     if (newTime == lastReportedUserInteraction)
         return;
index 93d572c..5ae32d7 100644 (file)
@@ -244,6 +244,9 @@ public:
     void setMediaCapabilitiesEnabled(bool isEnabled) { m_mediaCapabilitiesEnabled = isEnabled; }
     bool mediaCapabilitiesEnabled() const { return m_mediaCapabilitiesEnabled; }
 
+    void setResourceLoadStatisticsDebugMode(bool isEnabled) { m_resourceLoadStatisticsDebugMode = isEnabled; }
+    bool resourceLoadStatisticsDebugMode() const { return m_resourceLoadStatisticsDebugMode; }
+
     WEBCORE_EXPORT static RuntimeEnabledFeatures& sharedFeatures();
 
 private:
@@ -373,6 +376,8 @@ private:
 
     bool m_mediaCapabilitiesEnabled { false };
 
+    bool m_resourceLoadStatisticsDebugMode { false };
+    
     friend class WTF::NeverDestroyed<RuntimeEnabledFeatures>;
 };
 
index 4feac48..2be2892 100644 (file)
@@ -711,3 +711,6 @@ touchEventEmulationEnabled:
 
 mediaCapabilitiesEnabled:
   initial: false
+
+resourceLoadStatisticsDebugMode:
+  initial: false
index d559245..bad3b29 100644 (file)
@@ -1,3 +1,38 @@
+2018-01-29  John Wilander  <wilander@apple.com>
+
+        Resource Load Statistics: Introduce debug mode as experimental feature
+        https://bugs.webkit.org/show_bug.cgi?id=182199
+        <rdar://problem/36930364>
+
+        Reviewed by Alex Christensen.
+
+        The only changes to default behavior are:
+        - Increased resolution on timestamps which is needed to be able to set shorter
+          timeouts in debug mode.
+        - Only update partitioning and blocking table when needed. This is an optimization
+          which pays off in less XPC with shorter timeouts.
+
+        * Shared/WebPreferences.yaml:
+        * UIProcess/API/APIWebsiteDataStore.cpp:
+        (API::WebsiteDataStore::resourceLoadStatisticsDebugMode const):
+        (API::WebsiteDataStore::setResourceLoadStatisticsDebugMode):
+        * UIProcess/API/APIWebsiteDataStore.h:
+        * UIProcess/API/C/WKWebsiteDataStoreRef.cpp:
+        (WKWebsiteDataStoreSetResourceLoadStatisticsDebugMode):
+        * UIProcess/API/C/WKWebsiteDataStoreRef.h:
+        * UIProcess/API/Cocoa/WKWebsiteDataStore.mm:
+        (-[WKWebsiteDataStore _resourceLoadStatisticsDebugMode]):
+        (-[WKWebsiteDataStore _setResourceLoadStatisticsDebugMode:]):
+        * UIProcess/API/Cocoa/WKWebsiteDataStorePrivate.h:
+        * UIProcess/WebResourceLoadStatisticsStore.cpp:
+        (WebKit::WebResourceLoadStatisticsStore::setResourceLoadStatisticsDebugMode):
+        (WebKit::WebResourceLoadStatisticsStore::logUserInteraction):
+        * UIProcess/WebResourceLoadStatisticsStore.h:
+        * UIProcess/WebsiteData/WebsiteDataStore.cpp:
+        (WebKit::WebsiteDataStore::resourceLoadStatisticsDebugMode const):
+        (WebKit::WebsiteDataStore::setResourceLoadStatisticsDebugMode):
+        * UIProcess/WebsiteData/WebsiteDataStore.h:
+
 2018-01-29  Andy Estes  <aestes@apple.com>
 
         [iOS] Restrict synthetic clicks to the origin that handled the underlying touch event
index 43d5063..174afb4 100644 (file)
@@ -1193,4 +1193,12 @@ WebVREnabled:
   humanReadableDescription: "WebVR Module support"
   webcoreBinding: RuntimeEnabledFeatures
   category: experimental
-  condition: PLATFORM(GTK) || PLATFORM(WPE)
\ No newline at end of file
+  condition: PLATFORM(GTK) || PLATFORM(WPE)
+
+ResourceLoadStatisticsDebugMode:
+  type: bool
+  defaultValue: false
+  humanReadableName: "ITP Debug Mode"
+  humanReadableDescription: "Intelligent Tracking Prevention Debug Mode"
+  category: experimental
+  webcoreBinding: RuntimeEnabledFeatures
index 4b8e2a5..d0472c8 100644 (file)
@@ -95,6 +95,16 @@ void WebsiteDataStore::setResourceLoadStatisticsEnabled(bool enabled)
     m_websiteDataStore->setResourceLoadStatisticsEnabled(enabled);
 }
 
+bool WebsiteDataStore::resourceLoadStatisticsDebugMode() const
+{
+    return m_websiteDataStore->resourceLoadStatisticsDebugMode();
+}
+
+void WebsiteDataStore::setResourceLoadStatisticsDebugMode(bool enabled)
+{
+    m_websiteDataStore->setResourceLoadStatisticsDebugMode(enabled);
+}
+
 #if !PLATFORM(COCOA)
 String WebsiteDataStore::defaultMediaCacheDirectory()
 {
index 4d39f2f..768555d 100644 (file)
@@ -47,6 +47,8 @@ public:
 
     bool resourceLoadStatisticsEnabled() const;
     void setResourceLoadStatisticsEnabled(bool);
+    bool resourceLoadStatisticsDebugMode() const;
+    void setResourceLoadStatisticsDebugMode(bool);
 
     WebKit::WebsiteDataStore& websiteDataStore() { return m_websiteDataStore.get(); }
     HTTPCookieStore& httpCookieStore();
index 82b2814..7796fc9 100644 (file)
@@ -63,6 +63,11 @@ bool WKWebsiteDataStoreGetResourceLoadStatisticsEnabled(WKWebsiteDataStoreRef da
     return WebKit::toImpl(dataStoreRef)->resourceLoadStatisticsEnabled();
 }
 
+void WKWebsiteDataStoreSetResourceLoadStatisticsDebugMode(WKWebsiteDataStoreRef dataStoreRef, bool enable)
+{
+    WebKit::toImpl(dataStoreRef)->setResourceLoadStatisticsDebugMode(enable);
+}
+
 void WKWebsiteDataStoreSetStatisticsLastSeen(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, double seconds)
 {
     auto* store = WebKit::toImpl(dataStoreRef)->websiteDataStore().resourceLoadStatistics();
index f881a31..d666a44 100644 (file)
@@ -39,6 +39,7 @@ WK_EXPORT WKWebsiteDataStoreRef WKWebsiteDataStoreCreateNonPersistentDataStore()
 
 WK_EXPORT bool WKWebsiteDataStoreGetResourceLoadStatisticsEnabled(WKWebsiteDataStoreRef dataStoreRef);
 WK_EXPORT void WKWebsiteDataStoreSetResourceLoadStatisticsEnabled(WKWebsiteDataStoreRef dataStoreRef, bool enable);
+WK_EXPORT void WKWebsiteDataStoreSetResourceLoadStatisticsDebugMode(WKWebsiteDataStoreRef dataStoreRef, bool enable);
 WK_EXPORT void WKWebsiteDataStoreSetStatisticsLastSeen(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, double seconds);
 WK_EXPORT void WKWebsiteDataStoreSetStatisticsPrevalentResource(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, bool value);
 typedef void (*WKWebsiteDataStoreIsStatisticsPrevalentResourceFunction)(bool isPrevalentResource, void* functionContext);
index 1bc81e4..c1f1510 100644 (file)
@@ -234,6 +234,16 @@ static Vector<WebKit::WebsiteDataRecord> toWebsiteDataRecords(NSArray *dataRecor
     _websiteDataStore->websiteDataStore().setResourceLoadStatisticsEnabled(enabled);
 }
 
+- (BOOL)_resourceLoadStatisticsDebugMode
+{
+    return _websiteDataStore->websiteDataStore().resourceLoadStatisticsDebugMode();
+}
+
+- (void)_setResourceLoadStatisticsDebugMode:(BOOL)enabled
+{
+    _websiteDataStore->websiteDataStore().setResourceLoadStatisticsDebugMode(enabled);
+}
+
 - (NSUInteger)_cacheStoragePerOriginQuota
 {
     return _websiteDataStore->websiteDataStore().cacheStoragePerOriginQuota();
index e772869..2ef1c86 100644 (file)
@@ -45,6 +45,7 @@ typedef NS_OPTIONS(NSUInteger, _WKWebsiteDataStoreFetchOptions) {
 - (void)_fetchDataRecordsOfTypes:(NSSet<NSString *> *)dataTypes withOptions:(_WKWebsiteDataStoreFetchOptions)options completionHandler:(void (^)(NSArray<WKWebsiteDataRecord *> *))completionHandler;
 
 @property (nonatomic, setter=_setResourceLoadStatisticsEnabled:) BOOL _resourceLoadStatisticsEnabled WK_API_AVAILABLE(macosx(10.12), ios(10.0));
+@property (nonatomic, setter=_setResourceLoadStatisticsDebugMode:) BOOL _resourceLoadStatisticsDebugMode WK_API_AVAILABLE(macosx(WK_MAC_TBA), ios(WK_IOS_TBA));
 @property (nonatomic, setter=_setCacheStoragePerOriginQuota:) NSUInteger _cacheStoragePerOriginQuota WK_API_AVAILABLE(macosx(WK_MAC_TBA), ios(WK_IOS_TBA));
 @property (nonatomic, setter=_setCacheStorageDirectory:) NSString* _cacheStorageDirectory WK_API_AVAILABLE(macosx(WK_MAC_TBA), ios(WK_IOS_TBA));
 
index 7f8dea2..3f039af 100644 (file)
@@ -365,6 +365,14 @@ void WebResourceLoadStatisticsStore::submitTelemetry()
     });
 }
 
+void WebResourceLoadStatisticsStore::setResourceLoadStatisticsDebugMode(bool enable)
+{
+    if (enable)
+        setTimeToLiveCookiePartitionFree(30_s);
+    else
+        resetParametersToDefaultValues();
+}
+
 void WebResourceLoadStatisticsStore::logUserInteraction(const URL& url)
 {
     if (url.isBlankURL() || url.isEmpty())
@@ -375,7 +383,8 @@ void WebResourceLoadStatisticsStore::logUserInteraction(const URL& url)
         statistics.hadUserInteraction = true;
         statistics.mostRecentUserInteractionTime = WallTime::now();
 
-        updateCookiePartitioningForDomains({ }, { }, { primaryDomain }, ShouldClearFirst::No);
+        if (statistics.isMarkedForCookiePartitioning || statistics.isMarkedForCookieBlocking)
+            updateCookiePartitioningForDomains({ }, { }, { primaryDomain }, ShouldClearFirst::No);
     });
 }
 
index 418aa65..8ab968a 100644 (file)
@@ -139,6 +139,8 @@ public:
     void clearInMemory();
     void grandfatherExistingWebsiteData(CompletionHandler<void()>&&);
 
+    void setResourceLoadStatisticsDebugMode(bool);
+
     void setStatisticsTestingCallback(Function<void (const String&)>&& callback) { m_statisticsTestingCallback = WTFMove(callback); }
     void logTestingEvent(const String&);
 
index ccec4d1..bb0c45a 100644 (file)
@@ -1413,6 +1413,17 @@ void WebsiteDataStore::setResourceLoadStatisticsEnabled(bool enabled)
         processPool->setResourceLoadStatisticsEnabled(false);
 }
 
+bool WebsiteDataStore::resourceLoadStatisticsDebugMode() const
+{
+    return m_resourceLoadStatisticsDebugMode;
+}
+
+void WebsiteDataStore::setResourceLoadStatisticsDebugMode(bool enabled)
+{
+    m_resourceLoadStatisticsDebugMode = enabled;
+    m_resourceLoadStatistics->setResourceLoadStatisticsDebugMode(enabled);
+}
+
 void WebsiteDataStore::enableResourceLoadStatisticsAndSetTestingCallback(Function<void (const String&)>&& callback)
 {
     if (m_resourceLoadStatistics) {
index 66a0139..b546ca1 100644 (file)
@@ -100,6 +100,8 @@ public:
 
     bool resourceLoadStatisticsEnabled() const;
     void setResourceLoadStatisticsEnabled(bool);
+    bool resourceLoadStatisticsDebugMode() const;
+    void setResourceLoadStatisticsDebugMode(bool);
 
     uint64_t cacheStoragePerOriginQuota() const { return m_resolvedConfiguration.cacheStoragePerOriginQuota; }
     void setCacheStoragePerOriginQuota(uint64_t quota) { m_resolvedConfiguration.cacheStoragePerOriginQuota = quota; }
@@ -197,6 +199,7 @@ private:
 
     const RefPtr<StorageManager> m_storageManager;
     RefPtr<WebResourceLoadStatisticsStore> m_resourceLoadStatistics;
+    bool m_resourceLoadStatisticsDebugMode { false };
 
     Ref<WorkQueue> m_queue;