Resource Load Statistics: Flush the shared ResourceLoadObserver when the webpage...
authorwilander@apple.com <wilander@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 4 Nov 2019 22:13:40 +0000 (22:13 +0000)
committerwilander@apple.com <wilander@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 4 Nov 2019 22:13:40 +0000 (22:13 +0000)
https://bugs.webkit.org/show_bug.cgi?id=203623
<rdar://problem/56756427>

Reviewed by Alex Christensen.

Source/WebCore:

New API test added.

* loader/ResourceLoadObserver.cpp:
(WebCore::ResourceLoadObserver::shared):
    Now returns its own empty observer if no shared one has been set.
* loader/ResourceLoadObserver.h:
(WebCore::ResourceLoadObserver::logSubresourceLoadingForTesting):
    Test function to seed the web process' observer.
* page/DOMWindow.cpp:
(WebCore::DOMWindow::close):
    Now flushes the shared observer.

Source/WebKit:

This patch adds flushing of pending statistics when a window is closed by JavaScript,
when a webpage is removed from the web process, and when the web process prepares to
suspend.

New API test added.

* NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:
(WebKit::WebResourceLoadStatisticsStore::resourceLoadStatisticsUpdated):
    Now calls logTestingEvent() so that the test infrastructure can wait for updates.
* UIProcess/API/Cocoa/WKProcessPool.mm:
(-[WKProcessPool _seedResourceLoadStatisticsForTestingWithFirstParty:thirdParty:shouldScheduleNotification:completionHandler:]):
    Test infrastructure to seed every web process' WebCore::ResourceLoadObserver with test data.
* UIProcess/API/Cocoa/WKProcessPoolPrivate.h:
* UIProcess/API/Cocoa/WKWebsiteDataStore.mm:
(-[WKWebsiteDataStore _clearResourceLoadStatistics:]):
(-[WKWebsiteDataStore _isRegisteredAsSubresourceUnderFirstParty:thirdParty:completionHandler:]):
    Test infrastructure.
* UIProcess/API/Cocoa/WKWebsiteDataStorePrivate.h:
* UIProcess/WebProcessPool.cpp:
(WebKit::WebProcessPool::seedResourceLoadStatisticsForTesting):
    Test infrastructure.
* UIProcess/WebProcessPool.h:
* UIProcess/WebsiteData/WebsiteDataStore.cpp:
(WebKit::WebsiteDataStore::isRegisteredAsSubresourceUnder):
    Made sure the completion handler is called even if there is no network process.
(WebKit::WebsiteDataStore::setResourceLoadStatisticsEnabled):
    Now tells all web processes to turn ITP on or off.
* WebProcess/WebCoreSupport/WebResourceLoadObserver.cpp:
(WebKit::WebResourceLoadObserver::~WebResourceLoadObserver):
(WebKit::WebResourceLoadObserver::logSubresourceLoadingForTesting):
    Test infrastructure to seed the observer with test data.
* WebProcess/WebCoreSupport/WebResourceLoadObserver.h:
* WebProcess/WebProcess.cpp:
(WebKit::WebProcess::setWebsiteDataStoreParameters):
    Now checks whether a shared observer already exists before setting one.
(WebKit::WebProcess::removeWebPage):
(WebKit::WebProcess::prepareToSuspend):
    These two functions now call WebProcess::flushResourceLoadStatistics().
(WebKit::WebProcess::setResourceLoadStatisticsEnabled):
    This function now sets the process' shared WebCore::ResourceLoadObserver if none exists.
(WebKit::WebProcess::flushResourceLoadStatistics):
    This function tells the shared WebCore::ResourceLoadObserver to send any pending
    statistics to the central ITP store.
(WebKit::WebProcess::seedResourceLoadStatisticsForTesting):
    Test infrastructure to seed the shared observer with test data.
* WebProcess/WebProcess.h:
* WebProcess/WebProcess.messages.in:

Tools:

* TestWebKitAPI/Tests/WebKitCocoa/ResourceLoadStatistics.mm:
(TEST):

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@252014 268f45cc-cd09-0410-ab3c-d52691b4dbfc

20 files changed:
Source/WebCore/ChangeLog
Source/WebCore/loader/ResourceLoadObserver.cpp
Source/WebCore/loader/ResourceLoadObserver.h
Source/WebCore/page/DOMWindow.cpp
Source/WebKit/ChangeLog
Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp
Source/WebKit/UIProcess/API/Cocoa/WKProcessPool.mm
Source/WebKit/UIProcess/API/Cocoa/WKProcessPoolPrivate.h
Source/WebKit/UIProcess/API/Cocoa/WKWebsiteDataStore.mm
Source/WebKit/UIProcess/API/Cocoa/WKWebsiteDataStorePrivate.h
Source/WebKit/UIProcess/WebProcessPool.cpp
Source/WebKit/UIProcess/WebProcessPool.h
Source/WebKit/UIProcess/WebsiteData/WebsiteDataStore.cpp
Source/WebKit/WebProcess/WebCoreSupport/WebResourceLoadObserver.cpp
Source/WebKit/WebProcess/WebCoreSupport/WebResourceLoadObserver.h
Source/WebKit/WebProcess/WebProcess.cpp
Source/WebKit/WebProcess/WebProcess.h
Source/WebKit/WebProcess/WebProcess.messages.in
Tools/ChangeLog
Tools/TestWebKitAPI/Tests/WebKitCocoa/ResourceLoadStatistics.mm

index 7bf2acc..ae5eed3 100644 (file)
@@ -1,3 +1,23 @@
+2019-11-04  John Wilander  <wilander@apple.com>
+
+        Resource Load Statistics: Flush the shared ResourceLoadObserver when the webpage is closed by JavaScript
+        https://bugs.webkit.org/show_bug.cgi?id=203623
+        <rdar://problem/56756427>
+
+        Reviewed by Alex Christensen.
+
+        New API test added.
+
+        * loader/ResourceLoadObserver.cpp:
+        (WebCore::ResourceLoadObserver::shared):
+            Now returns its own empty observer if no shared one has been set.
+        * loader/ResourceLoadObserver.h:
+        (WebCore::ResourceLoadObserver::logSubresourceLoadingForTesting):
+            Test function to seed the web process' observer.
+        * page/DOMWindow.cpp:
+        (WebCore::DOMWindow::close):
+            Now flushes the shared observer.
+
 2019-11-04  Alex Christensen  <achristensen@webkit.org>
 
         Collect all documents before iterating in Page::forEachDocument
index e165637..5824150 100644 (file)
@@ -25,6 +25,7 @@
 
 #include "config.h"
 #include "ResourceLoadObserver.h"
+#include <wtf/NeverDestroyed.h>
 
 namespace WebCore {
 
@@ -42,8 +43,9 @@ void ResourceLoadObserver::setShared(ResourceLoadObserver& observer)
 
 ResourceLoadObserver& ResourceLoadObserver::shared()
 {
+    static NeverDestroyed<ResourceLoadObserver> emptyObserver;
     if (!sharedObserver())
-        sharedObserver() = new ResourceLoadObserver;
+        return emptyObserver;
     return *sharedObserver();
 }
 
index e74d5f6..fd65155 100644 (file)
@@ -52,6 +52,7 @@ public:
     virtual void logCanvasWriteOrMeasure(const Document&, const String& /* textWritten */) { }
     virtual void logNavigatorAPIAccessed(const Document&, const ResourceLoadStatistics::NavigatorAPI) { }
     virtual void logScreenAPIAccessed(const Document&, const ResourceLoadStatistics::ScreenAPI) { }
+    virtual void logSubresourceLoadingForTesting(const RegistrableDomain& /* firstPartyDomain */, const RegistrableDomain& /* thirdPartyDomain */, bool /* shouldScheduleNotification */) { }
 
     virtual String statisticsForURL(const URL&) { return { }; }
     virtual void updateCentralStatisticsStore() { }
index c70f337..0ace9ec 100644 (file)
@@ -1058,6 +1058,8 @@ void DOMWindow::close()
     if (!frame->loader().shouldClose())
         return;
 
+    ResourceLoadObserver::shared().updateCentralStatisticsStore();
+
     page->setIsClosing();
     page->chrome().closeWindowSoon();
 }
index 8eb8460..6490a2e 100644 (file)
@@ -1,3 +1,59 @@
+2019-11-04  John Wilander  <wilander@apple.com>
+
+        Resource Load Statistics: Flush the shared ResourceLoadObserver when the webpage is closed by JavaScript
+        https://bugs.webkit.org/show_bug.cgi?id=203623
+        <rdar://problem/56756427>
+
+        Reviewed by Alex Christensen.
+
+        This patch adds flushing of pending statistics when a window is closed by JavaScript,
+        when a webpage is removed from the web process, and when the web process prepares to
+        suspend.
+
+        New API test added.
+
+        * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:
+        (WebKit::WebResourceLoadStatisticsStore::resourceLoadStatisticsUpdated):
+            Now calls logTestingEvent() so that the test infrastructure can wait for updates.
+        * UIProcess/API/Cocoa/WKProcessPool.mm:
+        (-[WKProcessPool _seedResourceLoadStatisticsForTestingWithFirstParty:thirdParty:shouldScheduleNotification:completionHandler:]):
+            Test infrastructure to seed every web process' WebCore::ResourceLoadObserver with test data.
+        * UIProcess/API/Cocoa/WKProcessPoolPrivate.h:
+        * UIProcess/API/Cocoa/WKWebsiteDataStore.mm:
+        (-[WKWebsiteDataStore _clearResourceLoadStatistics:]):
+        (-[WKWebsiteDataStore _isRegisteredAsSubresourceUnderFirstParty:thirdParty:completionHandler:]):
+            Test infrastructure.
+        * UIProcess/API/Cocoa/WKWebsiteDataStorePrivate.h:
+        * UIProcess/WebProcessPool.cpp:
+        (WebKit::WebProcessPool::seedResourceLoadStatisticsForTesting):
+            Test infrastructure.
+        * UIProcess/WebProcessPool.h:
+        * UIProcess/WebsiteData/WebsiteDataStore.cpp:
+        (WebKit::WebsiteDataStore::isRegisteredAsSubresourceUnder):
+            Made sure the completion handler is called even if there is no network process.
+        (WebKit::WebsiteDataStore::setResourceLoadStatisticsEnabled):
+            Now tells all web processes to turn ITP on or off.
+        * WebProcess/WebCoreSupport/WebResourceLoadObserver.cpp:
+        (WebKit::WebResourceLoadObserver::~WebResourceLoadObserver):
+        (WebKit::WebResourceLoadObserver::logSubresourceLoadingForTesting):
+            Test infrastructure to seed the observer with test data.
+        * WebProcess/WebCoreSupport/WebResourceLoadObserver.h:
+        * WebProcess/WebProcess.cpp:
+        (WebKit::WebProcess::setWebsiteDataStoreParameters):
+            Now checks whether a shared observer already exists before setting one.
+        (WebKit::WebProcess::removeWebPage):
+        (WebKit::WebProcess::prepareToSuspend):
+            These two functions now call WebProcess::flushResourceLoadStatistics().
+        (WebKit::WebProcess::setResourceLoadStatisticsEnabled):
+            This function now sets the process' shared WebCore::ResourceLoadObserver if none exists.
+        (WebKit::WebProcess::flushResourceLoadStatistics):
+            This function tells the shared WebCore::ResourceLoadObserver to send any pending
+            statistics to the central ITP store.
+        (WebKit::WebProcess::seedResourceLoadStatisticsForTesting):
+            Test infrastructure to seed the shared observer with test data.
+        * WebProcess/WebProcess.h:
+        * WebProcess/WebProcess.messages.in:
+
 2019-11-04  Chris Dumez  <cdumez@apple.com>
 
         [iOS][WK2] Simplify process assertion handling for the network process and service worker processes
index dcbdd88..dd7dc5b 100644 (file)
@@ -273,7 +273,7 @@ void WebResourceLoadStatisticsStore::resourceLoadStatisticsUpdated(Vector<Resour
     // It is safe to move the origins to the background queue without isolated copy here because this is an r-value
     // coming from IPC. ResourceLoadStatistics only contains strings which are safe to move to other threads as long
     // as nobody on this thread holds a reference to those strings.
-    postTask([this, statistics = WTFMove(statistics)]() mutable {
+    postTask([this, protectedThis = makeRef(*this), statistics = WTFMove(statistics)]() mutable {
         if (!m_statisticsStore)
             return;
 
@@ -283,7 +283,11 @@ void WebResourceLoadStatisticsStore::resourceLoadStatisticsUpdated(Vector<Resour
         m_statisticsStore->cancelPendingStatisticsProcessingRequest();
 
         // Fire before processing statistics to propagate user interaction as fast as possible to the network process.
-        m_statisticsStore->updateCookieBlocking([]() { });
+        m_statisticsStore->updateCookieBlocking([this, protectedThis = protectedThis.copyRef()]() {
+            postTaskReply([this, protectedThis = protectedThis.copyRef()]() {
+                logTestingEvent("Statistics Updated"_s);
+            });
+        });
         m_statisticsStore->processStatisticsAndDataRecords();
     });
 }
index 0b810b5..34832b1 100644 (file)
@@ -49,6 +49,7 @@
 #import "_WKProcessPoolConfigurationInternal.h"
 #import <WebCore/CertificateInfo.h>
 #import <WebCore/PluginData.h>
+#import <WebCore/RegistrableDomain.h>
 #import <pal/spi/cf/CFNetworkSPI.h>
 #import <pal/spi/cocoa/NSKeyedArchiverSPI.h>
 #import <wtf/BlockPtr.h>
@@ -653,4 +654,18 @@ static NSDictionary *policiesHashMapToDictionary(const HashMap<String, HashMap<S
     _processPool->setAllowsAnySSLCertificateForWebSocket(true);
 }
 
+- (void)_seedResourceLoadStatisticsForTestingWithFirstParty:(NSURL *)firstPartyURL thirdParty:(NSURL *)thirdPartyURL shouldScheduleNotification:(BOOL)shouldScheduleNotification completionHandler:(void(^)(void))completionHandler
+{
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+    _processPool->seedResourceLoadStatisticsForTesting(WebCore::RegistrableDomain { firstPartyURL }, WebCore::RegistrableDomain { thirdPartyURL }, shouldScheduleNotification, [completionHandler = makeBlockPtr(completionHandler)] () {
+        completionHandler();
+    });
+#else
+    UNUSED_PARAM(firstPartyURL);
+    UNUSED_PARAM(thirdPartyURL);
+    UNUSED_PARAM(shouldScheduleNotification);
+    UNUSED_PARAM(completionHandler);
+#endif
+}
+
 @end
index 3a81d59..13035ca 100644 (file)
 @property (nonatomic, getter=_isStorageAccessAPIEnabled, setter=_setStorageAccessAPIEnabled:) BOOL _storageAccessAPIEnabled WK_API_AVAILABLE(macos(10.13.4), ios(11.3));
 - (void)_synthesizeAppIsBackground:(BOOL)background WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
 
+// Test only.
+- (void)_seedResourceLoadStatisticsForTestingWithFirstParty:(NSURL *)firstPartyURL thirdParty:(NSURL *)thirdPartyURL shouldScheduleNotification:(BOOL)shouldScheduleNotification completionHandler:(void(^)(void))completionHandler  WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
+
 @end
index 6142f0b..ad0b731 100644 (file)
@@ -29,6 +29,7 @@
 #import "APIString.h"
 #import "AuthenticationChallengeDispositionCocoa.h"
 #import "CompletionHandlerCallChecker.h"
+#import "ShouldGrandfatherStatistics.h"
 #import "WKHTTPCookieStoreInternal.h"
 #import "WKNSArray.h"
 #import "WKNSURLAuthenticationChallenge.h"
@@ -495,6 +496,28 @@ static Vector<WebKit::WebsiteDataRecord> toWebsiteDataRecords(NSArray *dataRecor
 #endif
 }
 
+- (void)_clearResourceLoadStatistics:(void (^)(void))completionHandler
+{
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+    _websiteDataStore->scheduleClearInMemoryAndPersistent(WebKit::ShouldGrandfatherStatistics::No, [completionHandler = makeBlockPtr(completionHandler)]() {
+        completionHandler();
+    });
+#else
+    completionHandler();
+#endif
+}
+
+- (void)_isRegisteredAsSubresourceUnderFirstParty:(NSURL *)firstPartyURL thirdParty:(NSURL *)thirdPartyURL completionHandler:(void (^)(BOOL))completionHandler
+{
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+    _websiteDataStore->isRegisteredAsSubresourceUnder(thirdPartyURL, firstPartyURL, [completionHandler = makeBlockPtr(completionHandler)](bool enabled) {
+        completionHandler(enabled);
+    });
+#else
+    completionHandler(NO);
+#endif
+}
+
 - (void)_processStatisticsAndDataRecords:(void (^)(void))completionHandler
 {
 #if ENABLE(RESOURCE_LOAD_STATISTICS)
index 04e3c78..31b80ab 100644 (file)
@@ -70,7 +70,9 @@ typedef NS_OPTIONS(NSUInteger, _WKWebsiteDataStoreFetchOptions) {
 - (void)_scheduleCookieBlockingUpdate:(void (^)(void))completionHandler WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
 - (void)_setPrevalentDomain:(NSURL *)domain completionHandler:(void (^)(void))completionHandler WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
 - (void)_getIsPrevalentDomain:(NSURL *)domain completionHandler:(void (^)(BOOL))completionHandler WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
-- (void)_clearPrevalentDomain:(NSURL *)domain completionHandler:(void (^)(void))completionHandler;
+- (void)_clearPrevalentDomain:(NSURL *)domain completionHandler:(void (^)(void))completionHandler WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
+- (void)_clearResourceLoadStatistics:(void (^)(void))completionHandler WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
+- (void)_isRegisteredAsSubresourceUnderFirstParty:(NSURL *)firstPartyURL thirdParty:(NSURL *)thirdPartyURL completionHandler:(void (^)(BOOL))completionHandler WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
 - (void)_processStatisticsAndDataRecords:(void (^)(void))completionHandler WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
 
 @property (nullable, nonatomic, weak) id <_WKWebsiteDataStoreDelegate> _delegate WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
index 81f77c8..8ecc420 100644 (file)
 #include <WebCore/PlatformScreen.h>
 #include <WebCore/ProcessIdentifier.h>
 #include <WebCore/ProcessWarming.h>
+#include <WebCore/RegistrableDomain.h>
 #include <WebCore/RegistrationDatabase.h>
 #include <WebCore/ResourceRequest.h>
 #include <WebCore/RuntimeApplicationChecks.h>
 #include <pal/SessionID.h>
+#include <wtf/CallbackAggregator.h>
 #include <wtf/Language.h>
 #include <wtf/MainThread.h>
 #include <wtf/NeverDestroyed.h>
@@ -2276,6 +2278,14 @@ void WebProcessPool::didCommitCrossSiteLoadWithDataTransfer(PAL::SessionID sessi
 
     m_networkProcess->didCommitCrossSiteLoadWithDataTransfer(sessionID, fromDomain, toDomain, navigationDataTransfer, webPageProxyID, webPageID);
 }
+
+void WebProcessPool::seedResourceLoadStatisticsForTesting(const RegistrableDomain& firstPartyDomain, const RegistrableDomain& thirdPartyDomain, bool shouldScheduleNotification, CompletionHandler<void()>&& completionHandler)
+{
+    auto callbackAggregator = CallbackAggregator::create(WTFMove(completionHandler));
+
+    for (auto& process : processes())
+        process->sendWithAsyncReply(Messages::WebProcess::SeedResourceLoadStatisticsForTesting(firstPartyDomain, thirdPartyDomain, shouldScheduleNotification), [callbackAggregator = callbackAggregator.copyRef()] { });
+}
 #endif
 
 void WebProcessPool::setWebProcessHasUploads(ProcessIdentifier processID)
index 8f800ff..5873460 100644 (file)
@@ -45,7 +45,6 @@
 #include "WebsiteDataStore.h"
 #include <WebCore/CrossSiteNavigationDataTransfer.h>
 #include <WebCore/ProcessIdentifier.h>
-#include <WebCore/RegistrableDomain.h>
 #include <WebCore/SecurityOriginHash.h>
 #include <WebCore/SharedStringHash.h>
 #include <pal/SessionID.h>
@@ -90,6 +89,7 @@ class PageConfiguration;
 }
 
 namespace WebCore {
+class RegistrableDomain;
 struct MockMediaDevice;
 }
 
@@ -495,6 +495,7 @@ public:
 
 #if ENABLE(RESOURCE_LOAD_STATISTICS)
     void didCommitCrossSiteLoadWithDataTransfer(PAL::SessionID, const WebCore::RegistrableDomain& fromDomain, const WebCore::RegistrableDomain& toDomain, OptionSet<WebCore::CrossSiteNavigationDataTransfer::Flag>, WebPageProxyIdentifier, WebCore::PageIdentifier);
+    void seedResourceLoadStatisticsForTesting(const WebCore::RegistrableDomain& firstPartyDomain, const WebCore::RegistrableDomain& thirdPartyDomain, bool shouldScheduleNotification, CompletionHandler<void()>&&);
 #endif
 
 #if PLATFORM(GTK) || PLATFORM(WPE)
index 15dcab2..2df9d7f 100644 (file)
@@ -1369,6 +1369,10 @@ void WebsiteDataStore::isRegisteredAsSubresourceUnder(const URL& subresourceURL,
             process->isRegisteredAsSubresourceUnder(m_sessionID, WebCore::RegistrableDomain { subresourceURL }, WebCore::RegistrableDomain { topFrameURL }, WTFMove(completionHandler));
             ASSERT(processPools().size() == 1);
             break;
+        } else {
+            ASSERT_NOT_REACHED();
+            completionHandler(false);
+            break;
         }
     }
 }
@@ -1961,13 +1965,17 @@ void WebsiteDataStore::setResourceLoadStatisticsEnabled(bool enabled)
         
         resolveDirectoriesIfNecessary();
         
-        for (auto& processPool : processPools(std::numeric_limits<size_t>::max(), false))
+        for (auto& processPool : processPools(std::numeric_limits<size_t>::max(), false)) {
             processPool->sendToNetworkingProcess(Messages::NetworkProcess::SetResourceLoadStatisticsEnabled(m_sessionID, true));
+            processPool->sendToAllProcesses(Messages::WebProcess::SetResourceLoadStatisticsEnabled(true));
+        }
         return;
     }
 
-    for (auto& processPool : processPools(std::numeric_limits<size_t>::max(), false))
+    for (auto& processPool : processPools(std::numeric_limits<size_t>::max(), false)) {
         processPool->sendToNetworkingProcess(Messages::NetworkProcess::SetResourceLoadStatisticsEnabled(m_sessionID, false));
+        processPool->sendToAllProcesses(Messages::WebProcess::SetResourceLoadStatisticsEnabled(false));
+    }
 
     m_resourceLoadStatisticsEnabled = false;
 #else
index 9ed2a05..5e4f067 100644 (file)
@@ -65,6 +65,12 @@ WebResourceLoadObserver::WebResourceLoadObserver()
 {
 }
 
+WebResourceLoadObserver::~WebResourceLoadObserver()
+{
+    if (hasStatistics())
+        updateCentralStatisticsStore();
+}
+
 void WebResourceLoadObserver::requestStorageAccessUnderOpener(const RegistrableDomain& domainInNeedOfStorageAccess, PageIdentifier openerPageID, Document& openerDocument)
 {
     auto openerUrl = openerDocument.url();
@@ -356,6 +362,19 @@ void WebResourceLoadObserver::logUserInteractionWithReducedTimeResolution(const
 #endif
 }
 
+void WebResourceLoadObserver::logSubresourceLoadingForTesting(const RegistrableDomain& firstPartyDomain, const RegistrableDomain& thirdPartyDomain, bool shouldScheduleNotification)
+{
+    auto& targetStatistics = ensureResourceStatisticsForRegistrableDomain(thirdPartyDomain);
+    auto lastSeen = ResourceLoadStatistics::reduceTimeResolution(WallTime::now());
+    targetStatistics.lastSeen = lastSeen;
+    targetStatistics.subresourceUnderTopFrameDomains.add(firstPartyDomain);
+
+    if (shouldScheduleNotification)
+        scheduleNotificationIfNeeded();
+    else
+        m_notificationTimer.stop();
+}
+
 } // namespace WebKit
 
 #endif // ENABLE(RESOURCE_LOAD_STATISTICS)
index d6b8d87..92de7bc 100644 (file)
@@ -37,7 +37,8 @@ namespace WebKit {
 class WebResourceLoadObserver final : public WebCore::ResourceLoadObserver {
 public:
     WebResourceLoadObserver();
-    
+    ~WebResourceLoadObserver();
+
     void logSubresourceLoading(const WebCore::Frame*, const WebCore::ResourceRequest& newRequest, const WebCore::ResourceResponse& redirectResponse) final;
     void logWebSocketLoading(const URL& targetURL, const URL& mainFrameURL) final;
     void logUserInteractionWithReducedTimeResolution(const WebCore::Document&) final;
@@ -46,6 +47,7 @@ public:
     void logCanvasWriteOrMeasure(const WebCore::Document&, const String& textWritten) final;
     void logNavigatorAPIAccessed(const WebCore::Document&, const WebCore::ResourceLoadStatistics::NavigatorAPI) final;
     void logScreenAPIAccessed(const WebCore::Document&, const WebCore::ResourceLoadStatistics::ScreenAPI) final;
+    void logSubresourceLoadingForTesting(const WebCore::RegistrableDomain& firstPartyDomain, const WebCore::RegistrableDomain& thirdPartyDomain, bool shouldScheduleNotification);
 
 #if !RELEASE_LOG_DISABLED
     static void setShouldLogUserInteraction(bool);
index 6c16fa0..42681ac 100644 (file)
@@ -458,7 +458,7 @@ void WebProcess::setWebsiteDataStoreParameters(WebProcessDataStoreParameters&& p
     setResourceLoadStatisticsEnabled(parameters.resourceLoadStatisticsEnabled);
 
 #if ENABLE(RESOURCE_LOAD_STATISTICS)
-    if (parameters.resourceLoadStatisticsEnabled && !parameters.sessionID.isEphemeral())
+    if (parameters.resourceLoadStatisticsEnabled && !parameters.sessionID.isEphemeral() && !ResourceLoadObserver::sharedIfExists())
         ResourceLoadObserver::setShared(*new WebResourceLoadObserver);
 #endif
 
@@ -671,6 +671,8 @@ void WebProcess::removeWebPage(PageIdentifier pageID)
 {
     ASSERT(m_pageMap.contains(pageID));
 
+    flushResourceLoadStatistics();
+
     pageWillLeaveWindow(pageID);
     m_pageMap.remove(pageID);
 
@@ -1383,6 +1385,8 @@ void WebProcess::prepareToSuspend(bool isSuspensionImminent, CompletionHandler<v
     SetForScope<bool> suspensionScope(m_isSuspending, true);
     m_processIsSuspended = true;
 
+    flushResourceLoadStatistics();
+
 #if PLATFORM(COCOA)
     if (m_processType == ProcessType::PrewarmedWebContent) {
         RELEASE_LOG(ProcessSuspension, "%p - WebProcess::prepareToSuspend() Process is ready to suspend", this);
@@ -1552,7 +1556,13 @@ StorageAreaMap* WebProcess::storageAreaMap(StorageAreaIdentifier identifier) con
 
 void WebProcess::setResourceLoadStatisticsEnabled(bool enabled)
 {
+    if (WebCore::DeprecatedGlobalSettings::resourceLoadStatisticsEnabled() == enabled || m_sessionID->isEphemeral())
+        return;
     WebCore::DeprecatedGlobalSettings::setResourceLoadStatisticsEnabled(enabled);
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+    if (enabled && !ResourceLoadObserver::sharedIfExists())
+        WebCore::ResourceLoadObserver::setShared(*new WebResourceLoadObserver);
+#endif
 }
 
 void WebProcess::clearResourceLoadStatistics()
@@ -1563,6 +1573,23 @@ void WebProcess::clearResourceLoadStatistics()
 #endif
 }
 
+void WebProcess::flushResourceLoadStatistics()
+{
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+    if (auto* observer = ResourceLoadObserver::sharedIfExists())
+        observer->updateCentralStatisticsStore();
+#endif
+}
+
+void WebProcess::seedResourceLoadStatisticsForTesting(const RegistrableDomain& firstPartyDomain, const RegistrableDomain& thirdPartyDomain, bool shouldScheduleNotification, CompletionHandler<void()>&& completionHandler)
+{
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+    if (auto* observer = ResourceLoadObserver::sharedIfExists())
+        observer->logSubresourceLoadingForTesting(firstPartyDomain, thirdPartyDomain, shouldScheduleNotification);
+#endif
+    completionHandler();
+}
+
 RefPtr<API::Object> WebProcess::transformHandlesToObjects(API::Object* object)
 {
     struct Transformer final : UserData::Transformer {
index c24bcbd..cd612ff 100644 (file)
@@ -87,6 +87,7 @@ class ApplicationCacheStorage;
 class CPUMonitor;
 class CertificateInfo;
 class PageGroup;
+class RegistrableDomain;
 class ResourceRequest;
 class UserGestureToken;
 struct BackForwardItemIdentifier;
@@ -346,6 +347,8 @@ private:
     void setShouldUseFontSmoothing(bool);
     void setResourceLoadStatisticsEnabled(bool);
     void clearResourceLoadStatistics();
+    void flushResourceLoadStatistics();
+    void seedResourceLoadStatisticsForTesting(const WebCore::RegistrableDomain& firstPartyDomain, const WebCore::RegistrableDomain& thirdPartyDomain, bool shouldScheduleNotification, CompletionHandler<void()>&&);
     void userPreferredLanguagesChanged(const Vector<String>&) const;
     void fullKeyboardAccessModeChanged(bool fullKeyboardAccessEnabled);
 
index 10941f0..58d4f67 100644 (file)
@@ -160,4 +160,8 @@ messages -> WebProcess LegacyReceiver {
 #if PLATFORM(GTK) || PLATFORM(WPE)
     SendMessageToWebExtension(struct WebKit::UserMessage userMessage)
 #endif
+
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+    SeedResourceLoadStatisticsForTesting(WebCore::RegistrableDomain firstPartyDomain, WebCore::RegistrableDomain thirdPartyDomain, bool shouldScheduleNotification) -> () Async
+#endif
 }
index f678c91..7270a16 100644 (file)
@@ -1,3 +1,14 @@
+2019-11-04  John Wilander  <wilander@apple.com>
+
+        Resource Load Statistics: Flush the shared ResourceLoadObserver when the webpage is closed by JavaScript
+        https://bugs.webkit.org/show_bug.cgi?id=203623
+        <rdar://problem/56756427>
+
+        Reviewed by Alex Christensen.
+
+        * TestWebKitAPI/Tests/WebKitCocoa/ResourceLoadStatistics.mm:
+        (TEST):
+
 2019-11-04  Wenson Hsieh  <wenson_hsieh@apple.com>
 
         Consolidate forEachViewInHierarchy and findAllViewsInHierarchyOfType into common helper file
index af64485..a721d24 100644 (file)
@@ -484,3 +484,66 @@ TEST(ResourceLoadStatistics, NoMessagesWhenNotTesting)
     [webView synchronouslyLoadTestPageNamed:@"simple"];
     EXPECT_FALSE([WKWebsiteDataStore _defaultDataStoreExists]);
 }
+
+TEST(ResourceLoadStatistics, FlushObserverWhenWebPageIsClosedByJavaScript)
+{
+    auto *sharedProcessPool = [WKProcessPool _sharedProcessPool];
+    auto *dataStore = [WKWebsiteDataStore defaultDataStore];
+
+    auto configuration = adoptNS([[WKWebViewConfiguration alloc] init]);
+    [configuration setProcessPool: sharedProcessPool];
+    configuration.get().websiteDataStore = dataStore;
+
+    auto webView = adoptNS([[WKWebView alloc] initWithFrame:NSMakeRect(0, 0, 800, 600) configuration:configuration.get()]);
+
+    [webView loadHTMLString:@"WebKit Test" baseURL:[NSURL URLWithString:@"http://webkit.org"]];
+    [webView _test_waitForDidFinishNavigation];
+
+    [dataStore _setResourceLoadStatisticsEnabled:YES];
+
+    static bool doneFlag = false;
+    [dataStore _clearResourceLoadStatistics:^(void) {
+        doneFlag = true;
+    }];
+
+    static bool statisticsUpdated = false;
+    [dataStore _setResourceLoadStatisticsTestingCallback:^(WKWebsiteDataStore *, NSString *message) {
+        if (![message isEqualToString:@"Statistics Updated"])
+            return;
+        statisticsUpdated = true;
+    }];
+
+    TestWebKitAPI::Util::run(&doneFlag);
+
+    // Seed test data in the web process' observer.
+    doneFlag = false;
+    [sharedProcessPool _seedResourceLoadStatisticsForTestingWithFirstParty:[NSURL URLWithString:@"http://webkit.org"] thirdParty:[NSURL URLWithString:@"http://evil.com"] shouldScheduleNotification:NO completionHandler: ^() {
+        doneFlag = true;
+    }];
+    
+    TestWebKitAPI::Util::run(&doneFlag);
+
+    // Check that the third-party is not yet registered.
+    doneFlag = false;
+    [dataStore _isRegisteredAsSubresourceUnderFirstParty:[NSURL URLWithString:@"http://webkit.org"] thirdParty:[NSURL URLWithString:@"http://evil.com"] completionHandler: ^(BOOL isRegistered) {
+        EXPECT_FALSE(isRegistered);
+        doneFlag = true;
+    }];
+
+    TestWebKitAPI::Util::run(&doneFlag);
+
+    statisticsUpdated = false;
+    [webView loadHTMLString:@"<body><script>close();</script></body>" baseURL:[NSURL URLWithString:@"http://webkit.org"]];
+
+    // Wait for the statistics to be updated in the network process.
+    TestWebKitAPI::Util::run(&statisticsUpdated);
+
+    // Check that the third-party is now registered.
+    doneFlag = false;
+    [dataStore _isRegisteredAsSubresourceUnderFirstParty:[NSURL URLWithString:@"http://webkit.org"] thirdParty:[NSURL URLWithString:@"http://evil.com"] completionHandler: ^(BOOL isRegistered) {
+        EXPECT_TRUE(isRegistered);
+        doneFlag = true;
+    }];
+
+    TestWebKitAPI::Util::run(&doneFlag);
+}