[GTK][WPE] Enable resource load statistics
authorcarlosgc@webkit.org <carlosgc@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 20 Apr 2020 08:33:58 +0000 (08:33 +0000)
committercarlosgc@webkit.org <carlosgc@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 20 Apr 2020 08:33:58 +0000 (08:33 +0000)
https://bugs.webkit.org/show_bug.cgi?id=210184

Reviewed by Žan Doberšek.

.:

Enable resource load statistics.

* Source/cmake/OptionsGTK.cmake:
* Source/cmake/OptionsWPE.cmake:

Source/WebCore:

* platform/network/soup/NetworkStorageSessionSoup.cpp:
(WebCore::NetworkStorageSession::setCookiesFromDOM const): Return early if cookies are blocked and update the
persistent cookies expiration if needed.
(WebCore::NetworkStorageSession::deleteCookiesForHostnames): Implement this when receiving
IncludeHttpOnlyCookies parameter.
(WebCore::NetworkStorageSession::hasCookies const): Implement this.
(WebCore::NetworkStorageSession::getRawCookies const): Honor shouldAskITP parameter.
(WebCore::cookiesForSession): Ditto.
(WebCore::NetworkStorageSession::cookiesForDOM const): Ditto.
(WebCore::NetworkStorageSession::cookieRequestHeaderFieldValue const): Ditto.

Source/WebKit:

* NetworkProcess/NetworkDataTask.cpp:
(WebKit::NetworkDataTask::create): Pass page and frame identifiers to the constructor.
(WebKit::NetworkDataTask::isThirdPartyRequest const): Moved from NetworkDataTaskCocoa.mm.
(WebKit::NetworkDataTask::restrictRequestReferrerToOriginIfNeeded): Ditto.
* NetworkProcess/NetworkDataTask.h:
* NetworkProcess/cocoa/NetworkDataTaskCocoa.h:
* NetworkProcess/cocoa/NetworkDataTaskCocoa.mm:
* NetworkProcess/soup/NetworkDataTaskSoup.cpp:
(WebKit::NetworkDataTaskSoup::NetworkDataTaskSoup): Initialize m_frameID and m_pageID.
(WebKit::NetworkDataTaskSoup::createRequest): Add WasBlockingCookies parameter. Call
restrictRequestReferrerToOriginIfNeeded() and disable cookies in the soup message if cookies should be blocked.
(WebKit::NetworkDataTaskSoup::clearRequest): Reset m_isBlockingCookies.
(WebKit::NetworkDataTaskSoup::continueHTTPRedirection): Pass WasBlockingCookies to createRequest().
* NetworkProcess/soup/NetworkDataTaskSoup.h:
* UIProcess/API/C/WKPage.cpp:
(WKPageLoadedThirdPartyDomains): Added for tests.
(WKPageClearLoadedThirdPartyDomains): Ditto.
* UIProcess/API/C/WKPagePrivate.h:
* UIProcess/glib/WebsiteDataStoreGLib.cpp:
(WebKit::WebsiteDataStore::defaultResourceLoadStatisticsDirectory): Use lowercase for consistency with other
default directories.

Tools:

Implement TestController::loadedThirdPartyDomains() and TestController::clearLoadedThirdPartyDomains() for
non-cocoa ports.

* WebKitTestRunner/TestController.cpp:
(WTR::LoadedThirdPartyDomainsCallbackContext::LoadedThirdPartyDomainsCallbackContext):
(WTR::loadedThirdPartyDomainsCallback):
(WTR::TestController::loadedThirdPartyDomains):
(WTR::TestController::clearLoadedThirdPartyDomains):

LayoutTests:

Unskip tests that are now passing and add expectations for the two reminaing failures.

* http/tests/resourceLoadStatistics/loaded-registrable-domains-get-reported.html: Use setTimeout in onload
handler to avoid a deadlock due to sync injected bundle messages.
* platform/gtk/TestExpectations:
* platform/wpe/TestExpectations:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@260356 268f45cc-cd09-0410-ab3c-d52691b4dbfc

21 files changed:
ChangeLog
LayoutTests/ChangeLog
LayoutTests/http/tests/resourceLoadStatistics/loaded-registrable-domains-get-reported.html
LayoutTests/platform/gtk/TestExpectations
LayoutTests/platform/wpe/TestExpectations
Source/WebCore/ChangeLog
Source/WebCore/platform/network/soup/NetworkStorageSessionSoup.cpp
Source/WebKit/ChangeLog
Source/WebKit/NetworkProcess/NetworkDataTask.cpp
Source/WebKit/NetworkProcess/NetworkDataTask.h
Source/WebKit/NetworkProcess/cocoa/NetworkDataTaskCocoa.h
Source/WebKit/NetworkProcess/cocoa/NetworkDataTaskCocoa.mm
Source/WebKit/NetworkProcess/soup/NetworkDataTaskSoup.cpp
Source/WebKit/NetworkProcess/soup/NetworkDataTaskSoup.h
Source/WebKit/UIProcess/API/C/WKPage.cpp
Source/WebKit/UIProcess/API/C/WKPagePrivate.h
Source/WebKit/UIProcess/glib/WebsiteDataStoreGLib.cpp
Source/cmake/OptionsGTK.cmake
Source/cmake/OptionsWPE.cmake
Tools/ChangeLog
Tools/WebKitTestRunner/TestController.cpp

index dcd78b7..1b96b83 100644 (file)
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,15 @@
+2020-04-20  Carlos Garcia Campos  <cgarcia@igalia.com>
+
+        [GTK][WPE] Enable resource load statistics
+        https://bugs.webkit.org/show_bug.cgi?id=210184
+
+        Reviewed by Žan Doberšek.
+
+        Enable resource load statistics.
+
+        * Source/cmake/OptionsGTK.cmake:
+        * Source/cmake/OptionsWPE.cmake:
+
 2020-04-17  Don Olmstead  <don.olmstead@sony.com>
 
         [CMake] Add WebKit::WebCore target
index 95ccc9a..ecabf4a 100644 (file)
@@ -1,3 +1,17 @@
+2020-04-20  Carlos Garcia Campos  <cgarcia@igalia.com>
+
+        [GTK][WPE] Enable resource load statistics
+        https://bugs.webkit.org/show_bug.cgi?id=210184
+
+        Reviewed by Žan Doberšek.
+
+        Unskip tests that are now passing and add expectations for the two reminaing failures.
+
+        * http/tests/resourceLoadStatistics/loaded-registrable-domains-get-reported.html: Use setTimeout in onload
+        handler to avoid a deadlock due to sync injected bundle messages.
+        * platform/gtk/TestExpectations:
+        * platform/wpe/TestExpectations:
+
 2020-04-19  Yusuke Suzuki  <ysuzuki@apple.com>
 
         Unreviewed, skip imported/w3c/web-platform-tests/IndexedDB/structured-clone
index a8b087e..3d00f2d 100644 (file)
@@ -24,7 +24,7 @@
     </script>
 </head>
 <body>
-<iframe onload='askForLoadedThirdPartyDomains()' src="http://localhost:8000/resourceLoadStatistics/resources/basic-iframe.html">
+<iframe onload="setTimeout('askForLoadedThirdPartyDomains()', 0)" src="http://localhost:8000/resourceLoadStatistics/resources/basic-iframe.html">
 </iframe>
 </body>
 </html>
index 179256a..3bbd105 100644 (file)
@@ -1180,10 +1180,6 @@ webkit.org/b/191005 inspector/canvas/shaderProgram-add-remove-webgpu.html [ Skip
 webkit.org/b/191005 inspector/canvas/updateShader-webgpu.html [ Skip ]
 webkit.org/b/191005 inspector/canvas/updateShader-webgpu-sharedVertexFragment.html [ Skip ]
 
-# No support for resource load statistics yet
-http/tests/resourceLoadStatistics/ [ Skip ]
-http/tests/websocket/web-socket-loads-captured-in-per-page-domains.html [ Skip ]
-
 # No support for screen capture
 fast/mediastream/screencapture-user-gesture.html [ Skip ]
 
@@ -1445,6 +1441,9 @@ webkit.org/b/210601 imported/w3c/web-platform-tests/web-animations/interfaces/An
 webkit.org/b/210601 imported/w3c/web-platform-tests/web-animations/interfaces/KeyframeEffect/style-change-events.html [ Failure ]
 webkit.org/b/210601 imported/w3c/web-platform-tests/web-animations/interfaces/KeyframeEffect/target.html [ Failure ]
 
+webkit.org/b/210487 http/tests/resourceLoadStatistics/switch-session-on-navigation-to-prevalent-with-interaction-database.php [ Failure ]
+webkit.org/b/210487 http/tests/resourceLoadStatistics/switch-session-on-navigation-to-prevalent-with-interaction.php [ Failure ]
+
 #////////////////////////////////////////////////////////////////////////////////////////
 # End of Expected failures.
 #
index 216bd79..b4689d1 100644 (file)
@@ -738,6 +738,9 @@ webkit.org/b/210263 fast/layoutformattingcontext/out-of-flow-positioned-line-bre
 # Currently enabled on Mac only
 media/media-usage-state.html [ Skip ]
 
+webkit.org/b/210487 http/tests/resourceLoadStatistics/switch-session-on-navigation-to-prevalent-with-interaction-database.php [ Failure ]
+webkit.org/b/210487 http/tests/resourceLoadStatistics/switch-session-on-navigation-to-prevalent-with-interaction.php [ Failure ]
+
 #////////////////////////////////////////////////////////////////////////////////////////
 # 3. UNRESOLVED TESTS
 #////////////////////////////////////////////////////////////////////////////////////////
index 92ccc9d..a6eeff4 100644 (file)
@@ -1,3 +1,21 @@
+2020-04-20  Carlos Garcia Campos  <cgarcia@igalia.com>
+
+        [GTK][WPE] Enable resource load statistics
+        https://bugs.webkit.org/show_bug.cgi?id=210184
+
+        Reviewed by Žan Doberšek.
+
+        * platform/network/soup/NetworkStorageSessionSoup.cpp:
+        (WebCore::NetworkStorageSession::setCookiesFromDOM const): Return early if cookies are blocked and update the
+        persistent cookies expiration if needed.
+        (WebCore::NetworkStorageSession::deleteCookiesForHostnames): Implement this when receiving
+        IncludeHttpOnlyCookies parameter.
+        (WebCore::NetworkStorageSession::hasCookies const): Implement this.
+        (WebCore::NetworkStorageSession::getRawCookies const): Honor shouldAskITP parameter.
+        (WebCore::cookiesForSession): Ditto.
+        (WebCore::NetworkStorageSession::cookiesForDOM const): Ditto.
+        (WebCore::NetworkStorageSession::cookieRequestHeaderFieldValue const): Ditto.
+
 2020-04-19  Simon Fraser  <simon.fraser@apple.com>
 
         Use Optional<FloatQuad> in TransformState
index 00015a2..cde64a7 100644 (file)
@@ -268,10 +268,16 @@ static inline bool httpOnlyCookieExists(const GSList* cookies, const gchar* name
     return false;
 }
 
-void NetworkStorageSession::setCookiesFromDOM(const URL& firstParty, const SameSiteInfo&, const URL& url, Optional<FrameIdentifier> frameID, Optional<PageIdentifier> pageID, ShouldAskITP, const String& value) const
+void NetworkStorageSession::setCookiesFromDOM(const URL& firstParty, const SameSiteInfo&, const URL& url, Optional<FrameIdentifier> frameID, Optional<PageIdentifier> pageID, ShouldAskITP shouldAskITP, const String& value) const
 {
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+    if (shouldAskITP == ShouldAskITP::Yes && shouldBlockCookies(firstParty, url, frameID, pageID))
+        return;
+#else
     UNUSED_PARAM(frameID);
     UNUSED_PARAM(pageID);
+    UNUSED_PARAM(shouldAskITP);
+#endif
     GUniquePtr<SoupURI> origin = urlToSoupURI(url);
     if (!origin)
         return;
@@ -280,6 +286,10 @@ void NetworkStorageSession::setCookiesFromDOM(const URL& firstParty, const SameS
     if (!firstPartyURI)
         return;
 
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+    auto cappedLifetime = clientSideCookieCap(RegistrableDomain { firstParty }, pageID);
+#endif
+
     // Get existing cookies for this origin.
     SoupCookieJar* jar = cookieStorage();
     GSList* existingCookies = soup_cookie_jar_get_cookie_list(jar, origin.get(), TRUE);
@@ -298,6 +308,17 @@ void NetworkStorageSession::setCookiesFromDOM(const URL& firstParty, const SameS
         if (httpOnlyCookieExists(existingCookies, soup_cookie_get_name(cookie.get()), soup_cookie_get_path(cookie.get())))
             continue;
 
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+        // Cap lifetime of persistent, client-side cookies to a week.
+        if (cappedLifetime) {
+            if (auto* expiresDate = soup_cookie_get_expires(cookie.get())) {
+                auto timeIntervalSinceNow = Seconds(static_cast<double>(soup_date_to_time_t(expiresDate))) - WallTime::now().secondsSinceEpoch();
+                if (timeIntervalSinceNow > cappedLifetime.value())
+                    soup_cookie_set_max_age(cookie.get(), cappedLifetime->secondsAs<int>());
+            }
+        }
+#endif
+
 #if SOUP_CHECK_VERSION(2, 67, 1)
         soup_cookie_jar_add_cookie_full(jar, cookie.release(), origin.get(), firstPartyURI.get());
 #else
@@ -380,28 +401,27 @@ void NetworkStorageSession::deleteAllCookiesModifiedSince(WallTime timestamp)
 
 void NetworkStorageSession::deleteCookiesForHostnames(const Vector<String>& hostnames, IncludeHttpOnlyCookies includeHttpOnlyCookies)
 {
-    // FIXME: Not yet implemented.
-    UNUSED_PARAM(includeHttpOnlyCookies);
-    deleteCookiesForHostnames(hostnames);
-}
-
-void NetworkStorageSession::deleteCookiesForHostnames(const Vector<String>& hostnames)
-{
     SoupCookieJar* cookieJar = cookieStorage();
-
     for (const auto& hostname : hostnames) {
         CString hostNameString = hostname.utf8();
 
         GUniquePtr<GSList> cookies(soup_cookie_jar_all_cookies(cookieJar));
-        for (GSList* item = cookies.get(); item; item = g_slist_next(item)) {
-            SoupCookie* cookie = static_cast<SoupCookie*>(item->data);
-            if (soup_cookie_domain_matches(cookie, hostNameString.data()))
-                soup_cookie_jar_delete_cookie(cookieJar, cookie);
-            soup_cookie_free(cookie);
+        for (auto* item = cookies.get(); item; item = g_slist_next(item)) {
+            GUniquePtr<SoupCookie> cookie(static_cast<SoupCookie*>(item->data));
+            if (includeHttpOnlyCookies == IncludeHttpOnlyCookies::No && soup_cookie_get_http_only(cookie.get()))
+                continue;
+
+            if (soup_cookie_domain_matches(cookie.get(), hostNameString.data()))
+                soup_cookie_jar_delete_cookie(cookieJar, cookie.get());
         }
     }
 }
 
+void NetworkStorageSession::deleteCookiesForHostnames(const Vector<String>& hostnames)
+{
+    deleteCookiesForHostnames(hostnames, IncludeHttpOnlyCookies::Yes);
+}
+
 void NetworkStorageSession::getHostnamesWithCookies(HashSet<String>& hostnames)
 {
     GUniquePtr<GSList> cookies(soup_cookie_jar_all_cookies(cookieStorage()));
@@ -440,22 +460,30 @@ Vector<Cookie> NetworkStorageSession::getCookies(const URL& url)
     return cookies;
 }
 
-void NetworkStorageSession::hasCookies(const RegistrableDomain&, CompletionHandler<void(bool)>&& completionHandler) const
+void NetworkStorageSession::hasCookies(const RegistrableDomain& domain, CompletionHandler<void(bool)>&& completionHandler) const
 {
-    // FIXME: Implement.
+    GUniquePtr<GSList> cookies(soup_cookie_jar_all_cookies(cookieStorage()));
+    for (auto* item = cookies.get(); item; item = g_slist_next(item)) {
+        GUniquePtr<SoupCookie> cookie(static_cast<SoupCookie*>(item->data));
+        if (RegistrableDomain::uncheckedCreateFromHost(cookie->domain) == domain) {
+            completionHandler(true);
+            return;
+        }
+    }
     completionHandler(false);
 }
 
-bool NetworkStorageSession::getRawCookies(const URL& firstParty, const SameSiteInfo& sameSiteInfo, const URL& url, Optional<FrameIdentifier> frameID, Optional<PageIdentifier> pageID, ShouldAskITP, Vector<Cookie>& rawCookies) const
+bool NetworkStorageSession::getRawCookies(const URL& firstParty, const SameSiteInfo& sameSiteInfo, const URL& url, Optional<FrameIdentifier> frameID, Optional<PageIdentifier> pageID, ShouldAskITP shouldAskITP, Vector<Cookie>& rawCookies) const
 {
     rawCookies.clear();
 
 #if ENABLE(RESOURCE_LOAD_STATISTICS)
-    if (shouldBlockCookies(firstParty, url, frameID, pageID))
+    if (shouldAskITP == ShouldAskITP::Yes && shouldBlockCookies(firstParty, url, frameID, pageID))
         return true;
 #else
     UNUSED_PARAM(frameID);
     UNUSED_PARAM(pageID);
+    UNUSED_PARAM(shouldAskITP);
 #endif
 
     GUniquePtr<SoupURI> uri = urlToSoupURI(url);
@@ -487,14 +515,15 @@ bool NetworkStorageSession::getRawCookies(const URL& firstParty, const SameSiteI
     return true;
 }
 
-static std::pair<String, bool> cookiesForSession(const NetworkStorageSession& session, const URL& firstParty, const URL& url, const SameSiteInfo& sameSiteInfo, Optional<FrameIdentifier> frameID, Optional<PageIdentifier> pageID, bool forHTTPHeader, IncludeSecureCookies includeSecureCookies)
+static std::pair<String, bool> cookiesForSession(const NetworkStorageSession& session, const URL& firstParty, const URL& url, const SameSiteInfo& sameSiteInfo, Optional<FrameIdentifier> frameID, Optional<PageIdentifier> pageID, bool forHTTPHeader, IncludeSecureCookies includeSecureCookies, ShouldAskITP shouldAskITP)
 {
 #if ENABLE(RESOURCE_LOAD_STATISTICS)
-    if (session.shouldBlockCookies(firstParty, url, frameID, pageID))
+    if (shouldAskITP == ShouldAskITP::Yes && session.shouldBlockCookies(firstParty, url, frameID, pageID))
         return { { }, false };
 #else
     UNUSED_PARAM(frameID);
     UNUSED_PARAM(pageID);
+    UNUSED_PARAM(shouldAskITP);
 #endif
 
     GUniquePtr<SoupURI> uri = urlToSoupURI(url);
@@ -543,15 +572,15 @@ static std::pair<String, bool> cookiesForSession(const NetworkStorageSession& se
     return { String::fromUTF8(cookieHeader.get()), didAccessSecureCookies };
 }
 
-std::pair<String, bool> NetworkStorageSession::cookiesForDOM(const URL& firstParty, const SameSiteInfo& sameSiteInfo, const URL& url, Optional<FrameIdentifier> frameID, Optional<PageIdentifier> pageID, IncludeSecureCookies includeSecureCookies, ShouldAskITP) const
+std::pair<String, bool> NetworkStorageSession::cookiesForDOM(const URL& firstParty, const SameSiteInfo& sameSiteInfo, const URL& url, Optional<FrameIdentifier> frameID, Optional<PageIdentifier> pageID, IncludeSecureCookies includeSecureCookies, ShouldAskITP shouldAskITP) const
 {
-    return cookiesForSession(*this, firstParty, url, sameSiteInfo, frameID, pageID, false, includeSecureCookies);
+    return cookiesForSession(*this, firstParty, url, sameSiteInfo, frameID, pageID, false, includeSecureCookies, shouldAskITP);
 }
 
-std::pair<String, bool> NetworkStorageSession::cookieRequestHeaderFieldValue(const URL& firstParty, const SameSiteInfo& sameSiteInfo, const URL& url, Optional<FrameIdentifier> frameID, Optional<PageIdentifier> pageID, IncludeSecureCookies includeSecureCookies, ShouldAskITP) const
+std::pair<String, bool> NetworkStorageSession::cookieRequestHeaderFieldValue(const URL& firstParty, const SameSiteInfo& sameSiteInfo, const URL& url, Optional<FrameIdentifier> frameID, Optional<PageIdentifier> pageID, IncludeSecureCookies includeSecureCookies, ShouldAskITP shouldAskITP) const
 {
     // Secure cookies will still only be included if url's protocol is https.
-    return cookiesForSession(*this, firstParty, url, sameSiteInfo, frameID, pageID, true, includeSecureCookies);
+    return cookiesForSession(*this, firstParty, url, sameSiteInfo, frameID, pageID, true, includeSecureCookies, shouldAskITP);
 }
 
 std::pair<String, bool> NetworkStorageSession::cookieRequestHeaderFieldValue(const CookieRequestHeaderFieldProxy& headerFieldProxy) const
index 263d5ce..78a3610 100644 (file)
@@ -1,3 +1,32 @@
+2020-04-20  Carlos Garcia Campos  <cgarcia@igalia.com>
+
+        [GTK][WPE] Enable resource load statistics
+        https://bugs.webkit.org/show_bug.cgi?id=210184
+
+        Reviewed by Žan Doberšek.
+
+        * NetworkProcess/NetworkDataTask.cpp:
+        (WebKit::NetworkDataTask::create): Pass page and frame identifiers to the constructor.
+        (WebKit::NetworkDataTask::isThirdPartyRequest const): Moved from NetworkDataTaskCocoa.mm.
+        (WebKit::NetworkDataTask::restrictRequestReferrerToOriginIfNeeded): Ditto.
+        * NetworkProcess/NetworkDataTask.h:
+        * NetworkProcess/cocoa/NetworkDataTaskCocoa.h:
+        * NetworkProcess/cocoa/NetworkDataTaskCocoa.mm:
+        * NetworkProcess/soup/NetworkDataTaskSoup.cpp:
+        (WebKit::NetworkDataTaskSoup::NetworkDataTaskSoup): Initialize m_frameID and m_pageID.
+        (WebKit::NetworkDataTaskSoup::createRequest): Add WasBlockingCookies parameter. Call
+        restrictRequestReferrerToOriginIfNeeded() and disable cookies in the soup message if cookies should be blocked.
+        (WebKit::NetworkDataTaskSoup::clearRequest): Reset m_isBlockingCookies.
+        (WebKit::NetworkDataTaskSoup::continueHTTPRedirection): Pass WasBlockingCookies to createRequest().
+        * NetworkProcess/soup/NetworkDataTaskSoup.h:
+        * UIProcess/API/C/WKPage.cpp:
+        (WKPageLoadedThirdPartyDomains): Added for tests.
+        (WKPageClearLoadedThirdPartyDomains): Ditto.
+        * UIProcess/API/C/WKPagePrivate.h:
+        * UIProcess/glib/WebsiteDataStoreGLib.cpp:
+        (WebKit::WebsiteDataStore::defaultResourceLoadStatisticsDirectory): Use lowercase for consistency with other
+        default directories.
+
 2020-04-20  David Kilzer  <ddkilzer@apple.com>
 
         REGRESSION (r259610): Leak of RBSAssertionInvalidationCallbackType due to missing -dealloc
index 9523766..dbcfce5 100644 (file)
@@ -30,7 +30,9 @@
 #include "NetworkDataTaskBlob.h"
 #include "NetworkLoadParameters.h"
 #include "NetworkSession.h"
+#include <WebCore/RegistrableDomain.h>
 #include <WebCore/ResourceError.h>
+#include <WebCore/ResourceRequest.h>
 #include <WebCore/ResourceResponse.h>
 #include <wtf/RunLoop.h>
 
@@ -54,7 +56,7 @@ Ref<NetworkDataTask> NetworkDataTask::create(NetworkSession& session, NetworkDat
     return NetworkDataTaskCocoa::create(session, client, parameters.request, parameters.webFrameID, parameters.webPageID, parameters.storedCredentialsPolicy, parameters.contentSniffingPolicy, parameters.contentEncodingSniffingPolicy, parameters.shouldClearReferrerOnHTTPSToHTTPRedirect, parameters.shouldPreconnectOnly, parameters.isMainFrameNavigation, parameters.isMainResourceNavigationForAnyFrame, parameters.networkActivityTracker, parameters.isNavigatingToAppBoundDomain);
 #endif
 #if USE(SOUP)
-    return NetworkDataTaskSoup::create(session, client, parameters.request, parameters.storedCredentialsPolicy, parameters.contentSniffingPolicy, parameters.contentEncodingSniffingPolicy, parameters.shouldClearReferrerOnHTTPSToHTTPRedirect, parameters.isMainFrameNavigation);
+    return NetworkDataTaskSoup::create(session, client, parameters.request, parameters.webFrameID, parameters.webPageID, parameters.storedCredentialsPolicy, parameters.contentSniffingPolicy, parameters.contentEncodingSniffingPolicy, parameters.shouldClearReferrerOnHTTPSToHTTPRedirect, parameters.isMainFrameNavigation);
 #endif
 #if USE(CURL)
     return NetworkDataTaskCurl::create(session, client, parameters.request, parameters.storedCredentialsPolicy, parameters.contentSniffingPolicy, parameters.contentEncodingSniffingPolicy, parameters.shouldClearReferrerOnHTTPSToHTTPRedirect, parameters.isMainFrameNavigation);
@@ -169,4 +171,15 @@ NetworkSession* NetworkDataTask::networkSession()
     return m_session.get();
 }
 
+bool NetworkDataTask::isThirdPartyRequest(const WebCore::ResourceRequest& request) const
+{
+    return !WebCore::areRegistrableDomainsEqual(request.url(), request.firstPartyForCookies());
+}
+
+void NetworkDataTask::restrictRequestReferrerToOriginIfNeeded(WebCore::ResourceRequest& request)
+{
+    if ((m_session->sessionID().isEphemeral() || m_session->isResourceLoadStatisticsEnabled()) && m_session->shouldDowngradeReferrer() && isThirdPartyRequest(request))
+        request.setExistingHTTPReferrerToOriginString();
+}
+
 } // namespace WebKit
index f5b4029..438f675 100644 (file)
@@ -148,6 +148,9 @@ protected:
     void failureTimerFired();
     void scheduleFailure(FailureType);
 
+    bool isThirdPartyRequest(const WebCore::ResourceRequest&) const;
+    void restrictRequestReferrerToOriginIfNeeded(WebCore::ResourceRequest&);
+
     FailureType m_scheduledFailureType { NoFailure };
     WebCore::Timer m_failureTimer;
     WeakPtr<NetworkSession> m_session;
index 7c95956..f2a8fd0 100644 (file)
@@ -81,15 +81,12 @@ private:
     bool tryPasswordBasedAuthentication(const WebCore::AuthenticationChallenge&, ChallengeCompletionHandler&);
     void applySniffingPoliciesAndBindRequestToInferfaceIfNeeded(__strong NSURLRequest*&, bool shouldContentSniff, bool shouldContentEncodingSniff);
 
-    void restrictRequestReferrerToOriginIfNeeded(WebCore::ResourceRequest&);
-
 #if ENABLE(RESOURCE_LOAD_STATISTICS)
     static NSHTTPCookieStorage *statelessCookieStorage();
     void blockCookies();
     void unblockCookies();
     bool needsFirstPartyCookieBlockingLatchModeQuirk(const URL& firstPartyURL, const URL& requestURL, const URL& redirectingURL) const;
 #endif
-    bool isThirdPartyRequest(const WebCore::ResourceRequest&) const;
     bool isAlwaysOnLoggingAllowed() const;
 
     WeakPtr<SessionWrapper> m_sessionWrapper;
index 0c88056..623a1ad 100644 (file)
@@ -38,7 +38,6 @@
 #import <WebCore/AuthenticationChallenge.h>
 #import <WebCore/NetworkStorageSession.h>
 #import <WebCore/NotImplemented.h>
-#import <WebCore/RegistrableDomain.h>
 #import <WebCore/ResourceRequest.h>
 #import <pal/spi/cf/CFNetworkSPI.h>
 #import <wtf/BlockPtr.h>
@@ -178,11 +177,6 @@ bool NetworkDataTaskCocoa::needsFirstPartyCookieBlockingLatchModeQuirk(const URL
 }
 #endif
 
-bool NetworkDataTaskCocoa::isThirdPartyRequest(const WebCore::ResourceRequest& request) const
-{
-    return !WebCore::areRegistrableDomainsEqual(request.url(), request.firstPartyForCookies());
-}
-
 static void updateTaskWithFirstPartyForSameSiteCookies(NSURLSessionDataTask* task, const WebCore::ResourceRequest& request)
 {
     if (request.isSameSiteUnspecified())
@@ -301,12 +295,6 @@ NetworkDataTaskCocoa::~NetworkDataTaskCocoa()
     m_sessionWrapper->dataTaskMap.remove([m_task taskIdentifier]);
 }
 
-void NetworkDataTaskCocoa::restrictRequestReferrerToOriginIfNeeded(WebCore::ResourceRequest& request)
-{
-    if ((m_session->sessionID().isEphemeral() || m_session->isResourceLoadStatisticsEnabled()) && m_session->shouldDowngradeReferrer() && isThirdPartyRequest(request))
-        request.setExistingHTTPReferrerToOriginString();
-}
-
 void NetworkDataTaskCocoa::didSendData(uint64_t totalBytesSent, uint64_t totalBytesExpectedToSend)
 {
     WTFEmitSignpost(m_task.get(), "DataTask", "sent %llu bytes (expected %llu bytes)", totalBytesSent, totalBytesExpectedToSend);
index 1de5ad9..fe1b63e 100644 (file)
@@ -50,8 +50,10 @@ using namespace WebCore;
 
 static const size_t gDefaultReadBufferSize = 8192;
 
-NetworkDataTaskSoup::NetworkDataTaskSoup(NetworkSession& session, NetworkDataTaskClient& client, const ResourceRequest& requestWithCredentials, StoredCredentialsPolicy storedCredentialsPolicy, ContentSniffingPolicy shouldContentSniff, WebCore::ContentEncodingSniffingPolicy, bool shouldClearReferrerOnHTTPSToHTTPRedirect, bool dataTaskIsForMainFrameNavigation)
+NetworkDataTaskSoup::NetworkDataTaskSoup(NetworkSession& session, NetworkDataTaskClient& client, const ResourceRequest& requestWithCredentials, FrameIdentifier frameID, PageIdentifier pageID, StoredCredentialsPolicy storedCredentialsPolicy, ContentSniffingPolicy shouldContentSniff, WebCore::ContentEncodingSniffingPolicy, bool shouldClearReferrerOnHTTPSToHTTPRedirect, bool dataTaskIsForMainFrameNavigation)
     : NetworkDataTask(session, client, requestWithCredentials, storedCredentialsPolicy, shouldClearReferrerOnHTTPSToHTTPRedirect, dataTaskIsForMainFrameNavigation)
+    , m_frameID(frameID)
+    , m_pageID(pageID)
     , m_shouldContentSniff(shouldContentSniff)
     , m_timeoutSource(RunLoop::main(), this, &NetworkDataTaskSoup::timeoutFired)
 {
@@ -75,7 +77,7 @@ NetworkDataTaskSoup::NetworkDataTaskSoup(NetworkSession& session, NetworkDataTas
         }
         applyAuthenticationToRequest(request);
     }
-    createRequest(WTFMove(request));
+    createRequest(WTFMove(request), WasBlockingCookies::No);
 }
 
 NetworkDataTaskSoup::~NetworkDataTaskSoup()
@@ -103,7 +105,7 @@ void NetworkDataTaskSoup::setPendingDownloadLocation(const String& filename, San
     m_allowOverwriteDownload = allowOverwrite;
 }
 
-void NetworkDataTaskSoup::createRequest(ResourceRequest&& request)
+void NetworkDataTaskSoup::createRequest(ResourceRequest&& request, WasBlockingCookies wasBlockingCookies)
 {
     m_currentRequest = WTFMove(request);
 
@@ -133,6 +135,8 @@ void NetworkDataTaskSoup::createRequest(ResourceRequest&& request)
         return;
     }
 
+    restrictRequestReferrerToOriginIfNeeded(m_currentRequest);
+
     unsigned messageFlags = SOUP_MESSAGE_NO_REDIRECT;
 
     m_currentRequest.updateSoupMessage(soupMessage.get(), m_session->blobRegistry());
@@ -148,6 +152,17 @@ void NetworkDataTaskSoup::createRequest(ResourceRequest&& request)
 #endif
     }
 
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+    bool shouldBlockCookies = wasBlockingCookies == WasBlockingCookies::Yes ? true : m_storedCredentialsPolicy == StoredCredentialsPolicy::EphemeralStateless;
+    if (!shouldBlockCookies) {
+        if (auto* networkStorageSession = m_session->networkStorageSession())
+            shouldBlockCookies = networkStorageSession->shouldBlockCookies(m_currentRequest, m_frameID, m_pageID);
+    }
+    if (shouldBlockCookies)
+        soup_message_disable_feature(soupMessage.get(), SOUP_TYPE_COOKIE_JAR);
+    m_isBlockingCookies = shouldBlockCookies;
+#endif
+
 #if SOUP_CHECK_VERSION(2, 67, 1)
     if ((m_currentRequest.url().protocolIs("https") && !shouldAllowHSTSPolicySetting()) || (m_currentRequest.url().protocolIs("http") && !shouldAllowHSTSProtocolUpgrade()))
         soup_message_disable_feature(soupMessage.get(), SOUP_TYPE_HSTS_ENFORCER);
@@ -194,6 +209,7 @@ void NetworkDataTaskSoup::clearRequest()
     m_downloadOutputStream = nullptr;
     g_cancellable_cancel(m_cancellable.get());
     m_cancellable = nullptr;
+    m_isBlockingCookies = false;
     if (m_soupMessage) {
         g_signal_handlers_disconnect_matched(m_soupMessage.get(), G_SIGNAL_MATCH_DATA, 0, 0, nullptr, nullptr, this);
         if (m_session)
@@ -703,10 +719,12 @@ void NetworkDataTaskSoup::continueHTTPRedirection()
         }
     }
 
+    auto wasBlockingCookies = m_isBlockingCookies ? WasBlockingCookies::Yes : WasBlockingCookies::No;
+
     clearRequest();
 
     auto response = ResourceResponse(m_response);
-    m_client->willPerformHTTPRedirection(WTFMove(response), WTFMove(request), [this, protectedThis = makeRef(*this), isCrossOrigin](const ResourceRequest& newRequest) {
+    m_client->willPerformHTTPRedirection(WTFMove(response), WTFMove(request), [this, protectedThis = makeRef(*this), isCrossOrigin, wasBlockingCookies](const ResourceRequest& newRequest) {
         if (newRequest.isNull() || m_state == State::Canceling)
             return;
 
@@ -719,7 +737,7 @@ void NetworkDataTaskSoup::continueHTTPRedirection()
 
             applyAuthenticationToRequest(request);
         }
-        createRequest(WTFMove(request));
+        createRequest(WTFMove(request), wasBlockingCookies);
         if (m_soupRequest && m_state != State::Suspended) {
             m_state = State::Suspended;
             resume();
index 1bde42d..74e31f3 100644 (file)
@@ -26,7 +26,9 @@
 #pragma once
 
 #include "NetworkDataTask.h"
+#include <WebCore/FrameIdentifier.h>
 #include <WebCore/NetworkLoadMetrics.h>
+#include <WebCore/PageIdentifier.h>
 #include <WebCore/ProtectionSpace.h>
 #include <WebCore/ResourceResponse.h>
 #include <wtf/RunLoop.h>
@@ -36,15 +38,15 @@ namespace WebKit {
 
 class NetworkDataTaskSoup final : public NetworkDataTask {
 public:
-    static Ref<NetworkDataTask> create(NetworkSession& session, NetworkDataTaskClient& client, const WebCore::ResourceRequest& request, WebCore::StoredCredentialsPolicy storedCredentialsPolicy, WebCore::ContentSniffingPolicy shouldContentSniff, WebCore::ContentEncodingSniffingPolicy shouldContentEncodingSniff, bool shouldClearReferrerOnHTTPSToHTTPRedirect, bool dataTaskIsForMainFrameNavigation)
+    static Ref<NetworkDataTask> create(NetworkSession& session, NetworkDataTaskClient& client, const WebCore::ResourceRequest& request, WebCore::FrameIdentifier frameID, WebCore::PageIdentifier pageID, WebCore::StoredCredentialsPolicy storedCredentialsPolicy, WebCore::ContentSniffingPolicy shouldContentSniff, WebCore::ContentEncodingSniffingPolicy shouldContentEncodingSniff, bool shouldClearReferrerOnHTTPSToHTTPRedirect, bool dataTaskIsForMainFrameNavigation)
     {
-        return adoptRef(*new NetworkDataTaskSoup(session, client, request, storedCredentialsPolicy, shouldContentSniff, shouldContentEncodingSniff, shouldClearReferrerOnHTTPSToHTTPRedirect, dataTaskIsForMainFrameNavigation));
+        return adoptRef(*new NetworkDataTaskSoup(session, client, request, frameID, pageID, storedCredentialsPolicy, shouldContentSniff, shouldContentEncodingSniff, shouldClearReferrerOnHTTPSToHTTPRedirect, dataTaskIsForMainFrameNavigation));
     }
 
     ~NetworkDataTaskSoup();
 
 private:
-    NetworkDataTaskSoup(NetworkSession&, NetworkDataTaskClient&, const WebCore::ResourceRequest&, WebCore::StoredCredentialsPolicy, WebCore::ContentSniffingPolicy, WebCore::ContentEncodingSniffingPolicy, bool shouldClearReferrerOnHTTPSToHTTPRedirect, bool dataTaskIsForMainFrameNavigation);
+    NetworkDataTaskSoup(NetworkSession&, NetworkDataTaskClient&, const WebCore::ResourceRequest&, WebCore::FrameIdentifier, WebCore::PageIdentifier, WebCore::StoredCredentialsPolicy, WebCore::ContentSniffingPolicy, WebCore::ContentEncodingSniffingPolicy, bool shouldClearReferrerOnHTTPSToHTTPRedirect, bool dataTaskIsForMainFrameNavigation);
 
     void cancel() override;
     void resume() override;
@@ -58,7 +60,8 @@ private:
     void startTimeout();
     void stopTimeout();
 
-    void createRequest(WebCore::ResourceRequest&&);
+    enum class WasBlockingCookies { No, Yes };
+    void createRequest(WebCore::ResourceRequest&&, WasBlockingCookies);
     void clearRequest();
     static void sendRequestCallback(SoupRequest*, GAsyncResult*, NetworkDataTaskSoup*);
     void didSendRequest(GRefPtr<GInputStream>&&);
@@ -122,6 +125,8 @@ private:
     static void restartedCallback(SoupMessage*, NetworkDataTaskSoup*);
     void didRestart();
 
+    WebCore::FrameIdentifier m_frameID;
+    WebCore::PageIdentifier m_pageID;
     State m_state { State::Suspended };
     WebCore::ContentSniffingPolicy m_shouldContentSniff;
     GRefPtr<SoupRequest> m_soupRequest;
@@ -143,6 +148,7 @@ private:
     bool m_allowOverwriteDownload { false };
     WebCore::NetworkLoadMetrics m_networkLoadMetrics;
     MonotonicTime m_startTime;
+    bool m_isBlockingCookies { false };
     RunLoop::Timer<NetworkDataTaskSoup> m_timeoutSource;
 };
 
index 09cfac9..b68e646 100644 (file)
@@ -2970,3 +2970,27 @@ WK_EXPORT bool WKPageIsMockRealtimeMediaSourceCenterEnabled(WKPageRef)
     return false;
 #endif
 }
+
+void WKPageLoadedThirdPartyDomains(WKPageRef page, WKPageLoadedThirdPartyDomainsFunction callback, void* callbackContext)
+{
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+    toImpl(page)->loadedThirdPartyDomains([callbackContext, callback](Vector<RegistrableDomain>&& domains) {
+        Vector<RefPtr<API::Object>> apiDomains = WTF::map(domains, [](auto& domain) {
+            return RefPtr<API::Object>(API::String::create(WTFMove(domain.string())));
+        });
+        callback(toAPI(API::Array::create(WTFMove(apiDomains)).ptr()), callbackContext);
+    });
+#else
+    UNUSED_PARAM(page);
+    callback(nullptr, callbackContext);
+#endif
+}
+
+void WKPageClearLoadedThirdPartyDomains(WKPageRef page)
+{
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+    toImpl(page)->clearLoadedThirdPartyDomains();
+#else
+    UNUSED_PARAM(page);
+#endif
+}
index 0e62f61..2d0ce3b 100644 (file)
@@ -194,6 +194,10 @@ WK_EXPORT void WKPageMarkAdClickAttributionsAsExpiredForTesting(WKPageRef page,
 WK_EXPORT void WKPageSetMockCameraOrientation(WKPageRef page, uint64_t orientation);
 WK_EXPORT bool WKPageIsMockRealtimeMediaSourceCenterEnabled(WKPageRef page);
 
+typedef void (*WKPageLoadedThirdPartyDomainsFunction)(WKArrayRef domains, void* functionContext);
+WK_EXPORT void WKPageLoadedThirdPartyDomains(WKPageRef page, WKPageLoadedThirdPartyDomainsFunction callback, void* callbackContext);
+WK_EXPORT void WKPageClearLoadedThirdPartyDomains(WKPageRef page);
+
 #ifdef __cplusplus
 }
 #endif
index e331616..8ea8ea2 100644 (file)
@@ -91,7 +91,7 @@ WTF::String WebsiteDataStore::defaultHSTSDirectory()
 
 WTF::String WebsiteDataStore::defaultResourceLoadStatisticsDirectory()
 {
-    return websiteDataDirectoryFileSystemRepresentation(BASE_DIRECTORY G_DIR_SEPARATOR_S "ResourceLoadStatistics");
+    return websiteDataDirectoryFileSystemRepresentation(BASE_DIRECTORY G_DIR_SEPARATOR_S "resourceloadstatistics");
 }
 
 WTF::String WebsiteDataStore::cacheDirectoryFileSystemRepresentation(const WTF::String& directoryName)
index ba17e3c..6546753 100644 (file)
@@ -185,6 +185,7 @@ WEBKIT_OPTION_DEFAULT_PORT_VALUE(ENABLE_WEB_RTC PRIVATE ${ENABLE_EXPERIMENTAL_FE
 WEBKIT_OPTION_DEFAULT_PORT_VALUE(ENABLE_CSS_PAINTING_API PRIVATE ${ENABLE_EXPERIMENTAL_FEATURES})
 WEBKIT_OPTION_DEFAULT_PORT_VALUE(ENABLE_CSS_TYPED_OM PRIVATE ${ENABLE_EXPERIMENTAL_FEATURES})
 WEBKIT_OPTION_DEFAULT_PORT_VALUE(ENABLE_CSS_CONIC_GRADIENTS PRIVATE ON)
+WEBKIT_OPTION_DEFAULT_PORT_VALUE(ENABLE_RESOURCE_LOAD_STATISTICS PRIVATE ON)
 
 if (USE_GTK4)
 WEBKIT_OPTION_DEFAULT_PORT_VALUE(ENABLE_NETSCAPE_PLUGIN_API PRIVATE OFF)
index e7ce799..1a26c4e 100644 (file)
@@ -64,6 +64,7 @@ WEBKIT_OPTION_DEFAULT_PORT_VALUE(ENABLE_MHTML PRIVATE ON)
 WEBKIT_OPTION_DEFAULT_PORT_VALUE(ENABLE_NETSCAPE_PLUGIN_API PRIVATE OFF)
 WEBKIT_OPTION_DEFAULT_PORT_VALUE(ENABLE_NOTIFICATIONS PRIVATE OFF)
 WEBKIT_OPTION_DEFAULT_PORT_VALUE(ENABLE_OFFSCREEN_CANVAS PRIVATE ${ENABLE_EXPERIMENTAL_FEATURES})
+WEBKIT_OPTION_DEFAULT_PORT_VALUE(ENABLE_RESOURCE_LOAD_STATISTICS PRIVATE ON)
 WEBKIT_OPTION_DEFAULT_PORT_VALUE(ENABLE_SERVICE_WORKER PRIVATE ON)
 WEBKIT_OPTION_DEFAULT_PORT_VALUE(ENABLE_SHAREABLE_RESOURCE PUBLIC ON)
 WEBKIT_OPTION_DEFAULT_PORT_VALUE(ENABLE_SMOOTH_SCROLLING PRIVATE OFF)
index bf60cb9..f5426ed 100644 (file)
@@ -1,3 +1,19 @@
+2020-04-20  Carlos Garcia Campos  <cgarcia@igalia.com>
+
+        [GTK][WPE] Enable resource load statistics
+        https://bugs.webkit.org/show_bug.cgi?id=210184
+
+        Reviewed by Žan Doberšek.
+
+        Implement TestController::loadedThirdPartyDomains() and TestController::clearLoadedThirdPartyDomains() for
+        non-cocoa ports.
+
+        * WebKitTestRunner/TestController.cpp:
+        (WTR::LoadedThirdPartyDomainsCallbackContext::LoadedThirdPartyDomainsCallbackContext):
+        (WTR::loadedThirdPartyDomainsCallback):
+        (WTR::TestController::loadedThirdPartyDomains):
+        (WTR::TestController::clearLoadedThirdPartyDomains):
+
 2020-04-20  David Kilzer  <ddkilzer@apple.com>
 
         check-webkit-style should not complain about missing config.h header in WebKitLegacy source files
index 9887052..0c6210d 100644 (file)
@@ -3152,12 +3152,43 @@ void TestController::getAllStorageAccessEntries()
 {
 }
 
+struct LoadedThirdPartyDomainsCallbackContext {
+    explicit LoadedThirdPartyDomainsCallbackContext(TestController& controller)
+        : testController(controller)
+    {
+    }
+
+    TestController& testController;
+    bool done { false };
+    Vector<String> result;
+};
+
+static void loadedThirdPartyDomainsCallback(WKArrayRef domains, void* userData)
+{
+    auto* context = static_cast<LoadedThirdPartyDomainsCallbackContext*>(userData);
+    context->done = true;
+
+    if (domains) {
+        auto size = WKArrayGetSize(domains);
+        context->result.reserveInitialCapacity(size);
+        for (size_t index = 0; index < size; ++index)
+            context->result.uncheckedAppend(toWTFString(static_cast<WKStringRef>(WKArrayGetItemAtIndex(domains, index))));
+    }
+
+    context->testController.notifyDone();
+}
+
 void TestController::loadedThirdPartyDomains()
 {
+    LoadedThirdPartyDomainsCallbackContext context(*this);
+    WKPageLoadedThirdPartyDomains(m_mainWebView->page(), loadedThirdPartyDomainsCallback, &context);
+    runUntil(context.done, noTimeout);
+    m_currentInvocation->didReceiveLoadedThirdPartyDomains(WTFMove(context.result));
 }
 
 void TestController::clearLoadedThirdPartyDomains()
 {
+    WKPageClearLoadedThirdPartyDomains(m_mainWebView->page());
 }
 
 void TestController::getWebViewCategory()