Resource Load Statistics: Downgrade all third-party referrer headers

Resource Load Statistics: Downgrade all third-party referrer headers
https://bugs.webkit.org/show_bug.cgi?id=201353
<rdar://problem/54895650>
Source/WebKit:

Majority of this patch was written by John Wilander <wilander@apple.com>.

Patch by Kate Cheney <katherine_cheney@apple.com> on 2019-09-26
Reviewed by Brent Fulgham.

When tracking protections are enabled, we should downgrade all third-party
referrers to their origins. Note that this downgrade will be specific to
Cocoa so other ports will have to adopt as they see fit.

Cocoa already does this downgrade in ephemeral sessions (shipping).

The majority of these changes are test infrastructure. The functional
change is in WebKit::NetworkDataTaskCocoa and WebKit::NetworkSession.

* NetworkProcess/NetworkProcess.cpp:
(WebKit::NetworkProcess::setShouldDowngradeReferrerForTesting):
* NetworkProcess/NetworkProcess.h:
* NetworkProcess/NetworkProcess.messages.in:
* NetworkProcess/NetworkSession.cpp:
(WebKit::NetworkSession::isResourceLoadStatisticsEnabled const):
(WebKit::NetworkSession::setShouldDowngradeReferrerForTesting):
(WebKit::NetworkSession::shouldDowngradeReferrer const):
* NetworkProcess/NetworkSession.h:
* NetworkProcess/cocoa/NetworkDataTaskCocoa.h:
* NetworkProcess/cocoa/NetworkDataTaskCocoa.mm:
(WebKit::NetworkDataTaskCocoa::isThirdPartyRequest const):
(WebKit::NetworkDataTaskCocoa::NetworkDataTaskCocoa):
(WebKit::NetworkDataTaskCocoa::restrictRequestReferrerToOriginIfNeeded):
(WebKit::NetworkDataTaskCocoa::willPerformHTTPRedirection):
(WebKit::NetworkDataTaskCocoa::isThirdPartyRequest): Deleted.
* UIProcess/API/C/WKWebsiteDataStoreRef.cpp:
(WKWebsiteDataStoreSetResourceLoadStatisticsShouldDowngradeReferrerForTesting):
(WKWebsiteDataStoreStatisticsResetToConsistentState):
* UIProcess/API/C/WKWebsiteDataStoreRef.h:
* UIProcess/Network/NetworkProcessProxy.cpp:
(WebKit::NetworkProcessProxy::setShouldDowngradeReferrerForTesting):
* UIProcess/Network/NetworkProcessProxy.h:
* UIProcess/WebsiteData/WebsiteDataStore.cpp:
(WebKit::WebsiteDataStore::setResourceLoadStatisticsShouldDowngradeReferrerForTesting):
* UIProcess/WebsiteData/WebsiteDataStore.h:

Tools:

Majority of this patch was written by John Wilander <wilander@apple.com>.

Patch by Kate Cheney <katherine_cheney@apple.com> on 2019-09-26
Reviewed by Brent Fulgham.

The changes to the TestRunner facilitates an opt-out for test cases
that either test the referrer mechanism explicitly or tests that
rely on the full referrer to be sent.

The new boolean variable and early return in
TestRunner::setStatisticsShouldDowngradeReferrer() prevent the same
event from trying to set multiple TestRunner callbacks.

* WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl:
* WebKitTestRunner/InjectedBundle/InjectedBundle.cpp:
(WTR::InjectedBundle::didReceiveMessageToPage):
* WebKitTestRunner/InjectedBundle/TestRunner.cpp:
(WTR::TestRunner::setStatisticsShouldDowngradeReferrer):
(WTR::TestRunner::statisticsCallDidSetShouldDowngradeReferrerCallback):
* WebKitTestRunner/InjectedBundle/TestRunner.h:
* WebKitTestRunner/TestController.cpp:
(WTR::TestController::setStatisticsShouldDowngradeReferrer):
* WebKitTestRunner/TestController.h:
* WebKitTestRunner/TestInvocation.cpp:
(WTR::TestInvocation::didReceiveSynchronousMessageFromInjectedBundle):
(WTR::TestInvocation::didSetShouldDowngradeReferrer):
* WebKitTestRunner/TestInvocation.h:

LayoutTests:

Majority of this patch was written by John Wilander <wilander@apple.com>

Patch by Kate Cheney <katherine_cheney@apple.com> on 2019-09-26
Reviewed by Brent Fulgham.

The changes in the two http/tests/resourceLoadStatistics/strip-referrer-to-origin*
tests and the http/tests/navigation/ping-attribute/* tests are for the functional
change. The other changes are to make use of the new
testRunner.setStatisticsShouldDowngradeReferrer() to maintain earlier functionality.

TestRunner::setStatisticsShouldDowngradeReferrer() function is not
supported for mac-wk1, win or wincairo.

* http/tests/blink/sendbeacon/beacon-cross-origin-expected.txt:
* http/tests/navigation/ping-attribute/anchor-cross-origin.html:
* http/tests/navigation/ping-attribute/area-cross-origin.html:
* http/tests/referrer-policy-script/no-referrer-when-downgrade/cross-origin-http-http.html:
* http/tests/referrer-policy-script/no-referrer-when-downgrade/cross-origin-http.https.html:
* http/tests/referrer-policy-script/no-referrer-when-downgrade/same-origin.html:
* http/tests/referrer-policy-script/no-referrer/cross-origin-http-http.html:
* http/tests/referrer-policy-script/no-referrer/cross-origin-http.https.html:
* http/tests/referrer-policy-script/no-referrer/same-origin.html:
* http/tests/referrer-policy-script/origin-when-cross-origin/cross-origin-http-http.html:
* http/tests/referrer-policy-script/origin-when-cross-origin/cross-origin-http.https.html:
* http/tests/referrer-policy-script/origin-when-cross-origin/same-origin.html:
* http/tests/referrer-policy-script/origin/cross-origin-http-http.html:
* http/tests/referrer-policy-script/origin/cross-origin-http.https.html:
* http/tests/referrer-policy-script/origin/same-origin.html:
* http/tests/referrer-policy-script/same-origin/cross-origin-http-http.html:
* http/tests/referrer-policy-script/same-origin/cross-origin-http.https.html:
* http/tests/referrer-policy-script/same-origin/same-origin.html:
* http/tests/referrer-policy-script/strict-origin-when-cross-origin/cross-origin-http-http.html:
* http/tests/referrer-policy-script/strict-origin-when-cross-origin/cross-origin-http.https.html:
* http/tests/referrer-policy-script/strict-origin-when-cross-origin/same-origin.html:
* http/tests/referrer-policy-script/strict-origin/cross-origin-http-http.html:
* http/tests/referrer-policy-script/strict-origin/cross-origin-http.https.html:
* http/tests/referrer-policy-script/strict-origin/same-origin.html:
* http/tests/referrer-policy-script/unsafe-url/cross-origin-http-http.html:
* http/tests/referrer-policy-script/unsafe-url/cross-origin-http.https.html:
* http/tests/referrer-policy-script/unsafe-url/same-origin.html:
* http/tests/referrer-policy/no-referrer-when-downgrade/cross-origin-http-http.html:
* http/tests/referrer-policy/no-referrer-when-downgrade/cross-origin-http.https.html:
* http/tests/referrer-policy/no-referrer-when-downgrade/same-origin.html:
* http/tests/referrer-policy/unsafe-url/cross-origin-http-http.html:
* http/tests/referrer-policy/unsafe-url/cross-origin-http.https.html:
* http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-prevalent-subresource-requests.html: Removed.
* http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-third-party-redirects-expected.txt: Renamed from LayoutTests/http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-prevalent-subresource-redirects-expected.txt.
* http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-third-party-redirects.html: Copied from LayoutTests/http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-prevalent-subresource-redirects.html.
* http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-third-party-requests-expected.txt: Renamed from LayoutTests/http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-prevalent-subresource-requests-expected.txt.
* http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-third-party-requests.html: Renamed from LayoutTests/http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-prevalent-subresource-redirects.html.
* http/tests/security/contentSecurityPolicy/report-cross-origin-no-cookies-when-private-browsing-enabled.php:
* http/tests/security/contentSecurityPolicy/report-cross-origin-no-cookies.php:
* http/tests/security/referrer-policy-header.html:
* platform/ios-wk2/TestExpectations:
* platform/ios/TestExpectations:
* platform/mac-wk1/TestExpectations:
* platform/mac-wk2/TestExpectations:
* platform/win/TestExpectations:
* platform/wincairo/TestExpectations:
* platform/wk2/TestExpectations:
* resources/testharnessreport.js:
* resources/js-test.js:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@250413 268f45cc-cd09-0410-ab3c-d52691b4dbfc