Resource Load Statistics: Downgrade all third-party referrer headers
authorcommit-queue@webkit.org <commit-queue@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 27 Sep 2019 05:13:10 +0000 (05:13 +0000)
committercommit-queue@webkit.org <commit-queue@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 27 Sep 2019 05:13:10 +0000 (05:13 +0000)
commit3f120524ad8f87eac080c7ed9bda8ce4210ac224
treeef7398b731a479679d68bd57c651f262205e4037
parent9e7749b9bef61da88318ac4a8bf0cbfe9c8e27fd
Resource Load Statistics: Downgrade all third-party referrer headers
https://bugs.webkit.org/show_bug.cgi?id=201353
<rdar://problem/54895650>
Source/WebKit:

Majority of this patch was written by John Wilander <wilander@apple.com>.

Patch by Kate Cheney <katherine_cheney@apple.com> on 2019-09-26
Reviewed by Brent Fulgham.

When tracking protections are enabled, we should downgrade all third-party
referrers to their origins. Note that this downgrade will be specific to
Cocoa so other ports will have to adopt as they see fit.

Cocoa already does this downgrade in ephemeral sessions (shipping).

The majority of these changes are test infrastructure. The functional
change is in WebKit::NetworkDataTaskCocoa and WebKit::NetworkSession.

* NetworkProcess/NetworkProcess.cpp:
(WebKit::NetworkProcess::setShouldDowngradeReferrerForTesting):
* NetworkProcess/NetworkProcess.h:
* NetworkProcess/NetworkProcess.messages.in:
* NetworkProcess/NetworkSession.cpp:
(WebKit::NetworkSession::isResourceLoadStatisticsEnabled const):
(WebKit::NetworkSession::setShouldDowngradeReferrerForTesting):
(WebKit::NetworkSession::shouldDowngradeReferrer const):
* NetworkProcess/NetworkSession.h:
* NetworkProcess/cocoa/NetworkDataTaskCocoa.h:
* NetworkProcess/cocoa/NetworkDataTaskCocoa.mm:
(WebKit::NetworkDataTaskCocoa::isThirdPartyRequest const):
(WebKit::NetworkDataTaskCocoa::NetworkDataTaskCocoa):
(WebKit::NetworkDataTaskCocoa::restrictRequestReferrerToOriginIfNeeded):
(WebKit::NetworkDataTaskCocoa::willPerformHTTPRedirection):
(WebKit::NetworkDataTaskCocoa::isThirdPartyRequest): Deleted.
* UIProcess/API/C/WKWebsiteDataStoreRef.cpp:
(WKWebsiteDataStoreSetResourceLoadStatisticsShouldDowngradeReferrerForTesting):
(WKWebsiteDataStoreStatisticsResetToConsistentState):
* UIProcess/API/C/WKWebsiteDataStoreRef.h:
* UIProcess/Network/NetworkProcessProxy.cpp:
(WebKit::NetworkProcessProxy::setShouldDowngradeReferrerForTesting):
* UIProcess/Network/NetworkProcessProxy.h:
* UIProcess/WebsiteData/WebsiteDataStore.cpp:
(WebKit::WebsiteDataStore::setResourceLoadStatisticsShouldDowngradeReferrerForTesting):
* UIProcess/WebsiteData/WebsiteDataStore.h:

Tools:

Majority of this patch was written by John Wilander <wilander@apple.com>.

Patch by Kate Cheney <katherine_cheney@apple.com> on 2019-09-26
Reviewed by Brent Fulgham.

The changes to the TestRunner facilitates an opt-out for test cases
that either test the referrer mechanism explicitly or tests that
rely on the full referrer to be sent.

The new boolean variable and early return in
TestRunner::setStatisticsShouldDowngradeReferrer() prevent the same
event from trying to set multiple TestRunner callbacks.

* WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl:
* WebKitTestRunner/InjectedBundle/InjectedBundle.cpp:
(WTR::InjectedBundle::didReceiveMessageToPage):
* WebKitTestRunner/InjectedBundle/TestRunner.cpp:
(WTR::TestRunner::setStatisticsShouldDowngradeReferrer):
(WTR::TestRunner::statisticsCallDidSetShouldDowngradeReferrerCallback):
* WebKitTestRunner/InjectedBundle/TestRunner.h:
* WebKitTestRunner/TestController.cpp:
(WTR::TestController::setStatisticsShouldDowngradeReferrer):
* WebKitTestRunner/TestController.h:
* WebKitTestRunner/TestInvocation.cpp:
(WTR::TestInvocation::didReceiveSynchronousMessageFromInjectedBundle):
(WTR::TestInvocation::didSetShouldDowngradeReferrer):
* WebKitTestRunner/TestInvocation.h:

LayoutTests:

Majority of this patch was written by John Wilander <wilander@apple.com>

Patch by Kate Cheney <katherine_cheney@apple.com> on 2019-09-26
Reviewed by Brent Fulgham.

The changes in the two http/tests/resourceLoadStatistics/strip-referrer-to-origin*
tests and the http/tests/navigation/ping-attribute/* tests are for the functional
change. The other changes are to make use of the new
testRunner.setStatisticsShouldDowngradeReferrer() to maintain earlier functionality.

TestRunner::setStatisticsShouldDowngradeReferrer() function is not
supported for mac-wk1, win or wincairo.

* http/tests/blink/sendbeacon/beacon-cross-origin-expected.txt:
* http/tests/navigation/ping-attribute/anchor-cross-origin.html:
* http/tests/navigation/ping-attribute/area-cross-origin.html:
* http/tests/referrer-policy-script/no-referrer-when-downgrade/cross-origin-http-http.html:
* http/tests/referrer-policy-script/no-referrer-when-downgrade/cross-origin-http.https.html:
* http/tests/referrer-policy-script/no-referrer-when-downgrade/same-origin.html:
* http/tests/referrer-policy-script/no-referrer/cross-origin-http-http.html:
* http/tests/referrer-policy-script/no-referrer/cross-origin-http.https.html:
* http/tests/referrer-policy-script/no-referrer/same-origin.html:
* http/tests/referrer-policy-script/origin-when-cross-origin/cross-origin-http-http.html:
* http/tests/referrer-policy-script/origin-when-cross-origin/cross-origin-http.https.html:
* http/tests/referrer-policy-script/origin-when-cross-origin/same-origin.html:
* http/tests/referrer-policy-script/origin/cross-origin-http-http.html:
* http/tests/referrer-policy-script/origin/cross-origin-http.https.html:
* http/tests/referrer-policy-script/origin/same-origin.html:
* http/tests/referrer-policy-script/same-origin/cross-origin-http-http.html:
* http/tests/referrer-policy-script/same-origin/cross-origin-http.https.html:
* http/tests/referrer-policy-script/same-origin/same-origin.html:
* http/tests/referrer-policy-script/strict-origin-when-cross-origin/cross-origin-http-http.html:
* http/tests/referrer-policy-script/strict-origin-when-cross-origin/cross-origin-http.https.html:
* http/tests/referrer-policy-script/strict-origin-when-cross-origin/same-origin.html:
* http/tests/referrer-policy-script/strict-origin/cross-origin-http-http.html:
* http/tests/referrer-policy-script/strict-origin/cross-origin-http.https.html:
* http/tests/referrer-policy-script/strict-origin/same-origin.html:
* http/tests/referrer-policy-script/unsafe-url/cross-origin-http-http.html:
* http/tests/referrer-policy-script/unsafe-url/cross-origin-http.https.html:
* http/tests/referrer-policy-script/unsafe-url/same-origin.html:
* http/tests/referrer-policy/no-referrer-when-downgrade/cross-origin-http-http.html:
* http/tests/referrer-policy/no-referrer-when-downgrade/cross-origin-http.https.html:
* http/tests/referrer-policy/no-referrer-when-downgrade/same-origin.html:
* http/tests/referrer-policy/unsafe-url/cross-origin-http-http.html:
* http/tests/referrer-policy/unsafe-url/cross-origin-http.https.html:
* http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-prevalent-subresource-requests.html: Removed.
* http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-third-party-redirects-expected.txt: Renamed from LayoutTests/http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-prevalent-subresource-redirects-expected.txt.
* http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-third-party-redirects.html: Copied from LayoutTests/http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-prevalent-subresource-redirects.html.
* http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-third-party-requests-expected.txt: Renamed from LayoutTests/http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-prevalent-subresource-requests-expected.txt.
* http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-third-party-requests.html: Renamed from LayoutTests/http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-prevalent-subresource-redirects.html.
* http/tests/security/contentSecurityPolicy/report-cross-origin-no-cookies-when-private-browsing-enabled.php:
* http/tests/security/contentSecurityPolicy/report-cross-origin-no-cookies.php:
* http/tests/security/referrer-policy-header.html:
* platform/ios-wk2/TestExpectations:
* platform/ios/TestExpectations:
* platform/mac-wk1/TestExpectations:
* platform/mac-wk2/TestExpectations:
* platform/win/TestExpectations:
* platform/wincairo/TestExpectations:
* platform/wk2/TestExpectations:
* resources/testharnessreport.js:
* resources/js-test.js:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@250413 268f45cc-cd09-0410-ab3c-d52691b4dbfc
73 files changed:
LayoutTests/ChangeLog
LayoutTests/http/tests/blink/sendbeacon/beacon-cross-origin-expected.txt
LayoutTests/http/tests/navigation/ping-attribute/anchor-cross-origin.html
LayoutTests/http/tests/navigation/ping-attribute/area-cross-origin.html
LayoutTests/http/tests/referrer-policy-script/no-referrer-when-downgrade/cross-origin-http-http.html
LayoutTests/http/tests/referrer-policy-script/no-referrer-when-downgrade/cross-origin-http.https.html
LayoutTests/http/tests/referrer-policy-script/no-referrer-when-downgrade/same-origin.html
LayoutTests/http/tests/referrer-policy-script/no-referrer/cross-origin-http-http.html
LayoutTests/http/tests/referrer-policy-script/no-referrer/cross-origin-http.https.html
LayoutTests/http/tests/referrer-policy-script/no-referrer/same-origin.html
LayoutTests/http/tests/referrer-policy-script/origin-when-cross-origin/cross-origin-http-http.html
LayoutTests/http/tests/referrer-policy-script/origin-when-cross-origin/cross-origin-http.https.html
LayoutTests/http/tests/referrer-policy-script/origin-when-cross-origin/same-origin.html
LayoutTests/http/tests/referrer-policy-script/origin/cross-origin-http-http.html
LayoutTests/http/tests/referrer-policy-script/origin/cross-origin-http.https.html
LayoutTests/http/tests/referrer-policy-script/origin/same-origin.html
LayoutTests/http/tests/referrer-policy-script/same-origin/cross-origin-http-http.html
LayoutTests/http/tests/referrer-policy-script/same-origin/cross-origin-http.https.html
LayoutTests/http/tests/referrer-policy-script/same-origin/same-origin.html
LayoutTests/http/tests/referrer-policy-script/strict-origin-when-cross-origin/cross-origin-http-http.html
LayoutTests/http/tests/referrer-policy-script/strict-origin-when-cross-origin/cross-origin-http.https.html
LayoutTests/http/tests/referrer-policy-script/strict-origin-when-cross-origin/same-origin.html
LayoutTests/http/tests/referrer-policy-script/strict-origin/cross-origin-http-http.html
LayoutTests/http/tests/referrer-policy-script/strict-origin/cross-origin-http.https.html
LayoutTests/http/tests/referrer-policy-script/strict-origin/same-origin.html
LayoutTests/http/tests/referrer-policy-script/unsafe-url/cross-origin-http-http.html
LayoutTests/http/tests/referrer-policy-script/unsafe-url/cross-origin-http.https.html
LayoutTests/http/tests/referrer-policy-script/unsafe-url/same-origin.html
LayoutTests/http/tests/referrer-policy/no-referrer-when-downgrade/cross-origin-http-http.html
LayoutTests/http/tests/referrer-policy/no-referrer-when-downgrade/cross-origin-http.https.html
LayoutTests/http/tests/referrer-policy/no-referrer-when-downgrade/same-origin.html
LayoutTests/http/tests/referrer-policy/unsafe-url/cross-origin-http-http.html
LayoutTests/http/tests/referrer-policy/unsafe-url/cross-origin-http.https.html
LayoutTests/http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-prevalent-subresource-requests.html [deleted file]
LayoutTests/http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-third-party-redirects-expected.txt [moved from LayoutTests/http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-prevalent-subresource-redirects-expected.txt with 60% similarity]
LayoutTests/http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-third-party-redirects.html [new file with mode: 0644]
LayoutTests/http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-third-party-requests-expected.txt [moved from LayoutTests/http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-prevalent-subresource-requests-expected.txt with 65% similarity]
LayoutTests/http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-third-party-requests.html [moved from LayoutTests/http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-prevalent-subresource-redirects.html with 53% similarity]
LayoutTests/http/tests/security/contentSecurityPolicy/report-cross-origin-no-cookies-when-private-browsing-enabled.php
LayoutTests/http/tests/security/contentSecurityPolicy/report-cross-origin-no-cookies.php
LayoutTests/http/tests/security/referrer-policy-header.html
LayoutTests/platform/ios-wk2/TestExpectations
LayoutTests/platform/ios/TestExpectations
LayoutTests/platform/mac-wk1/TestExpectations
LayoutTests/platform/mac-wk2/TestExpectations
LayoutTests/platform/win/TestExpectations
LayoutTests/platform/wincairo/TestExpectations
LayoutTests/platform/wk2/TestExpectations
LayoutTests/resources/js-test.js
LayoutTests/resources/testharnessreport.js
Source/WebKit/ChangeLog
Source/WebKit/NetworkProcess/NetworkProcess.cpp
Source/WebKit/NetworkProcess/NetworkProcess.h
Source/WebKit/NetworkProcess/NetworkProcess.messages.in
Source/WebKit/NetworkProcess/NetworkSession.cpp
Source/WebKit/NetworkProcess/NetworkSession.h
Source/WebKit/NetworkProcess/cocoa/NetworkDataTaskCocoa.h
Source/WebKit/NetworkProcess/cocoa/NetworkDataTaskCocoa.mm
Source/WebKit/UIProcess/API/C/WKWebsiteDataStoreRef.cpp
Source/WebKit/UIProcess/API/C/WKWebsiteDataStoreRef.h
Source/WebKit/UIProcess/Network/NetworkProcessProxy.cpp
Source/WebKit/UIProcess/Network/NetworkProcessProxy.h
Source/WebKit/UIProcess/WebsiteData/WebsiteDataStore.cpp
Source/WebKit/UIProcess/WebsiteData/WebsiteDataStore.h
Tools/ChangeLog
Tools/WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl
Tools/WebKitTestRunner/InjectedBundle/InjectedBundle.cpp
Tools/WebKitTestRunner/InjectedBundle/TestRunner.cpp
Tools/WebKitTestRunner/InjectedBundle/TestRunner.h
Tools/WebKitTestRunner/TestController.cpp
Tools/WebKitTestRunner/TestController.h
Tools/WebKitTestRunner/TestInvocation.cpp
Tools/WebKitTestRunner/TestInvocation.h