XMLHttpRequest should not sniff content encoding
authordbates@webkit.org <dbates@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 1 Nov 2017 22:23:41 +0000 (22:23 +0000)
committerdbates@webkit.org <dbates@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 1 Nov 2017 22:23:41 +0000 (22:23 +0000)
commit8c69e06b3103362468bc1df9642d53859cb04327
treed26cb88131d43beafcb4859f5aa1c8c131bc472e
parenta09997bc16247146560dccc05105cca1607ae965
XMLHttpRequest should not sniff content encoding
https://bugs.webkit.org/show_bug.cgi?id=175597
<rdar://problem/34912624>

Reviewed by Alex Christensen.

Source/WebCore:

Fixes an issue where the body of an HTTP response associated with an XHR request to a .gz file
would be automatically gzipped decompressed if the HTTP response omitted a Content-Encoding HTTP
header. Specifically, such a response would be treated analogous to a response with headers
"Content-Type: application/gzip" and "Content-Encoding: identity". This behavior does not conform
the HTTP 1.1 spec. and breaks Epic Zen Garden, <https://s3.amazonaws.com/mozilla-games/ZenGarden/EpicZenGarden.html>.

On macOS 10.13.2 opt out of content encoding sniffing when making an XHR request. We likely can
selectively opt out of content encoding sniffing for other network requests. This will be done
in subsequent commits to make it straightforward to identify site breakage (if any).

* loader/ResourceLoader.cpp:
(WebCore::ResourceLoader::start): Pass content encoding policy.
* loader/ResourceLoader.h:
(WebCore::ResourceLoader::shouldSniffContentEncoding const): Added.
* loader/ResourceLoaderOptions.h:
* loader/appcache/ApplicationCacheGroup.cpp:
(WebCore::ApplicationCacheGroup::createResourceHandle): Enable content encoding sniff to match existing behavior.
* platform/network/BlobResourceHandle.cpp:
(WebCore::BlobResourceHandle::BlobResourceHandle): Ditto. We should look to disable content encoding sniffing in
a subsequent change.
* platform/network/PingHandle.h: Ditto.
* platform/network/ResourceHandle.cpp:
(WebCore::ResourceHandle::ResourceHandle): Modified to take a boolean as to whether to enable content encoding sniffing.
(WebCore::ResourceHandle::create): Ditto.
(WebCore::ResourceHandle::shouldContentEncodingSniff const): Added.
* platform/network/ResourceHandle.h:
* platform/network/ResourceHandleInternal.h:
(WebCore::ResourceHandleInternal::ResourceHandleInternal): Modified to take a boolean as to whether to enable content
encoding sniffing.
* platform/network/cf/ResourceHandleCFNet.cpp:
(WebCore::ResourceHandle::createCFURLConnection): Modified to take a boolean as to whether to enable content encoding
sniffing and apply this policy to the CFMutableURLRequestRef object when building on macOS 10.13.2.
(WebCore::ResourceHandle::start):
(WebCore::ResourceHandle::platformLoadResourceSynchronously): Enable content encoding sniff to match existing behavior.
* platform/network/mac/ResourceHandleMac.mm:
(WebCore::ResourceHandle::applySniffingPoliciesAndStoragePartitionIfNeeded): Added helper function to apply sniffing policies
and storage partition, if applicable.
(WebCore::ResourceHandle::createNSURLConnection): Modified to take a boolean as to whether to enable content encoding
sniffing. Calls adjustNSRequestApplyingPolicies() to apply this policy.
(WebCore::ResourceHandle::start):
(WebCore::ResourceHandle::platformLoadResourceSynchronously): Enable content encoding sniff to match existing behavior.
* platform/network/soup/ResourceHandleSoup.cpp:
(WebCore::ResourceHandle::create): Modified to take a boolean as to whether to enable content encoding sniffing.
(WebCore::ResourceHandle::ResourceHandle): Ditto.
(WebCore::ResourceHandle::releaseForDownload): Pass content encoding policy.
* xml/XMLHttpRequest.cpp:
(WebCore::XMLHttpRequest::createRequest): Do not enable content encoding sniffing for the request.

Source/WebCore/PAL:

Forward declare CFNetwork SPI.

* pal/spi/cf/CFNetworkSPI.h:

Source/WebKit:

Fixes an issue where the body of an HTTP response associated with an XHR request to a .gz file
would be automatically gzipped decompressed if the HTTP response omitted a Content-Encoding HTTP
header. Specifically, such a response would be treated analogous to a response with headers
"Content-Type: application/gzip" and "Content-Encoding: identity". This behavior does not conform
the HTTP 1.1 spec. and breaks Epic Zen Garden, <https://s3.amazonaws.com/mozilla-games/ZenGarden/EpicZenGarden.html>.

On macOS 10.13.2 opt out of content encoding sniffing when making an XHR request. We likely can
selectively opt out of content encoding sniffing for other network requests. This will be done
in subsequent commits to make it straightforward to identify site breakage (if any).

* NetworkProcess/Downloads/Download.cpp:
(WebKit::Download::start): Enable content encoding sniff to match existing behavior.
(WebKit::Download::startWithHandle): Ditto.
* NetworkProcess/NetworkDataTask.cpp:
(WebKit::NetworkDataTask::create): Pass through the content encoding sniffing policy.
* NetworkProcess/NetworkLoad.cpp:
(WebKit::NetworkLoad::NetworkLoad): Ditto.
* NetworkProcess/NetworkLoadParameters.h:
* NetworkProcess/NetworkResourceLoadParameters.cpp:
(WebKit::NetworkResourceLoadParameters::encode const): Encode content encoding sniffing policy.
(WebKit::NetworkResourceLoadParameters::decode): Decode content encoding sniffing policy.
* NetworkProcess/cache/NetworkCacheSpeculativeLoad.cpp:
(WebKit::NetworkCache::SpeculativeLoad::SpeculativeLoad): Enable content encoding sniff to match existing
behavior. We should look to disable content encoding sniffing in a subsequent change.
* NetworkProcess/cocoa/NetworkDataTaskCocoa.h:
* NetworkProcess/cocoa/NetworkDataTaskCocoa.mm:
(WebKit::applySniffingPoliciesAndBindRequestToInferfaceIfNeeded): Added helper function
to apply sniffing policies and bind request to interface, if applicable.
(WebKit::NetworkDataTaskCocoa::NetworkDataTaskCocoa): Modified to take the content encoding sniffing
policy. Calls applySniffingPoliciesAndBindRequestToInferfaceIfNeeded() to apply this policy. Also use
convenience function URL::isLocalFile() to determine if the URL is a file URL.
* NetworkProcess/soup/NetworkDataTaskSoup.cpp:
(WebKit::NetworkDataTaskSoup::NetworkDataTaskSoup): Pass through the content encoding sniffing policy.
* NetworkProcess/soup/NetworkDataTaskSoup.h:
* WebProcess/Network/WebLoaderStrategy.cpp:
(WebKit::WebLoaderStrategy::scheduleLoadFromNetworkProcess): Pass through the content encoding sniffing policy.
(WebKit::WebLoaderStrategy::loadResourceSynchronously): Enable content encoding sniff to match existing
behavior.

LayoutTests:

Fix up the test http/tests/xmlhttprequest/gzip-content-type-no-content-encoding.html to
actually check that we do not sniff content encoding. CFNetwork only sniffs the URL
for the content encoding when the filename ends with known file extension (e.g. .gz)
and the HTTP response does not specifying a Content-Encoding HTTP header.

* TestExpectations: Skip the test on all platforms. Once <rdar://problem/33822249> ships
then we will enable the test on all platforms that use CFNetwork.
* http/tests/xmlhttprequest/gzip-content-type-no-content-encoding-expected.txt:
* http/tests/xmlhttprequest/gzip-content-type-no-content-encoding.html:
* http/tests/xmlhttprequest/resources/.htaccess: Interpret files with .php.gz extension as PHP files.
* http/tests/xmlhttprequest/resources/gzip-lorem-no-content-encoding.php.gz: Renamed from LayoutTests/http/tests/xmlhttprequest/resources/gzip-lorem-no-content-encoding.php.
* platform/mac/TestExpectations: Mark the test as flaky (Pass Failure) until <rdar://problem/33822249> ships.

git-svn-id: http://svn.webkit.org/repository/webkit/trunk@224299 268f45cc-cd09-0410-ab3c-d52691b4dbfc
35 files changed:
LayoutTests/ChangeLog
LayoutTests/TestExpectations
LayoutTests/http/tests/xmlhttprequest/gzip-content-type-no-content-encoding-expected.txt
LayoutTests/http/tests/xmlhttprequest/gzip-content-type-no-content-encoding.html
LayoutTests/http/tests/xmlhttprequest/resources/.htaccess
LayoutTests/http/tests/xmlhttprequest/resources/gzip-lorem-no-content-encoding.php.gz [moved from LayoutTests/http/tests/xmlhttprequest/resources/gzip-lorem-no-content-encoding.php with 100% similarity]
LayoutTests/platform/mac/TestExpectations
Source/WebCore/ChangeLog
Source/WebCore/PAL/ChangeLog
Source/WebCore/PAL/pal/spi/cf/CFNetworkSPI.h
Source/WebCore/loader/ResourceLoader.cpp
Source/WebCore/loader/ResourceLoader.h
Source/WebCore/loader/ResourceLoaderOptions.h
Source/WebCore/loader/appcache/ApplicationCacheGroup.cpp
Source/WebCore/platform/network/BlobResourceHandle.cpp
Source/WebCore/platform/network/PingHandle.h
Source/WebCore/platform/network/ResourceHandle.cpp
Source/WebCore/platform/network/ResourceHandle.h
Source/WebCore/platform/network/ResourceHandleInternal.h
Source/WebCore/platform/network/cf/ResourceHandleCFNet.cpp
Source/WebCore/platform/network/mac/ResourceHandleMac.mm
Source/WebCore/platform/network/soup/ResourceHandleSoup.cpp
Source/WebCore/xml/XMLHttpRequest.cpp
Source/WebKit/ChangeLog
Source/WebKit/NetworkProcess/Downloads/Download.cpp
Source/WebKit/NetworkProcess/NetworkDataTask.cpp
Source/WebKit/NetworkProcess/NetworkLoad.cpp
Source/WebKit/NetworkProcess/NetworkLoadParameters.h
Source/WebKit/NetworkProcess/NetworkResourceLoadParameters.cpp
Source/WebKit/NetworkProcess/cache/NetworkCacheSpeculativeLoad.cpp
Source/WebKit/NetworkProcess/cocoa/NetworkDataTaskCocoa.h
Source/WebKit/NetworkProcess/cocoa/NetworkDataTaskCocoa.mm
Source/WebKit/NetworkProcess/soup/NetworkDataTaskSoup.cpp
Source/WebKit/NetworkProcess/soup/NetworkDataTaskSoup.h
Source/WebKit/WebProcess/Network/WebLoaderStrategy.cpp