XMLHttpRequest should not sniff content encoding
authordbates@webkit.org <dbates@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 1 Nov 2017 22:23:41 +0000 (22:23 +0000)
committerdbates@webkit.org <dbates@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 1 Nov 2017 22:23:41 +0000 (22:23 +0000)
https://bugs.webkit.org/show_bug.cgi?id=175597
<rdar://problem/34912624>

Reviewed by Alex Christensen.

Source/WebCore:

Fixes an issue where the body of an HTTP response associated with an XHR request to a .gz file
would be automatically gzipped decompressed if the HTTP response omitted a Content-Encoding HTTP
header. Specifically, such a response would be treated analogous to a response with headers
"Content-Type: application/gzip" and "Content-Encoding: identity". This behavior does not conform
the HTTP 1.1 spec. and breaks Epic Zen Garden, <https://s3.amazonaws.com/mozilla-games/ZenGarden/EpicZenGarden.html>.

On macOS 10.13.2 opt out of content encoding sniffing when making an XHR request. We likely can
selectively opt out of content encoding sniffing for other network requests. This will be done
in subsequent commits to make it straightforward to identify site breakage (if any).

* loader/ResourceLoader.cpp:
(WebCore::ResourceLoader::start): Pass content encoding policy.
* loader/ResourceLoader.h:
(WebCore::ResourceLoader::shouldSniffContentEncoding const): Added.
* loader/ResourceLoaderOptions.h:
* loader/appcache/ApplicationCacheGroup.cpp:
(WebCore::ApplicationCacheGroup::createResourceHandle): Enable content encoding sniff to match existing behavior.
* platform/network/BlobResourceHandle.cpp:
(WebCore::BlobResourceHandle::BlobResourceHandle): Ditto. We should look to disable content encoding sniffing in
a subsequent change.
* platform/network/PingHandle.h: Ditto.
* platform/network/ResourceHandle.cpp:
(WebCore::ResourceHandle::ResourceHandle): Modified to take a boolean as to whether to enable content encoding sniffing.
(WebCore::ResourceHandle::create): Ditto.
(WebCore::ResourceHandle::shouldContentEncodingSniff const): Added.
* platform/network/ResourceHandle.h:
* platform/network/ResourceHandleInternal.h:
(WebCore::ResourceHandleInternal::ResourceHandleInternal): Modified to take a boolean as to whether to enable content
encoding sniffing.
* platform/network/cf/ResourceHandleCFNet.cpp:
(WebCore::ResourceHandle::createCFURLConnection): Modified to take a boolean as to whether to enable content encoding
sniffing and apply this policy to the CFMutableURLRequestRef object when building on macOS 10.13.2.
(WebCore::ResourceHandle::start):
(WebCore::ResourceHandle::platformLoadResourceSynchronously): Enable content encoding sniff to match existing behavior.
* platform/network/mac/ResourceHandleMac.mm:
(WebCore::ResourceHandle::applySniffingPoliciesAndStoragePartitionIfNeeded): Added helper function to apply sniffing policies
and storage partition, if applicable.
(WebCore::ResourceHandle::createNSURLConnection): Modified to take a boolean as to whether to enable content encoding
sniffing. Calls adjustNSRequestApplyingPolicies() to apply this policy.
(WebCore::ResourceHandle::start):
(WebCore::ResourceHandle::platformLoadResourceSynchronously): Enable content encoding sniff to match existing behavior.
* platform/network/soup/ResourceHandleSoup.cpp:
(WebCore::ResourceHandle::create): Modified to take a boolean as to whether to enable content encoding sniffing.
(WebCore::ResourceHandle::ResourceHandle): Ditto.
(WebCore::ResourceHandle::releaseForDownload): Pass content encoding policy.
* xml/XMLHttpRequest.cpp:
(WebCore::XMLHttpRequest::createRequest): Do not enable content encoding sniffing for the request.

Source/WebCore/PAL:

Forward declare CFNetwork SPI.

* pal/spi/cf/CFNetworkSPI.h:

Source/WebKit:

Fixes an issue where the body of an HTTP response associated with an XHR request to a .gz file
would be automatically gzipped decompressed if the HTTP response omitted a Content-Encoding HTTP
header. Specifically, such a response would be treated analogous to a response with headers
"Content-Type: application/gzip" and "Content-Encoding: identity". This behavior does not conform
the HTTP 1.1 spec. and breaks Epic Zen Garden, <https://s3.amazonaws.com/mozilla-games/ZenGarden/EpicZenGarden.html>.

On macOS 10.13.2 opt out of content encoding sniffing when making an XHR request. We likely can
selectively opt out of content encoding sniffing for other network requests. This will be done
in subsequent commits to make it straightforward to identify site breakage (if any).

* NetworkProcess/Downloads/Download.cpp:
(WebKit::Download::start): Enable content encoding sniff to match existing behavior.
(WebKit::Download::startWithHandle): Ditto.
* NetworkProcess/NetworkDataTask.cpp:
(WebKit::NetworkDataTask::create): Pass through the content encoding sniffing policy.
* NetworkProcess/NetworkLoad.cpp:
(WebKit::NetworkLoad::NetworkLoad): Ditto.
* NetworkProcess/NetworkLoadParameters.h:
* NetworkProcess/NetworkResourceLoadParameters.cpp:
(WebKit::NetworkResourceLoadParameters::encode const): Encode content encoding sniffing policy.
(WebKit::NetworkResourceLoadParameters::decode): Decode content encoding sniffing policy.
* NetworkProcess/cache/NetworkCacheSpeculativeLoad.cpp:
(WebKit::NetworkCache::SpeculativeLoad::SpeculativeLoad): Enable content encoding sniff to match existing
behavior. We should look to disable content encoding sniffing in a subsequent change.
* NetworkProcess/cocoa/NetworkDataTaskCocoa.h:
* NetworkProcess/cocoa/NetworkDataTaskCocoa.mm:
(WebKit::applySniffingPoliciesAndBindRequestToInferfaceIfNeeded): Added helper function
to apply sniffing policies and bind request to interface, if applicable.
(WebKit::NetworkDataTaskCocoa::NetworkDataTaskCocoa): Modified to take the content encoding sniffing
policy. Calls applySniffingPoliciesAndBindRequestToInferfaceIfNeeded() to apply this policy. Also use
convenience function URL::isLocalFile() to determine if the URL is a file URL.
* NetworkProcess/soup/NetworkDataTaskSoup.cpp:
(WebKit::NetworkDataTaskSoup::NetworkDataTaskSoup): Pass through the content encoding sniffing policy.
* NetworkProcess/soup/NetworkDataTaskSoup.h:
* WebProcess/Network/WebLoaderStrategy.cpp:
(WebKit::WebLoaderStrategy::scheduleLoadFromNetworkProcess): Pass through the content encoding sniffing policy.
(WebKit::WebLoaderStrategy::loadResourceSynchronously): Enable content encoding sniff to match existing
behavior.

LayoutTests:

Fix up the test http/tests/xmlhttprequest/gzip-content-type-no-content-encoding.html to
actually check that we do not sniff content encoding. CFNetwork only sniffs the URL
for the content encoding when the filename ends with known file extension (e.g. .gz)
and the HTTP response does not specifying a Content-Encoding HTTP header.

* TestExpectations: Skip the test on all platforms. Once <rdar://problem/33822249> ships
then we will enable the test on all platforms that use CFNetwork.
* http/tests/xmlhttprequest/gzip-content-type-no-content-encoding-expected.txt:
* http/tests/xmlhttprequest/gzip-content-type-no-content-encoding.html:
* http/tests/xmlhttprequest/resources/.htaccess: Interpret files with .php.gz extension as PHP files.
* http/tests/xmlhttprequest/resources/gzip-lorem-no-content-encoding.php.gz: Renamed from LayoutTests/http/tests/xmlhttprequest/resources/gzip-lorem-no-content-encoding.php.
* platform/mac/TestExpectations: Mark the test as flaky (Pass Failure) until <rdar://problem/33822249> ships.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@224299 268f45cc-cd09-0410-ab3c-d52691b4dbfc

35 files changed:
LayoutTests/ChangeLog
LayoutTests/TestExpectations
LayoutTests/http/tests/xmlhttprequest/gzip-content-type-no-content-encoding-expected.txt
LayoutTests/http/tests/xmlhttprequest/gzip-content-type-no-content-encoding.html
LayoutTests/http/tests/xmlhttprequest/resources/.htaccess
LayoutTests/http/tests/xmlhttprequest/resources/gzip-lorem-no-content-encoding.php.gz [moved from LayoutTests/http/tests/xmlhttprequest/resources/gzip-lorem-no-content-encoding.php with 100% similarity]
LayoutTests/platform/mac/TestExpectations
Source/WebCore/ChangeLog
Source/WebCore/PAL/ChangeLog
Source/WebCore/PAL/pal/spi/cf/CFNetworkSPI.h
Source/WebCore/loader/ResourceLoader.cpp
Source/WebCore/loader/ResourceLoader.h
Source/WebCore/loader/ResourceLoaderOptions.h
Source/WebCore/loader/appcache/ApplicationCacheGroup.cpp
Source/WebCore/platform/network/BlobResourceHandle.cpp
Source/WebCore/platform/network/PingHandle.h
Source/WebCore/platform/network/ResourceHandle.cpp
Source/WebCore/platform/network/ResourceHandle.h
Source/WebCore/platform/network/ResourceHandleInternal.h
Source/WebCore/platform/network/cf/ResourceHandleCFNet.cpp
Source/WebCore/platform/network/mac/ResourceHandleMac.mm
Source/WebCore/platform/network/soup/ResourceHandleSoup.cpp
Source/WebCore/xml/XMLHttpRequest.cpp
Source/WebKit/ChangeLog
Source/WebKit/NetworkProcess/Downloads/Download.cpp
Source/WebKit/NetworkProcess/NetworkDataTask.cpp
Source/WebKit/NetworkProcess/NetworkLoad.cpp
Source/WebKit/NetworkProcess/NetworkLoadParameters.h
Source/WebKit/NetworkProcess/NetworkResourceLoadParameters.cpp
Source/WebKit/NetworkProcess/cache/NetworkCacheSpeculativeLoad.cpp
Source/WebKit/NetworkProcess/cocoa/NetworkDataTaskCocoa.h
Source/WebKit/NetworkProcess/cocoa/NetworkDataTaskCocoa.mm
Source/WebKit/NetworkProcess/soup/NetworkDataTaskSoup.cpp
Source/WebKit/NetworkProcess/soup/NetworkDataTaskSoup.h
Source/WebKit/WebProcess/Network/WebLoaderStrategy.cpp

index e99d513..ab028e8 100644 (file)
@@ -1,3 +1,24 @@
+2017-11-01  Daniel Bates  <dabates@apple.com>
+
+        XMLHttpRequest should not sniff content encoding
+        https://bugs.webkit.org/show_bug.cgi?id=175597
+        <rdar://problem/34912624>
+
+        Reviewed by Alex Christensen.
+
+        Fix up the test http/tests/xmlhttprequest/gzip-content-type-no-content-encoding.html to
+        actually check that we do not sniff content encoding. CFNetwork only sniffs the URL
+        for the content encoding when the filename ends with known file extension (e.g. .gz)
+        and the HTTP response does not specifying a Content-Encoding HTTP header.
+
+        * TestExpectations: Skip the test on all platforms. Once <rdar://problem/33822249> ships
+        then we will enable the test on all platforms that use CFNetwork.
+        * http/tests/xmlhttprequest/gzip-content-type-no-content-encoding-expected.txt:
+        * http/tests/xmlhttprequest/gzip-content-type-no-content-encoding.html:
+        * http/tests/xmlhttprequest/resources/.htaccess: Interpret files with .php.gz extension as PHP files.
+        * http/tests/xmlhttprequest/resources/gzip-lorem-no-content-encoding.php.gz: Renamed from LayoutTests/http/tests/xmlhttprequest/resources/gzip-lorem-no-content-encoding.php.
+        * platform/mac/TestExpectations: Mark the test as flaky (Pass Failure) until <rdar://problem/33822249> ships.
+
 2017-11-01  Chris Dumez  <cdumez@apple.com>
 
         Mark a couple of service worker tests as flaky.
index 9b34492..9c3fe3d 100644 (file)
@@ -256,6 +256,9 @@ imported/w3c/web-platform-tests/fetch/api/abort/general.any.worker.html [ Skip ]
 # Not supported
 imported/w3c/web-platform-tests/background-fetch [ Skip ]
 
+# Content encoding sniffing is only supported by CFNetwork.
+http/tests/xmlhttprequest/gzip-content-type-no-content-encoding.html [ Skip ]
+
 #//////////////////////////////////////////////////////////////////////////////////////////
 # End platform-specific tests.
 #//////////////////////////////////////////////////////////////////////////////////////////
@@ -1553,8 +1556,6 @@ webkit.org/b/175288 imported/w3c/web-platform-tests/css/css-ui-3/outline-016.htm
 webkit.org/b/175288 imported/w3c/web-platform-tests/css/css-ui-3/outline-019.html [ ImageOnlyFailure ]
 webkit.org/b/175290 imported/w3c/web-platform-tests/css/css-ui-3/text-overflow-005.html [ ImageOnlyFailure ]
 
-webkit.org/b/175597 http/tests/xmlhttprequest/gzip-content-type-no-content-encoding.html [ Failure ]
-
 webkit.org/b/171945 perf/class-list-remove.html [ Pass Failure Timeout ]
 
 webkit.org/b/176183 fast/block/inside-inlines/opacity-on-inline.html [ ImageOnlyFailure ]
index 86d51df..74f91a8 100644 (file)
@@ -1,7 +1,7 @@
 Test that an XMLHttpRequest with Content-Type: gzip is properly uncompressed.
 
 Content Type: application/octet-stream
-Content Encoding: null
+Content Encoding: gzip
 Content Length: 282
 RESPONSE 200: [object ArrayBuffer]
 LENGTH: 445
index 8ae5549..7926287 100644 (file)
@@ -16,7 +16,7 @@ const log = text => {
 };
 
 const xhr = new XMLHttpRequest();
-xhr.open('GET', 'resources/gzip-lorem-no-content-encoding.php');
+xhr.open('GET', 'resources/gzip-lorem-no-content-encoding.php.gz');
 xhr.responseType = 'arraybuffer';
 xhr.onerror = e => log(`FAILED: ${xhr.status} ${xhr.statusText}`);
 xhr.onreadystatechange = () => {
index 9277940..0252607 100644 (file)
@@ -10,3 +10,5 @@ AddCharset windows-1251 .xml
 <Files "noContentType.asis">
 DefaultType None
 </Files>
+
+AddType application/x-httpd-php .php.gz
index 292fda5..447d903 100644 (file)
@@ -33,6 +33,9 @@ fast/xmlhttprequest/set-dangerous-headers-in-dashboard.html [ Pass ]
 
 http/tests/gzip-content-encoding [ Pass ]
 
+# FIXME: Mark as Pass once the fix for <rdar://problem/33822249> ships.
+http/tests/xmlhttprequest/gzip-content-type-no-content-encoding.html [ Pass Failure ]
+
 #//////////////////////////////////////////////////////////////////////////////////////////
 # End platform-specific directories.
 #//////////////////////////////////////////////////////////////////////////////////////////
index 94dc436..0c4e376 100644 (file)
@@ -1,3 +1,59 @@
+2017-11-01  Daniel Bates  <dabates@apple.com>
+
+        XMLHttpRequest should not sniff content encoding
+        https://bugs.webkit.org/show_bug.cgi?id=175597
+        <rdar://problem/34912624>
+
+        Reviewed by Alex Christensen.
+
+        Fixes an issue where the body of an HTTP response associated with an XHR request to a .gz file
+        would be automatically gzipped decompressed if the HTTP response omitted a Content-Encoding HTTP
+        header. Specifically, such a response would be treated analogous to a response with headers
+        "Content-Type: application/gzip" and "Content-Encoding: identity". This behavior does not conform
+        the HTTP 1.1 spec. and breaks Epic Zen Garden, <https://s3.amazonaws.com/mozilla-games/ZenGarden/EpicZenGarden.html>.
+
+        On macOS 10.13.2 opt out of content encoding sniffing when making an XHR request. We likely can
+        selectively opt out of content encoding sniffing for other network requests. This will be done
+        in subsequent commits to make it straightforward to identify site breakage (if any).
+
+        * loader/ResourceLoader.cpp:
+        (WebCore::ResourceLoader::start): Pass content encoding policy.
+        * loader/ResourceLoader.h:
+        (WebCore::ResourceLoader::shouldSniffContentEncoding const): Added.
+        * loader/ResourceLoaderOptions.h:
+        * loader/appcache/ApplicationCacheGroup.cpp:
+        (WebCore::ApplicationCacheGroup::createResourceHandle): Enable content encoding sniff to match existing behavior.
+        * platform/network/BlobResourceHandle.cpp:
+        (WebCore::BlobResourceHandle::BlobResourceHandle): Ditto. We should look to disable content encoding sniffing in
+        a subsequent change.
+        * platform/network/PingHandle.h: Ditto.
+        * platform/network/ResourceHandle.cpp:
+        (WebCore::ResourceHandle::ResourceHandle): Modified to take a boolean as to whether to enable content encoding sniffing.
+        (WebCore::ResourceHandle::create): Ditto.
+        (WebCore::ResourceHandle::shouldContentEncodingSniff const): Added.
+        * platform/network/ResourceHandle.h:
+        * platform/network/ResourceHandleInternal.h:
+        (WebCore::ResourceHandleInternal::ResourceHandleInternal): Modified to take a boolean as to whether to enable content
+        encoding sniffing.
+        * platform/network/cf/ResourceHandleCFNet.cpp:
+        (WebCore::ResourceHandle::createCFURLConnection): Modified to take a boolean as to whether to enable content encoding
+        sniffing and apply this policy to the CFMutableURLRequestRef object when building on macOS 10.13.2.
+        (WebCore::ResourceHandle::start):
+        (WebCore::ResourceHandle::platformLoadResourceSynchronously): Enable content encoding sniff to match existing behavior.
+        * platform/network/mac/ResourceHandleMac.mm:
+        (WebCore::ResourceHandle::applySniffingPoliciesAndStoragePartitionIfNeeded): Added helper function to apply sniffing policies
+        and storage partition, if applicable. 
+        (WebCore::ResourceHandle::createNSURLConnection): Modified to take a boolean as to whether to enable content encoding
+        sniffing. Calls adjustNSRequestApplyingPolicies() to apply this policy.
+        (WebCore::ResourceHandle::start):
+        (WebCore::ResourceHandle::platformLoadResourceSynchronously): Enable content encoding sniff to match existing behavior.
+        * platform/network/soup/ResourceHandleSoup.cpp:
+        (WebCore::ResourceHandle::create): Modified to take a boolean as to whether to enable content encoding sniffing.
+        (WebCore::ResourceHandle::ResourceHandle): Ditto.
+        (WebCore::ResourceHandle::releaseForDownload): Pass content encoding policy.
+        * xml/XMLHttpRequest.cpp:
+        (WebCore::XMLHttpRequest::createRequest): Do not enable content encoding sniffing for the request.
+
 2017-11-01  Jer Noble  <jer.noble@apple.com>
 
         [Performance] Painting <video> to canvas spends a lot of time in URL getting and parsing
index c7ca871..b714646 100644 (file)
@@ -1,3 +1,15 @@
+2017-11-01  Daniel Bates  <dabates@apple.com>
+
+        XMLHttpRequest should not sniff content encoding
+        https://bugs.webkit.org/show_bug.cgi?id=175597
+        <rdar://problem/34912624>
+
+        Reviewed by Alex Christensen.
+
+        Forward declare CFNetwork SPI.
+
+        * pal/spi/cf/CFNetworkSPI.h:
+
 2017-10-31  Tim Horton  <timothy_horton@apple.com>
 
         Clean up some drag and drop feature flags
index df39b8b..0acc89f 100644 (file)
@@ -208,6 +208,10 @@ extern const CFStringRef _kCFURLCachePartitionKey;
 extern const CFStringRef _kCFURLConnectionPropertyShouldSniff;
 extern const CFStringRef _kCFURLStorageSessionIsPrivate;
 
+#if PLATFORM(MAC) && __MAC_OS_X_VERSION_MIN_REQUIRED >= 101302
+extern const CFStringRef kCFURLRequestContentDecoderSkipURLCheck;
+#endif
+
 CFHTTPCookieStorageRef _CFHTTPCookieStorageGetDefault(CFAllocatorRef);
 CFHTTPCookieStorageRef CFHTTPCookieStorageCreateFromFile(CFAllocatorRef, CFURLRef, CFHTTPCookieStorageRef);
 void CFHTTPCookieStorageScheduleWithRunLoop(CFHTTPCookieStorageRef, CFRunLoopRef, CFStringRef);
index 39d64ea..36949d6 100644 (file)
@@ -216,7 +216,7 @@ void ResourceLoader::start()
         return;
     }
 
-    m_handle = ResourceHandle::create(frameLoader()->networkingContext(), m_request, this, m_defersLoading, m_options.sniffContent == SniffContent);
+    m_handle = ResourceHandle::create(frameLoader()->networkingContext(), m_request, this, m_defersLoading, m_options.sniffContent == SniffContent, m_options.sniffContentEncoding == ContentEncodingSniffingPolicy::Sniff);
 }
 
 void ResourceLoader::setDefersLoading(bool defers)
index b672d0b..9cacbb8 100644 (file)
@@ -123,6 +123,7 @@ public:
     bool shouldSendResourceLoadCallbacks() const { return m_options.sendLoadCallbacks == SendCallbacks; }
     void setSendCallbackPolicy(SendCallbackPolicy sendLoadCallbacks) { m_options.sendLoadCallbacks = sendLoadCallbacks; }
     bool shouldSniffContent() const { return m_options.sniffContent == SniffContent; }
+    bool shouldSniffContentEncoding() const { return m_options.sniffContentEncoding == ContentEncodingSniffingPolicy::Sniff; }
     WEBCORE_EXPORT bool isAllowedToAskUserForCredentials() const;
     bool shouldIncludeCertificateInfo() const { return m_options.certificateInfoPolicy == IncludeCertificateInfo; }
 
index 5c5a0d4..6574b37 100644 (file)
@@ -97,6 +97,11 @@ enum class ServiceWorkersMode {
     None,
 };
 
+enum class ContentEncodingSniffingPolicy {
+    Sniff,
+    DoNotSniff,
+};
+
 struct ResourceLoaderOptions : public FetchOptions {
     ResourceLoaderOptions() { }
 
@@ -120,6 +125,7 @@ struct ResourceLoaderOptions : public FetchOptions {
 
     SendCallbackPolicy sendLoadCallbacks { DoNotSendCallbacks };
     ContentSniffingPolicy sniffContent { DoNotSniffContent };
+    ContentEncodingSniffingPolicy sniffContentEncoding { ContentEncodingSniffingPolicy::Sniff };
     DataBufferingPolicy dataBufferingPolicy { BufferData };
     StoredCredentialsPolicy storedCredentialsPolicy { StoredCredentialsPolicy::DoNotUse };
     SecurityCheckPolicy securityCheck { DoSecurityCheck };
index 113c687..d813a8a 100644 (file)
@@ -475,7 +475,10 @@ RefPtr<ResourceHandle> ApplicationCacheGroup::createResourceHandle(const URL& ur
         }
     }
 
-    RefPtr<ResourceHandle> handle = ResourceHandle::create(m_frame->loader().networkingContext(), request, this, false, true);
+    bool defersLoading = false;
+    bool shouldContentSniff = true;
+    bool shouldContentEncodingSniff = true;
+    RefPtr<ResourceHandle> handle = ResourceHandle::create(m_frame->loader().networkingContext(), request, this, defersLoading, shouldContentSniff, shouldContentEncodingSniff);
 
     // Because willSendRequest only gets called during redirects, we initialize
     // the identifier and the first willSendRequest here.
index 035eb49..7b78937 100644 (file)
@@ -153,9 +153,9 @@ void BlobResourceHandle::loadResourceSynchronously(BlobData* blobData, const Res
 }
 
 BlobResourceHandle::BlobResourceHandle(BlobData* blobData, const ResourceRequest& request, ResourceHandleClient* client, bool async)
-    : ResourceHandle(nullptr, request, client, false, false)
-    , m_blobData(blobData)
-    , m_async(async)
+    : ResourceHandle { nullptr, request, client, false /* defersLoading */, false /* shouldContentSniff */, true /* shouldContentEncodingSniff */ }
+    , m_blobData { blobData }
+    , m_async { async }
 {
     if (m_async)
         m_asyncStream = std::make_unique<AsyncFileStream>(*this);
index 3e91da2..5640162 100644 (file)
@@ -46,7 +46,10 @@ public:
         , m_shouldFollowRedirects(shouldFollowRedirects)
         , m_completionHandler(WTFMove(completionHandler))
     {
-        m_handle = ResourceHandle::create(networkingContext, request, this, false, false);
+        bool defersLoading = false;
+        bool shouldContentSniff = false;
+        bool shouldContentEncodingSniff = true;
+        m_handle = ResourceHandle::create(networkingContext, request, this, defersLoading, shouldContentSniff, shouldContentEncodingSniff);
 
         // If the server never responds, this object will hang around forever.
         // Set a very generous timeout, just in case.
index e6aa508..a92e1a6 100644 (file)
@@ -72,8 +72,8 @@ void ResourceHandle::registerBuiltinSynchronousLoader(const AtomicString& protoc
     builtinResourceHandleSynchronousLoaderMap().add(protocol, loader);
 }
 
-ResourceHandle::ResourceHandle(NetworkingContext* context, const ResourceRequest& request, ResourceHandleClient* client, bool defersLoading, bool shouldContentSniff)
-    : d(std::make_unique<ResourceHandleInternal>(this, context, request, client, defersLoading, shouldContentSniff && shouldContentSniffURL(request.url())))
+ResourceHandle::ResourceHandle(NetworkingContext* context, const ResourceRequest& request, ResourceHandleClient* client, bool defersLoading, bool shouldContentSniff, bool shouldContentEncodingSniff)
+    : d(std::make_unique<ResourceHandleInternal>(this, context, request, client, defersLoading, shouldContentSniff && shouldContentSniffURL(request.url()), shouldContentEncodingSniff))
 {
     if (!request.url().isValid()) {
         scheduleFailure(InvalidURLFailure);
@@ -86,12 +86,12 @@ ResourceHandle::ResourceHandle(NetworkingContext* context, const ResourceRequest
     }
 }
 
-RefPtr<ResourceHandle> ResourceHandle::create(NetworkingContext* context, const ResourceRequest& request, ResourceHandleClient* client, bool defersLoading, bool shouldContentSniff)
+RefPtr<ResourceHandle> ResourceHandle::create(NetworkingContext* context, const ResourceRequest& request, ResourceHandleClient* client, bool defersLoading, bool shouldContentSniff, bool shouldContentEncodingSniff)
 {
     if (auto constructor = builtinResourceHandleConstructorMap().get(request.url().protocol().toStringWithoutCopying()))
         return constructor(request, client);
 
-    auto newHandle = adoptRef(*new ResourceHandle(context, request, client, defersLoading, shouldContentSniff));
+    auto newHandle = adoptRef(*new ResourceHandle(context, request, client, defersLoading, shouldContentSniff, shouldContentEncodingSniff));
 
     if (newHandle->d->m_scheduledFailureType != NoFailure)
         return WTFMove(newHandle);
@@ -206,6 +206,11 @@ bool ResourceHandle::shouldContentSniff() const
     return d->m_shouldContentSniff;
 }
 
+bool ResourceHandle::shouldContentEncodingSniff() const
+{
+    return d->m_shouldContentEncodingSniff;
+}
+
 bool ResourceHandle::shouldContentSniffURL(const URL& url)
 {
 #if PLATFORM(COCOA)
index cc196d8..b3f6a4f 100644 (file)
@@ -52,6 +52,7 @@ OBJC_CLASS NSData;
 OBJC_CLASS NSDictionary;
 OBJC_CLASS NSError;
 OBJC_CLASS NSURLConnection;
+OBJC_CLASS NSURLRequest;
 #ifndef __OBJC__
 typedef struct objc_object *id;
 #endif
@@ -93,11 +94,11 @@ class Timer;
 
 class ResourceHandle : public RefCounted<ResourceHandle>, public AuthenticationClient {
 public:
-    WEBCORE_EXPORT static RefPtr<ResourceHandle> create(NetworkingContext*, const ResourceRequest&, ResourceHandleClient*, bool defersLoading, bool shouldContentSniff);
+    WEBCORE_EXPORT static RefPtr<ResourceHandle> create(NetworkingContext*, const ResourceRequest&, ResourceHandleClient*, bool defersLoading, bool shouldContentSniff, bool shouldContentEncodingSniff);
     WEBCORE_EXPORT static void loadResourceSynchronously(NetworkingContext*, const ResourceRequest&, StoredCredentialsPolicy, ResourceError&, ResourceResponse&, Vector<char>& data);
 
 #if USE(SOUP)
-    static RefPtr<ResourceHandle> create(SoupNetworkSession&, const ResourceRequest&, ResourceHandleClient*, bool defersLoading, bool shouldContentSniff);
+    static RefPtr<ResourceHandle> create(SoupNetworkSession&, const ResourceRequest&, ResourceHandleClient*, bool defersLoading, bool shouldContentSniff, bool shouldContentEncodingSniff);
 #endif
 
     WEBCORE_EXPORT virtual ~ResourceHandle();
@@ -165,6 +166,8 @@ public:
     bool shouldContentSniff() const;
     static bool shouldContentSniffURL(const URL&);
 
+    bool shouldContentEncodingSniff() const;
+
     WEBCORE_EXPORT static void forceContentSniffing();
 
 #if USE(CURL) || USE(SOUP)
@@ -236,7 +239,7 @@ public:
     static void registerBuiltinSynchronousLoader(const AtomicString& protocol, BuiltinSynchronousLoader);
 
 protected:
-    ResourceHandle(NetworkingContext*, const ResourceRequest&, ResourceHandleClient*, bool defersLoading, bool shouldContentSniff);
+    ResourceHandle(NetworkingContext*, const ResourceRequest&, ResourceHandleClient*, bool defersLoading, bool shouldContentSniff, bool shouldContentEncodingSniff);
 
 private:
     enum FailureType {
@@ -246,7 +249,7 @@ private:
     };
 
 #if USE(SOUP)
-    ResourceHandle(SoupNetworkSession&, const ResourceRequest&, ResourceHandleClient*, bool defersLoading, bool shouldContentSniff);
+    ResourceHandle(SoupNetworkSession&, const ResourceRequest&, ResourceHandleClient*, bool defersLoading, bool shouldContentSniff, bool shouldContentEncodingSniff);
 #endif
 
     void platformSetDefersLoading(bool);
@@ -266,15 +269,19 @@ private:
 #endif
 
 #if USE(CFURLCONNECTION)
-    void createCFURLConnection(bool shouldUseCredentialStorage, bool shouldContentSniff, WTF::MessageQueue<WTF::Function<void()>>*, CFDictionaryRef clientProperties);
+    void createCFURLConnection(bool shouldUseCredentialStorage, bool shouldContentSniff, bool shouldContentEncodingSniff, WTF::MessageQueue<WTF::Function<void()>>*, CFDictionaryRef clientProperties);
 #endif
 
 #if PLATFORM(MAC) && !USE(CFURLCONNECTION)
-    void createNSURLConnection(id delegate, bool shouldUseCredentialStorage, bool shouldContentSniff, SchedulingBehavior);
+    void createNSURLConnection(id delegate, bool shouldUseCredentialStorage, bool shouldContentSniff, bool shouldContentEncodingSniff, SchedulingBehavior);
 #endif
 
 #if PLATFORM(IOS) && !USE(CFURLCONNECTION)
-    void createNSURLConnection(id delegate, bool shouldUseCredentialStorage, bool shouldContentSniff, SchedulingBehavior, NSDictionary *connectionProperties);
+    void createNSURLConnection(id delegate, bool shouldUseCredentialStorage, bool shouldContentSniff, bool shouldContentEncodingSniff, SchedulingBehavior, NSDictionary *connectionProperties);
+#endif
+
+#if PLATFORM(COCOA)
+    void applySniffingPoliciesAndStoragePartitionIfNeeded(NSURLRequest*&, bool shouldContentSniff, bool shouldContentEncodingSniff);
 #endif
 
 #if USE(SOUP)
index fbb6cad..eea7908 100644 (file)
@@ -72,7 +72,7 @@ namespace WebCore {
 class ResourceHandleInternal {
     WTF_MAKE_NONCOPYABLE(ResourceHandleInternal); WTF_MAKE_FAST_ALLOCATED;
 public:
-    ResourceHandleInternal(ResourceHandle* loader, NetworkingContext* context, const ResourceRequest& request, ResourceHandleClient* client, bool defersLoading, bool shouldContentSniff)
+    ResourceHandleInternal(ResourceHandle* loader, NetworkingContext* context, const ResourceRequest& request, ResourceHandleClient* client, bool defersLoading, bool shouldContentSniff, bool shouldContentEncodingSniff)
         : m_context(context)
         , m_client(client)
         , m_firstRequest(request)
@@ -80,6 +80,7 @@ public:
         , m_partition(request.cachePartition())
         , m_defersLoading(defersLoading)
         , m_shouldContentSniff(shouldContentSniff)
+        , m_shouldContentEncodingSniff(shouldContentEncodingSniff)
 #if USE(CFURLCONNECTION)
         , m_currentRequest(request)
 #endif
@@ -114,6 +115,7 @@ public:
 
     bool m_defersLoading;
     bool m_shouldContentSniff;
+    bool m_shouldContentEncodingSniff;
 #if USE(CFURLCONNECTION)
     RetainPtr<CFURLConnectionRef> m_connection;
     ResourceRequest m_currentRequest;
index 7a6c2ff..e5c8f31 100644 (file)
@@ -126,7 +126,7 @@ static inline CFStringRef shouldSniffConnectionProperty()
 #endif
 }
 
-void ResourceHandle::createCFURLConnection(bool shouldUseCredentialStorage, bool shouldContentSniff, MessageQueue<Function<void()>>* messageQueue, CFDictionaryRef clientProperties)
+void ResourceHandle::createCFURLConnection(bool shouldUseCredentialStorage, bool shouldContentSniff, bool shouldContentEncodingSniff, MessageQueue<Function<void()>>* messageQueue, CFDictionaryRef clientProperties)
 {
     if ((!d->m_user.isEmpty() || !d->m_pass.isEmpty()) && !firstRequest().url().protocolIsInHTTPFamily()) {
         // Credentials for ftp can only be passed in URL, the didReceiveAuthenticationChallenge delegate call won't be made.
@@ -161,7 +161,14 @@ void ResourceHandle::createCFURLConnection(bool shouldUseCredentialStorage, bool
     auto request = adoptCF(CFURLRequestCreateMutableCopy(kCFAllocatorDefault, firstRequest().cfURLRequest(UpdateHTTPBody)));
     if (auto storageSession = d->m_storageSession.get())
         _CFURLRequestSetStorageSession(request.get(), storageSession);
-    
+
+#if PLATFORM(MAC) && __MAC_OS_X_VERSION_MIN_REQUIRED >= 101302
+    if (!shouldContentEncodingSniff)
+        _CFURLRequestSetProtocolProperty(request.get(), kCFURLRequestContentDecoderSkipURLCheck, kCFBooleanTrue);
+#else
+    UNUSED_PARAM(shouldContentEncodingSniff);
+#endif
+
     if (!shouldContentSniff)
         _CFURLRequestSetProtocolProperty(request.get(), shouldSniffConnectionProperty(), kCFBooleanFalse);
 
@@ -259,7 +266,7 @@ bool ResourceHandle::start()
     setCollectsTimingData();
 #endif
 
-    createCFURLConnection(shouldUseCredentialStorage, d->m_shouldContentSniff, nullptr, client()->connectionProperties(this).get());
+    createCFURLConnection(shouldUseCredentialStorage, d->m_shouldContentSniff, d->m_shouldContentEncodingSniff, nullptr, client()->connectionProperties(this).get());
     ref();
 
     d->m_connectionDelegate->setupConnectionScheduling(d->m_connection.get());
@@ -554,7 +561,10 @@ void ResourceHandle::platformLoadResourceSynchronously(NetworkingContext* contex
     SynchronousLoaderClient client;
     client.setAllowStoredCredentials(storedCredentialsPolicy == StoredCredentialsPolicy::Use);
 
-    RefPtr<ResourceHandle> handle = adoptRef(new ResourceHandle(context, request, &client, false /*defersLoading*/, true /*shouldContentSniff*/));
+    bool defersLoading = false;
+    bool shouldContentSniff = true;
+    bool shouldContentEncodingSniff = true;
+    RefPtr<ResourceHandle> handle = adoptRef(new ResourceHandle(context, request, &client, defersLoading, shouldContentSniff, shouldContentEncodingSniff));
 
     handle->d->m_storageSession = context->storageSession().platformSession();
 
@@ -564,7 +574,7 @@ void ResourceHandle::platformLoadResourceSynchronously(NetworkingContext* contex
     }
 
     handle->ref();
-    handle->createCFURLConnection(storedCredentialsPolicy == StoredCredentialsPolicy::Use, ResourceHandle::shouldContentSniffURL(request.url()), &client.messageQueue(), handle->client()->connectionProperties(handle.get()).get());
+    handle->createCFURLConnection(storedCredentialsPolicy == StoredCredentialsPolicy::Use, ResourceHandle::shouldContentSniffURL(request.url()), handle->shouldContentEncodingSniff(), &client.messageQueue(), handle->client()->connectionProperties(handle.get()).get());
 
     static CFRunLoopRef runLoop = nullptr;
     if (!runLoop) {
index 06826f1..e9478b1 100644 (file)
@@ -122,10 +122,46 @@ static bool synchronousWillSendRequestEnabled()
 }
 #endif
 
+#if PLATFORM(COCOA)
+void ResourceHandle::applySniffingPoliciesAndStoragePartitionIfNeeded(NSURLRequest*& nsRequest, bool shouldContentSniff, bool shouldContentEncodingSniff)
+{
+#if !PLATFORM(MAC)
+    UNUSED_PARAM(shouldContentEncodingSniff);
+#elif __MAC_OS_X_VERSION_MIN_REQUIRED < 101302
+    shouldContentEncodingSniff = true;
+#endif
+#if HAVE(CFNETWORK_STORAGE_PARTITIONING)
+    String storagePartition = d->m_context->storageSession().cookieStoragePartition(firstRequest());
+#else
+    String storagePartition;
+#endif
+    if (shouldContentSniff && shouldContentEncodingSniff && storagePartition.isEmpty())
+        return;
+
+    auto mutableRequest = adoptNS([nsRequest mutableCopy]);
+
+#if PLATFORM(MAC) && __MAC_OS_X_VERSION_MIN_REQUIRED >= 101302
+    if (!shouldContentEncodingSniff)
+        [mutableRequest _setProperty:@(YES) forKey:(NSString *)kCFURLRequestContentDecoderSkipURLCheck];
+#endif
+
+    if (!shouldContentSniff)
+        [mutableRequest _setProperty:@(NO) forKey:(NSString *)_kCFURLConnectionPropertyShouldSniff];
+
+#if HAVE(CFNETWORK_STORAGE_PARTITIONING)
+    if (!storagePartition.isEmpty())
+        [mutableRequest _setProperty:storagePartition forKey:@"__STORAGE_PARTITION_IDENTIFIER"];
+#endif
+
+    nsRequest = mutableRequest.autorelease();
+
+}
+#endif
+
 #if !PLATFORM(IOS)
-void ResourceHandle::createNSURLConnection(id delegate, bool shouldUseCredentialStorage, bool shouldContentSniff, SchedulingBehavior schedulingBehavior)
+void ResourceHandle::createNSURLConnection(id delegate, bool shouldUseCredentialStorage, bool shouldContentSniff, bool shouldContentEncodingSniff, SchedulingBehavior schedulingBehavior)
 #else
-void ResourceHandle::createNSURLConnection(id delegate, bool shouldUseCredentialStorage, bool shouldContentSniff, SchedulingBehavior schedulingBehavior, NSDictionary *connectionProperties)
+void ResourceHandle::createNSURLConnection(id delegate, bool shouldUseCredentialStorage, bool shouldContentSniff, bool shouldContentEncodingSniff, SchedulingBehavior schedulingBehavior, NSDictionary *connectionProperties)
 #endif
 {
 #if !HAVE(TIMINGDATAOPTIONS)
@@ -159,20 +195,7 @@ void ResourceHandle::createNSURLConnection(id delegate, bool shouldUseCredential
     }
 
     NSURLRequest *nsRequest = firstRequest().nsURLRequest(UpdateHTTPBody);
-    if (!shouldContentSniff) {
-        NSMutableURLRequest *mutableRequest = [[nsRequest mutableCopy] autorelease];
-        [mutableRequest _setProperty:@(NO) forKey:(NSString *)_kCFURLConnectionPropertyShouldSniff];
-        nsRequest = mutableRequest;
-    }
-
-#if HAVE(CFNETWORK_STORAGE_PARTITIONING)
-    String storagePartition = d->m_context->storageSession().cookieStoragePartition(firstRequest());
-    if (!storagePartition.isEmpty()) {
-        NSMutableURLRequest *mutableRequest = [[nsRequest mutableCopy] autorelease];
-        [mutableRequest _setProperty:storagePartition forKey:@"__STORAGE_PARTITION_IDENTIFIER"];
-        nsRequest = mutableRequest;
-    }
-#endif
+    applySniffingPoliciesAndStoragePartitionIfNeeded(nsRequest, shouldContentSniff, shouldContentEncodingSniff);
 
     if (d->m_storageSession)
         nsRequest = [copyRequestWithStorageSession(d->m_storageSession.get(), nsRequest) autorelease];
@@ -250,12 +273,14 @@ bool ResourceHandle::start()
         ResourceHandle::makeDelegate(shouldUseCredentialStorage, nullptr),
         shouldUseCredentialStorage,
         d->m_shouldContentSniff || d->m_context->localFileContentSniffingEnabled(),
+        d->m_shouldContentEncodingSniff,
         schedulingBehavior);
 #else
     createNSURLConnection(
         ResourceHandle::makeDelegate(shouldUseCredentialStorage, nullptr),
         shouldUseCredentialStorage,
         d->m_shouldContentSniff || d->m_context->localFileContentSniffingEnabled(),
+        d->m_shouldContentEncodingSniff,
         schedulingBehavior,
         (NSDictionary *)client()->connectionProperties(this).get());
 #endif
@@ -362,7 +387,10 @@ void ResourceHandle::platformLoadResourceSynchronously(NetworkingContext* contex
     SynchronousLoaderClient client;
     client.setAllowStoredCredentials(storedCredentialsPolicy == StoredCredentialsPolicy::Use);
 
-    RefPtr<ResourceHandle> handle = adoptRef(new ResourceHandle(context, request, &client, false /*defersLoading*/, true /*shouldContentSniff*/));
+    bool defersLoading = false;
+    bool shouldContentSniff = true;
+    bool shouldContentEncodingSniff = true;
+    RefPtr<ResourceHandle> handle = adoptRef(new ResourceHandle(context, request, &client, defersLoading, shouldContentSniff, shouldContentEncodingSniff));
 
     handle->d->m_storageSession = context->storageSession().platformSession();
 
@@ -377,12 +405,14 @@ void ResourceHandle::platformLoadResourceSynchronously(NetworkingContext* contex
         handle->makeDelegate(shouldUseCredentialStorage, &client.messageQueue()),
         shouldUseCredentialStorage,
         handle->shouldContentSniff() || context->localFileContentSniffingEnabled(),
+        handle->shouldContentEncodingSniff(),
         SchedulingBehavior::Synchronous);
 #else
     handle->createNSURLConnection(
         handle->makeDelegate(shouldUseCredentialStorage, &client.messageQueue()), // A synchronous request cannot turn into a download, so there is no need to proxy the delegate.
         shouldUseCredentialStorage,
         handle->shouldContentSniff() || (context && context->localFileContentSniffingEnabled()),
+        handle->shouldContentEncodingSniff(),
         SchedulingBehavior::Synchronous,
         (NSDictionary *)handle->client()->connectionProperties(handle.get()).get());
 #endif
index 348870d..026ff12 100644 (file)
@@ -90,9 +90,9 @@ SoupSession* ResourceHandleInternal::soupSession()
     return m_session ? m_session->soupSession() : sessionFromContext(m_context.get());
 }
 
-RefPtr<ResourceHandle> ResourceHandle::create(SoupNetworkSession& session, const ResourceRequest& request, ResourceHandleClient* client, bool defersLoading, bool shouldContentSniff)
+RefPtr<ResourceHandle> ResourceHandle::create(SoupNetworkSession& session, const ResourceRequest& request, ResourceHandleClient* client, bool defersLoading, bool shouldContentSniff, bool shouldContentEncodingSniff)
 {
-    auto newHandle = adoptRef(*new ResourceHandle(session, request, client, defersLoading, shouldContentSniff));
+    auto newHandle = adoptRef(*new ResourceHandle(session, request, client, defersLoading, shouldContentSniff, shouldContentEncodingSniff));
 
     if (newHandle->d->m_scheduledFailureType != NoFailure)
         return WTFMove(newHandle);
@@ -103,8 +103,8 @@ RefPtr<ResourceHandle> ResourceHandle::create(SoupNetworkSession& session, const
     return nullptr;
 }
 
-ResourceHandle::ResourceHandle(SoupNetworkSession& session, const ResourceRequest& request, ResourceHandleClient* client, bool defersLoading, bool shouldContentSniff)
-    : d(std::make_unique<ResourceHandleInternal>(this, nullptr, request, client, defersLoading, shouldContentSniff && shouldContentSniffURL(request.url())))
+ResourceHandle::ResourceHandle(SoupNetworkSession& session, const ResourceRequest& request, ResourceHandleClient* client, bool defersLoading, bool shouldContentSniff, bool shouldContentEncodingSniff)
+    : d(std::make_unique<ResourceHandleInternal>(this, nullptr, request, client, defersLoading, shouldContentSniff && shouldContentSniffURL(request.url()), shouldContentEncodingSniff))
 {
     if (!request.url().isValid()) {
         scheduleFailure(InvalidURLFailure);
@@ -732,7 +732,7 @@ bool ResourceHandle::start()
 RefPtr<ResourceHandle> ResourceHandle::releaseForDownload(ResourceHandleClient* downloadClient)
 {
     // We don't adopt the ref, as it will be released by cleanupSoupRequestOperation, which should always run.
-    ResourceHandle* newHandle = new ResourceHandle(d->m_context.get(), firstRequest(), nullptr, d->m_defersLoading, d->m_shouldContentSniff);
+    ResourceHandle* newHandle = new ResourceHandle(d->m_context.get(), firstRequest(), nullptr, d->m_defersLoading, d->m_shouldContentSniff, d->m_shouldContentEncodingSniff);
     newHandle->relaxAdoptionRequirement();
     std::swap(d, newHandle->d);
 
index 79d8c38..096f055 100644 (file)
@@ -613,6 +613,7 @@ ExceptionOr<void> XMLHttpRequest::createRequest()
     options.initiator = cachedResourceRequestInitiators().xmlhttprequest;
     options.sameOriginDataURLFlag = SameOriginDataURLFlag::Set;
     options.filteringPolicy = ResponseFilteringPolicy::Enable;
+    options.sniffContentEncoding = ContentEncodingSniffingPolicy::DoNotSniff;
 
     if (m_timeoutMilliseconds) {
         if (!m_async)
index f30952d..031b446 100644 (file)
@@ -1,3 +1,50 @@
+2017-11-01  Daniel Bates  <dabates@apple.com>
+
+        XMLHttpRequest should not sniff content encoding
+        https://bugs.webkit.org/show_bug.cgi?id=175597
+        <rdar://problem/34912624>
+
+        Reviewed by Alex Christensen.
+
+        Fixes an issue where the body of an HTTP response associated with an XHR request to a .gz file
+        would be automatically gzipped decompressed if the HTTP response omitted a Content-Encoding HTTP
+        header. Specifically, such a response would be treated analogous to a response with headers
+        "Content-Type: application/gzip" and "Content-Encoding: identity". This behavior does not conform
+        the HTTP 1.1 spec. and breaks Epic Zen Garden, <https://s3.amazonaws.com/mozilla-games/ZenGarden/EpicZenGarden.html>.
+
+        On macOS 10.13.2 opt out of content encoding sniffing when making an XHR request. We likely can
+        selectively opt out of content encoding sniffing for other network requests. This will be done
+        in subsequent commits to make it straightforward to identify site breakage (if any).
+
+        * NetworkProcess/Downloads/Download.cpp:
+        (WebKit::Download::start): Enable content encoding sniff to match existing behavior.
+        (WebKit::Download::startWithHandle): Ditto.
+        * NetworkProcess/NetworkDataTask.cpp:
+        (WebKit::NetworkDataTask::create): Pass through the content encoding sniffing policy.
+        * NetworkProcess/NetworkLoad.cpp:
+        (WebKit::NetworkLoad::NetworkLoad): Ditto.
+        * NetworkProcess/NetworkLoadParameters.h:
+        * NetworkProcess/NetworkResourceLoadParameters.cpp:
+        (WebKit::NetworkResourceLoadParameters::encode const): Encode content encoding sniffing policy.
+        (WebKit::NetworkResourceLoadParameters::decode): Decode content encoding sniffing policy.
+        * NetworkProcess/cache/NetworkCacheSpeculativeLoad.cpp:
+        (WebKit::NetworkCache::SpeculativeLoad::SpeculativeLoad): Enable content encoding sniff to match existing
+        behavior. We should look to disable content encoding sniffing in a subsequent change.
+        * NetworkProcess/cocoa/NetworkDataTaskCocoa.h:
+        * NetworkProcess/cocoa/NetworkDataTaskCocoa.mm:
+        (WebKit::applySniffingPoliciesAndBindRequestToInferfaceIfNeeded): Added helper function
+        to apply sniffing policies and bind request to interface, if applicable.
+        (WebKit::NetworkDataTaskCocoa::NetworkDataTaskCocoa): Modified to take the content encoding sniffing
+        policy. Calls applySniffingPoliciesAndBindRequestToInferfaceIfNeeded() to apply this policy. Also use
+        convenience function URL::isLocalFile() to determine if the URL is a file URL.
+        * NetworkProcess/soup/NetworkDataTaskSoup.cpp:
+        (WebKit::NetworkDataTaskSoup::NetworkDataTaskSoup): Pass through the content encoding sniffing policy.
+        * NetworkProcess/soup/NetworkDataTaskSoup.h:
+        * WebProcess/Network/WebLoaderStrategy.cpp:
+        (WebKit::WebLoaderStrategy::scheduleLoadFromNetworkProcess): Pass through the content encoding sniffing policy.
+        (WebKit::WebLoaderStrategy::loadResourceSynchronously): Enable content encoding sniff to match existing
+        behavior.
+
 2017-11-01  Brady Eidson  <beidson@apple.com>
 
         Plumbing for handling SW scripts failing to evaluate
index 1eafeed..f63adaf 100644 (file)
@@ -112,7 +112,10 @@ void Download::start()
 {
     if (m_request.url().protocolIsBlob()) {
         m_downloadClient = std::make_unique<BlobDownloadClient>(*this);
-        m_resourceHandle = ResourceHandle::create(nullptr, m_request, m_downloadClient.get(), false, false);
+        bool defersLoading = false;
+        bool shouldContentSniff = false;
+        bool shouldContentEncodingSniff = true;
+        m_resourceHandle = ResourceHandle::create(nullptr, m_request, m_downloadClient.get(), defersLoading, shouldContentSniff, shouldContentEncodingSniff);
         didStart();
         return;
     }
@@ -124,7 +127,10 @@ void Download::startWithHandle(ResourceHandle* handle, const ResourceResponse& r
 {
     if (m_request.url().protocolIsBlob()) {
         m_downloadClient = std::make_unique<BlobDownloadClient>(*this);
-        m_resourceHandle = ResourceHandle::create(nullptr, m_request, m_downloadClient.get(), false, false);
+        bool defersLoading = false;
+        bool shouldContentSniff = false;
+        bool shouldContentEncodingSniff = true;
+        m_resourceHandle = ResourceHandle::create(nullptr, m_request, m_downloadClient.get(), defersLoading, shouldContentSniff, shouldContentEncodingSniff);
         didStart();
         return;
     }
index bb7f175..b891879 100644 (file)
@@ -53,10 +53,10 @@ Ref<NetworkDataTask> NetworkDataTask::create(NetworkSession& session, NetworkDat
         return NetworkDataTaskBlob::create(session, client, parameters.request, parameters.contentSniffingPolicy, parameters.blobFileReferences);
 
 #if PLATFORM(COCOA)
-    return NetworkDataTaskCocoa::create(session, client, parameters.request, parameters.storedCredentialsPolicy, parameters.contentSniffingPolicy, parameters.shouldClearReferrerOnHTTPSToHTTPRedirect, parameters.shouldPreconnectOnly);
+    return NetworkDataTaskCocoa::create(session, client, parameters.request, parameters.storedCredentialsPolicy, parameters.contentSniffingPolicy, parameters.contentEncodingSniffingPolicy, parameters.shouldClearReferrerOnHTTPSToHTTPRedirect, parameters.shouldPreconnectOnly);
 #endif
 #if USE(SOUP)
-    return NetworkDataTaskSoup::create(session, client, parameters.request, parameters.storedCredentialsPolicy, parameters.contentSniffingPolicy, parameters.shouldClearReferrerOnHTTPSToHTTPRedirect);
+    return NetworkDataTaskSoup::create(session, client, parameters.request, parameters.storedCredentialsPolicy, parameters.contentSniffingPolicy, parameters.contentEncodingSniffingPolicy, parameters.shouldClearReferrerOnHTTPSToHTTPRedirect);
 #endif
 }
 
index 447d8a1..9f51d90 100644 (file)
@@ -125,7 +125,7 @@ NetworkLoad::NetworkLoad(NetworkLoadClient& client, NetworkLoadParameters&& para
     , m_networkingContext(RemoteNetworkingContext::create(m_parameters.sessionID, m_parameters.shouldClearReferrerOnHTTPSToHTTPRedirect))
     , m_currentRequest(m_parameters.request)
 {
-    m_handle = ResourceHandle::create(m_networkingContext.get(), m_parameters.request, this, m_parameters.defersLoading, m_parameters.contentSniffingPolicy == SniffContent);
+    m_handle = ResourceHandle::create(m_networkingContext.get(), m_parameters.request, this, m_parameters.defersLoading, m_parameters.contentSniffingPolicy == SniffContent, m_parameters.contentEncodingSniffingPolicy == ContentEncodingSniffingPolicy::Sniff);
 }
 
 #endif
index 25aa505..8327897 100644 (file)
@@ -42,6 +42,7 @@ public:
     PAL::SessionID sessionID { PAL::SessionID::emptySessionID() };
     WebCore::ResourceRequest request;
     WebCore::ContentSniffingPolicy contentSniffingPolicy { WebCore::SniffContent };
+    WebCore::ContentEncodingSniffingPolicy contentEncodingSniffingPolicy { WebCore::ContentEncodingSniffingPolicy::Sniff };
     WebCore::StoredCredentialsPolicy storedCredentialsPolicy { WebCore::StoredCredentialsPolicy::DoNotUse };
     WebCore::ClientCredentialPolicy clientCredentialPolicy { WebCore::ClientCredentialPolicy::CannotAskClientForCredentials };
     bool shouldFollowRedirects { true };
index e963576..cd21657 100644 (file)
@@ -74,6 +74,7 @@ void NetworkResourceLoadParameters::encode(IPC::Encoder& encoder) const
     }
 
     encoder.encodeEnum(contentSniffingPolicy);
+    encoder.encodeEnum(contentEncodingSniffingPolicy);
     encoder.encodeEnum(storedCredentialsPolicy);
     encoder.encodeEnum(clientCredentialPolicy);
     encoder.encodeEnum(shouldPreconnectOnly);
@@ -141,6 +142,8 @@ bool NetworkResourceLoadParameters::decode(IPC::Decoder& decoder, NetworkResourc
 
     if (!decoder.decodeEnum(result.contentSniffingPolicy))
         return false;
+    if (!decoder.decodeEnum(result.contentEncodingSniffingPolicy))
+        return false;
     if (!decoder.decodeEnum(result.storedCredentialsPolicy))
         return false;
     if (!decoder.decodeEnum(result.clientCredentialPolicy))
index 6913965..af63670 100644 (file)
@@ -56,6 +56,7 @@ SpeculativeLoad::SpeculativeLoad(Cache& cache, const GlobalFrameID& frameID, con
     parameters.sessionID = PAL::SessionID::defaultSessionID();
     parameters.storedCredentialsPolicy = StoredCredentialsPolicy::Use;
     parameters.contentSniffingPolicy = DoNotSniffContent;
+    parameters.contentEncodingSniffingPolicy = ContentEncodingSniffingPolicy::Sniff;
     parameters.request = m_originalRequest;
 #if USE(NETWORK_SESSION)
     m_networkLoad = std::make_unique<NetworkLoad>(*this, WTFMove(parameters), *SessionTracker::networkSession(PAL::SessionID::defaultSessionID()));
index 75fc97d..eb26364 100644 (file)
@@ -41,9 +41,9 @@ class NetworkSessionCocoa;
 class NetworkDataTaskCocoa final : public NetworkDataTask {
     friend class NetworkSessionCocoa;
 public:
-    static Ref<NetworkDataTask> create(NetworkSession& session, NetworkDataTaskClient& client, const WebCore::ResourceRequest& request, WebCore::StoredCredentialsPolicy storedCredentialsPolicy, WebCore::ContentSniffingPolicy shouldContentSniff, bool shouldClearReferrerOnHTTPSToHTTPRedirect, PreconnectOnly shouldPreconnectOnly)
+    static Ref<NetworkDataTask> create(NetworkSession& session, NetworkDataTaskClient& client, const WebCore::ResourceRequest& request, WebCore::StoredCredentialsPolicy storedCredentialsPolicy, WebCore::ContentSniffingPolicy shouldContentSniff, WebCore::ContentEncodingSniffingPolicy shouldContentEncodingSniff, bool shouldClearReferrerOnHTTPSToHTTPRedirect, PreconnectOnly shouldPreconnectOnly)
     {
-        return adoptRef(*new NetworkDataTaskCocoa(session, client, request, storedCredentialsPolicy, shouldContentSniff, shouldClearReferrerOnHTTPSToHTTPRedirect, shouldPreconnectOnly));
+        return adoptRef(*new NetworkDataTaskCocoa(session, client, request, storedCredentialsPolicy, shouldContentSniff, shouldContentEncodingSniff, shouldClearReferrerOnHTTPSToHTTPRedirect, shouldPreconnectOnly));
     }
 
     ~NetworkDataTaskCocoa();
@@ -70,9 +70,10 @@ public:
     WebCore::NetworkLoadMetrics& networkLoadMetrics() { return m_networkLoadMetrics; }
 
 private:
-    NetworkDataTaskCocoa(NetworkSession&, NetworkDataTaskClient&, const WebCore::ResourceRequest&, WebCore::StoredCredentialsPolicy, WebCore::ContentSniffingPolicy, bool shouldClearReferrerOnHTTPSToHTTPRedirect, PreconnectOnly);
+    NetworkDataTaskCocoa(NetworkSession&, NetworkDataTaskClient&, const WebCore::ResourceRequest&, WebCore::StoredCredentialsPolicy, WebCore::ContentSniffingPolicy, WebCore::ContentEncodingSniffingPolicy, bool shouldClearReferrerOnHTTPSToHTTPRedirect, PreconnectOnly);
 
     bool tryPasswordBasedAuthentication(const WebCore::AuthenticationChallenge&, ChallengeCompletionHandler&);
+    void applySniffingPoliciesAndBindRequestToInferfaceIfNeeded(NSURLRequest*&, bool shouldContentSniff, bool shouldContentEncodingSniff);
 
     RefPtr<SandboxExtension> m_sandboxExtension;
     RetainPtr<NSURLSessionDataTask> m_task;
index df64e39..eb53f6c 100644 (file)
@@ -74,7 +74,34 @@ static float toNSURLSessionTaskPriority(WebCore::ResourceLoadPriority priority)
     return NSURLSessionTaskPriorityDefault;
 }
 
-NetworkDataTaskCocoa::NetworkDataTaskCocoa(NetworkSession& session, NetworkDataTaskClient& client, const WebCore::ResourceRequest& requestWithCredentials, WebCore::StoredCredentialsPolicy storedCredentialsPolicy, WebCore::ContentSniffingPolicy shouldContentSniff, bool shouldClearReferrerOnHTTPSToHTTPRedirect, PreconnectOnly shouldPreconnectOnly)
+void NetworkDataTaskCocoa::applySniffingPoliciesAndBindRequestToInferfaceIfNeeded(NSURLRequest*& nsRequest, bool shouldContentSniff, bool shouldContentEncodingSniff)
+{
+#if !PLATFORM(MAC)
+    UNUSED_PARAM(shouldContentEncodingSniff);
+#elif __MAC_OS_X_VERSION_MIN_REQUIRED < 101302
+    shouldContentEncodingSniff = true;
+#endif
+    auto& cocoaSession = static_cast<NetworkSessionCocoa&>(m_session.get());
+    if (shouldContentSniff && shouldContentEncodingSniff && cocoaSession.m_boundInterfaceIdentifier.isNull())
+        return;
+
+    auto mutableRequest = adoptNS([nsRequest mutableCopy]);
+
+#if PLATFORM(MAC) && __MAC_OS_X_VERSION_MIN_REQUIRED >= 101302
+    if (!shouldContentEncodingSniff)
+        [mutableRequest _setProperty:@(YES) forKey:(NSString *)kCFURLRequestContentDecoderSkipURLCheck];
+#endif
+
+    if (!shouldContentSniff)
+        [mutableRequest _setProperty:@(NO) forKey:(NSString *)_kCFURLConnectionPropertyShouldSniff];
+
+    if (!cocoaSession.m_boundInterfaceIdentifier.isNull())
+        [mutableRequest setBoundInterfaceIdentifier:cocoaSession.m_boundInterfaceIdentifier];
+
+    nsRequest = mutableRequest.autorelease();
+}
+
+NetworkDataTaskCocoa::NetworkDataTaskCocoa(NetworkSession& session, NetworkDataTaskClient& client, const WebCore::ResourceRequest& requestWithCredentials, WebCore::StoredCredentialsPolicy storedCredentialsPolicy, WebCore::ContentSniffingPolicy shouldContentSniff, WebCore::ContentEncodingSniffingPolicy shouldContentEncodingSniff, bool shouldClearReferrerOnHTTPSToHTTPRedirect, PreconnectOnly shouldPreconnectOnly)
     : NetworkDataTask(session, client, requestWithCredentials, storedCredentialsPolicy, shouldClearReferrerOnHTTPSToHTTPRedirect)
 {
     if (m_scheduledFailureType != NoFailure)
@@ -104,24 +131,14 @@ NetworkDataTaskCocoa::NetworkDataTaskCocoa(NetworkSession& session, NetworkDataT
 #endif
     
     NSURLRequest *nsRequest = request.nsURLRequest(WebCore::UpdateHTTPBody);
-    if (shouldContentSniff == WebCore::DoNotSniffContent || url.protocolIs("file")) {
-        NSMutableURLRequest *mutableRequest = [[nsRequest mutableCopy] autorelease];
-        [mutableRequest _setProperty:@(NO) forKey:(NSString *)_kCFURLConnectionPropertyShouldSniff];
-        nsRequest = mutableRequest;
-    }
+    applySniffingPoliciesAndBindRequestToInferfaceIfNeeded(nsRequest, shouldContentSniff == WebCore::SniffContent && !url.isLocalFile(), shouldContentEncodingSniff == WebCore::ContentEncodingSniffingPolicy::Sniff);
 
-    auto& cocoaSession = static_cast<NetworkSessionCocoa&>(m_session.get());
     if (session.networkStorageSession().shouldBlockCookies(request)) {
         storedCredentialsPolicy = WebCore::StoredCredentialsPolicy::DoNotUse;
         m_storedCredentialsPolicy = WebCore::StoredCredentialsPolicy::DoNotUse;
     }
 
-    if (!cocoaSession.m_boundInterfaceIdentifier.isNull()) {
-        NSMutableURLRequest *mutableRequest = [[nsRequest mutableCopy] autorelease];
-        [mutableRequest setBoundInterfaceIdentifier:cocoaSession.m_boundInterfaceIdentifier];
-        nsRequest = mutableRequest;
-    }
-
+    auto& cocoaSession = static_cast<NetworkSessionCocoa&>(m_session.get());
     if (storedCredentialsPolicy == WebCore::StoredCredentialsPolicy::Use) {
         m_task = [cocoaSession.m_sessionWithCredentialStorage dataTaskWithRequest:nsRequest];
         ASSERT(!cocoaSession.m_dataTaskMapWithCredentials.contains([m_task taskIdentifier]));
index e72c1f6..4a7e916 100644 (file)
@@ -49,7 +49,7 @@ namespace WebKit {
 
 static const size_t gDefaultReadBufferSize = 8192;
 
-NetworkDataTaskSoup::NetworkDataTaskSoup(NetworkSession& session, NetworkDataTaskClient& client, const ResourceRequest& requestWithCredentials, StoredCredentialsPolicy storedCredentialsPolicy, ContentSniffingPolicy shouldContentSniff, bool shouldClearReferrerOnHTTPSToHTTPRedirect)
+NetworkDataTaskSoup::NetworkDataTaskSoup(NetworkSession& session, NetworkDataTaskClient& client, const ResourceRequest& requestWithCredentials, StoredCredentialsPolicy storedCredentialsPolicy, ContentSniffingPolicy shouldContentSniff, WebCore::ContentEncodingSniffingPolicy, bool shouldClearReferrerOnHTTPSToHTTPRedirect)
     : NetworkDataTask(session, client, requestWithCredentials, storedCredentialsPolicy, shouldClearReferrerOnHTTPSToHTTPRedirect)
     , m_shouldContentSniff(shouldContentSniff)
     , m_timeoutSource(RunLoop::main(), this, &NetworkDataTaskSoup::timeoutFired)
index 390cd48..379a7f0 100644 (file)
@@ -36,15 +36,15 @@ namespace WebKit {
 
 class NetworkDataTaskSoup final : public NetworkDataTask {
 public:
-    static Ref<NetworkDataTask> create(NetworkSession& session, NetworkDataTaskClient& client, const WebCore::ResourceRequest& request, WebCore::StoredCredentialsPolicy storedCredentialsPolicy, WebCore::ContentSniffingPolicy shouldContentSniff, bool shouldClearReferrerOnHTTPSToHTTPRedirect)
+    static Ref<NetworkDataTask> create(NetworkSession& session, NetworkDataTaskClient& client, const WebCore::ResourceRequest& request, WebCore::StoredCredentialsPolicy storedCredentialsPolicy, WebCore::ContentSniffingPolicy shouldContentSniff, WebCore::ContentEncodingSniffingPolicy shouldContentEncodingSniff, bool shouldClearReferrerOnHTTPSToHTTPRedirect)
     {
-        return adoptRef(*new NetworkDataTaskSoup(session, client, request, storedCredentialsPolicy, shouldContentSniff, shouldClearReferrerOnHTTPSToHTTPRedirect));
+        return adoptRef(*new NetworkDataTaskSoup(session, client, request, storedCredentialsPolicy, shouldContentSniff, shouldContentEncodingSniff, shouldClearReferrerOnHTTPSToHTTPRedirect));
     }
 
     ~NetworkDataTaskSoup();
 
 private:
-    NetworkDataTaskSoup(NetworkSession&, NetworkDataTaskClient&, const WebCore::ResourceRequest&, WebCore::StoredCredentialsPolicy, WebCore::ContentSniffingPolicy, bool shouldClearReferrerOnHTTPSToHTTPRedirect);
+    NetworkDataTaskSoup(NetworkSession&, NetworkDataTaskClient&, const WebCore::ResourceRequest&, WebCore::StoredCredentialsPolicy, WebCore::ContentSniffingPolicy, WebCore::ContentEncodingSniffingPolicy, bool shouldClearReferrerOnHTTPSToHTTPRedirect);
 
     void suspend() override;
     void cancel() override;
index 85afa27..e25bc79 100644 (file)
@@ -237,6 +237,7 @@ void WebLoaderStrategy::scheduleLoadFromNetworkProcess(ResourceLoader& resourceL
     LOG(NetworkScheduling, "(WebProcess) WebLoaderStrategy::scheduleLoad, url '%s' will be scheduled with the NetworkProcess with priority %d", resourceLoader.url().string().latin1().data(), static_cast<int>(resourceLoader.request().priority()));
 
     ContentSniffingPolicy contentSniffingPolicy = resourceLoader.shouldSniffContent() ? SniffContent : DoNotSniffContent;
+    ContentEncodingSniffingPolicy contentEncodingSniffingPolicy = resourceLoader.shouldSniffContentEncoding() ? ContentEncodingSniffingPolicy::Sniff : ContentEncodingSniffingPolicy::DoNotSniff;
     StoredCredentialsPolicy storedCredentialsPolicy = resourceLoader.shouldUseCredentialStorage() ? StoredCredentialsPolicy::Use : StoredCredentialsPolicy::DoNotUse;
 
     NetworkResourceLoadParameters loadParameters;
@@ -246,6 +247,7 @@ void WebLoaderStrategy::scheduleLoadFromNetworkProcess(ResourceLoader& resourceL
     loadParameters.sessionID = sessionID;
     loadParameters.request = request;
     loadParameters.contentSniffingPolicy = contentSniffingPolicy;
+    loadParameters.contentEncodingSniffingPolicy = contentEncodingSniffingPolicy;
     loadParameters.storedCredentialsPolicy = storedCredentialsPolicy;
     // If there is no WebFrame then this resource cannot be authenticated with the client.
     loadParameters.clientCredentialPolicy = (loadParameters.webFrameID && loadParameters.webPageID && resourceLoader.isAllowedToAskUserForCredentials()) ? ClientCredentialPolicy::MayAskClientForCredentials : ClientCredentialPolicy::CannotAskClientForCredentials;
@@ -400,6 +402,7 @@ void WebLoaderStrategy::loadResourceSynchronously(NetworkingContext* context, un
     loadParameters.sessionID = webPage ? webPage->sessionID() : PAL::SessionID::defaultSessionID();
     loadParameters.request = request;
     loadParameters.contentSniffingPolicy = SniffContent;
+    loadParameters.contentEncodingSniffingPolicy = ContentEncodingSniffingPolicy::Sniff;
     loadParameters.storedCredentialsPolicy = storedCredentialsPolicy;
     loadParameters.clientCredentialPolicy = clientCredentialPolicy;
     loadParameters.shouldClearReferrerOnHTTPSToHTTPRedirect = context->shouldClearReferrerOnHTTPSToHTTPRedirect();