Speculative revalidation requests do not have their HTTP user-agent set
authorcdumez@apple.com <cdumez@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 18 Mar 2016 23:23:09 +0000 (23:23 +0000)
committercdumez@apple.com <cdumez@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 18 Mar 2016 23:23:09 +0000 (23:23 +0000)
https://bugs.webkit.org/show_bug.cgi?id=155620
<rdar://problem/24657567>

Reviewed by Brady Eidson.

Source/WebCore:

Export a couple of symbols so they can be used from WebKit2.

Test: http/tests/cache/disk-cache/speculative-validation/validation-request.html

* platform/network/ResourceRequestBase.h:

Source/WebKit2:

Speculative revalidation requests do not have their HTTP user-agent set
which could lead to different content getting returned by the server.

To address the problem, we save the original request's 'user-agent' in
the disk cache and re-use it later on for speculative validation
requests.

* NetworkProcess/cache/NetworkCacheSpeculativeLoadManager.cpp:
(WebKit::NetworkCache::constructRevalidationRequest):
(WebKit::NetworkCache::SpeculativeLoadManager::revalidateEntry):
(WebKit::NetworkCache::SpeculativeLoadManager::preloadEntry):
* NetworkProcess/cache/NetworkCacheSubresourcesEntry.cpp:
(WebKit::NetworkCache::SubresourceInfo::encode):
(WebKit::NetworkCache::SubresourceInfo::decode):
(WebKit::NetworkCache::SubresourceInfo::SubresourceInfo):
(WebKit::NetworkCache::SubresourceInfo::operator=):
(WebKit::NetworkCache::SubresourcesEntry::SubresourcesEntry):
(WebKit::NetworkCache::SubresourcesEntry::updateSubresourceLoads):
(WebKit::NetworkCache::SubresourcesEntry::decodeStorageRecord): Deleted.
* NetworkProcess/cache/NetworkCacheSubresourcesEntry.h:
(WebKit::NetworkCache::SubresourceInfo::SubresourceInfo):

LayoutTests:

Add layout test coverage.

* http/tests/cache/disk-cache/speculative-validation/resources/request-headers-script.php: Added.
* http/tests/cache/disk-cache/speculative-validation/resources/validation-request-frame.php: Added.
* http/tests/cache/disk-cache/speculative-validation/validation-request-expected.txt: Added.
* http/tests/cache/disk-cache/speculative-validation/validation-request.html: Added.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@198450 268f45cc-cd09-0410-ab3c-d52691b4dbfc

LayoutTests/ChangeLog
LayoutTests/http/tests/cache/disk-cache/speculative-validation/resources/request-headers-script.php [new file with mode: 0644]
LayoutTests/http/tests/cache/disk-cache/speculative-validation/resources/validation-request-frame.php [new file with mode: 0644]
LayoutTests/http/tests/cache/disk-cache/speculative-validation/validation-request-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/cache/disk-cache/speculative-validation/validation-request.html [new file with mode: 0644]
Source/WebCore/ChangeLog
Source/WebCore/platform/network/ResourceRequestBase.h
Source/WebKit2/ChangeLog
Source/WebKit2/NetworkProcess/cache/NetworkCacheSpeculativeLoadManager.cpp
Source/WebKit2/NetworkProcess/cache/NetworkCacheSubresourcesEntry.cpp
Source/WebKit2/NetworkProcess/cache/NetworkCacheSubresourcesEntry.h

index 57b2d9b..392d907 100644 (file)
@@ -1,3 +1,18 @@
+2016-03-18  Chris Dumez  <cdumez@apple.com>
+
+        Speculative revalidation requests do not have their HTTP user-agent set
+        https://bugs.webkit.org/show_bug.cgi?id=155620
+        <rdar://problem/24657567>
+
+        Reviewed by Brady Eidson.
+
+        Add layout test coverage.
+
+        * http/tests/cache/disk-cache/speculative-validation/resources/request-headers-script.php: Added.
+        * http/tests/cache/disk-cache/speculative-validation/resources/validation-request-frame.php: Added.
+        * http/tests/cache/disk-cache/speculative-validation/validation-request-expected.txt: Added.
+        * http/tests/cache/disk-cache/speculative-validation/validation-request.html: Added.
+
 2016-03-18  Mark Lam  <mark.lam@apple.com>
 
         ES6 spec requires that RegExpPrototype not be a RegExp object.
diff --git a/LayoutTests/http/tests/cache/disk-cache/speculative-validation/resources/request-headers-script.php b/LayoutTests/http/tests/cache/disk-cache/speculative-validation/resources/request-headers-script.php
new file mode 100644 (file)
index 0000000..8054868
--- /dev/null
@@ -0,0 +1,12 @@
+<?php
+header('Content-Type: text/javascript');
+header('Cache-Control: max-age=0');
+header('Etag: 123456789');
+
+?>
+allRequestHeaders = [];
+<?php
+foreach (getallheaders() as $name => $value) {
+    echo "allRequestHeaders['" . $name . "'] = '" . $value . "';";
+}
+?>
diff --git a/LayoutTests/http/tests/cache/disk-cache/speculative-validation/resources/validation-request-frame.php b/LayoutTests/http/tests/cache/disk-cache/speculative-validation/resources/validation-request-frame.php
new file mode 100644 (file)
index 0000000..d88c966
--- /dev/null
@@ -0,0 +1,10 @@
+<?php
+header('Content-Type: text/html');
+header('Cache-Control: max-age=0');
+header('Etag: 123456789');
+
+?>
+<!DOCTYPE html>
+<body>
+<script src="request-headers-script.php"></script>
+</body>
diff --git a/LayoutTests/http/tests/cache/disk-cache/speculative-validation/validation-request-expected.txt b/LayoutTests/http/tests/cache/disk-cache/speculative-validation/validation-request-expected.txt
new file mode 100644 (file)
index 0000000..98b4aec
--- /dev/null
@@ -0,0 +1,11 @@
+Validates HTTP headers of speculative validation requests.
+
+On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
+
+
+PASS validationRequestHeader('If-None-Match') is "123456789"
+PASS validationRequestHeader('User-Agent') is initialUserAgent
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
diff --git a/LayoutTests/http/tests/cache/disk-cache/speculative-validation/validation-request.html b/LayoutTests/http/tests/cache/disk-cache/speculative-validation/validation-request.html
new file mode 100644 (file)
index 0000000..ad84b62
--- /dev/null
@@ -0,0 +1,41 @@
+<script src="/js-test-resources/js-test-pre.js"></script>
+<script>
+description("Validates HTTP headers of speculative validation requests.");
+jsTestIsAsync = true;
+
+state = "warmup";
+
+function validationRequestHeader(headerName)
+{
+    return frames[0].allRequestHeaders[headerName];
+    
+}
+
+function frameLoaded()
+{
+    if (state == "warmup") {
+        initialUserAgent = validationRequestHeader('User-Agent');
+        // Navigate frame to 'about:blank' to flush to subresource loads metadata to disk.
+        state = "flushingMetadata";
+        document.getElementById("testFrame").src = "about:blank";
+        return;
+    }
+    if (state == "flushingMetadata") {
+        // Navigate frame to its original location again. This time it should speculatively
+        // validate subresources as we have subresource loads metadata in the disk cache.
+        state = "speculativeRevalidation";
+        document.getElementById("testFrame").src = "resources/validation-request-frame.php";
+        return;
+    }
+    if (state == "speculativeRevalidation") {
+        // Validate the HTTP headers of the speculative validation request.
+        shouldBeEqualToString("validationRequestHeader('If-None-Match')", "123456789");
+        shouldBe("validationRequestHeader('User-Agent')", "initialUserAgent");
+        finishJSTest();
+        return;
+    }
+}
+
+</script>
+<iframe id="testFrame" src="resources/validation-request-frame.php" onload="frameLoaded()"></iframe>
+<script src="/js-test-resources/js-test-post.js"></script>
index bfbd572..fbf13ce 100644 (file)
@@ -1,3 +1,17 @@
+2016-03-18  Chris Dumez  <cdumez@apple.com>
+
+        Speculative revalidation requests do not have their HTTP user-agent set
+        https://bugs.webkit.org/show_bug.cgi?id=155620
+        <rdar://problem/24657567>
+
+        Reviewed by Brady Eidson.
+
+        Export a couple of symbols so they can be used from WebKit2.
+
+        Test: http/tests/cache/disk-cache/speculative-validation/validation-request.html
+
+        * platform/network/ResourceRequestBase.h:
+
 2016-03-18  Zhuo Li  <zachli@apple.com>
 
         Need to forward declare NSScrollerImpSPI::scrollerLayoutDirection.
index 48fb24c..d01602d 100644 (file)
@@ -107,8 +107,8 @@ namespace WebCore {
         void setHTTPOrigin(const String&);
         WEBCORE_EXPORT void clearHTTPOrigin();
 
-        String httpUserAgent() const;
-        void setHTTPUserAgent(const String&);
+        WEBCORE_EXPORT String httpUserAgent() const;
+        WEBCORE_EXPORT void setHTTPUserAgent(const String&);
         void clearHTTPUserAgent();
 
         String httpAccept() const;
index a91feca..3858800 100644 (file)
@@ -1,3 +1,33 @@
+2016-03-18  Chris Dumez  <cdumez@apple.com>
+
+        Speculative revalidation requests do not have their HTTP user-agent set
+        https://bugs.webkit.org/show_bug.cgi?id=155620
+        <rdar://problem/24657567>
+
+        Reviewed by Brady Eidson.
+
+        Speculative revalidation requests do not have their HTTP user-agent set
+        which could lead to different content getting returned by the server.
+
+        To address the problem, we save the original request's 'user-agent' in
+        the disk cache and re-use it later on for speculative validation
+        requests.
+
+        * NetworkProcess/cache/NetworkCacheSpeculativeLoadManager.cpp:
+        (WebKit::NetworkCache::constructRevalidationRequest):
+        (WebKit::NetworkCache::SpeculativeLoadManager::revalidateEntry):
+        (WebKit::NetworkCache::SpeculativeLoadManager::preloadEntry):
+        * NetworkProcess/cache/NetworkCacheSubresourcesEntry.cpp:
+        (WebKit::NetworkCache::SubresourceInfo::encode):
+        (WebKit::NetworkCache::SubresourceInfo::decode):
+        (WebKit::NetworkCache::SubresourceInfo::SubresourceInfo):
+        (WebKit::NetworkCache::SubresourceInfo::operator=):
+        (WebKit::NetworkCache::SubresourcesEntry::SubresourcesEntry):
+        (WebKit::NetworkCache::SubresourcesEntry::updateSubresourceLoads):
+        (WebKit::NetworkCache::SubresourcesEntry::decodeStorageRecord): Deleted.
+        * NetworkProcess/cache/NetworkCacheSubresourcesEntry.h:
+        (WebKit::NetworkCache::SubresourceInfo::SubresourceInfo):
+
 2016-03-18  Enrica Casucci  <enrica@apple.com>
 
         Add id attribute to _WKActivatedElementInfo.
index 554fb11..f8f02fd 100644 (file)
@@ -84,10 +84,11 @@ static inline Key makeSubresourcesKey(const Key& resourceKey)
     return Key(resourceKey.partition(), subresourcesType(), resourceKey.range(), resourceKey.identifier());
 }
 
-static inline ResourceRequest constructRevalidationRequest(const Entry& entry, const URL& firstPartyForCookies)
+static inline ResourceRequest constructRevalidationRequest(const Entry& entry, const SubresourceInfo& subResourceInfo)
 {
     ResourceRequest revalidationRequest(entry.key().identifier());
-    revalidationRequest.setFirstPartyForCookies(firstPartyForCookies);
+    revalidationRequest.setHTTPUserAgent(subResourceInfo.httpUserAgent);
+    revalidationRequest.setFirstPartyForCookies(subResourceInfo.firstPartyForCookies);
 #if ENABLE(CACHE_PARTITIONING)
     if (entry.key().hasPartition())
         revalidationRequest.setCachePartition(entry.key().partition());
@@ -380,7 +381,7 @@ void SpeculativeLoadManager::revalidateEntry(std::unique_ptr<Entry> entry, const
     if (!key.range().isEmpty())
         return;
 
-    ResourceRequest revalidationRequest = constructRevalidationRequest(*entry, subresourceInfo.firstPartyForCookies);
+    ResourceRequest revalidationRequest = constructRevalidationRequest(*entry, subresourceInfo);
 
     LOG(NetworkCacheSpeculativePreloading, "(NetworkProcess) Speculatively revalidating '%s':", key.identifier().utf8().data());
     auto revalidator = std::make_unique<SpeculativeLoad>(frameID, revalidationRequest, WTFMove(entry), [this, key, frameID](std::unique_ptr<Entry> revalidatedEntry) {
@@ -405,8 +406,7 @@ void SpeculativeLoadManager::revalidateEntry(std::unique_ptr<Entry> entry, const
 void SpeculativeLoadManager::preloadEntry(const Key& key, const SubresourceInfo& subResourceInfo, const GlobalFrameID& frameID)
 {
     m_pendingPreloads.add(key, nullptr);
-    URLCapture firstPartyForCookies(subResourceInfo.firstPartyForCookies);
-    retrieveEntryFromStorage(key, [this, key, firstPartyForCookies, frameID](std::unique_ptr<Entry> entry) {
+    retrieveEntryFromStorage(key, [this, key, subResourceInfo, frameID](std::unique_ptr<Entry> entry) {
         m_pendingPreloads.remove(key);
 
         if (satisfyPendingRequests(key, entry.get())) {
@@ -419,7 +419,7 @@ void SpeculativeLoadManager::preloadEntry(const Key& key, const SubresourceInfo&
             return;
 
         if (entry->needsValidation())
-            revalidateEntry(WTFMove(entry), firstPartyForCookies.url(), frameID);
+            revalidateEntry(WTFMove(entry), subResourceInfo, frameID);
         else
             addPreloadedEntry(WTFMove(entry), frameID, WasRevalidated::No);
     });
index e003ca1..564b85d 100644 (file)
@@ -40,6 +40,7 @@ void SubresourceInfo::encode(Encoder& encoder) const
 {
     encoder << firstPartyForCookies;
     encoder << isTransient;
+    encoder << httpUserAgent;
 }
 
 bool SubresourceInfo::decode(Decoder& decoder, SubresourceInfo& info)
@@ -48,9 +49,26 @@ bool SubresourceInfo::decode(Decoder& decoder, SubresourceInfo& info)
         return false;
     if (!decoder.decode(info.isTransient))
         return false;
+    if (!decoder.decode(info.httpUserAgent))
+        return false;
     return true;
 }
 
+SubresourceInfo::SubresourceInfo(const SubresourceInfo& subresourceInfo)
+    : firstPartyForCookies(subresourceInfo.firstPartyForCookies.isolatedCopy())
+    , httpUserAgent(subresourceInfo.httpUserAgent.isolatedCopy())
+    , isTransient(subresourceInfo.isTransient)
+{
+}
+
+SubresourceInfo& SubresourceInfo::operator=(const SubresourceInfo& other)
+{
+    firstPartyForCookies = other.firstPartyForCookies.isolatedCopy();
+    httpUserAgent = other.httpUserAgent.isolatedCopy();
+    isTransient = other.isTransient;
+    return *this;
+}
+
 Storage::Record SubresourcesEntry::encodeAsStorageRecord() const
 {
     Encoder encoder;
@@ -90,7 +108,7 @@ SubresourcesEntry::SubresourcesEntry(Key&& key, const Vector<std::unique_ptr<Sub
 {
     ASSERT(m_key.type() == "subresources");
     for (auto& subresourceLoad : subresourceLoads)
-        m_subresources.add(subresourceLoad->key, SubresourceInfo(subresourceLoad->request.firstPartyForCookies()));
+        m_subresources.add(subresourceLoad->key, SubresourceInfo(subresourceLoad->request));
 }
 
 void SubresourcesEntry::updateSubresourceLoads(const Vector<std::unique_ptr<SubresourceLoad>>& subresourceLoads)
@@ -100,7 +118,7 @@ void SubresourcesEntry::updateSubresourceLoads(const Vector<std::unique_ptr<Subr
     // Mark keys that are common with last load as non-Transient.
     for (auto& subresourceLoad : subresourceLoads) {
         bool isTransient = !oldSubresources.contains(subresourceLoad->key);
-        m_subresources.add(subresourceLoad->key, SubresourceInfo(subresourceLoad->request.firstPartyForCookies(), isTransient));
+        m_subresources.add(subresourceLoad->key, SubresourceInfo(subresourceLoad->request, isTransient));
     }
 }
 
index b92d44a..c7b1c50 100644 (file)
@@ -43,12 +43,19 @@ struct SubresourceInfo {
     static bool decode(Decoder&, SubresourceInfo&);
 
     SubresourceInfo() = default;
-    SubresourceInfo(const WebCore::URL& firstPartyForCookies, bool isTransient = false)
-        : firstPartyForCookies(firstPartyForCookies)
+    SubresourceInfo(const WebCore::ResourceRequest& request, bool isTransient = false)
+        : firstPartyForCookies(request.firstPartyForCookies())
+        , httpUserAgent(request.httpUserAgent())
         , isTransient(isTransient)
     { }
 
+    SubresourceInfo(const SubresourceInfo&);
+    SubresourceInfo(SubresourceInfo&&) = default;
+    SubresourceInfo& operator=(const SubresourceInfo&);
+    SubresourceInfo& operator=(SubresourceInfo&&) = default;
+
     WebCore::URL firstPartyForCookies;
+    String httpUserAgent;
     bool isTransient { false };
 };