Cache API and IDB space usages should be initialized on first quota check
authoryouenn@apple.com <youenn@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 21 Mar 2019 22:52:18 +0000 (22:52 +0000)
committeryouenn@apple.com <youenn@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 21 Mar 2019 22:52:18 +0000 (22:52 +0000)
https://bugs.webkit.org/show_bug.cgi?id=195707

Reviewed by Chris Dumez.

Source/WebCore:

Add a way to require IDBServer to create a quota user for a given origin.
Make sure that removing a user might kick-in processing of pending requests.
In the case of processing pending requests, we should not decide on the first task
except if it is triggered by a request space response.
Update processPendingRequests accordingly.

Tests: http/tests/IndexedDB/storage-limit-1.https.html
       http/tests/IndexedDB/storage-limit-2.https.html

* Modules/indexeddb/server/IDBServer.h:
(WebCore::IDBServer::IDBServer::initializeQuotaUser):
* storage/StorageQuotaManager.cpp:
(WebCore::StorageQuotaManager::removeUser):
(WebCore::StorageQuotaManager::askForMoreSpace):
(WebCore::StorageQuotaManager::processPendingRequests):
* storage/StorageQuotaManager.h:

Source/WebKit:

When the quota manager is created, make sure it delays quota check decisions until IDB and Cache API quota users are initialized.
For IDB, the creation is synchronous but it may not be synchronous for Cache API.
For that purpose, add a temporary quota user that will stay uninitialized until these two quota users are added.
Once added, the temporary quota user is removed.
The addition of the real users is made asynchronously as this is triggered by the creation of one of the two quota users.

In the case of a Cache API caches being cleared, make sure to reset the size to zero and to redo the quota user initialization dance.

* NetworkProcess/NetworkProcess.cpp:
(WebKit::QuotaUserInitializer::initialize):
(WebKit::QuotaUserInitializer::~QuotaUserInitializer):
(WebKit::QuotaUserInitializer::QuotaUserInitializer):
(WebKit::NetworkProcess::storageQuotaManager):
* NetworkProcess/cache/CacheStorageEngine.cpp:
(WebKit::CacheStorage::Engine::initializeQuotaUser):
* NetworkProcess/cache/CacheStorageEngine.h:
* NetworkProcess/cache/CacheStorageEngineCaches.cpp:
(WebKit::CacheStorage::Caches::clear):

LayoutTests:

* TestExpectations:
Marking http/tests/cache-storage/cache-clearing-origin.https.html as
flaky on iOS as it is already marked flaky in MacOS.
* platform/mac-wk2/TestExpectations:
* http/tests/IndexedDB/storage-limit-1.https-expected.txt: Added.
* http/tests/IndexedDB/storage-limit-1.https.html: Added.
* http/tests/IndexedDB/storage-limit-2.https-expected.txt: Added.
* http/tests/IndexedDB/storage-limit-2.https.html: Added.
* http/tests/IndexedDB/storage-limit.https.html:
* platform/mac-wk1/TestExpectations:
* platform/win/TestExpectations:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@243339 268f45cc-cd09-0410-ab3c-d52691b4dbfc

22 files changed:
LayoutTests/ChangeLog
LayoutTests/TestExpectations
LayoutTests/http/tests/IndexedDB/storage-limit-1.https-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/IndexedDB/storage-limit-1.https.html [moved from LayoutTests/http/tests/IndexedDB/resources/storage-limit.js with 61% similarity]
LayoutTests/http/tests/IndexedDB/storage-limit-2.https-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/IndexedDB/storage-limit-2.https.html [new file with mode: 0644]
LayoutTests/http/tests/IndexedDB/storage-limit.https.html
LayoutTests/platform/mac-wk1/TestExpectations
LayoutTests/platform/mac-wk2/TestExpectations
LayoutTests/platform/win/TestExpectations
Source/WebCore/ChangeLog
Source/WebCore/Modules/indexeddb/server/IDBServer.cpp
Source/WebCore/Modules/indexeddb/server/IDBServer.h
Source/WebCore/storage/StorageQuotaManager.cpp
Source/WebCore/storage/StorageQuotaManager.h
Source/WebKit/ChangeLog
Source/WebKit/NetworkProcess/NetworkProcess.cpp
Source/WebKit/NetworkProcess/NetworkProcess.h
Source/WebKit/NetworkProcess/cache/CacheStorageEngine.cpp
Source/WebKit/NetworkProcess/cache/CacheStorageEngine.h
Source/WebKit/NetworkProcess/cache/CacheStorageEngineCaches.cpp
Source/WebKit/NetworkProcess/cache/CacheStorageEngineCaches.h

index a26d3f2..438befc 100644 (file)
@@ -1,3 +1,22 @@
+2019-03-21  Youenn Fablet  <youenn@apple.com>
+
+        Cache API and IDB space usages should be initialized on first quota check
+        https://bugs.webkit.org/show_bug.cgi?id=195707
+
+        Reviewed by Chris Dumez.
+
+        * TestExpectations:
+        Marking http/tests/cache-storage/cache-clearing-origin.https.html as
+        flaky on iOS as it is already marked flaky in MacOS.
+        * platform/mac-wk2/TestExpectations:
+        * http/tests/IndexedDB/storage-limit-1.https-expected.txt: Added.
+        * http/tests/IndexedDB/storage-limit-1.https.html: Added.
+        * http/tests/IndexedDB/storage-limit-2.https-expected.txt: Added.
+        * http/tests/IndexedDB/storage-limit-2.https.html: Added.
+        * http/tests/IndexedDB/storage-limit.https.html:
+        * platform/mac-wk1/TestExpectations:
+        * platform/win/TestExpectations:
+
 2019-03-21  Said Abou-Hallawa  <sabouhallawa@apple.com>
 
         Remove the SVG tear off objects for SVGPoint, SVGPointList and SVGAnimatedPointList
index c71b29c..bb5aff6 100644 (file)
@@ -2236,6 +2236,7 @@ webkit.org/b/182311 imported/w3c/web-platform-tests/service-workers/service-work
 webkit.org/b/90980 fast/forms/textarea/textarea-state-restore.html [ Pass Timeout ]
 
 webkit.org/b/182928 http/tests/cache-storage/cache-representation.https.html [ Pass Failure ]
+webkit.org/b/193976 http/tests/cache-storage/cache-clearing-origin.https.html [ Pass Failure ]
 
 webkit.org/b/116621 fast/replaced/preferred-widths.html [ Pass Failure ]
 
diff --git a/LayoutTests/http/tests/IndexedDB/storage-limit-1.https-expected.txt b/LayoutTests/http/tests/IndexedDB/storage-limit-1.https-expected.txt
new file mode 100644 (file)
index 0000000..8604f10
--- /dev/null
@@ -0,0 +1,21 @@
+This test makes sure that storage of indexedDB and Cache API do not grow unboundedly.
+
+On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
+
+
+indexedDB = self.indexedDB || self.webkitIndexedDB || self.mozIndexedDB || self.msIndexedDB || self.OIndexedDB;
+
+indexedDB.deleteDatabase(dbname)
+indexedDB.open(dbname)
+db = event.target.result
+store = db.createObjectStore('store')
+db = event.target.result
+store = db.transaction('store', 'readwrite').objectStore('store')
+request = store.add(new Uint8Array(300 * 1024), 'key')
+PASS 'error' in request is true
+PASS request.error.code is DOMException.QUOTA_EXCEEDED_ERR
+PASS request.error.name is "QuotaExceededError"
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
@@ -1,11 +1,27 @@
+<html>
+<head>
+<script src="/js-test-resources/js-test.js"></script>
+<script src="resources/shared.js"></script>
+</head>
+<body>
+<script>
 if (window.testRunner)
     testRunner.setAllowStorageQuotaIncrease(false);
 
-description("This test makes sure that storage of indexedDB does not grow unboundedly.");
+description("This test makes sure that storage of indexedDB  and Cache API do not grow unboundedly.");
 
 window.caches.open("test").then(cache => {
-    return cache.put(new Request("/test"), new Response(new Uint8Array(204800)));
-}).then(() => {
+    return cache.put(new Request("/test"), new Response(new Uint8Array(200 * 1024)));
+}).then(async() => {
+    // Let's terminate the network process so that all the opened quota users are gone for IDB quota check.
+    if (window.testRunner)
+        testRunner.terminateNetworkProcess();
+    while (true) {
+        try {
+            await fetch(".");
+            break;
+        } catch (e) { }
+    }
     indexedDBTest(prepareDatabase, onOpenSuccess, {'suffix': '-1'});
 }).catch(e => {
     testFailed("Cache API store operation failed: " + e);
@@ -19,12 +35,12 @@ function prepareDatabase(event)
 }
 
 // Quota for test is 400ko, but IDB is eating some of it when initializing files.
-// Let's make sure that 200ko is fine but 200ko after 200ko is not fine.
+// Let's make sure that 200ko is fine but 300ko after 200ko is not fine.
 async function onOpenSuccess(event)
 {
     evalAndLog("db = event.target.result");
     evalAndLog("store = db.transaction('store', 'readwrite').objectStore('store')");
-    evalAndLog("request = store.add(new Uint8Array(204800), 'key')");
+    evalAndLog("request = store.add(new Uint8Array(300 * 1024), 'key')");
     request.onerror = function(event) {
         shouldBeTrue("'error' in request");
         shouldBe("request.error.code", "DOMException.QUOTA_EXCEEDED_ERR");
@@ -36,4 +52,6 @@ async function onOpenSuccess(event)
         testFailed("Add operation should fail because storage limit is reached, but succeeded.");
         finishJSTest();
     }
-}
+}</script>
+</body>
+</html>
diff --git a/LayoutTests/http/tests/IndexedDB/storage-limit-2.https-expected.txt b/LayoutTests/http/tests/IndexedDB/storage-limit-2.https-expected.txt
new file mode 100644 (file)
index 0000000..51b7a88
--- /dev/null
@@ -0,0 +1,21 @@
+CONSOLE MESSAGE: Cache API operation failed: Quota exceeded
+This test makes sure that storage of indexedDB and Cache API do not grow unboundedly.
+
+On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
+
+
+indexedDB = self.indexedDB || self.webkitIndexedDB || self.mozIndexedDB || self.msIndexedDB || self.OIndexedDB;
+
+indexedDB.deleteDatabase(dbname)
+indexedDB.open(dbname)
+db = event.target.result
+store = db.createObjectStore('store')
+db = event.target.result
+store = db.transaction('store', 'readwrite').objectStore('store')
+request = store.add(new Uint8Array(204800), 'key')
+finished idb processing
+Cache API store operation failed: QuotaExceededError: Quota exceeded
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
diff --git a/LayoutTests/http/tests/IndexedDB/storage-limit-2.https.html b/LayoutTests/http/tests/IndexedDB/storage-limit-2.https.html
new file mode 100644 (file)
index 0000000..42718ec
--- /dev/null
@@ -0,0 +1,62 @@
+<html>
+<head>
+<script src="/js-test-resources/js-test.js"></script>
+<script src="resources/shared.js"></script>
+</head>
+<body>
+<script>
+if (window.testRunner)
+    testRunner.setAllowStorageQuotaIncrease(false);
+
+description("This test makes sure that storage of indexedDB and Cache API do not grow unboundedly.");
+
+indexedDBTest(prepareDatabase, onOpenSuccess, {'suffix': '-1'});
+
+function prepareDatabase(event)
+{
+    evalAndLog("db = event.target.result");
+    evalAndLog("store = db.createObjectStore('store')");
+}
+
+// Quota for test is 400ko, but IDB is eating some of it when initializing files.
+// Let's make sure that 200ko is fine but 200ko after 200ko is not fine.
+async function onOpenSuccess(event)
+{
+    evalAndLog("db = event.target.result");
+    evalAndLog("store = db.transaction('store', 'readwrite').objectStore('store')");
+    evalAndLog("request = store.add(new Uint8Array(204800), 'key')");
+    request.onerror = function(event) {
+        testFailed("Add operation should fail because storage limit is reached, but succeeded.");
+        finishJSTest();
+    }
+
+    request.onsuccess = async (event) => {
+        debug("finished idb processing");
+        // Let's terminate the network process so that all the opened quota users are gone for Cache API quota check.
+        if (window.testRunner)
+            testRunner.terminateNetworkProcess();
+        while (true) {
+            try {
+                await fetch(".");
+                break;
+            } catch (e) { }
+        }
+        cacheTest();
+    }
+}
+
+function cacheTest()
+{
+    window.caches.open("test").then(cache => {
+        return cache.put(new Request("/test"), new Response(new Uint8Array(304800)));
+    }).then(() => {
+        testFailed("Cache API store operation succeeded");
+        finishJSTest();
+    }).catch(e => {
+        debug("Cache API store operation failed: " + e);
+        finishJSTest();
+    });
+}
+</script>
+</body>
+</html>
index 6b4df80..7eb9299 100644 (file)
@@ -4,6 +4,46 @@
 <script src="resources/shared.js"></script>
 </head>
 <body>
-<script src="resources/storage-limit.js"></script>
+<script>
+if (window.testRunner)
+    testRunner.setAllowStorageQuotaIncrease(false);
+
+description("This test makes sure that storage of indexedDB does not grow unboundedly.");
+
+window.caches.open("test").then(cache => {
+    return cache.put(new Request("/test"), new Response(new Uint8Array(204800)));
+}).then(() => {
+    indexedDBTest(prepareDatabase, onOpenSuccess, {'suffix': '-1'});
+}).catch(e => {
+    testFailed("Cache API store operation failed: " + e);
+    finishJSTest();
+});
+
+function prepareDatabase(event)
+{
+    evalAndLog("db = event.target.result");
+    evalAndLog("store = db.createObjectStore('store')");
+}
+
+// Quota for test is 400ko, but IDB is eating some of it when initializing files.
+// Let's make sure that 200ko is fine but 200ko after 200ko is not fine.
+async function onOpenSuccess(event)
+{
+    evalAndLog("db = event.target.result");
+    evalAndLog("store = db.transaction('store', 'readwrite').objectStore('store')");
+    evalAndLog("request = store.add(new Uint8Array(204800), 'key')");
+    request.onerror = function(event) {
+        shouldBeTrue("'error' in request");
+        shouldBe("request.error.code", "DOMException.QUOTA_EXCEEDED_ERR");
+        shouldBeEqualToString("request.error.name", "QuotaExceededError");
+        finishJSTest();
+    }
+
+    request.onsuccess = function(event) {
+        testFailed("Add operation should fail because storage limit is reached, but succeeded.");
+        finishJSTest();
+    }
+}
+</script>
 </body>
 </html>
index 21fde6b..73f7346 100644 (file)
@@ -164,6 +164,8 @@ imported/w3c/web-platform-tests/fetch/cross-origin-resource-policy [ Skip ]
 
 # Quota check missing in WK1
 http/tests/IndexedDB/storage-limit.https.html [ Skip ]
+http/tests/IndexedDB/storage-limit-1.https.html [ Skip ]
+http/tests/IndexedDB/storage-limit-2.https.html [ Skip ]
 storage/indexeddb/storage-limit.html [ Skip ]
 
 # Skip WebRTC for now in WK1
index 0ee5889..5430647 100644 (file)
@@ -912,8 +912,6 @@ webkit.org/b/191644 [ Sierra ] fast/workers/worker-cloneport.html [ Pass Failure
 
 webkit.org/b/191658 [ Sierra Release ] fast/layers/no-clipping-overflow-hidden-added-after-transform.html [ Pass ImageOnlyFailure ]
 
-webkit.org/b/193976 http/tests/cache-storage/cache-clearing-origin.https.html [ Pass Failure ]
-
 webkit.org/b/194826 http/tests/resourceLoadStatistics/do-not-block-top-level-navigation-redirect.html [ Pass Timeout ]
 
 webkit.org/b/194368 http/tests/workers/service/basic-register-exceptions.html [ Pass Failure ]
index 8875c96..8a30b9c 100644 (file)
@@ -2375,6 +2375,8 @@ http/tests/xmlhttprequest/web-apps/013.html
 
 storage/indexeddb/storage-limit.html [ Skip ]
 http/tests/IndexedDB/storage-limit.https.html [ Skip ]
+http/tests/IndexedDB/storage-limit-1.https.html [ Skip ]
+http/tests/IndexedDB/storage-limit-2.https.html [ Skip ]
 
 # Assertion failures: Not investigated
 [ Debug ] http/tests/security/isolatedWorld/storage-properties.html [ Skip ]
index 77a9cbd..f370e96 100644 (file)
@@ -1,3 +1,27 @@
+2019-03-21  Youenn Fablet  <youenn@apple.com>
+
+        Cache API and IDB space usages should be initialized on first quota check
+        https://bugs.webkit.org/show_bug.cgi?id=195707
+
+        Reviewed by Chris Dumez.
+
+        Add a way to require IDBServer to create a quota user for a given origin.
+        Make sure that removing a user might kick-in processing of pending requests.
+        In the case of processing pending requests, we should not decide on the first task
+        except if it is triggered by a request space response.
+        Update processPendingRequests accordingly.
+
+        Tests: http/tests/IndexedDB/storage-limit-1.https.html
+               http/tests/IndexedDB/storage-limit-2.https.html
+
+        * Modules/indexeddb/server/IDBServer.h:
+        (WebCore::IDBServer::IDBServer::initializeQuotaUser):
+        * storage/StorageQuotaManager.cpp:
+        (WebCore::StorageQuotaManager::removeUser):
+        (WebCore::StorageQuotaManager::askForMoreSpace):
+        (WebCore::StorageQuotaManager::processPendingRequests):
+        * storage/StorageQuotaManager.h:
+
 2019-03-21  Alex Christensen  <achristensen@webkit.org>
 
         Fix iOS build after r243337
index cd93bf8..62f320c 100644 (file)
@@ -784,7 +784,7 @@ void IDBServer::QuotaUser::initializeSpaceUsed(uint64_t spaceUsed)
         callback();
 }
 
-IDBServer::QuotaUser& IDBServer::quotaUser(const ClientOrigin& origin)
+IDBServer::QuotaUser& IDBServer::ensureQuotaUser(const ClientOrigin& origin)
 {
     return *m_quotaUsers.ensure(origin, [this, &origin] {
         return std::make_unique<QuotaUser>(*this, m_quotaManagerGetter(m_sessionID, origin), ClientOrigin { origin });
@@ -810,12 +810,12 @@ void IDBServer::computeSpaceUsedForOrigin(const ClientOrigin& origin)
 
 void IDBServer::finishComputingSpaceUsedForOrigin(const ClientOrigin& origin, uint64_t spaceUsed)
 {
-    quotaUser(origin).initializeSpaceUsed(spaceUsed);
+    ensureQuotaUser(origin).initializeSpaceUsed(spaceUsed);
 }
 
 void IDBServer::requestSpace(const ClientOrigin& origin, uint64_t taskSize, CompletionHandler<void(StorageQuotaManager::Decision)>&& callback)
 {
-    auto* quotaManager = quotaUser(origin).manager();
+    auto* quotaManager = ensureQuotaUser(origin).manager();
     if (!quotaManager) {
         callback(StorageQuotaManager::Decision::Deny);
         return;
@@ -832,17 +832,17 @@ void IDBServer::resetSpaceUsed(const ClientOrigin& origin)
 
 void IDBServer::setSpaceUsed(const ClientOrigin& origin, uint64_t taskSize)
 {
-    quotaUser(origin).setSpaceUsed(taskSize);
+    ensureQuotaUser(origin).setSpaceUsed(taskSize);
 }
 
 void IDBServer::increasePotentialSpaceUsed(const ClientOrigin& origin, uint64_t taskSize)
 {
-    quotaUser(origin).increasePotentialSpaceUsed(taskSize);
+    ensureQuotaUser(origin).increasePotentialSpaceUsed(taskSize);
 }
 
 void IDBServer::decreasePotentialSpaceUsed(const ClientOrigin& origin, uint64_t spaceUsed)
 {
-    quotaUser(origin).decreasePotentialSpaceUsed(spaceUsed);
+    ensureQuotaUser(origin).decreasePotentialSpaceUsed(spaceUsed);
 }
 
 void IDBServer::upgradeFilesIfNecessary()
index 2ce656f..ae4bfd8 100644 (file)
@@ -125,6 +125,8 @@ public:
     void setSpaceUsed(const ClientOrigin&, uint64_t spaceUsed);
     void resetSpaceUsed(const ClientOrigin&);
 
+    void initializeQuotaUser(const ClientOrigin& origin) { ensureQuotaUser(origin); }
+
 private:
     IDBServer(PAL::SessionID, IDBBackingStoreTemporaryFileHandler&, QuotaManagerGetter&&, WTF::Function<void(bool)>&&);
     IDBServer(PAL::SessionID, const String& databaseDirectoryPath, IDBBackingStoreTemporaryFileHandler&, QuotaManagerGetter&&, WTF::Function<void(bool)>&&);
@@ -175,7 +177,7 @@ private:
         CompletionHandler<void()> m_initializationCallback;
     };
 
-    QuotaUser& quotaUser(const ClientOrigin&);
+    WEBCORE_EXPORT QuotaUser& ensureQuotaUser(const ClientOrigin&);
     void startComputingSpaceUsedForOrigin(const ClientOrigin&);
     void computeSpaceUsedForOrigin(const ClientOrigin&);
     void finishComputingSpaceUsedForOrigin(const ClientOrigin&, uint64_t spaceUsed);
index b7e71be..d2884af 100644 (file)
@@ -69,7 +69,7 @@ void StorageQuotaManager::addUser(StorageQuotaUser& user)
             return;
 
         updateQuotaBasedOnSpaceUsage();
-        processPendingRequests({ });
+        processPendingRequests({ }, ShouldDequeueFirstPendingRequest::No);
     });
 }
 
@@ -81,6 +81,14 @@ bool StorageQuotaManager::shouldAskForMoreSpace(uint64_t spaceIncrease) const
     return spaceUsage() + spaceIncrease > m_quota;
 }
 
+void StorageQuotaManager::removeUser(StorageQuotaUser& user)
+{
+    ASSERT(m_users.contains(&user) || m_pendingInitializationUsers.contains(&user));
+    m_users.remove(&user);
+    if (m_pendingInitializationUsers.remove(&user) && m_pendingInitializationUsers.isEmpty())
+        processPendingRequests({ }, ShouldDequeueFirstPendingRequest::No);
+}
+
 void StorageQuotaManager::requestSpace(uint64_t spaceIncrease, RequestCallback&& callback)
 {
     if (!m_pendingRequests.isEmpty() || !m_pendingInitializationUsers.isEmpty()) {
@@ -99,14 +107,17 @@ void StorageQuotaManager::requestSpace(uint64_t spaceIncrease, RequestCallback&&
 void StorageQuotaManager::askForMoreSpace(uint64_t spaceIncrease)
 {
     ASSERT(shouldAskForMoreSpace(spaceIncrease));
+    ASSERT(!m_isWaitingForSpaceIncreaseResponse);
+    m_isWaitingForSpaceIncreaseResponse = true;
     m_spaceIncreaseRequester(m_quota, spaceUsage(), spaceIncrease, [this, weakThis = makeWeakPtr(*this)](Optional<uint64_t> newQuota) {
         if (!weakThis)
             return;
-        processPendingRequests(newQuota);
+        m_isWaitingForSpaceIncreaseResponse = false;
+        processPendingRequests(newQuota, ShouldDequeueFirstPendingRequest::Yes);
     });
 }
 
-void StorageQuotaManager::processPendingRequests(Optional<uint64_t> newQuota)
+void StorageQuotaManager::processPendingRequests(Optional<uint64_t> newQuota, ShouldDequeueFirstPendingRequest shouldDequeueFirstPendingRequest)
 {
     if (m_pendingRequests.isEmpty())
         return;
@@ -114,9 +125,17 @@ void StorageQuotaManager::processPendingRequests(Optional<uint64_t> newQuota)
     if (newQuota)
         m_quota = *newQuota;
 
-    auto request = m_pendingRequests.takeFirst();
-    auto decision = shouldAskForMoreSpace(request.spaceIncrease) ? Decision::Deny : Decision::Grant;
-    request.callback(decision);
+    if (m_isWaitingForSpaceIncreaseResponse)
+        return;
+
+    if (!m_pendingInitializationUsers.isEmpty())
+        return;
+
+    if (shouldDequeueFirstPendingRequest == ShouldDequeueFirstPendingRequest::Yes) {
+        auto request = m_pendingRequests.takeFirst();
+        auto decision = shouldAskForMoreSpace(request.spaceIncrease) ? Decision::Deny : Decision::Grant;
+        request.callback(decision);
+    }
 
     while (!m_pendingRequests.isEmpty()) {
         auto& request = m_pendingRequests.first();
index a0585ab..44d4fbf 100644 (file)
@@ -50,12 +50,7 @@ public:
     static constexpr uint64_t defaultThirdPartyQuota() { return 100 * MB; }
 
     WEBCORE_EXPORT void addUser(StorageQuotaUser&);
-    void removeUser(StorageQuotaUser& user)
-    {
-        ASSERT(m_users.contains(&user) || m_pendingInitializationUsers.contains(&user));
-        m_pendingInitializationUsers.remove(&user);
-        m_users.remove(&user);
-    }
+    WEBCORE_EXPORT void removeUser(StorageQuotaUser&);
 
     enum class Decision { Deny, Grant };
     using RequestCallback = CompletionHandler<void(Decision)>;
@@ -68,9 +63,13 @@ private:
     uint64_t spaceUsage() const;
     bool shouldAskForMoreSpace(uint64_t spaceIncrease) const;
     void askForMoreSpace(uint64_t spaceIncrease);
-    void processPendingRequests(Optional<uint64_t>);
+
+    enum class ShouldDequeueFirstPendingRequest { No, Yes };
+    void processPendingRequests(Optional<uint64_t>, ShouldDequeueFirstPendingRequest);
 
     uint64_t m_quota { 0 };
+
+    bool m_isWaitingForSpaceIncreaseResponse { false };
     SpaceIncreaseRequester m_spaceIncreaseRequester;
     HashSet<const StorageQuotaUser*> m_pendingInitializationUsers;
     HashSet<const StorageQuotaUser*> m_users;
index 1f67344..cc173eb 100644 (file)
@@ -1,3 +1,29 @@
+2019-03-21  Youenn Fablet  <youenn@apple.com>
+
+        Cache API and IDB space usages should be initialized on first quota check
+        https://bugs.webkit.org/show_bug.cgi?id=195707
+
+        Reviewed by Chris Dumez.
+
+        When the quota manager is created, make sure it delays quota check decisions until IDB and Cache API quota users are initialized.
+        For IDB, the creation is synchronous but it may not be synchronous for Cache API.
+        For that purpose, add a temporary quota user that will stay uninitialized until these two quota users are added.
+        Once added, the temporary quota user is removed.
+        The addition of the real users is made asynchronously as this is triggered by the creation of one of the two quota users.
+
+        In the case of a Cache API caches being cleared, make sure to reset the size to zero and to redo the quota user initialization dance.
+
+        * NetworkProcess/NetworkProcess.cpp:
+        (WebKit::QuotaUserInitializer::initialize):
+        (WebKit::QuotaUserInitializer::~QuotaUserInitializer):
+        (WebKit::QuotaUserInitializer::QuotaUserInitializer):
+        (WebKit::NetworkProcess::storageQuotaManager):
+        * NetworkProcess/cache/CacheStorageEngine.cpp:
+        (WebKit::CacheStorage::Engine::initializeQuotaUser):
+        * NetworkProcess/cache/CacheStorageEngine.h:
+        * NetworkProcess/cache/CacheStorageEngineCaches.cpp:
+        (WebKit::CacheStorage::Caches::clear):
+
 2019-03-21  Per Arne Vollan  <pvollan@apple.com>
 
         [iOS][macOS] Fix sandbox call violations
index 937a97d..b5b7bec 100644 (file)
@@ -2394,6 +2394,49 @@ void NetworkProcess::requestStorageSpace(PAL::SessionID sessionID, const ClientO
     parentProcessConnection()->sendWithAsyncReply(Messages::NetworkProcessProxy::RequestStorageSpace { sessionID, origin, quota, currentSize, spaceRequired }, WTFMove(callback), 0);
 }
 
+class QuotaUserInitializer final : public WebCore::StorageQuotaUser {
+public:
+    explicit QuotaUserInitializer(StorageQuotaManager& manager)
+        : m_manager(makeWeakPtr(manager))
+    {
+        manager.addUser(*this);
+    }
+
+    ~QuotaUserInitializer()
+    {
+        if (m_manager)
+            m_manager->removeUser(*this);
+        if (m_callback)
+            m_callback();
+    }
+
+private:
+    // StorageQuotaUser API.
+    uint64_t spaceUsed() const final
+    {
+        ASSERT_NOT_REACHED();
+        return 0;
+    }
+
+    void whenInitialized(CompletionHandler<void()>&& callback) final
+    {
+        m_callback = WTFMove(callback);
+    }
+
+    WeakPtr<StorageQuotaManager> m_manager;
+    CompletionHandler<void()> m_callback;
+};
+
+void NetworkProcess::initializeQuotaUsers(StorageQuotaManager& manager, PAL::SessionID sessionID, const ClientOrigin& origin)
+{
+    RunLoop::main().dispatch([this, weakThis = makeWeakPtr(this), sessionID, origin, user = std::make_unique<QuotaUserInitializer>(manager)]() mutable {
+        if (!weakThis)
+            return;
+        this->idbServer(sessionID).initializeQuotaUser(origin);
+        CacheStorage::Engine::initializeQuotaUser(*this, sessionID, origin, [user = WTFMove(user)] { });
+    });
+}
+
 StorageQuotaManager& NetworkProcess::storageQuotaManager(PAL::SessionID sessionID, const ClientOrigin& origin)
 {
     auto& storageQuotaManagers = m_storageQuotaManagers.ensure(sessionID, [] {
@@ -2401,9 +2444,11 @@ StorageQuotaManager& NetworkProcess::storageQuotaManager(PAL::SessionID sessionI
     }).iterator->value;
     return *storageQuotaManagers.managersPerOrigin.ensure(origin, [this, &storageQuotaManagers, sessionID, &origin] {
         auto quota = origin.topOrigin == origin.clientOrigin ? storageQuotaManagers.defaultQuota : storageQuotaManagers.defaultThirdPartyQuota;
-        return std::make_unique<StorageQuotaManager>(quota, [this, sessionID, origin](uint64_t quota, uint64_t currentSpace, uint64_t spaceIncrease, auto callback) {
+        auto manager = std::make_unique<StorageQuotaManager>(quota, [this, sessionID, origin](uint64_t quota, uint64_t currentSpace, uint64_t spaceIncrease, auto callback) {
             this->requestStorageSpace(sessionID, origin, quota, currentSpace, spaceIncrease, WTFMove(callback));
         });
+        initializeQuotaUsers(*manager, sessionID, origin);
+        return manager;
     }).iterator->value;
 }
 
index 133beaa..cc268dc 100644 (file)
@@ -383,6 +383,7 @@ private:
     void clearCachedCredentials();
 
     void setCacheStorageParameters(PAL::SessionID, uint64_t quota, String&& cacheStorageDirectory, SandboxExtension::Handle&&);
+    void initializeQuotaUsers(WebCore::StorageQuotaManager&, PAL::SessionID, const WebCore::ClientOrigin&);
 
     // FIXME: This should take a session ID so we can identify which disk cache to delete.
     void clearDiskCache(WallTime modifiedSince, CompletionHandler<void()>&&);
index e390b5e..e6f9fc8 100644 (file)
@@ -185,6 +185,15 @@ void Engine::clearCachesForOrigin(NetworkProcess& networkProcess, PAL::SessionID
     });
 }
 
+void Engine::initializeQuotaUser(NetworkProcess& networkProcess, PAL::SessionID sessionID, const WebCore::ClientOrigin& clientOrigin, CompletionHandler<void()>&& completionHandler)
+{
+    from(networkProcess, sessionID, [clientOrigin, completionHandler = WTFMove(completionHandler)](auto& engine) mutable {
+        engine.readCachesFromDisk(clientOrigin, [completionHandler = WTFMove(completionHandler)](auto&& cachesOrError) mutable {
+            completionHandler();
+        });
+    });
+}
+
 Engine::Engine(PAL::SessionID sessionID, NetworkProcess& process, String&& rootPath)
     : m_sessionID(sessionID)
     , m_networkProcess(makeWeakPtr(process))
index 870ab7b..1768501 100644 (file)
@@ -80,6 +80,8 @@ public:
     static void clearAllCaches(NetworkProcess&, PAL::SessionID, CompletionHandler<void()>&&);
     static void clearCachesForOrigin(NetworkProcess&, PAL::SessionID, WebCore::SecurityOriginData&&, CompletionHandler<void()>&&);
 
+    static void initializeQuotaUser(NetworkProcess&, PAL::SessionID, const WebCore::ClientOrigin&, CompletionHandler<void()>&&);
+
     bool shouldPersist() const { return !!m_ioQueue;}
 
     void writeFile(const String& filename, NetworkCache::Data&&, WebCore::DOMCacheEngine::CompletionCallback&&);
index c532f1c..08b94b4 100644 (file)
@@ -260,11 +260,13 @@ void Caches::clear(CompletionHandler<void()>&& completionHandler)
         m_storage->clear(String { }, -WallTime::infinity(), [protectedThis = makeRef(*this), completionHandler = WTFMove(completionHandler)]() mutable {
             ASSERT(RunLoop::isMain());
             protectedThis->clearMemoryRepresentation();
+            protectedThis->resetSpaceUsed();
             completionHandler();
         });
         return;
     }
     clearMemoryRepresentation();
+    resetSpaceUsed();
     clearPendingWritingCachesToDiskCallbacks();
     completionHandler();
 }
@@ -601,6 +603,15 @@ void Caches::removeCacheEntry(const NetworkCache::Key& key)
     m_storage->remove(key);
 }
 
+void Caches::resetSpaceUsed()
+{
+    m_size = 0;
+    if (m_quotaManager) {
+        m_quotaManager->removeUser(*this);
+        m_quotaManager->addUser(*this);
+    }
+}
+
 void Caches::clearMemoryRepresentation()
 {
     if (!m_isInitialized) {
index 33a162c..3c451a7 100644 (file)
@@ -78,6 +78,7 @@ public:
 
     void clear(WTF::CompletionHandler<void()>&&);
     void clearMemoryRepresentation();
+    void resetSpaceUsed();
 
     uint64_t storageSize() const;