Check IDB quota usage through QuotaManager
authoryouenn@apple.com <youenn@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 13 Mar 2019 21:47:13 +0000 (21:47 +0000)
committeryouenn@apple.com <youenn@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 13 Mar 2019 21:47:13 +0000 (21:47 +0000)
https://bugs.webkit.org/show_bug.cgi?id=195302

Reviewed by Chris Dumez.

Source/WebCore:

For every write operation, compute an estimate size and check for quota before proceeding.
When proceeding, store the estimate size in a map.
If size of the database is to be computed when the task is not done,
the estimate size will be added to the current size of the databases.
At the end of the task, the estimate size is removed from the map,
and the databases size is refreshed.

This patch implements size estimation for write tasks.
Put/add operations might overestimate the size
when an old value will be replaced by a new value.
In that case, we do not substract the old value size since we do not know it.

This patch implements database opening by adding a fixed small cost,
as we do not know whether the database is new or not.

For the first IDB request, we have not computed the size of the database.
To do so, we need to go to a background thread and do that file size computation.
For that purpose, we add support for being-initialized quota user.
Quota manager is calling whenInitialized on its quota user and will
delay any quota check requests until its quota user is answering this callback.

For in process IDB, use the default storage quota per origin and do not increase it.
Future work should move it to NetworkProcess and implement some quota checking.

Cache API and IDB quota management are not yet fully unified.
If IDB is used on start-up, we should check for Cache API storage size.
Conversely, on Cache API first wite task, even if IDB is not being used,
we should compute the size of the IDB data for the given origin.

Test: http/tests/IndexedDB/storage-limit.https.html

* Modules/indexeddb/server/IDBBackingStore.h:
* Modules/indexeddb/server/IDBServer.cpp:
(WebCore::IDBServer::IDBServer::create):
(WebCore::IDBServer::IDBServer::IDBServer):
(WebCore::IDBServer::m_quotaManagerGetter):
(WebCore::IDBServer::IDBServer::QuotaUser::QuotaUser):
(WebCore::IDBServer::IDBServer::QuotaUser::~QuotaUser):
(WebCore::IDBServer::IDBServer::QuotaUser::clearSpaceUsed):
(WebCore::IDBServer::IDBServer::QuotaUser::whenInitialized):
(WebCore::IDBServer::IDBServer::QuotaUser::initializeSpaceUsed):
(WebCore::IDBServer::IDBServer::quotaUser):
(WebCore::IDBServer::IDBServer::startComputingSpaceUsedForOrigin):
(WebCore::IDBServer::IDBServer::computeSpaceUsedForOrigin):
(WebCore::IDBServer::IDBServer::finishComputingSpaceUsedForOrigin):
(WebCore::IDBServer::IDBServer::requestSpace):
(WebCore::IDBServer::IDBServer::clearSpaceUsed):
(WebCore::IDBServer::IDBServer::setSpaceUsed):
(WebCore::IDBServer::IDBServer::increasePotentialSpaceUsed):
(WebCore::IDBServer::IDBServer::decreasePotentialSpaceUsed):
* Modules/indexeddb/server/IDBServer.h:
(WebCore::IDBServer::IDBServer::create):
* Modules/indexeddb/server/MemoryIDBBackingStore.cpp:
(WebCore::IDBServer::MemoryIDBBackingStore::databasesSizeForOrigin const):
* Modules/indexeddb/server/MemoryIDBBackingStore.h:
* Modules/indexeddb/server/SQLiteIDBBackingStore.cpp:
(WebCore::IDBServer::SQLiteIDBBackingStore::databasesSizeForFolder):
(WebCore::IDBServer::SQLiteIDBBackingStore::databasesSizeForOrigin const):
(WebCore::IDBServer::SQLiteIDBBackingStore::maximumSize const):
* Modules/indexeddb/server/SQLiteIDBBackingStore.h:
* Modules/indexeddb/server/UniqueIDBDatabase.cpp:
(WebCore::IDBServer::estimateSize):
(WebCore::IDBServer::UniqueIDBDatabase::UniqueIDBDatabase):
(WebCore::IDBServer::quotaErrorMessageName):
(WebCore::IDBServer::UniqueIDBDatabase::requestSpace):
(WebCore::IDBServer::UniqueIDBDatabase::performCurrentOpenOperation):
(WebCore::IDBServer::UniqueIDBDatabase::storeCallbackOrFireError):
(WebCore::IDBServer::UniqueIDBDatabase::createObjectStore):
(WebCore::IDBServer::UniqueIDBDatabase::createObjectStoreAfterQuotaCheck):
(WebCore::IDBServer::UniqueIDBDatabase::renameObjectStore):
(WebCore::IDBServer::UniqueIDBDatabase::renameObjectStoreAfterQuotaCheck):
(WebCore::IDBServer::UniqueIDBDatabase::createIndex):
(WebCore::IDBServer::UniqueIDBDatabase::createIndexAfterQuotaCheck):
(WebCore::IDBServer::UniqueIDBDatabase::renameIndex):
(WebCore::IDBServer::UniqueIDBDatabase::renameIndexAfterQuotaCheck):
(WebCore::IDBServer::UniqueIDBDatabase::putOrAdd):
(WebCore::IDBServer::UniqueIDBDatabase::putOrAddAfterQuotaCheck):
(WebCore::IDBServer::UniqueIDBDatabase::postDatabaseTaskReply):
(WebCore::IDBServer::UniqueIDBDatabase::immediateCloseForUserDelete):
(WebCore::IDBServer::UniqueIDBDatabase::updateSpaceUsedIfNeeded):
(WebCore::IDBServer::UniqueIDBDatabase::performErrorCallback):
(WebCore::IDBServer::UniqueIDBDatabase::performKeyDataCallback):
* Modules/indexeddb/server/UniqueIDBDatabase.h:
(WebCore::IDBServer::UniqueIDBDatabase::server):
* Modules/indexeddb/shared/InProcessIDBServer.cpp:
(WebCore::InProcessIDBServer::create):
(WebCore::InProcessIDBServer::quotaManager):
(WebCore::storageQuotaManagerGetter):
(WebCore::InProcessIDBServer::InProcessIDBServer):
* Modules/indexeddb/shared/InProcessIDBServer.h:
* loader/EmptyClients.cpp:
* storage/StorageQuotaManager.cpp:
(WebCore::StorageQuotaManager::addUser):
(WebCore::StorageQuotaManager::requestSpace):
* storage/StorageQuotaManager.h:
(WebCore::StorageQuotaManager::defaultQuota):
(WebCore::StorageQuotaManager::removeUser):
* storage/StorageQuotaUser.h:
(WebCore::StorageQuotaUser::whenInitialized):

Source/WebKit:

Set the quota manager getter for IDBServer at creation time.

* NetworkProcess/NetworkProcess.cpp:
(WebKit::NetworkProcess::createIDBServer):
(WebKit::NetworkProcess::idbServer):
* NetworkProcess/NetworkProcess.h:
* WebProcess/Databases/WebDatabaseProvider.cpp:
(WebKit::WebDatabaseProvider::idbConnectionToServerForSession):

Source/WebKitLegacy:

* Storage/WebDatabaseProvider.cpp:
(WebDatabaseProvider::idbConnectionToServerForSession):

LayoutTests:

Update IDB quota test according quota limit of 400ko.
Update WK1 test expectations to skip quota check tests.

* http/tests/IndexedDB/resources/shared.js: Added.
* http/tests/IndexedDB/resources/storage-limit.js: Added.
* http/tests/IndexedDB/storage-limit.https-expected.txt: Added.
* http/tests/IndexedDB/storage-limit.https.html: Added.
* platform/mac-wk1/TestExpectations:
* platform/win/TestExpectations:
* storage/indexeddb/resources/storage-limit.js:
* storage/indexeddb/storage-limit-expected.txt:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@242911 268f45cc-cd09-0410-ab3c-d52691b4dbfc

31 files changed:
LayoutTests/ChangeLog
LayoutTests/http/tests/IndexedDB/resources/shared.js [new file with mode: 0644]
LayoutTests/http/tests/IndexedDB/resources/storage-limit.js [new file with mode: 0644]
LayoutTests/http/tests/IndexedDB/storage-limit.https-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/IndexedDB/storage-limit.https.html [new file with mode: 0644]
LayoutTests/platform/mac-wk1/TestExpectations
LayoutTests/platform/win/TestExpectations
LayoutTests/storage/indexeddb/resources/storage-limit.js
LayoutTests/storage/indexeddb/storage-limit-expected.txt
Source/WebCore/ChangeLog
Source/WebCore/Modules/indexeddb/server/IDBBackingStore.h
Source/WebCore/Modules/indexeddb/server/IDBServer.cpp
Source/WebCore/Modules/indexeddb/server/IDBServer.h
Source/WebCore/Modules/indexeddb/server/MemoryIDBBackingStore.cpp
Source/WebCore/Modules/indexeddb/server/MemoryIDBBackingStore.h
Source/WebCore/Modules/indexeddb/server/SQLiteIDBBackingStore.cpp
Source/WebCore/Modules/indexeddb/server/SQLiteIDBBackingStore.h
Source/WebCore/Modules/indexeddb/server/UniqueIDBDatabase.cpp
Source/WebCore/Modules/indexeddb/server/UniqueIDBDatabase.h
Source/WebCore/Modules/indexeddb/shared/InProcessIDBServer.cpp
Source/WebCore/Modules/indexeddb/shared/InProcessIDBServer.h
Source/WebCore/loader/EmptyClients.cpp
Source/WebCore/storage/StorageQuotaManager.cpp
Source/WebCore/storage/StorageQuotaManager.h
Source/WebCore/storage/StorageQuotaUser.h
Source/WebKit/ChangeLog
Source/WebKit/NetworkProcess/NetworkProcess.cpp
Source/WebKit/NetworkProcess/NetworkProcess.h
Source/WebKit/WebProcess/Databases/WebDatabaseProvider.cpp
Source/WebKitLegacy/ChangeLog
Source/WebKitLegacy/Storage/WebDatabaseProvider.cpp

index 6484e6f..b292384 100644 (file)
@@ -1,3 +1,22 @@
+2019-03-13  Youenn Fablet  <youenn@apple.com>
+
+        Check IDB quota usage through QuotaManager
+        https://bugs.webkit.org/show_bug.cgi?id=195302
+
+        Reviewed by Chris Dumez.
+
+        Update IDB quota test according quota limit of 400ko.
+        Update WK1 test expectations to skip quota check tests.
+
+        * http/tests/IndexedDB/resources/shared.js: Added.
+        * http/tests/IndexedDB/resources/storage-limit.js: Added.
+        * http/tests/IndexedDB/storage-limit.https-expected.txt: Added.
+        * http/tests/IndexedDB/storage-limit.https.html: Added.
+        * platform/mac-wk1/TestExpectations:
+        * platform/win/TestExpectations:
+        * storage/indexeddb/resources/storage-limit.js:
+        * storage/indexeddb/storage-limit-expected.txt:
+
 2019-03-13  Truitt Savell  <tsavell@apple.com>
 
         (r242595) Layout Tests in imported/w3c/web-platform-tests/html/semantics/embedded-content/media-elements/* are failing
diff --git a/LayoutTests/http/tests/IndexedDB/resources/shared.js b/LayoutTests/http/tests/IndexedDB/resources/shared.js
new file mode 100644 (file)
index 0000000..fc9c1e8
--- /dev/null
@@ -0,0 +1,227 @@
+var jsTestIsAsync = true;
+if (self.importScripts && !self.postMessage) {
+    // Shared worker.  Make postMessage send to the newest client, which in
+    // our tests is the only client.
+
+    // Store messages for sending until we have somewhere to send them.
+    self.postMessage = function(message)
+    {
+        if (typeof self.pendingMessages === "undefined")
+            self.pendingMessages = [];
+        self.pendingMessages.push(message);
+    };
+    self.onconnect = function(event)
+    {
+        self.postMessage = function(message)
+        {
+            event.ports[0].postMessage(message);
+        };
+        // Offload any stored messages now that someone has connected to us.
+        if (typeof self.pendingMessages === "undefined")
+            return;
+        while (self.pendingMessages.length)
+            event.ports[0].postMessage(self.pendingMessages.shift());
+    };
+}
+
+function removeVendorPrefixes()
+{
+    IDBCursor = self.IDBCursor || self.webkitIDBCursor;
+    IDBDatabase = self.IDBDatabase || self.webkitIDBDatabase;
+    IDBFactory = self.IDBFactory || self.webkitIDBFactory;
+    IDBIndex = self.IDBIndex || self.webkitIDBIndex;
+    IDBKeyRange = self.IDBKeyRange || self.webkitIDBKeyRange;
+    IDBObjectStore = self.IDBObjectStore || self.webkitIDBObjectStore;
+    IDBRequest = self.IDBRequest || self.webkitIDBRequest;
+    IDBTransaction = self.IDBTransaction || self.webkitIDBTransaction;
+
+    indexedDB = evalAndLog("indexedDB = self.indexedDB || self.webkitIndexedDB || self.mozIndexedDB || self.msIndexedDB || self.OIndexedDB;");
+    debug("");
+}
+
+function unexpectedSuccessCallback()
+{
+    testFailed("Success function called unexpectedly.");
+    finishJSTest();
+}
+
+function unexpectedErrorCallback(event)
+{
+    testFailed("Error function called unexpectedly: (" + event.target.error.name + ") " + event.target.error.message);
+    finishJSTest();
+}
+
+function unexpectedAbortCallback(e)
+{
+    testFailed("Abort function called unexpectedly! Message: [" + e.target.error.message + "]");
+    finishJSTest();
+}
+
+function unexpectedCompleteCallback()
+{
+    testFailed("oncomplete function called unexpectedly!");
+    finishJSTest();
+}
+
+function unexpectedBlockedCallback(e)
+{
+    testFailed("onblocked called unexpectedly. oldVersion = " + e.oldVersion + ", newVersion = " + e.newVersion);
+    finishJSTest();
+}
+
+function unexpectedUpgradeNeededCallback()
+{
+    testFailed("onupgradeneeded called unexpectedly");
+    finishJSTest();
+}
+
+function unexpectedVersionChangeCallback(e)
+{
+    testFailed("onversionchange called unexpectedly. oldVersion = " + e.oldVersion + ". newVersion = " + e.newVersion);
+    finishJSTest();
+}
+
+function evalAndExpectException(cmd, exceptionCode, exceptionName, _quiet)
+{
+    if (!_quiet)
+        debug("Expecting exception from " + cmd);
+    try {
+        eval(cmd);
+        testFailed("No exception thrown! Should have been " + exceptionCode);
+    } catch (e) {
+        code = e.code;
+        if (!_quiet)
+            testPassed("Exception was thrown.");
+        shouldBe("code", exceptionCode, _quiet);
+        if (exceptionName) {
+            ename = e.name;
+            shouldBe("ename", exceptionName, _quiet);
+        }
+        if (!_quiet)
+            debug("Exception message: " + e.message);
+    }
+}
+
+function evalAndExpectExceptionClass(cmd, expected)
+{
+    debug("Expecting " + expected + " exception from " + cmd);
+    try {
+        eval(cmd);
+        testFailed("No exception thrown!" );
+    } catch (e) {
+        testPassed("Exception was thrown.");
+        if (eval("e instanceof " + expected))
+            testPassed(cmd + " threw " + e);
+        else
+            testFailed("Expected " + expected + " but saw " + e);
+    }
+}
+
+function evalAndLogCallback(cmd) {
+  function callback() {
+    evalAndLog(cmd);
+  }
+  return callback;
+}
+
+// If this function is deleted, a standalone layout test exercising its
+// functionality should be added.
+function deleteAllObjectStores(db)
+{
+    while (db.objectStoreNames.length)
+        db.deleteObjectStore(db.objectStoreNames.item(0));
+    debug("Deleted all object stores.");
+}
+
+function setDBNameFromPath(suffix) {
+    var name = self.location.pathname.substring(1 + self.location.pathname.lastIndexOf("/"));
+    if (suffix)
+        name += suffix;
+    eval('dbname = "' + name + '"');
+    return name;
+}
+
+function preamble(evt)
+{
+    if (evt)
+        event = evt;
+    debug("");
+    debug(preamble.caller.name + "():");
+}
+
+// For Workers
+if (!self.DOMException) {
+    self.DOMException = {
+        INDEX_SIZE_ERR: 1,
+        DOMSTRING_SIZE_ERR: 2,
+        HIERARCHY_REQUEST_ERR: 3,
+        WRONG_DOCUMENT_ERR: 4,
+        INVALID_CHARACTER_ERR: 5,
+        NO_DATA_ALLOWED_ERR: 6,
+        NO_MODIFICATION_ALLOWED_ERR: 7,
+        NOT_FOUND_ERR: 8,
+        NOT_SUPPORTED_ERR: 9,
+        INUSE_ATTRIBUTE_ERR: 10,
+        INVALID_STATE_ERR: 11,
+        SYNTAX_ERR: 12,
+        INVALID_MODIFICATION_ERR: 13,
+        NAMESPACE_ERR: 14,
+        INVALID_ACCESS_ERR: 15,
+        VALIDATION_ERR: 16,
+        TYPE_MISMATCH_ERR: 17,
+        SECURITY_ERR: 18,
+        NETWORK_ERR: 19,
+        ABORT_ERR: 20,
+        URL_MISMATCH_ERR: 21,
+        QUOTA_EXCEEDED_ERR: 22,
+        TIMEOUT_ERR: 23,
+        INVALID_NODE_TYPE_ERR: 24,
+        DATA_CLONE_ERR: 25
+    };
+}
+
+function indexedDBTest(upgradeCallback, optionalOpenCallback, optionalParameters) {
+    removeVendorPrefixes();
+    if (optionalParameters && 'suffix' in optionalParameters) {
+        setDBNameFromPath(optionalParameters['suffix']);
+    } else {
+        setDBNameFromPath();
+    }
+    var deleteRequest = evalAndLog("indexedDB.deleteDatabase(dbname)");
+    deleteRequest.onerror = unexpectedErrorCallback;
+    deleteRequest.onblocked = unexpectedBlockedCallback;
+    deleteRequest.onsuccess = function() {
+        self.openRequest = null;
+        if (optionalParameters && 'version' in optionalParameters)
+            openRequest = evalAndLog("indexedDB.open(dbname, " + optionalParameters['version'] + ")");
+        else
+            openRequest = evalAndLog("indexedDB.open(dbname)");
+        shouldBe("openRequest.readyState", "'pending'", true/*quiet*/);
+        openRequest.onerror = unexpectedErrorCallback;
+        openRequest.onupgradeneeded = upgradeCallback;
+        openRequest.onblocked = unexpectedBlockedCallback;
+        if (optionalOpenCallback)
+            openRequest.onsuccess = optionalOpenCallback;
+        delete self.openRequest;
+        if (optionalParameters && 'runAfterOpen' in optionalParameters)
+            (optionalParameters['runAfterOpen'])();
+    };
+}
+
+function waitForRequests(requests, callback) {
+    var count = requests.length;
+
+    if (!count) {
+        callback(requests);
+        return;
+    }
+
+    requests.forEach(function(req) {
+        req.onsuccess = function() {
+            --count;
+            if (!count)
+                callback(requests);
+        };
+        req.onerror = unexpectedErrorCallback;
+    });
+}
diff --git a/LayoutTests/http/tests/IndexedDB/resources/storage-limit.js b/LayoutTests/http/tests/IndexedDB/resources/storage-limit.js
new file mode 100644 (file)
index 0000000..51c15b8
--- /dev/null
@@ -0,0 +1,39 @@
+if (window.testRunner)
+    testRunner.setAllowStorageQuotaIncrease(false);
+
+description("This test makes sure that storage of indexedDB does not grow unboundedly.");
+
+window.caches.open("test").then(cache => {
+    return cache.put(new Request("/test"), new Response(new Uint8Array(204800)));
+}).then(() => {
+    indexedDBTest(prepareDatabase, onOpenSuccess, {'suffix': '-1'});
+}).catch(e => {
+    testFailed("Cache API store operation failed: " + e);
+    finishJSTest();
+});
+
+function prepareDatabase(event)
+{
+    evalAndLog("db = event.target.result");
+    evalAndLog("store = db.createObjectStore('store')");
+}
+
+// Quota for test is 400ko, but IDB is eating some of it when initializing files.
+// Let's make sure that 200ko is fine but 200ko after 200ko is not fine.
+async function onOpenSuccess(event)
+{
+    evalAndLog("db = event.target.result");
+    evalAndLog("store = db.transaction('store', 'readwrite').objectStore('store')");
+    evalAndLog("request = store.add(new Uint8Array(204800), 'key')");
+    request.onerror = function(event) {
+        shouldBeTrue("'error' in request");
+        shouldBe("request.error.code", "DOMException.QUOTA_EXCEEDED_ERR");
+        shouldBeEqualToString("request.error.name", "QuotaExceededError");
+        finishJSTest();
+    }
+
+    request.onsuccess = function(event) {
+        testFailed("Add operation should fail because storage limit is reached, but succeeded.");
+        finishJSTest();
+    }
+}
diff --git a/LayoutTests/http/tests/IndexedDB/storage-limit.https-expected.txt b/LayoutTests/http/tests/IndexedDB/storage-limit.https-expected.txt
new file mode 100644 (file)
index 0000000..996ab19
--- /dev/null
@@ -0,0 +1,21 @@
+This test makes sure that storage of indexedDB does not grow unboundedly.
+
+On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
+
+
+indexedDB = self.indexedDB || self.webkitIndexedDB || self.mozIndexedDB || self.msIndexedDB || self.OIndexedDB;
+
+indexedDB.deleteDatabase(dbname)
+indexedDB.open(dbname)
+db = event.target.result
+store = db.createObjectStore('store')
+db = event.target.result
+store = db.transaction('store', 'readwrite').objectStore('store')
+request = store.add(new Uint8Array(204800), 'key')
+PASS 'error' in request is true
+PASS request.error.code is DOMException.QUOTA_EXCEEDED_ERR
+PASS request.error.name is "QuotaExceededError"
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
diff --git a/LayoutTests/http/tests/IndexedDB/storage-limit.https.html b/LayoutTests/http/tests/IndexedDB/storage-limit.https.html
new file mode 100644 (file)
index 0000000..6b4df80
--- /dev/null
@@ -0,0 +1,9 @@
+<html>
+<head>
+<script src="/js-test-resources/js-test.js"></script>
+<script src="resources/shared.js"></script>
+</head>
+<body>
+<script src="resources/storage-limit.js"></script>
+</body>
+</html>
index a09343e..f0968dc 100644 (file)
@@ -162,6 +162,10 @@ imported/w3c/web-platform-tests/server-timing/service_worker_idl.html [ Skip ]
 imported/w3c/web-platform-tests/fetch/api/request/destination [ Skip ]
 imported/w3c/web-platform-tests/fetch/cross-origin-resource-policy [ Skip ]
 
+# Quota check missing in WK1
+http/tests/IndexedDB/storage-limit.https.html [ Skip ]
+storage/indexeddb/storage-limit.html [ Skip ]
+
 # Skip WebRTC for now in WK1
 imported/w3c/web-platform-tests/webrtc [ Skip ]
 webrtc [ Skip ]
index b30420d..b7ddb67 100644 (file)
@@ -2373,6 +2373,9 @@ http/tests/xmlhttprequest/response-special-characters.html [ Failure ]
 http/tests/xmlhttprequest/web-apps/012.html
 http/tests/xmlhttprequest/web-apps/013.html
 
+storage/indexeddb/storage-limit.html [ Skip ]
+http/tests/IndexedDB/storage-limit.https.html [ Skip ]
+
 # Assertion failures: Not investigated
 [ Debug ] http/tests/security/isolatedWorld/storage-properties.html [ Skip ]
 [ Debug ] http/tests/security/javascriptURL/javascriptURL-execution-context-frame-location-htmldom.html [ Skip ]
index 0d57b6c..81be6d2 100644 (file)
@@ -3,11 +3,11 @@ if (this.importScripts) {
     importScripts('shared.js');
 }
 
-var quota = 1024 * 1024; // 1 MB
-description("This test makes sure that storage of indexedDB does not grow unboundedly.");
-
 if (window.testRunner)
-    testRunner.setIDBPerOriginQuota(quota);
+    testRunner.setAllowStorageQuotaIncrease(false);
+
+var quota = 400 * 1024; // default quota for testing.
+description("This test makes sure that storage of indexedDB does not grow unboundedly.");
 
 indexedDBTest(prepareDatabase, onOpenSuccess);
 
@@ -35,4 +35,4 @@ function onOpenSuccess(event)
         testFailed("Add operation should fail because storage limit is reached, but succeeded.");
         finishJSTest();
     }
-}
\ No newline at end of file
+}
index c314a44..a4cf6a9 100644 (file)
@@ -15,7 +15,7 @@ store = db.createObjectStore('store')
 onOpenSuccess():
 db = event.target.result
 store = db.transaction('store', 'readwrite').objectStore('store')
-request = store.add(new Uint8Array(1048577), 0)
+request = store.add(new Uint8Array(409601), 0)
 PASS 'error' in request is true
 PASS request.error.code is DOMException.QUOTA_EXCEEDED_ERR
 PASS request.error.name is "QuotaExceededError"
index e0a2021..ae10238 100644 (file)
@@ -1,3 +1,110 @@
+2019-03-13  Youenn Fablet  <youenn@apple.com>
+
+        Check IDB quota usage through QuotaManager
+        https://bugs.webkit.org/show_bug.cgi?id=195302
+
+        Reviewed by Chris Dumez.
+
+        For every write operation, compute an estimate size and check for quota before proceeding.
+        When proceeding, store the estimate size in a map.
+        If size of the database is to be computed when the task is not done,
+        the estimate size will be added to the current size of the databases.
+        At the end of the task, the estimate size is removed from the map,
+        and the databases size is refreshed.
+
+        This patch implements size estimation for write tasks.
+        Put/add operations might overestimate the size
+        when an old value will be replaced by a new value.
+        In that case, we do not substract the old value size since we do not know it.
+
+        This patch implements database opening by adding a fixed small cost,
+        as we do not know whether the database is new or not.
+
+        For the first IDB request, we have not computed the size of the database.
+        To do so, we need to go to a background thread and do that file size computation.
+        For that purpose, we add support for being-initialized quota user.
+        Quota manager is calling whenInitialized on its quota user and will
+        delay any quota check requests until its quota user is answering this callback.
+
+        For in process IDB, use the default storage quota per origin and do not increase it.
+        Future work should move it to NetworkProcess and implement some quota checking.
+
+        Cache API and IDB quota management are not yet fully unified.
+        If IDB is used on start-up, we should check for Cache API storage size.
+        Conversely, on Cache API first wite task, even if IDB is not being used,
+        we should compute the size of the IDB data for the given origin.
+
+        Test: http/tests/IndexedDB/storage-limit.https.html
+
+        * Modules/indexeddb/server/IDBBackingStore.h:
+        * Modules/indexeddb/server/IDBServer.cpp:
+        (WebCore::IDBServer::IDBServer::create):
+        (WebCore::IDBServer::IDBServer::IDBServer):
+        (WebCore::IDBServer::m_quotaManagerGetter):
+        (WebCore::IDBServer::IDBServer::QuotaUser::QuotaUser):
+        (WebCore::IDBServer::IDBServer::QuotaUser::~QuotaUser):
+        (WebCore::IDBServer::IDBServer::QuotaUser::clearSpaceUsed):
+        (WebCore::IDBServer::IDBServer::QuotaUser::whenInitialized):
+        (WebCore::IDBServer::IDBServer::QuotaUser::initializeSpaceUsed):
+        (WebCore::IDBServer::IDBServer::quotaUser):
+        (WebCore::IDBServer::IDBServer::startComputingSpaceUsedForOrigin):
+        (WebCore::IDBServer::IDBServer::computeSpaceUsedForOrigin):
+        (WebCore::IDBServer::IDBServer::finishComputingSpaceUsedForOrigin):
+        (WebCore::IDBServer::IDBServer::requestSpace):
+        (WebCore::IDBServer::IDBServer::clearSpaceUsed):
+        (WebCore::IDBServer::IDBServer::setSpaceUsed):
+        (WebCore::IDBServer::IDBServer::increasePotentialSpaceUsed):
+        (WebCore::IDBServer::IDBServer::decreasePotentialSpaceUsed):
+        * Modules/indexeddb/server/IDBServer.h:
+        (WebCore::IDBServer::IDBServer::create):
+        * Modules/indexeddb/server/MemoryIDBBackingStore.cpp:
+        (WebCore::IDBServer::MemoryIDBBackingStore::databasesSizeForOrigin const):
+        * Modules/indexeddb/server/MemoryIDBBackingStore.h:
+        * Modules/indexeddb/server/SQLiteIDBBackingStore.cpp:
+        (WebCore::IDBServer::SQLiteIDBBackingStore::databasesSizeForFolder):
+        (WebCore::IDBServer::SQLiteIDBBackingStore::databasesSizeForOrigin const):
+        (WebCore::IDBServer::SQLiteIDBBackingStore::maximumSize const):
+        * Modules/indexeddb/server/SQLiteIDBBackingStore.h:
+        * Modules/indexeddb/server/UniqueIDBDatabase.cpp:
+        (WebCore::IDBServer::estimateSize):
+        (WebCore::IDBServer::UniqueIDBDatabase::UniqueIDBDatabase):
+        (WebCore::IDBServer::quotaErrorMessageName):
+        (WebCore::IDBServer::UniqueIDBDatabase::requestSpace):
+        (WebCore::IDBServer::UniqueIDBDatabase::performCurrentOpenOperation):
+        (WebCore::IDBServer::UniqueIDBDatabase::storeCallbackOrFireError):
+        (WebCore::IDBServer::UniqueIDBDatabase::createObjectStore):
+        (WebCore::IDBServer::UniqueIDBDatabase::createObjectStoreAfterQuotaCheck):
+        (WebCore::IDBServer::UniqueIDBDatabase::renameObjectStore):
+        (WebCore::IDBServer::UniqueIDBDatabase::renameObjectStoreAfterQuotaCheck):
+        (WebCore::IDBServer::UniqueIDBDatabase::createIndex):
+        (WebCore::IDBServer::UniqueIDBDatabase::createIndexAfterQuotaCheck):
+        (WebCore::IDBServer::UniqueIDBDatabase::renameIndex):
+        (WebCore::IDBServer::UniqueIDBDatabase::renameIndexAfterQuotaCheck):
+        (WebCore::IDBServer::UniqueIDBDatabase::putOrAdd):
+        (WebCore::IDBServer::UniqueIDBDatabase::putOrAddAfterQuotaCheck):
+        (WebCore::IDBServer::UniqueIDBDatabase::postDatabaseTaskReply):
+        (WebCore::IDBServer::UniqueIDBDatabase::immediateCloseForUserDelete):
+        (WebCore::IDBServer::UniqueIDBDatabase::updateSpaceUsedIfNeeded):
+        (WebCore::IDBServer::UniqueIDBDatabase::performErrorCallback):
+        (WebCore::IDBServer::UniqueIDBDatabase::performKeyDataCallback):
+        * Modules/indexeddb/server/UniqueIDBDatabase.h:
+        (WebCore::IDBServer::UniqueIDBDatabase::server):
+        * Modules/indexeddb/shared/InProcessIDBServer.cpp:
+        (WebCore::InProcessIDBServer::create):
+        (WebCore::InProcessIDBServer::quotaManager):
+        (WebCore::storageQuotaManagerGetter):
+        (WebCore::InProcessIDBServer::InProcessIDBServer):
+        * Modules/indexeddb/shared/InProcessIDBServer.h:
+        * loader/EmptyClients.cpp:
+        * storage/StorageQuotaManager.cpp:
+        (WebCore::StorageQuotaManager::addUser):
+        (WebCore::StorageQuotaManager::requestSpace):
+        * storage/StorageQuotaManager.h:
+        (WebCore::StorageQuotaManager::defaultQuota):
+        (WebCore::StorageQuotaManager::removeUser):
+        * storage/StorageQuotaUser.h:
+        (WebCore::StorageQuotaUser::whenInitialized):
+
 2019-03-13  Chris Dumez  <cdumez@apple.com>
 
         Better build fix after r242901.
index 4f61c72..c6d7d43 100644 (file)
@@ -99,6 +99,7 @@ public:
     virtual bool supportsSimultaneousTransactions() = 0;
     virtual bool isEphemeral() = 0;
 
+    virtual uint64_t databasesSizeForOrigin() const = 0;
     virtual void setQuota(uint64_t) = 0;
 protected:
     IDBBackingStore() { RELEASE_ASSERT(!isMainThread()); }
index 2da4f16..8271ed8 100644 (file)
@@ -35,6 +35,7 @@
 #include "SQLiteFileSystem.h"
 #include "SQLiteIDBBackingStore.h"
 #include "SecurityOrigin.h"
+#include "StorageQuotaManager.h"
 #include <wtf/CrossThreadCopier.h>
 #include <wtf/Locker.h>
 #include <wtf/MainThread.h>
 namespace WebCore {
 namespace IDBServer {
 
-Ref<IDBServer> IDBServer::create(IDBBackingStoreTemporaryFileHandler& fileHandler, WTF::Function<void(bool)>&& isClosingDatabaseCallback)
+Ref<IDBServer> IDBServer::create(PAL::SessionID sessionID, IDBBackingStoreTemporaryFileHandler& fileHandler, QuotaManagerGetter&& quotaManagerGetter, WTF::Function<void(bool)>&& isClosingDatabaseCallback)
 {
-    return adoptRef(*new IDBServer(fileHandler, WTFMove(isClosingDatabaseCallback)));
+    return adoptRef(*new IDBServer(sessionID, fileHandler, WTFMove(quotaManagerGetter), WTFMove(isClosingDatabaseCallback)));
 }
 
-Ref<IDBServer> IDBServer::create(const String& databaseDirectoryPath, IDBBackingStoreTemporaryFileHandler& fileHandler, WTF::Function<void(bool)>&& isClosingDatabaseCallback)
+Ref<IDBServer> IDBServer::create(PAL::SessionID sessionID, const String& databaseDirectoryPath, IDBBackingStoreTemporaryFileHandler& fileHandler, QuotaManagerGetter&& quotaManagerGetter, WTF::Function<void(bool)>&& isClosingDatabaseCallback)
 {
-    return adoptRef(*new IDBServer(databaseDirectoryPath, fileHandler, WTFMove(isClosingDatabaseCallback)));
+    return adoptRef(*new IDBServer(sessionID, databaseDirectoryPath, fileHandler, WTFMove(quotaManagerGetter), WTFMove(isClosingDatabaseCallback)));
 }
 
-IDBServer::IDBServer(IDBBackingStoreTemporaryFileHandler& fileHandler, WTF::Function<void(bool)>&& isClosingDatabaseCallback)
+IDBServer::IDBServer(PAL::SessionID sessionID, IDBBackingStoreTemporaryFileHandler& fileHandler, QuotaManagerGetter&& quotaManagerGetter, WTF::Function<void(bool)>&& isClosingDatabaseCallback)
     : CrossThreadTaskHandler("IndexedDatabase Server")
+    , m_sessionID(sessionID)
     , m_backingStoreTemporaryFileHandler(fileHandler)
     , m_isClosingDatabaseCallback(WTFMove(isClosingDatabaseCallback))
     , m_isClosingDatabaseHysteresis([&](PAL::HysteresisState state) { m_isClosingDatabaseCallback(state == PAL::HysteresisState::Started); })
+    , m_quotaManagerGetter(WTFMove(quotaManagerGetter))
 {
 }
 
-IDBServer::IDBServer(const String& databaseDirectoryPath, IDBBackingStoreTemporaryFileHandler& fileHandler, WTF::Function<void(bool)>&& isClosingDatabaseCallback)
+IDBServer::IDBServer(PAL::SessionID sessionID, const String& databaseDirectoryPath, IDBBackingStoreTemporaryFileHandler& fileHandler, QuotaManagerGetter&& quotaManagerGetter, WTF::Function<void(bool)>&& isClosingDatabaseCallback)
     : CrossThreadTaskHandler("IndexedDatabase Server")
+    , m_sessionID(sessionID)
     , m_databaseDirectoryPath(databaseDirectoryPath)
     , m_backingStoreTemporaryFileHandler(fileHandler)
     , m_isClosingDatabaseCallback(WTFMove(isClosingDatabaseCallback))
     , m_isClosingDatabaseHysteresis([&](PAL::HysteresisState state) { m_isClosingDatabaseCallback(state == PAL::HysteresisState::Started); })
+    , m_quotaManagerGetter(WTFMove(quotaManagerGetter))
 {
     LOG(IndexedDB, "IDBServer created at path %s", databaseDirectoryPath.utf8().data());
 }
@@ -679,6 +684,125 @@ void IDBServer::didCloseDatabase(UniqueIDBDatabase* database)
     }
 }
 
+IDBServer::QuotaUser::QuotaUser(IDBServer& server, StorageQuotaManager* manager, ClientOrigin&& origin)
+    : m_server(server)
+    , m_manager(makeWeakPtr(manager))
+    , m_origin(WTFMove(origin))
+    , m_isInitialized(m_server.m_sessionID.isEphemeral())
+{
+    if (manager)
+        manager->addUser(*this);
+}
+
+IDBServer::QuotaUser::~QuotaUser()
+{
+    if (m_manager)
+        m_manager->removeUser(*this);
+}
+
+void IDBServer::QuotaUser::resetSpaceUsed()
+{
+    m_spaceUsed = 0;
+    m_estimatedSpaceIncrease = 0;
+
+    if (!m_manager)
+        return;
+
+    if (m_server.m_sessionID.isEphemeral())
+        return;
+
+    if (!m_isInitialized)
+        return;
+
+    ASSERT(!m_initializationCallback);
+
+    m_isInitialized = false;
+
+    // Do add/remove to trigger call to whenInitialized.
+    m_manager->removeUser(*this);
+    m_manager->addUser(*this);
+}
+
+void IDBServer::QuotaUser::whenInitialized(CompletionHandler<void()>&& callback)
+{
+    if (m_isInitialized) {
+        callback();
+        return;
+    }
+    m_initializationCallback = WTFMove(callback);
+    m_server.startComputingSpaceUsedForOrigin(m_origin);
+}
+
+void IDBServer::QuotaUser::initializeSpaceUsed(uint64_t spaceUsed)
+{
+    ASSERT(m_isInitialized || !m_estimatedSpaceIncrease);
+    m_spaceUsed = spaceUsed;
+    m_isInitialized = true;
+
+    if (auto callback = WTFMove(m_initializationCallback))
+        callback();
+}
+
+IDBServer::QuotaUser& IDBServer::quotaUser(const ClientOrigin& origin)
+{
+    return *m_quotaUsers.ensure(origin, [this, &origin] {
+        return std::make_unique<QuotaUser>(*this, m_quotaManagerGetter(m_sessionID, origin), ClientOrigin { origin });
+    }).iterator->value;
+}
+
+void IDBServer::startComputingSpaceUsedForOrigin(const ClientOrigin& origin)
+{
+    ASSERT(!m_sessionID.isEphemeral());
+    postDatabaseTask(createCrossThreadTask(*this, &IDBServer::computeSpaceUsedForOrigin, origin));
+}
+
+void IDBServer::computeSpaceUsedForOrigin(const ClientOrigin& origin)
+{
+    ASSERT(!isMainThread());
+
+    auto path = IDBDatabaseIdentifier::databaseDirectoryRelativeToRoot(origin.topOrigin, origin.clientOrigin, m_databaseDirectoryPath);
+    auto size = SQLiteIDBBackingStore::databasesSizeForFolder(path);
+
+    postDatabaseTaskReply(createCrossThreadTask(*this, &IDBServer::finishComputingSpaceUsedForOrigin, origin, size));
+}
+
+void IDBServer::finishComputingSpaceUsedForOrigin(const ClientOrigin& origin, uint64_t spaceUsed)
+{
+    quotaUser(origin).initializeSpaceUsed(spaceUsed);
+}
+
+void IDBServer::requestSpace(const ClientOrigin& origin, uint64_t taskSize, CompletionHandler<void(StorageQuotaManager::Decision)>&& callback)
+{
+    auto* quotaManager = quotaUser(origin).manager();
+    if (!quotaManager) {
+        callback(StorageQuotaManager::Decision::Deny);
+        return;
+    }
+
+    quotaManager->requestSpace(taskSize, WTFMove(callback));
+}
+
+void IDBServer::resetSpaceUsed(const ClientOrigin& origin)
+{
+    if (auto* user = m_quotaUsers.get(origin))
+        user->resetSpaceUsed();
+}
+
+void IDBServer::setSpaceUsed(const ClientOrigin& origin, uint64_t taskSize)
+{
+    quotaUser(origin).setSpaceUsed(taskSize);
+}
+
+void IDBServer::increasePotentialSpaceUsed(const ClientOrigin& origin, uint64_t taskSize)
+{
+    quotaUser(origin).increasePotentialSpaceUsed(taskSize);
+}
+
+void IDBServer::decreasePotentialSpaceUsed(const ClientOrigin& origin, uint64_t spaceUsed)
+{
+    quotaUser(origin).decreasePotentialSpaceUsed(spaceUsed);
+}
+
 } // namespace IDBServer
 } // namespace WebCore
 
index f6f55d3..011da2e 100644 (file)
 
 #include "IDBConnectionToClient.h"
 #include "IDBDatabaseIdentifier.h"
+#include "StorageQuotaManager.h"
+#include "StorageQuotaUser.h"
 #include "UniqueIDBDatabase.h"
 #include "UniqueIDBDatabaseConnection.h"
 #include <pal/HysteresisActivity.h>
+#include <pal/SessionID.h>
 #include <wtf/CrossThreadTaskHandler.h>
 #include <wtf/HashMap.h>
 #include <wtf/Lock.h>
@@ -44,6 +47,7 @@ namespace WebCore {
 class IDBCursorInfo;
 class IDBRequestData;
 class IDBValue;
+class StorageQuotaManager;
 
 struct IDBGetRecordData;
 
@@ -55,8 +59,9 @@ class IDBBackingStoreTemporaryFileHandler;
 
 class IDBServer : public RefCounted<IDBServer>, public CrossThreadTaskHandler {
 public:
-    static Ref<IDBServer> create(IDBBackingStoreTemporaryFileHandler&, WTF::Function<void(bool)>&& isClosingDatabaseCallback = [](bool) { });
-    WEBCORE_EXPORT static Ref<IDBServer> create(const String& databaseDirectoryPath, IDBBackingStoreTemporaryFileHandler&, WTF::Function<void(bool)>&& isClosingDatabaseCallback = [](bool) { });
+    using QuotaManagerGetter = WTF::Function<StorageQuotaManager*(PAL::SessionID, const ClientOrigin&)>;
+    static Ref<IDBServer> create(PAL::SessionID, IDBBackingStoreTemporaryFileHandler&, QuotaManagerGetter&&, WTF::Function<void(bool)>&& isClosingDatabaseCallback = [](bool) { });
+    WEBCORE_EXPORT static Ref<IDBServer> create(PAL::SessionID, const String& databaseDirectoryPath, IDBBackingStoreTemporaryFileHandler&, QuotaManagerGetter&&, WTF::Function<void(bool)>&& isClosingDatabaseCallback = [](bool) { });
 
     WEBCORE_EXPORT void registerConnection(IDBConnectionToClient&);
     WEBCORE_EXPORT void unregisterConnection(IDBConnectionToClient&);
@@ -114,9 +119,15 @@ public:
     void didCloseDatabase(UniqueIDBDatabase*);
     void hysteresisUpdated(PAL::HysteresisState);
 
+    void requestSpace(const ClientOrigin&, uint64_t taskSize, CompletionHandler<void(StorageQuotaManager::Decision)>&&);
+    void increasePotentialSpaceUsed(const ClientOrigin&, uint64_t taskSize);
+    void decreasePotentialSpaceUsed(const ClientOrigin&, uint64_t taskSize);
+    void setSpaceUsed(const ClientOrigin&, uint64_t spaceUsed);
+    void resetSpaceUsed(const ClientOrigin&);
+
 private:
-    IDBServer(IDBBackingStoreTemporaryFileHandler&, WTF::Function<void(bool)>&&);
-    IDBServer(const String& databaseDirectoryPath, IDBBackingStoreTemporaryFileHandler&, WTF::Function<void(bool)>&&);
+    IDBServer(PAL::SessionID, IDBBackingStoreTemporaryFileHandler&, QuotaManagerGetter&&, WTF::Function<void(bool)>&&);
+    IDBServer(PAL::SessionID, const String& databaseDirectoryPath, IDBBackingStoreTemporaryFileHandler&, QuotaManagerGetter&&, WTF::Function<void(bool)>&&);
 
     UniqueIDBDatabase& getOrCreateUniqueIDBDatabase(const IDBDatabaseIdentifier&);
 
@@ -127,6 +138,45 @@ private:
     void performCloseAndDeleteDatabasesForOrigins(const Vector<SecurityOriginData>&, uint64_t callbackID);
     void didPerformCloseAndDeleteDatabases(uint64_t callbackID);
 
+    class QuotaUser final : public StorageQuotaUser {
+        WTF_MAKE_FAST_ALLOCATED;
+    public:
+        QuotaUser(IDBServer&, StorageQuotaManager*, ClientOrigin&&);
+        ~QuotaUser();
+
+        StorageQuotaManager* manager() { return m_manager.get(); }
+
+        void setSpaceUsed(uint64_t spaceUsed) { m_spaceUsed = spaceUsed; }
+        void resetSpaceUsed();
+
+        void increasePotentialSpaceUsed(uint64_t increase) { m_estimatedSpaceIncrease += increase; }
+        void decreasePotentialSpaceUsed(uint64_t decrease)
+        {
+            ASSERT(m_estimatedSpaceIncrease >= decrease);
+            m_estimatedSpaceIncrease -= decrease;
+        }
+
+        void initializeSpaceUsed(uint64_t spaceUsed);
+
+    private:
+        uint64_t spaceUsed() const final { return m_spaceUsed + m_estimatedSpaceIncrease; }
+        void whenInitialized(CompletionHandler<void()>&&) final;
+
+        IDBServer& m_server;
+        WeakPtr<StorageQuotaManager> m_manager;
+        ClientOrigin m_origin;
+        bool m_isInitialized { false };
+        uint64_t m_spaceUsed { 0 };
+        uint64_t m_estimatedSpaceIncrease { 0 };
+        CompletionHandler<void()> m_initializationCallback;
+    };
+
+    QuotaUser& quotaUser(const ClientOrigin&);
+    void startComputingSpaceUsedForOrigin(const ClientOrigin&);
+    void computeSpaceUsedForOrigin(const ClientOrigin&);
+    void finishComputingSpaceUsedForOrigin(const ClientOrigin&, uint64_t spaceUsed);
+
+    PAL::SessionID m_sessionID;
     HashMap<uint64_t, RefPtr<IDBConnectionToClient>> m_connectionMap;
     HashMap<IDBDatabaseIdentifier, std::unique_ptr<UniqueIDBDatabase>> m_uniqueIDBDatabaseMap;
     HashSet<UniqueIDBDatabase*> m_uniqueIDBDatabasesInClose;
@@ -143,6 +193,9 @@ private:
 
     WTF::Function<void(bool)> m_isClosingDatabaseCallback;
     PAL::HysteresisActivity m_isClosingDatabaseHysteresis;
+
+    HashMap<ClientOrigin, std::unique_ptr<QuotaUser>> m_quotaUsers;
+    QuotaManagerGetter m_quotaManagerGetter;
 };
 
 } // namespace IDBServer
index 953cace..f6d1658 100644 (file)
@@ -592,6 +592,12 @@ void MemoryIDBBackingStore::deleteBackingStore()
     // The in-memory IDB backing store doesn't need to do any cleanup when it is deleted.
 }
 
+uint64_t MemoryIDBBackingStore::databasesSizeForOrigin() const
+{
+    // FIXME: Implement this.
+    return 0;
+}
+
 } // namespace IDBServer
 } // namespace WebCore
 
index 7806730..64e6cc7 100644 (file)
@@ -80,6 +80,7 @@ public:
     bool isEphemeral() final { return true; }
 
     void setQuota(uint64_t quota) final { UNUSED_PARAM(quota); };
+    uint64_t databasesSizeForOrigin() const final;
 
     void removeObjectStoreForVersionChangeAbort(MemoryObjectStore&);
     void restoreObjectStoreForVersionChangeAbort(Ref<MemoryObjectStore>&&);
index 7415ec8..5989cb3 100644 (file)
@@ -850,6 +850,21 @@ uint64_t SQLiteIDBBackingStore::quotaForOrigin() const
     return std::min(diskFreeSpaceSize / 2, m_quota);
 }
 
+uint64_t SQLiteIDBBackingStore::databasesSizeForFolder(const String& folder)
+{
+    uint64_t diskUsage = 0;
+    for (auto& directory : FileSystem::listDirectory(folder, "*")) {
+        for (auto& file : FileSystem::listDirectory(directory, "*.sqlite3"_s))
+            diskUsage += SQLiteFileSystem::getDatabaseFileSize(file);
+    }
+    return diskUsage;
+}
+
+uint64_t SQLiteIDBBackingStore::databasesSizeForOrigin() const
+{
+    return databasesSizeForFolder(m_absoluteDatabaseDirectory);
+}
+
 uint64_t SQLiteIDBBackingStore::maximumSize() const
 {
     ASSERT(!isMainThread());
@@ -859,11 +874,7 @@ uint64_t SQLiteIDBBackingStore::maximumSize() const
     uint64_t databaseFileSize = SQLiteFileSystem::getDatabaseFileSize(fullDatabasePath());
     uint64_t quota = quotaForOrigin();
 
-    uint64_t diskUsage = 0;
-    for (auto& directory : FileSystem::listDirectory(m_absoluteDatabaseDirectory, "*")) {
-        for (auto& file : FileSystem::listDirectory(directory, "*.sqlite3"_s))
-            diskUsage += SQLiteFileSystem::getDatabaseFileSize(file);
-    }
+    uint64_t diskUsage = databasesSizeForOrigin();
     ASSERT(diskUsage >= databaseFileSize);
 
     if (quota < diskUsage)
index 14004b3..4f974ba 100644 (file)
@@ -82,6 +82,7 @@ public:
     void deleteBackingStore() final;
 
     void setQuota(uint64_t quota) final { m_quota = quota; }
+    uint64_t databasesSizeForOrigin() const final;
 
     bool supportsSimultaneousTransactions() final { return false; }
     bool isEphemeral() final { return false; }
@@ -95,6 +96,7 @@ public:
     IDBError getBlobRecordsForObjectStoreRecord(int64_t objectStoreRecord, Vector<String>& blobURLs, PAL::SessionID&, Vector<String>& blobFilePaths);
 
     static String databaseNameFromEncodedFilename(const String&);
+    static uint64_t databasesSizeForFolder(const String& folder);
 
 private:
     String filenameForDatabaseName() const;
index 65a092b..d5b96b0 100644 (file)
@@ -41,6 +41,7 @@
 #include "IDBValue.h"
 #include "Logging.h"
 #include "SerializedScriptValue.h"
+#include "StorageQuotaManager.h"
 #include "UniqueIDBDatabaseConnection.h"
 #include <JavaScriptCore/AuxiliaryBarrierInlines.h>
 #include <JavaScriptCore/HeapInlines.h>
@@ -54,8 +55,59 @@ namespace WebCore {
 using namespace JSC;
 namespace IDBServer {
 
+static const uint64_t defaultWriteOperationCost = 4;
+
+static inline uint64_t estimateSize(const IDBKeyData& keyData)
+{
+    uint64_t size = 4;
+    switch (keyData.type()) {
+    case IndexedDB::KeyType::String:
+        size += keyData.string().sizeInBytes();
+        break;
+    case IndexedDB::KeyType::Binary: {
+        size += keyData.binary().size();
+        break;
+    }
+    case IndexedDB::KeyType::Array:
+        for (auto& data : keyData.array())
+            size += estimateSize(data);
+        break;
+    default:
+        break;
+    }
+    return size;
+}
+
+static inline uint64_t estimateSize(const IDBValue& value)
+{
+    uint64_t size = 4;
+    size += value.data().size();
+    for (auto& url : value.blobURLs())
+        size += url.sizeInBytes();
+    for (auto& path : value.blobFilePaths())
+        size += path.sizeInBytes();
+    return size;
+}
+
+static inline uint64_t estimateSize(const IDBIndexInfo& info)
+{
+    uint64_t size = 4;
+    size += info.name().sizeInBytes();
+    return size;
+}
+
+static inline uint64_t estimateSize(const IDBObjectStoreInfo& info)
+{
+    uint64_t size = 4;
+    size += info.name().sizeInBytes();
+    // FIXME: estimate keyPath.
+    for (auto& indexInfo : info.indexMap().values())
+        size += estimateSize(indexInfo);
+    return size;
+}
+
 UniqueIDBDatabase::UniqueIDBDatabase(IDBServer& server, const IDBDatabaseIdentifier& identifier)
-    : m_server(&server)
+    : m_server(server)
     , m_identifier(identifier)
     , m_operationAndTransactionTimer(*this, &UniqueIDBDatabase::operationAndTransactionTimerFired)
 {
@@ -118,6 +170,28 @@ bool UniqueIDBDatabase::isVersionChangeInProgress()
     return m_versionChangeDatabaseConnection;
 }
 
+static inline String quotaErrorMessageName(const char* taskName)
+{
+    return makeString("Failed to ", taskName, " in database because not enough space for domain");
+}
+
+void UniqueIDBDatabase::requestSpace(uint64_t taskSize, const char* taskName, CompletionHandler<void(Optional<IDBError>&&)>&& callback)
+{
+    m_server->requestSpace(m_identifier.origin(), taskSize, [weakThis = makeWeakPtr(this), taskName, callback = WTFMove(callback)](auto decision) mutable {
+        if (!weakThis) {
+            callback(IDBError { UnknownError });
+            return;
+        }
+        switch (decision) {
+        case StorageQuotaManager::Decision::Deny:
+            callback(IDBError { QuotaExceededError, quotaErrorMessageName(taskName) });
+            return;
+        case StorageQuotaManager::Decision::Grant:
+            callback({ });
+        };
+    });
+}
+
 void UniqueIDBDatabase::performCurrentOpenOperation()
 {
     LOG(IndexedDB, "(main) UniqueIDBDatabase::performCurrentOpenOperation (%p)", this);
@@ -128,9 +202,24 @@ void UniqueIDBDatabase::performCurrentOpenOperation()
     if (!m_databaseInfo) {
         if (!m_isOpeningBackingStore) {
             m_isOpeningBackingStore = true;
-            postDatabaseTask(createCrossThreadTask(*this, &UniqueIDBDatabase::openBackingStore, m_identifier));
+            // We do not know whether this is an existing or a new database.
+            // We set a small cost so that it is not possible to open an infinite number of database.
+            m_server->requestSpace(m_identifier.origin(), defaultWriteOperationCost, [this, weakThis = makeWeakPtr(this)](auto decision) mutable {
+                if (!weakThis)
+                    return;
+
+                switch (decision) {
+                case StorageQuotaManager::Decision::Deny: {
+                    auto result = IDBResultData::error(m_currentOpenDBRequest->requestData().requestIdentifier(), IDBError { QuotaExceededError, quotaErrorMessageName("openDatabase") });
+                    m_currentOpenDBRequest->connection().didOpenDatabase(result);
+                    m_currentOpenDBRequest = nullptr;
+                    break;
+                }
+                case StorageQuotaManager::Decision::Grant:
+                    this->postDatabaseTask(createCrossThreadTask(*this, &UniqueIDBDatabase::openBackingStore, m_identifier));
+                };
+            });
         }
-
         return;
     }
 
@@ -427,7 +516,7 @@ static uint64_t generateUniqueCallbackIdentifier()
     return ++currentID;
 }
 
-uint64_t UniqueIDBDatabase::storeCallbackOrFireError(ErrorCallback&& callback)
+uint64_t UniqueIDBDatabase::storeCallbackOrFireError(ErrorCallback&& callback, uint64_t taskSize)
 {
     if (m_hardClosedForUserDelete) {
         callback(IDBError::userDeleteError());
@@ -437,11 +526,17 @@ uint64_t UniqueIDBDatabase::storeCallbackOrFireError(ErrorCallback&& callback)
     uint64_t identifier = generateUniqueCallbackIdentifier();
     ASSERT(!m_errorCallbacks.contains(identifier));
     m_errorCallbacks.add(identifier, WTFMove(callback));
+
+    if (taskSize) {
+        m_server->increasePotentialSpaceUsed(m_identifier.origin(), taskSize);
+        m_pendingSpaceIncreasingTasks.add(identifier, taskSize);
+    }
+
     m_callbackQueue.append(identifier);
     return identifier;
 }
 
-uint64_t UniqueIDBDatabase::storeCallbackOrFireError(KeyDataCallback&& callback)
+uint64_t UniqueIDBDatabase::storeCallbackOrFireError(KeyDataCallback&& callback, uint64_t taskSize)
 {
     if (m_hardClosedForUserDelete) {
         callback(IDBError::userDeleteError(), { });
@@ -451,6 +546,12 @@ uint64_t UniqueIDBDatabase::storeCallbackOrFireError(KeyDataCallback&& callback)
     uint64_t identifier = generateUniqueCallbackIdentifier();
     ASSERT(!m_keyDataCallbacks.contains(identifier));
     m_keyDataCallbacks.add(identifier, WTFMove(callback));
+
+    if (taskSize) {
+        m_server->increasePotentialSpaceUsed(m_identifier.origin(), taskSize);
+        m_pendingSpaceIncreasingTasks.add(identifier, taskSize);
+    }
+
     m_callbackQueue.append(identifier);
     return identifier;
 }
@@ -668,7 +769,19 @@ void UniqueIDBDatabase::createObjectStore(UniqueIDBDatabaseTransaction& transact
     ASSERT(isMainThread());
     LOG(IndexedDB, "(main) UniqueIDBDatabase::createObjectStore");
 
-    uint64_t callbackID = storeCallbackOrFireError(WTFMove(callback));
+    auto taskSize = defaultWriteOperationCost + estimateSize(info);
+    requestSpace(taskSize, "createObjectStore", [this, taskSize, transaction = makeRef(transaction), info, callback = WTFMove(callback)](auto error) mutable {
+        if (error) {
+            callback(WTFMove(error.value()));
+            return;
+        }
+        this->createObjectStoreAfterQuotaCheck(taskSize, transaction.get(), info, WTFMove(callback));
+    });
+}
+
+void UniqueIDBDatabase::createObjectStoreAfterQuotaCheck(uint64_t taskSize, UniqueIDBDatabaseTransaction& transaction, const IDBObjectStoreInfo& info, ErrorCallback callback)
+{
+    uint64_t callbackID = storeCallbackOrFireError(WTFMove(callback), taskSize);
     if (!callbackID)
         return;
 
@@ -744,7 +857,19 @@ void UniqueIDBDatabase::renameObjectStore(UniqueIDBDatabaseTransaction& transact
     ASSERT(isMainThread());
     LOG(IndexedDB, "(main) UniqueIDBDatabase::renameObjectStore");
 
-    uint64_t callbackID = storeCallbackOrFireError(WTFMove(callback));
+    auto taskSize = defaultWriteOperationCost + newName.sizeInBytes();
+    requestSpace(taskSize, "renameObjectStore", [this, taskSize, transaction = makeRef(transaction), objectStoreIdentifier, newName, callback = WTFMove(callback)](auto error) mutable {
+        if (error) {
+            callback(WTFMove(error.value()));
+            return;
+        }
+        this->renameObjectStoreAfterQuotaCheck(taskSize, transaction.get(), objectStoreIdentifier, newName, WTFMove(callback));
+    });
+}
+
+void UniqueIDBDatabase::renameObjectStoreAfterQuotaCheck(uint64_t taskSize, UniqueIDBDatabaseTransaction& transaction, uint64_t objectStoreIdentifier, const String& newName, ErrorCallback callback)
+{
+    uint64_t callbackID = storeCallbackOrFireError(WTFMove(callback), taskSize);
     if (!callbackID)
         return;
 
@@ -816,7 +941,19 @@ void UniqueIDBDatabase::createIndex(UniqueIDBDatabaseTransaction& transaction, c
     ASSERT(isMainThread());
     LOG(IndexedDB, "(main) UniqueIDBDatabase::createIndex");
 
-    uint64_t callbackID = storeCallbackOrFireError(WTFMove(callback));
+    auto taskSize = defaultWriteOperationCost + estimateSize(info);
+    requestSpace(taskSize, "createIndex", [this, taskSize, transaction = makeRef(transaction), info, callback = WTFMove(callback)](auto error) mutable {
+        if (error) {
+            callback(WTFMove(error.value()));
+            return;
+        }
+        this->createIndexAfterQuotaCheck(taskSize, transaction.get(), info, WTFMove(callback));
+    });
+}
+
+void UniqueIDBDatabase::createIndexAfterQuotaCheck(uint64_t taskSize, UniqueIDBDatabaseTransaction& transaction, const IDBIndexInfo& info, ErrorCallback callback)
+{
+    uint64_t callbackID = storeCallbackOrFireError(WTFMove(callback), taskSize);
     if (!callbackID)
         return;
     postDatabaseTask(createCrossThreadTask(*this, &UniqueIDBDatabase::performCreateIndex, callbackID, transaction.info().identifier(), info));
@@ -903,7 +1040,19 @@ void UniqueIDBDatabase::renameIndex(UniqueIDBDatabaseTransaction& transaction, u
     ASSERT(isMainThread());
     LOG(IndexedDB, "(main) UniqueIDBDatabase::renameIndex");
 
-    uint64_t callbackID = storeCallbackOrFireError(WTFMove(callback));
+    auto taskSize = defaultWriteOperationCost + newName.sizeInBytes();
+    requestSpace(taskSize, "renameIndex", [this, taskSize, transaction = makeRef(transaction), objectStoreIdentifier, indexIdentifier, newName, callback = WTFMove(callback)](auto error) mutable {
+        if (error) {
+            callback(WTFMove(error.value()));
+            return;
+        }
+        this->renameIndexAfterQuotaCheck(taskSize, transaction.get(), objectStoreIdentifier, indexIdentifier, newName, WTFMove(callback));
+    });
+}
+
+void UniqueIDBDatabase::renameIndexAfterQuotaCheck(uint64_t taskSize, UniqueIDBDatabaseTransaction& transaction, uint64_t objectStoreIdentifier, uint64_t indexIdentifier, const String& newName, ErrorCallback callback)
+{
+    uint64_t callbackID = storeCallbackOrFireError(WTFMove(callback), taskSize);
     if (!callbackID)
         return;
 
@@ -957,7 +1106,19 @@ void UniqueIDBDatabase::putOrAdd(const IDBRequestData& requestData, const IDBKey
     ASSERT(isMainThread());
     LOG(IndexedDB, "(main) UniqueIDBDatabase::putOrAdd");
 
-    uint64_t callbackID = storeCallbackOrFireError(WTFMove(callback));
+    auto taskSize = defaultWriteOperationCost + estimateSize(keyData) + estimateSize(value);
+    requestSpace(taskSize, "putOrAdd", [this, taskSize, requestData, keyData, value, callback = WTFMove(callback), overwriteMode](auto error) mutable {
+        if (error) {
+            callback(WTFMove(error.value()), { });
+            return;
+        }
+        this->putOrAddAfterQuotaCheck(taskSize, requestData, keyData, value, overwriteMode, WTFMove(callback));
+    });
+}
+
+void UniqueIDBDatabase::putOrAddAfterQuotaCheck(uint64_t taskSize, const IDBRequestData& requestData, const IDBKeyData& keyData, const IDBValue& value, IndexedDB::ObjectStoreOverwriteMode overwriteMode, KeyDataCallback callback)
+{
+    uint64_t callbackID = storeCallbackOrFireError(WTFMove(callback), taskSize);
     if (!callbackID)
         return;
     postDatabaseTask(createCrossThreadTask(*this, &UniqueIDBDatabase::performPutOrAdd, callbackID, requestData.transactionIdentifier(), requestData.objectStoreIdentifier(), keyData, value, overwriteMode));
@@ -1768,6 +1929,9 @@ void UniqueIDBDatabase::postDatabaseTask(CrossThreadTask&& task)
 
 void UniqueIDBDatabase::postDatabaseTaskReply(CrossThreadTask&& task)
 {
+    // FIXME: We might want to compute total size only for modification operations.
+    if (m_backingStore)
+        m_databasesSizeForOrigin = m_backingStore->databasesSizeForOrigin();
     m_databaseReplyQueue.append(WTFMove(task));
     m_server->postDatabaseTaskReply(createCrossThreadTask(*this, &UniqueIDBDatabase::executeNextDatabaseTaskReply));
 }
@@ -1832,6 +1996,9 @@ void UniqueIDBDatabase::immediateCloseForUserDelete()
 
     ASSERT(isMainThread());
 
+    m_pendingSpaceIncreasingTasks.clear();
+    m_server->resetSpaceUsed(m_identifier.origin());
+
     // Error out all transactions
     for (auto& identifier : copyToVector(m_inProgressTransactions.keys()))
         m_inProgressTransactions.get(identifier)->abortWithoutCallback();
@@ -1902,8 +2069,21 @@ void UniqueIDBDatabase::immediateCloseForUserDelete()
     postDatabaseTask(createCrossThreadTask(*this, &UniqueIDBDatabase::performUnconditionalDeleteBackingStore));
 }
 
+void UniqueIDBDatabase::updateSpaceUsedIfNeeded(uint64_t callbackIdentifier)
+{
+    auto iterator = m_pendingSpaceIncreasingTasks.find(callbackIdentifier);
+    if (iterator == m_pendingSpaceIncreasingTasks.end())
+        return;
+
+    m_server->decreasePotentialSpaceUsed(m_identifier.origin(), iterator->value);
+    m_server->setSpaceUsed(m_identifier.origin(), m_databasesSizeForOrigin);
+    m_pendingSpaceIncreasingTasks.remove(iterator);
+}
+
 void UniqueIDBDatabase::performErrorCallback(uint64_t callbackIdentifier, const IDBError& error)
 {
+    updateSpaceUsedIfNeeded(callbackIdentifier);
+
     auto callback = m_errorCallbacks.take(callbackIdentifier);
     ASSERT(callback || m_hardClosedForUserDelete);
     if (callback) {
@@ -1915,6 +2095,8 @@ void UniqueIDBDatabase::performErrorCallback(uint64_t callbackIdentifier, const
 
 void UniqueIDBDatabase::performKeyDataCallback(uint64_t callbackIdentifier, const IDBError& error, const IDBKeyData& resultKey)
 {
+    updateSpaceUsedIfNeeded(callbackIdentifier);
+
     auto callback = m_keyDataCallbacks.take(callbackIdentifier);
     ASSERT(callback || m_hardClosedForUserDelete);
     if (callback) {
@@ -1959,6 +2141,8 @@ void UniqueIDBDatabase::performCountCallback(uint64_t callbackIdentifier, const
 
 void UniqueIDBDatabase::forgetErrorCallback(uint64_t callbackIdentifier)
 {
+    updateSpaceUsedIfNeeded(callbackIdentifier);
+
     ASSERT(m_errorCallbacks.contains(callbackIdentifier));
     ASSERT(m_callbackQueue.last() == callbackIdentifier);
     m_callbackQueue.removeLast();
index fc094ee..683ae83 100644 (file)
@@ -54,6 +54,7 @@ class IDBError;
 class IDBGetAllResult;
 class IDBRequestData;
 class IDBTransactionInfo;
+class StorageQuotaManager;
 
 enum class IDBGetRecordDataType;
 
@@ -82,7 +83,7 @@ public:
     void openDatabaseConnection(IDBConnectionToClient&, const IDBRequestData&);
 
     const IDBDatabaseInfo& info() const;
-    IDBServer& server() { return *m_server; }
+    IDBServer& server() { return m_server.get(); }
     const IDBDatabaseIdentifier& identifier() const { return m_identifier; }
 
     void createObjectStore(UniqueIDBDatabaseTransaction&, const IDBObjectStoreInfo&, ErrorCallback);
@@ -119,7 +120,10 @@ public:
 
     bool hardClosedForUserDelete() const { return m_hardClosedForUserDelete; }
 
+    uint64_t spaceUsed() const;
+
     void setQuota(uint64_t);
+
 private:
     enum class CloseState { Start, Done };
 
@@ -143,6 +147,12 @@ private:
 
     void scheduleShutdownForClose();
 
+    void createObjectStoreAfterQuotaCheck(uint64_t taskSize, UniqueIDBDatabaseTransaction&, const IDBObjectStoreInfo&, ErrorCallback);
+    void renameObjectStoreAfterQuotaCheck(uint64_t taskSize, UniqueIDBDatabaseTransaction&, uint64_t objectStoreIdentifier, const String& newName, ErrorCallback);
+    void createIndexAfterQuotaCheck(uint64_t taskSize, UniqueIDBDatabaseTransaction&, const IDBIndexInfo&, ErrorCallback);
+    void renameIndexAfterQuotaCheck(uint64_t taskSize, UniqueIDBDatabaseTransaction&, uint64_t objectStoreIdentifier, uint64_t indexIdentifier, const String& newName, ErrorCallback);
+    void putOrAddAfterQuotaCheck(uint64_t taskSize, const IDBRequestData&, const IDBKeyData&, const IDBValue&, IndexedDB::ObjectStoreOverwriteMode, KeyDataCallback);
+
     // Database thread operations
     void deleteBackingStore(const IDBDatabaseIdentifier&);
     void openBackingStore(const IDBDatabaseIdentifier&);
@@ -193,8 +203,8 @@ private:
     void didPerformUnconditionalDeleteBackingStore();
     void didShutdownForClose();
 
-    uint64_t storeCallbackOrFireError(ErrorCallback&&);
-    uint64_t storeCallbackOrFireError(KeyDataCallback&&);
+    uint64_t storeCallbackOrFireError(ErrorCallback&&, uint64_t taskSize = 0);
+    uint64_t storeCallbackOrFireError(KeyDataCallback&&, uint64_t taskSize = 0);
     uint64_t storeCallbackOrFireError(GetAllResultsCallback&&);
     uint64_t storeCallbackOrFireError(GetResultCallback&&);
     uint64_t storeCallbackOrFireError(CountCallback&&);
@@ -229,9 +239,12 @@ private:
 
     void notifyServerAboutClose(CloseState);
 
-    RefPtr<IDBServer> m_server;
+    void requestSpace(uint64_t taskSize, const char* errorMessage, CompletionHandler<void(Optional<IDBError>&&)>&&);
+    void updateSpaceUsedIfNeeded(uint64_t callbackIdentifier);
+
+    Ref<IDBServer> m_server;
     IDBDatabaseIdentifier m_identifier;
-    
+
     ListHashSet<RefPtr<ServerOpenDBRequest>> m_pendingOpenDBRequests;
     RefPtr<ServerOpenDBRequest> m_currentOpenDBRequest;
 
@@ -279,6 +292,9 @@ private:
     std::unique_ptr<UniqueIDBDatabase> m_owningPointerForClose;
 
     HashSet<IDBResourceIdentifier> m_cursorPrefetches;
+
+    HashMap<uint64_t, uint64_t> m_pendingSpaceIncreasingTasks;
+    uint64_t m_databasesSizeForOrigin { 0 };
 };
 
 } // namespace IDBServer
index dd9f5ad..0472867 100644 (file)
 
 namespace WebCore {
 
-Ref<InProcessIDBServer> InProcessIDBServer::create()
+Ref<InProcessIDBServer> InProcessIDBServer::create(PAL::SessionID sessionID)
 {
-    Ref<InProcessIDBServer> server = adoptRef(*new InProcessIDBServer);
+    Ref<InProcessIDBServer> server = adoptRef(*new InProcessIDBServer(sessionID));
     server->m_server->registerConnection(server->connectionToClient());
     return server;
 }
 
-Ref<InProcessIDBServer> InProcessIDBServer::create(const String& databaseDirectoryPath)
+Ref<InProcessIDBServer> InProcessIDBServer::create(PAL::SessionID sessionID, const String& databaseDirectoryPath)
 {
-    Ref<InProcessIDBServer> server = adoptRef(*new InProcessIDBServer(databaseDirectoryPath));
+    Ref<InProcessIDBServer> server = adoptRef(*new InProcessIDBServer(sessionID, databaseDirectoryPath));
     server->m_server->registerConnection(server->connectionToClient());
     return server;
 }
 
-InProcessIDBServer::InProcessIDBServer()
-    : m_server(IDBServer::IDBServer::create(*this))
+StorageQuotaManager* InProcessIDBServer::quotaManager(const ClientOrigin& origin)
+{
+    return m_quotaManagers.ensure(origin, [] {
+        return std::make_unique<StorageQuotaManager>(StorageQuotaManager::defaultQuota(), [](uint64_t quota, uint64_t currentSpace, uint64_t spaceIncrease, auto callback) {
+            callback(quota + currentSpace + spaceIncrease);
+        });
+    }).iterator->value.get();
+}
+
+static inline IDBServer::IDBServer::QuotaManagerGetter storageQuotaManagerGetter(InProcessIDBServer& server)
+{
+    return [&server, weakServer = makeWeakPtr(server)](PAL::SessionID, const auto& origin) {
+        return weakServer ? server.quotaManager(origin) : nullptr;
+    };
+}
+
+InProcessIDBServer::InProcessIDBServer(PAL::SessionID sessionID)
+    : m_server(IDBServer::IDBServer::create(sessionID, *this, storageQuotaManagerGetter(*this)))
 {
     relaxAdoptionRequirement();
     m_connectionToServer = IDBClient::IDBConnectionToServer::create(*this);
     m_connectionToClient = IDBServer::IDBConnectionToClient::create(*this);
 }
 
-InProcessIDBServer::InProcessIDBServer(const String& databaseDirectoryPath)
-    : m_server(IDBServer::IDBServer::create(databaseDirectoryPath, *this))
+InProcessIDBServer::InProcessIDBServer(PAL::SessionID sessionID, const String& databaseDirectoryPath)
+    : m_server(IDBServer::IDBServer::create(sessionID, databaseDirectoryPath, *this, storageQuotaManagerGetter(*this)))
 {
     relaxAdoptionRequirement();
     m_connectionToServer = IDBClient::IDBConnectionToServer::create(*this);
index c7944d1..5483a98 100644 (file)
 #include "IDBConnectionToClient.h"
 #include "IDBConnectionToServer.h"
 #include "IDBServer.h"
+#include "StorageQuotaManager.h"
 #include <wtf/RefCounted.h>
 #include <wtf/RefPtr.h>
+#include <wtf/WeakPtr.h>
+
+namespace PAL {
+class SessionID;
+}
 
 namespace WebCore {
 
+struct ClientOrigin;
+
 namespace IDBClient {
 class IDBConnectionToServer;
 }
@@ -45,8 +53,8 @@ class IDBServer;
 
 class InProcessIDBServer final : public IDBClient::IDBConnectionToServerDelegate, public IDBServer::IDBConnectionToClientDelegate, public RefCounted<InProcessIDBServer>, public IDBServer::IDBBackingStoreTemporaryFileHandler {
 public:
-    WEBCORE_EXPORT static Ref<InProcessIDBServer> create();
-    WEBCORE_EXPORT static Ref<InProcessIDBServer> create(const String& databaseDirectoryPath);
+    WEBCORE_EXPORT static Ref<InProcessIDBServer> create(PAL::SessionID);
+    WEBCORE_EXPORT static Ref<InProcessIDBServer> create(PAL::SessionID, const String& databaseDirectoryPath);
 
     WEBCORE_EXPORT IDBClient::IDBConnectionToServer& connectionToServer() const;
     IDBServer::IDBConnectionToClient& connectionToClient() const;
@@ -114,13 +122,19 @@ public:
 
     void accessToTemporaryFileComplete(const String& path) override;
 
+    StorageQuotaManager* quotaManager(const ClientOrigin&);
+
+    const WeakPtrFactory<IDBClient::IDBConnectionToServerDelegate>& weakPtrFactory() const { return IDBClient::IDBConnectionToServerDelegate::weakPtrFactory(); }
+
 private:
-    InProcessIDBServer();
-    InProcessIDBServer(const String& databaseDirectoryPath);
+    explicit InProcessIDBServer(PAL::SessionID);
+    InProcessIDBServer(PAL::SessionID, const String& databaseDirectoryPath);
 
     Ref<IDBServer::IDBServer> m_server;
     RefPtr<IDBClient::IDBConnectionToServer> m_connectionToServer;
     RefPtr<IDBServer::IDBConnectionToClient> m_connectionToClient;
+
+    HashMap<ClientOrigin, std::unique_ptr<StorageQuotaManager>> m_quotaManagers;
 };
 
 } // namespace WebCore
index d56e337..e53b404 100644 (file)
@@ -118,9 +118,9 @@ class EmptyContextMenuClient final : public ContextMenuClient {
 
 class EmptyDatabaseProvider final : public DatabaseProvider {
 #if ENABLE(INDEXED_DATABASE)
-    IDBClient::IDBConnectionToServer& idbConnectionToServerForSession(const PAL::SessionID&) final
+    IDBClient::IDBConnectionToServer& idbConnectionToServerForSession(const PAL::SessionID& sessionID) final
     {
-        static auto& sharedConnection = InProcessIDBServer::create().leakRef();
+        static auto& sharedConnection = InProcessIDBServer::create(sessionID).leakRef();
         return sharedConnection.connectionToServer();
     }
 #endif
index b595da8..e30a40a 100644 (file)
@@ -44,9 +44,23 @@ uint64_t StorageQuotaManager::spaceUsage() const
     return usage;
 }
 
+void StorageQuotaManager::addUser(StorageQuotaUser& user)
+{
+    ASSERT(!m_pendingInitializationUsers.contains(&user));
+    ASSERT(!m_users.contains(&user));
+    m_pendingInitializationUsers.add(&user);
+    user.whenInitialized([this, &user, weakThis = makeWeakPtr(this)]() {
+        if (!weakThis)
+            return;
+        m_pendingInitializationUsers.remove(&user);
+        m_users.add(&user);
+        processPendingRequests({ });
+    });
+}
+
 void StorageQuotaManager::requestSpace(uint64_t spaceIncrease, RequestCallback&& callback)
 {
-    if (!m_pendingRequests.isEmpty()) {
+    if (!m_pendingRequests.isEmpty() || !m_pendingInitializationUsers.isEmpty()) {
         m_pendingRequests.append({ spaceIncrease, WTFMove(callback) });
         return;
     }
index 2a84437..dc8141b 100644 (file)
@@ -46,15 +46,13 @@ public:
     }
     WEBCORE_EXPORT ~StorageQuotaManager();
 
-    void addUser(StorageQuotaUser& user)
-    {
-        ASSERT(!m_users.contains(&user));
-        m_users.add(&user);
-    }
+    static constexpr uint64_t defaultQuota() { return 500 * MB; }
 
+    WEBCORE_EXPORT void addUser(StorageQuotaUser&);
     void removeUser(StorageQuotaUser& user)
     {
-        ASSERT(m_users.contains(&user));
+        ASSERT(m_users.contains(&user) || m_pendingInitializationUsers.contains(&user));
+        m_pendingInitializationUsers.remove(&user);
         m_users.remove(&user);
     }
 
@@ -69,6 +67,7 @@ private:
 
     uint64_t m_quota { 0 };
     SpaceIncreaseRequester m_spaceIncreaseRequester;
+    HashSet<const StorageQuotaUser*> m_pendingInitializationUsers;
     HashSet<const StorageQuotaUser*> m_users;
 
     struct PendingRequest {
index 3dddfab..b4230c5 100644 (file)
@@ -25,6 +25,8 @@
 
 #pragma once
 
+#include <wtf/CompletionHandler.h>
+
 namespace WebCore {
 
 class StorageQuotaUser {
@@ -32,6 +34,7 @@ public:
     virtual ~StorageQuotaUser() = default;
 
     virtual uint64_t spaceUsed() const = 0;
+    virtual void whenInitialized(CompletionHandler<void()>&& callback) { callback(); }
 };
 
 } // namespace WebCore
index c291322..8012412 100644 (file)
@@ -1,3 +1,19 @@
+2019-03-13  Youenn Fablet  <youenn@apple.com>
+
+        Check IDB quota usage through QuotaManager
+        https://bugs.webkit.org/show_bug.cgi?id=195302
+
+        Reviewed by Chris Dumez.
+
+        Set the quota manager getter for IDBServer at creation time.
+
+        * NetworkProcess/NetworkProcess.cpp:
+        (WebKit::NetworkProcess::createIDBServer):
+        (WebKit::NetworkProcess::idbServer):
+        * NetworkProcess/NetworkProcess.h:
+        * WebProcess/Databases/WebDatabaseProvider.cpp:
+        (WebKit::WebDatabaseProvider::idbConnectionToServerForSession):
+
 2019-03-13  Timothy Hatcher  <timothy@apple.com>
 
         Consolidate ArgumentCodersMac and ArgumentCodersCocoa.
index 937f284..094c00e 100644 (file)
@@ -2035,24 +2035,31 @@ void NetworkProcess::didSyncAllCookies()
 }
 
 #if ENABLE(INDEXED_DATABASE)
-IDBServer::IDBServer& NetworkProcess::idbServer(PAL::SessionID sessionID)
+Ref<IDBServer::IDBServer> NetworkProcess::createIDBServer(PAL::SessionID sessionID)
 {
-    auto addResult = m_idbServers.add(sessionID, nullptr);
-    if (!addResult.isNewEntry) {
-        ASSERT(addResult.iterator->value);
-        return *addResult.iterator->value;
-    }
-    
     auto path = m_idbDatabasePaths.get(sessionID);
     // There should already be a registered path for this PAL::SessionID.
     // If there's not, then where did this PAL::SessionID come from?
     ASSERT(!path.isEmpty());
-    
-    addResult.iterator->value = IDBServer::IDBServer::create(path, *this, [this](bool isHoldingLockedFiles) {
-        notifyHoldingLockedFiles(isHoldingLockedFiles);
+
+    auto server = IDBServer::IDBServer::create(sessionID, path, *this, [this, weakThis = makeWeakPtr(this)](PAL::SessionID sessionID, const auto& origin) -> StorageQuotaManager* {
+        if (!weakThis)
+            return nullptr;
+        return &this->storageQuotaManager(sessionID, origin);
+    }, [this, weakThis = makeWeakPtr(this)](bool isHoldingLockedFiles) {
+        if (!weakThis)
+            return;
+        this->notifyHoldingLockedFiles(isHoldingLockedFiles);
     });
-    addResult.iterator->value->setPerOriginQuota(m_idbPerOriginQuota);
-    return *addResult.iterator->value;
+    server->setPerOriginQuota(m_idbPerOriginQuota);
+    return server;
+}
+
+IDBServer::IDBServer& NetworkProcess::idbServer(PAL::SessionID sessionID)
+{
+    return *m_idbServers.ensure(sessionID, [this, sessionID] {
+        return this->createIDBServer(sessionID);
+    }).iterator->value;
 }
 
 void NetworkProcess::ensurePathExists(const String& path)
index 600f1c1..aea3342 100644 (file)
@@ -430,6 +430,7 @@ private:
 #if ENABLE(INDEXED_DATABASE)
     void addIndexedDatabaseSession(PAL::SessionID, String&, SandboxExtension::Handle&);
     HashSet<WebCore::SecurityOriginData> indexedDatabaseOrigins(const String& path);
+    Ref<WebCore::IDBServer::IDBServer> createIDBServer(PAL::SessionID);
 #endif
 
 #if ENABLE(SERVICE_WORKER)
@@ -524,7 +525,7 @@ private:
 #endif
 
     struct StorageQuotaManagers {
-        uint64_t defaultQuota { 0 };
+        uint64_t defaultQuota { WebCore::StorageQuotaManager::defaultQuota() };
         HashMap<WebCore::ClientOrigin, std::unique_ptr<WebCore::StorageQuotaManager>> managersPerOrigin;
     };
     HashMap<PAL::SessionID, StorageQuotaManagers> m_storageQuotaManagers;
index b0798e9..1c9de65 100644 (file)
@@ -72,7 +72,7 @@ WebCore::IDBClient::IDBConnectionToServer& WebDatabaseProvider::idbConnectionToS
     if (sessionID.isEphemeral()) {
         auto result = m_idbEphemeralConnectionMap.add(sessionID.sessionID(), nullptr);
         if (result.isNewEntry)
-            result.iterator->value = WebCore::InProcessIDBServer::create();
+            result.iterator->value = WebCore::InProcessIDBServer::create(sessionID);
 
         return result.iterator->value->connectionToServer();
     }
index de8f0f8..25fe35b 100644 (file)
@@ -1,3 +1,13 @@
+2019-03-13  Youenn Fablet  <youenn@apple.com>
+
+        Check IDB quota usage through QuotaManager
+        https://bugs.webkit.org/show_bug.cgi?id=195302
+
+        Reviewed by Chris Dumez.
+
+        * Storage/WebDatabaseProvider.cpp:
+        (WebDatabaseProvider::idbConnectionToServerForSession):
+
 2019-03-06  Sam Weinig  <sam@webkit.org>
 
         WebKitLegacy does not need to generate an export file for i386 anymore
index fb71661..5ee5a7b 100644 (file)
@@ -48,9 +48,9 @@ WebCore::IDBClient::IDBConnectionToServer& WebDatabaseProvider::idbConnectionToS
     auto result = m_idbServerMap.add(sessionID.sessionID(), nullptr);
     if (result.isNewEntry) {
         if (sessionID.isEphemeral())
-            result.iterator->value = WebCore::InProcessIDBServer::create();
+            result.iterator->value = WebCore::InProcessIDBServer::create(sessionID);
         else
-            result.iterator->value = WebCore::InProcessIDBServer::create(indexedDatabaseDirectoryPath());
+            result.iterator->value = WebCore::InProcessIDBServer::create(sessionID, indexedDatabaseDirectoryPath());
     }
 
     result.iterator->value->idbServer().setPerOriginQuota(m_idbPerOriginQuota);