IndexedDB: include size of index records in size estimate of put/add task
authorsihui_liu@apple.com <sihui_liu@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 9 Oct 2019 22:20:29 +0000 (22:20 +0000)
committersihui_liu@apple.com <sihui_liu@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 9 Oct 2019 22:20:29 +0000 (22:20 +0000)
https://bugs.webkit.org/show_bug.cgi?id=202483

Reviewed by Geoffrey Garen.

Source/WebCore:

SQLiteIDBBackingStore has two tables IndexRecords and Records. For add operation add(VALUE, KEY), we add a new
record to Records table. If we can extract index value from VALUE for some index, we would add a new record to
IndexRecords table.

We estimated the task szie of add task with (KEY_SIZE + VALUE_SIZE), but we didn't count the size that could be
added to the IndexRecords table. This could lead to storage abuse.

Test: storage/indexeddb/request-size-estimate.html

* Modules/indexeddb/server/UniqueIDBDatabase.cpp:
(WebCore::IDBServer::UniqueIDBDatabase::putOrAdd):

LayoutTests:

* platform/mac-wk1/TestExpectations:
* platform/win/TestExpectations:
* platform/wincairo/TestExpectations:
* storage/indexeddb/request-size-estimate-expected.txt: Added.
* storage/indexeddb/request-size-estimate.html: Added.
* storage/indexeddb/resources/request-size-estimate.js: Added.
(randomKey):
(randomPropertyValue):
(createObject):
(prepareDatabase):
(onOpenSuccess):

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@250936 268f45cc-cd09-0410-ab3c-d52691b4dbfc

LayoutTests/ChangeLog
LayoutTests/platform/mac-wk1/TestExpectations
LayoutTests/platform/win/TestExpectations
LayoutTests/platform/wincairo/TestExpectations
LayoutTests/storage/indexeddb/request-size-estimate-expected.txt [new file with mode: 0644]
LayoutTests/storage/indexeddb/request-size-estimate.html [new file with mode: 0644]
LayoutTests/storage/indexeddb/resources/request-size-estimate.js [new file with mode: 0644]
Source/WebCore/ChangeLog
Source/WebCore/Modules/indexeddb/server/UniqueIDBDatabase.cpp

index 24e55e2..24e246a 100644 (file)
@@ -1,3 +1,22 @@
+2019-10-09  Sihui Liu  <sihui_liu@apple.com>
+
+        IndexedDB: include size of index records in size estimate of put/add task
+        https://bugs.webkit.org/show_bug.cgi?id=202483
+
+        Reviewed by Geoffrey Garen.
+
+        * platform/mac-wk1/TestExpectations:
+        * platform/win/TestExpectations:
+        * platform/wincairo/TestExpectations:
+        * storage/indexeddb/request-size-estimate-expected.txt: Added.
+        * storage/indexeddb/request-size-estimate.html: Added.
+        * storage/indexeddb/resources/request-size-estimate.js: Added.
+        (randomKey):
+        (randomPropertyValue):
+        (createObject):
+        (prepareDatabase):
+        (onOpenSuccess):
+
 2019-10-09  Chris Dumez  <cdumez@apple.com>
 
         Youtube.com is unable to enter the back/forward cache on macOS
index c7b1318..6914ddd 100644 (file)
@@ -304,6 +304,7 @@ http/tests/IndexedDB/storage-limit.https.html [ Skip ]
 http/tests/IndexedDB/storage-limit-1.https.html [ Skip ]
 http/tests/IndexedDB/storage-limit-2.https.html [ Skip ]
 storage/indexeddb/storage-limit.html [ Skip ]
+storage/indexeddb/request-size-estimate.html [ Skip ]
 
 # Skip WebRTC for now in WK1
 imported/w3c/web-platform-tests/webrtc [ Skip ]
index c016acf..0d9e217 100644 (file)
@@ -2417,6 +2417,7 @@ http/tests/xmlhttprequest/web-apps/012.html
 http/tests/xmlhttprequest/web-apps/013.html
 
 storage/indexeddb/storage-limit.html [ Skip ]
+storage/indexeddb/request-size-estimate.html [ Skip ]
 http/tests/IndexedDB/storage-limit.https.html [ Skip ]
 http/tests/IndexedDB/storage-limit-1.https.html [ Skip ]
 http/tests/IndexedDB/storage-limit-2.https.html [ Skip ]
index 71679a7..fda76c2 100644 (file)
@@ -1471,6 +1471,7 @@ storage/indexeddb/modern/idbtransaction-objectstore-failures.html [ Failure Pass
 storage/indexeddb/objectstore-basics.html [ Failure Pass ]
 storage/indexeddb/objectstore-cursor.html [ Pass Timeout ]
 storage/indexeddb/storage-limit.html [ Failure ]
+storage/indexeddb/request-size-estimate.html [ Failure ]
 storage/indexeddb/structured-clone.html [ Pass Timeout ]
 storage/indexeddb/version-change-abort.html [ Failure Pass ]
 storage/indexeddb/wasm-exceptions.html [ Failure ]
diff --git a/LayoutTests/storage/indexeddb/request-size-estimate-expected.txt b/LayoutTests/storage/indexeddb/request-size-estimate-expected.txt
new file mode 100644 (file)
index 0000000..862ccbb
--- /dev/null
@@ -0,0 +1,24 @@
+This test verifies that estimated size of IDB database task is not smaller than or close to actual space increase (maybe subject to our implementation of backing store.)
+
+On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
+
+
+indexedDB = self.indexedDB || self.webkitIndexedDB || self.mozIndexedDB || self.msIndexedDB || self.OIndexedDB;
+
+indexedDB.deleteDatabase(dbname)
+indexedDB.open(dbname)
+
+prepareDatabase():
+Create 20 indexes
+
+onOpenSuccess():
+db = event.target.result
+store = db.transaction('store', 'readwrite').objectStore('store')
+request = store.add(createObject(), randomKey(keySize))
+PASS 'error' in request is true
+PASS request.error.code is DOMException.QUOTA_EXCEEDED_ERR
+PASS request.error.name is "QuotaExceededError"
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
diff --git a/LayoutTests/storage/indexeddb/request-size-estimate.html b/LayoutTests/storage/indexeddb/request-size-estimate.html
new file mode 100644 (file)
index 0000000..cf39b63
--- /dev/null
@@ -0,0 +1,9 @@
+<html>
+<head>
+<script src="../../resources/js-test.js"></script>
+<script src="resources/shared.js"></script>
+</head>
+<body>
+<script src="resources/request-size-estimate.js"></script>
+</body>
+</html>
\ No newline at end of file
diff --git a/LayoutTests/storage/indexeddb/resources/request-size-estimate.js b/LayoutTests/storage/indexeddb/resources/request-size-estimate.js
new file mode 100644 (file)
index 0000000..935a085
--- /dev/null
@@ -0,0 +1,72 @@
+if (this.importScripts) {
+    importScripts('../../../resources/js-test.js');
+    importScripts('shared.js');
+}
+
+if (window.testRunner)
+    testRunner.setAllowStorageQuotaIncrease(false);
+
+description('This test verifies that estimated size of IDB database task is not smaller than or close to actual space increase (maybe subject to our implementation of backing store.)');
+
+const quota = 400 * 1024; // Current default quota for test.
+const indexCount = 20;
+const indexValueSize = 1024;
+const keySize = 1024;
+
+indexedDBTest(prepareDatabase, onOpenSuccess);
+
+function randomKey(length) {
+    var str = "";
+    for (var i = 0; i < length; i++)
+        str += (Math.floor(Math.random() * 10))
+    return str;
+}
+
+function randomPropertyValue(length) {
+    var value = [];
+    for (var i = 0; i < length; i++)
+        value.push(Math.floor(Math.random() * 10))
+    return value;
+}
+
+function createObject() {
+    var obj = { };
+    for (var i = 0; i < indexCount; i++) 
+        obj["index" + i] = randomPropertyValue(indexValueSize);
+
+    return obj;
+}
+
+function prepareDatabase(event)
+{
+    preamble(event);
+    db = event.target.result;
+    store = db.createObjectStore('store');
+    var indexes = [];
+
+    debug("Create " + indexCount + " indexes");
+    // This is an extreme case where each of indexes is composed of all properties of the object.
+    for (var i = 0; i < indexCount; i ++)
+        indexes.push("index" + i);
+    for (var i = 0; i < indexCount; i ++)
+        store.createIndex("allIndexes" + i, indexes);
+}
+
+function onOpenSuccess(event)
+{
+    preamble(event);
+    evalAndLog("db = event.target.result");
+    evalAndLog("store = db.transaction('store', 'readwrite').objectStore('store')");
+    request = evalAndLog("request = store.add(createObject(), randomKey(keySize))");
+    request.onerror = () => {
+        shouldBeTrue("'error' in request");
+        shouldBe("request.error.code", "DOMException.QUOTA_EXCEEDED_ERR");
+        shouldBeEqualToString("request.error.name", "QuotaExceededError");
+
+        finishJSTest();
+    }
+    request.onsuccess = () => {
+        testFailed("Add operation should fail because task size is bigger than space available");
+        finishJSTest();
+    }
+}
\ No newline at end of file
index a5de855..07e1132 100644 (file)
@@ -1,3 +1,22 @@
+2019-10-09  Sihui Liu  <sihui_liu@apple.com>
+
+        IndexedDB: include size of index records in size estimate of put/add task
+        https://bugs.webkit.org/show_bug.cgi?id=202483
+
+        Reviewed by Geoffrey Garen.
+
+        SQLiteIDBBackingStore has two tables IndexRecords and Records. For add operation add(VALUE, KEY), we add a new 
+        record to Records table. If we can extract index value from VALUE for some index, we would add a new record to
+        IndexRecords table.
+
+        We estimated the task szie of add task with (KEY_SIZE + VALUE_SIZE), but we didn't count the size that could be
+        added to the IndexRecords table. This could lead to storage abuse.
+
+        Test: storage/indexeddb/request-size-estimate.html
+
+        * Modules/indexeddb/server/UniqueIDBDatabase.cpp:
+        (WebCore::IDBServer::UniqueIDBDatabase::putOrAdd):
+
 2019-10-09  Chris Dumez  <cdumez@apple.com>
 
         Youtube.com is unable to enter the back/forward cache on macOS
index c939a00..8a635bd 100644 (file)
@@ -1228,6 +1228,11 @@ void UniqueIDBDatabase::putOrAdd(UniqueIDBDatabaseTransaction& transaction, cons
     LOG(IndexedDB, "(main) UniqueIDBDatabase::putOrAdd");
 
     auto taskSize = defaultWriteOperationCost + estimateSize(keyData) + estimateSize(value);
+    ASSERT(m_databaseInfo);
+    auto* objectStore = m_databaseInfo->infoForExistingObjectStore(requestData.objectStoreIdentifier());
+    if (objectStore)
+        taskSize += objectStore->indexNames().size() * taskSize;
+
     requestSpace(transaction, taskSize, "putOrAdd", [this, taskSize, requestData, keyData, value, callback = WTFMove(callback), overwriteMode](auto error) mutable {
         if (!error.isNull() && *error.code() != QuotaExceededError) {
             callback(WTFMove(error), { });