IndexedDB: speed up index records deletion
authorsihui_liu@apple.com <sihui_liu@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 28 Jan 2020 23:17:33 +0000 (23:17 +0000)
committersihui_liu@apple.com <sihui_liu@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 28 Jan 2020 23:17:33 +0000 (23:17 +0000)
https://bugs.webkit.org/show_bug.cgi?id=206196
PerformanceTests:

Reviewed by Brady Eidson.

* IndexedDB/basic/index-cursor-delete-2.html: Added.

Source/WebCore:

<rdar://problem/53596307>

Reviewed by Brady Eidson.

This patch does a few things to accelerate deletion for index records:
1. make indexID unique in database instead of objectStore
2. create an index on IndexRecords table
3. optimize some SQLite statements to take advantage of 1 and 2

Test: IndexedDB.IDBObjectStoreInfoUpgradeToV2

Make test PerformanceTests/IndexedDB/basic/index-cursor-delete-2.html 7.5x faster.

* Modules/indexeddb/IDBDatabase.h:
* Modules/indexeddb/IDBObjectStore.cpp:
(WebCore::IDBObjectStore::createIndex):
* Modules/indexeddb/server/MemoryIDBBackingStore.cpp:
(WebCore::IDBServer::MemoryIDBBackingStore::getOrEstablishDatabaseInfo):
(WebCore::IDBServer::MemoryIDBBackingStore::createIndex):
* Modules/indexeddb/server/SQLiteIDBBackingStore.cpp:
(WebCore::IDBServer::createV1ObjectStoreInfoSchema):
(WebCore::IDBServer::createV2ObjectStoreInfoSchema):
(WebCore::IDBServer::SQLiteIDBBackingStore::ensureValidIndexRecordsRecordIndex):
(WebCore::IDBServer::SQLiteIDBBackingStore::createAndPopulateInitialDatabaseInfo):
(WebCore::IDBServer::SQLiteIDBBackingStore::ensureValidObjectStoreInfoTable):
(WebCore::IDBServer::SQLiteIDBBackingStore::extractExistingDatabaseInfo):
(WebCore::IDBServer::SQLiteIDBBackingStore::getOrEstablishDatabaseInfo):
(WebCore::IDBServer::SQLiteIDBBackingStore::createObjectStore):
(WebCore::IDBServer::SQLiteIDBBackingStore::createIndex):
(WebCore::IDBServer::SQLiteIDBBackingStore::uncheckedHasIndexRecord):
(WebCore::IDBServer::SQLiteIDBBackingStore::deleteIndex):
(WebCore::IDBServer::SQLiteIDBBackingStore::deleteRecord):
(WebCore::IDBServer::SQLiteIDBBackingStore::updateAllIndexesForAddRecord):
(WebCore::IDBServer::SQLiteIDBBackingStore::uncheckedGetIndexRecordForOneKey):
(WebCore::IDBServer::SQLiteIDBBackingStore::deleteOneIndexRecord): Deleted.
* Modules/indexeddb/server/SQLiteIDBBackingStore.h:
* Modules/indexeddb/server/SQLiteIDBCursor.cpp:
(WebCore::IDBServer::buildIndexStatement):
(WebCore::IDBServer::SQLiteIDBCursor::bindArguments):
* Modules/indexeddb/server/UniqueIDBDatabase.cpp:
(WebCore::IDBServer::UniqueIDBDatabase::didDeleteBackingStore):
(WebCore::IDBServer::UniqueIDBDatabase::createIndex):
* Modules/indexeddb/shared/IDBDatabaseInfo.cpp:
(WebCore::IDBDatabaseInfo::IDBDatabaseInfo):
(WebCore::IDBDatabaseInfo::setMaxIndexID):
* Modules/indexeddb/shared/IDBDatabaseInfo.h:
(WebCore::IDBDatabaseInfo::generateNextIndexID):
(WebCore::IDBDatabaseInfo::encode const):
(WebCore::IDBDatabaseInfo::decode):
* Modules/indexeddb/shared/IDBObjectStoreInfo.cpp:
(WebCore::IDBObjectStoreInfo::createNewIndex):
(WebCore::IDBObjectStoreInfo::addExistingIndex):
(WebCore::IDBObjectStoreInfo::isolatedCopy const):
* Modules/indexeddb/shared/IDBObjectStoreInfo.h:
(WebCore::IDBObjectStoreInfo::autoIncrement const):
(WebCore::IDBObjectStoreInfo::encode const):
(WebCore::IDBObjectStoreInfo::decode):
(WebCore::IDBObjectStoreInfo::maxIndexID const): Deleted.

Tools:

Reviewed by Brady Eidson.

* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WebKitCocoa/IDBObjectStoreInfoUpgrade.sqlite3: Added.
* TestWebKitAPI/Tests/WebKitCocoa/IDBObjectStoreInfoUpgradeToV2.html: Added.
* TestWebKitAPI/Tests/WebKitCocoa/IDBObjectStoreInfoUpgradeToV2.mm: Added.
(-[IDBObjectStoreInfoUpgradeToV2MessageHandler userContentController:didReceiveScriptMessage:]):
(TEST):

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@255318 268f45cc-cd09-0410-ab3c-d52691b4dbfc

19 files changed:
PerformanceTests/ChangeLog
PerformanceTests/IndexedDB/basic/index-cursor-delete-2.html [new file with mode: 0644]
Source/WebCore/ChangeLog
Source/WebCore/Modules/indexeddb/IDBDatabase.h
Source/WebCore/Modules/indexeddb/IDBObjectStore.cpp
Source/WebCore/Modules/indexeddb/server/MemoryIDBBackingStore.cpp
Source/WebCore/Modules/indexeddb/server/SQLiteIDBBackingStore.cpp
Source/WebCore/Modules/indexeddb/server/SQLiteIDBBackingStore.h
Source/WebCore/Modules/indexeddb/server/SQLiteIDBCursor.cpp
Source/WebCore/Modules/indexeddb/server/UniqueIDBDatabase.cpp
Source/WebCore/Modules/indexeddb/shared/IDBDatabaseInfo.cpp
Source/WebCore/Modules/indexeddb/shared/IDBDatabaseInfo.h
Source/WebCore/Modules/indexeddb/shared/IDBObjectStoreInfo.cpp
Source/WebCore/Modules/indexeddb/shared/IDBObjectStoreInfo.h
Tools/ChangeLog
Tools/TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj
Tools/TestWebKitAPI/Tests/WebKitCocoa/IDBObjectStoreInfoUpgrade.sqlite3 [new file with mode: 0644]
Tools/TestWebKitAPI/Tests/WebKitCocoa/IDBObjectStoreInfoUpgradeToV2.html [new file with mode: 0644]
Tools/TestWebKitAPI/Tests/WebKitCocoa/IDBObjectStoreInfoUpgradeToV2.mm [new file with mode: 0644]

index 0f03f57..97cda05 100644 (file)
@@ -1,3 +1,12 @@
+2020-01-28  Sihui Liu  <sihui_liu@apple.com>
+
+        IndexedDB: speed up index records deletion
+        https://bugs.webkit.org/show_bug.cgi?id=206196
+
+        Reviewed by Brady Eidson.
+
+        * IndexedDB/basic/index-cursor-delete-2.html: Added.
+
 2020-01-08  Sihui Liu  <sihui_liu@apple.com>
 
         REGRESSION (r242911?): High Sierra Release WK2 Perf bot timing out while running IndexedDB/large-number-of-inserts.html
diff --git a/PerformanceTests/IndexedDB/basic/index-cursor-delete-2.html b/PerformanceTests/IndexedDB/basic/index-cursor-delete-2.html
new file mode 100644 (file)
index 0000000..6bfbc5f
--- /dev/null
@@ -0,0 +1,93 @@
+<!DOCTYPE html>
+<html>
+<body>
+<script src="../../resources/runner.js"></script>
+<script>
+
+const numberOfIterations = 20;
+const numberOfItems = 10000;
+const numberOfItemsToDelete = 5000;
+
+// Delete database(s) for the test ahead of time.
+var databaseName = "index-cursor-delete-DB-2";
+indexedDB.deleteDatabase(databaseName).onsuccess = function() {
+    startIteration();
+}
+
+var testGenerator = null;
+var db = null;
+var transaction = null;
+
+PerfTestRunner.prepareToMeasureValuesAsync({
+    customIterationCount: numberOfIterations,
+    unit: 'ms',
+    done: function () {
+        transaction = null;
+        db = null;
+        testGenerator = null;
+        PerfTestRunner.gc();
+    }
+});
+
+function startIteration()
+{
+    testGenerator = runIteration();
+    nextStep();
+}
+
+function nextStep()
+{
+    testGenerator.next();
+}
+
+function *runIteration()
+{
+    var openRequest = indexedDB.open(databaseName);
+    openRequest.onupgradeneeded = function(event) {
+        db = event.target.result;
+        var objectStore = db.createObjectStore('store', { keyPath: 'uniqueIndexKey' });
+        objectStore.createIndex('uniqueIndex', 'uniqueIndexKey', { unique: true });
+        objectStore.createIndex('index', 'indexKey');
+    }
+    openRequest.onsuccess = nextStep;
+
+    yield;
+
+    // Store items for deletion.
+    transaction = db.transaction('store', 'readwrite');
+    var objectStore = transaction.objectStore('store');
+    for (var i = 0; i < numberOfItems; ++i)
+        objectStore.put({ value: 'value', uniqueIndexKey: "uniqueIndexKey" + i, indexKey: "index" + (i%2) });
+
+    transaction.oncomplete = function(event) {
+        nextStep();
+    }
+
+    yield;
+
+    var startTime = PerfTestRunner.now();
+
+    var index = db.transaction('store', 'readwrite').objectStore('store').index('index');
+    var completedDeletes = 0;
+    index.openCursor(IDBKeyRange.only('index1')).onsuccess = function(event) {
+        var cursor = event.target.result;
+        if(cursor) {
+            cursor.delete().onsuccess = function(event) {
+                if (++completedDeletes == numberOfItemsToDelete)
+                    nextStep();
+            }
+
+            cursor.continue();
+        }
+    }
+
+    yield;
+
+    if (!PerfTestRunner.measureValueAsync(PerfTestRunner.now() - startTime))
+        return;
+
+    setTimeout(startIteration, 0);
+}
+</script>
+</body>
+</html>
index 3038ab2..0881711 100644 (file)
@@ -1,3 +1,66 @@
+2020-01-28  Sihui Liu  <sihui_liu@apple.com>
+
+        IndexedDB: speed up index records deletion
+        https://bugs.webkit.org/show_bug.cgi?id=206196
+        <rdar://problem/53596307>
+
+        Reviewed by Brady Eidson.
+
+        This patch does a few things to accelerate deletion for index records:
+        1. make indexID unique in database instead of objectStore
+        2. create an index on IndexRecords table
+        3. optimize some SQLite statements to take advantage of 1 and 2
+
+        Test: IndexedDB.IDBObjectStoreInfoUpgradeToV2
+
+        Make test PerformanceTests/IndexedDB/basic/index-cursor-delete-2.html 7.5x faster.
+
+        * Modules/indexeddb/IDBDatabase.h:
+        * Modules/indexeddb/IDBObjectStore.cpp:
+        (WebCore::IDBObjectStore::createIndex):
+        * Modules/indexeddb/server/MemoryIDBBackingStore.cpp:
+        (WebCore::IDBServer::MemoryIDBBackingStore::getOrEstablishDatabaseInfo):
+        (WebCore::IDBServer::MemoryIDBBackingStore::createIndex):
+        * Modules/indexeddb/server/SQLiteIDBBackingStore.cpp:
+        (WebCore::IDBServer::createV1ObjectStoreInfoSchema):
+        (WebCore::IDBServer::createV2ObjectStoreInfoSchema):
+        (WebCore::IDBServer::SQLiteIDBBackingStore::ensureValidIndexRecordsRecordIndex):
+        (WebCore::IDBServer::SQLiteIDBBackingStore::createAndPopulateInitialDatabaseInfo):
+        (WebCore::IDBServer::SQLiteIDBBackingStore::ensureValidObjectStoreInfoTable):
+        (WebCore::IDBServer::SQLiteIDBBackingStore::extractExistingDatabaseInfo):
+        (WebCore::IDBServer::SQLiteIDBBackingStore::getOrEstablishDatabaseInfo):
+        (WebCore::IDBServer::SQLiteIDBBackingStore::createObjectStore):
+        (WebCore::IDBServer::SQLiteIDBBackingStore::createIndex):
+        (WebCore::IDBServer::SQLiteIDBBackingStore::uncheckedHasIndexRecord):
+        (WebCore::IDBServer::SQLiteIDBBackingStore::deleteIndex):
+        (WebCore::IDBServer::SQLiteIDBBackingStore::deleteRecord):
+        (WebCore::IDBServer::SQLiteIDBBackingStore::updateAllIndexesForAddRecord):
+        (WebCore::IDBServer::SQLiteIDBBackingStore::uncheckedGetIndexRecordForOneKey):
+        (WebCore::IDBServer::SQLiteIDBBackingStore::deleteOneIndexRecord): Deleted.
+        * Modules/indexeddb/server/SQLiteIDBBackingStore.h:
+        * Modules/indexeddb/server/SQLiteIDBCursor.cpp:
+        (WebCore::IDBServer::buildIndexStatement):
+        (WebCore::IDBServer::SQLiteIDBCursor::bindArguments):
+        * Modules/indexeddb/server/UniqueIDBDatabase.cpp:
+        (WebCore::IDBServer::UniqueIDBDatabase::didDeleteBackingStore):
+        (WebCore::IDBServer::UniqueIDBDatabase::createIndex):
+        * Modules/indexeddb/shared/IDBDatabaseInfo.cpp:
+        (WebCore::IDBDatabaseInfo::IDBDatabaseInfo):
+        (WebCore::IDBDatabaseInfo::setMaxIndexID):
+        * Modules/indexeddb/shared/IDBDatabaseInfo.h:
+        (WebCore::IDBDatabaseInfo::generateNextIndexID):
+        (WebCore::IDBDatabaseInfo::encode const):
+        (WebCore::IDBDatabaseInfo::decode):
+        * Modules/indexeddb/shared/IDBObjectStoreInfo.cpp:
+        (WebCore::IDBObjectStoreInfo::createNewIndex):
+        (WebCore::IDBObjectStoreInfo::addExistingIndex):
+        (WebCore::IDBObjectStoreInfo::isolatedCopy const):
+        * Modules/indexeddb/shared/IDBObjectStoreInfo.h:
+        (WebCore::IDBObjectStoreInfo::autoIncrement const):
+        (WebCore::IDBObjectStoreInfo::encode const):
+        (WebCore::IDBObjectStoreInfo::decode):
+        (WebCore::IDBObjectStoreInfo::maxIndexID const): Deleted.
+
 2020-01-28  Justin Fan  <justin_fan@apple.com>
 
         [WebGL2] Implement sub-source texImage2D and texSubImage2D
index b45091b..ec97257 100644 (file)
@@ -84,7 +84,7 @@ public:
     const char* activeDOMObjectName() const final;
     void stop() final;
 
-    const IDBDatabaseInfo& info() const { return m_info; }
+    IDBDatabaseInfo& info() { return m_info; }
     uint64_t databaseConnectionIdentifier() const { return m_databaseConnectionIdentifier; }
 
     Ref<IDBTransaction> startVersionChangeTransaction(const IDBTransactionInfo&, IDBOpenDBRequest&);
index 3124434..30371dc 100644 (file)
@@ -475,7 +475,7 @@ ExceptionOr<Ref<IDBIndex>> IDBObjectStore::createIndex(JSGlobalObject&, const St
         return Exception { InvalidAccessError, "Failed to execute 'createIndex' on 'IDBObjectStore': The keyPath argument was an array and the multiEntry option is true."_s };
 
     // Install the new Index into the ObjectStore's info.
-    IDBIndexInfo info = m_info.createNewIndex(name, WTFMove(keyPath), parameters.unique, parameters.multiEntry);
+    IDBIndexInfo info = m_info.createNewIndex(m_transaction.database().info().generateNextIndexID(), name, WTFMove(keyPath), parameters.unique, parameters.multiEntry);
     m_transaction.database().didCreateIndexInfo(info);
 
     // Create the actual IDBObjectStore from the transaction, which also schedules the operation server side.
index 4352229..401ce3d 100644 (file)
@@ -57,7 +57,7 @@ MemoryIDBBackingStore::~MemoryIDBBackingStore() = default;
 IDBError MemoryIDBBackingStore::getOrEstablishDatabaseInfo(IDBDatabaseInfo& info)
 {
     if (!m_databaseInfo)
-        m_databaseInfo = makeUnique<IDBDatabaseInfo>(m_identifier.databaseName(), 0);
+        m_databaseInfo = makeUnique<IDBDatabaseInfo>(m_identifier.databaseName(), 0, 0);
 
     info = *m_databaseInfo;
     return IDBError { };
@@ -236,8 +236,10 @@ IDBError MemoryIDBBackingStore::createIndex(const IDBResourceIdentifier& transac
         return IDBError { ConstraintError };
 
     auto error = objectStore->createIndex(*rawTransaction, info);
-    if (error.isNull())
+    if (error.isNull()) {
         objectStoreInfo->addExistingIndex(info);
+        m_databaseInfo->setMaxIndexID(info.identifier());
+    }
 
     return error;
 }
index fcc481c..bf382e6 100644 (file)
@@ -60,6 +60,11 @@ namespace WebCore {
 using namespace JSC;
 namespace IDBServer {
 
+constexpr auto objectStoreInfoTableName = "ObjectStoreInfo"_s;
+constexpr auto objectStoreInfoTableNameAlternate = "\"ObjectStoreInfo\""_s;
+constexpr auto v2ObjectStoreInfoSchema = "CREATE TABLE ObjectStoreInfo (id INTEGER PRIMARY KEY NOT NULL ON CONFLICT FAIL UNIQUE ON CONFLICT FAIL, name TEXT NOT NULL ON CONFLICT FAIL UNIQUE ON CONFLICT FAIL, keyPath BLOB NOT NULL ON CONFLICT FAIL, autoInc INTEGER NOT NULL ON CONFLICT FAIL)"_s;
+constexpr auto v1IndexRecordsRecordIndexSchema = "CREATE INDEX IndexRecordsRecordIndex ON IndexRecords (objectStoreID, objectStoreRecordID)"_s;
+
 // Current version of the metadata schema being used in the metadata database.
 static const int currentMetadataVersion = 1;
 
@@ -229,6 +234,16 @@ static const String& blobFilesTableSchemaAlternate()
     return blobFilesTableSchemaString;
 }
 
+static String createV1ObjectStoreInfoSchema(ASCIILiteral tableName)
+{
+    return makeString("CREATE TABLE ", tableName, " (id INTEGER PRIMARY KEY NOT NULL ON CONFLICT FAIL UNIQUE ON CONFLICT FAIL, name TEXT NOT NULL ON CONFLICT FAIL UNIQUE ON CONFLICT FAIL, keyPath BLOB NOT NULL ON CONFLICT FAIL, autoInc INTEGER NOT NULL ON CONFLICT FAIL, maxIndexID INTEGER NOT NULL ON CONFLICT FAIL)");
+}
+
+static String createV2ObjectStoreInfoSchema(ASCIILiteral tableName)
+{
+    return makeString("CREATE TABLE ", tableName, " (id INTEGER PRIMARY KEY NOT NULL ON CONFLICT FAIL UNIQUE ON CONFLICT FAIL, name TEXT NOT NULL ON CONFLICT FAIL UNIQUE ON CONFLICT FAIL, keyPath BLOB NOT NULL ON CONFLICT FAIL, autoInc INTEGER NOT NULL ON CONFLICT FAIL)");
+}
+
 SQLiteIDBBackingStore::SQLiteIDBBackingStore(PAL::SessionID sessionID, const IDBDatabaseIdentifier& identifier, const String& databaseRootDirectory)
     : m_sessionID(sessionID)
     , m_identifier(identifier)
@@ -525,6 +540,46 @@ bool SQLiteIDBBackingStore::ensureValidIndexRecordsIndex()
     return false;
 }
 
+bool SQLiteIDBBackingStore::ensureValidIndexRecordsRecordIndex()
+{
+    ASSERT(m_sqliteDB);
+    ASSERT(m_sqliteDB->isOpen());
+
+    String currentSchema;
+    {
+        SQLiteStatement statement(*m_sqliteDB, "SELECT sql FROM sqlite_master WHERE name='IndexRecordsRecordIndex'");
+        if (statement.prepare() != SQLITE_OK) {
+            LOG_ERROR("Unable to prepare statement to fetch schema for the IndexRecordsRecordIndex index.");
+            return false;
+        }
+
+        int sqliteResult = statement.step();
+
+        if (sqliteResult == SQLITE_DONE) {
+            if (!m_sqliteDB->executeCommand(v1IndexRecordsRecordIndexSchema)) {
+                LOG_ERROR("Could not create IndexRecordsRecordIndex index in database (%i) - %s", m_sqliteDB->lastError(), m_sqliteDB->lastErrorMsg());
+                return false;
+            }
+
+            return true;
+        }
+
+        if (sqliteResult != SQLITE_ROW) {
+            LOG_ERROR("Error executing statement to fetch schema for the IndexRecordsRecordIndex index.");
+            return false;
+        }
+
+        currentSchema = statement.getColumnText(0);
+    }
+
+    ASSERT(!currentSchema.isEmpty());
+
+    if (currentSchema == v1IndexRecordsRecordIndexSchema)
+        return true;
+
+    return false;
+}
+
 std::unique_ptr<IDBDatabaseInfo> SQLiteIDBBackingStore::createAndPopulateInitialDatabaseInfo()
 {
     ASSERT(m_sqliteDB);
@@ -536,7 +591,7 @@ std::unique_ptr<IDBDatabaseInfo> SQLiteIDBBackingStore::createAndPopulateInitial
         return nullptr;
     }
 
-    if (!m_sqliteDB->executeCommand("CREATE TABLE ObjectStoreInfo (id INTEGER PRIMARY KEY NOT NULL ON CONFLICT FAIL UNIQUE ON CONFLICT FAIL, name TEXT NOT NULL ON CONFLICT FAIL UNIQUE ON CONFLICT FAIL, keyPath BLOB NOT NULL ON CONFLICT FAIL, autoInc INTEGER NOT NULL ON CONFLICT FAIL, maxIndexID INTEGER NOT NULL ON CONFLICT FAIL);")) {
+    if (!m_sqliteDB->executeCommand(v2ObjectStoreInfoSchema)) {
         LOG_ERROR("Could not create ObjectStoreInfo table in database (%i) - %s", m_sqliteDB->lastError(), m_sqliteDB->lastErrorMsg());
         closeSQLiteDB();
         return nullptr;
@@ -594,7 +649,65 @@ std::unique_ptr<IDBDatabaseInfo> SQLiteIDBBackingStore::createAndPopulateInitial
     }
 
     // This initial database info matches the default values we just put into the metadata database.
-    return makeUnique<IDBDatabaseInfo>(m_identifier.databaseName(), 0);
+    return makeUnique<IDBDatabaseInfo>(m_identifier.databaseName(), 0, 0);
+}
+
+Optional<IsSchemaUpgraded> SQLiteIDBBackingStore::ensureValidObjectStoreInfoTable()
+{
+    ASSERT(m_sqliteDB);
+    ASSERT(m_sqliteDB->isOpen());
+
+    String currentSchema;
+    {
+        // Fetch the schema for ObjectStoreInfo table.
+        SQLiteStatement statement(*m_sqliteDB, "SELECT sql FROM sqlite_master WHERE tbl_name='ObjectStoreInfo'");
+        if (statement.prepare() != SQLITE_OK) {
+            LOG_ERROR("Unable to prepare statement to fetch schema for the ObjectStoreInfo table.");
+            return WTF::nullopt;
+        }
+
+        int sqliteResult = statement.step();
+        if (sqliteResult != SQLITE_ROW) {
+            LOG_ERROR("Error executing statement to fetch schema for the ObjectStoreInfo table.");
+            return WTF::nullopt;
+        }
+
+        currentSchema = statement.getColumnText(0);
+    }
+
+    ASSERT(!currentSchema.isEmpty());
+    if (currentSchema == v2ObjectStoreInfoSchema || currentSchema == createV2ObjectStoreInfoSchema(objectStoreInfoTableNameAlternate))
+        return { IsSchemaUpgraded::No };
+
+    RELEASE_ASSERT(currentSchema == createV1ObjectStoreInfoSchema(objectStoreInfoTableName) || currentSchema == createV1ObjectStoreInfoSchema(objectStoreInfoTableNameAlternate));
+
+    // Drop column maxIndexID from table.
+    SQLiteTransaction transaction(*m_sqliteDB);
+    transaction.begin();
+
+    if (!m_sqliteDB->executeCommand(createV2ObjectStoreInfoSchema("_Temp_ObjectStoreInfo"_s))) {
+        LOG_ERROR("Could not create temporary ObjectStoreInfo table in database (%i) - %s", m_sqliteDB->lastError(), m_sqliteDB->lastErrorMsg());
+        return WTF::nullopt;
+    }
+
+    if (!m_sqliteDB->executeCommand("INSERT INTO _Temp_ObjectStoreInfo (id, name, keyPath, autoInc) SELECT id, name, keyPath, autoInc FROM ObjectStoreInfo")) {
+        LOG_ERROR("Could not migrate existing ObjectStoreInfo content (%i) - %s", m_sqliteDB->lastError(), m_sqliteDB->lastErrorMsg());
+        return WTF::nullopt;
+    }
+
+    if (!m_sqliteDB->executeCommand("DROP TABLE ObjectStoreInfo")) {
+        LOG_ERROR("Could not drop existing ObjectStoreInfo table (%i) - %s", m_sqliteDB->lastError(), m_sqliteDB->lastErrorMsg());
+        return WTF::nullopt;
+    }
+
+    if (!m_sqliteDB->executeCommand("ALTER TABLE _Temp_ObjectStoreInfo RENAME TO ObjectStoreInfo")) {
+        LOG_ERROR("Could not rename temporary ObjectStoreInfo table (%i) - %s", m_sqliteDB->lastError(), m_sqliteDB->lastErrorMsg());
+        return WTF::nullopt;
+    }
+
+    transaction.commit();
+
+    return { IsSchemaUpgraded::Yes };
 }
 
 std::unique_ptr<IDBDatabaseInfo> SQLiteIDBBackingStore::extractExistingDatabaseInfo()
@@ -629,10 +742,16 @@ std::unique_ptr<IDBDatabaseInfo> SQLiteIDBBackingStore::extractExistingDatabaseI
         }
     }
 
-    auto databaseInfo = makeUnique<IDBDatabaseInfo>(databaseName, databaseVersion);
+    auto databaseInfo = makeUnique<IDBDatabaseInfo>(databaseName, databaseVersion, 0);
+
+    auto result = ensureValidObjectStoreInfoTable();
+    if (!result)
+        return nullptr;
+
+    bool shouldUpdateIndexID = (result.value() == IsSchemaUpgraded::Yes);
 
     {
-        SQLiteStatement sql(*m_sqliteDB, "SELECT id, name, keyPath, autoInc, maxIndexID FROM ObjectStoreInfo;"_s);
+        SQLiteStatement sql(*m_sqliteDB, "SELECT id, name, keyPath, autoInc FROM ObjectStoreInfo;"_s);
         if (sql.prepare() != SQLITE_OK)
             return nullptr;
 
@@ -663,10 +782,14 @@ std::unique_ptr<IDBDatabaseInfo> SQLiteIDBBackingStore::extractExistingDatabaseI
         }
     }
 
+    uint64_t maxIndexID = 0;
+    HashMap<std::pair<uint64_t, uint64_t>, uint64_t> indexIDMap;
     {
         SQLiteStatement sql(*m_sqliteDB, "SELECT id, name, objectStoreID, keyPath, isUnique, multiEntry FROM IndexInfo;"_s);
-        if (sql.prepare() != SQLITE_OK)
+        if (sql.prepare() != SQLITE_OK) {
+            LOG_ERROR("Unable to prepare statement to fetch records from the IndexInfo table.");
             return nullptr;
+        }
 
         int result = sql.step();
         while (result == SQLITE_ROW) {
@@ -696,7 +819,13 @@ std::unique_ptr<IDBDatabaseInfo> SQLiteIDBBackingStore::extractExistingDatabaseI
                 return nullptr;
             }
 
+            if (shouldUpdateIndexID) {
+                indexIDMap.set({ objectStoreID, indexID }, ++maxIndexID);
+                indexID = maxIndexID;
+            }
+
             objectStore->addExistingIndex({ indexID, objectStoreID, indexName, WTFMove(indexKeyPath.value()), unique, multiEntry });
+            maxIndexID = maxIndexID < indexID ? indexID : maxIndexID;
 
             result = sql.step();
         }
@@ -705,6 +834,32 @@ std::unique_ptr<IDBDatabaseInfo> SQLiteIDBBackingStore::extractExistingDatabaseI
             LOG_ERROR("Error fetching index info from database on disk");
             return nullptr;
         }
+        databaseInfo->setMaxIndexID(maxIndexID);
+
+        if (!shouldUpdateIndexID)
+            return databaseInfo;
+
+        for (auto& entry : indexIDMap) {
+            SQLiteStatement sql(*m_sqliteDB, "UPDATE IndexInfo SET id = ? WHERE id = ? AND objectStoreID = ?;"_s);
+            if (sql.prepare() != SQLITE_OK
+                || sql.bindInt64(1, entry.value) != SQLITE_OK
+                || sql.bindInt64(2, entry.key.second) != SQLITE_OK
+                || sql.bindInt64(3, entry.key.first) != SQLITE_OK
+                || sql.step() != SQLITE_DONE) {
+                LOG_ERROR("Unable to update id of IndexInfo table");
+                return nullptr;
+            }
+
+            SQLiteStatement recordSql(*m_sqliteDB, "UPDATE IndexRecords SET indexID = ? WHERE indexID = ? AND objectStoreID = ?;"_s);
+            if (recordSql.prepare() != SQLITE_OK
+                || recordSql.bindInt64(1, entry.value) != SQLITE_OK
+                || recordSql.bindInt64(2, entry.key.second) != SQLITE_OK
+                || recordSql.bindInt64(3, entry.key.first) != SQLITE_OK
+                || recordSql.step() != SQLITE_DONE) {
+                LOG_ERROR("Unable to update indexID of IndexRecords table");
+                return nullptr;
+            }
+        }
     }
 
     return databaseInfo;
@@ -823,6 +978,12 @@ IDBError SQLiteIDBBackingStore::getOrEstablishDatabaseInfo(IDBDatabaseInfo& info
         return IDBError { UnknownError, "Error creating or migrating Index Records index in database"_s };
     }
 
+    if (!ensureValidIndexRecordsRecordIndex()) {
+        LOG_ERROR("Error creating or migrating Index Records second index for in database");
+        closeSQLiteDB();
+        return IDBError { UnknownError, "Error creating or migrating Index Records second index in database"_s };
+    }
+
     if (!ensureValidBlobTables()) {
         LOG_ERROR("Error creating or confirming Blob Records tables in database");
         closeSQLiteDB();
@@ -952,13 +1113,12 @@ IDBError SQLiteIDBBackingStore::createObjectStore(const IDBResourceIdentifier& t
     }
 
     {
-        auto* sql = cachedStatement(SQL::CreateObjectStoreInfo, "INSERT INTO ObjectStoreInfo VALUES (?, ?, ?, ?, ?);"_s);
+        auto* sql = cachedStatement(SQL::CreateObjectStoreInfo, "INSERT INTO ObjectStoreInfo VALUES (?, ?, ?, ?);"_s);
         if (!sql
             || sql->bindInt64(1, info.identifier()) != SQLITE_OK
             || sql->bindText(2, info.name()) != SQLITE_OK
             || sql->bindBlob(3, keyPathBlob->data(), keyPathBlob->size()) != SQLITE_OK
             || sql->bindInt(4, info.autoIncrement()) != SQLITE_OK
-            || sql->bindInt64(5, info.maxIndexID()) != SQLITE_OK
             || sql->step() != SQLITE_DONE) {
             LOG_ERROR("Could not add object store '%s' to ObjectStoreInfo table (%i) - %s", info.name().utf8().data(), m_sqliteDB->lastError(), m_sqliteDB->lastErrorMsg());
             return IDBError { UnknownError, "Could not create object store"_s };
@@ -1223,6 +1383,7 @@ IDBError SQLiteIDBBackingStore::createIndex(const IDBResourceIdentifier& transac
     auto* objectStore = m_databaseInfo->infoForExistingObjectStore(info.objectStoreIdentifier());
     ASSERT(objectStore);
     objectStore->addExistingIndex(info);
+    m_databaseInfo->setMaxIndexID(info.identifier());
 
     return IDBError { };
 }
@@ -1237,11 +1398,10 @@ IDBError SQLiteIDBBackingStore::uncheckedHasIndexRecord(const IDBIndexInfo& info
         return IDBError { UnknownError, "Unable to serialize IDBKey to check for index record in database"_s };
     }
 
-    auto* sql = cachedStatement(SQL::HasIndexRecord, "SELECT rowid FROM IndexRecords WHERE indexID = ? AND objectStoreID = ? AND key = CAST(? AS TEXT);"_s);
+    auto* sql = cachedStatement(SQL::HasIndexRecord, "SELECT rowid FROM IndexRecords WHERE indexID = ? AND key = CAST(? AS TEXT);"_s);
     if (!sql
         || sql->bindInt64(1, info.identifier()) != SQLITE_OK
-        || sql->bindInt64(2, info.objectStoreIdentifier()) != SQLITE_OK
-        || sql->bindBlob(3, indexKeyBuffer->data(), indexKeyBuffer->size()) != SQLITE_OK) {
+        || sql->bindBlob(2, indexKeyBuffer->data(), indexKeyBuffer->size()) != SQLITE_OK) {
         LOG_ERROR("Error checking for index record in database");
         return IDBError { UnknownError, "Error checking for index record in database"_s };
     }
@@ -1359,10 +1519,9 @@ IDBError SQLiteIDBBackingStore::deleteIndex(const IDBResourceIdentifier& transac
     }
 
     {
-        auto* sql = cachedStatement(SQL::DeleteIndexRecords, "DELETE FROM IndexRecords WHERE indexID = ? AND objectStoreID = ?;"_s);
+        auto* sql = cachedStatement(SQL::DeleteIndexRecords, "DELETE FROM IndexRecords WHERE indexID = ?;"_s);
         if (!sql
             || sql->bindInt64(1, indexIdentifier) != SQLITE_OK
-            || sql->bindInt64(2, objectStoreIdentifier) != SQLITE_OK
             || sql->step() != SQLITE_DONE) {
             LOG_ERROR("Could not delete index records for index id %" PRIi64 " from IndexRecords table (%i) - %s", indexIdentifier, m_sqliteDB->lastError(), m_sqliteDB->lastErrorMsg());
             return IDBError { UnknownError, "Error deleting index records from database"_s };
@@ -1582,61 +1741,21 @@ IDBError SQLiteIDBBackingStore::deleteRecord(SQLiteIDBTransaction& transaction,
     }
 
     // Delete record from indexes store
-    JSLockHolder locker(m_serializationContext->vm());
-    auto jsValue = deserializeIDBValueToJSValue(m_serializationContext->execState(), value);
-    if (jsValue.isUndefinedOrNull())
-        return IDBError { };
-
-    for (auto& objectStoreInfo : m_databaseInfo->objectStoreMap().values()) {
-        for (auto& indexInfo : objectStoreInfo.indexMap().values()) {
-            IndexKey indexKey;
-            generateIndexKeyForValue(m_serializationContext->execState(), indexInfo, jsValue, indexKey, objectStoreInfo.keyPath(), keyData);
-            if (indexKey.isNull())
-                continue;
+    {
+        auto* sql = cachedStatement(SQL::DeleteObjectStoreIndexRecord, "DELETE FROM IndexRecords WHERE objectStoreID = ? AND objectStoreRecordID = ?;"_s);
 
-            if (!indexInfo.multiEntry()) {
-                auto error = deleteOneIndexRecord(objectStoreID, recordID, indexKey.asOneKey());
-                if (!error.isNull())
-                    return error;
-            } else {
-                auto indexKeys = indexKey.multiEntry();
-                for (auto& key : indexKeys) {
-                    auto error = deleteOneIndexRecord(objectStoreID, recordID, key);
-                    if (!error.isNull())
-                        return error;
-                }
-                
-            }
+        if (!sql
+            || sql->bindInt64(1, objectStoreID) != SQLITE_OK
+            || sql->bindInt64(2, recordID) != SQLITE_OK
+            || sql->step() != SQLITE_DONE) {
+            LOG_ERROR("Could not delete record from indexes for object store %" PRIi64 " (%i) - %s", objectStoreID, m_sqliteDB->lastError(), m_sqliteDB->lastErrorMsg());
+            return IDBError { UnknownError, "Failed to delete index entries for object store record"_s };
         }
     }
 
     return IDBError { };
 }
 
-IDBError SQLiteIDBBackingStore::deleteOneIndexRecord(int64_t objectStoreID, int64_t objectStoreRecordID, const IDBKeyData& indexKey)
-{
-    if (!indexKey.isValid())
-        return IDBError { };
-
-    RefPtr<SharedBuffer> indexKeyBuffer = serializeIDBKeyData(indexKey);
-    if (!indexKeyBuffer) {
-        LOG_ERROR("Could not delete record from object store %" PRIi64 " (Could not serialize index key)", objectStoreID);
-        return IDBError { UnknownError, "Failed to delete index entries for object store record"_s };
-    }
-
-    auto* sql = cachedStatement(SQL::DeleteObjectStoreIndexRecord, "DELETE FROM IndexRecords WHERE objectStoreID = ? AND objectStoreRecordID = ? AND key = CAST(? AS TEXT);"_s);
-    if (!sql
-        || sql->bindInt64(1, objectStoreID) != SQLITE_OK
-        || sql->bindInt64(2, objectStoreRecordID) != SQLITE_OK
-        || sql->bindBlob(3, indexKeyBuffer->data(), indexKeyBuffer->size()) != SQLITE_OK
-        || sql->step() != SQLITE_DONE) {
-        LOG_ERROR("Could not delete record from indexes for object store %" PRIi64 " (%i) - %s", objectStoreID, m_sqliteDB->lastError(), m_sqliteDB->lastErrorMsg());
-        return IDBError { UnknownError, "Failed to delete index entries for object store record"_s };
-    }
-
-    return IDBError { };
-}
-
 IDBError SQLiteIDBBackingStore::deleteRange(const IDBResourceIdentifier& transactionIdentifier, uint64_t objectStoreID, const IDBKeyRangeData& keyRange)
 {
     LOG(IndexedDB, "SQLiteIDBBackingStore::deleteRange - range %s, object store %" PRIu64, keyRange.loggingString().utf8().data(), objectStoreID);
@@ -1741,13 +1860,11 @@ IDBError SQLiteIDBBackingStore::updateAllIndexesForAddRecord(const IDBObjectStor
     }
 
     if (!error.isNull() && anyRecordsSucceeded) {
-        RefPtr<SharedBuffer> keyBuffer = serializeIDBKeyData(key);
-
-        auto* sql = cachedStatement(SQL::DeleteObjectStoreIndexRecord, "DELETE FROM IndexRecords WHERE objectStoreID = ? AND value = CAST(? AS TEXT);"_s);
+        auto* sql = cachedStatement(SQL::DeleteObjectStoreIndexRecord, "DELETE FROM IndexRecords WHERE objectStoreID = ? AND objectStoreRecordID = ?;"_s);
 
         if (!sql
             || sql->bindInt64(1, info.identifier()) != SQLITE_OK
-            || sql->bindBlob(2, keyBuffer->data(), keyBuffer->size()) != SQLITE_OK
+            || sql->bindInt64(2, recordID) != SQLITE_OK
             || sql->step() != SQLITE_DONE) {
             LOG_ERROR("Adding one Index record failed, but failed to remove all others that previously succeeded");
             return IDBError { UnknownError, "Adding one Index record failed, but failed to remove all others that previously succeeded"_s };
@@ -2286,12 +2403,11 @@ IDBError SQLiteIDBBackingStore::uncheckedGetIndexRecordForOneKey(int64_t indexID
         return IDBError { UnknownError, "Unable to serialize IDBKey to look up one index record"_s };
     }
 
-    auto* sql = cachedStatement(SQL::GetIndexRecordForOneKey, "SELECT IndexRecords.value, Records.value, Records.recordID FROM Records INNER JOIN IndexRecords ON Records.recordID = IndexRecords.objectStoreRecordID WHERE IndexRecords.indexID = ? AND IndexRecords.objectStoreID = ? AND IndexRecords.key = CAST(? AS TEXT) ORDER BY IndexRecords.key, IndexRecords.value"_s);
+    auto* sql = cachedStatement(SQL::GetIndexRecordForOneKey, "SELECT IndexRecords.value, Records.value, Records.recordID FROM Records INNER JOIN IndexRecords ON Records.objectStoreID = IndexRecords.objectStoreID AND Records.recordID = IndexRecords.objectStoreRecordID WHERE IndexRecords.indexID = ? AND IndexRecords.key = CAST(? AS TEXT) ORDER BY IndexRecords.key, IndexRecords.value"_s);
 
     if (!sql
         || sql->bindInt64(1, indexID) != SQLITE_OK
-        || sql->bindInt64(2, objectStoreID) != SQLITE_OK
-        || sql->bindBlob(3, buffer->data(), buffer->size()) != SQLITE_OK) {
+        || sql->bindBlob(2, buffer->data(), buffer->size()) != SQLITE_OK) {
         LOG_ERROR("Unable to lookup index record in database");
         return IDBError { UnknownError, "Unable to lookup index record in database"_s };
     }
index 75f7e8d..ce29d2c 100644 (file)
@@ -44,6 +44,8 @@ class SQLiteStatement;
 
 namespace IDBServer {
 
+enum class IsSchemaUpgraded : bool { No, Yes };
+
 class IDBSerializationContext;
 class SQLiteIDBCursor;
 
@@ -111,12 +113,13 @@ private:
     bool ensureValidRecordsTable();
     bool ensureValidIndexRecordsTable();
     bool ensureValidIndexRecordsIndex();
+    bool ensureValidIndexRecordsRecordIndex();
     bool ensureValidBlobTables();
+    Optional<IsSchemaUpgraded> ensureValidObjectStoreInfoTable();
     std::unique_ptr<IDBDatabaseInfo> createAndPopulateInitialDatabaseInfo();
     std::unique_ptr<IDBDatabaseInfo> extractExistingDatabaseInfo();
 
     IDBError deleteRecord(SQLiteIDBTransaction&, int64_t objectStoreID, const IDBKeyData&);
-    IDBError deleteOneIndexRecord(int64_t objectStoreID, int64_t objectStoreRecordID, const IDBKeyData&);
     IDBError uncheckedGetKeyGeneratorValue(int64_t objectStoreID, uint64_t& outValue);
     IDBError uncheckedSetKeyGeneratorValue(int64_t objectStoreID, uint64_t value);
 
index 90c0ad3..21dff84 100644 (file)
@@ -120,7 +120,7 @@ static String buildIndexStatement(const IDBKeyRangeData& keyRange, IndexedDB::Cu
 {
     StringBuilder builder;
 
-    builder.appendLiteral("SELECT rowid, key, value FROM IndexRecords WHERE indexID = ? AND objectStoreID = ? AND key ");
+    builder.appendLiteral("SELECT rowid, key, value FROM IndexRecords WHERE indexID = ? AND key ");
     if (!keyRange.lowerKey.isNull() && !keyRange.lowerOpen)
         builder.appendLiteral(">=");
     else
@@ -289,11 +289,6 @@ bool SQLiteIDBCursor::bindArguments()
         return false;
     }
 
-    if (m_indexID != IDBIndexInfo::InvalidId && m_statement->bindInt64(currentBindArgument++, m_objectStoreID) != SQLITE_OK) {
-        LOG_ERROR("Could not bind object store id argument for an index cursor");
-        return false;
-    }
-
     RefPtr<SharedBuffer> buffer = serializeIDBKeyData(m_currentLowerKey);
     if (m_statement->bindBlob(currentBindArgument++, buffer->data(), buffer->size()) != SQLITE_OK) {
         LOG_ERROR("Could not create cursor statement (lower key)");
index b72b637..a0a0516 100644 (file)
@@ -309,7 +309,7 @@ void UniqueIDBDatabase::didDeleteBackingStore(uint64_t deletedVersion)
     // we won't have a m_mostRecentDeletedDatabaseInfo. In that case, we'll manufacture one using the
     // passed in deletedVersion argument.
     if (!m_mostRecentDeletedDatabaseInfo)
-        m_mostRecentDeletedDatabaseInfo = makeUnique<IDBDatabaseInfo>(m_identifier.databaseName(), deletedVersion);
+        m_mostRecentDeletedDatabaseInfo = makeUnique<IDBDatabaseInfo>(m_identifier.databaseName(), deletedVersion, 0);
 
     if (m_currentOpenDBRequest) {
         m_currentOpenDBRequest->notifyDidDeleteDatabase(*m_mostRecentDeletedDatabaseInfo);
@@ -633,6 +633,7 @@ void UniqueIDBDatabase::createIndex(UniqueIDBDatabaseTransaction& transaction, c
         auto* objectStoreInfo = m_databaseInfo->infoForExistingObjectStore(info.objectStoreIdentifier());
         ASSERT(objectStoreInfo);
         objectStoreInfo->addExistingIndex(info);
+        m_databaseInfo->setMaxIndexID(info.identifier());
     }
 
     callback(error);
index ea312cc..8603fe6 100644 (file)
@@ -36,9 +36,10 @@ IDBDatabaseInfo::IDBDatabaseInfo()
 {
 }
 
-IDBDatabaseInfo::IDBDatabaseInfo(const String& name, uint64_t version)
+IDBDatabaseInfo::IDBDatabaseInfo(const String& name, uint64_t version, uint64_t maxIndexID)
     : m_name(name)
     , m_version(version)
+    , m_maxIndexID(maxIndexID)
 {
 }
 
@@ -46,6 +47,7 @@ IDBDatabaseInfo::IDBDatabaseInfo(const IDBDatabaseInfo& other, IsolatedCopyTag)
     : m_name(other.m_name.isolatedCopy())
     , m_version(other.m_version)
     , m_maxObjectStoreID(other.m_maxObjectStoreID)
+    , m_maxIndexID(other.m_maxIndexID)
 {
     for (const auto& entry : other.m_objectStoreMap)
         m_objectStoreMap.set(entry.key, entry.value.isolatedCopy());
@@ -168,6 +170,12 @@ String IDBDatabaseInfo::loggingString() const
 
 #endif
 
+void IDBDatabaseInfo::setMaxIndexID(uint64_t maxIndexID)
+{
+    ASSERT(maxIndexID > m_maxIndexID || (!maxIndexID && !m_maxIndexID));
+    m_maxIndexID = maxIndexID;
+}
+
 } // namespace WebCore
 
 #endif // ENABLE(INDEXED_DATABASE)
index 8235bb8..e2e152a 100644 (file)
@@ -35,7 +35,7 @@ namespace WebCore {
 class IDBDatabaseInfo {
     WTF_MAKE_FAST_ALLOCATED;
 public:
-    explicit IDBDatabaseInfo(const String& name, uint64_t version);
+    explicit IDBDatabaseInfo(const String& name, uint64_t version, uint64_t maxIndexID);
 
     enum IsolatedCopyTag { IsolatedCopy };
     IDBDatabaseInfo(const IDBDatabaseInfo&, IsolatedCopyTag);
@@ -68,6 +68,9 @@ public:
     template<class Encoder> void encode(Encoder&) const;
     template<class Decoder> static bool decode(Decoder&, IDBDatabaseInfo&);
 
+    void setMaxIndexID(uint64_t maxIndexID);
+    uint64_t generateNextIndexID() { return ++m_maxIndexID; }
+
 #if !LOG_DISABLED
     String loggingString() const;
 #endif
@@ -79,6 +82,7 @@ private:
     String m_name;
     uint64_t m_version { 0 };
     uint64_t m_maxObjectStoreID { 0 };
+    uint64_t m_maxIndexID { 0 };
 
     HashMap<uint64_t, IDBObjectStoreInfo> m_objectStoreMap;
 
@@ -87,7 +91,7 @@ private:
 template<class Encoder>
 void IDBDatabaseInfo::encode(Encoder& encoder) const
 {
-    encoder << m_name << m_version << m_maxObjectStoreID << m_objectStoreMap;
+    encoder << m_name << m_version << m_maxObjectStoreID << m_maxIndexID << m_objectStoreMap;
 }
 
 template<class Decoder>
@@ -102,6 +106,9 @@ bool IDBDatabaseInfo::decode(Decoder& decoder, IDBDatabaseInfo& info)
     if (!decoder.decode(info.m_maxObjectStoreID))
         return false;
 
+    if (!decoder.decode(info.m_maxIndexID))
+        return false;
+
     if (!decoder.decode(info.m_objectStoreMap))
         return false;
 
index 5c7282a..4ead38b 100644 (file)
@@ -43,9 +43,9 @@ IDBObjectStoreInfo::IDBObjectStoreInfo(uint64_t identifier, const String& name,
 {
 }
 
-IDBIndexInfo IDBObjectStoreInfo::createNewIndex(const String& name, IDBKeyPath&& keyPath, bool unique, bool multiEntry)
+IDBIndexInfo IDBObjectStoreInfo::createNewIndex(uint64_t indexID, const String& name, IDBKeyPath&& keyPath, bool unique, bool multiEntry)
 {
-    IDBIndexInfo info(++m_maxIndexID, m_identifier, name, WTFMove(keyPath), unique, multiEntry);
+    IDBIndexInfo info(indexID, m_identifier, name, WTFMove(keyPath), unique, multiEntry);
     m_indexMap.set(info.identifier(), info);
     return info;
 }
@@ -54,9 +54,6 @@ void IDBObjectStoreInfo::addExistingIndex(const IDBIndexInfo& info)
 {
     ASSERT(!m_indexMap.contains(info.identifier()));
 
-    if (info.identifier() > m_maxIndexID)
-        m_maxIndexID = info.identifier();
-
     m_indexMap.set(info.identifier(), info);
 }
 
@@ -101,8 +98,6 @@ IDBObjectStoreInfo IDBObjectStoreInfo::isolatedCopy() const
     for (auto& iterator : m_indexMap)
         result.m_indexMap.set(iterator.key, iterator.value.isolatedCopy());
 
-    result.m_maxIndexID = m_maxIndexID;
-
     return result;
 }
 
index dce95be..17fe422 100644 (file)
@@ -43,13 +43,12 @@ public:
     const String& name() const { return m_name; }
     const Optional<IDBKeyPath>& keyPath() const { return m_keyPath; }
     bool autoIncrement() const { return m_autoIncrement; }
-    uint64_t maxIndexID() const { return m_maxIndexID; }
 
     void rename(const String& newName) { m_name = newName; }
 
     WEBCORE_EXPORT IDBObjectStoreInfo isolatedCopy() const;
 
-    IDBIndexInfo createNewIndex(const String& name, IDBKeyPath&&, bool unique, bool multiEntry);
+    IDBIndexInfo createNewIndex(uint64_t indexID, const String& name, IDBKeyPath&&, bool unique, bool multiEntry);
     void addExistingIndex(const IDBIndexInfo&);
     bool hasIndex(const String& name) const;
     bool hasIndex(uint64_t indexIdentifier) const;
@@ -75,7 +74,6 @@ private:
     String m_name;
     Optional<IDBKeyPath> m_keyPath;
     bool m_autoIncrement { false };
-    uint64_t m_maxIndexID { 0 };
 
     HashMap<uint64_t, IDBIndexInfo> m_indexMap;
 };
@@ -83,7 +81,7 @@ private:
 template<class Encoder>
 void IDBObjectStoreInfo::encode(Encoder& encoder) const
 {
-    encoder << m_identifier << m_name << m_keyPath << m_autoIncrement << m_maxIndexID << m_indexMap;
+    encoder << m_identifier << m_name << m_keyPath << m_autoIncrement << m_indexMap;
 }
 
 template<class Decoder>
@@ -101,9 +99,6 @@ bool IDBObjectStoreInfo::decode(Decoder& decoder, IDBObjectStoreInfo& info)
     if (!decoder.decode(info.m_autoIncrement))
         return false;
 
-    if (!decoder.decode(info.m_maxIndexID))
-        return false;
-
     if (!decoder.decode(info.m_indexMap))
         return false;
 
index 65e66ca..06bcf1c 100644 (file)
@@ -1,3 +1,17 @@
+2020-01-28  Sihui Liu  <sihui_liu@apple.com>
+
+        IndexedDB: speed up index records deletion
+        https://bugs.webkit.org/show_bug.cgi?id=206196
+
+        Reviewed by Brady Eidson.
+
+        * TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
+        * TestWebKitAPI/Tests/WebKitCocoa/IDBObjectStoreInfoUpgrade.sqlite3: Added.
+        * TestWebKitAPI/Tests/WebKitCocoa/IDBObjectStoreInfoUpgradeToV2.html: Added.
+        * TestWebKitAPI/Tests/WebKitCocoa/IDBObjectStoreInfoUpgradeToV2.mm: Added.
+        (-[IDBObjectStoreInfoUpgradeToV2MessageHandler userContentController:didReceiveScriptMessage:]):
+        (TEST):
+
 2020-01-28  James Darpinian  <jdarpinian@chromium.org>
 
         Quote file argument to git check-attr
index a55cc33..325ed01 100644 (file)
                9368A25F229EFB4700A829CA /* local-storage-process-suspends-2.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = 9368A25C229EFB3A00A829CA /* local-storage-process-suspends-2.html */; };
                936F72801CD7D9EC0068A0FB /* large-video-with-audio.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = 936F727E1CD7D9D00068A0FB /* large-video-with-audio.html */; };
                936F72811CD7D9EC0068A0FB /* large-video-with-audio.mp4 in Copy Resources */ = {isa = PBXBuildFile; fileRef = 936F727F1CD7D9D00068A0FB /* large-video-with-audio.mp4 */; };
+               937B569E23CD23DB002AE640 /* IDBObjectStoreInfoUpgrade.sqlite3 in Copy Resources */ = {isa = PBXBuildFile; fileRef = 93BCBC8123CC6EF500CA2221 /* IDBObjectStoreInfoUpgrade.sqlite3 */; };
                9399BA0423711140008392BF /* IndexedDBInPageCache.mm in Sources */ = {isa = PBXBuildFile; fileRef = 9399BA01237110AE008392BF /* IndexedDBInPageCache.mm */; };
                9399BA052371114F008392BF /* IndexedDBInPageCache.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = 9399BA02237110BF008392BF /* IndexedDBInPageCache.html */; };
                9399BA062371114F008392BF /* IndexedDBNotInPageCache.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = 9399BA03237110BF008392BF /* IndexedDBNotInPageCache.html */; };
                93AF4ECE1506F064007FD57E /* NewFirstVisuallyNonEmptyLayoutForImages_Bundle.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 93AF4ECD1506F064007FD57E /* NewFirstVisuallyNonEmptyLayoutForImages_Bundle.cpp */; };
                93AF4ED11506F130007FD57E /* lots-of-images.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = 93AF4ECF1506F123007FD57E /* lots-of-images.html */; };
+               93BCBC8323CC6F2A00CA2221 /* IDBObjectStoreInfoUpgradeToV2.mm in Sources */ = {isa = PBXBuildFile; fileRef = 93BCBC8023CC6EE800CA2221 /* IDBObjectStoreInfoUpgradeToV2.mm */; };
+               93BCBC8423CC6F4400CA2221 /* IDBObjectStoreInfoUpgradeToV2.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = 93BCBC8223CC6EF500CA2221 /* IDBObjectStoreInfoUpgradeToV2.html */; };
                93CFA8671CEB9E38000565A8 /* autofocused-text-input.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = 93CFA8661CEB9DE1000565A8 /* autofocused-text-input.html */; };
                93D119FC22C680F7009BE3C7 /* localstorage-open-window-private.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = 93D119FB22C57112009BE3C7 /* localstorage-open-window-private.html */; };
                93E2C5551FD3204100E1DF6A /* LineEnding.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 93E2C5541FD3204100E1DF6A /* LineEnding.cpp */; };
                                510477731D298DDD009747EB /* IDBDeleteRecovery.sqlite3-shm in Copy Resources */,
                                510477741D298DDD009747EB /* IDBDeleteRecovery.sqlite3-wal in Copy Resources */,
                                5110FCF11E01CD64006F8D0B /* IDBIndexUpgradeToV2.html in Copy Resources */,
+                               937B569E23CD23DB002AE640 /* IDBObjectStoreInfoUpgrade.sqlite3 in Copy Resources */,
+                               93BCBC8423CC6F4400CA2221 /* IDBObjectStoreInfoUpgradeToV2.html in Copy Resources */,
                                F41AB9A41EF4696B0083FA08 /* image-and-contenteditable.html in Copy Resources */,
                                F4E0A2B42122402B00AF7C7F /* image-and-file-upload.html in Copy Resources */,
                                F41AB9A51EF4696B0083FA08 /* image-and-textarea.html in Copy Resources */,
                93AF4ECA1506F035007FD57E /* NewFirstVisuallyNonEmptyLayoutForImages.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = NewFirstVisuallyNonEmptyLayoutForImages.cpp; sourceTree = "<group>"; };
                93AF4ECD1506F064007FD57E /* NewFirstVisuallyNonEmptyLayoutForImages_Bundle.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = NewFirstVisuallyNonEmptyLayoutForImages_Bundle.cpp; sourceTree = "<group>"; };
                93AF4ECF1506F123007FD57E /* lots-of-images.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = "lots-of-images.html"; sourceTree = "<group>"; };
+               93BCBC8023CC6EE800CA2221 /* IDBObjectStoreInfoUpgradeToV2.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = IDBObjectStoreInfoUpgradeToV2.mm; sourceTree = "<group>"; };
+               93BCBC8123CC6EF500CA2221 /* IDBObjectStoreInfoUpgrade.sqlite3 */ = {isa = PBXFileReference; lastKnownFileType = file; path = IDBObjectStoreInfoUpgrade.sqlite3; sourceTree = "<group>"; };
+               93BCBC8223CC6EF500CA2221 /* IDBObjectStoreInfoUpgradeToV2.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = IDBObjectStoreInfoUpgradeToV2.html; sourceTree = "<group>"; };
                93CFA8661CEB9DE1000565A8 /* autofocused-text-input.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = "autofocused-text-input.html"; sourceTree = "<group>"; };
                93CFA8681CEBCFED000565A8 /* CandidateTests.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = CandidateTests.mm; sourceTree = "<group>"; };
                93D119FB22C57112009BE3C7 /* localstorage-open-window-private.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = "localstorage-open-window-private.html"; sourceTree = "<group>"; };
                                51AF23DE1EF1A3720072F281 /* IconLoadingDelegate.mm */,
                                510477751D298E03009747EB /* IDBDeleteRecovery.mm */,
                                5110FCEF1E01CBAA006F8D0B /* IDBIndexUpgradeToV2.mm */,
+                               93BCBC8023CC6EE800CA2221 /* IDBObjectStoreInfoUpgradeToV2.mm */,
                                51A587841D272EF3004BA9AF /* IndexedDBDatabaseProcessKill.mm */,
                                CAB0FF5422332C3A006CA5B0 /* IndexedDBFileName.mm */,
                                9399BA01237110AE008392BF /* IndexedDBInPageCache.mm */,
                                510477701D298D85009747EB /* IDBDeleteRecovery.sqlite3-shm */,
                                510477711D298D85009747EB /* IDBDeleteRecovery.sqlite3-wal */,
                                5110FCF01E01CD53006F8D0B /* IDBIndexUpgradeToV2.html */,
+                               93BCBC8123CC6EF500CA2221 /* IDBObjectStoreInfoUpgrade.sqlite3 */,
+                               93BCBC8223CC6EF500CA2221 /* IDBObjectStoreInfoUpgradeToV2.html */,
                                F41AB9991EF4692C0083FA08 /* image-and-contenteditable.html */,
                                F4E0A2B321223F2D00AF7C7F /* image-and-file-upload.html */,
                                F41AB9931EF4692C0083FA08 /* image-and-textarea.html */,
                                51AF23DF1EF1A3730072F281 /* IconLoadingDelegate.mm in Sources */,
                                510477781D29923B009747EB /* IDBDeleteRecovery.mm in Sources */,
                                5110FCFA1E01CDB8006F8D0B /* IDBIndexUpgradeToV2.mm in Sources */,
+                               93BCBC8323CC6F2A00CA2221 /* IDBObjectStoreInfoUpgradeToV2.mm in Sources */,
                                51A587861D273AA9004BA9AF /* IndexedDBDatabaseProcessKill.mm in Sources */,
                                CAB0FF5522332C57006CA5B0 /* IndexedDBFileName.mm in Sources */,
                                9399BA0423711140008392BF /* IndexedDBInPageCache.mm in Sources */,
diff --git a/Tools/TestWebKitAPI/Tests/WebKitCocoa/IDBObjectStoreInfoUpgrade.sqlite3 b/Tools/TestWebKitAPI/Tests/WebKitCocoa/IDBObjectStoreInfoUpgrade.sqlite3
new file mode 100644 (file)
index 0000000..64426ea
Binary files /dev/null and b/Tools/TestWebKitAPI/Tests/WebKitCocoa/IDBObjectStoreInfoUpgrade.sqlite3 differ
diff --git a/Tools/TestWebKitAPI/Tests/WebKitCocoa/IDBObjectStoreInfoUpgradeToV2.html b/Tools/TestWebKitAPI/Tests/WebKitCocoa/IDBObjectStoreInfoUpgradeToV2.html
new file mode 100644 (file)
index 0000000..e16fe7a
--- /dev/null
@@ -0,0 +1,38 @@
+<script>
+
+var databaseName = "objectstoreinfo-upgrade-test";
+var openRequest = indexedDB.open(databaseName);
+var db;
+openRequest.onupgradeneeded = function(event) {
+    window.webkit.messageHandlers.testHandler.postMessage("Unexpected upgrade needed");
+}
+
+openRequest.onsuccess = function(event) {
+    window.webkit.messageHandlers.testHandler.postMessage("Success");
+    var transaction = event.target.result.transaction(['objectStore', 'anotherObjectStore']);
+    var indexCount = transaction.objectStore('objectStore').indexNames;
+    if (indexes.length != 2)
+        window.webkit.messageHandlers.testHandler.postMessage("Unexpected count of indexes in objectStore 'objectStore': " + indexes.length);
+    
+    indexCount = transaction.objectStore('anotherObjectStore').indexNames;
+    if (indexes.length != 1)
+        window.webkit.messageHandlers.testHandler.postMessage("Unexpected count of indexes in objectStore 'anotherObjectStore': " + indexes.length);
+
+    req = transaction.objectStore('objectStore').index('indexKey').get("indexKey1");
+    req.onsuccess = function(event) {
+        if (req.result.value != "value1")
+            window.webkit.messageHandlers.testHandler.postMessage("Unexpected get result: " +  req.result.value);
+        else
+            window.webkit.messageHandlers.testHandler.postMessage("Success");
+    }
+    req.onerror = function(event) {
+        window.webkit.messageHandlers.testHandler.postMessage("Unexpected get result: " +  req.result.value);
+    }
+}
+
+openRequest.onerror = function(event) {
+    window.webkit.messageHandlers.testHandler.postMessage("Unexpected open error");
+}
+
+</script>
+
diff --git a/Tools/TestWebKitAPI/Tests/WebKitCocoa/IDBObjectStoreInfoUpgradeToV2.mm b/Tools/TestWebKitAPI/Tests/WebKitCocoa/IDBObjectStoreInfoUpgradeToV2.mm
new file mode 100644 (file)
index 0000000..ef1c1e5
--- /dev/null
@@ -0,0 +1,82 @@
+/*
+* Copyright (C) 2019 Apple Inc. All rights reserved.
+*
+* Redistribution and use in source and binary forms, with or without
+* modification, are permitted provided that the following conditions
+* are met:
+* 1. Redistributions of source code must retain the above copyright
+*    notice, this list of conditions and the following disclaimer.
+* 2. Redistributions in binary form must reproduce the above copyright
+*    notice, this list of conditions and the following disclaimer in the
+*    documentation and/or other materials provided with the distribution.
+*
+* THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+* THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+* BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+* THE POSSIBILITY OF SUCH DAMAGE.
+*/
+
+#import "config.h"
+
+#import "PlatformUtilities.h"
+#import "Test.h"
+#import <WebCore/SQLiteFileSystem.h>
+#import <WebKit/WKProcessPoolPrivate.h>
+#import <WebKit/WKUserContentControllerPrivate.h>
+#import <WebKit/WKWebViewConfigurationPrivate.h>
+#import <WebKit/WebKit.h>
+#import <WebKit/_WKProcessPoolConfiguration.h>
+#import <WebKit/_WKUserStyleSheet.h>
+#import <wtf/RetainPtr.h>
+
+static bool receivedScriptMessage;
+static RetainPtr<WKScriptMessage> lastScriptMessage;
+
+@interface IDBObjectStoreInfoUpgradeToV2MessageHandler : NSObject <WKScriptMessageHandler>
+@end
+
+@implementation IDBObjectStoreInfoUpgradeToV2MessageHandler
+
+- (void)userContentController:(WKUserContentController *)userContentController didReceiveScriptMessage:(WKScriptMessage *)message
+{
+    receivedScriptMessage = true;
+    lastScriptMessage = message;
+}
+@end
+
+TEST(IndexedDB, IDBObjectStoreInfoUpgradeToV2)
+{
+    RetainPtr<IDBObjectStoreInfoUpgradeToV2MessageHandler> handler = adoptNS([[IDBObjectStoreInfoUpgradeToV2MessageHandler alloc] init]);
+    RetainPtr<WKWebViewConfiguration> configuration = adoptNS([[WKWebViewConfiguration alloc] init]);
+    [[configuration userContentController] addScriptMessageHandler:handler.get() name:@"testHandler"];
+
+    [configuration.get().processPool _terminateNetworkProcess];
+
+    // Copy database files with old ObjectStoreInfo schema to the database directory.
+    NSURL *url1 = [[NSBundle mainBundle] URLForResource:@"IDBObjectStoreInfoUpgrade" withExtension:@"sqlite3" subdirectory:@"TestWebKitAPI.resources"];
+
+    NSString *hash = WebCore::SQLiteFileSystem::computeHashForFileName("objectstoreinfo-upgrade-test");
+    NSString *originDirectory = @"~/Library/WebKit/TestWebKitAPI/WebsiteData/IndexedDB/v1/file__0/";
+    NSString *databaseDirectory = [[originDirectory stringByAppendingString:hash] stringByExpandingTildeInPath];
+    NSURL *targetURL = [NSURL fileURLWithPath:databaseDirectory];
+    [[NSFileManager defaultManager] removeItemAtURL:targetURL error:nil];
+    [[NSFileManager defaultManager] createDirectoryAtURL:targetURL withIntermediateDirectories:YES attributes:nil error:nil];
+
+    [[NSFileManager defaultManager] copyItemAtURL:url1 toURL:[targetURL URLByAppendingPathComponent:@"IndexedDB.sqlite3"] error:nil];
+
+    RetainPtr<WKWebView> webView = adoptNS([[WKWebView alloc] initWithFrame:NSMakeRect(0, 0, 800, 600) configuration:configuration.get()]);
+
+    NSURLRequest *request = [NSURLRequest requestWithURL:[[NSBundle mainBundle] URLForResource:@"IDBObjectStoreInfoUpgradeToV2" withExtension:@"html" subdirectory:@"TestWebKitAPI.resources"]];
+    [webView loadRequest:request];
+
+    TestWebKitAPI::Util::run(&receivedScriptMessage);
+
+    EXPECT_WK_STREQ(@"Success", [lastScriptMessage body]);
+}