Modern IDB: Properly handle blobs in Workers.
authorbeidson@apple.com <beidson@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 20 May 2016 23:15:15 +0000 (23:15 +0000)
committerbeidson@apple.com <beidson@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 20 May 2016 23:15:15 +0000 (23:15 +0000)
https://bugs.webkit.org/show_bug.cgi?id=157947

Reviewed by Alex Christensen.

Source/WebCore:

Test: storage/indexeddb/modern/blob-simple-workers.html

* Modules/indexeddb/IDBTransaction.cpp:
(WebCore::IDBTransaction::putOrAddOnServer): Use writeBlobsToDiskForIndexedDBSynchronously from
  background threads instead of the asynchronous form.

Add ability to set an existing empty IDBValue to be an isolated copy of a different IDBValue:
* Modules/indexeddb/IDBValue.cpp:
(WebCore::IDBValue::setAsIsolatedCopy):
(WebCore::IDBValue::isolatedCopy):
* Modules/indexeddb/IDBValue.h:

Add a method - only to be called from a non-main thread - that synchronously writes blobs to disk:
* bindings/js/SerializedScriptValue.cpp:
(WebCore::SerializedScriptValue::writeBlobsToDiskForIndexedDBSynchronously):
* bindings/js/SerializedScriptValue.h:

LayoutTests:

* storage/indexeddb/modern/blob-simple-workers-expected.txt: Added.
* storage/indexeddb/modern/blob-simple-workers.html: Added.
* storage/indexeddb/modern/resources/blob-simple-workers.js: Added.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@201232 268f45cc-cd09-0410-ab3c-d52691b4dbfc

LayoutTests/ChangeLog
LayoutTests/storage/indexeddb/modern/blob-simple-workers-expected.txt [new file with mode: 0644]
LayoutTests/storage/indexeddb/modern/blob-simple-workers.html [new file with mode: 0644]
LayoutTests/storage/indexeddb/modern/resources/blob-simple-workers.js [new file with mode: 0644]
Source/WebCore/ChangeLog
Source/WebCore/Modules/indexeddb/IDBTransaction.cpp
Source/WebCore/Modules/indexeddb/IDBValue.cpp
Source/WebCore/Modules/indexeddb/IDBValue.h
Source/WebCore/bindings/js/SerializedScriptValue.cpp
Source/WebCore/bindings/js/SerializedScriptValue.h

index bae19af..e864c89 100644 (file)
@@ -1,3 +1,14 @@
+2016-05-20  Brady Eidson  <beidson@apple.com>
+
+        Modern IDB: Properly handle blobs in Workers.
+        https://bugs.webkit.org/show_bug.cgi?id=157947
+
+        Reviewed by Alex Christensen.
+
+        * storage/indexeddb/modern/blob-simple-workers-expected.txt: Added.
+        * storage/indexeddb/modern/blob-simple-workers.html: Added.
+        * storage/indexeddb/modern/resources/blob-simple-workers.js: Added.
+        
 2016-05-20  Myles C. Maxfield  <mmaxfield@apple.com>
 
         [Cocoa] REGRESSION(r184899): Ascent adjustments are applied to web fonts
diff --git a/LayoutTests/storage/indexeddb/modern/blob-simple-workers-expected.txt b/LayoutTests/storage/indexeddb/modern/blob-simple-workers-expected.txt
new file mode 100644 (file)
index 0000000..b2e64c4
--- /dev/null
@@ -0,0 +1,48 @@
+[Worker] This tests basic operations putting blobs into an object store and then retrieving them.
+
+On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
+
+
+Starting worker: resources/blob-simple-workers.js
+[Worker] indexedDB = self.indexedDB || self.webkitIndexedDB || self.mozIndexedDB || self.msIndexedDB || self.OIndexedDB;
+[Worker] 
+[Worker] indexedDB.deleteDatabase(dbname)
+[Worker] indexedDB.open(dbname)
+[Worker] Initial upgrade needed: Old version - 0 New version - 1
+[Worker] Let's create a blob and store it in IndexedDB twice.
+[Worker] Added blob to database once
+[Worker] Added blob to database twice
+[Worker] Let's retrieve the blob again and verify the contents is the same.
+[Worker] Got blob from database
+[Worker] Correct text: true
+[Worker] Let's retrieve it again, create an object URL for the blob, load it via an XMLHttpRequest, and verify the contents are the same.
+[Worker] Got blob from database
+[Worker] Correct responseText: true
+[Worker] Retrieve both blob entries from the database and verify contents.
+[Worker] Correct text on first value: true
+[Worker] Correct text on second value: true
+[Worker] Got item from cursor
+[Worker] Got item from cursor
+[Worker] Finished cursor
+[Worker] Got right number of items: true
+[Worker] Correct text: true
+[Worker] Retrieve blobs from database via index and verify contents.
+[Worker] Got blob from database
+[Worker] Correct text: true
+[Worker] Got item from cursor
+[Worker] Got item from cursor
+[Worker] Finished cursor
+[Worker] Got right number of items: true
+[Worker] Correct text: true
+[Worker] Correct text: true
+[Worker] Slice the the retrieved blob and verify its contents.
+[Worker] Correct text: true
+[Worker] Store a blob back in the database, and keep holding on to the blob, verifying that it still can be read.
+[Worker] Got blob from database
+[Worker] Stored blob back into database
+[Worker] Correct text: true
+[Worker] Correct responseText: true
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
diff --git a/LayoutTests/storage/indexeddb/modern/blob-simple-workers.html b/LayoutTests/storage/indexeddb/modern/blob-simple-workers.html
new file mode 100644 (file)
index 0000000..30aa0ed
--- /dev/null
@@ -0,0 +1,12 @@
+<html>
+<head>
+<script src="../../../resources/js-test.js"></script>
+<script src="../resources/shared.js"></script>
+</head>
+<body>
+<script>
+if (window.internals)
+    window.internals.settings.setIndexedDBWorkersEnabled(true);
+worker = startWorker('resources/blob-simple-workers.js');</script>
+</body>
+</html>
diff --git a/LayoutTests/storage/indexeddb/modern/resources/blob-simple-workers.js b/LayoutTests/storage/indexeddb/modern/resources/blob-simple-workers.js
new file mode 100644 (file)
index 0000000..f8efa29
--- /dev/null
@@ -0,0 +1,270 @@
+if (this.importScripts) {
+    importScripts('../../../../resources/js-test.js');
+    importScripts('../../resources/shared.js');
+}
+
+// Original implementation of test from Mozilla: ./dom/indexedDB/test/test_blob_simple.html
+description("This tests basic operations putting blobs into an object store and then retrieving them.");
+
+indexedDBTest(prepareDatabase);
+
+function done()
+{
+    finishJSTest();
+}
+
+function log(message)
+{
+    debug(message);
+}
+
+var testGenerator;
+
+function continueWithEvent(event)
+{
+    testGenerator.next(event);
+}
+
+function asyncContinue()
+{
+    setTimeout("testGenerator.next();", 0);
+}
+
+function idbRequest(request)
+{
+    request.onerror = continueWithEvent;
+    request.onsuccess = continueWithEvent;
+}
+
+var db;
+
+function prepareDatabase(event)
+{
+    debug("Initial upgrade needed: Old version - " + event.oldVersion + " New version - " + event.newVersion);
+
+    db = event.target.result;
+    db.createObjectStore("foo", { autoIncrement: true }).createIndex("foo", "index");
+    event.target.onsuccess = function() {
+        testGenerator = testSteps();
+        testGenerator.next();
+    };
+}
+
+function is(a, b, message) {
+    result = a == b ? "true" : "false";
+    debug(message + ": " + result);
+}
+
+function* testSteps()
+{
+    debug("Let's create a blob and store it in IndexedDB twice.");
+
+    const BLOB_DATA = ["fun ", "times ", "all ", "around!"];
+    const INDEX_KEY = 5;
+    let blob = new Blob(BLOB_DATA, { type: "text/plain" });
+    let data = { blob: blob, index: INDEX_KEY };
+
+    objectStore = db.transaction("foo", "readwrite").objectStore("foo");
+    
+    idbRequest(objectStore.add(data));
+    event = yield;
+    
+    debug("Added blob to database once");
+
+    let key1 = event.target.result;
+
+    idbRequest(objectStore.add(data));
+    event = yield;
+
+    debug("Added blob to database twice");
+    
+    let key2 = event.target.result;
+
+    debug("Let's retrieve the blob again and verify the contents is the same.");
+
+    objectStore = db.transaction("foo").objectStore("foo");
+    idbRequest(objectStore.get(key1));
+    event = yield;
+
+    debug("Got blob from database");
+    
+    let fileReader = new FileReader();
+    fileReader.onload = continueWithEvent;
+    fileReader.readAsText(event.target.result.blob);
+    event = yield;
+
+    is(event.target.result, BLOB_DATA.join(""), "Correct text");
+
+    debug("Let's retrieve it again, create an object URL for the blob, load it via an XMLHttpRequest, and verify the contents are the same.");
+    
+    objectStore = db.transaction("foo").objectStore("foo");
+    idbRequest(objectStore.get(key1));
+    event = yield;
+    
+    debug("Got blob from database");
+    
+    let blobURL = URL.createObjectURL(event.target.result.blob);
+    
+    let xhr = new XMLHttpRequest();
+    xhr.open("GET", blobURL);
+    xhr.onload = continueWithEvent;
+    xhr.send();
+    yield;
+    
+    URL.revokeObjectURL(blobURL);    
+    is(xhr.responseText, BLOB_DATA.join(""), "Correct responseText");
+    
+    debug("Retrieve both blob entries from the database and verify contents.");
+
+    var objects = []
+    
+    objectStore = db.transaction("foo").objectStore("foo");
+    idbRequest(objectStore.get(key1));
+    event = yield;
+    
+    objects.push(event.target.result);
+    
+    objectStore = db.transaction("foo").objectStore("foo");
+    idbRequest(objectStore.get(key2));
+    event = yield;
+    
+    objects.push(event.target.result);
+        
+    fileReader = new FileReader();
+    fileReader.onload = continueWithEvent;
+    fileReader.readAsText(objects[0].blob);
+    event = yield;
+
+    is(event.target.result, BLOB_DATA.join(""), "Correct text on first value");
+
+    fileReader = new FileReader();
+    fileReader.onload = continueWithEvent;
+    fileReader.readAsText(objects[1].blob);
+    event = yield;
+
+    is(event.target.result, BLOB_DATA.join(""), "Correct text on second value");
+
+    let cursorResults = [];
+    
+    objectStore = db.transaction("foo").objectStore("foo");
+    objectStore.openCursor().onsuccess = function(event) {
+      let cursor = event.target.result;
+      if (cursor) {
+        debug("Got item from cursor");
+        cursorResults.push(cursor.value);
+        cursor.continue();
+      }
+      else {
+        debug("Finished cursor");
+        asyncContinue();
+      }
+    };
+    yield;
+
+    is(cursorResults.length, 2, "Got right number of items");
+
+    fileReader = new FileReader();
+    fileReader.onload = continueWithEvent;
+    fileReader.readAsText(cursorResults[0].blob);
+    event = yield;
+    
+    is(event.target.result, BLOB_DATA.join(""), "Correct text");
+
+    debug("Retrieve blobs from database via index and verify contents.");
+    
+    index = db.transaction("foo").objectStore("foo").index("foo");
+    idbRequest(index.get(INDEX_KEY)); 
+    event = yield;
+    
+    debug("Got blob from database");
+    
+    fileReader = new FileReader();
+    fileReader.onload = continueWithEvent;
+    fileReader.readAsText(event.target.result.blob);
+    event = yield;
+    
+    is(event.target.result, BLOB_DATA.join(""), "Correct text");
+        
+    cursorResults = [];
+    
+    index = db.transaction("foo").objectStore("foo").index("foo");
+    index.openCursor().onsuccess = function(event) {
+      let cursor = event.target.result;
+      if (cursor) {
+        debug("Got item from cursor");
+        cursorResults.push(cursor.value);
+        cursor.continue();
+      }
+      else {
+        debug("Finished cursor");
+        asyncContinue();
+      }
+    };
+    yield;
+    
+    is(cursorResults.length, 2, "Got right number of items");
+    
+    fileReader = new FileReader();
+    fileReader.onload = continueWithEvent;
+    fileReader.readAsText(cursorResults[0].blob);
+    event = yield;
+    
+    is(event.target.result, BLOB_DATA.join(""), "Correct text");
+    
+    fileReader = new FileReader();
+    fileReader.onload = continueWithEvent;
+    fileReader.readAsText(cursorResults[1].blob);
+    event = yield;
+    
+    is(event.target.result, BLOB_DATA.join(""), "Correct text");
+    
+    debug("Slice the the retrieved blob and verify its contents.");
+    
+    let slice = cursorResults[1].blob.slice(0, BLOB_DATA[0].length);
+    
+    fileReader = new FileReader();
+    fileReader.onload = continueWithEvent;
+    fileReader.readAsText(slice);
+    event = yield;
+    
+    is(event.target.result, BLOB_DATA[0], "Correct text");
+    
+    debug("Store a blob back in the database, and keep holding on to the " +
+         "blob, verifying that it still can be read.");
+    
+    objectStore = db.transaction("foo").objectStore("foo");
+    idbRequest(objectStore.get(key1)); 
+    event = yield;
+    
+    let blobFromDB = event.target.result.blob;
+    debug("Got blob from database");
+    
+    let txn = db.transaction("foo", "readwrite");
+    txn.objectStore("foo").put(event.target.result, key1);
+    txn.oncomplete = continueWithEvent;
+    event = yield;
+    
+    debug("Stored blob back into database");
+    
+    fileReader = new FileReader();
+    fileReader.onload = continueWithEvent;
+    fileReader.readAsText(blobFromDB);
+    event = yield;
+    
+    is(event.target.result, BLOB_DATA.join(""), "Correct text");
+    
+    blobURL = URL.createObjectURL(blobFromDB);
+    
+    xhr = new XMLHttpRequest();
+    xhr.open("GET", blobURL);
+    xhr.onload = continueWithEvent;
+    xhr.send();
+    yield;
+    
+    URL.revokeObjectURL(blobURL);
+    
+    is(xhr.responseText, BLOB_DATA.join(""), "Correct responseText");
+    
+    finishJSTest();
+ }
\ No newline at end of file
index c2932bd..7c5464e 100644 (file)
@@ -1,3 +1,27 @@
+2016-05-20  Brady Eidson  <beidson@apple.com>
+
+        Modern IDB: Properly handle blobs in Workers.
+        https://bugs.webkit.org/show_bug.cgi?id=157947
+
+        Reviewed by Alex Christensen.
+
+        Test: storage/indexeddb/modern/blob-simple-workers.html
+
+        * Modules/indexeddb/IDBTransaction.cpp:
+        (WebCore::IDBTransaction::putOrAddOnServer): Use writeBlobsToDiskForIndexedDBSynchronously from
+          background threads instead of the asynchronous form.
+        
+        Add ability to set an existing empty IDBValue to be an isolated copy of a different IDBValue:
+        * Modules/indexeddb/IDBValue.cpp:
+        (WebCore::IDBValue::setAsIsolatedCopy):
+        (WebCore::IDBValue::isolatedCopy):
+        * Modules/indexeddb/IDBValue.h:
+        
+        Add a method - only to be called from a non-main thread - that synchronously writes blobs to disk:
+        * bindings/js/SerializedScriptValue.cpp:
+        (WebCore::SerializedScriptValue::writeBlobsToDiskForIndexedDBSynchronously):
+        * bindings/js/SerializedScriptValue.h:
+        
 2016-05-20  Anders Carlsson  <andersca@apple.com>
 
         Allow named images on iOS
index ef4ced0..bf4ec6f 100644 (file)
@@ -938,7 +938,7 @@ Ref<IDBRequest> IDBTransaction::requestPutOrAdd(ScriptExecutionContext& context,
 void IDBTransaction::putOrAddOnServer(IDBClient::TransactionOperation& operation, RefPtr<IDBKey> key, RefPtr<SerializedScriptValue> value, const IndexedDB::ObjectStoreOverwriteMode& overwriteMode)
 {
     LOG(IndexedDB, "IDBTransaction::putOrAddOnServer");
-    ASSERT(currentThread() == m_database->originThreadID());
+    ASSERT(currentThread() == originThreadID());
     ASSERT(!isReadOnly());
     ASSERT(value);
 
@@ -947,9 +947,30 @@ void IDBTransaction::putOrAddOnServer(IDBClient::TransactionOperation& operation
         return;
     }
 
+    // Due to current limitations on our ability to post tasks back to a worker thread,
+    // workers currently write blobs to disk synchronously.
+    // FIXME: https://bugs.webkit.org/show_bug.cgi?id=157958 - Make this asynchronous after refactoring allows it.
+    if (!isMainThread()) {
+        auto idbValue = value->writeBlobsToDiskForIndexedDBSynchronously();
+        if (idbValue.data().data())
+            m_database->connectionProxy().putOrAdd(operation, key.get(), idbValue, overwriteMode);
+        else {
+            // If the IDBValue doesn't have any data, then something went wrong writing the blobs to disk.
+            // In that case, we cannot successfully store this record, so we callback with an error.
+            RefPtr<IDBClient::TransactionOperation> protectedOperation(&operation);
+            auto result = IDBResultData::error(operation.identifier(), { IDBDatabaseException::UnknownError, ASCIILiteral("Error preparing Blob/File data to be stored in object store") });
+            scriptExecutionContext()->postTask([protectedOperation, result](ScriptExecutionContext&) {
+                protectedOperation->completed(result);
+            });
+        }
+        return;
+    }
+
     RefPtr<IDBTransaction> protectedThis(this);
     RefPtr<IDBClient::TransactionOperation> protectedOperation(&operation);
     value->writeBlobsToDiskForIndexedDB([protectedThis, this, protectedOperation, key, value, overwriteMode](const IDBValue& idbValue) {
+        ASSERT(currentThread() == originThreadID());
+        ASSERT(isMainThread());
         if (idbValue.data().data()) {
             m_database->connectionProxy().putOrAdd(*protectedOperation, key.get(), idbValue, overwriteMode);
             return;
index 9f70e40..6ea1c24 100644 (file)
@@ -28,6 +28,7 @@
 
 #if ENABLE(INDEXED_DATABASE)
 
+#include "CrossThreadCopier.h"
 #include "SerializedScriptValue.h"
 
 namespace WebCore {
@@ -69,19 +70,19 @@ IDBValue::IDBValue(const ThreadSafeDataBuffer& value, const Vector<String>& blob
 {
 }
 
-IDBValue IDBValue::isolatedCopy() const
+void IDBValue::setAsIsolatedCopy(const IDBValue& other)
 {
-    IDBValue result;
-    result.m_data = m_data;
+    ASSERT(m_blobURLs.isEmpty() && m_blobFilePaths.isEmpty());
 
-    result.m_blobURLs.reserveInitialCapacity(m_blobURLs.size());
-    for (auto& url : m_blobURLs)
-        result.m_blobURLs.uncheckedAppend(url.isolatedCopy());
-
-    result.m_blobFilePaths.reserveInitialCapacity(m_blobFilePaths.size());
-    for (auto& path : m_blobFilePaths)
-        result.m_blobFilePaths.uncheckedAppend(path.isolatedCopy());
+    m_data = other.m_data;
+    m_blobURLs = CrossThreadCopier<Vector<String>>::copy(other.m_blobURLs);
+    m_blobFilePaths = CrossThreadCopier<Vector<String>>::copy(other.m_blobFilePaths);
+}
 
+IDBValue IDBValue::isolatedCopy() const
+{
+    IDBValue result;
+    result.setAsIsolatedCopy(*this);
     return result;
 }
 
index 43ec4e4..2b2cfac 100644 (file)
@@ -42,6 +42,7 @@ public:
     IDBValue(const ThreadSafeDataBuffer&, Vector<String>&& blobURLs, Vector<String>&& blobFilePaths);
     IDBValue(const ThreadSafeDataBuffer&, const Vector<String>& blobURLs, const Vector<String>& blobFilePaths);
 
+    void setAsIsolatedCopy(const IDBValue&);
     IDBValue isolatedCopy() const;
 
     const ThreadSafeDataBuffer& data() const { return m_data; }
index e90a2dc..dc58153 100644 (file)
@@ -79,6 +79,7 @@
 #include <runtime/TypedArrays.h>
 #include <wtf/HashTraits.h>
 #include <wtf/MainThread.h>
+#include <wtf/RunLoop.h>
 #include <wtf/Vector.h>
 
 using namespace JSC;
@@ -2816,6 +2817,33 @@ void SerializedScriptValue::writeBlobsToDiskForIndexedDB(std::function<void (con
         completionHandler({ *this, m_blobURLs, blobFilePaths });
     });
 }
+
+IDBValue SerializedScriptValue::writeBlobsToDiskForIndexedDBSynchronously()
+{
+    ASSERT(!isMainThread());
+
+    IDBValue value;
+    IDBValue* valuePtr = &value;
+
+    Lock lock;
+    Condition condition;
+    Condition* conditionPtr = &condition;
+    lock.lock();
+
+    RunLoop::main().dispatch([this, conditionPtr, valuePtr] {
+        writeBlobsToDiskForIndexedDB([conditionPtr, valuePtr](const IDBValue& result) {
+            ASSERT(isMainThread());
+            valuePtr->setAsIsolatedCopy(result);
+
+            conditionPtr->notifyAll();
+        });
+    });
+
+    condition.wait(lock);
+
+    return value;
+}
+
 #endif // ENABLE(INDEXED_DATABASE)
 
 } // namespace WebCore
index d4b88bc..7c75682 100644 (file)
@@ -87,6 +87,7 @@ public:
 #if ENABLE(INDEXED_DATABASE)
     Vector<String> blobURLsIsolatedCopy() const;
     void writeBlobsToDiskForIndexedDB(std::function<void (const IDBValue&)> completionHandler);
+    IDBValue writeBlobsToDiskForIndexedDBSynchronously();
 #endif // ENABLE(INDEXED_DATABASE)
 
     static Ref<SerializedScriptValue> createFromWireBytes(Vector<uint8_t>&& data)