Add SPI to configure WebsiteDataStores with a URL for standalone web applications...
authorwilander@apple.com <wilander@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 16 Apr 2020 03:38:09 +0000 (03:38 +0000)
committerwilander@apple.com <wilander@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 16 Apr 2020 03:38:09 +0000 (03:38 +0000)
https://bugs.webkit.org/show_bug.cgi?id=209634
<rdar://problem/60943970>

Reviewed by Alex Christensen.

Source/WebKit:

This change adds a new property to _WKWebsiteDataStoreConfiguration.h called
standaloneApplicationURL with which the hosting application can inform the
website data store that it's running as a standalone web application.

This change also forwards an existing standaloneApplicationURL as a
WebCore::RegistrableDomain into ITP so that explicit exemptions can be made
to first parties of standalone web applications. The exemptions made here
all for all of ITP's website data removal. This part of the change is
covered by the new layout tests.

Tests: http/tests/resourceLoadStatistics/standalone-web-application-exempt-from-website-data-deletion-database.html
       http/tests/resourceLoadStatistics/standalone-web-application-exempt-from-website-data-deletion.html

* NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:
* NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp:
(WebKit::ResourceLoadStatisticsMemoryStore::registrableDomainsToDeleteOrRestrictWebsiteDataFor):
* NetworkProcess/Classifier/ResourceLoadStatisticsStore.h:
(WebKit::ResourceLoadStatisticsStore::setStandaloneApplicationDomain):
(WebKit::ResourceLoadStatisticsStore::standaloneApplicationDomain const):
* NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:
(WebKit::WebResourceLoadStatisticsStore::setStandaloneApplicationDomain):
* NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h:
* NetworkProcess/NetworkSession.cpp:
(WebKit::NetworkSession::NetworkSession):
(WebKit::NetworkSession::forwardResourceLoadStatisticsSettings):
* NetworkProcess/NetworkSession.h:
* Shared/ResourceLoadStatisticsParameters.h:
(WebKit::ResourceLoadStatisticsParameters::encode const):
(WebKit::ResourceLoadStatisticsParameters::decode):
* UIProcess/API/Cocoa/_WKWebsiteDataStoreConfiguration.h:
* UIProcess/API/Cocoa/_WKWebsiteDataStoreConfiguration.mm:
(-[_WKWebsiteDataStoreConfiguration standaloneApplicationURL]):
(-[_WKWebsiteDataStoreConfiguration setStandaloneApplication:]):
* UIProcess/WebProcessPool.cpp:
(WebKit::WebProcessPool::ensureNetworkProcess):
* UIProcess/WebsiteData/Cocoa/WebsiteDataStoreCocoa.mm:
(WebKit::WebsiteDataStore::platformSetNetworkParameters):
* UIProcess/WebsiteData/WebsiteDataStoreConfiguration.cpp:
(WebKit::WebsiteDataStoreConfiguration::copy const):
* UIProcess/WebsiteData/WebsiteDataStoreConfiguration.h:
(WebKit::WebsiteDataStoreConfiguration::standaloneApplicationURL const):
(WebKit::WebsiteDataStoreConfiguration::setStandaloneApplicationURL):

Tools:

Added a new test option called standaloneWebApplicationURL so that layout tests can
configure the website data store accordingly. Picking it up and using it requires
creating a new website data store with a configuration that has the standalone web
application URL.

* WebKitTestRunner/TestController.cpp:
(WTR::TestController::configureWebsiteDataStoreTemporaryDirectories):
(WTR::TestController::defaultWebsiteDataStore):
(WTR::parseStringTestHeaderValueAsURL):
(WTR::updateTestOptionsFromTestHeader):
* WebKitTestRunner/TestController.h:
* WebKitTestRunner/TestOptions.h:
(WTR::TestOptions::hasSameInitializationOptions const):
* WebKitTestRunner/cocoa/TestControllerCocoa.mm:
(WTR::TestController::platformCreateWebView):

LayoutTests:

* http/tests/resourceLoadStatistics/standalone-web-application-exempt-from-website-data-deletion-database-expected.txt: Added.
* http/tests/resourceLoadStatistics/standalone-web-application-exempt-from-website-data-deletion-database.html: Added.
* http/tests/resourceLoadStatistics/standalone-web-application-exempt-from-website-data-deletion-expected.txt: Added.
* http/tests/resourceLoadStatistics/standalone-web-application-exempt-from-website-data-deletion.html: Added.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@260169 268f45cc-cd09-0410-ab3c-d52691b4dbfc

25 files changed:
LayoutTests/ChangeLog
LayoutTests/http/tests/resourceLoadStatistics/standalone-web-application-exempt-from-website-data-deletion-database-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/resourceLoadStatistics/standalone-web-application-exempt-from-website-data-deletion-database.html [new file with mode: 0644]
LayoutTests/http/tests/resourceLoadStatistics/standalone-web-application-exempt-from-website-data-deletion-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/resourceLoadStatistics/standalone-web-application-exempt-from-website-data-deletion.html [new file with mode: 0644]
Source/WebKit/ChangeLog
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp
Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsStore.h
Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp
Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h
Source/WebKit/NetworkProcess/NetworkSession.cpp
Source/WebKit/NetworkProcess/NetworkSession.h
Source/WebKit/Shared/ResourceLoadStatisticsParameters.h
Source/WebKit/UIProcess/API/Cocoa/_WKWebsiteDataStoreConfiguration.h
Source/WebKit/UIProcess/API/Cocoa/_WKWebsiteDataStoreConfiguration.mm
Source/WebKit/UIProcess/WebProcessPool.cpp
Source/WebKit/UIProcess/WebsiteData/Cocoa/WebsiteDataStoreCocoa.mm
Source/WebKit/UIProcess/WebsiteData/WebsiteDataStoreConfiguration.cpp
Source/WebKit/UIProcess/WebsiteData/WebsiteDataStoreConfiguration.h
Tools/ChangeLog
Tools/WebKitTestRunner/TestController.cpp
Tools/WebKitTestRunner/TestController.h
Tools/WebKitTestRunner/TestOptions.h
Tools/WebKitTestRunner/cocoa/TestControllerCocoa.mm

index afb72bb..6184af7 100644 (file)
@@ -1,3 +1,16 @@
+2020-04-15  John Wilander  <wilander@apple.com>
+
+        Add SPI to configure WebsiteDataStores with a URL for standalone web applications and use it to disable first-party website data removal in ITP
+        https://bugs.webkit.org/show_bug.cgi?id=209634
+        <rdar://problem/60943970>
+
+        Reviewed by Alex Christensen.
+
+        * http/tests/resourceLoadStatistics/standalone-web-application-exempt-from-website-data-deletion-database-expected.txt: Added.
+        * http/tests/resourceLoadStatistics/standalone-web-application-exempt-from-website-data-deletion-database.html: Added.
+        * http/tests/resourceLoadStatistics/standalone-web-application-exempt-from-website-data-deletion-expected.txt: Added.
+        * http/tests/resourceLoadStatistics/standalone-web-application-exempt-from-website-data-deletion.html: Added.
+
 2020-04-15  Diego Pino Garcia  <dpino@igalia.com>
 
         [GTK] Gardening, emit baselines after r260139
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/standalone-web-application-exempt-from-website-data-deletion-database-expected.txt b/LayoutTests/http/tests/resourceLoadStatistics/standalone-web-application-exempt-from-website-data-deletion-database-expected.txt
new file mode 100644 (file)
index 0000000..5b6a435
--- /dev/null
@@ -0,0 +1,15 @@
+Check that non-cookie website data does not get removed after a period of no user interaction if the website is a standalone web application.
+
+Before deletion: Client-side cookie exists.
+Before deletion: HttpOnly cookie exists.
+Before deletion: Regular server-side cookie exists.
+Before deletion: LocalStorage entry does exist.
+Before deletion: IDB entry does exist.
+
+After deletion: HttpOnly cookie exists.
+After deletion: Client-side cookie exists.
+After deletion: Regular server-side cookie exists.
+After deletion: LocalStorage entry does exist.
+After deletion: IDB entry does exist.
+
+
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/standalone-web-application-exempt-from-website-data-deletion-database.html b/LayoutTests/http/tests/resourceLoadStatistics/standalone-web-application-exempt-from-website-data-deletion-database.html
new file mode 100644 (file)
index 0000000..64f6423
--- /dev/null
@@ -0,0 +1,255 @@
+<!-- webkit-test-runner [ standaloneWebApplicationURL=http://127.0.0.1 ] -->
+<!DOCTYPE html>
+<html>
+<head>
+    <script src="/cookies/resources/cookie-utilities.js"></script>
+    <script src="resources/util.js"></script>
+</head>
+<body onload="setTimeout('runTest()', 0)">
+<div id="description">Check that non-cookie website data does not get removed after a period of no user interaction if the website is a standalone web application.</div>
+<br>
+<div id="output"></div>
+<br>
+<script>
+    testRunner.waitUntilDone();
+    testRunner.dumpAsText();
+    testRunner.setUseITPDatabase(true);
+
+    const httpOnlyCookieName = "http-only-cookie";
+    const serverSideCookieName = "server-side-cookie";
+    const clientSideCookieName = "client-side-cookie";
+
+    function sortStringArray(a, b) {
+        a = a.toLowerCase();
+        b = b.toLowerCase();
+
+        return a > b ? 1 : b > a ? -1 : 0;
+    }
+
+    function addLinebreakToOutput() {
+        let element = document.createElement("br");
+        output.appendChild(element);
+    }
+
+    function addOutput(message) {
+        let element = document.createElement("div");
+        element.innerText = message;
+        output.appendChild(element);
+    }
+
+    function checkCookies(isAfterDeletion) {
+        let unsortedTestPassedMessages = [];
+        let cookies = internals.getCookies();
+        if (!cookies.length)
+            addOutput((isAfterDeletion ? "After" : "Before") + " script-accessible deletion: No cookies found.");
+        for (let cookie of cookies) {
+            switch (cookie.name) {
+                case httpOnlyCookieName:
+                    unsortedTestPassedMessages.push((isAfterDeletion ? "After" : "Before") + " deletion: " + (isAfterDeletion ? " " : "") + "HttpOnly cookie exists.");
+                    break;
+                case serverSideCookieName:
+                    unsortedTestPassedMessages.push((isAfterDeletion ? "After" : "Before") + " deletion: Regular server-side cookie exists.");
+                    break;
+                case clientSideCookieName:
+                    unsortedTestPassedMessages.push((isAfterDeletion ? "After" : "Before") + " deletion: Client-side cookie exists.");
+                    break;
+            }
+        }
+        let sortedTestPassedMessages = unsortedTestPassedMessages.sort(sortStringArray);
+        for (let testPassedMessage of sortedTestPassedMessages) {
+            addOutput(testPassedMessage);
+        }
+    }
+
+    const dbName = "TestDatabase";
+
+    function createIDBDataStore(callback) {
+        let request = indexedDB.open(dbName);
+        request.onerror = function() {
+            addOutput("Couldn't create indexedDB.");
+            finishTest();
+        };
+        request.onupgradeneeded = function(event) {
+            let db = event.target.result;
+            let objStore = db.createObjectStore("test", {autoIncrement: true});
+            objStore.add("value");
+            callback();
+        }
+    }
+
+    const maxIntervals = 20;
+
+    let intervalCounterIDB;
+    let checkIDBCallback;
+    let checkIDBIntervalID;
+    let semaphoreIDBCheck = false;
+    function checkIDBDataStoreExists(isAfterDeletion, callback) {
+        let request;
+        intervalCounterIDB = 0;
+        checkIDBCallback = callback;
+        if (!isAfterDeletion) {
+            // Check until there is a IDB.
+            checkIDBIntervalID = setInterval(function() {
+                if (semaphoreIDBCheck)
+                    return;
+                semaphoreIDBCheck = true;
+
+                if (++intervalCounterIDB >= maxIntervals) {
+                    clearInterval(checkIDBIntervalID);
+                    addOutput("Before deletion: IDB entry does not exist.");
+                    semaphoreIDBCheck = false;
+                    checkIDBCallback();
+                } else {
+                    request = indexedDB.open(dbName);
+                    request.onerror = function () {
+                        clearInterval(checkIDBIntervalID);
+                        addOutput("Couldn't open indexedDB.");
+                        semaphoreIDBCheck = false;
+                        finishTest();
+                    };
+                    request.onupgradeneeded = function () {
+                        // Let the next interval check again.
+                        semaphoreIDBCheck = false;
+                    };
+                    request.onsuccess = function () {
+                        clearInterval(checkIDBIntervalID);
+                        addOutput("Before deletion: IDB entry does exist.");
+                        semaphoreIDBCheck = false;
+                        checkIDBCallback();
+                    };
+                }
+            }, 200);
+        } else {
+            // Check until there is a IDB.
+            checkIDBIntervalID = setInterval(function () {
+                if (semaphoreIDBCheck)
+                    return;
+                semaphoreIDBCheck = true;
+
+                if (++intervalCounterIDB >= maxIntervals) {
+                    clearInterval(checkIDBIntervalID);
+                    addOutput("After deletion: IDB entry checks exhausted.");
+                    semaphoreIDBCheck = false;
+                    checkIDBCallback();
+                } else {
+                    request = indexedDB.open(dbName);
+                    request.onerror = function () {
+                        clearInterval(checkIDBIntervalID);
+                        addOutput("Couldn't open indexedDB.");
+                        semaphoreIDBCheck = false;
+                        finishTest();
+                    };
+                    request.onupgradeneeded = function () {
+                        // Let the next interval check again.
+                        semaphoreIDBCheck = false;
+                    };
+                    request.onsuccess = function () {
+                        clearInterval(checkIDBIntervalID);
+                        addOutput("After deletion: IDB entry does exist.");
+                        semaphoreIDBCheck = false;
+                        checkIDBCallback();
+                    };
+                }
+            }, 200);
+        }
+    }
+
+    let intervalCounterLocalStorage;
+    let checkLocalStorageCallback;
+    let checkLocalStorageIntervalID;
+    const localStorageName = "test";
+    const localStorageValue = "value";
+    function checkLocalStorageExists(isAfterDeletion, callback) {
+        intervalCounterLocalStorage = 0;
+        checkLocalStorageCallback = callback;
+        if (!isAfterDeletion) {
+            // Check until there is LocalStorage.
+            checkLocalStorageIntervalID = setInterval(function() {
+                if (++intervalCounterLocalStorage >= maxIntervals) {
+                    clearInterval(checkLocalStorageIntervalID);
+                    let value = localStorage.getItem(localStorageName);
+                    addOutput("Before deletion: LocalStorage entry " + (value === localStorageValue ? "does" : "does not") + " exist.");
+                    checkLocalStorageCallback();
+                } else if (testRunner.isStatisticsHasLocalStorage(originUnderTest)) {
+                    clearInterval(checkLocalStorageIntervalID);
+                    let value = localStorage.getItem(localStorageName);
+                    addOutput("Before deletion: LocalStorage entry " + (value === localStorageValue ? "does" : "does not") + " exist.");
+                    checkLocalStorageCallback();
+                }
+            }, 100);
+        } else {
+            // Check until there is no LocalStorage.
+            checkLocalStorageIntervalID = setInterval(function() {
+                if (++intervalCounterLocalStorage >= maxIntervals) {
+                    clearInterval(checkLocalStorageIntervalID);
+                    let value = localStorage.getItem(localStorageName);
+                    addOutput("After deletion: LocalStorage entry " + (value === localStorageValue ? "does" : "does not") + " exist.");
+                    checkLocalStorageCallback();
+                } else if (!testRunner.isStatisticsHasLocalStorage(originUnderTest)) {
+                    clearInterval(checkLocalStorageIntervalID);
+                    let value = localStorage.getItem(localStorageName);
+                    addOutput("After deletion: LocalStorage entry " + (value === localStorageValue ? "does" : "does not") + " exist.");
+                    checkLocalStorageCallback();
+                }
+            }, 100);
+        }
+    }
+
+    async function writeWebsiteDataAndContinue() {
+        // Write cookies.
+        await fetch("/cookies/resources/set-http-only-cookie.php?cookieName=" + httpOnlyCookieName, { credentials: "same-origin" });
+        await fetch("/cookies/resources/setCookies.cgi", { headers: { "Set-Cookie": serverSideCookieName + "=1; path=/;" }, credentials: "same-origin" });
+        document.cookie = clientSideCookieName + "=1";
+
+        checkCookies(false);
+
+        // Write LocalStorage
+        localStorage.setItem(localStorageName, localStorageValue);
+        checkLocalStorageExists(false, function() {
+
+            // Write IndexedDB.
+            createIDBDataStore(function () {
+                checkIDBDataStoreExists(false, function() {
+                    addLinebreakToOutput();
+                    processWebsiteDataAndContinue();
+                });
+            });
+        });
+    }
+
+    function processWebsiteDataAndContinue() {
+        testRunner.installStatisticsDidScanDataRecordsCallback(checkWebsiteDataAndContinue);
+        testRunner.statisticsProcessStatisticsAndDataRecords();
+    }
+
+    function checkWebsiteDataAndContinue() {
+        checkCookies(true);
+        checkLocalStorageExists(true, function () {
+            checkIDBDataStoreExists(true, finishTest);
+        });
+    }
+
+    function finishTest() {
+        resetCookies();
+        testRunner.setStatisticsFirstPartyWebsiteDataRemovalMode(false, function() {
+            setEnableFeature(false, function() {
+                testRunner.notifyDone();
+            });
+        });
+    }
+
+    const originUnderTest  = "http://127.0.0.1:8000";
+    function runTest() {
+        setEnableFeature(true, function () {
+            testRunner.setStatisticsFirstPartyWebsiteDataRemovalMode(true, function() {
+                testRunner.setStatisticsPrevalentResource(originUnderTest, true, function() {
+                    if (!testRunner.isStatisticsPrevalentResource(originUnderTest))
+                        addOutput("FAIL: " + originUnderTest + " didn't get classified as prevalent.");
+                    writeWebsiteDataAndContinue();
+                });
+            });
+        });
+    }
+</script>
+</body>
+</html>
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/standalone-web-application-exempt-from-website-data-deletion-expected.txt b/LayoutTests/http/tests/resourceLoadStatistics/standalone-web-application-exempt-from-website-data-deletion-expected.txt
new file mode 100644 (file)
index 0000000..5b6a435
--- /dev/null
@@ -0,0 +1,15 @@
+Check that non-cookie website data does not get removed after a period of no user interaction if the website is a standalone web application.
+
+Before deletion: Client-side cookie exists.
+Before deletion: HttpOnly cookie exists.
+Before deletion: Regular server-side cookie exists.
+Before deletion: LocalStorage entry does exist.
+Before deletion: IDB entry does exist.
+
+After deletion: HttpOnly cookie exists.
+After deletion: Client-side cookie exists.
+After deletion: Regular server-side cookie exists.
+After deletion: LocalStorage entry does exist.
+After deletion: IDB entry does exist.
+
+
diff --git a/LayoutTests/http/tests/resourceLoadStatistics/standalone-web-application-exempt-from-website-data-deletion.html b/LayoutTests/http/tests/resourceLoadStatistics/standalone-web-application-exempt-from-website-data-deletion.html
new file mode 100644 (file)
index 0000000..db844c0
--- /dev/null
@@ -0,0 +1,254 @@
+<!-- webkit-test-runner [ standaloneWebApplicationURL=http://127.0.0.1 ] -->
+<!DOCTYPE html>
+<html>
+<head>
+    <script src="/cookies/resources/cookie-utilities.js"></script>
+    <script src="resources/util.js"></script>
+</head>
+<body onload="setTimeout('runTest()', 0)">
+<div id="description">Check that non-cookie website data does not get removed after a period of no user interaction if the website is a standalone web application.</div>
+<br>
+<div id="output"></div>
+<br>
+<script>
+    testRunner.waitUntilDone();
+    testRunner.dumpAsText();
+
+    const httpOnlyCookieName = "http-only-cookie";
+    const serverSideCookieName = "server-side-cookie";
+    const clientSideCookieName = "client-side-cookie";
+
+    function sortStringArray(a, b) {
+        a = a.toLowerCase();
+        b = b.toLowerCase();
+
+        return a > b ? 1 : b > a ? -1 : 0;
+    }
+
+    function addLinebreakToOutput() {
+        let element = document.createElement("br");
+        output.appendChild(element);
+    }
+
+    function addOutput(message) {
+        let element = document.createElement("div");
+        element.innerText = message;
+        output.appendChild(element);
+    }
+
+    function checkCookies(isAfterDeletion) {
+        let unsortedTestPassedMessages = [];
+        let cookies = internals.getCookies();
+        if (!cookies.length)
+            addOutput((isAfterDeletion ? "After" : "Before") + " script-accessible deletion: No cookies found.");
+        for (let cookie of cookies) {
+            switch (cookie.name) {
+                case httpOnlyCookieName:
+                    unsortedTestPassedMessages.push((isAfterDeletion ? "After" : "Before") + " deletion: " + (isAfterDeletion ? " " : "") + "HttpOnly cookie exists.");
+                    break;
+                case serverSideCookieName:
+                    unsortedTestPassedMessages.push((isAfterDeletion ? "After" : "Before") + " deletion: Regular server-side cookie exists.");
+                    break;
+                case clientSideCookieName:
+                    unsortedTestPassedMessages.push((isAfterDeletion ? "After" : "Before") + " deletion: Client-side cookie exists.");
+                    break;
+            }
+        }
+        let sortedTestPassedMessages = unsortedTestPassedMessages.sort(sortStringArray);
+        for (let testPassedMessage of sortedTestPassedMessages) {
+            addOutput(testPassedMessage);
+        }
+    }
+
+    const dbName = "TestDatabase";
+
+    function createIDBDataStore(callback) {
+        let request = indexedDB.open(dbName);
+        request.onerror = function() {
+            addOutput("Couldn't create indexedDB.");
+            finishTest();
+        };
+        request.onupgradeneeded = function(event) {
+            let db = event.target.result;
+            let objStore = db.createObjectStore("test", {autoIncrement: true});
+            objStore.add("value");
+            callback();
+        }
+    }
+
+    const maxIntervals = 20;
+
+    let intervalCounterIDB;
+    let checkIDBCallback;
+    let checkIDBIntervalID;
+    let semaphoreIDBCheck = false;
+    function checkIDBDataStoreExists(isAfterDeletion, callback) {
+        let request;
+        intervalCounterIDB = 0;
+        checkIDBCallback = callback;
+        if (!isAfterDeletion) {
+            // Check until there is a IDB.
+            checkIDBIntervalID = setInterval(function() {
+                if (semaphoreIDBCheck)
+                    return;
+                semaphoreIDBCheck = true;
+
+                if (++intervalCounterIDB >= maxIntervals) {
+                    clearInterval(checkIDBIntervalID);
+                    addOutput("Before deletion: IDB entry does not exist.");
+                    semaphoreIDBCheck = false;
+                    checkIDBCallback();
+                } else {
+                    request = indexedDB.open(dbName);
+                    request.onerror = function () {
+                        clearInterval(checkIDBIntervalID);
+                        addOutput("Couldn't open indexedDB.");
+                        semaphoreIDBCheck = false;
+                        finishTest();
+                    };
+                    request.onupgradeneeded = function () {
+                        // Let the next interval check again.
+                        semaphoreIDBCheck = false;
+                    };
+                    request.onsuccess = function () {
+                        clearInterval(checkIDBIntervalID);
+                        addOutput("Before deletion: IDB entry does exist.");
+                        semaphoreIDBCheck = false;
+                        checkIDBCallback();
+                    };
+                }
+            }, 200);
+        } else {
+            // Check until there is a IDB.
+            checkIDBIntervalID = setInterval(function () {
+                if (semaphoreIDBCheck)
+                    return;
+                semaphoreIDBCheck = true;
+
+                if (++intervalCounterIDB >= maxIntervals) {
+                    clearInterval(checkIDBIntervalID);
+                    addOutput("After deletion: IDB entry checks exhausted.");
+                    semaphoreIDBCheck = false;
+                    checkIDBCallback();
+                } else {
+                    request = indexedDB.open(dbName);
+                    request.onerror = function () {
+                        clearInterval(checkIDBIntervalID);
+                        addOutput("Couldn't open indexedDB.");
+                        semaphoreIDBCheck = false;
+                        finishTest();
+                    };
+                    request.onupgradeneeded = function () {
+                        // Let the next interval check again.
+                        semaphoreIDBCheck = false;
+                    };
+                    request.onsuccess = function () {
+                        clearInterval(checkIDBIntervalID);
+                        addOutput("After deletion: IDB entry does exist.");
+                        semaphoreIDBCheck = false;
+                        checkIDBCallback();
+                    };
+                }
+            }, 200);
+        }
+    }
+
+    let intervalCounterLocalStorage;
+    let checkLocalStorageCallback;
+    let checkLocalStorageIntervalID;
+    const localStorageName = "test";
+    const localStorageValue = "value";
+    function checkLocalStorageExists(isAfterDeletion, callback) {
+        intervalCounterLocalStorage = 0;
+        checkLocalStorageCallback = callback;
+        if (!isAfterDeletion) {
+            // Check until there is LocalStorage.
+            checkLocalStorageIntervalID = setInterval(function() {
+                if (++intervalCounterLocalStorage >= maxIntervals) {
+                    clearInterval(checkLocalStorageIntervalID);
+                    let value = localStorage.getItem(localStorageName);
+                    addOutput("Before deletion: LocalStorage entry " + (value === localStorageValue ? "does" : "does not") + " exist.");
+                    checkLocalStorageCallback();
+                } else if (testRunner.isStatisticsHasLocalStorage(originUnderTest)) {
+                    clearInterval(checkLocalStorageIntervalID);
+                    let value = localStorage.getItem(localStorageName);
+                    addOutput("Before deletion: LocalStorage entry " + (value === localStorageValue ? "does" : "does not") + " exist.");
+                    checkLocalStorageCallback();
+                }
+            }, 100);
+        } else {
+            // Check until there is no LocalStorage.
+            checkLocalStorageIntervalID = setInterval(function() {
+                if (++intervalCounterLocalStorage >= maxIntervals) {
+                    clearInterval(checkLocalStorageIntervalID);
+                    let value = localStorage.getItem(localStorageName);
+                    addOutput("After deletion: LocalStorage entry " + (value === localStorageValue ? "does" : "does not") + " exist.");
+                    checkLocalStorageCallback();
+                } else if (!testRunner.isStatisticsHasLocalStorage(originUnderTest)) {
+                    clearInterval(checkLocalStorageIntervalID);
+                    let value = localStorage.getItem(localStorageName);
+                    addOutput("After deletion: LocalStorage entry " + (value === localStorageValue ? "does" : "does not") + " exist.");
+                    checkLocalStorageCallback();
+                }
+            }, 100);
+        }
+    }
+
+    async function writeWebsiteDataAndContinue() {
+        // Write cookies.
+        await fetch("/cookies/resources/set-http-only-cookie.php?cookieName=" + httpOnlyCookieName, { credentials: "same-origin" });
+        await fetch("/cookies/resources/setCookies.cgi", { headers: { "Set-Cookie": serverSideCookieName + "=1; path=/;" }, credentials: "same-origin" });
+        document.cookie = clientSideCookieName + "=1";
+
+        checkCookies(false);
+
+        // Write LocalStorage
+        localStorage.setItem(localStorageName, localStorageValue);
+        checkLocalStorageExists(false, function() {
+
+            // Write IndexedDB.
+            createIDBDataStore(function () {
+                checkIDBDataStoreExists(false, function() {
+                    addLinebreakToOutput();
+                    processWebsiteDataAndContinue();
+                });
+            });
+        });
+    }
+
+    function processWebsiteDataAndContinue() {
+        testRunner.installStatisticsDidScanDataRecordsCallback(checkWebsiteDataAndContinue);
+        testRunner.statisticsProcessStatisticsAndDataRecords();
+    }
+
+    function checkWebsiteDataAndContinue() {
+        checkCookies(true);
+        checkLocalStorageExists(true, function () {
+            checkIDBDataStoreExists(true, finishTest);
+        });
+    }
+
+    function finishTest() {
+        resetCookies();
+        testRunner.setStatisticsFirstPartyWebsiteDataRemovalMode(false, function() {
+            setEnableFeature(false, function() {
+                testRunner.notifyDone();
+            });
+        });
+    }
+
+    const originUnderTest  = "http://127.0.0.1:8000";
+    function runTest() {
+        setEnableFeature(true, function () {
+            testRunner.setStatisticsFirstPartyWebsiteDataRemovalMode(true, function() {
+                testRunner.setStatisticsPrevalentResource(originUnderTest, true, function() {
+                    if (!testRunner.isStatisticsPrevalentResource(originUnderTest))
+                        addOutput("FAIL: " + originUnderTest + " didn't get classified as prevalent.");
+                    writeWebsiteDataAndContinue();
+                });
+            });
+        });
+    }
+</script>
+</body>
+</html>
index de2aae2..caf7b49 100644 (file)
@@ -1,3 +1,54 @@
+2020-04-15  John Wilander  <wilander@apple.com>
+
+        Add SPI to configure WebsiteDataStores with a URL for standalone web applications and use it to disable first-party website data removal in ITP
+        https://bugs.webkit.org/show_bug.cgi?id=209634
+        <rdar://problem/60943970>
+
+        Reviewed by Alex Christensen.
+
+        This change adds a new property to _WKWebsiteDataStoreConfiguration.h called
+        standaloneApplicationURL with which the hosting application can inform the
+        website data store that it's running as a standalone web application.
+
+        This change also forwards an existing standaloneApplicationURL as a
+        WebCore::RegistrableDomain into ITP so that explicit exemptions can be made
+        to first parties of standalone web applications. The exemptions made here
+        all for all of ITP's website data removal. This part of the change is
+        covered by the new layout tests.
+
+        Tests: http/tests/resourceLoadStatistics/standalone-web-application-exempt-from-website-data-deletion-database.html
+               http/tests/resourceLoadStatistics/standalone-web-application-exempt-from-website-data-deletion.html
+
+        * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:
+        * NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp:
+        (WebKit::ResourceLoadStatisticsMemoryStore::registrableDomainsToDeleteOrRestrictWebsiteDataFor):
+        * NetworkProcess/Classifier/ResourceLoadStatisticsStore.h:
+        (WebKit::ResourceLoadStatisticsStore::setStandaloneApplicationDomain):
+        (WebKit::ResourceLoadStatisticsStore::standaloneApplicationDomain const):
+        * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:
+        (WebKit::WebResourceLoadStatisticsStore::setStandaloneApplicationDomain):
+        * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h:
+        * NetworkProcess/NetworkSession.cpp:
+        (WebKit::NetworkSession::NetworkSession):
+        (WebKit::NetworkSession::forwardResourceLoadStatisticsSettings):
+        * NetworkProcess/NetworkSession.h:
+        * Shared/ResourceLoadStatisticsParameters.h:
+        (WebKit::ResourceLoadStatisticsParameters::encode const):
+        (WebKit::ResourceLoadStatisticsParameters::decode):
+        * UIProcess/API/Cocoa/_WKWebsiteDataStoreConfiguration.h:
+        * UIProcess/API/Cocoa/_WKWebsiteDataStoreConfiguration.mm:
+        (-[_WKWebsiteDataStoreConfiguration standaloneApplicationURL]):
+        (-[_WKWebsiteDataStoreConfiguration setStandaloneApplication:]):
+        * UIProcess/WebProcessPool.cpp:
+        (WebKit::WebProcessPool::ensureNetworkProcess):
+        * UIProcess/WebsiteData/Cocoa/WebsiteDataStoreCocoa.mm:
+        (WebKit::WebsiteDataStore::platformSetNetworkParameters):
+        * UIProcess/WebsiteData/WebsiteDataStoreConfiguration.cpp:
+        (WebKit::WebsiteDataStoreConfiguration::copy const):
+        * UIProcess/WebsiteData/WebsiteDataStoreConfiguration.h:
+        (WebKit::WebsiteDataStoreConfiguration::standaloneApplicationURL const):
+        (WebKit::WebsiteDataStoreConfiguration::setStandaloneApplicationURL):
+
 2020-04-15  Chris Dumez  <cdumez@apple.com>
 
         [IPC Hardening] MachMessage::messageSize() should use checked arithmetic
index 9e9d544..f2c4ffd 100644 (file)
@@ -2461,6 +2461,8 @@ RegistrableDomainsToDeleteOrRestrictWebsiteDataFor ResourceLoadStatisticsDatabas
     Vector<DomainData> domains = this->domains();
     Vector<unsigned> domainIDsToClearGrandfathering;
     for (auto& statistic : domains) {
+        if (statistic.registrableDomain == standaloneApplicationDomain())
+            continue;
         oldestUserInteraction = std::min(oldestUserInteraction, statistic.mostRecentUserInteractionTime);
         if (shouldRemoveAllWebsiteDataFor(statistic, shouldCheckForGrandfathering)) {
             toDeleteOrRestrictFor.domainsToDeleteAllCookiesFor.append(statistic.registrableDomain);
index ca5b905..63e63f9 100644 (file)
@@ -980,6 +980,8 @@ RegistrableDomainsToDeleteOrRestrictWebsiteDataFor ResourceLoadStatisticsMemoryS
     auto oldestUserInteraction = now;
     RegistrableDomainsToDeleteOrRestrictWebsiteDataFor toDeleteOrRestrictFor;
     for (auto& statistic : m_resourceStatisticsMap.values()) {
+        if (statistic.registrableDomain == standaloneApplicationDomain())
+            continue;
         oldestUserInteraction = std::min(oldestUserInteraction, statistic.mostRecentUserInteractionTime);
         if (shouldRemoveAllWebsiteDataFor(statistic, shouldCheckForGrandfathering)) {
             toDeleteOrRestrictFor.domainsToDeleteAllCookiesFor.append(statistic.registrableDomain);
index e3df400..ef4c66f 100644 (file)
@@ -168,6 +168,7 @@ public:
     void setSameSiteStrictEnforcementEnabled(WebCore::SameSiteStrictEnforcementEnabled enabled) { m_sameSiteStrictEnforcementEnabled = enabled; };
     bool isSameSiteStrictEnforcementEnabled() const { return m_sameSiteStrictEnforcementEnabled == WebCore::SameSiteStrictEnforcementEnabled::Yes; };
     void setFirstPartyWebsiteDataRemovalMode(WebCore::FirstPartyWebsiteDataRemovalMode mode) { m_firstPartyWebsiteDataRemovalMode = mode; }
+    void setStandaloneApplicationDomain(RegistrableDomain&& domain) { m_standaloneApplicationDomain = WTFMove(domain); }
 
     virtual bool areAllThirdPartyCookiesBlockedUnder(const TopFrameDomain&) = 0;
     virtual void hasStorageAccess(const SubFrameDomain&, const TopFrameDomain&, Optional<WebCore::FrameIdentifier>, WebCore::PageIdentifier, CompletionHandler<void(bool)>&&) = 0;
@@ -246,6 +247,7 @@ protected:
     bool debugLoggingEnabled() const { return m_debugLoggingEnabled; };
     bool debugModeEnabled() const { return m_debugModeEnabled; }
     WebCore::FirstPartyWebsiteDataRemovalMode firstPartyWebsiteDataRemovalMode() const { return m_firstPartyWebsiteDataRemovalMode; }
+    RegistrableDomain standaloneApplicationDomain() const { return m_standaloneApplicationDomain; }
 
     static constexpr unsigned maxNumberOfRecursiveCallsInRedirectTraceBack { 50 };
     
@@ -286,6 +288,7 @@ private:
     bool m_dataRecordsBeingRemoved { false };
     ShouldIncludeLocalhost m_shouldIncludeLocalhost { ShouldIncludeLocalhost::Yes };
     WebCore::FirstPartyWebsiteDataRemovalMode m_firstPartyWebsiteDataRemovalMode { WebCore::FirstPartyWebsiteDataRemovalMode::AllButCookies };
+    RegistrableDomain m_standaloneApplicationDomain;
 };
 
 } // namespace WebKit
index 08ede6c..1ef8acd 100644 (file)
@@ -635,6 +635,26 @@ void WebResourceLoadStatisticsStore::setFirstPartyWebsiteDataRemovalMode(FirstPa
     });
 }
 
+void WebResourceLoadStatisticsStore::setStandaloneApplicationDomain(const RegistrableDomain& domain, CompletionHandler<void()>&& completionHandler)
+{
+    ASSERT(RunLoop::isMain());
+
+    if (isEphemeral() || domain.isEmpty()) {
+        completionHandler();
+        return;
+    }
+
+    RELEASE_LOG(ResourceLoadStatistics, "WebResourceLoadStatisticsStore::setStandaloneApplicationDomain() called with non-empty domain.");
+
+    postTask([this, domain = domain.isolatedCopy(), completionHandler = WTFMove(completionHandler)]() mutable {
+        if (m_statisticsStore)
+            m_statisticsStore->setStandaloneApplicationDomain(WTFMove(domain));
+        postTaskReply([completionHandler = WTFMove(completionHandler)]() mutable {
+            completionHandler();
+        });
+    });
+}
+
 void WebResourceLoadStatisticsStore::didCreateNetworkProcess()
 {
     ASSERT(RunLoop::isMain());
index 35b8303..3030a4d 100644 (file)
@@ -277,6 +277,7 @@ struct ThirdPartyData {
     void setThirdPartyCookieBlockingMode(WebCore::ThirdPartyCookieBlockingMode);
     void setSameSiteStrictEnforcementEnabled(WebCore::SameSiteStrictEnforcementEnabled);
     void setFirstPartyWebsiteDataRemovalMode(WebCore::FirstPartyWebsiteDataRemovalMode, CompletionHandler<void()>&&);
+    void setStandaloneApplicationDomain(const RegistrableDomain&, CompletionHandler<void()>&&);
     void didCreateNetworkProcess();
 
     void notifyResourceLoadStatisticsProcessed();
index 330427b..7524e48 100644 (file)
@@ -89,6 +89,7 @@ NetworkSession::NetworkSession(NetworkProcess& networkProcess, const NetworkSess
     , m_thirdPartyCookieBlockingMode(parameters.resourceLoadStatisticsParameters.thirdPartyCookieBlockingMode)
     , m_sameSiteStrictEnforcementEnabled(parameters.resourceLoadStatisticsParameters.sameSiteStrictEnforcementEnabled)
     , m_firstPartyWebsiteDataRemovalMode(parameters.resourceLoadStatisticsParameters.firstPartyWebsiteDataRemovalMode)
+    , m_standaloneApplicationDomain(parameters.resourceLoadStatisticsParameters.standaloneApplicationDomain)
 #endif
     , m_adClickAttribution(makeUniqueRef<AdClickAttributionManager>(networkProcess, parameters.sessionID))
     , m_testSpeedMultiplier(parameters.testSpeedMultiplier)
@@ -204,6 +205,7 @@ void NetworkSession::forwardResourceLoadStatisticsSettings()
     m_resourceLoadStatistics->setThirdPartyCookieBlockingMode(m_thirdPartyCookieBlockingMode);
     m_resourceLoadStatistics->setSameSiteStrictEnforcementEnabled(m_sameSiteStrictEnforcementEnabled);
     m_resourceLoadStatistics->setFirstPartyWebsiteDataRemovalMode(m_firstPartyWebsiteDataRemovalMode, [] { });
+    m_resourceLoadStatistics->setStandaloneApplicationDomain(m_standaloneApplicationDomain, [] { });
 }
 
 bool NetworkSession::isResourceLoadStatisticsEnabled() const
index 4634293..099ce74 100644 (file)
@@ -164,6 +164,7 @@ protected:
     WebCore::ThirdPartyCookieBlockingMode m_thirdPartyCookieBlockingMode { WebCore::ThirdPartyCookieBlockingMode::All };
     WebCore::SameSiteStrictEnforcementEnabled m_sameSiteStrictEnforcementEnabled { WebCore::SameSiteStrictEnforcementEnabled::No };
     WebCore::FirstPartyWebsiteDataRemovalMode m_firstPartyWebsiteDataRemovalMode { WebCore::FirstPartyWebsiteDataRemovalMode::AllButCookies };
+    WebCore::RegistrableDomain m_standaloneApplicationDomain;
 #endif
     bool m_isStaleWhileRevalidateEnabled { false };
     UniqueRef<AdClickAttributionManager> m_adClickAttribution;
index f497609..927aec3 100644 (file)
@@ -47,6 +47,7 @@ struct ResourceLoadStatisticsParameters {
     WebCore::SameSiteStrictEnforcementEnabled sameSiteStrictEnforcementEnabled { WebCore::SameSiteStrictEnforcementEnabled::No };
 #endif
     WebCore::FirstPartyWebsiteDataRemovalMode firstPartyWebsiteDataRemovalMode { WebCore::FirstPartyWebsiteDataRemovalMode::AllButCookies };
+    WebCore::RegistrableDomain standaloneApplicationDomain { };
     WebCore::RegistrableDomain manualPrevalentResource { };
     
     void encode(IPC::Encoder& encoder) const
@@ -63,6 +64,7 @@ struct ResourceLoadStatisticsParameters {
         encoder << sameSiteStrictEnforcementEnabled;
 #endif
         encoder << firstPartyWebsiteDataRemovalMode;
+        encoder << standaloneApplicationDomain;
         encoder << manualPrevalentResource;
     }
 
@@ -120,6 +122,11 @@ struct ResourceLoadStatisticsParameters {
         if (!firstPartyWebsiteDataRemovalMode)
             return WTF::nullopt;
 
+        Optional<WebCore::RegistrableDomain> standaloneApplicationDomain;
+        decoder >> standaloneApplicationDomain;
+        if (!standaloneApplicationDomain)
+            return WTF::nullopt;
+
         Optional<WebCore::RegistrableDomain> manualPrevalentResource;
         decoder >> manualPrevalentResource;
         if (!manualPrevalentResource)
@@ -138,6 +145,7 @@ struct ResourceLoadStatisticsParameters {
             WTFMove(*sameSiteStrictEnforcementEnabled),
 #endif
             WTFMove(*firstPartyWebsiteDataRemovalMode),
+            WTFMove(*standaloneApplicationDomain),
             WTFMove(*manualPrevalentResource),
         }};
     }
index 4801379..eb596b9 100644 (file)
@@ -76,6 +76,8 @@ WK_CLASS_AVAILABLE(macos(10.13), ios(11.0))
 
 @property (nonatomic, nullable, copy) NSURL *alternativeServicesStorageDirectory WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
 
+@property (nonatomic, nullable, copy) NSURL *standaloneApplicationURL WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
+
 // Testing only.
 @property (nonatomic) BOOL allLoadsBlockedByDeviceManagementRestrictionsForTesting WK_API_AVAILABLE(macos(10.15), ios(13.0));
 
index d428d77..0153116 100644 (file)
@@ -443,6 +443,16 @@ static void checkURLArgument(NSURL *url)
     _configuration->setProxyConfiguration((__bridge CFDictionaryRef)[configuration copy]);
 }
 
+- (NSURL *)standaloneApplicationURL
+{
+    return _configuration->standaloneApplicationURL();
+}
+
+- (void)setStandaloneApplicationURL:(NSURL *)url
+{
+    _configuration->setStandaloneApplicationURL(url);
+}
+
 - (BOOL)allLoadsBlockedByDeviceManagementRestrictionsForTesting
 {
     return _configuration->allLoadsBlockedByDeviceManagementRestrictionsForTesting();
index b649dbd..9737fd3 100644 (file)
@@ -604,6 +604,7 @@ NetworkProcessProxy& WebProcessPool::ensureNetworkProcess(WebsiteDataStore* with
     WebCore::ThirdPartyCookieBlockingMode thirdPartyCookieBlockingMode = WebCore::ThirdPartyCookieBlockingMode::All;
     WebCore::SameSiteStrictEnforcementEnabled sameSiteStrictEnforcementEnabled = WebCore::SameSiteStrictEnforcementEnabled::No;
 #endif
+    WebCore::RegistrableDomain standaloneApplicationDomain { };
     WebCore::FirstPartyWebsiteDataRemovalMode firstPartyWebsiteDataRemovalMode = WebCore::FirstPartyWebsiteDataRemovalMode::AllButCookies;
     WebCore::RegistrableDomain manualPrevalentResource { };
     WEB_PROCESS_POOL_ADDITIONS_2
@@ -621,6 +622,7 @@ NetworkProcessProxy& WebProcessPool::ensureNetworkProcess(WebsiteDataStore* with
             sameSiteStrictEnforcementEnabled = networkSessionParameters.resourceLoadStatisticsParameters.sameSiteStrictEnforcementEnabled;
 #endif
             firstPartyWebsiteDataRemovalMode = networkSessionParameters.resourceLoadStatisticsParameters.firstPartyWebsiteDataRemovalMode;
+            standaloneApplicationDomain = networkSessionParameters.resourceLoadStatisticsParameters.standaloneApplicationDomain;
             manualPrevalentResource = networkSessionParameters.resourceLoadStatisticsParameters.manualPrevalentResource;
         }
 
@@ -646,6 +648,7 @@ NetworkProcessProxy& WebProcessPool::ensureNetworkProcess(WebsiteDataStore* with
             sameSiteStrictEnforcementEnabled = networkSessionParameters.resourceLoadStatisticsParameters.sameSiteStrictEnforcementEnabled;
 #endif
             firstPartyWebsiteDataRemovalMode = networkSessionParameters.resourceLoadStatisticsParameters.firstPartyWebsiteDataRemovalMode;
+            standaloneApplicationDomain = networkSessionParameters.resourceLoadStatisticsParameters.standaloneApplicationDomain;
             manualPrevalentResource = networkSessionParameters.resourceLoadStatisticsParameters.manualPrevalentResource;
         }
 
index 87cb8b7..5fbf562 100644 (file)
@@ -180,6 +180,7 @@ void WebsiteDataStore::platformSetNetworkParameters(WebsiteDataStoreParameters&
     parameters.networkSessionParameters.resourceLoadStatisticsParameters.enableDebugMode = enableResourceLoadStatisticsDebugMode;
     parameters.networkSessionParameters.resourceLoadStatisticsParameters.sameSiteStrictEnforcementEnabled = sameSiteStrictEnforcementEnabled;
     parameters.networkSessionParameters.resourceLoadStatisticsParameters.firstPartyWebsiteDataRemovalMode = firstPartyWebsiteDataRemovalMode;
+    parameters.networkSessionParameters.resourceLoadStatisticsParameters.standaloneApplicationDomain = WebCore::RegistrableDomain { m_configuration->standaloneApplicationURL() };
     parameters.networkSessionParameters.resourceLoadStatisticsParameters.manualPrevalentResource = WTFMove(resourceLoadStatisticsManualPrevalentResource);
 
     auto cookieFile = resolvedCookieStorageFile();
index 963e458..a75a790 100644 (file)
@@ -93,6 +93,7 @@ Ref<WebsiteDataStoreConfiguration> WebsiteDataStoreConfiguration::copy() const
     copy->m_allowsServerPreconnect = this->m_allowsServerPreconnect;
     copy->m_requiresSecureHTTPSProxyConnection = this->m_requiresSecureHTTPSProxyConnection;
     copy->m_preventsSystemHTTPProxyAuthentication = this->m_preventsSystemHTTPProxyAuthentication;
+    copy->m_standaloneApplicationURL = this->m_standaloneApplicationURL;
 #if PLATFORM(COCOA)
     if (m_proxyConfiguration)
         copy->m_proxyConfiguration = adoptCF(CFDictionaryCreateCopy(nullptr, this->m_proxyConfiguration.get()));
index ee7a04c..0a1087b 100644 (file)
@@ -159,6 +159,9 @@ public:
     bool requiresSecureHTTPSProxyConnection() const { return m_requiresSecureHTTPSProxyConnection; };
     void setRequiresSecureHTTPSProxyConnection(bool requires) { m_requiresSecureHTTPSProxyConnection = requires; }
 
+    const URL& standaloneApplicationURL() const { return m_standaloneApplicationURL; }
+    void setStandaloneApplicationURL(URL&& url) { m_standaloneApplicationURL = WTFMove(url); }
+
 private:
     IsPersistent m_isPersistent { IsPersistent::No };
 
@@ -203,6 +206,7 @@ private:
     bool m_preventsSystemHTTPProxyAuthentication { false };
     bool m_requiresSecureHTTPSProxyConnection { false };
     unsigned m_testSpeedMultiplier { 1 };
+    URL m_standaloneApplicationURL;
 #if PLATFORM(COCOA)
     RetainPtr<CFDictionaryRef> m_proxyConfiguration;
 #endif
index ec0f89f..f49b011 100644 (file)
@@ -1,3 +1,27 @@
+2020-04-15  John Wilander  <wilander@apple.com>
+
+        Add SPI to configure WebsiteDataStores with a URL for standalone web applications and use it to disable first-party website data removal in ITP
+        https://bugs.webkit.org/show_bug.cgi?id=209634
+        <rdar://problem/60943970>
+
+        Reviewed by Alex Christensen.
+
+        Added a new test option called standaloneWebApplicationURL so that layout tests can
+        configure the website data store accordingly. Picking it up and using it requires
+        creating a new website data store with a configuration that has the standalone web
+        application URL.
+
+        * WebKitTestRunner/TestController.cpp:
+        (WTR::TestController::configureWebsiteDataStoreTemporaryDirectories):
+        (WTR::TestController::defaultWebsiteDataStore):
+        (WTR::parseStringTestHeaderValueAsURL):
+        (WTR::updateTestOptionsFromTestHeader):
+        * WebKitTestRunner/TestController.h:
+        * WebKitTestRunner/TestOptions.h:
+        (WTR::TestOptions::hasSameInitializationOptions const):
+        * WebKitTestRunner/cocoa/TestControllerCocoa.mm:
+        (WTR::TestController::platformCreateWebView):
+
 2020-04-15  Chris Dumez  <cdumez@apple.com>
 
         [IPC Hardening] MachMessage::messageSize() should use checked arithmetic
index 1d29fad..9887052 100644 (file)
@@ -521,28 +521,33 @@ WKRetainPtr<WKContextConfigurationRef> TestController::generateContextConfigurat
     return configuration;
 }
 
+void TestController::configureWebsiteDataStoreTemporaryDirectories(WKWebsiteDataStoreConfigurationRef configuration)
+{
+    if (const char* dumpRenderTreeTemp = libraryPathForTesting()) {
+        String temporaryFolder = String::fromUTF8(dumpRenderTreeTemp);
+
+        WKWebsiteDataStoreConfigurationSetApplicationCacheDirectory(configuration, toWK(temporaryFolder + pathSeparator + "ApplicationCache").get());
+        WKWebsiteDataStoreConfigurationSetNetworkCacheDirectory(configuration, toWK(temporaryFolder + pathSeparator + "Cache").get());
+        WKWebsiteDataStoreConfigurationSetCacheStorageDirectory(configuration, toWK(temporaryFolder + pathSeparator + "CacheStorage").get());
+        WKWebsiteDataStoreConfigurationSetIndexedDBDatabaseDirectory(configuration, toWK(temporaryFolder + pathSeparator + "Databases" + pathSeparator + "IndexedDB").get());
+        WKWebsiteDataStoreConfigurationSetLocalStorageDirectory(configuration, toWK(temporaryFolder + pathSeparator + "LocalStorage").get());
+        WKWebsiteDataStoreConfigurationSetWebSQLDatabaseDirectory(configuration, toWK(temporaryFolder + pathSeparator + "Databases" + pathSeparator + "WebSQL").get());
+        WKWebsiteDataStoreConfigurationSetMediaKeysStorageDirectory(configuration, toWK(temporaryFolder + pathSeparator + "MediaKeys").get());
+        WKWebsiteDataStoreConfigurationSetResourceLoadStatisticsDirectory(configuration, toWK(temporaryFolder + pathSeparator + "ResourceLoadStatistics").get());
+        WKWebsiteDataStoreConfigurationSetServiceWorkerRegistrationDirectory(configuration, toWK(temporaryFolder + pathSeparator + "ServiceWorkers").get());
+        WKWebsiteDataStoreConfigurationSetPerOriginStorageQuota(configuration, 400 * 1024);
+        WKWebsiteDataStoreConfigurationSetNetworkCacheSpeculativeValidationEnabled(configuration, true);
+        WKWebsiteDataStoreConfigurationSetStaleWhileRevalidateEnabled(configuration, true);
+        WKWebsiteDataStoreConfigurationSetTestingSessionEnabled(configuration, true);
+    }
+}
+
 WKWebsiteDataStoreRef TestController::defaultWebsiteDataStore()
 {
     static WKWebsiteDataStoreRef dataStore = nullptr;
     if (!dataStore) {
         auto configuration = adoptWK(WKWebsiteDataStoreConfigurationCreate());
-        if (const char* dumpRenderTreeTemp = libraryPathForTesting()) {
-            String temporaryFolder = String::fromUTF8(dumpRenderTreeTemp);
-
-            WKWebsiteDataStoreConfigurationSetApplicationCacheDirectory(configuration.get(), toWK(temporaryFolder + pathSeparator + "ApplicationCache").get());
-            WKWebsiteDataStoreConfigurationSetNetworkCacheDirectory(configuration.get(), toWK(temporaryFolder + pathSeparator + "Cache").get());
-            WKWebsiteDataStoreConfigurationSetCacheStorageDirectory(configuration.get(), toWK(temporaryFolder + pathSeparator + "CacheStorage").get());
-            WKWebsiteDataStoreConfigurationSetIndexedDBDatabaseDirectory(configuration.get(), toWK(temporaryFolder + pathSeparator + "Databases" + pathSeparator + "IndexedDB").get());
-            WKWebsiteDataStoreConfigurationSetLocalStorageDirectory(configuration.get(), toWK(temporaryFolder + pathSeparator + "LocalStorage").get());
-            WKWebsiteDataStoreConfigurationSetWebSQLDatabaseDirectory(configuration.get(), toWK(temporaryFolder + pathSeparator + "Databases" + pathSeparator + "WebSQL").get());
-            WKWebsiteDataStoreConfigurationSetMediaKeysStorageDirectory(configuration.get(), toWK(temporaryFolder + pathSeparator + "MediaKeys").get());
-            WKWebsiteDataStoreConfigurationSetResourceLoadStatisticsDirectory(configuration.get(), toWK(temporaryFolder + pathSeparator + "ResourceLoadStatistics").get());
-            WKWebsiteDataStoreConfigurationSetServiceWorkerRegistrationDirectory(configuration.get(), toWK(temporaryFolder + pathSeparator + "ServiceWorkers").get());
-            WKWebsiteDataStoreConfigurationSetPerOriginStorageQuota(configuration.get(), 400 * 1024);
-            WKWebsiteDataStoreConfigurationSetNetworkCacheSpeculativeValidationEnabled(configuration.get(), true);
-            WKWebsiteDataStoreConfigurationSetStaleWhileRevalidateEnabled(configuration.get(), true);
-            WKWebsiteDataStoreConfigurationSetTestingSessionEnabled(configuration.get(), true);
-        }
+        configureWebsiteDataStoreTemporaryDirectories(configuration.get());
         dataStore = WKWebsiteDataStoreCreateWithConfiguration(configuration.get());
     }
     return dataStore;
@@ -1360,6 +1365,11 @@ static std::string parseStringTestHeaderValueAsRelativePath(const std::string& v
     return toSTD(adoptWK(WKURLCopyPath(relativeURL.get())));
 }
 
+static std::string parseStringTestHeaderValueAsURL(const std::string& value)
+{
+    return toSTD(adoptWK(WKURLCopyString(createTestURL(value.c_str()))));
+}
+
 static void updateTestOptionsFromTestHeader(TestOptions& testOptions, const std::string& pathOrURL, const std::string& absolutePath)
 {
     std::string filename = absolutePath;
@@ -1515,7 +1525,9 @@ static void updateTestOptionsFromTestHeader(TestOptions& testOptions, const std:
             testOptions.allowTopNavigationToDataURLs = parseBooleanTestHeaderValue(value);
         else if (key == "enableInAppBrowserPrivacy")
             testOptions.enableInAppBrowserPrivacy = parseBooleanTestHeaderValue(value);
-        
+        else if (key == "standaloneWebApplicationURL")
+            testOptions.standaloneWebApplicationURL = parseStringTestHeaderValueAsURL(value);
+
         pairStart = pairEnd + 1;
     }
 }
index 8bca645..bc3ad75 100644 (file)
@@ -88,6 +88,7 @@ private:
 class TestController {
 public:
     static TestController& singleton();
+    static void configureWebsiteDataStoreTemporaryDirectories(WKWebsiteDataStoreConfigurationRef);
     static WKWebsiteDataStoreRef defaultWebsiteDataStore();
 
     static const unsigned viewWidth;
index 701de9c..a2af12f 100644 (file)
@@ -111,6 +111,7 @@ struct TestOptions {
     std::string applicationManifest;
     std::string jscOptions;
     std::string additionalSupportedImageTypes;
+    std::string standaloneWebApplicationURL;
     HashMap<String, bool> experimentalFeatures;
     HashMap<String, bool> internalDebugFeatures;
     String contentMode;
@@ -170,7 +171,8 @@ struct TestOptions {
             || enableCaptureVideoInGPUProcess != options.enableCaptureVideoInGPUProcess
             || enableCaptureAudioInGPUProcess != options.enableCaptureAudioInGPUProcess
             || allowTopNavigationToDataURLs != options.allowTopNavigationToDataURLs
-            || enableInAppBrowserPrivacy != options.enableInAppBrowserPrivacy)
+            || enableInAppBrowserPrivacy != options.enableInAppBrowserPrivacy
+            || standaloneWebApplicationURL != options.standaloneWebApplicationURL)
             return false;
 
         if (!contextOptions.hasSameInitializationOptions(options.contextOptions))
index daa8a07..2562e6d 100644 (file)
@@ -51,6 +51,7 @@
 #import <WebKit/_WKApplicationManifest.h>
 #import <WebKit/_WKUserContentExtensionStore.h>
 #import <WebKit/_WKUserContentExtensionStorePrivate.h>
+#import <WebKit/_WKWebsiteDataStoreConfiguration.h>
 #import <wtf/MainThread.h>
 #import <wtf/cocoa/VectorCocoa.h>
 #import <wtf/spi/cocoa/SecuritySPI.h>
@@ -154,10 +155,15 @@ void TestController::platformCreateWebView(WKPageConfigurationRef, const TestOpt
     if (options.enableEditableImages)
         [copiedConfiguration _setEditableImagesEnabled:YES];
 
-    if (options.useEphemeralSession) {
-        auto ephemeralWebsiteDataStore = [WKWebsiteDataStore nonPersistentDataStore];
-        [ephemeralWebsiteDataStore _setResourceLoadStatisticsEnabled:YES];
-        [copiedConfiguration setWebsiteDataStore:ephemeralWebsiteDataStore];
+    if (options.useEphemeralSession || options.standaloneWebApplicationURL.length()) {
+        auto websiteDataStoreConfig = options.useEphemeralSession ? [[[_WKWebsiteDataStoreConfiguration alloc] initNonPersistentConfiguration] autorelease] : [[[_WKWebsiteDataStoreConfiguration alloc] init] autorelease];
+        if (!options.useEphemeralSession)
+            configureWebsiteDataStoreTemporaryDirectories((WKWebsiteDataStoreConfigurationRef)websiteDataStoreConfig);
+        if (options.standaloneWebApplicationURL.length())
+            [websiteDataStoreConfig setStandaloneApplicationURL:[NSURL URLWithString:[NSString stringWithUTF8String:options.standaloneWebApplicationURL.c_str()]]];
+        auto websiteDataStore = [[[WKWebsiteDataStore alloc] _initWithConfiguration:websiteDataStoreConfig] autorelease];
+        [websiteDataStore _setResourceLoadStatisticsEnabled:YES];
+        [copiedConfiguration setWebsiteDataStore:websiteDataStore];
     }
 
     [copiedConfiguration _setAllowTopNavigationToDataURLs:options.allowTopNavigationToDataURLs];