https://bugs.webkit.org/show_bug.cgi?id=181136
<rdar://problem/
36116604>
Reviewed by Chris Dumez.
Source/WebKit:
Some uses of WebKit involve running a UIProcess as an ephemeral session for the life of the process. In this
case, we do not initialize the data path for the set of load statistics triggering an assertion.
We actually intended ephemeral sessions to consume the existing resource load data (presumably captured during
non-ephemeral browsing). This would be a read-only mode, where it would not add new entries to the load
statistics, but would take advantage of existing observations. Currently that does not happen (for this type
of WebKit embed), which forces each run as an ephemeral session to build up in-memory browsing data until it has
enough observations to begin modifying loads.
We need to set the ResourceLoadStatisticsPersistentStorage object to a "read only" mode in this case, so
that it read (but does not write) from this database.
Tested by ephemeral website data TestWebKitAPI tests.
* UIProcess/ResourceLoadStatisticsPersistentStorage.cpp:
(WebKit::ResourceLoadStatisticsPersistentStorage::create): Added to allow creation of the right style of
Persistent Storage.
(WebKit::ResourceLoadStatisticsPersistentStorage::ResourceLoadStatisticsPersistentStorage): Initialize the
new data member.
(WebKit::ResourceLoadStatisticsPersistentStorage::asyncWriteTimerFired): RELEASE_ASSERT that we never run
this method when in "read only" mode.
(WebKit::ResourceLoadStatisticsPersistentStorage::writeMemoryStoreToDisk): Ditto.
(WebKit::ResourceLoadStatisticsPersistentStorage::scheduleOrWriteMemoryStore): Return early if asked to
schedule a write operation for a "read only" persistent store.
(WebKit::ResourceLoadStatisticsPersistentStorage::finishAllPendingWorkSynchronously): RELEASE_ASSERT if we
ever shut down in "read only" mode with an active write timer.
* UIProcess/ResourceLoadStatisticsPersistentStorage.h:
* UIProcess/WebResourceLoadStatisticsStore.cpp:
(WebKit::WebResourceLoadStatisticsStore::WebResourceLoadStatisticsStore): Pass a flag indicating whether the
storage session is ephemeral or not.
* UIProcess/WebResourceLoadStatisticsStore.h:
Tools:
Add a new API test to confirm that ResourceLoadStatistics can be turned on safely for ephemeral
browsing sessions.
* Scripts/run-gtk-tests:
(GtkTestRunner): Unskip test now that it passes.
* TestWebKitAPI/Tests/WebKitCocoa/WebsiteDataStoreCustomPaths.mm:
(TEST): Add new WebsiteDataStoreEphemeral test.
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@226838
268f45cc-cd09-0410-ab3c-
d52691b4dbfc
+2018-01-11 Brent Fulgham <bfulgham@apple.com>
+
+ REGRESSION(r219530): ResourceLoadStatisticsPersistentStorage should be read-only in ephemeral sessions
+ https://bugs.webkit.org/show_bug.cgi?id=181136
+ <rdar://problem/36116604>
+
+ Reviewed by Chris Dumez.
+
+ Some uses of WebKit involve running a UIProcess as an ephemeral session for the life of the process. In this
+ case, we do not initialize the data path for the set of load statistics triggering an assertion.
+
+ We actually intended ephemeral sessions to consume the existing resource load data (presumably captured during
+ non-ephemeral browsing). This would be a read-only mode, where it would not add new entries to the load
+ statistics, but would take advantage of existing observations. Currently that does not happen (for this type
+ of WebKit embed), which forces each run as an ephemeral session to build up in-memory browsing data until it has
+ enough observations to begin modifying loads.
+
+ We need to set the ResourceLoadStatisticsPersistentStorage object to a "read only" mode in this case, so
+ that it read (but does not write) from this database.
+
+ Tested by ephemeral website data TestWebKitAPI tests.
+
+ * UIProcess/ResourceLoadStatisticsPersistentStorage.cpp:
+ (WebKit::ResourceLoadStatisticsPersistentStorage::create): Added to allow creation of the right style of
+ Persistent Storage.
+ (WebKit::ResourceLoadStatisticsPersistentStorage::ResourceLoadStatisticsPersistentStorage): Initialize the
+ new data member.
+ (WebKit::ResourceLoadStatisticsPersistentStorage::asyncWriteTimerFired): RELEASE_ASSERT that we never run
+ this method when in "read only" mode.
+ (WebKit::ResourceLoadStatisticsPersistentStorage::writeMemoryStoreToDisk): Ditto.
+ (WebKit::ResourceLoadStatisticsPersistentStorage::scheduleOrWriteMemoryStore): Return early if asked to
+ schedule a write operation for a "read only" persistent store.
+ (WebKit::ResourceLoadStatisticsPersistentStorage::finishAllPendingWorkSynchronously): RELEASE_ASSERT if we
+ ever shut down in "read only" mode with an active write timer.
+ * UIProcess/ResourceLoadStatisticsPersistentStorage.h:
+ * UIProcess/WebResourceLoadStatisticsStore.cpp:
+ (WebKit::WebResourceLoadStatisticsStore::WebResourceLoadStatisticsStore): Pass a flag indicating whether the
+ storage session is ephemeral or not.
+ * UIProcess/WebResourceLoadStatisticsStore.h:
+
2018-01-11 Keith Rollin <krollin@apple.com>
Add optional logging of ITP-related user interaction information
/*
- * Copyright (C) 2017 Apple Inc. All rights reserved.
+ * Copyright (C) 2017-2018 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
return KeyedDecoder::decoder(reinterpret_cast<const uint8_t*>(buffer.data()), buffer.size());
}
-ResourceLoadStatisticsPersistentStorage::ResourceLoadStatisticsPersistentStorage(WebResourceLoadStatisticsStore& store, const String& storageDirectoryPath)
+ResourceLoadStatisticsPersistentStorage::ResourceLoadStatisticsPersistentStorage(WebResourceLoadStatisticsStore& store, const String& storageDirectoryPath, IsReadOnly isReadOnly)
: m_memoryStore(store)
, m_storageDirectoryPath(storageDirectoryPath)
, m_asyncWriteTimer(RunLoop::main(), this, &ResourceLoadStatisticsPersistentStorage::asyncWriteTimerFired)
+ , m_isReadOnly(isReadOnly)
{
}
void ResourceLoadStatisticsPersistentStorage::asyncWriteTimerFired()
{
ASSERT(RunLoop::isMain());
+ RELEASE_ASSERT(m_isReadOnly != IsReadOnly::Yes);
m_memoryStore.statisticsQueue().dispatch([this] () mutable {
writeMemoryStoreToDisk();
});
void ResourceLoadStatisticsPersistentStorage::writeMemoryStoreToDisk()
{
ASSERT(!RunLoop::isMain());
+ RELEASE_ASSERT(m_isReadOnly != IsReadOnly::Yes);
m_hasPendingWrite = false;
stopMonitoringDisk();
void ResourceLoadStatisticsPersistentStorage::scheduleOrWriteMemoryStore(ForceImmediateWrite forceImmediateWrite)
{
ASSERT(!RunLoop::isMain());
+ if (m_isReadOnly == IsReadOnly::Yes)
+ return;
auto timeSinceLastWrite = MonotonicTime::now() - m_lastStatisticsWriteTime;
if (forceImmediateWrite != ForceImmediateWrite::Yes && timeSinceLastWrite < minimumWriteInterval) {
void ResourceLoadStatisticsPersistentStorage::finishAllPendingWorkSynchronously()
{
+ if (m_isReadOnly == IsReadOnly::Yes) {
+ RELEASE_ASSERT(!m_asyncWriteTimer.isActive());
+ return;
+ }
+
m_asyncWriteTimer.stop();
BinarySemaphore semaphore;
/*
- * Copyright (C) 2017 Apple Inc. All rights reserved.
+ * Copyright (C) 2017-2018 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
class ResourceLoadStatisticsPersistentStorage {
public:
- ResourceLoadStatisticsPersistentStorage(WebResourceLoadStatisticsStore&, const String& storageDirectoryPath);
+ enum class IsReadOnly { No, Yes };
+ ResourceLoadStatisticsPersistentStorage(WebResourceLoadStatisticsStore&, const String& storageDirectoryPath, IsReadOnly);
~ResourceLoadStatisticsPersistentStorage();
void initialize();
std::unique_ptr<WebCore::FileMonitor> m_fileMonitor;
WallTime m_lastStatisticsFileSyncTime;
MonotonicTime m_lastStatisticsWriteTime;
+ IsReadOnly m_isReadOnly { IsReadOnly::No };
bool m_hasPendingWrite { false };
};
/*
- * Copyright (C) 2016-2017 Apple Inc. All rights reserved.
+ * Copyright (C) 2016-2018 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
return mergedDates;
}
-WebResourceLoadStatisticsStore::WebResourceLoadStatisticsStore(const String& resourceLoadStatisticsDirectory, Function<void(const String&)>&& testingCallback, UpdatePrevalentDomainsToPartitionOrBlockCookiesHandler&& updatePrevalentDomainsToPartitionOrBlockCookiesHandler, HasStorageAccessForFrameHandler&& hasStorageAccessForFrameHandler, GrantStorageAccessForFrameHandler&& grantStorageAccessForFrameHandler, RemovePrevalentDomainsHandler&& removeDomainsHandler)
+WebResourceLoadStatisticsStore::WebResourceLoadStatisticsStore(const String& resourceLoadStatisticsDirectory, Function<void(const String&)>&& testingCallback, bool isEphemeral, UpdatePrevalentDomainsToPartitionOrBlockCookiesHandler&& updatePrevalentDomainsToPartitionOrBlockCookiesHandler, HasStorageAccessForFrameHandler&& hasStorageAccessForFrameHandler, GrantStorageAccessForFrameHandler&& grantStorageAccessForFrameHandler, RemovePrevalentDomainsHandler&& removeDomainsHandler)
: m_statisticsQueue(WorkQueue::create("WebResourceLoadStatisticsStore Process Data Queue", WorkQueue::Type::Serial, WorkQueue::QOS::Utility))
- , m_persistentStorage(*this, resourceLoadStatisticsDirectory)
+ , m_persistentStorage(*this, resourceLoadStatisticsDirectory, isEphemeral ? ResourceLoadStatisticsPersistentStorage::IsReadOnly::Yes : ResourceLoadStatisticsPersistentStorage::IsReadOnly::No)
, m_updatePrevalentDomainsToPartitionOrBlockCookiesHandler(WTFMove(updatePrevalentDomainsToPartitionOrBlockCookiesHandler))
, m_hasStorageAccessForFrameHandler(WTFMove(hasStorageAccessForFrameHandler))
, m_grantStorageAccessForFrameHandler(WTFMove(grantStorageAccessForFrameHandler))
/*
- * Copyright (C) 2016-2017 Apple Inc. All rights reserved.
+ * Copyright (C) 2016-2018 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
using HasStorageAccessForFrameHandler = WTF::Function<void(const String& resourceDomain, const String& firstPartyDomain, uint64_t frameID, uint64_t pageID, WTF::Function<void(bool hasAccess)>&& callback)>;
using GrantStorageAccessForFrameHandler = WTF::Function<void(const String& resourceDomain, const String& firstPartyDomain, uint64_t frameID, uint64_t pageID, WTF::Function<void(bool wasGranted)>&& callback)>;
using RemovePrevalentDomainsHandler = WTF::Function<void (const Vector<String>&)>;
- static Ref<WebResourceLoadStatisticsStore> create(const String& resourceLoadStatisticsDirectory, Function<void (const String&)>&& testingCallback, UpdatePrevalentDomainsToPartitionOrBlockCookiesHandler&& updatePrevalentDomainsToPartitionOrBlockCookiesHandler = [](const Vector<String>&, const Vector<String>&, const Vector<String>&, ShouldClearFirst) { }, HasStorageAccessForFrameHandler&& hasStorageAccessForFrameHandler = [](const String&, const String&, uint64_t, uint64_t, WTF::Function<void(bool)>&&) { }, GrantStorageAccessForFrameHandler&& grantStorageAccessForFrameHandler = [](const String&, const String&, uint64_t, uint64_t, WTF::Function<void(bool)>&&) { }, RemovePrevalentDomainsHandler&& removeDomainsHandler = [] (const Vector<String>&) { })
+ static Ref<WebResourceLoadStatisticsStore> create(const String& resourceLoadStatisticsDirectory, Function<void (const String&)>&& testingCallback, bool isEphemeral, UpdatePrevalentDomainsToPartitionOrBlockCookiesHandler&& updatePrevalentDomainsToPartitionOrBlockCookiesHandler = [](const Vector<String>&, const Vector<String>&, const Vector<String>&, ShouldClearFirst) { }, HasStorageAccessForFrameHandler&& hasStorageAccessForFrameHandler = [](const String&, const String&, uint64_t, uint64_t, WTF::Function<void(bool)>&&) { }, GrantStorageAccessForFrameHandler&& grantStorageAccessForFrameHandler = [](const String&, const String&, uint64_t, uint64_t, WTF::Function<void(bool)>&&) { }, RemovePrevalentDomainsHandler&& removeDomainsHandler = [] (const Vector<String>&) { })
{
- return adoptRef(*new WebResourceLoadStatisticsStore(resourceLoadStatisticsDirectory, WTFMove(testingCallback), WTFMove(updatePrevalentDomainsToPartitionOrBlockCookiesHandler), WTFMove(hasStorageAccessForFrameHandler), WTFMove(grantStorageAccessForFrameHandler), WTFMove(removeDomainsHandler)));
+ return adoptRef(*new WebResourceLoadStatisticsStore(resourceLoadStatisticsDirectory, WTFMove(testingCallback), isEphemeral, WTFMove(updatePrevalentDomainsToPartitionOrBlockCookiesHandler), WTFMove(hasStorageAccessForFrameHandler), WTFMove(grantStorageAccessForFrameHandler), WTFMove(removeDomainsHandler)));
}
~WebResourceLoadStatisticsStore();
void logTestingEvent(const String&);
private:
- WebResourceLoadStatisticsStore(const String&, Function<void(const String&)>&& testingCallback, UpdatePrevalentDomainsToPartitionOrBlockCookiesHandler&&, HasStorageAccessForFrameHandler&&, GrantStorageAccessForFrameHandler&&, RemovePrevalentDomainsHandler&&);
+ WebResourceLoadStatisticsStore(const String&, Function<void(const String&)>&& testingCallback, bool isEphemeral, UpdatePrevalentDomainsToPartitionOrBlockCookiesHandler&&, HasStorageAccessForFrameHandler&&, GrantStorageAccessForFrameHandler&&, RemovePrevalentDomainsHandler&&);
void removeDataRecords();
#include "APIProcessPoolConfiguration.h"
#include "APIWebsiteDataRecord.h"
+#include "APIWebsiteDataStore.h"
#include "NetworkProcessMessages.h"
#include "StorageManager.h"
#include "StorageProcessCreationParameters.h"
return adoptRef(*new WebsiteDataStore(WTFMove(configuration), sessionID));
}
+WebsiteDataStore::Configuration::Configuration()
+ : resourceLoadStatisticsDirectory(API::WebsiteDataStore::defaultResourceLoadStatisticsDirectory())
+{
+}
+
WebsiteDataStore::WebsiteDataStore(Configuration configuration, PAL::SessionID sessionID)
: m_sessionID(sessionID)
, m_configuration(WTFMove(configuration))
}
#if HAVE(CFNETWORK_STORAGE_PARTITIONING)
- m_resourceLoadStatistics = WebResourceLoadStatisticsStore::create(m_configuration.resourceLoadStatisticsDirectory, WTFMove(callback), [this, protectedThis = makeRef(*this)] (const Vector<String>& domainsToPartition, const Vector<String>& domainsToBlock, const Vector<String>& domainsToNeitherPartitionNorBlock, ShouldClearFirst shouldClearFirst) {
+ m_resourceLoadStatistics = WebResourceLoadStatisticsStore::create(m_configuration.resourceLoadStatisticsDirectory, WTFMove(callback), m_sessionID.isEphemeral(), [this, protectedThis = makeRef(*this)] (const Vector<String>& domainsToPartition, const Vector<String>& domainsToBlock, const Vector<String>& domainsToNeitherPartitionNorBlock, ShouldClearFirst shouldClearFirst) {
updatePrevalentDomainsToPartitionOrBlockCookies(domainsToPartition, domainsToBlock, domainsToNeitherPartitionNorBlock, shouldClearFirst);
}, [this, protectedThis = makeRef(*this)] (const String& resourceDomain, const String& firstPartyDomain, uint64_t frameID, uint64_t pageID, WTF::CompletionHandler<void(bool hasAccess)>&& callback) {
hasStorageAccessForFrameHandler(resourceDomain, firstPartyDomain, frameID, pageID, WTFMove(callback));
removePrevalentDomains(domainsToRemove);
});
#else
- m_resourceLoadStatistics = WebResourceLoadStatisticsStore::create(m_configuration.resourceLoadStatisticsDirectory, WTFMove(callback));
+ m_resourceLoadStatistics = WebResourceLoadStatisticsStore::create(m_configuration.resourceLoadStatisticsDirectory, WTFMove(callback), m_sessionID.isEphemeral());
#endif
for (auto& processPool : processPools())
String resourceLoadStatisticsDirectory;
String javaScriptConfigurationDirectory;
String cookieStorageFile;
+
+ explicit Configuration();
};
static Ref<WebsiteDataStore> createNonPersistent();
static Ref<WebsiteDataStore> create(Configuration, PAL::SessionID);
+2018-01-11 Brent Fulgham <bfulgham@apple.com>
+
+ REGRESSION(r219530): ResourceLoadStatisticsPersistentStorage should be read-only in ephemeral sessions
+ https://bugs.webkit.org/show_bug.cgi?id=181136
+ <rdar://problem/36116604>
+
+ Reviewed by Chris Dumez.
+
+ Add a new API test to confirm that ResourceLoadStatistics can be turned on safely for ephemeral
+ browsing sessions.
+
+ * Scripts/run-gtk-tests:
+ (GtkTestRunner): Unskip test now that it passes.
+ * TestWebKitAPI/Tests/WebKitCocoa/WebsiteDataStoreCustomPaths.mm:
+ (TEST): Add new WebsiteDataStoreEphemeral test.
+
2018-01-11 Ryan Haddad <ryanhaddad@apple.com>
Unreviewed, rolling out r226816.
SkippedTest("WebKit2Gtk/TestWebViewEditor", "/webkit2/WebKitWebView/editable/editable", "Test hits an assertion in Debug builds", 151654, "Debug"),
SkippedTest("WebKit2Gtk/TestWebExtensions", "/webkit2/WebKitWebView/install-missing-plugins-permission-request", "Test times out", 147822),
SkippedTest("WebKit2Gtk/TestWebsiteData", "/webkit2/WebKitWebsiteData/databases", "Test fails to clear database data", 181251),
- SkippedTest("WebKit2Gtk/TestWebsiteData", "/webkit2/WebKitWebsiteData/ephemeral", "Test hits an assertion in Debug builds", 181136, "Debug"),
SkippedTest("WebKit/TestWebKit", "WebKit.MouseMoveAfterCrash", "Test is flaky", 85066),
SkippedTest("WebKit/TestWebKit", "WebKit.NewFirstVisuallyNonEmptyLayoutForImages", "Test is flaky", 85066),
SkippedTest("WebKit/TestWebKit", "WebKit.NewFirstVisuallyNonEmptyLayoutFrames", "Test fails", 85037),
EXPECT_FALSE([WKWebsiteDataStore _defaultDataStoreExists]);
}
+TEST(WebKit, WebsiteDataStoreEphemeral)
+{
+ RetainPtr<WebsiteDataStoreCustomPathsMessageHandler> handler = adoptNS([[WebsiteDataStoreCustomPathsMessageHandler alloc] init]);
+ RetainPtr<WKWebViewConfiguration> configuration = adoptNS([[WKWebViewConfiguration alloc] init]);
+ [[configuration userContentController] addScriptMessageHandler:handler.get() name:@"testHandler"];
+
+ NSURL *defaultResourceLoadStatisticsPath = [NSURL fileURLWithPath:[@"~/Library/WebKit/TestWebKitAPI/WebsiteData/ResourceLoadStatistics/" stringByExpandingTildeInPath] isDirectory:YES];
+
+ [[NSFileManager defaultManager] removeItemAtURL:defaultResourceLoadStatisticsPath error:nil];
+
+ EXPECT_FALSE([[NSFileManager defaultManager] fileExistsAtPath:defaultResourceLoadStatisticsPath.path]);
+
+ configuration.get().websiteDataStore = [WKWebsiteDataStore nonPersistentDataStore];
+ [configuration.get().websiteDataStore _setResourceLoadStatisticsEnabled:YES];
+
+ // We expect the directory to be created by starting up the data store machinery, but not the data file.
+ EXPECT_TRUE([[NSFileManager defaultManager] fileExistsAtPath:defaultResourceLoadStatisticsPath.path]);
+
+ NSURL *defaultResourceLoadStatisticsFilePath = [NSURL fileURLWithPath:[@"~/Library/WebKit/TestWebKitAPI/WebsiteData/ResourceLoadStatistics/full_browsing_session_resourceLog.plist" stringByExpandingTildeInPath] isDirectory:NO];
+ EXPECT_FALSE([[NSFileManager defaultManager] fileExistsAtPath:defaultResourceLoadStatisticsFilePath.path]);
+
+ RetainPtr<WKWebView> webView = adoptNS([[WKWebView alloc] initWithFrame:NSMakeRect(0, 0, 800, 600) configuration:configuration.get()]);
+
+ NSURLRequest *request = [NSURLRequest requestWithURL:[[NSBundle mainBundle] URLForResource:@"WebsiteDataStoreCustomPaths" withExtension:@"html" subdirectory:@"TestWebKitAPI.resources"]];
+ [webView loadRequest:request];
+
+ [[[webView configuration] processPool] _syncNetworkProcessCookies];
+
+ // Forcibly shut down everything of WebKit that we can.
+ [[[webView configuration] processPool] _terminateStorageProcess];
+ auto pid = [webView _webProcessIdentifier];
+ if (pid)
+ kill(pid, SIGKILL);
+
+ webView = nil;
+ handler = nil;
+ configuration = nil;
+
+ EXPECT_TRUE([[NSFileManager defaultManager] fileExistsAtPath:defaultResourceLoadStatisticsPath.path]);
+ EXPECT_FALSE([[NSFileManager defaultManager] fileExistsAtPath:defaultResourceLoadStatisticsFilePath.path]);
+}
+
#endif