Resource Load Statistics: Add alternate classification method
authorwilander@apple.com <wilander@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 24 Feb 2017 09:19:53 +0000 (09:19 +0000)
committerwilander@apple.com <wilander@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 24 Feb 2017 09:19:53 +0000 (09:19 +0000)
commit7378504195abdf87898ccd5a05c01eed1024b665
treea30238adbdd7a47ee8938d8bf60537117c7a67d8
parentdefed15375afe9f09153c749f9b80f0f6ae27214
Resource Load Statistics: Add alternate classification method
https://bugs.webkit.org/show_bug.cgi?id=168347
<rdar://problem/30352793>
<rdar://problem/30646710>
<rdar://problem/30660708>

Reviewed by Alex Christensen.

Source/WebCore:

This patch only adds test infrastructure in WebCore.

Tests: http/tests/loading/resourceLoadStatistics/classify-as-non-prevalent-based-on-mixed-statistics.html
       http/tests/loading/resourceLoadStatistics/classify-as-non-prevalent-based-on-sub-frame-under-top-frame-origins.html
       http/tests/loading/resourceLoadStatistics/classify-as-non-prevalent-based-on-subresource-under-top-frame-origins.html
       http/tests/loading/resourceLoadStatistics/classify-as-non-prevalent-based-on-subresource-unique-redirects-to.html
       http/tests/loading/resourceLoadStatistics/classify-as-prevalent-based-on-mixed-statistics.html
       http/tests/loading/resourceLoadStatistics/classify-as-prevalent-based-on-sub-frame-under-top-frame-origins.html
       http/tests/loading/resourceLoadStatistics/classify-as-prevalent-based-on-subresource-under-top-frame-origins.html
       http/tests/loading/resourceLoadStatistics/classify-as-prevalent-based-on-subresource-unique-redirects-to.html

* loader/ResourceLoadObserver.cpp:
(WebCore::ResourceLoadObserver::setSubframeUnderTopFrameOrigin):
(WebCore::ResourceLoadObserver::setSubresourceUnderTopFrameOrigin):
(WebCore::ResourceLoadObserver::setSubresourceUniqueRedirectTo):
* loader/ResourceLoadObserver.h:

Source/WebKit2:

This patch adds a CorePrediction-based classifier to the WebResourceLoadStatisticsStore.
The CorePrediction framework is introduced as a dependency for macOS and iOS. The patch
also adds functions to support layout tests of the feature.

* Platform/Logging.h:
    Added channel ResourceLoadStatistics.
* Platform/classifier/ResourceLoadStatisticsClassifier.h: Added.
    Pulls in the Cocoa-specific classifier for Cocoa-based platforms.
(WebKit::ResourceLoadStatisticsClassifier::ResourceLoadStatisticsClassifier):
* Platform/classifier/ResourceLoadStatisticsClassifierBase.cpp: Added.
(WebKit::ResourceLoadStatisticsClassifierBase::hasPrevalentResourceCharacteristics):
    Shared classifier logic.
(WebKit::ResourceLoadStatisticsClassifierBase::classifyWithVectorThreshold):
    Fallback classifier for when we don't have CorePrediction.
* Platform/classifier/ResourceLoadStatisticsClassifierBase.h: Added.
(WebKit::ResourceLoadStatisticsClassifierBase::ResourceLoadStatisticsClassifierBase):
(WebKit::ResourceLoadStatisticsClassifierBase::~ResourceLoadStatisticsClassifierBase):
* Platform/classifier/cocoa/CorePredictionSoftLink.h: Added.
    Soft links to CorePrediction and the two functions we use from it.
* Platform/classifier/cocoa/ResourceLoadStatisticsClassifierCocoa.cpp: Added.
(WebKit::ResourceLoadStatisticsClassifier::classify):
(WebKit::ResourceLoadStatisticsClassifier::storagePath):
(WebKit::ResourceLoadStatisticsClassifier::shouldUseCorePredictionAndHaveModelLoaded):
(WebKit::ResourceLoadStatisticsClassifier::singletonPredictionModel):
    The new classifier for Cocoa platforms.
* Platform/classifier/cocoa/ResourceLoadStatisticsClassifierCocoa.h: Added.
* PlatformGTK.cmake:
    Added Platform/classifier directory and ResourceLoadStatisticsClassifierBase.cpp.
* PlatformMac.cmake:
    Added Platform/classifier and Platform/classifier/cocoa directories, and
    source files ResourceLoadStatisticsClassifierBase.cpp and
    ResourceLoadStatisticsClassifierCocoa.cpp
* Resources/ResourceLoadStatistics/corePrediction_model: Added.
    Model file to load into CorePrediction.
* UIProcess/API/C/WKResourceLoadStatisticsManager.cpp:
(WKResourceLoadStatisticsManagerSetSubframeUnderTopFrameOrigin):
(WKResourceLoadStatisticsManagerSetSubresourceUnderTopFrameOrigin):
(WKResourceLoadStatisticsManagerSetSubresourceUniqueRedirectTo):
    Test infrastructure.
* UIProcess/API/C/WKResourceLoadStatisticsManager.h:
* UIProcess/WebResourceLoadStatisticsManager.cpp:
(WebKit::WebResourceLoadStatisticsManager::setSubframeUnderTopFrameOrigin):
(WebKit::WebResourceLoadStatisticsManager::setSubresourceUnderTopFrameOrigin):
(WebKit::WebResourceLoadStatisticsManager::setSubresourceUniqueRedirectTo):
    Test infrastructure.
* UIProcess/WebResourceLoadStatisticsManager.h:
* UIProcess/WebResourceLoadStatisticsStore.cpp:
(WebKit::WebResourceLoadStatisticsStore::WebResourceLoadStatisticsStore):
    Renamed m_storagePath to m_statisticsStoragePath for clarity.
(WebKit::WebResourceLoadStatisticsStore::classifyResource):
    Now calls the classifier through its ResourceLoadStatisticsClassifier
    member variable.
(WebKit::WebResourceLoadStatisticsStore::persistentStoragePath):
    Renamed m_storagePath to m_statisticsStoragePath for clarity.
(WebKit::WebResourceLoadStatisticsStore::writeEncoderToDisk):
    Renamed m_storagePath to m_statisticsStoragePath for clarity.
(WebKit::WebResourceLoadStatisticsStore::hasPrevalentResourceCharacteristics): Deleted.
* UIProcess/WebResourceLoadStatisticsStore.h:
* WebKit2.xcodeproj/project.pbxproj:
    Added the new classifier source files under Platform/classifier and the
    corePrediction_model file under Resources/ResourceLoadStatistics.

Tools:

Added three testRunner functions to facilitate layout tests:
- setStatisticsSubframeUnderTopFrameOrigin()
- setStatisticsSubresourceUnderTopFrameOrigin()
- setStatisticsSubresourceUniqueRedirectTo()

* WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl:
* WebKitTestRunner/InjectedBundle/TestRunner.cpp:
(WTR::TestRunner::setStatisticsSubframeUnderTopFrameOrigin):
(WTR::TestRunner::setStatisticsSubresourceUnderTopFrameOrigin):
(WTR::TestRunner::setStatisticsSubresourceUniqueRedirectTo):
* WebKitTestRunner/InjectedBundle/TestRunner.h:
* WebKitTestRunner/TestController.cpp:
(WTR::TestController::setStatisticsSubframeUnderTopFrameOrigin):
(WTR::TestController::setStatisticsSubresourceUnderTopFrameOrigin):
(WTR::TestController::setStatisticsSubresourceUniqueRedirectTo):
* WebKitTestRunner/TestController.h:
* WebKitTestRunner/TestInvocation.cpp:
(WTR::TestInvocation::didReceiveSynchronousMessageFromInjectedBundle):

LayoutTests:

* http/tests/loading/resourceLoadStatistics/classify-as-non-prevalent-based-on-mixed-statistics-expected.txt: Added.
* http/tests/loading/resourceLoadStatistics/classify-as-non-prevalent-based-on-mixed-statistics.html: Added.
* http/tests/loading/resourceLoadStatistics/classify-as-non-prevalent-based-on-sub-frame-under-top-frame-origins-expected.txt: Added.
* http/tests/loading/resourceLoadStatistics/classify-as-non-prevalent-based-on-sub-frame-under-top-frame-origins.html: Added.
* http/tests/loading/resourceLoadStatistics/classify-as-non-prevalent-based-on-subresource-under-top-frame-origins-expected.txt: Added.
* http/tests/loading/resourceLoadStatistics/classify-as-non-prevalent-based-on-subresource-under-top-frame-origins.html: Added.
* http/tests/loading/resourceLoadStatistics/classify-as-non-prevalent-based-on-subresource-unique-redirects-to-expected.txt: Added.
* http/tests/loading/resourceLoadStatistics/classify-as-non-prevalent-based-on-subresource-unique-redirects-to.html: Added.
* http/tests/loading/resourceLoadStatistics/classify-as-prevalent-based-on-mixed-statistics-expected.txt: Added.
* http/tests/loading/resourceLoadStatistics/classify-as-prevalent-based-on-mixed-statistics.html: Added.
* http/tests/loading/resourceLoadStatistics/classify-as-prevalent-based-on-sub-frame-under-top-frame-origins-expected.txt: Added.
* http/tests/loading/resourceLoadStatistics/classify-as-prevalent-based-on-sub-frame-under-top-frame-origins.html: Added.
* http/tests/loading/resourceLoadStatistics/classify-as-prevalent-based-on-subresource-under-top-frame-origins-expected.txt: Added.
* http/tests/loading/resourceLoadStatistics/classify-as-prevalent-based-on-subresource-under-top-frame-origins.html: Added.
* http/tests/loading/resourceLoadStatistics/classify-as-prevalent-based-on-subresource-unique-redirects-to-expected.txt: Added.
* http/tests/loading/resourceLoadStatistics/classify-as-prevalent-based-on-subresource-unique-redirects-to.html: Added.
* http/tests/loading/resourceLoadStatistics/non-prevalent-resource-with-user-interaction.html:
    Added a call to internals.setResourceLoadStatisticsEnabled(false) before testRunner.notifyDone().
* http/tests/loading/resourceLoadStatistics/non-prevalent-resource-without-user-interaction.html:
    Added a call to internals.setResourceLoadStatisticsEnabled(false) before testRunner.notifyDone().
* http/tests/loading/resourceLoadStatistics/prevalent-resource-with-user-interaction-timeout.html:
    Added a call to internals.setResourceLoadStatisticsEnabled(false) before testRunner.notifyDone().
* http/tests/loading/resourceLoadStatistics/prevalent-resource-with-user-interaction.html:
    Added a call to internals.setResourceLoadStatisticsEnabled(false) before testRunner.notifyDone().
* http/tests/loading/resourceLoadStatistics/prevalent-resource-without-user-interaction.html:
    Added a call to internals.setResourceLoadStatisticsEnabled(false) before testRunner.notifyDone().
* platform/gtk/TestExpectations:
    The whole http/tests/loading/resourceLoadStatistics marked as crashing based on
    Carlos Garcia Campos's assessment in https://bugs.webkit.org/show_bug.cgi?id=168171.
* platform/wk2/TestExpectations:
    The above tests are only valid for WebKit2. Marked as [ Pass ].

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@212949 268f45cc-cd09-0410-ab3c-d52691b4dbfc
51 files changed:
LayoutTests/ChangeLog
LayoutTests/http/tests/loading/resourceLoadStatistics/classify-as-non-prevalent-based-on-mixed-statistics-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/loading/resourceLoadStatistics/classify-as-non-prevalent-based-on-mixed-statistics.html [new file with mode: 0644]
LayoutTests/http/tests/loading/resourceLoadStatistics/classify-as-non-prevalent-based-on-sub-frame-under-top-frame-origins-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/loading/resourceLoadStatistics/classify-as-non-prevalent-based-on-sub-frame-under-top-frame-origins.html [new file with mode: 0644]
LayoutTests/http/tests/loading/resourceLoadStatistics/classify-as-non-prevalent-based-on-subresource-under-top-frame-origins-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/loading/resourceLoadStatistics/classify-as-non-prevalent-based-on-subresource-under-top-frame-origins.html [new file with mode: 0644]
LayoutTests/http/tests/loading/resourceLoadStatistics/classify-as-non-prevalent-based-on-subresource-unique-redirects-to-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/loading/resourceLoadStatistics/classify-as-non-prevalent-based-on-subresource-unique-redirects-to.html [new file with mode: 0644]
LayoutTests/http/tests/loading/resourceLoadStatistics/classify-as-prevalent-based-on-mixed-statistics-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/loading/resourceLoadStatistics/classify-as-prevalent-based-on-mixed-statistics.html [new file with mode: 0644]
LayoutTests/http/tests/loading/resourceLoadStatistics/classify-as-prevalent-based-on-sub-frame-under-top-frame-origins-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/loading/resourceLoadStatistics/classify-as-prevalent-based-on-sub-frame-under-top-frame-origins.html [new file with mode: 0644]
LayoutTests/http/tests/loading/resourceLoadStatistics/classify-as-prevalent-based-on-subresource-under-top-frame-origins-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/loading/resourceLoadStatistics/classify-as-prevalent-based-on-subresource-under-top-frame-origins.html [new file with mode: 0644]
LayoutTests/http/tests/loading/resourceLoadStatistics/classify-as-prevalent-based-on-subresource-unique-redirects-to-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/loading/resourceLoadStatistics/classify-as-prevalent-based-on-subresource-unique-redirects-to.html [new file with mode: 0644]
LayoutTests/http/tests/loading/resourceLoadStatistics/non-prevalent-resource-with-user-interaction.html
LayoutTests/http/tests/loading/resourceLoadStatistics/non-prevalent-resource-without-user-interaction.html
LayoutTests/http/tests/loading/resourceLoadStatistics/prevalent-resource-with-user-interaction-timeout.html
LayoutTests/http/tests/loading/resourceLoadStatistics/prevalent-resource-with-user-interaction.html
LayoutTests/http/tests/loading/resourceLoadStatistics/prevalent-resource-without-user-interaction.html
LayoutTests/platform/gtk/TestExpectations
LayoutTests/platform/wk2/TestExpectations
Source/WebCore/ChangeLog
Source/WebCore/loader/ResourceLoadObserver.cpp
Source/WebCore/loader/ResourceLoadObserver.h
Source/WebKit2/ChangeLog
Source/WebKit2/Platform/Logging.h
Source/WebKit2/Platform/classifier/ResourceLoadStatisticsClassifier.cpp [new file with mode: 0644]
Source/WebKit2/Platform/classifier/ResourceLoadStatisticsClassifier.h [new file with mode: 0644]
Source/WebKit2/Platform/classifier/cocoa/CorePredictionSoftLink.h [new file with mode: 0644]
Source/WebKit2/Platform/classifier/cocoa/ResourceLoadStatisticsClassifierCocoa.cpp [new file with mode: 0644]
Source/WebKit2/Platform/classifier/cocoa/ResourceLoadStatisticsClassifierCocoa.h [new file with mode: 0644]
Source/WebKit2/PlatformGTK.cmake
Source/WebKit2/PlatformMac.cmake
Source/WebKit2/Resources/ResourceLoadStatistics/corePrediction_model [new file with mode: 0644]
Source/WebKit2/UIProcess/API/C/WKResourceLoadStatisticsManager.cpp
Source/WebKit2/UIProcess/API/C/WKResourceLoadStatisticsManager.h
Source/WebKit2/UIProcess/WebResourceLoadStatisticsManager.cpp
Source/WebKit2/UIProcess/WebResourceLoadStatisticsManager.h
Source/WebKit2/UIProcess/WebResourceLoadStatisticsStore.cpp
Source/WebKit2/UIProcess/WebResourceLoadStatisticsStore.h
Source/WebKit2/WebKit2.xcodeproj/project.pbxproj
Tools/ChangeLog
Tools/WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl
Tools/WebKitTestRunner/InjectedBundle/TestRunner.cpp
Tools/WebKitTestRunner/InjectedBundle/TestRunner.h
Tools/WebKitTestRunner/TestController.cpp
Tools/WebKitTestRunner/TestController.h
Tools/WebKitTestRunner/TestInvocation.cpp