[Content Filtering] Determine navigation and content policy before continuing to...
authoraestes@apple.com <aestes@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 31 Aug 2015 23:27:20 +0000 (23:27 +0000)
committeraestes@apple.com <aestes@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 31 Aug 2015 23:27:20 +0000 (23:27 +0000)
commitcf4e561d5229e29a29195d03ff0b0b9368bcca90
treecbac922a003944d36869cfc2529abc3139e33b80
parent9715a9dbb497566ab7bc7a58b630289f1e8b364b
[Content Filtering] Determine navigation and content policy before continuing to filter a load
https://bugs.webkit.org/show_bug.cgi?id=148506

Reviewed by Brady Eidson.

Source/WebCore:

Prior to this change, ContentFilter would hide from DocumentLoader all CachedRawResourceClient callbacks until
a decision was made, then replay the missed callbacks. This approach interacted poorly with some features of
the loader, notably appcache and downloads. In the case of appcache, DocumentLoader might not have a chance to
check for substitute data until the original load has finished, wasting bandwidth, and might receive duplicate
or out-of-order callbacks. In the case of downloads, it would often be too late to convert the existing
connection to a download, leading to restarted downloads or outright failures.

Bandaids were put in place for these issues in r188150, r188486, and r188851 to fix crashes or serious
regressions in behavior, but these weren't complete fixes. They did not solve any of the duplicate data loading
problems, and they did not make downloads work reliably in all cases.

This patch rolls out the bandaids (but keeps their tests) and replaces them with a more robust fix. Instead of
hiding callbacks from DocumentLoader, ContentFilter now delivers willSendRequest(), redirectReceived(), and
responseReceived() to DocumentLoader immediately, and cancels filtering if DocumentLoader decides to ignore the
load, download it, or load substitute data. ContentFilter continues to buffer incoming data to prevent partial
rendering of blocked content.

The existing tests for r188150 and r188851 were kept, the test for r188486 was rewritten to be specific to
content filtering, and new tests were added to cover the case where ContentFilter is still undecided after a
load finishes.

Tests: contentfiltering/allow-never.html
       contentfiltering/block-never.html
       ContentFiltering.AllowDownloadAfterAddData
       ContentFiltering.AllowDownloadAfterFinishedAddingData
       ContentFiltering.AllowDownloadAfterRedirect
       ContentFiltering.AllowDownloadAfterResponse
       ContentFiltering.AllowDownloadAfterWillSendRequest
       ContentFiltering.AllowDownloadNever
       ContentFiltering.BlockDownloadAfterAddData
       ContentFiltering.BlockDownloadAfterFinishedAddingData
       ContentFiltering.BlockDownloadAfterRedirect
       ContentFiltering.BlockDownloadAfterResponse
       ContentFiltering.BlockDownloadAfterWillSendRequest
       ContentFiltering.BlockDownloadNever

* bindings/js/JSMockContentFilterSettingsCustom.cpp:
(WebCore::JSMockContentFilterSettings::decisionPoint): Taught to handle DecisionPoint::Never, and rewrote to
not need a set of const uint8_ts that mirror the DecisionPoint enum.
(WebCore::JSMockContentFilterSettings::setDecisionPoint): Ditto.
(WebCore::toJSValue): Rewrote to not need a set of const uint8_ts that mirror the Decision enum.
(WebCore::toDecision): Ditto.
* loader/ContentFilter.cpp:
(WebCore::ContentFilter::createIfEnabled): Renamed from createIfNeeded, and changed to take a DocumentLoader&
instead of a DecisionFunction.
(WebCore::ContentFilter::ContentFilter):
(WebCore::ContentFilter::responseReceived): If m_state != Blocked after filtering, call DocumentLoader::responseReceived().
(WebCore::ContentFilter::dataReceived): If m_state == Allowed after filtering, deliver buffered data to DocumentLoader.
If no filtering was necessary, call DocumentLoader::dataReceived() directly.
(WebCore::ContentFilter::redirectReceived): If m_state != Blocked after filtering, call DocumentLoader::redirectReceived().
(WebCore::ContentFilter::notifyFinished): If an error occured, call DocumentLoader::notifyFinished() immediately and return.
If m_state != Blocked after filtering, deliver buffered data to DocumentLoader and call DocumentLoader::notifyFinished().
If no filtering was necessary and m_state != Blocked, call DocumentLoader::notifyFinished() directly.
(WebCore::ContentFilter::didDecide): Called DocumentLoader::contentFilterDidDecide() instead of m_decisionFunction().
(WebCore::ContentFilter::deliverResourceData): Added a helper function to deliver buffered data to DocumentLoader.
(WebCore::ContentFilter::createIfNeeded): Renamed to createIfEnabled().
* loader/ContentFilter.h:
* loader/DocumentLoader.cpp:
(WebCore::DocumentLoader::DocumentLoader):
(WebCore::DocumentLoader::willSendRequest): Stopped asserting that redirectResponse is null and made it part of
the if condition instead, since willSendRequest() will now be called on redirects when there is an active ContentFilter.
(WebCore::DocumentLoader::startLoadingMainResource): Called becomeMainResourceClient() instead of becomeMainResourceClientIfFilterAllows().
(WebCore::DocumentLoader::becomeMainResourceClient): Renamed from becomeMainResourceClientIfFilterAllows().
Only called ContentFilter::startFilteringMainResource() if the filter state is Initialized, since ContentFilter
might have already made a decision in willSendRequest().
(WebCore::DocumentLoader::contentFilterDidDecide): Stopped deleting m_contentFilter, since it will continue to deliver callbacks
even after making a decision. Fixed a bug where we were creating two copies of ContentFilter's replacement data.
(WebCore::DocumentLoader::syntheticRedirectReceived): Deleted.
(WebCore::DocumentLoader::becomeMainResourceClientIfFilterAllows): Renamed to becomeMainResourceClient().
* loader/DocumentLoader.h:
* loader/EmptyClients.h:
* loader/FrameLoaderClient.h:
* loader/ResourceLoader.cpp:
(WebCore::ResourceLoader::willSendRequestInternal): Removed part of r188851.
* loader/SubresourceLoader.cpp:
(WebCore::SubresourceLoader::didReceiveResponse): Removed part of r188486.
* loader/SubresourceLoader.h:
* loader/cache/CachedRawResource.cpp:
(WebCore::CachedRawResource::didAddClient): Removed part of r188150.
* loader/cache/CachedRawResourceClient.h:
(WebCore::CachedRawResourceClient::syntheticRedirectReceived): Removed part of r188150.
* testing/MockContentFilterSettings.h: Defined DecisionPoint::Never.
* testing/MockContentFilterSettings.idl: Defined DECISION_POINT_NEVER.

Source/WebKit/mac:

Deleted parts of r188486 and r188851.

* WebCoreSupport/WebFrameLoaderClient.h:
* WebCoreSupport/WebFrameLoaderClient.mm:
(WebFrameLoaderClient::convertMainResourceLoadToDownload):
(WebFrameLoaderClient::dispatchDidReceiveServerRedirectForProvisionalLoad):

Source/WebKit/win:

Deleted part of r188851.

* WebCoreSupport/WebFrameLoaderClient.cpp:
(WebFrameLoaderClient::dispatchDidReceiveServerRedirectForProvisionalLoad):
* WebCoreSupport/WebFrameLoaderClient.h:

Source/WebKit2:

Deleted parts of r188486 and r188851.

* WebProcess/WebCoreSupport/WebFrameLoaderClient.cpp:
(WebKit::WebFrameLoaderClient::dispatchDidReceiveServerRedirectForProvisionalLoad):
* WebProcess/WebCoreSupport/WebFrameLoaderClient.h:
* WebProcess/WebPage/WebFrame.cpp:
(WebKit::WebFrame::convertMainResourceLoadToDownload):

Tools:

Added download API tests covering every decision and decision point.
Removed _WKDownload.AsynchronousDownloadPolicy in favor of these new tests.

* TestWebKitAPI/Configurations/Base.xcconfig: Added $(BUILT_PRODUCTS_DIR)/WebCoreTestSupport to the header search path.
* TestWebKitAPI/Configurations/WebProcessPlugIn.xcconfig: Removed it from here.
* TestWebKitAPI/Tests/WebKit2Cocoa/BundleParametersPlugIn.mm:
(-[BundleParametersPlugIn observeValueForKeyPath:ofObject:change:context:]): Called -valueForKeyPath:, which
returns an id, instead of -valueForKey:, which always returns an NSString even if the object is of another type.
* TestWebKitAPI/Tests/WebKit2Cocoa/ContentFiltering.mm: Added a class that can send a MockContentFilter
decision and decision point as a bundle parameter.
(+[MockContentFilterEnabler supportsSecureCoding]):
(-[MockContentFilterEnabler copyWithZone:]):
(-[MockContentFilterEnabler initWithCoder:]):
(-[MockContentFilterEnabler initWithDecision:decisionPoint:]):
(-[MockContentFilterEnabler encodeWithCoder:]):
(configurationWithContentFilterSettings): Added a helper function to create a WKWebViewConfiguration with MockConentFilter settings.
(TEST): Renamed ContentFiltering.ServerRedirect to ContentFiltering.URLAfterServerRedirect.
(-[BecomeDownloadDelegate webView:decidePolicyForNavigationResponse:decisionHandler:]): Decided _WKNavigationResponsePolicyBecomeDownload.
(-[BecomeDownloadDelegate webView:didFailProvisionalNavigation:withError:]): Set isDone to true.
(-[BecomeDownloadDelegate webView:didFinishNavigation:]): Ditto.
(-[ContentFilteringDownloadDelegate _downloadDidStart:]): Set downloadDidStart to true.
(downloadTest): Added a helper function to test downloads with a given decision and decision point.
* TestWebKitAPI/Tests/WebKit2Cocoa/ContentFilteringPlugIn.mm: Added a corresponding bundle class that decodes
the MockContentFilter decision and decision point, and sets these values in the MockContentFilterSettings
singleton. This class is duplicated in the bundle to prevent TestWebKitAPI from having to link to WebCoreTestSupport.
(+[MockContentFilterEnabler supportsSecureCoding]):
(-[MockContentFilterEnabler copyWithZone:]):
(-[MockContentFilterEnabler initWithCoder:]):
(-[MockContentFilterEnabler dealloc]):
(-[MockContentFilterEnabler encodeWithCoder:]):
(-[ContentFilteringPlugIn webProcessPlugIn:initializeWithObject:]): Start observing changes to the MockContentFilterEnabler key path.
(-[ContentFilteringPlugIn dealloc]): Stop observing.
(-[ContentFilteringPlugIn observeValueForKeyPath:ofObject:change:context:]): Store a MockContentFilterEnabler in _contentFilterEnabler.
(+[ServerRedirectPlugIn initialize]): Deleted.
* TestWebKitAPI/Tests/WebKit2Cocoa/Download.mm:
(-[AsynchronousDownloadNavigationDelegate webView:decidePolicyForNavigationResponse:decisionHandler:]): Deleted.
(-[AsynchronousDownloadDelegate _downloadDidStart:]): Deleted.
(TEST): Deleted.

LayoutTests:

Added tests for what happens if the content filter does not make a decision when the load finishes.

* contentfiltering/allow-never-expected.html: Added.
* contentfiltering/allow-never.html: Added.
* contentfiltering/block-never-expected.html: Added.
* contentfiltering/block-never.html: Added.
* contentfiltering/resources/contentfiltering.js:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@189193 268f45cc-cd09-0410-ab3c-d52691b4dbfc
38 files changed:
LayoutTests/ChangeLog
LayoutTests/contentfiltering/allow-never-expected.html [new file with mode: 0644]
LayoutTests/contentfiltering/allow-never.html [new file with mode: 0644]
LayoutTests/contentfiltering/block-never-expected.html [new file with mode: 0644]
LayoutTests/contentfiltering/block-never.html [new file with mode: 0644]
LayoutTests/contentfiltering/resources/contentfiltering.js
Source/WebCore/ChangeLog
Source/WebCore/bindings/js/JSMockContentFilterSettingsCustom.cpp
Source/WebCore/loader/ContentFilter.cpp
Source/WebCore/loader/ContentFilter.h
Source/WebCore/loader/DocumentLoader.cpp
Source/WebCore/loader/DocumentLoader.h
Source/WebCore/loader/EmptyClients.h
Source/WebCore/loader/FrameLoaderClient.h
Source/WebCore/loader/ResourceLoader.cpp
Source/WebCore/loader/SubresourceLoader.cpp
Source/WebCore/loader/SubresourceLoader.h
Source/WebCore/loader/cache/CachedRawResource.cpp
Source/WebCore/loader/cache/CachedRawResourceClient.h
Source/WebCore/testing/MockContentFilterSettings.h
Source/WebCore/testing/MockContentFilterSettings.idl
Source/WebKit/mac/ChangeLog
Source/WebKit/mac/WebCoreSupport/WebFrameLoaderClient.h
Source/WebKit/mac/WebCoreSupport/WebFrameLoaderClient.mm
Source/WebKit/win/ChangeLog
Source/WebKit/win/WebCoreSupport/WebFrameLoaderClient.cpp
Source/WebKit/win/WebCoreSupport/WebFrameLoaderClient.h
Source/WebKit2/ChangeLog
Source/WebKit2/WebProcess/WebCoreSupport/WebFrameLoaderClient.cpp
Source/WebKit2/WebProcess/WebCoreSupport/WebFrameLoaderClient.h
Source/WebKit2/WebProcess/WebPage/WebFrame.cpp
Tools/ChangeLog
Tools/TestWebKitAPI/Configurations/Base.xcconfig
Tools/TestWebKitAPI/Configurations/WebProcessPlugIn.xcconfig
Tools/TestWebKitAPI/Tests/WebKit2Cocoa/BundleParametersPlugIn.mm
Tools/TestWebKitAPI/Tests/WebKit2Cocoa/ContentFiltering.mm
Tools/TestWebKitAPI/Tests/WebKit2Cocoa/ContentFilteringPlugIn.mm
Tools/TestWebKitAPI/Tests/WebKit2Cocoa/Download.mm