Web Automation: support enter/exit fullscreen and hide/restore window operations
authorbburg@apple.com <bburg@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 27 Mar 2018 16:37:43 +0000 (16:37 +0000)
committerbburg@apple.com <bburg@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 27 Mar 2018 16:37:43 +0000 (16:37 +0000)
https://bugs.webkit.org/show_bug.cgi?id=182837
<rdar://problem/37580732>

Reviewed by Tim Horton.

The W3C specification is more explicit about when to exit fullscreen and
restore the window for a browsing context. So, WebKit needs to have support
for performing these operations on behalf of a driver.

Based on prototyping, it is sufficient to use a JavaScript atom to enter
fullscreen mode. This is included in the patch as EnterFullscreen.js and
can be used to implement the §10.7.5 Fullscreen Window command.

Other window operations cannot be peformed from JavaScript, so we need to
delegate these operations to the session client (i.e., Safari).
This patch adds session client callouts for restoring, minimizing, and
switching to a browsing context.

Exiting fullscreen happens implicitly (per specification) when setting a
window frame without an actual frame, or when switching/restoring/minimizing a window.
If needed, a driver can call Set Window Rect in this way to unfullscreen a context.
Similarly, a driver can restore a minimized window using Set Window Rect.

* UIProcess/API/APIAutomationSessionClient.h:
(API::AutomationSessionClient::requestHideWindowOfPage):
(API::AutomationSessionClient::requestRestoreWindowOfPage):
(API::AutomationSessionClient::requestSwitchToPage):
Add new API client methods.

* UIProcess/API/Cocoa/_WKAutomationSessionDelegate.h:
Add new Cocoa API delegate methods.

* UIProcess/Automation/Automation.json:
Make the switch to browsing context command asynchronous, since this functionality
is not always synchronous, and we prefer to use completion handlers in the delegates.

Add new protocol method for hiding the window of a browsing context.
This is expected to minimize/miniaturize a window for desktop window managers.

* UIProcess/Automation/WebAutomationSession.h:
* UIProcess/Automation/WebAutomationSession.cpp:
(WebKit::WebAutomationSession::switchToBrowsingContext):
Make this function asynchronous. Call out to the session client.

(WebKit::WebAutomationSession::setWindowFrameOfBrowsingContext):
Follow the steps in the specification to restore window and exit fullscreen.

(WebKit::WebAutomationSession::hideWindowOfBrowsingContext):
Exit fullscreen and call out to the session client.

(WebKit::WebAutomationSession::exitFullscreenWindowForPage):
This is a little strange. Because there is no async API for exiting fullscreen
from C++ code, we hook into willEnterFullScreen and didExitFullScreen and send
out the response if the page exited fullscreen after we requested it to do so.
Because the W3C specification mandates that drivers only process one command at
a time, there will only ever be one callback installed by this method at a time.

(WebKit::WebAutomationSession::restoreWindowForPage):
(WebKit::WebAutomationSession::hideWindowForPage):
Call out to the session client.

(WebKit::WebAutomationSession::didEnterFullScreenForPage):
(WebKit::WebAutomationSession::didExitFullScreenForPage):
Add methods to be called by instrumentation hooks in WebFullScreenManagerProxy.

* UIProcess/Automation/atoms/EnterFullscreen.js: Added.
(enterFullscreen):

* UIProcess/Cocoa/AutomationSessionClient.h:
* UIProcess/Cocoa/AutomationSessionClient.mm:
(WebKit::AutomationSessionClient::AutomationSessionClient):
(WebKit::AutomationSessionClient::requestSwitchToPage):
(WebKit::AutomationSessionClient::requestHideWindowOfPage):
(WebKit::AutomationSessionClient::requestRestoreWindowOfPage):
(WebKit::AutomationSessionClient::isShowingJavaScriptDialogOnPage):
Add boilerplate to convert C++ API client to Objective-C delegate methods.

* UIProcess/WebFullScreenManagerProxy.cpp:
(WebKit::WebFullScreenManagerProxy::didEnterFullScreen):
(WebKit::WebFullScreenManagerProxy::didExitFullScreen):
Notify the automation session if the page is under automation and
enters or exits fullscreen.

* WebKit.xcodeproj/project.pbxproj:
Add EnterFullscreen.js to the list of WebDriver atoms. These are copied
as WebKit2 private headers and used by driver implementations.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@229998 268f45cc-cd09-0410-ab3c-d52691b4dbfc

Source/WebKit/ChangeLog
Source/WebKit/UIProcess/API/APIAutomationSessionClient.h
Source/WebKit/UIProcess/API/Cocoa/_WKAutomationSessionDelegate.h
Source/WebKit/UIProcess/Automation/Automation.json
Source/WebKit/UIProcess/Automation/WebAutomationSession.cpp
Source/WebKit/UIProcess/Automation/WebAutomationSession.h
Source/WebKit/UIProcess/Automation/atoms/EnterFullscreen.js [new file with mode: 0644]
Source/WebKit/UIProcess/Cocoa/AutomationSessionClient.h
Source/WebKit/UIProcess/Cocoa/AutomationSessionClient.mm
Source/WebKit/UIProcess/WebFullScreenManagerProxy.cpp
Source/WebKit/WebKit.xcodeproj/project.pbxproj

index 265ab8b..8600c5b 100644 (file)
@@ -1,3 +1,93 @@
+2018-03-27  Brian Burg  <bburg@apple.com>
+
+        Web Automation: support enter/exit fullscreen and hide/restore window operations
+        https://bugs.webkit.org/show_bug.cgi?id=182837
+        <rdar://problem/37580732>
+
+        Reviewed by Tim Horton.
+
+        The W3C specification is more explicit about when to exit fullscreen and
+        restore the window for a browsing context. So, WebKit needs to have support
+        for performing these operations on behalf of a driver.
+
+        Based on prototyping, it is sufficient to use a JavaScript atom to enter
+        fullscreen mode. This is included in the patch as EnterFullscreen.js and
+        can be used to implement the §10.7.5 Fullscreen Window command.
+
+        Other window operations cannot be peformed from JavaScript, so we need to
+        delegate these operations to the session client (i.e., Safari).
+        This patch adds session client callouts for restoring, minimizing, and
+        switching to a browsing context.
+
+        Exiting fullscreen happens implicitly (per specification) when setting a
+        window frame without an actual frame, or when switching/restoring/minimizing a window.
+        If needed, a driver can call Set Window Rect in this way to unfullscreen a context.
+        Similarly, a driver can restore a minimized window using Set Window Rect.
+
+        * UIProcess/API/APIAutomationSessionClient.h:
+        (API::AutomationSessionClient::requestHideWindowOfPage):
+        (API::AutomationSessionClient::requestRestoreWindowOfPage):
+        (API::AutomationSessionClient::requestSwitchToPage):
+        Add new API client methods.
+
+        * UIProcess/API/Cocoa/_WKAutomationSessionDelegate.h:
+        Add new Cocoa API delegate methods.
+
+        * UIProcess/Automation/Automation.json:
+        Make the switch to browsing context command asynchronous, since this functionality
+        is not always synchronous, and we prefer to use completion handlers in the delegates.
+
+        Add new protocol method for hiding the window of a browsing context.
+        This is expected to minimize/miniaturize a window for desktop window managers.
+
+        * UIProcess/Automation/WebAutomationSession.h:
+        * UIProcess/Automation/WebAutomationSession.cpp:
+        (WebKit::WebAutomationSession::switchToBrowsingContext):
+        Make this function asynchronous. Call out to the session client.
+
+        (WebKit::WebAutomationSession::setWindowFrameOfBrowsingContext):
+        Follow the steps in the specification to restore window and exit fullscreen.
+
+        (WebKit::WebAutomationSession::hideWindowOfBrowsingContext):
+        Exit fullscreen and call out to the session client.
+
+        (WebKit::WebAutomationSession::exitFullscreenWindowForPage):
+        This is a little strange. Because there is no async API for exiting fullscreen
+        from C++ code, we hook into willEnterFullScreen and didExitFullScreen and send
+        out the response if the page exited fullscreen after we requested it to do so.
+        Because the W3C specification mandates that drivers only process one command at
+        a time, there will only ever be one callback installed by this method at a time.
+
+        (WebKit::WebAutomationSession::restoreWindowForPage):
+        (WebKit::WebAutomationSession::hideWindowForPage):
+        Call out to the session client.
+
+        (WebKit::WebAutomationSession::didEnterFullScreenForPage):
+        (WebKit::WebAutomationSession::didExitFullScreenForPage):
+        Add methods to be called by instrumentation hooks in WebFullScreenManagerProxy.
+
+        * UIProcess/Automation/atoms/EnterFullscreen.js: Added.
+        (enterFullscreen):
+
+        * UIProcess/Cocoa/AutomationSessionClient.h:
+        * UIProcess/Cocoa/AutomationSessionClient.mm:
+        (WebKit::AutomationSessionClient::AutomationSessionClient):
+        (WebKit::AutomationSessionClient::requestSwitchToPage):
+        (WebKit::AutomationSessionClient::requestHideWindowOfPage):
+        (WebKit::AutomationSessionClient::requestRestoreWindowOfPage):
+        (WebKit::AutomationSessionClient::isShowingJavaScriptDialogOnPage):
+        Add boilerplate to convert C++ API client to Objective-C delegate methods.
+
+        * UIProcess/WebFullScreenManagerProxy.cpp:
+        (WebKit::WebFullScreenManagerProxy::didEnterFullScreen):
+        (WebKit::WebFullScreenManagerProxy::didExitFullScreen):
+        Notify the automation session if the page is under automation and
+        enters or exits fullscreen.
+
+        * WebKit.xcodeproj/project.pbxproj:
+        Add EnterFullscreen.js to the list of WebDriver atoms. These are copied
+        as WebKit2 private headers and used by driver implementations.
+
 2018-03-27  Eric Carlson  <eric.carlson@apple.com>
 
         Make AVFoundationEnabled preference available on iOS
index 68353bb..8076e44 100644 (file)
@@ -54,6 +54,9 @@ public:
     virtual String sessionIdentifier() const { return String(); }
     virtual void didDisconnectFromRemote(WebKit::WebAutomationSession&) { }
     virtual void requestNewPageWithOptions(WebKit::WebAutomationSession&, AutomationSessionBrowsingContextOptions, CompletionHandler<void(WebKit::WebPageProxy*)>&& completionHandler) { completionHandler(nullptr); }
+    virtual void requestHideWindowOfPage(WebKit::WebAutomationSession&, WebKit::WebPageProxy&, CompletionHandler<void()>&& completionHandler) { completionHandler(); }
+    virtual void requestRestoreWindowOfPage(WebKit::WebAutomationSession&, WebKit::WebPageProxy&, CompletionHandler<void()>&& completionHandler) { completionHandler(); }
+    virtual void requestSwitchToPage(WebKit::WebAutomationSession&, WebKit::WebPageProxy&, CompletionHandler<void()>&& completionHandler) { completionHandler(); }
     virtual bool isShowingJavaScriptDialogOnPage(WebKit::WebAutomationSession&, WebKit::WebPageProxy&) { return false; }
     virtual void dismissCurrentJavaScriptDialogOnPage(WebKit::WebAutomationSession&, WebKit::WebPageProxy&) { }
     virtual void acceptCurrentJavaScriptDialogOnPage(WebKit::WebAutomationSession&, WebKit::WebPageProxy&) { }
index 48fa7eb..ec6b336 100644 (file)
@@ -52,6 +52,9 @@ typedef NS_ENUM(NSUInteger, _WKAutomationSessionBrowsingContextOptions) {
 - (void)_automationSessionDidDisconnectFromRemote:(_WKAutomationSession *)automationSession;
 
 - (void)_automationSession:(_WKAutomationSession *)automationSession requestNewWebViewWithOptions:(_WKAutomationSessionBrowsingContextOptions)options completionHandler:(void(^)(WKWebView * _Nullable))completionHandler WK_API_AVAILABLE(macosx(WK_MAC_TBA), ios(WK_IOS_TBA));
+- (void)_automationSession:(_WKAutomationSession *)automationSession requestHideWindowOfWebView:(WKWebView *)webView completionHandler:(void(^)(void))completionHandler WK_API_AVAILABLE(macosx(WK_MAC_TBA), ios(WK_IOS_TBA));
+- (void)_automationSession:(_WKAutomationSession *)automationSession requestRestoreWindowOfWebView:(WKWebView *)webView completionHandler:(void(^)(void))completionHandler WK_API_AVAILABLE(macosx(WK_MAC_TBA), ios(WK_IOS_TBA));
+- (void)_automationSession:(_WKAutomationSession *)automationSession requestSwitchToWebView:(WKWebView *)webView completionHandler:(void(^)(void))completionHandler WK_API_AVAILABLE(macosx(WK_MAC_TBA), ios(WK_IOS_TBA));
 - (BOOL)_automationSession:(_WKAutomationSession *)automationSession isShowingJavaScriptDialogForWebView:(WKWebView *)webView WK_API_AVAILABLE(macosx(10.13), ios(11.0));
 - (void)_automationSession:(_WKAutomationSession *)automationSession dismissCurrentJavaScriptDialogForWebView:(WKWebView *)webView WK_API_AVAILABLE(macosx(10.13), ios(11.0));
 - (void)_automationSession:(_WKAutomationSession *)automationSession acceptCurrentJavaScriptDialogForWebView:(WKWebView *)webView WK_API_AVAILABLE(macosx(10.13), ios(11.0));
@@ -62,6 +65,9 @@ typedef NS_ENUM(NSUInteger, _WKAutomationSessionBrowsingContextOptions) {
 // FIXME 37408718: Objective-C delegate methods shouldn't use C API types like WKPageRef. We need to
 // migrate clients to use WKWebView, or expose the same behavior via a C SPI for those clients.
 - (void)_automationSession:(_WKAutomationSession *)automationSession requestNewPageWithOptions:(_WKAutomationSessionBrowsingContextOptions)options completionHandler:(void(^)(WKPageRef))completionHandler WK_API_AVAILABLE(macosx(WK_MAC_TBA), ios(WK_IOS_TBA));
+- (void)_automationSession:(_WKAutomationSession *)automationSession requestHideWindowOfPage:(WKPageRef)page completionHandler:(void(^)(void))completionHandler WK_API_AVAILABLE(macosx(WK_MAC_TBA), ios(WK_IOS_TBA));
+- (void)_automationSession:(_WKAutomationSession *)automationSession requestRestoreWindowOfPage:(WKPageRef)page completionHandler:(void(^)(void))completionHandler WK_API_AVAILABLE(macosx(WK_MAC_TBA), ios(WK_IOS_TBA));
+- (void)_automationSession:(_WKAutomationSession *)automationSession requestSwitchToPage:(WKPageRef)page completionHandler:(void(^)(void))completionHandler WK_API_AVAILABLE(macosx(WK_MAC_TBA), ios(WK_IOS_TBA));
 - (BOOL)_automationSession:(_WKAutomationSession *)automationSession isShowingJavaScriptDialogOnPage:(WKPageRef)page WK_API_AVAILABLE(macosx(10.13), ios(11.0));
 - (void)_automationSession:(_WKAutomationSession *)automationSession dismissCurrentJavaScriptDialogOnPage:(WKPageRef)page WK_API_AVAILABLE(macosx(10.13), ios(11.0));
 - (void)_automationSession:(_WKAutomationSession *)automationSession acceptCurrentJavaScriptDialogOnPage:(WKPageRef)page WK_API_AVAILABLE(macosx(10.13), ios(11.0));
index 7ae68cb..7e86d14 100644 (file)
             "parameters": [
                 { "name": "browsingContextHandle", "$ref": "BrowsingContextHandle", "description": "The handle for the browsing context that should be made focused." },
                 { "name": "frameHandle", "$ref": "FrameHandle", "optional": true, "description": "The handle for the frame that should be focused. Defaults to the main frame if omitted." }
-            ]
+            ],
+            "async": true
         },
         {
             "name": "setWindowFrameOfBrowsingContext",
-            "description": "Moves and/or resizes the window of the specified top-level browsing context to the specified size and origin. Both the size and origin may be omitted. Regardless of the arguments, this command will exit fullscreen and deminiaturize the window if applicable.",
+            "description": "Moves and/or resizes the window of the specified top-level browsing context to the specified position and size. Both position and size may be omitted. This command implicitly exits fullscreen and restores the window (if previously hidden), then optionally sets the window frame.",
             "parameters": [
                 { "name": "handle", "$ref": "BrowsingContextHandle", "description": "The handle for the browsing context to be resized." },
                 { "name": "origin", "$ref": "Point", "optional": true, "description": "The new origin for the browsing context's window. The position is interpreted in screen coordinate space, relative to the upper left corner of the screen." },
             "async": true
         },
         {
+            "name": "hideWindowOfBrowsingContext",
+            "description": "Causes the window of the specified browsing context to be hidden/minimized/iconified. This command implicitly exits fullscreen mode.",
+            "parameters": [
+                { "name": "handle", "$ref": "BrowsingContextHandle", "description": "The handle for the browsing context whose window should be hidden." }
+            ],
+            "async": true
+        },
+        {
             "name": "navigateBrowsingContext",
             "description": "Navigates a browsing context to a specified URL.",
             "parameters": [
index e6d8a97..0970ac7 100644 (file)
@@ -35,6 +35,7 @@
 #include "WebAutomationSessionMessages.h"
 #include "WebAutomationSessionProxyMessages.h"
 #include "WebCookieManagerProxy.h"
+#include "WebFullScreenManagerProxy.h"
 #include "WebInspectorProxy.h"
 #include "WebOpenPanelResultListenerProxy.h"
 #include "WebProcessPool.h"
@@ -299,25 +300,27 @@ void WebAutomationSession::closeBrowsingContext(Inspector::ErrorString& errorStr
     page->closePage(false);
 }
 
-void WebAutomationSession::switchToBrowsingContext(Inspector::ErrorString& errorString, const String& browsingContextHandle, const String* optionalFrameHandle)
+void WebAutomationSession::switchToBrowsingContext(const String& browsingContextHandle, const String* optionalFrameHandle, Ref<SwitchToBrowsingContextCallback>&& callback)
 {
     WebPageProxy* page = webPageProxyForHandle(browsingContextHandle);
     if (!page)
-        SYNC_FAIL_WITH_PREDEFINED_ERROR(WindowNotFound);
+        ASYNC_FAIL_WITH_PREDEFINED_ERROR(WindowNotFound);
 
     std::optional<uint64_t> frameID = webFrameIDForHandle(optionalFrameHandle ? *optionalFrameHandle : emptyString());
     if (!frameID)
-        SYNC_FAIL_WITH_PREDEFINED_ERROR(FrameNotFound);
+        ASYNC_FAIL_WITH_PREDEFINED_ERROR(FrameNotFound);
 
-    page->setFocus(true);
-    page->process().send(Messages::WebAutomationSessionProxy::FocusFrame(page->pageID(), frameID.value()), 0);
+
+    m_client->requestSwitchToPage(*this, *page, [frameID, page = makeRef(*page), callback = WTFMove(callback)]() {
+        page->setFocus(true);
+        page->process().send(Messages::WebAutomationSessionProxy::FocusFrame(page->pageID(), frameID.value()), 0);
+
+        callback->sendSuccess();
+    });
 }
 
 void WebAutomationSession::setWindowFrameOfBrowsingContext(const String& handle, const JSON::Object* optionalOriginObject, const JSON::Object* optionalSizeObject, Ref<SetWindowFrameOfBrowsingContextCallback>&& callback)
 {
-#if PLATFORM(IOS)
-    ASYNC_FAIL_WITH_PREDEFINED_ERROR(NotImplemented);
-#else
     std::optional<float> x;
     std::optional<float> y;
     if (optionalOriginObject) {
@@ -354,18 +357,17 @@ void WebAutomationSession::setWindowFrameOfBrowsingContext(const String& handle,
     if (!page)
         ASYNC_FAIL_WITH_PREDEFINED_ERROR(WindowNotFound);
 
-    // FIXME (§10.7.2 Step 10): We need to exit fullscreen before setting the frame.
-    // FIXME (§10.7.2 Step 11): We need to de-miniaturize the window before setting the frame.
-
-    page->getWindowFrameWithCallback([callback = WTFMove(callback), page = makeRef(*page), width, height, x, y](WebCore::FloatRect originalFrame) mutable {
-        WebCore::FloatRect newFrame = WebCore::FloatRect(WebCore::FloatPoint(x.value_or(originalFrame.location().x()), y.value_or(originalFrame.location().y())), WebCore::FloatSize(width.value_or(originalFrame.size().width()), height.value_or(originalFrame.size().height())));
-        if (newFrame == originalFrame)
-            return callback->sendSuccess();
-
-        page->setWindowFrame(newFrame);
-        callback->sendSuccess();
+    exitFullscreenWindowForPage(*page, [this, protectedThis = makeRef(*this), callback = WTFMove(callback), page = makeRefPtr(page), width, height, x, y]() mutable {
+        this->restoreWindowForPage(*page, [callback = WTFMove(callback), page = WTFMove(page), width, height, x, y]() mutable {
+            page->getWindowFrameWithCallback([callback = WTFMove(callback), page = WTFMove(page), width, height, x, y](WebCore::FloatRect originalFrame) mutable {
+                WebCore::FloatRect newFrame = WebCore::FloatRect(WebCore::FloatPoint(x.value_or(originalFrame.location().x()), y.value_or(originalFrame.location().y())), WebCore::FloatSize(width.value_or(originalFrame.size().width()), height.value_or(originalFrame.size().height())));
+                if (newFrame != originalFrame)
+                    page->setWindowFrame(newFrame);
+                
+                callback->sendSuccess();
+            });
+        });
     });
-#endif
 }
 
 static std::optional<Inspector::Protocol::Automation::PageLoadStrategy> pageLoadStrategyFromStringParameter(const String* optionalPageLoadStrategyString)
@@ -507,6 +509,54 @@ void WebAutomationSession::loadTimerFired()
     respondToPendingPageNavigationCallbacksWithTimeout(m_pendingEagerNavigationInBrowsingContextCallbacksPerPage);
 }
 
+void WebAutomationSession::hideWindowOfBrowsingContext(const String& browsingContextHandle, Ref<HideWindowOfBrowsingContextCallback>&& callback)
+{
+    WebPageProxy* page = webPageProxyForHandle(browsingContextHandle);
+    if (!page)
+        ASYNC_FAIL_WITH_PREDEFINED_ERROR(WindowNotFound);
+    
+    exitFullscreenWindowForPage(*page, [protectedThis = makeRef(*this), callback = WTFMove(callback), page = makeRefPtr(page)]() mutable {
+        protectedThis->hideWindowForPage(*page, [callback = WTFMove(callback)]() mutable {
+            callback->sendSuccess();
+        });
+    });
+}
+
+void WebAutomationSession::exitFullscreenWindowForPage(WebPageProxy& page, WTF::CompletionHandler<void()>&& completionHandler)
+{
+#if ENABLE(FULLSCREEN_API)
+    ASSERT(!m_windowStateTransitionCallback);
+    if (!page.fullScreenManager()->isFullScreen()) {
+        completionHandler();
+        return;
+    }
+    
+    m_windowStateTransitionCallback = WTF::Function<void(WindowTransitionedToState)> { [this, protectedThis = makeRef(*this), completionHandler = WTFMove(completionHandler)](WindowTransitionedToState state) mutable {
+        // If fullscreen exited and we didn't request that, just ignore it.
+        if (state != WindowTransitionedToState::Unfullscreen)
+            return;
+
+        // Keep this callback in scope so completionHandler does not get destroyed before we call it.
+        auto protectedCallback = WTFMove(m_windowStateTransitionCallback);
+        completionHandler();
+    } };
+    
+    page.fullScreenManager()->requestExitFullScreen();
+#else
+    completionHandler();
+#endif
+}
+
+void WebAutomationSession::restoreWindowForPage(WebPageProxy& page, WTF::CompletionHandler<void()>&& completionHandler)
+{
+    m_client->requestRestoreWindowOfPage(*this, page, WTFMove(completionHandler));
+}
+
+void WebAutomationSession::hideWindowForPage(WebPageProxy& page, WTF::CompletionHandler<void()>&& completionHandler)
+{
+    m_client->requestHideWindowOfPage(*this, page, WTFMove(completionHandler));
+}
+
 void WebAutomationSession::willShowJavaScriptDialog(WebPageProxy& page)
 {
     // Wait until the next run loop iteration to give time for the client to show the dialog,
@@ -533,6 +583,18 @@ void WebAutomationSession::willShowJavaScriptDialog(WebPageProxy& page)
         }
     });
 }
+    
+void WebAutomationSession::didEnterFullScreenForPage(const WebPageProxy&)
+{
+    if (m_windowStateTransitionCallback)
+        m_windowStateTransitionCallback(WindowTransitionedToState::Fullscreen);
+}
+
+void WebAutomationSession::didExitFullScreenForPage(const WebPageProxy&)
+{
+    if (m_windowStateTransitionCallback)
+        m_windowStateTransitionCallback(WindowTransitionedToState::Unfullscreen);
+}
 
 void WebAutomationSession::navigateBrowsingContext(const String& handle, const String& url, const String* optionalPageLoadStrategyString, const int* optionalPageLoadTimeout, Ref<NavigateBrowsingContextCallback>&& callback)
 {
index 8846241..5706782 100644 (file)
@@ -31,6 +31,7 @@
 #include "Connection.h"
 #include "ShareableBitmap.h"
 #include "WebEvent.h"
+#include <wtf/CompletionHandler.h>
 #include <wtf/Forward.h>
 #include <wtf/RunLoop.h>
 
@@ -100,6 +101,8 @@ public:
     void willClosePage(const WebPageProxy&);
     void handleRunOpenPanel(const WebPageProxy&, const WebFrameProxy&, const API::OpenPanelParameters&, WebOpenPanelResultListenerProxy&);
     void willShowJavaScriptDialog(WebPageProxy&);
+    void didEnterFullScreenForPage(const WebPageProxy&);
+    void didExitFullScreenForPage(const WebPageProxy&);
 
     bool shouldAllowGetUserMediaForPage(const WebPageProxy&) const;
 
@@ -119,10 +122,11 @@ public:
     // Platform: Generic
     void getBrowsingContexts(Ref<GetBrowsingContextsCallback>&&) final;
     void getBrowsingContext(const String&, Ref<GetBrowsingContextCallback>&&) final;
-    void closeBrowsingContext(Inspector::ErrorString&, const String&) override;
-    void switchToBrowsingContext(Inspector::ErrorString&, const String& browsingContextHandle, const String* optionalFrameHandle) override;
     void createBrowsingContext(const bool* preferNewTab, Ref<CreateBrowsingContextCallback>&&) final;
+    void closeBrowsingContext(Inspector::ErrorString&, const String&) final;
+    void switchToBrowsingContext(const String& browsingContextHandle, const String* optionalFrameHandle, Ref<SwitchToBrowsingContextCallback>&&) final;
     void setWindowFrameOfBrowsingContext(const String& handle, const JSON::Object* origin, const JSON::Object* size, Ref<SetWindowFrameOfBrowsingContextCallback>&&) final;
+    void hideWindowOfBrowsingContext(const String& handle, Ref<HideWindowOfBrowsingContextCallback>&&) final;
     void navigateBrowsingContext(const String& handle, const String& url, const String* optionalPageLoadStrategyString, const int* optionalPageLoadTimeout, Ref<NavigateBrowsingContextCallback>&&) override;
     void goBackInBrowsingContext(const String&, const String* optionalPageLoadStrategyString, const int* optionalPageLoadTimeout, Ref<GoBackInBrowsingContextCallback>&&) override;
     void goForwardInBrowsingContext(const String&, const String* optionalPageLoadStrategyString, const int* optionalPageLoadTimeout, Ref<GoForwardInBrowsingContextCallback>&&) override;
@@ -177,6 +181,10 @@ private:
     void respondToPendingFrameNavigationCallbacksWithTimeout(HashMap<uint64_t, RefPtr<Inspector::BackendDispatcher::CallbackBase>>&);
     void loadTimerFired();
 
+    void exitFullscreenWindowForPage(WebPageProxy&, WTF::CompletionHandler<void()>&&);
+    void restoreWindowForPage(WebPageProxy&, WTF::CompletionHandler<void()>&&);
+    void hideWindowForPage(WebPageProxy&, WTF::CompletionHandler<void()>&&);
+
     // Implemented in generated WebAutomationSessionMessageReceiver.cpp.
     void didReceiveMessage(IPC::Connection&, IPC::Decoder&) override;
 
@@ -253,6 +261,12 @@ private:
     uint64_t m_nextSelectOptionElementCallbackID { 1 };
     HashMap<uint64_t, RefPtr<Inspector::AutomationBackendDispatcherHandler::SelectOptionElementCallback>> m_selectOptionElementCallbacks;
 
+    enum class WindowTransitionedToState {
+        Fullscreen,
+        Unfullscreen,
+    };
+    Function<void(WindowTransitionedToState)> m_windowStateTransitionCallback { };
+
     RunLoop::Timer<WebAutomationSession> m_loadTimer;
     Vector<String> m_filesToSelectForFileUpload;
 
diff --git a/Source/WebKit/UIProcess/Automation/atoms/EnterFullscreen.js b/Source/WebKit/UIProcess/Automation/atoms/EnterFullscreen.js
new file mode 100644 (file)
index 0000000..0e946e6
--- /dev/null
@@ -0,0 +1,59 @@
+/*
+ * Copyright (C) 2018 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+function enterFullscreen() {
+
+    let callback = arguments[0];
+    if (!document.webkitFullscreenEnabled)
+        callback(false);
+
+    let fullscreenChangeListener, fullscreenErrorListener;
+    fullscreenChangeListener = (e) => {
+        // Some other element went fullscreen, we didn't request it.
+        if (e.target !== document.documentElement)
+            return;
+
+        if (!document.webkitIsFullScreen)
+            return;
+
+        document.removeEventListener("webkitfullscreenerror", fullscreenChangeListener);
+        document.documentElement.removeEventListener("webkitfullscreenchange", fullscreenErrorListener);
+        callback(true);
+    };
+    fullscreenErrorListener = (e) => {
+        // Some other element caused an error, we didn't request it.
+        if (e.target !== document.documentElement)
+            return;
+
+        document.removeEventListener("webkitfullscreenchange", fullscreenChangeListener);
+        document.documentElement.removeEventListener("webkitfullscreenerror", fullscreenErrorListener);
+        callback(document.webkitIsFullscreen);
+    };
+
+    // The document fires change events, but the fullscreen element fires error events.
+    document.addEventListener("webkitfullscreenchange", fullscreenChangeListener);
+    document.documentElement.addEventListener("webkitfullscreenerror", fullscreenErrorListener);
+    document.documentElement.webkitRequestFullscreen();
+}
index 7f6144a..b2b9d44 100644 (file)
@@ -46,6 +46,10 @@ private:
     void didDisconnectFromRemote(WebAutomationSession&) override;
 
     void requestNewPageWithOptions(WebAutomationSession&, API::AutomationSessionBrowsingContextOptions, CompletionHandler<void(WebPageProxy*)>&&) override;
+    void requestSwitchToPage(WebKit::WebAutomationSession&, WebKit::WebPageProxy&, CompletionHandler<void()>&&) override;
+    void requestHideWindowOfPage(WebKit::WebAutomationSession&, WebKit::WebPageProxy&, CompletionHandler<void()>&&) override;
+    void requestRestoreWindowOfPage(WebKit::WebAutomationSession&, WebKit::WebPageProxy&, CompletionHandler<void()>&&) override;
+
     bool isShowingJavaScriptDialogOnPage(WebAutomationSession&, WebPageProxy&) override;
     void dismissCurrentJavaScriptDialogOnPage(WebAutomationSession&, WebPageProxy&) override;
     void acceptCurrentJavaScriptDialogOnPage(WebAutomationSession&, WebPageProxy&) override;
@@ -59,6 +63,9 @@ private:
         bool didDisconnectFromRemote : 1;
 
         bool requestNewWebViewWithOptions : 1;
+        bool requestSwitchToWebView : 1;
+        bool requestHideWindowOfWebView : 1;
+        bool requestRestoreWindowOfWebView : 1;
         bool isShowingJavaScriptDialogForWebView : 1;
         bool dismissCurrentJavaScriptDialogForWebView : 1;
         bool acceptCurrentJavaScriptDialogForWebView : 1;
@@ -68,6 +75,9 @@ private:
 
         // FIXME 37408718: these delegate methods should be removed.
         bool requestNewPageWithOptions : 1;
+        bool requestSwitchToPage : 1;
+        bool requestHideWindowOfPage : 1;
+        bool requestRestoreWindowOfPage : 1;
         bool isShowingJavaScriptDialogOnPage : 1;
         bool dismissCurrentJavaScriptDialogOnPage : 1;
         bool acceptCurrentJavaScriptDialogOnPage : 1;
index ccd890c..7970ba3 100644 (file)
@@ -44,6 +44,9 @@ AutomationSessionClient::AutomationSessionClient(id <_WKAutomationSessionDelegat
     m_delegateMethods.didDisconnectFromRemote = [delegate respondsToSelector:@selector(_automationSessionDidDisconnectFromRemote:)];
 
     m_delegateMethods.requestNewWebViewWithOptions = [delegate respondsToSelector:@selector(_automationSession:requestNewWebViewWithOptions:completionHandler:)];
+    m_delegateMethods.requestSwitchToWebView = [delegate respondsToSelector:@selector(_automationSession:requestSwitchToWebView:completionHandler:)];
+    m_delegateMethods.requestHideWindowOfWebView = [delegate respondsToSelector:@selector(_automationSession:requestHideWindowOfWebView:completionHandler:)];
+    m_delegateMethods.requestRestoreWindowOfWebView = [delegate respondsToSelector:@selector(_automationSession:requestRestoreWindowOfWebView:completionHandler:)];
     m_delegateMethods.isShowingJavaScriptDialogForWebView = [delegate respondsToSelector:@selector(_automationSession:isShowingJavaScriptDialogForWebView:)];
     m_delegateMethods.dismissCurrentJavaScriptDialogForWebView = [delegate respondsToSelector:@selector(_automationSession:dismissCurrentJavaScriptDialogForWebView:)];
     m_delegateMethods.acceptCurrentJavaScriptDialogForWebView = [delegate respondsToSelector:@selector(_automationSession:acceptCurrentJavaScriptDialogForWebView:)];
@@ -53,6 +56,9 @@ AutomationSessionClient::AutomationSessionClient(id <_WKAutomationSessionDelegat
 
     // FIXME 37408718: these delegate methods should be removed.
     m_delegateMethods.requestNewPageWithOptions = [delegate respondsToSelector:@selector(_automationSession:requestNewPageWithOptions:completionHandler:)];
+    m_delegateMethods.requestSwitchToPage = [delegate respondsToSelector:@selector(_automationSession:requestSwitchToPage:completionHandler:)];
+    m_delegateMethods.requestHideWindowOfPage = [delegate respondsToSelector:@selector(_automationSession:requestHideWindowOfPage:completionHandler:)];
+    m_delegateMethods.requestRestoreWindowOfPage = [delegate respondsToSelector:@selector(_automationSession:requestRestoreWindowOfPage:completionHandler:)];
     m_delegateMethods.isShowingJavaScriptDialogOnPage = [delegate respondsToSelector:@selector(_automationSession:isShowingJavaScriptDialogOnPage:)];
     m_delegateMethods.dismissCurrentJavaScriptDialogOnPage = [delegate respondsToSelector:@selector(_automationSession:dismissCurrentJavaScriptDialogOnPage:)];
     m_delegateMethods.acceptCurrentJavaScriptDialogOnPage = [delegate respondsToSelector:@selector(_automationSession:acceptCurrentJavaScriptDialogOnPage:)];
@@ -92,14 +98,55 @@ void AutomationSessionClient::requestNewPageWithOptions(WebAutomationSession& se
     }
 }
 
+void AutomationSessionClient::requestSwitchToPage(WebAutomationSession& session, WebPageProxy& page, CompletionHandler<void()>&& completionHandler)
+{
+    if (!m_delegateMethods.requestSwitchToWebView && !m_delegateMethods.requestSwitchToPage) {
+        completionHandler();
+        return;
+    }
+
+    auto completionBlock = BlockPtr<void()>::fromCallable([completionHandler = WTFMove(completionHandler)]() { completionHandler(); });
+    if (m_delegateMethods.requestSwitchToWebView)
+        [m_delegate.get() _automationSession:wrapper(session) requestSwitchToWebView:fromWebPageProxy(page) completionHandler:completionBlock.get()];
+    else if (m_delegateMethods.requestSwitchToPage)
+        [m_delegate.get() _automationSession:wrapper(session) requestSwitchToPage:toAPI(&page) completionHandler:completionBlock.get()];
+}
+
+void AutomationSessionClient::requestHideWindowOfPage(WebAutomationSession& session, WebPageProxy& page, CompletionHandler<void()>&& completionHandler)
+{
+    if (!m_delegateMethods.requestHideWindowOfWebView && !m_delegateMethods.requestHideWindowOfPage) {
+        completionHandler();
+        return;
+    }
+
+    auto completionBlock = BlockPtr<void()>::fromCallable([completionHandler = WTFMove(completionHandler)]() { completionHandler(); });
+    if (m_delegateMethods.requestHideWindowOfWebView)
+        [m_delegate.get() _automationSession:wrapper(session) requestHideWindowOfWebView:fromWebPageProxy(page) completionHandler:completionBlock.get()];
+    else if (m_delegateMethods.requestHideWindowOfPage)
+        [m_delegate.get() _automationSession:wrapper(session) requestHideWindowOfPage:toAPI(&page) completionHandler:completionBlock.get()];
+}
+
+void AutomationSessionClient::requestRestoreWindowOfPage(WebAutomationSession& session, WebPageProxy& page, CompletionHandler<void()>&& completionHandler)
+{
+    if (!m_delegateMethods.requestRestoreWindowOfWebView && !m_delegateMethods.requestRestoreWindowOfPage) {
+        completionHandler();
+        return;
+    }
+
+    auto completionBlock = BlockPtr<void()>::fromCallable([completionHandler = WTFMove(completionHandler)]() { completionHandler(); });
+    if (m_delegateMethods.requestRestoreWindowOfWebView)
+        [m_delegate.get() _automationSession:wrapper(session) requestRestoreWindowOfWebView:fromWebPageProxy(page) completionHandler:completionBlock.get()];
+    else if (m_delegateMethods.requestRestoreWindowOfPage)
+        [m_delegate.get() _automationSession:wrapper(session) requestRestoreWindowOfPage:toAPI(&page) completionHandler:completionBlock.get()];
+}
+
 bool AutomationSessionClient::isShowingJavaScriptDialogOnPage(WebAutomationSession& session, WebPageProxy& page)
 {
     if (m_delegateMethods.isShowingJavaScriptDialogForWebView)
         return [m_delegate.get() _automationSession:wrapper(session) isShowingJavaScriptDialogForWebView:fromWebPageProxy(page)];
-
-    if (m_delegateMethods.isShowingJavaScriptDialogOnPage)
+    else if (m_delegateMethods.isShowingJavaScriptDialogOnPage)
         return [m_delegate.get() _automationSession:wrapper(session) isShowingJavaScriptDialogOnPage:toAPI(&page)];
-
+    
     return false;
 }
 
index 0a2fb9b..f5818c6 100644 (file)
 #if ENABLE(FULLSCREEN_API)
 
 #include "APIFullscreenClient.h"
+#include "WebAutomationSession.h"
 #include "WebFullScreenManagerMessages.h"
 #include "WebFullScreenManagerProxyMessages.h"
 #include "WebPageProxy.h"
+#include "WebProcessPool.h"
 #include "WebProcessProxy.h"
 #include <WebCore/IntRect.h>
 
@@ -65,6 +67,11 @@ void WebFullScreenManagerProxy::didEnterFullScreen()
 {
     m_page->fullscreenClient().didEnterFullscreen(m_page);
     m_page->process().send(Messages::WebFullScreenManager::DidEnterFullScreen(), m_page->pageID());
+
+    if (m_page->isControlledByAutomation()) {
+        if (WebAutomationSession* automationSession = m_page->process().processPool().automationSession())
+            automationSession->didEnterFullScreenForPage(*m_page);
+    }
 }
 
 void WebFullScreenManagerProxy::willExitFullScreen()
@@ -77,6 +84,11 @@ void WebFullScreenManagerProxy::didExitFullScreen()
 {
     m_page->fullscreenClient().didExitFullscreen(m_page);
     m_page->process().send(Messages::WebFullScreenManager::DidExitFullScreen(), m_page->pageID());
+    
+    if (m_page->isControlledByAutomation()) {
+        if (WebAutomationSession* automationSession = m_page->process().processPool().automationSession())
+            automationSession->didExitFullScreenForPage(*m_page);
+    }
 }
 
 void WebFullScreenManagerProxy::setAnimatingFullScreen(bool animating)
index 292f200..0c74c5e 100644 (file)
                99C81D591C20E1E5005C4C82 /* AutomationClient.mm in Sources */ = {isa = PBXBuildFile; fileRef = 99C81D561C20DFBE005C4C82 /* AutomationClient.mm */; };
                99C81D5A1C20E7E2005C4C82 /* AutomationClient.h in Headers */ = {isa = PBXBuildFile; fileRef = 99C81D551C20DFBE005C4C82 /* AutomationClient.h */; };
                99C81D5D1C21F38B005C4C82 /* APIAutomationClient.h in Headers */ = {isa = PBXBuildFile; fileRef = 99C81D5B1C20E817005C4C82 /* APIAutomationClient.h */; };
+               99CA66CA2036685D0074F35E /* EnterFullscreen.js in Copy WebDriver Atoms */ = {isa = PBXBuildFile; fileRef = 99CA66C8203668220074F35E /* EnterFullscreen.js */; };
                99E714C51C124A0400665B3A /* _WKAutomationDelegate.h in Headers */ = {isa = PBXBuildFile; fileRef = 99E714C11C1249E600665B3A /* _WKAutomationDelegate.h */; settings = {ATTRIBUTES = (Private, ); }; };
                9F54F88F16488E87007DF81A /* ChildProcessMac.mm in Sources */ = {isa = PBXBuildFile; fileRef = 9F54F88E16488E87007DF81A /* ChildProcessMac.mm */; };
                9F54F8951648AE0F007DF81A /* PluginProcessManagerMac.mm in Sources */ = {isa = PBXBuildFile; fileRef = 9F54F8941648AE0E007DF81A /* PluginProcessManagerMac.mm */; };
                        dstPath = PrivateHeaders/atoms;
                        dstSubfolderSpec = 1;
                        files = (
+                               99CA66CA2036685D0074F35E /* EnterFullscreen.js in Copy WebDriver Atoms */,
                                99B750F21F33ED5B00C1CFB5 /* ElementAttribute.js in Copy WebDriver Atoms */,
                                99B750F31F33ED5B00C1CFB5 /* ElementDisplayed.js in Copy WebDriver Atoms */,
                                99B750F41F33ED5B00C1CFB5 /* FindNodes.js in Copy WebDriver Atoms */,
                99C81D551C20DFBE005C4C82 /* AutomationClient.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = AutomationClient.h; sourceTree = "<group>"; };
                99C81D561C20DFBE005C4C82 /* AutomationClient.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = AutomationClient.mm; sourceTree = "<group>"; };
                99C81D5B1C20E817005C4C82 /* APIAutomationClient.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = APIAutomationClient.h; sourceTree = "<group>"; };
+               99CA66C8203668220074F35E /* EnterFullscreen.js */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.javascript; path = EnterFullscreen.js; sourceTree = "<group>"; };
                99E714C11C1249E600665B3A /* _WKAutomationDelegate.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = _WKAutomationDelegate.h; sourceTree = "<group>"; };
                99F642D21FABE378009621E9 /* CoordinateSystem.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = CoordinateSystem.h; sourceTree = "<group>"; };
                9BC59D6C1EFCCCB6001E8D09 /* CallbackID.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = CallbackID.h; sourceTree = "<group>"; };
                        children = (
                                990657341F323CBF00944F9C /* ElementAttribute.js */,
                                990657331F323CBF00944F9C /* ElementDisplayed.js */,
+                               99CA66C8203668220074F35E /* EnterFullscreen.js */,
                                990657311F323CBF00944F9C /* FindNodes.js */,
                                990657321F323CBF00944F9C /* FormElementClear.js */,
                                990657351F323CBF00944F9C /* FormSubmit.js */,