AX: Support returning relative frames for accessibility
authorcfleizach@apple.com <cfleizach@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 21 Jan 2019 01:26:37 +0000 (01:26 +0000)
committercfleizach@apple.com <cfleizach@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 21 Jan 2019 01:26:37 +0000 (01:26 +0000)
https://bugs.webkit.org/show_bug.cgi?id=193414
<rdar://problem/47268501>

Reviewed by Zalan Bujtas.

Source/WebCore:

Create a way for assistive technologies to retrieve a frame in page space that can be transformed to its final screen space by having the AT message the UI process separately.

Consolidate rect/point conversion methods for macOS and iOS.
This is only needed on WebKit2, where we have to reach back across to the hosting process to get the final frame, so we can skip this test on WK1.

Tests: accessibility/mac/relative-frame.html

* accessibility/ios/WebAccessibilityObjectWrapperIOS.mm:
(-[WebAccessibilityObjectWrapper _accessibilityConvertPointToViewSpace:]):
(-[WebAccessibilityObjectWrapper _accessibilityRelativeFrame]):
(-[WebAccessibilityObjectWrapper accessibilityVisibleContentRect]):
(-[WebAccessibilityObjectWrapper accessibilityActivationPoint]):
(-[WebAccessibilityObjectWrapper accessibilityFrame]):
(-[WebAccessibilityObjectWrapper frameForTextMarkers:]):
(-[WebAccessibilityObjectWrapper rectsForSelectionRects:]):
(-[WebAccessibilityObjectWrapper convertPointToScreenSpace:]): Deleted.
(-[WebAccessibilityObjectWrapper convertRectToScreenSpace:]): Deleted.
* accessibility/mac/WebAccessibilityObjectWrapperBase.h:
* accessibility/mac/WebAccessibilityObjectWrapperBase.mm:
(convertPathToScreenSpaceFunction):
(-[WebAccessibilityObjectWrapperBase convertRectToSpace:space:]):
(-[WebAccessibilityObjectWrapperBase convertPointToScreenSpace:]): Deleted.
* accessibility/mac/WebAccessibilityObjectWrapperMac.mm:
(-[WebAccessibilityObjectWrapper IGNORE_WARNINGS_END]):
(-[WebAccessibilityObjectWrapper position]):
(-[WebAccessibilityObjectWrapper accessibilityAttributeValue:]):
(-[WebAccessibilityObjectWrapper convertPointToScreenSpace:]): Deleted.

Source/WebKit:

* UIProcess/API/Cocoa/WKWebView.mm:
(-[WKWebView accessibilityAttributeValue:forParameter:]):
(-[WKWebView IGNORE_WARNINGS_END]):
* UIProcess/API/mac/WKView.mm:
(-[WKView accessibilityAttributeValue:forParameter:]):
(-[WKView IGNORE_WARNINGS_END]):
* UIProcess/Cocoa/WebViewImpl.h:
* UIProcess/Cocoa/WebViewImpl.mm:
(WebKit::WebViewImpl::accessibilityAttributeValue):
* UIProcess/ios/WKContentView.mm:
(-[WKContentView accessibilityConvertRelativeFrameFromPage:]):

Tools:

* WebKitTestRunner/InjectedBundle/AccessibilityUIElement.h:
* WebKitTestRunner/InjectedBundle/Bindings/AccessibilityUIElement.idl:
* WebKitTestRunner/InjectedBundle/ios/AccessibilityUIElementIOS.mm:
(WTR::AccessibilityUIElement::stringDescriptionOfAttributeValue):
* WebKitTestRunner/InjectedBundle/mac/AccessibilityUIElementMac.mm:
(WTR::attributesOfElement):
(WTR::AccessibilityUIElement::stringDescriptionOfAttributeValue):

LayoutTests:

* accessibility/mac/relative-frame-expected.txt: Added.
* accessibility/mac/relative-frame.html: Added.
* platform/mac-wk1/TestExpectations:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@240219 268f45cc-cd09-0410-ab3c-d52691b4dbfc

23 files changed:
LayoutTests/ChangeLog
LayoutTests/accessibility/mac/relative-frame-expected.txt [new file with mode: 0644]
LayoutTests/accessibility/mac/relative-frame.html [new file with mode: 0644]
LayoutTests/platform/mac-wk1/TestExpectations
Source/WebCore/ChangeLog
Source/WebCore/accessibility/ios/WebAccessibilityObjectWrapperIOS.mm
Source/WebCore/accessibility/mac/WebAccessibilityObjectWrapperBase.h
Source/WebCore/accessibility/mac/WebAccessibilityObjectWrapperBase.mm
Source/WebCore/accessibility/mac/WebAccessibilityObjectWrapperMac.mm
Source/WebKit/ChangeLog
Source/WebKit/UIProcess/API/Cocoa/WKWebView.mm
Source/WebKit/UIProcess/API/mac/WKView.mm
Source/WebKit/UIProcess/Cocoa/WebViewImpl.h
Source/WebKit/UIProcess/Cocoa/WebViewImpl.mm
Tools/ChangeLog
Tools/DumpRenderTree/mac/AccessibilityUIElementMac.mm
Tools/WebKitTestRunner/InjectedBundle/AccessibilityUIElement.h
Tools/WebKitTestRunner/InjectedBundle/Bindings/AccessibilityUIElement.idl
Tools/WebKitTestRunner/InjectedBundle/atk/AccessibilityUIElementAtk.cpp
Tools/WebKitTestRunner/InjectedBundle/ios/AccessibilityUIElementIOS.mm
Tools/WebKitTestRunner/InjectedBundle/mac/AccessibilityUIElementMac.mm
Tools/WebKitTestRunner/InjectedBundle/win/AccessibilityUIElementWin.cpp
Tools/WebKitTestRunner/InjectedBundle/wpe/AccessibilityUIElementWPE.cpp

index 61f2e07..91111fb 100644 (file)
@@ -1,3 +1,15 @@
+2019-01-20  chris fleizach  <cfleizach@apple.com>
+
+        AX: Support returning relative frames for accessibility
+        https://bugs.webkit.org/show_bug.cgi?id=193414
+        <rdar://problem/47268501>
+
+        Reviewed by Zalan Bujtas.
+
+        * accessibility/mac/relative-frame-expected.txt: Added.
+        * accessibility/mac/relative-frame.html: Added.
+        * platform/mac-wk1/TestExpectations:
+
 2019-01-20  Michael Catanzaro  <mcatanzaro@igalia.com>
 
         Unreviewed gardening, add failure expectation for js/intl-numberformat.html
diff --git a/LayoutTests/accessibility/mac/relative-frame-expected.txt b/LayoutTests/accessibility/mac/relative-frame-expected.txt
new file mode 100644 (file)
index 0000000..cfa2602
--- /dev/null
@@ -0,0 +1,11 @@
+focusable link
+This tests the relative frame attribute returns accurate data.
+
+On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
+
+
+relative frame: NSRect: {{8, 8}, {90, 18}}
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
diff --git a/LayoutTests/accessibility/mac/relative-frame.html b/LayoutTests/accessibility/mac/relative-frame.html
new file mode 100644 (file)
index 0000000..3e247a7
--- /dev/null
@@ -0,0 +1,31 @@
+<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
+<html>
+<head>
+<script>
+var successfullyParsed = false;
+</script>
+<script src="../../resources/js-test-pre.js"></script>
+</head>
+<body id="body">
+
+<a id="link" href="#">focusable link</a>
+
+<p id="description"></p>
+<div id="console"></div>
+
+<script>
+
+    description("This tests the relative frame attribute returns accurate data.");
+
+    if (window.accessibilityController) {
+
+        var link = accessibilityController.accessibleElementById("link");
+        debug("relative frame: " + link.stringDescriptionOfAttributeValue("AXRelativeFrame"));
+    }
+
+    successfullyParsed = true;
+</script>
+
+<script src="../../resources/js-test-post.js"></script>
+</body>
+</html>
index e3f2645..d0887d7 100644 (file)
@@ -537,6 +537,7 @@ webkit.org/b/183352 accessibility/ios-simulator/AOM-dismiss-event.html [ Skip ]
 webkit.org/b/184742 accessibility/mac/async-increment-decrement-action.html [ Skip ]
 webkit.org/b/185897 accessibility/mac/AOM-event-accessiblesetvalue.html [ Skip ]
 webkit.org/b/185897 accessibility/mac/set-value-editable-types.html [ Skip ]
+webkit.org/b/193414 accessibility/mac/relative-frame.html [ Skip ]
 
 webkit.org/b/182752 accessibility/mac/accessibility-make-first-responder.html [ Skip ]
 
@@ -665,4 +666,4 @@ webkit.org/b/190383 fast/images/animated-image-different-dest-size.html [ Pass I
 
 webkit.org/b/191639 imported/blink/compositing/squashing/squashing-into-ancestor-painted-layer.html [ Pass ImageOnlyFailure ]
 
-webkit.org/b/190627  [ Mojave+ ] compositing/masks/compositing-clip-path-change-no-repaint.html  [ Pass Failure ]
\ No newline at end of file
+webkit.org/b/190627  [ Mojave+ ] compositing/masks/compositing-clip-path-change-no-repaint.html  [ Pass Failure ]
index 1e999e4..1a5f36d 100644 (file)
@@ -1,3 +1,39 @@
+2019-01-20  chris fleizach  <cfleizach@apple.com>
+
+        AX: Support returning relative frames for accessibility
+        https://bugs.webkit.org/show_bug.cgi?id=193414
+        <rdar://problem/47268501>
+
+        Reviewed by Zalan Bujtas.
+
+        Create a way for assistive technologies to retrieve a frame in page space that can be transformed to its final screen space by having the AT message the UI process separately.
+
+        Consolidate rect/point conversion methods for macOS and iOS.
+        This is only needed on WebKit2, where we have to reach back across to the hosting process to get the final frame, so we can skip this test on WK1.
+
+        Tests: accessibility/mac/relative-frame.html
+
+        * accessibility/ios/WebAccessibilityObjectWrapperIOS.mm:
+        (-[WebAccessibilityObjectWrapper _accessibilityConvertPointToViewSpace:]):
+        (-[WebAccessibilityObjectWrapper _accessibilityRelativeFrame]):
+        (-[WebAccessibilityObjectWrapper accessibilityVisibleContentRect]):
+        (-[WebAccessibilityObjectWrapper accessibilityActivationPoint]):
+        (-[WebAccessibilityObjectWrapper accessibilityFrame]):
+        (-[WebAccessibilityObjectWrapper frameForTextMarkers:]):
+        (-[WebAccessibilityObjectWrapper rectsForSelectionRects:]):
+        (-[WebAccessibilityObjectWrapper convertPointToScreenSpace:]): Deleted.
+        (-[WebAccessibilityObjectWrapper convertRectToScreenSpace:]): Deleted.
+        * accessibility/mac/WebAccessibilityObjectWrapperBase.h:
+        * accessibility/mac/WebAccessibilityObjectWrapperBase.mm:
+        (convertPathToScreenSpaceFunction):
+        (-[WebAccessibilityObjectWrapperBase convertRectToSpace:space:]):
+        (-[WebAccessibilityObjectWrapperBase convertPointToScreenSpace:]): Deleted.
+        * accessibility/mac/WebAccessibilityObjectWrapperMac.mm:
+        (-[WebAccessibilityObjectWrapper IGNORE_WARNINGS_END]):
+        (-[WebAccessibilityObjectWrapper position]):
+        (-[WebAccessibilityObjectWrapper accessibilityAttributeValue:]):
+        (-[WebAccessibilityObjectWrapper convertPointToScreenSpace:]): Deleted.
+
 2019-01-20  Simon Fraser  <simon.fraser@apple.com>
 
         On RenderBox, make client sizing be derived from padding box sizing
index 7fb62f8..25e8bc4 100644 (file)
@@ -77,7 +77,6 @@ enum {
 @end
 
 @interface WebAccessibilityObjectWrapper (AccessibilityPrivate)
-- (id)_accessibilityWebDocumentView;
 - (id)accessibilityContainer;
 - (void)setAccessibilityLabel:(NSString *)label;
 - (void)setAccessibilityValue:(NSString *)value;
@@ -1498,8 +1497,9 @@ static void appendStringToResult(NSMutableString *result, NSString *string)
     if (![self _prepareAccessibilityCall])
         return point;
     
-    FloatPoint floatPoint = FloatPoint(point);
-    return [self convertPointToScreenSpace:floatPoint];
+    auto floatPoint = FloatPoint(point);
+    auto floatRect = FloatRect(floatPoint, FloatSize());
+    return [self convertRectToSpace:floatRect space:ScreenSpace].origin;
 }
 
 - (BOOL)accessibilityPerformEscape
@@ -1560,102 +1560,10 @@ static void appendStringToResult(NSMutableString *result, NSString *string)
     return result;
 }
 
-- (CGPoint)convertPointToScreenSpace:(FloatPoint &)point
+- (CGRect)_accessibilityRelativeFrame
 {
-    if (!m_object)
-        return CGPointZero;
-    
-    CGPoint cgPoint = CGPointMake(point.x(), point.y());
-    
-    FrameView* frameView = m_object->documentFrameView();
-    WAKView* documentView = frameView ? frameView->documentView() : nullptr;
-    if (documentView) {
-        cgPoint = [documentView convertPoint:cgPoint toView:nil];
-
-        // we need the web document view to give us our final screen coordinates
-        // because that can take account of the scroller
-        id webDocument = [self _accessibilityWebDocumentView];
-        if (webDocument)
-            cgPoint = [webDocument convertPoint:cgPoint toView:nil];
-    }
-    else {
-        // Find the appropriate scroll view to use to convert the contents to the window.
-        ScrollView* scrollView = nullptr;
-        const AccessibilityObject* parent = AccessibilityObject::matchedParent(*m_object, false, [] (const AccessibilityObject& object) {
-            return is<AccessibilityScrollView>(object);
-        });
-        if (parent)
-            scrollView = downcast<AccessibilityScrollView>(*parent).scrollView();
-        
-        IntPoint intPoint = flooredIntPoint(point);
-        if (scrollView)
-            intPoint = scrollView->contentsToRootView(intPoint);
-        
-        Page* page = m_object->page();
-        
-        // If we have an empty chrome client (like SVG) then we should use the page
-        // of the scroll view parent to help us get to the screen rect.
-        if (parent && page && page->chrome().client().isEmptyChromeClient())
-            page = parent->page();
-        
-        if (page) {
-            IntRect rect = IntRect(intPoint, IntSize(0, 0));
-            intPoint = page->chrome().rootViewToAccessibilityScreen(rect).location();
-        }
-        
-        cgPoint = (CGPoint)intPoint;
-    }
-    
-    return cgPoint;
-}
-
-- (CGRect)convertRectToScreenSpace:(IntRect &)rect
-{
-    if (!m_object)
-        return CGRectZero;
-    
-    CGSize size = CGSizeMake(rect.size().width(), rect.size().height());
-    CGPoint point = CGPointMake(rect.x(), rect.y());
-    
-    CGRect frame = CGRectMake(point.x, point.y, size.width, size.height);
-    
-    FrameView* frameView = m_object->documentFrameView();
-    WAKView* documentView = frameView ? frameView->documentView() : nil;
-    if (documentView) {
-        frame = [documentView convertRect:frame toView:nil];
-        
-        // we need the web document view to give us our final screen coordinates
-        // because that can take account of the scroller
-        id webDocument = [self _accessibilityWebDocumentView];
-        if (webDocument)
-            frame = [webDocument convertRect:frame toView:nil];
-        
-    } else {
-        // Find the appropriate scroll view to use to convert the contents to the window.
-        ScrollView* scrollView = nullptr;
-        const AccessibilityObject* parent = AccessibilityObject::matchedParent(*m_object, false, [] (const AccessibilityObject& object) {
-            return is<AccessibilityScrollView>(object);
-        });
-        if (parent)
-            scrollView = downcast<AccessibilityScrollView>(*parent).scrollView();
-        
-        if (scrollView)
-            rect = scrollView->contentsToRootView(rect);
-        
-        Page* page = m_object->page();
-        
-        // If we have an empty chrome client (like SVG) then we should use the page
-        // of the scroll view parent to help us get to the screen rect.
-        if (parent && page && page->chrome().client().isEmptyChromeClient())
-            page = parent->page();
-        
-        if (page)
-            rect = page->chrome().rootViewToAccessibilityScreen(rect);
-        
-        frame = (CGRect)rect;
-    }
-    
-    return frame;
+    auto rect = FloatRect(snappedIntRect(m_object->elementRect()));
+    return [self convertRectToSpace:rect space:PageSpace];
 }
 
 // Used by UIKit accessibility bundle to help determine distance during a hit-test.
@@ -1673,11 +1581,11 @@ static void appendStringToResult(NSMutableString *result, NSString *string)
     if (![self _prepareAccessibilityCall])
         return CGRectZero;
     
-    Document* document = m_object->document();
+    auto document = m_object->document();
     if (!document || !document->view())
         return CGRectZero;
-    IntRect rect = snappedIntRect(document->view()->unobscuredContentRect());
-    return [self convertRectToScreenSpace:rect];
+    auto rect = FloatRect(snappedIntRect(document->view()->unobscuredContentRect()));
+    return [self convertRectToSpace:rect space:ScreenSpace];
 }
 
 // The "center point" is where VoiceOver will "press" an object. This may not be the actual
@@ -1687,8 +1595,8 @@ static void appendStringToResult(NSMutableString *result, NSString *string)
     if (![self _prepareAccessibilityCall])
         return CGPointZero;
     
-    IntRect rect = snappedIntRect(m_object->boundingBoxRect());
-    CGRect cgRect = [self convertRectToScreenSpace:rect];
+    auto rect = FloatRect(snappedIntRect(m_object->boundingBoxRect()));
+    CGRect cgRect = [self convertRectToSpace:rect space:ScreenSpace];
     return CGPointMake(CGRectGetMidX(cgRect), CGRectGetMidY(cgRect));
 }
 
@@ -1697,8 +1605,8 @@ static void appendStringToResult(NSMutableString *result, NSString *string)
     if (![self _prepareAccessibilityCall])
         return CGRectZero;
     
-    IntRect rect = snappedIntRect(m_object->elementRect());
-    return [self convertRectToScreenSpace:rect];
+    auto rect = FloatRect(snappedIntRect(m_object->elementRect()));
+    return [self convertRectToSpace:rect space:ScreenSpace];
 }
 
 // Checks whether a link contains only static text and images (and has been divided unnaturally by <spans> and other nefarious mechanisms).
@@ -2742,8 +2650,8 @@ static void AXAttributedStringAppendText(NSMutableAttributedString* attrString,
     if (!range)
         return CGRectZero;
     
-    IntRect rect = m_object->boundsForRange(range);
-    return [self convertRectToScreenSpace:rect];
+    auto rect = FloatRect(m_object->boundsForRange(range));
+    return [self convertRectToSpace:rect space:ScreenSpace];
 }
 
 - (RefPtr<Range>)rangeFromMarkers:(NSArray *)markers withText:(NSString *)text
@@ -2788,8 +2696,8 @@ static void AXAttributedStringAppendText(NSMutableAttributedString* attrString,
     NSMutableArray *rects = [NSMutableArray arrayWithCapacity:size];
     for (unsigned i = 0; i < size; i++) {
         const WebCore::SelectionRect& coreRect = selectionRects[i];
-        IntRect selectionRect = coreRect.rect();
-        CGRect rect = [self convertRectToScreenSpace:selectionRect];
+        auto selectionRect = FloatRect(coreRect.rect());
+        CGRect rect = [self convertRectToSpace:selectionRect space:ScreenSpace];
         [rects addObject:[NSValue valueWithRect:rect]];
     }
     
index 04749be..c31cab8 100644 (file)
@@ -65,7 +65,9 @@ class VisiblePosition;
 - (void)accessibilityPostedNotification:(NSString *)notificationName userInfo:(NSDictionary *)userInfo;
 
 - (CGPathRef)convertPathToScreenSpace:(WebCore::Path &)path;
-- (CGPoint)convertPointToScreenSpace:(WebCore::FloatPoint &)point;
+
+enum ConversionSpace { ScreenSpace, PageSpace };
+- (CGRect)convertRectToSpace:(WebCore::FloatRect &)rect space:(ConversionSpace)space;
 
 // Math related functions
 - (NSArray *)accessibilityMathPostscriptPairs;
@@ -74,6 +76,10 @@ class VisiblePosition;
 extern WebCore::AccessibilitySearchCriteria accessibilitySearchCriteriaForSearchPredicateParameterizedAttribute(const NSDictionary *);
 extern NSArray *convertToNSArray(const WebCore::AccessibilityObject::AccessibilityChildrenVector&);
 
+#if PLATFORM(IOS_FAMILY)
+- (id)_accessibilityWebDocumentView;
+#endif
+
 @end
 
 #endif // WebAccessibilityObjectWrapperBase_h
index ae66acd..7ad8510 100644 (file)
@@ -42,6 +42,8 @@
 #import "AccessibilityTableCell.h"
 #import "AccessibilityTableColumn.h"
 #import "AccessibilityTableRow.h"
+#import "Chrome.h"
+#import "ChromeClient.h"
 #import "ColorMac.h"
 #import "ContextMenuController.h"
 #import "Editing.h"
@@ -51,6 +53,7 @@
 #import "FrameLoaderClient.h"
 #import "FrameSelection.h"
 #import "HTMLNames.h"
+#import "LayoutRect.h"
 #import "LocalizedStrings.h"
 #import "Page.h"
 #import "RenderTextControl.h"
@@ -60,6 +63,8 @@
 #import "TextCheckerClient.h"
 #import "TextCheckingHelper.h"
 #import "VisibleUnits.h"
+#import "WAKView.h"
+#import "WAKWindow.h"
 #import "WebCoreFrameView.h"
 
 using namespace WebCore;
@@ -461,31 +466,42 @@ static void convertPathToScreenSpaceFunction(PathConversionInfo& conversion, con
 {
     WebAccessibilityObjectWrapperBase *wrapper = conversion.wrapper;
     CGMutablePathRef newPath = conversion.path;
+    FloatRect rect;
     switch (element.type) {
     case PathElementMoveToPoint:
     {
-        CGPoint newPoint = [wrapper convertPointToScreenSpace:element.points[0]];
+        rect = FloatRect(element.points[0], FloatSize());
+        CGPoint newPoint = [wrapper convertRectToSpace:rect space:ScreenSpace].origin;
         CGPathMoveToPoint(newPath, nil, newPoint.x, newPoint.y);
         break;
     }
     case PathElementAddLineToPoint:
     {
-        CGPoint newPoint = [wrapper convertPointToScreenSpace:element.points[0]];
+        rect = FloatRect(element.points[0], FloatSize());
+        CGPoint newPoint = [wrapper convertRectToSpace:rect space:ScreenSpace].origin;
         CGPathAddLineToPoint(newPath, nil, newPoint.x, newPoint.y);
         break;
     }
     case PathElementAddQuadCurveToPoint:
     {
-        CGPoint newPoint1 = [wrapper convertPointToScreenSpace:element.points[0]];
-        CGPoint newPoint2 = [wrapper convertPointToScreenSpace:element.points[1]];
+        rect = FloatRect(element.points[0], FloatSize());
+        CGPoint newPoint1 = [wrapper convertRectToSpace:rect space:ScreenSpace].origin;
+
+        rect = FloatRect(element.points[1], FloatSize());
+        CGPoint newPoint2 = [wrapper convertRectToSpace:rect space:ScreenSpace].origin;
         CGPathAddQuadCurveToPoint(newPath, nil, newPoint1.x, newPoint1.y, newPoint2.x, newPoint2.y);
         break;
     }
     case PathElementAddCurveToPoint:
     {
-        CGPoint newPoint1 = [wrapper convertPointToScreenSpace:element.points[0]];
-        CGPoint newPoint2 = [wrapper convertPointToScreenSpace:element.points[1]];
-        CGPoint newPoint3 = [wrapper convertPointToScreenSpace:element.points[2]];
+        rect = FloatRect(element.points[0], FloatSize());
+        CGPoint newPoint1 = [wrapper convertRectToSpace:rect space:ScreenSpace].origin;
+
+        rect = FloatRect(element.points[1], FloatSize());
+        CGPoint newPoint2 = [wrapper convertRectToSpace:rect space:ScreenSpace].origin;
+
+        rect = FloatRect(element.points[2], FloatSize());
+        CGPoint newPoint3 = [wrapper convertRectToSpace:rect space:ScreenSpace].origin;
         CGPathAddCurveToPoint(newPath, nil, newPoint1.x, newPoint1.y, newPoint2.x, newPoint2.y, newPoint3.x, newPoint3.y);
         break;
     }
@@ -507,11 +523,80 @@ static void convertPathToScreenSpaceFunction(PathConversionInfo& conversion, con
     return conversion.path;
 }
 
-- (CGPoint)convertPointToScreenSpace:(FloatPoint &)point
+- (id)_accessibilityWebDocumentView
 {
-    UNUSED_PARAM(point);
     ASSERT_NOT_REACHED();
-    return CGPointZero;
+    // Overridden by sub-classes
+    return nil;
+}
+
+- (CGRect)convertRectToSpace:(WebCore::FloatRect &)rect space:(ConversionSpace)space
+{
+    if (!m_object)
+        return CGRectZero;
+    
+    CGSize size = CGSizeMake(rect.size().width(), rect.size().height());
+    CGPoint point = CGPointMake(rect.x(), rect.y());
+    
+    CGRect cgRect = CGRectMake(point.x, point.y, size.width, size.height);
+
+    // WebKit1 code path... platformWidget() exists.
+    FrameView* frameView = m_object->documentFrameView();
+#if PLATFORM(IOS_FAMILY)
+    WAKView* documentView = frameView ? frameView->documentView() : nullptr;
+    if (documentView) {
+        cgRect = [documentView convertRect:cgRect toView:nil];
+        
+        // we need the web document view to give us our final screen coordinates
+        // because that can take account of the scroller
+        id webDocument = [self _accessibilityWebDocumentView];
+        if (webDocument)
+            cgRect = [webDocument convertRect:cgRect toView:nil];
+    }
+#else
+    if (frameView && frameView->platformWidget()) {
+        NSRect nsRect = NSRectFromCGRect(cgRect);
+        NSView* view = frameView->documentView();
+        ALLOW_DEPRECATED_DECLARATIONS_BEGIN
+        nsRect = [[view window] convertRectToScreen:[view convertRect:nsRect toView:nil]];
+        ALLOW_DEPRECATED_DECLARATIONS_END
+        cgRect = NSRectToCGRect(nsRect);
+    }
+#endif
+    else {
+        // Find the appropriate scroll view to use to convert the contents to the window.
+        ScrollView* scrollView = nullptr;
+        const AccessibilityObject* parent = AccessibilityObject::matchedParent(*m_object, false, [] (const AccessibilityObject& object) {
+            return is<AccessibilityScrollView>(object);
+        });
+        if (parent)
+            scrollView = downcast<AccessibilityScrollView>(*parent).scrollView();
+        
+        auto intRect = snappedIntRect(IntRect(cgRect));
+        if (scrollView)
+            intRect = scrollView->contentsToRootView(intRect);
+        
+        if (space == ScreenSpace) {
+            auto page = m_object->page();
+            
+            // If we have an empty chrome client (like SVG) then we should use the page
+            // of the scroll view parent to help us get to the screen rect.
+            if (parent && page && page->chrome().client().isEmptyChromeClient())
+                page = parent->page();
+            
+            if (page) {
+#if PLATFORM(IOS_FAMILY)
+                intRect = page->chrome().rootViewToAccessibilityScreen(intRect);
+#else
+                intRect = page->chrome().rootViewToScreen(intRect);
+#endif
+            }
+        }
+        
+        cgRect = (CGRect)intRect;
+    }
+    
+    return cgRect;
 }
 
 - (NSString *)ariaLandmarkRoleDescription
index cafd2b6..370597f 100644 (file)
@@ -412,6 +412,10 @@ using namespace HTMLNames;
 #define NSAccessibilityLinkRelationshipTypeAttribute @"AXLinkRelationshipType"
 #endif
 
+#ifndef NSAccessibilityRelativeFrameAttribute
+#define NSAccessibilityRelativeFrameAttribute @"AXRelativeFrame"
+#endif
+
 extern "C" AXUIElementRef NSAccessibilityCreateAXUIElementRef(id element);
 
 @implementation WebAccessibilityObjectWrapper
@@ -1320,6 +1324,7 @@ IGNORE_WARNINGS_END
             NSAccessibilityFocusableAncestorAttribute,
             NSAccessibilityEditableAncestorAttribute,
             NSAccessibilityHighestEditableAncestorAttribute,
+            NSAccessibilityRelativeFrameAttribute,
             nil];
     }
     if (commonMenuAttrs == nil) {
@@ -1746,50 +1751,6 @@ static NSMutableArray *convertStringsToNSArray(const Vector<String>& vector)
     return static_cast<PluginViewBase*>(pluginWidget)->accessibilityAssociatedPluginParentForElement(m_object->element());
 }
 
-- (CGPoint)convertPointToScreenSpace:(FloatPoint &)point
-{
-    FrameView* frameView = m_object->documentFrameView();
-    
-    // WebKit1 code path... platformWidget() exists.
-    if (frameView && frameView->platformWidget()) {
-        NSPoint nsPoint = (NSPoint)point;
-        NSView* view = frameView->documentView();
-        ALLOW_DEPRECATED_DECLARATIONS_BEGIN
-        nsPoint = [[view window] convertBaseToScreen:[view convertPoint:nsPoint toView:nil]];
-        ALLOW_DEPRECATED_DECLARATIONS_END
-        return CGPointMake(nsPoint.x, nsPoint.y);
-    } else {
-        
-        // Find the appropriate scroll view to use to convert the contents to the window.
-        ScrollView* scrollView = nullptr;
-        AccessibilityObject* parent = nullptr;
-        for (parent = m_object->parentObject(); parent; parent = parent->parentObject()) {
-            if (is<AccessibilityScrollView>(*parent)) {
-                scrollView = downcast<AccessibilityScrollView>(*parent).scrollView();
-                break;
-            }
-        }
-        
-        IntPoint intPoint = flooredIntPoint(point);
-        if (scrollView)
-            intPoint = scrollView->contentsToRootView(intPoint);
-        
-        Page* page = m_object->page();
-        
-        // If we have an empty chrome client (like SVG) then we should use the page
-        // of the scroll view parent to help us get to the screen rect.
-        if (parent && page && page->chrome().client().isEmptyChromeClient())
-            page = parent->page();
-        
-        if (page) {
-            IntRect rect = IntRect(intPoint, IntSize(0, 0));            
-            intPoint = page->chrome().rootViewToScreen(rect).location();
-        }
-        
-        return intPoint;
-    }
-}
-
 static void WebTransformCGPathToNSBezierPath(void* info, const CGPathElement *element)
 {
     NSBezierPath *bezierPath = (__bridge NSBezierPath *)info;
@@ -1836,14 +1797,14 @@ static void WebTransformCGPathToNSBezierPath(void* info, const CGPathElement *el
 
 - (NSValue *)position
 {
-    IntRect rect = snappedIntRect(m_object->elementRect());
+    auto rect = snappedIntRect(m_object->elementRect());
     
     // The Cocoa accessibility API wants the lower-left corner.
-    FloatPoint floatPoint = FloatPoint(rect.x(), rect.maxY());
+    auto floatPoint = FloatPoint(rect.x(), rect.maxY());
 
-    CGPoint cgPoint = [self convertPointToScreenSpace:floatPoint];
-    
-    return [NSValue valueWithPoint:NSMakePoint(cgPoint.x, cgPoint.y)];
+    auto floatRect = FloatRect(floatPoint, FloatSize());
+    CGPoint cgPoint = [self convertRectToSpace:floatRect space:ScreenSpace].origin;
+    return [NSValue valueWithPoint:NSPointFromCGPoint(cgPoint)];
 }
 
 using AccessibilityRoleMap = HashMap<int, CFStringRef>;
@@ -3223,6 +3184,11 @@ IGNORE_WARNINGS_END
         return convertToNSArray(details);
     }
 
+    if ([attributeName isEqualToString:NSAccessibilityRelativeFrameAttribute]) {
+        auto rect = FloatRect(snappedIntRect(m_object->elementRect()));
+        return [NSValue valueWithRect:NSRectFromCGRect([self convertRectToSpace:rect space:PageSpace])];
+    }
+    
     if ([attributeName isEqualToString:@"AXErrorMessageElements"]) {
         AccessibilityObject::AccessibilityChildrenVector errorMessages;
         m_object->ariaErrorMessageElements(errorMessages);
index 9898596..47d2fd1 100644 (file)
@@ -1,3 +1,23 @@
+2019-01-20  chris fleizach  <cfleizach@apple.com>
+
+        AX: Support returning relative frames for accessibility
+        https://bugs.webkit.org/show_bug.cgi?id=193414
+        <rdar://problem/47268501>
+
+        Reviewed by Zalan Bujtas.
+
+        * UIProcess/API/Cocoa/WKWebView.mm:
+        (-[WKWebView accessibilityAttributeValue:forParameter:]):
+        (-[WKWebView IGNORE_WARNINGS_END]):
+        * UIProcess/API/mac/WKView.mm:
+        (-[WKView accessibilityAttributeValue:forParameter:]):
+        (-[WKView IGNORE_WARNINGS_END]):
+        * UIProcess/Cocoa/WebViewImpl.h:
+        * UIProcess/Cocoa/WebViewImpl.mm:
+        (WebKit::WebViewImpl::accessibilityAttributeValue):
+        * UIProcess/ios/WKContentView.mm:
+        (-[WKContentView accessibilityConvertRelativeFrameFromPage:]):
+
 2019-01-20  Michael Catanzaro  <mcatanzaro@igalia.com>
 
         Unreviewed, rolling out r240209.
index 265e6fe..5f754aa 100644 (file)
@@ -4041,6 +4041,21 @@ IGNORE_WARNINGS_END
     return _impl->accessibilityAttributeValue(attribute);
 }
 
+IGNORE_WARNINGS_BEGIN("deprecated-implementations")
+- (id)accessibilityAttributeValue:(NSString *)attribute forParameter:(id)parameter
+IGNORE_WARNINGS_END
+{
+    return _impl->accessibilityAttributeValue(attribute, parameter);
+}
+
+IGNORE_WARNINGS_BEGIN("deprecated-implementations")
+- (NSArray<NSString *> *)accessibilityParameterizedAttributeNames
+IGNORE_WARNINGS_END
+{
+    NSArray<NSString *> *names = [super accessibilityParameterizedAttributeNames];
+    return [names arrayByAddingObject:@"AXConvertRelativeFrame"];
+}
+
 - (NSView *)hitTest:(NSPoint)point
 {
     if (!_impl)
index c7670b6..6bde539 100644 (file)
@@ -809,6 +809,21 @@ IGNORE_WARNINGS_END
     return _data->_impl->accessibilityAttributeValue(attribute);
 }
 
+IGNORE_WARNINGS_BEGIN("deprecated-implementations")
+- (id)accessibilityAttributeValue:(NSString *)attribute forParameter:(id)parameter
+IGNORE_WARNINGS_END
+{
+    return _data->_impl->accessibilityAttributeValue(attribute, parameter);
+}
+
+IGNORE_WARNINGS_BEGIN("deprecated-implementations")
+- (NSArray<NSString *> *)accessibilityParameterizedAttributeNames
+IGNORE_WARNINGS_END
+{
+    NSArray<NSString *> *names = [super accessibilityParameterizedAttributeNames];
+    return [names arrayByAddingObject:@"AXConvertRelativeFrame"];
+}
+
 - (NSView *)hitTest:(NSPoint)point
 {
     if (!_data)
index 5b2138d..6c8cecf 100644 (file)
@@ -409,7 +409,7 @@ public:
     bool accessibilityIsIgnored() const { return false; }
     id accessibilityHitTest(CGPoint);
     void enableAccessibilityIfNecessary();
-    id accessibilityAttributeValue(NSString *);
+    id accessibilityAttributeValue(NSString *, id parameter = nil);
 
     NSTrackingArea *primaryTrackingArea() const { return m_primaryTrackingArea.get(); }
     void setPrimaryTrackingArea(NSTrackingArea *);
index 23a96d9..064e0df 100644 (file)
@@ -3589,7 +3589,7 @@ void WebViewImpl::enableAccessibilityIfNecessary()
     updateWindowAndViewFrames();
 }
 
-id WebViewImpl::accessibilityAttributeValue(NSString *attribute)
+id WebViewImpl::accessibilityAttributeValue(NSString *attribute, id parameter)
 {
     enableAccessibilityIfNecessary();
 
@@ -3612,6 +3612,13 @@ id WebViewImpl::accessibilityAttributeValue(NSString *attribute)
             if ([attribute isEqualToString:NSAccessibilityEnabledAttribute])
                 return @YES;
     
+    if ([attribute isEqualToString:@"AXConvertRelativeFrame"]) {
+        if ([parameter isKindOfClass:[NSValue class]]) {
+            NSRect rect = [(NSValue *)parameter rectValue];
+            return [NSValue valueWithRect:m_pageClient->rootViewToScreen(IntRect(rect))];
+        }
+    }
+    
     return [m_view _web_superAccessibilityAttributeValue:attribute];
 }
 
index 7e837b3..d4814cb 100644 (file)
@@ -1,3 +1,19 @@
+2019-01-20  chris fleizach  <cfleizach@apple.com>
+
+        AX: Support returning relative frames for accessibility
+        https://bugs.webkit.org/show_bug.cgi?id=193414
+        <rdar://problem/47268501>
+
+        Reviewed by Zalan Bujtas.
+
+        * WebKitTestRunner/InjectedBundle/AccessibilityUIElement.h:
+        * WebKitTestRunner/InjectedBundle/Bindings/AccessibilityUIElement.idl:
+        * WebKitTestRunner/InjectedBundle/ios/AccessibilityUIElementIOS.mm:
+        (WTR::AccessibilityUIElement::stringDescriptionOfAttributeValue):
+        * WebKitTestRunner/InjectedBundle/mac/AccessibilityUIElementMac.mm:
+        (WTR::attributesOfElement):
+        (WTR::AccessibilityUIElement::stringDescriptionOfAttributeValue):
+
 2019-01-20  Yusuke Suzuki  <ysuzuki@apple.com>
 
         Unreviewed, add my new email address
index b242a96..fd60d40 100644 (file)
@@ -162,9 +162,9 @@ static NSString* attributesOfElement(id accessibilityObject)
             continue;
         
         // Skip screen-specific information.
-        if ([attribute isEqualToString:@"_AXPrimaryScreenHeight"])
+        if ([attribute isEqualToString:@"_AXPrimaryScreenHeight"] || [attribute isEqualToString:@"AXRelativeFrame"])
             continue;
-        
+
         // accessibilityAttributeValue: can throw an if an attribute is not returned.
         // For DumpRenderTree's purpose, we should ignore those exceptions
         BEGIN_AX_OBJC_EXCEPTIONS
index f24dce8..45c31c0 100644 (file)
@@ -109,6 +109,7 @@ public:
 #endif
 
     // Attributes - platform-independent implementations
+    JSRetainPtr<JSStringRef> stringDescriptionOfAttributeValue(JSStringRef attribute);
     JSRetainPtr<JSStringRef> stringAttributeValue(JSStringRef attribute);
     double numberAttributeValue(JSStringRef attribute);
     JSValueRef uiElementArrayAttributeValue(JSStringRef attribute) const;
index 7d5d69d..9be0b1c 100644 (file)
@@ -58,6 +58,7 @@ interface AccessibilityUIElement {
     readonly attribute long insertionPointLineNumber;
     readonly attribute DOMString selectedTextRange;
 
+    DOMString stringDescriptionOfAttributeValue(DOMString attr);
     DOMString stringAttributeValue(DOMString attr);
     double numberAttributeValue(DOMString attr);
     object uiElementArrayAttributeValue(DOMString attr);
index 430c022..8efba4e 100644 (file)
@@ -1076,6 +1076,12 @@ JSRetainPtr<JSStringRef> AccessibilityUIElement::allAttributes()
     return JSStringCreateWithUTF8CString(attributesOfElement(this).utf8().data());
 }
 
+JSRetainPtr<JSStringRef> AccessibilityUIElement::stringDescriptionOfAttributeValue(JSStringRef attribute)
+{
+    // FIXME: implement
+    return JSStringCreateWithCharacters(0, 0);
+}
+
 JSRetainPtr<JSStringRef> AccessibilityUIElement::stringAttributeValue(JSStringRef attribute)
 {
     if (!ATK_IS_OBJECT(m_element.get()))
index 9f8108a..5913b5d 100644 (file)
@@ -348,6 +348,11 @@ JSRetainPtr<JSStringRef> AccessibilityUIElement::allAttributes()
     return createEmptyJSString();
 }
 
+JSRetainPtr<JSStringRef> AccessibilityUIElement::stringDescriptionOfAttributeValue(JSStringRef attribute)
+{
+    return createEmptyJSString();
+}
+
 JSRetainPtr<JSStringRef> AccessibilityUIElement::stringAttributeValue(JSStringRef attribute)
 {
     if (JSStringIsEqualToUTF8CString(attribute, "AXPlaceholderValue"))
index 669d6ee..02e83a6 100644 (file)
@@ -176,9 +176,9 @@ static NSString* attributesOfElement(id accessibilityObject)
             continue;
         
         // Skip screen-specific information.
-        if ([attribute isEqualToString:@"_AXPrimaryScreenHeight"])
+        if ([attribute isEqualToString:@"_AXPrimaryScreenHeight"] || [attribute isEqualToString:@"AXRelativeFrame"])
             continue;
-        
+
         // accessibilityAttributeValue: can throw an if an attribute is not returned.
         // For DumpRenderTree's purpose, we should ignore those exceptions
         BEGIN_AX_OBJC_EXCEPTIONS
@@ -556,6 +556,15 @@ JSRetainPtr<JSStringRef> AccessibilityUIElement::allAttributes()
     NSString* attributes = attributesOfElement(m_element);
     return [attributes createJSStringRef];
 }
+    
+JSRetainPtr<JSStringRef> AccessibilityUIElement::stringDescriptionOfAttributeValue(JSStringRef attribute)
+{
+    BEGIN_AX_OBJC_EXCEPTIONS
+    NSString *value = descriptionOfValue([m_element accessibilityAttributeValue:[NSString stringWithJSStringRef:attribute]], m_element);
+    return [value createJSStringRef];
+    END_AX_OBJC_EXCEPTIONS
+    return nullptr;
+}
 
 JSRetainPtr<JSStringRef> AccessibilityUIElement::stringAttributeValue(JSStringRef attribute)
 {
index e60eca6..dbb0495 100644 (file)
@@ -190,6 +190,12 @@ JSRetainPtr<JSStringRef> AccessibilityUIElement::stringAttributeValue(JSStringRe
     return nullptr;
 }
 
+JSRetainPtr<JSStringRef> AccessibilityUIElement::stringDescriptionOfAttributeValue(JSStringRef)
+{
+    notImplemented();
+    return nullptr;
+}
+
 double AccessibilityUIElement::numberAttributeValue(JSStringRef attribute)
 {
     notImplemented();
index bcc9d54..8343cfe 100644 (file)
@@ -183,6 +183,12 @@ JSRetainPtr<JSStringRef> AccessibilityUIElement::stringAttributeValue(JSStringRe
     return nullptr;
 }
 
+JSRetainPtr<JSStringRef> AccessibilityUIElement::stringDescriptionOfAttributeValue(JSStringRef)
+{
+    notImplemented();
+    return nullptr;
+}
+
 double AccessibilityUIElement::numberAttributeValue(JSStringRef attribute)
 {
     notImplemented();