Data interaction should register type identifiers in order of priority
authorwenson_hsieh@apple.com <wenson_hsieh@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 4 Apr 2017 04:31:48 +0000 (04:31 +0000)
committerwenson_hsieh@apple.com <wenson_hsieh@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 4 Apr 2017 04:31:48 +0000 (04:31 +0000)
https://bugs.webkit.org/show_bug.cgi?id=170428
<rdar://problem/30633296>

Reviewed by Tim Horton.

Source/WebCore:

Currently, due to the shared logic for writing to a UIPasteboard (in the case of copy/paste) and the shared
UIItemProvider-based pasteboard wrapper (in the case of data interaction), we don't enforce any particular
ordering in which type identifiers are registered in the generated item provider. This is because
-[UIPasteboard setItems:] only takes an unordered mapping of type identifiers to objects.

To fix this, we introduce a separate way to set pasteboard data that first writes a list of object
representations in order of priority (which is dependent on the content being interacted with) and then register
individual type-data mappings at the end.

Augmented existing API tests in DataInteractionTests to check for the existence and priority of type identifiers
in the UIItemProviders created upon starting data interaction. Also adds a new unit test:
DataInteractionTests.TextAreaToInput.

* WebCore.xcodeproj/project.pbxproj:
* editing/cocoa/EditorCocoa.mm:
(WebCore::archivedDataForAttributedString):
(WebCore::Editor::writeSelectionToPasteboard):
(WebCore::Editor::writeSelection):
* platform/Pasteboard.h:
* platform/PasteboardWriterData.h:
* platform/PlatformPasteboard.h:
* platform/ios/AbstractPasteboard.h:
* platform/ios/AbstractPasteboard.mm: Copied from Source/WebCore/platform/ios/AbstractPasteboard.h.

Introduce WebPasteboardItemData, a wrapper around a list of objects representating the pasteboard data in order
of priority, and a dictionary containing additional NSData blobs that contain data useful for private clients.

(+[WebPasteboardItemData itemWithRepresentingObjects:additionalData:]):
(-[WebPasteboardItemData initWithRepresentingObjects:additionalData:]):
(-[WebPasteboardItemData representingObjects]):
(-[WebPasteboardItemData additionalData]):
* platform/ios/PlatformPasteboardIOS.mm:
(WebCore::richTextRepresentationsForPasteboardWebContent):
(WebCore::PlatformPasteboard::writeObjectRepresentations):
(WebCore::PlatformPasteboard::write):

Tweaked to check whether the pasteboard responds to -setItemsFromObjectRepresentations:. If so, uses the
PlatformPasteboard::writeObjectRepresentations codepath to write data to the pasteboard, respecting type
priority.

* platform/ios/WebItemProviderPasteboard.mm:
(-[WebItemProviderPasteboard pasteboardTypes]):
(-[WebItemProviderPasteboard setItemProviders:]):
(-[WebItemProviderPasteboard setItemsFromObjectRepresentations:]):

Replaces -setItems: with -setItemsFromObjectRepresentations:, which respects the priority of each object
representation of the data in the pasteboard.

(-[WebItemProviderPasteboard setItems:]): Deleted.

Source/WebKit2:

Serialize PasteboardWebContent.dataInAttributedStringFormat when sending over IPC. Refer to WebCore ChangeLog
for more details.

* Shared/WebCoreArgumentCoders.cpp:
(IPC::ArgumentCoder<PasteboardWebContent>::encode):
(IPC::ArgumentCoder<PasteboardWebContent>::decode):
* UIProcess/API/Cocoa/WKWebView.mm:

Tools:

Augments existing unit tests to check for the existence and priority of type identifiers in the UIItemProviders
created upon starting data interaction. Also fixes a race condition in one of the unit tests and adds a new unit
test for data interaction from a textarea to an input.

* TestWebKitAPI/Tests/ios/DataInteractionTests.mm:
(checkTypeIdentifierPrecedesOtherTypeIdentifier):
(TestWebKitAPI::TEST):
* TestWebKitAPI/ios/DataInteractionSimulator.h:
* TestWebKitAPI/ios/DataInteractionSimulator.mm:
(-[DataInteractionSimulator _advanceProgress]):
(-[DataInteractionSimulator sourceItemProviders]):
(-[DataInteractionSimulator _webView:showCustomSheetForElement:]):

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214865 268f45cc-cd09-0410-ab3c-d52691b4dbfc

17 files changed:
Source/WebCore/ChangeLog
Source/WebCore/WebCore.xcodeproj/project.pbxproj
Source/WebCore/editing/cocoa/EditorCocoa.mm
Source/WebCore/platform/Pasteboard.h
Source/WebCore/platform/PasteboardWriterData.h
Source/WebCore/platform/PlatformPasteboard.h
Source/WebCore/platform/ios/AbstractPasteboard.h
Source/WebCore/platform/ios/AbstractPasteboard.mm [new file with mode: 0644]
Source/WebCore/platform/ios/PlatformPasteboardIOS.mm
Source/WebCore/platform/ios/WebItemProviderPasteboard.mm
Source/WebKit2/ChangeLog
Source/WebKit2/Shared/WebCoreArgumentCoders.cpp
Source/WebKit2/UIProcess/API/Cocoa/WKWebView.mm
Tools/ChangeLog
Tools/TestWebKitAPI/Tests/ios/DataInteractionTests.mm
Tools/TestWebKitAPI/ios/DataInteractionSimulator.h
Tools/TestWebKitAPI/ios/DataInteractionSimulator.mm

index 0d65c03..efe0368 100644 (file)
@@ -1,3 +1,61 @@
+2017-04-03  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        Data interaction should register type identifiers in order of priority
+        https://bugs.webkit.org/show_bug.cgi?id=170428
+        <rdar://problem/30633296>
+
+        Reviewed by Tim Horton.
+
+        Currently, due to the shared logic for writing to a UIPasteboard (in the case of copy/paste) and the shared
+        UIItemProvider-based pasteboard wrapper (in the case of data interaction), we don't enforce any particular
+        ordering in which type identifiers are registered in the generated item provider. This is because
+        -[UIPasteboard setItems:] only takes an unordered mapping of type identifiers to objects.
+
+        To fix this, we introduce a separate way to set pasteboard data that first writes a list of object
+        representations in order of priority (which is dependent on the content being interacted with) and then register
+        individual type-data mappings at the end.
+
+        Augmented existing API tests in DataInteractionTests to check for the existence and priority of type identifiers
+        in the UIItemProviders created upon starting data interaction. Also adds a new unit test:
+        DataInteractionTests.TextAreaToInput.
+
+        * WebCore.xcodeproj/project.pbxproj:
+        * editing/cocoa/EditorCocoa.mm:
+        (WebCore::archivedDataForAttributedString):
+        (WebCore::Editor::writeSelectionToPasteboard):
+        (WebCore::Editor::writeSelection):
+        * platform/Pasteboard.h:
+        * platform/PasteboardWriterData.h:
+        * platform/PlatformPasteboard.h:
+        * platform/ios/AbstractPasteboard.h:
+        * platform/ios/AbstractPasteboard.mm: Copied from Source/WebCore/platform/ios/AbstractPasteboard.h.
+
+        Introduce WebPasteboardItemData, a wrapper around a list of objects representating the pasteboard data in order
+        of priority, and a dictionary containing additional NSData blobs that contain data useful for private clients.
+
+        (+[WebPasteboardItemData itemWithRepresentingObjects:additionalData:]):
+        (-[WebPasteboardItemData initWithRepresentingObjects:additionalData:]):
+        (-[WebPasteboardItemData representingObjects]):
+        (-[WebPasteboardItemData additionalData]):
+        * platform/ios/PlatformPasteboardIOS.mm:
+        (WebCore::richTextRepresentationsForPasteboardWebContent):
+        (WebCore::PlatformPasteboard::writeObjectRepresentations):
+        (WebCore::PlatformPasteboard::write):
+
+        Tweaked to check whether the pasteboard responds to -setItemsFromObjectRepresentations:. If so, uses the
+        PlatformPasteboard::writeObjectRepresentations codepath to write data to the pasteboard, respecting type
+        priority.
+
+        * platform/ios/WebItemProviderPasteboard.mm:
+        (-[WebItemProviderPasteboard pasteboardTypes]):
+        (-[WebItemProviderPasteboard setItemProviders:]):
+        (-[WebItemProviderPasteboard setItemsFromObjectRepresentations:]):
+
+        Replaces -setItems: with -setItemsFromObjectRepresentations:, which respects the priority of each object
+        representation of the data in the pasteboard.
+
+        (-[WebItemProviderPasteboard setItems:]): Deleted.
+
 2017-04-03  Javier Fernandez  <jfernandez@igalia.com>
 
         [css-align] Adapt place-content alignment shorthand to the new baseline syntax
index b13d393..79485a2 100644 (file)
                F48223131E386E240066FC79 /* AbstractPasteboard.h in Headers */ = {isa = PBXBuildFile; fileRef = F48223121E386E240066FC79 /* AbstractPasteboard.h */; settings = {ATTRIBUTES = (Private, ); }; };
                F4BFB9851E1DDF9B00862C24 /* DumpEditingHistory.js in Copy Scripts */ = {isa = PBXBuildFile; fileRef = F48389831E1DDF2B0076B7EA /* DumpEditingHistory.js */; };
                F4BFB9861E1DDF9B00862C24 /* EditingHistoryUtil.js in Copy Scripts */ = {isa = PBXBuildFile; fileRef = F48389841E1DDF2B0076B7EA /* EditingHistoryUtil.js */; };
+               F4D831621E908C0700941174 /* AbstractPasteboard.mm in Sources */ = {isa = PBXBuildFile; fileRef = F4D831611E908C0700941174 /* AbstractPasteboard.mm */; };
                F50664F7157F52DC00AC226F /* FormController.cpp in Sources */ = {isa = PBXBuildFile; fileRef = F50664F5157F52DC00AC226F /* FormController.cpp */; };
                F50664F8157F52DC00AC226F /* FormController.h in Headers */ = {isa = PBXBuildFile; fileRef = F50664F6157F52DC00AC226F /* FormController.h */; };
                F513A3EA15FF4841001526DB /* ValidationMessageClient.h in Headers */ = {isa = PBXBuildFile; fileRef = F513A3E915FF4841001526DB /* ValidationMessageClient.h */; settings = {ATTRIBUTES = (Private, ); }; };
                F48223121E386E240066FC79 /* AbstractPasteboard.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = AbstractPasteboard.h; sourceTree = "<group>"; };
                F48389831E1DDF2B0076B7EA /* DumpEditingHistory.js */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.javascript; name = DumpEditingHistory.js; path = Scripts/DumpEditingHistory.js; sourceTree = "<group>"; };
                F48389841E1DDF2B0076B7EA /* EditingHistoryUtil.js */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.javascript; name = EditingHistoryUtil.js; path = Scripts/EditingHistoryUtil.js; sourceTree = "<group>"; };
+               F4D831611E908C0700941174 /* AbstractPasteboard.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = AbstractPasteboard.mm; sourceTree = "<group>"; };
                F50664F5157F52DC00AC226F /* FormController.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = FormController.cpp; sourceTree = "<group>"; };
                F50664F6157F52DC00AC226F /* FormController.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = FormController.h; sourceTree = "<group>"; };
                F513A3E915FF4841001526DB /* ValidationMessageClient.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ValidationMessageClient.h; sourceTree = "<group>"; };
                        children = (
                                A148328B187F506800DA63A6 /* wak */,
                                F48223121E386E240066FC79 /* AbstractPasteboard.h */,
+                               F4D831611E908C0700941174 /* AbstractPasteboard.mm */,
                                2655414B1489AA2B000DFC5D /* CursorIOS.cpp */,
                                A1ED778A1BE3293F00DC1791 /* Device.cpp */,
                                A1ED778B1BE3294000DC1791 /* Device.h */,
                                A6148A7812E41E3B0044A784 /* JSHTMLKeygenElement.cpp in Sources */,
                                1AE2AB210A1CE63B00B42B25 /* JSHTMLLabelElement.cpp in Sources */,
                                1AE2AB230A1CE63B00B42B25 /* JSHTMLLegendElement.cpp in Sources */,
+                               F4D831621E908C0700941174 /* AbstractPasteboard.mm in Sources */,
                                1AE2AB250A1CE63B00B42B25 /* JSHTMLLIElement.cpp in Sources */,
                                A80E7B100A19D606007FB8C5 /* JSHTMLLinkElement.cpp in Sources */,
                                1AE2AB270A1CE63B00B42B25 /* JSHTMLMapElement.cpp in Sources */,
index 40286f4..05c8908 100644 (file)
@@ -152,6 +152,14 @@ FragmentAndResources Editor::createFragment(NSAttributedString *string)
     return result;
 }
 
+static RefPtr<SharedBuffer> archivedDataForAttributedString(NSAttributedString *attributedString)
+{
+    if (!attributedString.length)
+        return nullptr;
+
+    return SharedBuffer::wrapNSData([NSKeyedArchiver archivedDataWithRootObject:attributedString]);
+}
+
 void Editor::writeSelectionToPasteboard(Pasteboard& pasteboard)
 {
     NSAttributedString *attributedString = attributedStringFromRange(*selectedRange());
@@ -161,6 +169,7 @@ void Editor::writeSelectionToPasteboard(Pasteboard& pasteboard)
     content.dataInWebArchiveFormat = selectionInWebArchiveFormat();
     content.dataInRTFDFormat = attributedString.containsAttachments ? dataInRTFDFormat(attributedString) : nullptr;
     content.dataInRTFFormat = dataInRTFFormat(attributedString);
+    content.dataInAttributedStringFormat = archivedDataForAttributedString(attributedString);
     // FIXME: Why don't we want this on iOS?
 #if PLATFORM(MAC)
     content.dataInHTMLFormat = selectionInHTMLFormat();
@@ -180,6 +189,7 @@ void Editor::writeSelection(PasteboardWriterData& pasteboardWriterData)
     webContent.dataInWebArchiveFormat = selectionInWebArchiveFormat();
     webContent.dataInRTFDFormat = attributedString.containsAttachments ? dataInRTFDFormat(attributedString) : nullptr;
     webContent.dataInRTFFormat = dataInRTFFormat(attributedString);
+    webContent.dataInAttributedStringFormat = archivedDataForAttributedString(attributedString);
     // FIXME: Why don't we want this on iOS?
 #if PLATFORM(MAC)
     webContent.dataInHTMLFormat = selectionInHTMLFormat();
index 3dd00d3..b94d70a 100644 (file)
@@ -69,6 +69,7 @@ struct PasteboardWebContent {
     RefPtr<SharedBuffer> dataInWebArchiveFormat;
     RefPtr<SharedBuffer> dataInRTFDFormat;
     RefPtr<SharedBuffer> dataInRTFFormat;
+    RefPtr<SharedBuffer> dataInAttributedStringFormat;
     String dataInHTMLFormat;
     String dataInStringFormat;
     Vector<String> clientTypes;
index 2c5adc2..0d1b4b7 100644 (file)
@@ -54,6 +54,7 @@ public:
         RefPtr<SharedBuffer> dataInWebArchiveFormat;
         RefPtr<SharedBuffer> dataInRTFDFormat;
         RefPtr<SharedBuffer> dataInRTFFormat;
+        RefPtr<SharedBuffer> dataInAttributedStringFormat;
         // FIXME: Why don't we want this on iOS?
 #if PLATFORM(MAC)
         String dataInHTMLFormat;
index ba5426f..3430118 100644 (file)
@@ -88,6 +88,12 @@ public:
 #endif
 
 private:
+#if PLATFORM(IOS)
+    WEBCORE_EXPORT void writeObjectRepresentations(const PasteboardWebContent&);
+    WEBCORE_EXPORT void writeObjectRepresentations(const PasteboardImage&);
+    WEBCORE_EXPORT void writeObjectRepresentations(const String& pasteboardType, const String& text);
+#endif
+
 #if PLATFORM(MAC)
     RetainPtr<NSPasteboard> m_pasteboard;
 #endif
index a1fe040..6e69fc8 100644 (file)
 
 #import <Foundation/Foundation.h>
 
+#if TARGET_OS_IPHONE
+
+WEBCORE_EXPORT @interface WebPasteboardItemData : NSObject
+
++ (instancetype)itemWithRepresentingObjects:(NSArray *)representingObjects additionalData:(NSDictionary *)additionalData;
+
+@property (nonatomic, readonly, strong) NSArray *representingObjects;
+@property (nonatomic, readonly, strong) NSDictionary *additionalData;
+
+@end
+
 @protocol AbstractPasteboard <NSObject>
 @required
 
 @property (readonly, nonatomic) NSInteger numberOfItems;
 
 - (NSArray<NSString *> *)pasteboardTypes;
-- (void)setItems:(NSArray *)items;
 - (NSArray *)dataForPasteboardType:(NSString *)pasteboardType inItemSet:(NSIndexSet *)itemSet;
 - (NSArray *)valuesForPasteboardType:(NSString *)pasteboardType inItemSet:(NSIndexSet *)itemSet;
 - (NSInteger)changeCount;
 
+@optional
+- (void)setItemsFromObjectRepresentations:(NSArray<WebPasteboardItemData *> *)itemData;
+- (void)setItems:(NSArray<NSDictionary *> *)items;
+
 @end
+
+#endif // TARGET_OS_IPHONE
diff --git a/Source/WebCore/platform/ios/AbstractPasteboard.mm b/Source/WebCore/platform/ios/AbstractPasteboard.mm
new file mode 100644 (file)
index 0000000..27a2a57
--- /dev/null
@@ -0,0 +1,69 @@
+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "AbstractPasteboard.h"
+
+#import <wtf/RetainPtr.h>
+
+#if PLATFORM(IOS)
+
+@interface WebPasteboardItemData ()
+{
+    RetainPtr<NSArray> _representingObjects;
+    RetainPtr<NSDictionary> _additionalData;
+}
+
+@end
+
+@implementation WebPasteboardItemData
+
++ (instancetype)itemWithRepresentingObjects:(NSArray *)representingObjects additionalData:(NSDictionary *)additionalData
+{
+    return [[[self alloc] initWithRepresentingObjects:representingObjects additionalData:additionalData] autorelease];
+}
+
+- (instancetype)initWithRepresentingObjects:(NSArray *)representingObjects additionalData:(NSDictionary *)additionalData
+{
+    if (self = [super init]) {
+        _representingObjects = representingObjects;
+        _additionalData = additionalData;
+    }
+    return self;
+}
+
+- (NSArray *)representingObjects
+{
+    return _representingObjects.get();
+}
+
+- (NSDictionary *)additionalData
+{
+    return _additionalData.get();
+}
+
+@end
+
+#endif // PLATFORM(IOS)
index eb6b0d2..55c9c17 100644 (file)
 #import "SoftLinking.h"
 #import "WebItemProviderPasteboard.h"
 #import <MobileCoreServices/MobileCoreServices.h>
+#import <UIKit/UIImage.h>
 
 SOFT_LINK_FRAMEWORK(UIKit)
+SOFT_LINK_CLASS(UIKit, UIImage)
 SOFT_LINK_CLASS(UIKit, UIPasteboard)
 
 @interface UIPasteboard
@@ -147,9 +149,9 @@ String PlatformPasteboard::uniqueName()
     return String();
 }
 
-void PlatformPasteboard::write(const PasteboardWebContent& content)
+static RetainPtr<NSDictionary> richTextRepresentationsForPasteboardWebContent(const PasteboardWebContent& content)
 {
-    RetainPtr<NSDictionary> representations = adoptNS([[NSMutableDictionary alloc] init]);
+    RetainPtr<NSMutableDictionary> representations = adoptNS([[NSMutableDictionary alloc] init]);
 
     ASSERT(content.clientTypes.size() == content.clientData.size());
     for (size_t i = 0, size = content.clientTypes.size(); i < size; ++i)
@@ -158,7 +160,7 @@ void PlatformPasteboard::write(const PasteboardWebContent& content)
     if (content.dataInWebArchiveFormat) {
         [representations setValue:(NSData *)content.dataInWebArchiveFormat->createNSData().get() forKey:WebArchivePboardType];
         // Flag for UIKit to know that this copy contains rich content. This will trigger a two-step paste.
-        NSStringwebIOSPastePboardType = @"iOS rich content paste pasteboard type";
+        NSString *webIOSPastePboardType = @"iOS rich content paste pasteboard type";
         [representations setValue:webIOSPastePboardType forKey:webIOSPastePboardType];
     }
 
@@ -167,13 +169,70 @@ void PlatformPasteboard::write(const PasteboardWebContent& content)
     if (content.dataInRTFFormat)
         [representations setValue:content.dataInRTFFormat->createNSData().get() forKey:(NSString *)kUTTypeRTF];
 
+    return representations;
+}
+
+void PlatformPasteboard::writeObjectRepresentations(const PasteboardWebContent& content)
+{
+    RetainPtr<NSMutableArray> objectRepresentations = adoptNS([[NSMutableArray alloc] init]);
+    RetainPtr<NSDictionary> additionalData = richTextRepresentationsForPasteboardWebContent(content);
+
+    if (content.dataInAttributedStringFormat) {
+        NSAttributedString *attributedString = [NSKeyedUnarchiver unarchiveObjectWithData:content.dataInAttributedStringFormat->createNSData().get()];
+        if (attributedString)
+            [objectRepresentations addObject:attributedString];
+    }
+
+    if (!content.dataInStringFormat.isEmpty())
+        [objectRepresentations addObject:(NSString *)content.dataInStringFormat];
+
+    [m_pasteboard setItemsFromObjectRepresentations:@[[WebPasteboardItemData itemWithRepresentingObjects:objectRepresentations.get() additionalData:additionalData.get()]]];
+}
+
+void PlatformPasteboard::write(const PasteboardWebContent& content)
+{
+    if ([m_pasteboard respondsToSelector:@selector(setItemsFromObjectRepresentations:)]) {
+        writeObjectRepresentations(content);
+        return;
+    }
+
+    RetainPtr<NSMutableDictionary> representations = adoptNS([[NSMutableDictionary alloc] init]);
+    [representations addEntriesFromDictionary:richTextRepresentationsForPasteboardWebContent(content).autorelease()];
     [representations setValue:content.dataInStringFormat forKey:(NSString *)kUTTypeText];
-    [representations setValue:[(NSString *)content.dataInStringFormat dataUsingEncoding:NSUTF8StringEncoding] forKey:(NSString *)kUTTypeUTF8PlainText];
+
     [m_pasteboard setItems:@[representations.get()]];
 }
 
+void PlatformPasteboard::writeObjectRepresentations(const PasteboardImage& pasteboardImage)
+{
+    RetainPtr<NSMutableArray> objectRepresentations = adoptNS([[NSMutableArray alloc] init]);
+    RetainPtr<NSMutableDictionary> additionalData = adoptNS([[NSMutableDictionary alloc] init]);
+
+    if (auto nativeImage = pasteboardImage.image->nativeImage()) {
+        UIImage *uiImage = (UIImage *)[getUIImageClass() imageWithCGImage:nativeImage.get()];
+        if (uiImage)
+            [objectRepresentations addObject:uiImage];
+    }
+
+    if (!pasteboardImage.url.url.isEmpty()) {
+        NSURL *nsURL = pasteboardImage.url.url;
+        if (nsURL)
+            [objectRepresentations addObject:nsURL];
+    }
+
+    if (!pasteboardImage.resourceMIMEType.isNull())
+        [additionalData setObject:pasteboardImage.resourceData->createNSData().get() forKey:pasteboardImage.resourceMIMEType];
+
+    [m_pasteboard setItemsFromObjectRepresentations:@[[WebPasteboardItemData itemWithRepresentingObjects:objectRepresentations.get() additionalData:additionalData.get()]]];
+}
+
 void PlatformPasteboard::write(const PasteboardImage& pasteboardImage)
 {
+    if ([m_pasteboard respondsToSelector:@selector(setItemsFromObjectRepresentations:)]) {
+        writeObjectRepresentations(pasteboardImage);
+        return;
+    }
+
     RetainPtr<NSMutableDictionary> representations = adoptNS([[NSMutableDictionary alloc] init]);
     if (!pasteboardImage.resourceMIMEType.isNull()) {
         [representations setObject:pasteboardImage.resourceData->createNSData().get() forKey:pasteboardImage.resourceMIMEType];
@@ -182,8 +241,33 @@ void PlatformPasteboard::write(const PasteboardImage& pasteboardImage)
     [m_pasteboard setItems:@[representations.get()]];
 }
 
+void PlatformPasteboard::writeObjectRepresentations(const String& pasteboardType, const String& text)
+{
+    RetainPtr<NSMutableArray> objectRepresentations = adoptNS([[NSMutableArray alloc] init]);
+    RetainPtr<NSMutableDictionary> additionalData = adoptNS([[NSMutableDictionary alloc] init]);
+
+    NSString *pasteboardTypeAsNSString = pasteboardType;
+    NSString *textAsNSString = text;
+    if (textAsNSString && pasteboardTypeAsNSString.length) {
+        if (UTTypeConformsTo((__bridge CFStringRef)pasteboardTypeAsNSString, kUTTypeURL))
+            [objectRepresentations addObject:[[[NSURL alloc] initWithString:textAsNSString] autorelease]];
+
+        if (UTTypeConformsTo((__bridge CFStringRef)pasteboardTypeAsNSString, kUTTypeText))
+            [objectRepresentations addObject:textAsNSString];
+        else
+            [additionalData setObject:textAsNSString forKey:pasteboardTypeAsNSString];
+    }
+
+    [m_pasteboard setItemsFromObjectRepresentations:@[[WebPasteboardItemData itemWithRepresentingObjects:objectRepresentations.get() additionalData:additionalData.get()]]];
+}
+
 void PlatformPasteboard::write(const String& pasteboardType, const String& text)
 {
+    if ([m_pasteboard respondsToSelector:@selector(setItemsFromObjectRepresentations:)]) {
+        writeObjectRepresentations(pasteboardType, text);
+        return;
+    }
+
     RetainPtr<NSDictionary> representations = adoptNS([[NSMutableDictionary alloc] init]);
 
     if (pasteboardType == String(kUTTypeURL)) {
index a89aaed..920fab0 100644 (file)
@@ -80,6 +80,7 @@ static BOOL isImageType(NSString *type)
 
 @implementation WebItemProviderPasteboard {
     RetainPtr<NSArray> _itemProviders;
+    RetainPtr<NSArray> _cachedTypeIdentifiers;
 }
 
 + (instancetype)sharedInstance
@@ -104,10 +105,22 @@ static BOOL isImageType(NSString *type)
 
 - (NSArray<NSString *> *)pasteboardTypes
 {
+    if (_cachedTypeIdentifiers)
+        return _cachedTypeIdentifiers.get();
+
     NSMutableSet<NSString *> *allTypes = [NSMutableSet set];
-    for (UIItemProvider *provider in _itemProviders.get())
-        [allTypes addObjectsFromArray:provider.registeredTypeIdentifiers];
-    return allTypes.allObjects;
+    NSMutableArray<NSString *> *allTypesInOrder = [NSMutableArray array];
+    for (UIItemProvider *provider in _itemProviders.get()) {
+        for (NSString *typeIdentifier in provider.registeredTypeIdentifiers) {
+            if ([allTypes containsObject:typeIdentifier])
+                continue;
+
+            [allTypes addObject:typeIdentifier];
+            [allTypesInOrder addObject:typeIdentifier];
+        }
+    }
+    _cachedTypeIdentifiers = allTypesInOrder;
+    return _cachedTypeIdentifiers.get();
 }
 
 - (NSArray<UIItemProvider *> *)itemProviders
@@ -123,6 +136,7 @@ static BOOL isImageType(NSString *type)
 
     _itemProviders = itemProviders;
     _changeCount++;
+    _cachedTypeIdentifiers = nil;
 }
 
 - (NSInteger)numberOfItems
@@ -130,40 +144,38 @@ static BOOL isImageType(NSString *type)
     return [_itemProviders count];
 }
 
-- (void)setItems:(NSArray *)items
+- (void)setItemsFromObjectRepresentations:(NSArray<WebPasteboardItemData *> *)itemData
 {
     NSMutableArray *providers = [NSMutableArray array];
-    for (NSDictionary *item in items) {
-        if (!item.count)
+    for (WebPasteboardItemData *data in itemData) {
+        if (!data.representingObjects.count && !data.additionalData.count)
             continue;
+
         RetainPtr<UIItemProvider> itemProvider = adoptNS([[getUIItemProviderClass() alloc] init]);
-        RetainPtr<NSMutableDictionary> itemRepresentationsCopy = adoptNS([item mutableCopy]);
-        // First, let the platform write all the default object types it can recognize, such as NSString and NSURL.
-        for (NSString *typeIdentifier in [itemRepresentationsCopy allKeys]) {
-            id representingObject = [itemRepresentationsCopy objectForKey:typeIdentifier];
+        // First, register all platform objects, prioritizing objects at the beginning of the array.
+        for (id representingObject in data.representingObjects) {
             if (![representingObject conformsToProtocol:@protocol(UIItemProviderWriting)])
                 continue;
 
-            id <UIItemProviderWriting> objectToWrite = (id <UIItemProviderWriting>)representingObject;
-            if (![objectToWrite.writableTypeIdentifiersForItemProvider containsObject:typeIdentifier])
-                continue;
-
-            [itemRepresentationsCopy removeObjectForKey:typeIdentifier];
-            [itemProvider registerObject:objectToWrite options:nil];
+            [itemProvider registerObject:(id <UIItemProviderWriting>)representingObject options:nil];
         }
 
-        // Secondly, WebKit uses some custom type representations and/or type identifiers, so we need to write these as well.
-        for (NSString *typeIdentifier in itemRepresentationsCopy.get()) {
+        // Next, register other custom data representations for type identifiers.
+        NSDictionary <NSString *, NSData *> *additionalData = data.additionalData;
+        for (NSString *typeIdentifier in additionalData) {
+            if (![additionalData[typeIdentifier] isKindOfClass:[NSData class]])
+                continue;
+
             [itemProvider registerDataRepresentationForTypeIdentifier:typeIdentifier options:nil loadHandler:^NSProgress *(UIItemProviderDataLoadCompletionBlock completionBlock)
             {
-                completionBlock([itemRepresentationsCopy objectForKey:typeIdentifier], nil);
+                completionBlock(additionalData[typeIdentifier], nil);
                 return [NSProgress discreteProgressWithTotalUnitCount:100];
             }];
         }
         [providers addObject:itemProvider.get()];
     }
-    _changeCount++;
-    _itemProviders = providers;
+
+    self.itemProviders = providers;
 }
 
 - (NSArray *)dataForPasteboardType:(NSString *)pasteboardType inItemSet:(NSIndexSet *)itemSet
index 8098eef..966c623 100644 (file)
@@ -1,3 +1,19 @@
+2017-04-03  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        Data interaction should register type identifiers in order of priority
+        https://bugs.webkit.org/show_bug.cgi?id=170428
+        <rdar://problem/30633296>
+
+        Reviewed by Tim Horton.
+
+        Serialize PasteboardWebContent.dataInAttributedStringFormat when sending over IPC. Refer to WebCore ChangeLog
+        for more details.
+
+        * Shared/WebCoreArgumentCoders.cpp:
+        (IPC::ArgumentCoder<PasteboardWebContent>::encode):
+        (IPC::ArgumentCoder<PasteboardWebContent>::decode):
+        * UIProcess/API/Cocoa/WKWebView.mm:
+
 2017-04-03  Joseph Pecoraro  <pecoraro@apple.com>
 
         Remove no longer needed forward declarations
index a192955..71e7e7d 100644 (file)
@@ -1404,6 +1404,7 @@ void ArgumentCoder<PasteboardWebContent>::encode(Encoder& encoder, const Pastebo
     encodeSharedBuffer(encoder, content.dataInWebArchiveFormat.get());
     encodeSharedBuffer(encoder, content.dataInRTFDFormat.get());
     encodeSharedBuffer(encoder, content.dataInRTFFormat.get());
+    encodeSharedBuffer(encoder, content.dataInAttributedStringFormat.get());
 
     encoder << content.clientTypes;
     encoder << static_cast<uint64_t>(content.clientData.size());
@@ -1423,6 +1424,8 @@ bool ArgumentCoder<PasteboardWebContent>::decode(Decoder& decoder, PasteboardWeb
         return false;
     if (!decodeSharedBuffer(decoder, content.dataInRTFFormat))
         return false;
+    if (!decodeSharedBuffer(decoder, content.dataInAttributedStringFormat))
+        return false;
     if (!decoder.decode(content.clientTypes))
         return false;
     uint64_t clientDataSize;
index c842d8e..11877cb 100644 (file)
@@ -5378,6 +5378,8 @@ static WebCore::UserInterfaceLayoutDirection toUserInterfaceLayoutDirection(UISe
     WebKit::ViewSnapshotStore::singleton().setDisableSnapshotVolatilityForTesting(true);
 }
 
+#if PLATFORM(IOS)
+
 - (void)_simulateDataInteractionEntered:(id)info
 {
 #if ENABLE(DATA_INTERACTION)
@@ -5436,6 +5438,8 @@ static WebCore::UserInterfaceLayoutDirection toUserInterfaceLayoutDirection(UISe
 #endif
 }
 
+#endif // PLATFORM(IOS)
+
 @end
 
 
index 1d18f69..041a6cd 100644 (file)
@@ -1,3 +1,24 @@
+2017-04-03  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        Data interaction should register type identifiers in order of priority
+        https://bugs.webkit.org/show_bug.cgi?id=170428
+        <rdar://problem/30633296>
+
+        Reviewed by Tim Horton.
+
+        Augments existing unit tests to check for the existence and priority of type identifiers in the UIItemProviders
+        created upon starting data interaction. Also fixes a race condition in one of the unit tests and adds a new unit
+        test for data interaction from a textarea to an input.
+
+        * TestWebKitAPI/Tests/ios/DataInteractionTests.mm:
+        (checkTypeIdentifierPrecedesOtherTypeIdentifier):
+        (TestWebKitAPI::TEST):
+        * TestWebKitAPI/ios/DataInteractionSimulator.h:
+        * TestWebKitAPI/ios/DataInteractionSimulator.mm:
+        (-[DataInteractionSimulator _advanceProgress]):
+        (-[DataInteractionSimulator sourceItemProviders]):
+        (-[DataInteractionSimulator _webView:showCustomSheetForElement:]):
+
 2017-04-03  Carlos Alberto Lopez Perez  <clopez@igalia.com>
 
         [GTK][JHBuild] Update mesa repository url for tarballs
index 88b5cad..8f3bcd2 100644 (file)
@@ -62,6 +62,14 @@ static void checkSelectionRectsWithLogging(NSArray *expected, NSArray *observed)
     EXPECT_TRUE([expected isEqualToArray:observed]);
 }
 
+static void checkTypeIdentifierPrecedesOtherTypeIdentifier(DataInteractionSimulator *simulator, NSString *firstType, NSString *secondType)
+{
+    NSArray *registeredTypes = [simulator.sourceItemProviders.firstObject registeredTypeIdentifiers];
+    EXPECT_TRUE([registeredTypes containsObject:firstType]);
+    EXPECT_TRUE([registeredTypes containsObject:secondType]);
+    EXPECT_TRUE([registeredTypes indexOfObject:firstType] < [registeredTypes indexOfObject:secondType]);
+}
+
 namespace TestWebKitAPI {
 
 TEST(DataInteractionTests, ImageToContentEditable)
@@ -79,6 +87,7 @@ TEST(DataInteractionTests, ImageToContentEditable)
     EXPECT_TRUE([observedEventNames containsObject:DataInteractionOverEventName]);
     EXPECT_TRUE([observedEventNames containsObject:DataInteractionPerformOperationEventName]);
     checkSelectionRectsWithLogging(@[ makeCGRectValue(1, 201, 215, 174) ], [dataInteractionSimulator finalSelectionRects]);
+    checkTypeIdentifierPrecedesOtherTypeIdentifier(dataInteractionSimulator.get(), (NSString *)kUTTypePNG, (NSString *)kUTTypeFileURL);
 }
 
 TEST(DataInteractionTests, ImageToTextarea)
@@ -99,6 +108,7 @@ TEST(DataInteractionTests, ImageToTextarea)
 
     NSArray *expectedSelectionRects = [NSArray arrayWithObjects:makeCGRectValue(6, 203, 188, 14), makeCGRectValue(6, 217, 188, 14), makeCGRectValue(6, 231, 66, 14), nil];
     checkSelectionRectsWithLogging(expectedSelectionRects, [dataInteractionSimulator finalSelectionRects]);
+    checkTypeIdentifierPrecedesOtherTypeIdentifier(dataInteractionSimulator.get(), (NSString *)kUTTypePNG, (NSString *)kUTTypeFileURL);
 }
 
 TEST(DataInteractionTests, ContentEditableToContentEditable)
@@ -117,6 +127,7 @@ TEST(DataInteractionTests, ContentEditableToContentEditable)
     EXPECT_TRUE([observedEventNames containsObject:DataInteractionOverEventName]);
     EXPECT_TRUE([observedEventNames containsObject:DataInteractionPerformOperationEventName]);
     checkSelectionRectsWithLogging(@[ makeCGRectValue(1, 201, 961, 227) ], [dataInteractionSimulator finalSelectionRects]);
+    checkTypeIdentifierPrecedesOtherTypeIdentifier(dataInteractionSimulator.get(), (NSString *)kUTTypeRTFD, (NSString *)kUTTypeUTF8PlainText);
 }
 
 TEST(DataInteractionTests, ContentEditableToTextarea)
@@ -135,6 +146,20 @@ TEST(DataInteractionTests, ContentEditableToTextarea)
     EXPECT_TRUE([observedEventNames containsObject:DataInteractionOverEventName]);
     EXPECT_TRUE([observedEventNames containsObject:DataInteractionPerformOperationEventName]);
     checkSelectionRectsWithLogging(@[ makeCGRectValue(6, 203, 990, 232) ], [dataInteractionSimulator finalSelectionRects]);
+    checkTypeIdentifierPrecedesOtherTypeIdentifier(dataInteractionSimulator.get(), (NSString *)kUTTypeRTFD, (NSString *)kUTTypeUTF8PlainText);
+}
+
+TEST(DataInteractionTests, TextAreaToInput)
+{
+    RetainPtr<TestWKWebView> webView = adoptNS([[TestWKWebView alloc] initWithFrame:CGRectMake(0, 0, 320, 500)]);
+    [webView synchronouslyLoadTestPageNamed:@"textarea-to-input"];
+
+    RetainPtr<DataInteractionSimulator> dataInteractionSimulator = adoptNS([[DataInteractionSimulator alloc] initWithWebView:webView.get()]);
+    [dataInteractionSimulator runFrom:CGPointMake(100, 50) to:CGPointMake(100, 300)];
+
+    EXPECT_EQ([webView stringByEvaluatingJavaScript:@"source.value"].length, 0UL);
+    EXPECT_WK_STREQ("Hello world", [webView editorValue].UTF8String);
+    checkSelectionRectsWithLogging(@[ makeCGRectValue(101, 241, 990, 232) ], [dataInteractionSimulator finalSelectionRects]);
 }
 
 TEST(DataInteractionTests, LinkToInput)
@@ -152,6 +177,7 @@ TEST(DataInteractionTests, LinkToInput)
     EXPECT_TRUE([observedEventNames containsObject:DataInteractionOverEventName]);
     EXPECT_TRUE([observedEventNames containsObject:DataInteractionPerformOperationEventName]);
     checkSelectionRectsWithLogging(@[ makeCGRectValue(101, 273, 2057, 232) ], [dataInteractionSimulator finalSelectionRects]);
+    checkTypeIdentifierPrecedesOtherTypeIdentifier(dataInteractionSimulator.get(), (NSString *)kUTTypeURL, (NSString *)kUTTypeUTF8PlainText);
 }
 
 TEST(DataInteractionTests, BackgroundImageLinkToInput)
@@ -169,6 +195,7 @@ TEST(DataInteractionTests, BackgroundImageLinkToInput)
     EXPECT_TRUE([observedEventNames containsObject:DataInteractionOverEventName]);
     EXPECT_TRUE([observedEventNames containsObject:DataInteractionPerformOperationEventName]);
     checkSelectionRectsWithLogging(@[ makeCGRectValue(101, 241, 2057, 232) ], [dataInteractionSimulator finalSelectionRects]);
+    checkTypeIdentifierPrecedesOtherTypeIdentifier(dataInteractionSimulator.get(), (NSString *)kUTTypeURL, (NSString *)kUTTypeUTF8PlainText);
 }
 
 TEST(DataInteractionTests, CanPreventStart)
@@ -298,6 +325,7 @@ TEST(DataInteractionTests, LargeImageToTargetDiv)
     RetainPtr<DataInteractionSimulator> dataInteractionSimulator = adoptNS([[DataInteractionSimulator alloc] initWithWebView:webView.get()]);
     [dataInteractionSimulator runFrom:CGPointMake(200, 400) to:CGPointMake(200, 150)];
     EXPECT_WK_STREQ("PASS", [webView stringByEvaluatingJavaScript:@"target.textContent"].UTF8String);
+    checkTypeIdentifierPrecedesOtherTypeIdentifier(dataInteractionSimulator.get(), (NSString *)kUTTypePNG, (NSString *)kUTTypeFileURL);
 }
 
 TEST(DataInteractionTests, LinkWithEmptyHREF)
index fa110a6..336bdfd 100644 (file)
@@ -55,6 +55,7 @@ typedef NS_ENUM(NSInteger, DataInteractionPhase) {
     RetainPtr<MockDataOperationSession> _dataOperationSession;
     RetainPtr<NSMutableArray> _observedEventNames;
     RetainPtr<UIItemProvider> _externalItemProvider;
+    RetainPtr<NSArray *> _sourceItemProviders;
     RetainPtr<NSArray *> _finalSelectionRects;
     CGPoint _startLocation;
     CGPoint _endLocation;
@@ -71,6 +72,7 @@ typedef NS_ENUM(NSInteger, DataInteractionPhase) {
 @property (nonatomic) BlockPtr<BOOL(_WKActivatedElementInfo *)> showCustomActionSheetBlock;
 @property (nonatomic) BlockPtr<NSArray *(NSArray *)> convertItemProvidersBlock;
 @property (nonatomic, strong) UIItemProvider *externalItemProvider;
+@property (nonatomic, readonly) NSArray *sourceItemProviders;
 @property (nonatomic, readonly) NSArray *observedEventNames;
 @property (nonatomic, readonly) NSArray *finalSelectionRects;
 @property (nonatomic, readonly) DataInteractionPhase phase;
index 13f4a0b..ae16c9d 100644 (file)
@@ -202,10 +202,14 @@ static NSArray *dataInteractionEventNames()
 
         _dataOperationSession = adoptNS([[MockDataOperationSession alloc] initWithProvider:itemProviders.firstObject location:self._currentLocation window:[_webView window]]);
         [_dataInteractionSession setItems:items];
-        if (!self.showCustomActionSheetBlock) {
-            [_webView _simulateWillBeginDataInteractionWithSession:_dataInteractionSession.get()];
-            _phase = DataInteractionBegan;
+        _sourceItemProviders = itemProviders;
+        if (self.showCustomActionSheetBlock) {
+            // Defer progress until the custom action sheet is dismissed.
+            return;
         }
+
+        [_webView _simulateWillBeginDataInteractionWithSession:_dataInteractionSession.get()];
+        _phase = DataInteractionBegan;
         break;
     }
     case DataInteractionBegan:
@@ -235,6 +239,11 @@ static NSArray *dataInteractionEventNames()
     [self performSelector:@selector(_advanceProgress) withObject:nil afterDelay:progressTimeStep];
 }
 
+- (NSArray *)sourceItemProviders
+{
+    return _sourceItemProviders.get();
+}
+
 - (UIItemProvider *)externalItemProvider
 {
     return _externalItemProvider.get();
@@ -274,6 +283,7 @@ static NSArray *dataInteractionEventNames()
         DataInteractionSimulator *weakSelf = strongSelf.get();
         [weakSelf->_webView _simulateWillBeginDataInteractionWithSession:weakSelf->_dataInteractionSession.get()];
         weakSelf->_phase = DataInteractionBegan;
+        [weakSelf _scheduleAdvanceProgress];
     });
 
     return self.showCustomActionSheetBlock(element);