getUserMedia video streams should follow device orientation
authorcommit-queue@webkit.org <commit-queue@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 28 Apr 2017 23:26:31 +0000 (23:26 +0000)
committercommit-queue@webkit.org <commit-queue@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 28 Apr 2017 23:26:31 +0000 (23:26 +0000)
https://bugs.webkit.org/show_bug.cgi?id=171284

Patch by Youenn Fablet <youenn@apple.com> on 2017-04-28
Reviewed by Eric Carlson.

Source/WebCore:

Only really affects AVVideoCaptureSource on iOS. Manually testing the following cases:
- Mac: no change of behavior
- iOS: page loaded in portrait, video is portrait. page loaded in landscape, video is landscape
- iOS: changing the device orientation makes the video samples to be moved to landscape/portrait.
- Same tests as above with peer connection.

Making Document having an OrientationNotifier that AVVideoCaptureSources get registered to.
Making AVVideoCaptureSource an OrientationNotifier::Observer so that it can changes width/height if needed.

Refactoring to specialize CaptureFactory in VideoCaptureFactory and AudioCaptureFactory.
Refactoring to return a CaptureSourceOrError instead of passing an out parameter plus returning a RefPtr.

* Modules/mediastream/UserMediaRequest.cpp:
(WebCore::UserMediaRequest::allow):
* WebCore.xcodeproj/project.pbxproj:
* dom/Document.cpp:
(WebCore::Document::orientationChanged):
* dom/Document.h:
* page/Frame.cpp:
(WebCore::Frame::orientationChanged):
* platform/OrientationNotifer.h: Copied from Source/WebKit2/UIProcess/Cocoa/UserMediaCaptureManagerProxy.h.
(WebCore::OrientationNotifier::Observer::setNotifier):
(WebCore::OrientationNotifier::Observer::~Observer):
(WebCore::OrientationNotifier::orientationChanged):
(WebCore::OrientationNotifier::addObserver):
(WebCore::OrientationNotifier::removeObserver):
* platform/mediastream/MediaStreamPrivate.cpp:
(WebCore::MediaStreamPrivate::monitorOrientation):
* platform/mediastream/MediaStreamPrivate.h:
* platform/mediastream/RealtimeMediaSource.h:
(WebCore::CaptureSourceOrError::CaptureSourceOrError):
(WebCore::CaptureSourceOrError::operator bool):
(WebCore::CaptureSourceOrError::source):
* platform/mediastream/RealtimeMediaSourceCenter.cpp:
(WebCore::RealtimeMediaSourceCenter::setAudioFactory):
(WebCore::RealtimeMediaSourceCenter::unsetAudioFactory):
(WebCore::RealtimeMediaSourceCenter::setVideoFactory):
(WebCore::RealtimeMediaSourceCenter::unsetVideoFactory):
* platform/mediastream/RealtimeMediaSourceCenter.h:
(WebCore::RealtimeMediaSourceCenter::defaultAudioFactory):
(WebCore::RealtimeMediaSourceCenter::defaultVideoFactory):
(WebCore::RealtimeMediaSourceCenter::audioFactory):
(WebCore::RealtimeMediaSourceCenter::videoFactory):
* platform/mediastream/mac/AVAudioCaptureSource.h:
* platform/mediastream/mac/AVAudioCaptureSource.mm:
(WebCore::AVAudioCaptureSource::create):
(WebCore::AVAudioCaptureSource::factory):
* platform/mediastream/mac/AVVideoCaptureSource.h:
* platform/mediastream/mac/AVVideoCaptureSource.mm:
(WebCore::AVVideoCaptureSource::create):
(WebCore::AVVideoCaptureSource::factory):
(WebCore::AVVideoCaptureSource::monitorOrientation):
(WebCore::AVVideoCaptureSource::orientationChanged):
(WebCore::AVVideoCaptureSource::processNewFrame):
* platform/mediastream/mac/CoreAudioCaptureSource.cpp:
(WebCore::CoreAudioCaptureSource::create):
(WebCore::CoreAudioCaptureSource::factory):
* platform/mediastream/mac/CoreAudioCaptureSource.h:
* platform/mediastream/mac/MockRealtimeAudioSourceMac.mm:
(WebCore::MockRealtimeAudioSource::create):
* platform/mediastream/mac/MockRealtimeVideoSourceMac.h:
* platform/mediastream/mac/MockRealtimeVideoSourceMac.mm:
(WebCore::MockRealtimeVideoSource::create):
(WebCore::MockRealtimeVideoSourceMac::orientationChanged):
* platform/mediastream/mac/RealtimeMediaSourceCenterMac.cpp:
(WebCore::RealtimeMediaSourceCenterMac::createMediaStream):
(WebCore::RealtimeMediaSourceCenterMac::bestSourcesForTypeAndConstraints):
(WebCore::RealtimeMediaSourceCenterMac::defaultAudioFactory):
(WebCore::RealtimeMediaSourceCenterMac::defaultVideoFactory):
* platform/mediastream/mac/RealtimeMediaSourceCenterMac.h:
* platform/mediastream/openwebrtc/RealtimeMediaSourceCenterOwr.cpp:
(WebCore::RealtimeMediaSourceCenterOwr::createMediaStream):
* platform/mediastream/openwebrtc/RealtimeMediaSourceCenterOwr.h:
* platform/mock/MockRealtimeAudioSource.cpp:
(WebCore::MockRealtimeAudioSource::create):
(WebCore::MockRealtimeAudioSource::factory):
* platform/mock/MockRealtimeAudioSource.h:
* platform/mock/MockRealtimeMediaSourceCenter.cpp:
(WebCore::MockRealtimeMediaSourceCenter::validateRequestConstraints):
(WebCore::MockRealtimeMediaSourceCenter::createMediaStream):
(WebCore::MockRealtimeMediaSourceCenter::defaultAudioFactory):
(WebCore::MockRealtimeMediaSourceCenter::defaultVideoFactory):
* platform/mock/MockRealtimeMediaSourceCenter.h:
* platform/mock/MockRealtimeVideoSource.cpp:
(WebCore::MockRealtimeVideoSource::create):
(WebCore::MockRealtimeVideoSource::factory):
* platform/mock/MockRealtimeVideoSource.h:

Source/WebKit2:

Refactoring to specialize CaptureFactory in VideoCaptureFactory and AudioCaptureFactory.
Refactoring to return a CaptureSourceOrError instead of passing an out parameter in addition to returning a RefPtr.

* Shared/WebCoreArgumentCoders.h:
* UIProcess/Cocoa/UserMediaCaptureManagerProxy.cpp:
(WebKit::UserMediaCaptureManagerProxy::createMediaSourceForCaptureDeviceWithConstraints):
* UIProcess/Cocoa/UserMediaCaptureManagerProxy.h:
* UIProcess/Cocoa/UserMediaCaptureManagerProxy.messages.in:
* WebProcess/cocoa/UserMediaCaptureManager.cpp:
(WebKit::UserMediaCaptureManager::createCaptureSource):
(WebKit::UserMediaCaptureManager::createMediaSourceForCaptureDeviceWithConstraints): Deleted.
* WebProcess/cocoa/UserMediaCaptureManager.h:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215955 268f45cc-cd09-0410-ab3c-d52691b4dbfc

38 files changed:
Source/WebCore/ChangeLog
Source/WebCore/Modules/mediastream/UserMediaRequest.cpp
Source/WebCore/WebCore.xcodeproj/project.pbxproj
Source/WebCore/dom/Document.cpp
Source/WebCore/dom/Document.h
Source/WebCore/page/Frame.cpp
Source/WebCore/platform/OrientationNotifer.h [new file with mode: 0644]
Source/WebCore/platform/mediastream/MediaStreamPrivate.cpp
Source/WebCore/platform/mediastream/MediaStreamPrivate.h
Source/WebCore/platform/mediastream/RealtimeMediaSource.h
Source/WebCore/platform/mediastream/RealtimeMediaSourceCenter.cpp
Source/WebCore/platform/mediastream/RealtimeMediaSourceCenter.h
Source/WebCore/platform/mediastream/mac/AVAudioCaptureSource.h
Source/WebCore/platform/mediastream/mac/AVAudioCaptureSource.mm
Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.h
Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.mm
Source/WebCore/platform/mediastream/mac/CoreAudioCaptureSource.cpp
Source/WebCore/platform/mediastream/mac/CoreAudioCaptureSource.h
Source/WebCore/platform/mediastream/mac/MockRealtimeAudioSourceMac.mm
Source/WebCore/platform/mediastream/mac/MockRealtimeVideoSourceMac.h
Source/WebCore/platform/mediastream/mac/MockRealtimeVideoSourceMac.mm
Source/WebCore/platform/mediastream/mac/RealtimeMediaSourceCenterMac.cpp
Source/WebCore/platform/mediastream/mac/RealtimeMediaSourceCenterMac.h
Source/WebCore/platform/mediastream/openwebrtc/RealtimeMediaSourceCenterOwr.cpp
Source/WebCore/platform/mock/MockRealtimeAudioSource.cpp
Source/WebCore/platform/mock/MockRealtimeAudioSource.h
Source/WebCore/platform/mock/MockRealtimeMediaSourceCenter.cpp
Source/WebCore/platform/mock/MockRealtimeMediaSourceCenter.h
Source/WebCore/platform/mock/MockRealtimeVideoSource.cpp
Source/WebCore/platform/mock/MockRealtimeVideoSource.h
Source/WebKit2/ChangeLog
Source/WebKit2/Shared/WebCoreArgumentCoders.h
Source/WebKit2/UIProcess/Cocoa/UserMediaCaptureManagerProxy.cpp
Source/WebKit2/UIProcess/Cocoa/UserMediaCaptureManagerProxy.h
Source/WebKit2/UIProcess/Cocoa/UserMediaCaptureManagerProxy.messages.in
Source/WebKit2/WebProcess/WebPage/WebBackForwardListProxy.cpp
Source/WebKit2/WebProcess/cocoa/UserMediaCaptureManager.cpp
Source/WebKit2/WebProcess/cocoa/UserMediaCaptureManager.h

index e2db61b..196ccde 100644 (file)
@@ -1,3 +1,98 @@
+2017-04-28  Youenn Fablet  <youenn@apple.com>
+
+        getUserMedia video streams should follow device orientation
+        https://bugs.webkit.org/show_bug.cgi?id=171284
+
+        Reviewed by Eric Carlson.
+
+        Only really affects AVVideoCaptureSource on iOS. Manually testing the following cases:
+        - Mac: no change of behavior
+        - iOS: page loaded in portrait, video is portrait. page loaded in landscape, video is landscape
+        - iOS: changing the device orientation makes the video samples to be moved to landscape/portrait.
+        - Same tests as above with peer connection.
+
+        Making Document having an OrientationNotifier that AVVideoCaptureSources get registered to.
+        Making AVVideoCaptureSource an OrientationNotifier::Observer so that it can changes width/height if needed.
+
+        Refactoring to specialize CaptureFactory in VideoCaptureFactory and AudioCaptureFactory.
+        Refactoring to return a CaptureSourceOrError instead of passing an out parameter plus returning a RefPtr.
+
+        * Modules/mediastream/UserMediaRequest.cpp:
+        (WebCore::UserMediaRequest::allow):
+        * WebCore.xcodeproj/project.pbxproj:
+        * dom/Document.cpp:
+        (WebCore::Document::orientationChanged):
+        * dom/Document.h:
+        * page/Frame.cpp:
+        (WebCore::Frame::orientationChanged):
+        * platform/OrientationNotifer.h: Copied from Source/WebKit2/UIProcess/Cocoa/UserMediaCaptureManagerProxy.h.
+        (WebCore::OrientationNotifier::Observer::setNotifier):
+        (WebCore::OrientationNotifier::Observer::~Observer):
+        (WebCore::OrientationNotifier::orientationChanged):
+        (WebCore::OrientationNotifier::addObserver):
+        (WebCore::OrientationNotifier::removeObserver):
+        * platform/mediastream/MediaStreamPrivate.cpp:
+        (WebCore::MediaStreamPrivate::monitorOrientation):
+        * platform/mediastream/MediaStreamPrivate.h:
+        * platform/mediastream/RealtimeMediaSource.h:
+        (WebCore::CaptureSourceOrError::CaptureSourceOrError):
+        (WebCore::CaptureSourceOrError::operator bool):
+        (WebCore::CaptureSourceOrError::source):
+        * platform/mediastream/RealtimeMediaSourceCenter.cpp:
+        (WebCore::RealtimeMediaSourceCenter::setAudioFactory):
+        (WebCore::RealtimeMediaSourceCenter::unsetAudioFactory):
+        (WebCore::RealtimeMediaSourceCenter::setVideoFactory):
+        (WebCore::RealtimeMediaSourceCenter::unsetVideoFactory):
+        * platform/mediastream/RealtimeMediaSourceCenter.h:
+        (WebCore::RealtimeMediaSourceCenter::defaultAudioFactory):
+        (WebCore::RealtimeMediaSourceCenter::defaultVideoFactory):
+        (WebCore::RealtimeMediaSourceCenter::audioFactory):
+        (WebCore::RealtimeMediaSourceCenter::videoFactory):
+        * platform/mediastream/mac/AVAudioCaptureSource.h:
+        * platform/mediastream/mac/AVAudioCaptureSource.mm:
+        (WebCore::AVAudioCaptureSource::create):
+        (WebCore::AVAudioCaptureSource::factory):
+        * platform/mediastream/mac/AVVideoCaptureSource.h:
+        * platform/mediastream/mac/AVVideoCaptureSource.mm:
+        (WebCore::AVVideoCaptureSource::create):
+        (WebCore::AVVideoCaptureSource::factory):
+        (WebCore::AVVideoCaptureSource::monitorOrientation):
+        (WebCore::AVVideoCaptureSource::orientationChanged):
+        (WebCore::AVVideoCaptureSource::processNewFrame):
+        * platform/mediastream/mac/CoreAudioCaptureSource.cpp:
+        (WebCore::CoreAudioCaptureSource::create):
+        (WebCore::CoreAudioCaptureSource::factory):
+        * platform/mediastream/mac/CoreAudioCaptureSource.h:
+        * platform/mediastream/mac/MockRealtimeAudioSourceMac.mm:
+        (WebCore::MockRealtimeAudioSource::create):
+        * platform/mediastream/mac/MockRealtimeVideoSourceMac.h:
+        * platform/mediastream/mac/MockRealtimeVideoSourceMac.mm:
+        (WebCore::MockRealtimeVideoSource::create):
+        (WebCore::MockRealtimeVideoSourceMac::orientationChanged):
+        * platform/mediastream/mac/RealtimeMediaSourceCenterMac.cpp:
+        (WebCore::RealtimeMediaSourceCenterMac::createMediaStream):
+        (WebCore::RealtimeMediaSourceCenterMac::bestSourcesForTypeAndConstraints):
+        (WebCore::RealtimeMediaSourceCenterMac::defaultAudioFactory):
+        (WebCore::RealtimeMediaSourceCenterMac::defaultVideoFactory):
+        * platform/mediastream/mac/RealtimeMediaSourceCenterMac.h:
+        * platform/mediastream/openwebrtc/RealtimeMediaSourceCenterOwr.cpp:
+        (WebCore::RealtimeMediaSourceCenterOwr::createMediaStream):
+        * platform/mediastream/openwebrtc/RealtimeMediaSourceCenterOwr.h:
+        * platform/mock/MockRealtimeAudioSource.cpp:
+        (WebCore::MockRealtimeAudioSource::create):
+        (WebCore::MockRealtimeAudioSource::factory):
+        * platform/mock/MockRealtimeAudioSource.h:
+        * platform/mock/MockRealtimeMediaSourceCenter.cpp:
+        (WebCore::MockRealtimeMediaSourceCenter::validateRequestConstraints):
+        (WebCore::MockRealtimeMediaSourceCenter::createMediaStream):
+        (WebCore::MockRealtimeMediaSourceCenter::defaultAudioFactory):
+        (WebCore::MockRealtimeMediaSourceCenter::defaultVideoFactory):
+        * platform/mock/MockRealtimeMediaSourceCenter.h:
+        * platform/mock/MockRealtimeVideoSource.cpp:
+        (WebCore::MockRealtimeVideoSource::create):
+        (WebCore::MockRealtimeVideoSource::factory):
+        * platform/mock/MockRealtimeVideoSource.h:
+
 2017-04-28  Said Abou-Hallawa  <sabouhallawa@apple.com>
 
         Unify how BitmapImage handles the availability of a decoded for large and animated images
index 6eb4194..4d2bfbc 100644 (file)
@@ -167,6 +167,7 @@ void UserMediaRequest::allow(const String& audioDeviceUID, const String& videoDe
             deny(MediaAccessDenialReason::HardwareError, emptyString());
             return;
         }
+        privateStream->monitorOrientation(downcast<Document>(m_scriptExecutionContext)->orientationNotifier());
 
         auto stream = MediaStream::create(*m_scriptExecutionContext, privateStream.releaseNonNull());
         if (stream->getTracks().isEmpty()) {
index a1f96c5..d93c40d 100644 (file)
                41815C1F138319830057AAA4 /* WebCoreTestSupport.h in Headers */ = {isa = PBXBuildFile; fileRef = 41815C1D138319830057AAA4 /* WebCoreTestSupport.h */; settings = {ATTRIBUTES = (Private, ); }; };
                418205471E53E98C00D62207 /* RTCController.h in Headers */ = {isa = PBXBuildFile; fileRef = 418205451E53C8CD00D62207 /* RTCController.h */; settings = {ATTRIBUTES = (Private, ); }; };
                418205491E53EAB000D62207 /* RTCController.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 418205481E53EAAD00D62207 /* RTCController.cpp */; };
+               4184F5161EAF05A800F18BF0 /* OrientationNotifer.h in Headers */ = {isa = PBXBuildFile; fileRef = 4184F5151EAF059800F18BF0 /* OrientationNotifer.h */; settings = {ATTRIBUTES = (Private, ); }; };
                41885B9311B6FDA6003383BB /* FormSubmission.h in Headers */ = {isa = PBXBuildFile; fileRef = 41885B9111B6FDA6003383BB /* FormSubmission.h */; settings = {ATTRIBUTES = (Private, ); }; };
                41885B9411B6FDA6003383BB /* FormSubmission.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 41885B9211B6FDA6003383BB /* FormSubmission.cpp */; };
                418A06D0133C04D500CD379C /* EventDispatcher.h in Headers */ = {isa = PBXBuildFile; fileRef = 418A06CE133C04D500CD379C /* EventDispatcher.h */; };
                4181707D1386EDF20057AAA4 /* WebCoreTestSupport.xcconfig */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.xcconfig; path = WebCoreTestSupport.xcconfig; sourceTree = "<group>"; };
                418205451E53C8CD00D62207 /* RTCController.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = RTCController.h; sourceTree = "<group>"; };
                418205481E53EAAD00D62207 /* RTCController.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = RTCController.cpp; sourceTree = "<group>"; };
+               4184F5151EAF059800F18BF0 /* OrientationNotifer.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = OrientationNotifer.h; sourceTree = "<group>"; };
                41885B9111B6FDA6003383BB /* FormSubmission.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = FormSubmission.h; sourceTree = "<group>"; };
                41885B9211B6FDA6003383BB /* FormSubmission.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = FormSubmission.cpp; sourceTree = "<group>"; };
                418A06CE133C04D500CD379C /* EventDispatcher.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = EventDispatcher.h; sourceTree = "<group>"; };
                                BC772C4D0C4EB3040083285F /* MIMETypeRegistry.h */,
                                E1513D501677F08800149FCB /* NotImplemented.cpp */,
                                98EB1F941313FE0500D0E1EA /* NotImplemented.h */,
+                               4184F5151EAF059800F18BF0 /* OrientationNotifer.h */,
                                4B2708C50AF19EE40065127F /* Pasteboard.h */,
                                C5F765B414E1D414006C899B /* PasteboardStrategy.h */,
                                1AF5E4D21E56735A004A1F01 /* PasteboardWriterData.cpp */,
                                081093DC1255F0E700ED9D29 /* SVGTextLayoutAttributesBuilder.h in Headers */,
                                081668DA125603D5006F25DE /* SVGTextLayoutEngine.h in Headers */,
                                080E49261255F3BD00EFCA27 /* SVGTextLayoutEngineBaseline.h in Headers */,
+                               4184F5161EAF05A800F18BF0 /* OrientationNotifer.h in Headers */,
                                080E49281255F3BD00EFCA27 /* SVGTextLayoutEngineSpacing.h in Headers */,
                                08F0BFC61255C53C00075185 /* SVGTextMetrics.h in Headers */,
                                7117445914BC34EC00EE5FC8 /* SVGTextMetricsBuilder.h in Headers */,
index a4ead2f..8eec854 100644 (file)
@@ -433,6 +433,17 @@ HashSet<Document*>& Document::allDocuments()
     return documents;
 }
 
+static inline int currentOrientation(Frame* frame)
+{
+#if ENABLE(ORIENTATION_EVENTS)
+    if (frame)
+        return frame->orientation();
+#else
+    UNUSED_PARAM(frame);
+#endif
+    return 0;
+}
+
 Document::Document(Frame* frame, const URL& url, unsigned documentClasses, unsigned constructionFlags)
     : ContainerNode(*this, CreateDocument)
     , TreeScope(*this)
@@ -483,6 +494,7 @@ Document::Document(Frame* frame, const URL& url, unsigned documentClasses, unsig
 #endif
     , m_isSynthesized(constructionFlags & Synthesized)
     , m_isNonRenderedPlaceholder(constructionFlags & NonRenderedPlaceholder)
+    , m_orientationNotifier(currentOrientation(frame))
 {
     allDocuments().add(this);
 
@@ -6954,4 +6966,10 @@ void Document::didRemoveInDocumentShadowRoot(ShadowRoot& shadowRoot)
     m_inDocumentShadowRoots.remove(&shadowRoot);
 }
 
+void Document::orientationChanged(int orientation)
+{
+    dispatchWindowEvent(Event::create(eventNames().orientationchangeEvent, false, false));
+    m_orientationNotifier.orientationChanged(orientation);
+}
+
 } // namespace WebCore
index fb2d97f..398e539 100644 (file)
@@ -38,6 +38,7 @@
 #include "FrameDestructionObserver.h"
 #include "MediaProducer.h"
 #include "MutationObserver.h"
+#include "OrientationNotifer.h"
 #include "PageVisibilityState.h"
 #include "PlatformEvent.h"
 #include "ReferrerPolicy.h"
@@ -1303,6 +1304,9 @@ public:
 
     ConstantPropertyMap& constantProperties() const { return *m_constantPropertyMap; }
 
+    void orientationChanged(int orientation);
+    OrientationNotifier& orientationNotifier() { return m_orientationNotifier; }
+
 protected:
     enum ConstructionFlags { Synthesized = 1, NonRenderedPlaceholder = 1 << 1 };
     Document(Frame*, const URL&, unsigned = DefaultDocumentClass, unsigned constructionFlags = 0);
@@ -1758,6 +1762,8 @@ private:
     bool m_didDispatchViewportPropertiesChanged { false };
 #endif
 
+    OrientationNotifier m_orientationNotifier;
+
     static bool hasEverCreatedAnAXObjectCache;
 };
 
index fdefcd2..c6c5b23 100644 (file)
@@ -296,9 +296,10 @@ void Frame::orientationChanged()
     for (Frame* frame = this; frame; frame = frame->tree().traverseNext())
         frames.append(*frame);
 
+    auto newOrientation = orientation();
     for (auto& frame : frames) {
         if (Document* document = frame->document())
-            document->dispatchWindowEvent(Event::create(eventNames().orientationchangeEvent, false, false));
+            document->orientationChanged(newOrientation);
     }
 }
 
diff --git a/Source/WebCore/platform/OrientationNotifer.h b/Source/WebCore/platform/OrientationNotifer.h
new file mode 100644 (file)
index 0000000..f04428f
--- /dev/null
@@ -0,0 +1,91 @@
+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+namespace WebCore {
+
+class OrientationNotifier {
+public:
+    explicit OrientationNotifier(int orientation) { m_orientation = orientation; }
+    ~OrientationNotifier();
+
+    void orientationChanged(int orientation);
+
+    class Observer {
+    public:
+        virtual ~Observer();
+        virtual void orientationChanged(int orientation) = 0;
+        void setNotifier(OrientationNotifier* notifier) { m_notifier = notifier; }
+
+    private:
+        OrientationNotifier* m_notifier { nullptr };
+    };
+
+    void addObserver(Observer&);
+    void removeObserver(Observer&);
+    int orientation() const { return m_orientation; }
+
+private:
+    Vector<std::reference_wrapper<Observer>> m_observers;
+    int m_orientation;
+};
+
+inline OrientationNotifier::~OrientationNotifier()
+{
+    for (Observer& observer : m_observers)
+        observer.setNotifier(nullptr);
+}
+
+inline OrientationNotifier::Observer::~Observer()
+{
+    if (m_notifier)
+        m_notifier->removeObserver(*this);
+}
+
+inline void OrientationNotifier::orientationChanged(int orientation)
+{
+    m_orientation = orientation;
+    for (Observer& observer : m_observers)
+        observer.orientationChanged(orientation);
+}
+
+inline void OrientationNotifier::addObserver(Observer& observer)
+{
+    m_observers.append(observer);
+    observer.setNotifier(this);
+}
+
+inline void OrientationNotifier::removeObserver(Observer& observer)
+{
+    m_observers.removeFirstMatching([&observer](auto item) {
+        if (&observer != &item.get())
+            return false;
+        observer.setNotifier(nullptr);
+        return true;
+    });
+}
+
+}
index c8fab68..cbb3e55 100644 (file)
@@ -298,6 +298,14 @@ void MediaStreamPrivate::scheduleDeferredTask(Function<void ()>&& function)
     });
 }
 
+void MediaStreamPrivate::monitorOrientation(OrientationNotifier& notifier)
+{
+    for (auto& track : m_trackSet.values()) {
+        if (track->source().isCaptureSource() && track->type() == RealtimeMediaSource::Type::Video)
+            track->source().monitorOrientation(notifier);
+    }
+}
+
 } // namespace WebCore
 
 #endif // ENABLE(MEDIA_STREAM)
index 4e980f9..ad1d501 100644 (file)
@@ -56,6 +56,7 @@
 namespace WebCore {
 
 class MediaStream;
+class OrientationNotifier;
 
 class MediaStreamPrivate : public MediaStreamTrackPrivate::Observer, public RefCounted<MediaStreamPrivate> {
 public:
@@ -106,6 +107,8 @@ public:
 
     WeakPtr<MediaStreamPrivate> createWeakPtr() { return m_weakPtrFactory.createWeakPtr(); }
 
+    void monitorOrientation(OrientationNotifier&);
+
 #if USE(GSTREAMER)
     void setVideoRenderer(OwrGstVideoRenderer* renderer, GstElement* sink) { m_gstVideoRenderer = renderer; m_gstVideoSinkElement = sink; }
     GRefPtr<GstElement> getVideoSinkElement() const { return m_gstVideoSinkElement; }
index 90e222b..2af611f 100644 (file)
@@ -31,8 +31,7 @@
  * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
-#ifndef RealtimeMediaSource_h
-#define RealtimeMediaSource_h
+#pragma once
 
 #if ENABLE(MEDIA_STREAM)
 
@@ -58,9 +57,12 @@ class AudioStreamDescription;
 class FloatRect;
 class GraphicsContext;
 class MediaStreamPrivate;
+class OrientationNotifier;
 class PlatformAudioData;
 class RealtimeMediaSourceSettings;
 
+struct CaptureSourceOrError;
+
 class WEBCORE_EXPORT RealtimeMediaSource : public RefCounted<RealtimeMediaSource> {
 public:
     class Observer {
@@ -83,13 +85,22 @@ public:
         virtual void audioSamplesAvailable(const MediaTime&, const PlatformAudioData&, const AudioStreamDescription&, size_t /*numberOfFrames*/) { }
     };
 
-    class CaptureFactory {
+    class AudioCaptureFactory {
+    public:
+        virtual ~AudioCaptureFactory() = default;
+        virtual CaptureSourceOrError createAudioCaptureSource(const String& audioDeviceID, const MediaConstraints*) = 0;
+        
+    protected:
+        AudioCaptureFactory() = default;
+    };
+
+    class VideoCaptureFactory {
     public:
-        virtual ~CaptureFactory() = default;
-        virtual RefPtr<RealtimeMediaSource> createMediaSourceForCaptureDeviceWithConstraints(const String& audioDeviceID, CaptureDevice::DeviceType, const MediaConstraints*, String&) = 0;
+        virtual ~VideoCaptureFactory() = default;
+        virtual CaptureSourceOrError createVideoCaptureSource(const String& videoDeviceID, const MediaConstraints*) = 0;
 
     protected:
-        CaptureFactory() = default;
+        VideoCaptureFactory() = default;
     };
 
     virtual ~RealtimeMediaSource() { }
@@ -135,6 +146,8 @@ public:
 
     virtual bool isCaptureSource() const { return false; }
 
+    virtual void monitorOrientation(OrientationNotifier&) { }
+    
     WEBCORE_EXPORT void addObserver(Observer&);
     WEBCORE_EXPORT void removeObserver(Observer&);
 
@@ -229,8 +242,18 @@ private:
     bool m_suppressNotifications { true };
 };
 
+struct CaptureSourceOrError {
+    CaptureSourceOrError() = default;
+    CaptureSourceOrError(Ref<RealtimeMediaSource>&& source) : captureSource(WTFMove(source)) { }
+    CaptureSourceOrError(String&& message) : errorMessage(WTFMove(message)) { }
+    
+    operator bool()  const { return !!captureSource; }
+    Ref<RealtimeMediaSource> source() { return captureSource.releaseNonNull(); }
+    
+    RefPtr<RealtimeMediaSource> captureSource;
+    String errorMessage;
+};
+
 } // namespace WebCore
 
 #endif // ENABLE(MEDIA_STREAM)
-
-#endif // RealtimeMediaSource_h
index 7bed99c..b9d3d47 100644 (file)
@@ -67,23 +67,23 @@ RealtimeMediaSourceCenter::~RealtimeMediaSourceCenter()
 {
 }
 
-void RealtimeMediaSourceCenter::setAudioFactory(RealtimeMediaSource::CaptureFactory& factory)
+void RealtimeMediaSourceCenter::setAudioFactory(RealtimeMediaSource::AudioCaptureFactory& factory)
 {
     m_audioFactory = &factory;
 }
 
-void RealtimeMediaSourceCenter::unsetAudioFactory(RealtimeMediaSource::CaptureFactory& factory)
+void RealtimeMediaSourceCenter::unsetAudioFactory(RealtimeMediaSource::AudioCaptureFactory& factory)
 {
     if (m_audioFactory == &factory)
         m_audioFactory = nullptr;
 }
 
-void RealtimeMediaSourceCenter::setVideoFactory(RealtimeMediaSource::CaptureFactory& factory)
+void RealtimeMediaSourceCenter::setVideoFactory(RealtimeMediaSource::VideoCaptureFactory& factory)
 {
     m_videoFactory = &factory;
 }
 
-void RealtimeMediaSourceCenter::unsetVideoFactory(RealtimeMediaSource::CaptureFactory& factory)
+void RealtimeMediaSourceCenter::unsetVideoFactory(RealtimeMediaSource::VideoCaptureFactory& factory)
 {
     if (m_videoFactory == &factory)
         m_videoFactory = nullptr;
index 1b07a88..7bc7d8e 100644 (file)
@@ -66,16 +66,16 @@ public:
     
     virtual const RealtimeMediaSourceSupportedConstraints& supportedConstraints() { return m_supportedConstraints; }
 
-    virtual RealtimeMediaSource::CaptureFactory* defaultAudioFactory() { return nullptr; }
-    virtual RealtimeMediaSource::CaptureFactory* defaultVideoFactory() { return nullptr; }
+    virtual RealtimeMediaSource::AudioCaptureFactory* defaultAudioFactory() { return nullptr; }
+    virtual RealtimeMediaSource::VideoCaptureFactory* defaultVideoFactory() { return nullptr; }
 
-    WEBCORE_EXPORT void setAudioFactory(RealtimeMediaSource::CaptureFactory&);
-    WEBCORE_EXPORT void unsetAudioFactory(RealtimeMediaSource::CaptureFactory&);
-    RealtimeMediaSource::CaptureFactory* audioFactory() const { return m_audioFactory; }
+    WEBCORE_EXPORT void setAudioFactory(RealtimeMediaSource::AudioCaptureFactory&);
+    WEBCORE_EXPORT void unsetAudioFactory(RealtimeMediaSource::AudioCaptureFactory&);
+    RealtimeMediaSource::AudioCaptureFactory* audioFactory() const { return m_audioFactory; }
 
-    WEBCORE_EXPORT void setVideoFactory(RealtimeMediaSource::CaptureFactory&);
-    WEBCORE_EXPORT void unsetVideoFactory(RealtimeMediaSource::CaptureFactory&);
-    RealtimeMediaSource::CaptureFactory* videoFactory() const { return m_videoFactory; }
+    WEBCORE_EXPORT void setVideoFactory(RealtimeMediaSource::VideoCaptureFactory&);
+    WEBCORE_EXPORT void unsetVideoFactory(RealtimeMediaSource::VideoCaptureFactory&);
+    RealtimeMediaSource::VideoCaptureFactory* videoFactory() const { return m_videoFactory; }
 
     virtual CaptureDeviceManager* defaultAudioCaptureDeviceManager() { return nullptr; }
     virtual CaptureDeviceManager* defaultVideoCaptureDeviceManager() { return nullptr; }
@@ -103,8 +103,8 @@ protected:
     static RealtimeMediaSourceCenter& platformCenter();
     RealtimeMediaSourceSupportedConstraints m_supportedConstraints;
 
-    RealtimeMediaSource::CaptureFactory* m_audioFactory { nullptr };
-    RealtimeMediaSource::CaptureFactory* m_videoFactory { nullptr };
+    RealtimeMediaSource::AudioCaptureFactory* m_audioFactory { nullptr };
+    RealtimeMediaSource::VideoCaptureFactory* m_videoFactory { nullptr };
 
     CaptureDeviceManager* m_audioCaptureDeviceManager { nullptr };
     CaptureDeviceManager* m_videoCaptureDeviceManager { nullptr };
index 6c8362f..7988c1e 100644 (file)
@@ -44,9 +44,9 @@ class WebAudioSourceProviderAVFObjC;
 class AVAudioCaptureSource : public AVMediaCaptureSource {
 public:
 
-    static RefPtr<AVMediaCaptureSource> create(AVCaptureDevice*, const AtomicString&, const MediaConstraints*, String&);
+    static CaptureSourceOrError create(AVCaptureDevice*, const AtomicString&, const MediaConstraints*);
 
-    WEBCORE_EXPORT static CaptureFactory& factory();
+    WEBCORE_EXPORT static AudioCaptureFactory& factory();
 
 private:
     AVAudioCaptureSource(AVCaptureDevice*, const AtomicString&);
index 6db1c75..6e59d2e 100644 (file)
@@ -77,30 +77,27 @@ SOFT_LINK_POINTER(AVFoundation, AVMediaTypeAudio, NSString *)
 
 namespace WebCore {
 
-class AVAudioCaptureSourceFactory : public RealtimeMediaSource::CaptureFactory {
+class AVAudioCaptureSourceFactory : public RealtimeMediaSource::AudioCaptureFactory {
 public:
-    RefPtr<RealtimeMediaSource> createMediaSourceForCaptureDeviceWithConstraints(const String& deviceID, CaptureDevice::DeviceType type, const MediaConstraints* constraints, String& invalidConstraint) final {
+    CaptureSourceOrError createAudioCaptureSource(const String& deviceID, const MediaConstraints* constraints) final {
         AVCaptureDeviceTypedef *device = [getAVCaptureDeviceClass() deviceWithUniqueID:deviceID];
-        ASSERT_UNUSED(type, !device || (device && type == CaptureDevice::DeviceType::Audio));
-        return device ? AVAudioCaptureSource::create(device, emptyString(), constraints, invalidConstraint) : nullptr;
+        return device ? AVAudioCaptureSource::create(device, emptyString(), constraints) : CaptureSourceOrError();
     }
 };
 
-RefPtr<AVMediaCaptureSource> AVAudioCaptureSource::create(AVCaptureDeviceTypedef* device, const AtomicString& id, const MediaConstraints* constraints, String& invalidConstraint)
+CaptureSourceOrError AVAudioCaptureSource::create(AVCaptureDeviceTypedef* device, const AtomicString& id, const MediaConstraints* constraints)
 {
-    auto source = adoptRef(new AVAudioCaptureSource(device, id));
+    auto source = adoptRef(*new AVAudioCaptureSource(device, id));
     if (constraints) {
         auto result = source->applyConstraints(*constraints);
-        if (result) {
-            invalidConstraint = result.value().first;
-            source = nullptr;
-        }
+        if (result)
+            return String(result.value().first);
     }
 
-    return source;
+    return CaptureSourceOrError(WTFMove(source));
 }
 
-RealtimeMediaSource::CaptureFactory& AVAudioCaptureSource::factory()
+RealtimeMediaSource::AudioCaptureFactory& AVAudioCaptureSource::factory()
 {
     static NeverDestroyed<AVAudioCaptureSourceFactory> factory;
     return factory.get();
index a821580..9a6c26c 100644 (file)
  * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
-#ifndef AVVideoCaptureSource_h
-#define AVVideoCaptureSource_h
+#pragma once
 
 #if ENABLE(MEDIA_STREAM) && USE(AVFOUNDATION)
 
 #include "AVMediaCaptureSource.h"
+#include "OrientationNotifer.h"
 
 OBJC_CLASS CALayer;
 OBJC_CLASS AVFrameRateRange;
@@ -43,11 +43,11 @@ class FloatRect;
 class GraphicsContext;
 class PixelBufferConformerCV;
 
-class AVVideoCaptureSource : public AVMediaCaptureSource {
+class AVVideoCaptureSource : public AVMediaCaptureSource, private OrientationNotifier::Observer {
 public:
-    static RefPtr<AVMediaCaptureSource> create(AVCaptureDevice*, const AtomicString&, const MediaConstraints*, String&);
+    static CaptureSourceOrError create(AVCaptureDevice*, const AtomicString&, const MediaConstraints*);
 
-    WEBCORE_EXPORT static CaptureFactory& factory();
+    WEBCORE_EXPORT static VideoCaptureFactory& factory();
 
     int32_t width() const { return m_width; }
     int32_t height() const { return m_height; }
@@ -66,12 +66,18 @@ private:
     bool applyFrameRate(double) final;
     bool setPreset(NSString*);
 
+    void monitorOrientation(OrientationNotifier&) final;
+    void computeSampleRotation();
+
     NSString *bestSessionPresetForVideoDimensions(std::optional<int> width, std::optional<int> height) const;
     bool supportsSizeAndFrameRate(std::optional<int> width, std::optional<int> height, std::optional<double>) final;
 
     void initializeCapabilities(RealtimeMediaSourceCapabilities&) final;
     void initializeSupportedConstraints(RealtimeMediaSourceSupportedConstraints&) final;
 
+    // OrientationNotifier::Observer API
+    void orientationChanged(int orientation) final;
+
     bool setFrameRateConstraint(double minFrameRate, double maxFrameRate);
 
     bool updateFramerate(CMSampleBufferRef);
@@ -90,10 +96,11 @@ private:
     Float64 m_frameRate { 0 };
     int32_t m_width { 0 };
     int32_t m_height { 0 };
+    int m_sensorOrientation { 0 };
+    int m_deviceOrientation { 0 };
+    MediaSample::VideoRotation m_sampleRotation { MediaSample::VideoRotation::None };
 };
 
 } // namespace WebCore
 
 #endif // ENABLE(MEDIA_STREAM)
-
-#endif // AVVideoCaptureSource_h
index 88131b8..3ee25c8 100644 (file)
@@ -85,6 +85,7 @@ SOFT_LINK_CLASS(AVFoundation, AVFrameRateRange)
 #define AVCaptureVideoPreviewLayer getAVCaptureVideoPreviewLayerClass()
 #define AVFrameRateRange getAVFrameRateRangeClass()
 
+SOFT_LINK_POINTER(AVFoundation, AVMediaTypeVideo, NSString *)
 SOFT_LINK_POINTER_OPTIONAL(AVFoundation, AVCaptureSessionPreset1280x720, NSString *)
 SOFT_LINK_POINTER_OPTIONAL(AVFoundation, AVCaptureSessionPreset960x540, NSString *)
 SOFT_LINK_POINTER_OPTIONAL(AVFoundation, AVCaptureSessionPreset640x480, NSString *)
@@ -109,30 +110,29 @@ const OSType videoCaptureFormat = kCVPixelFormatType_420YpCbCr8Planar;
 const OSType videoCaptureFormat = kCVPixelFormatType_420YpCbCr8BiPlanarFullRange;
 #endif
 
-class AVVideoCaptureSourceFactory : public RealtimeMediaSource::CaptureFactory {
+class AVVideoCaptureSourceFactory : public RealtimeMediaSource::VideoCaptureFactory {
 public:
-    RefPtr<RealtimeMediaSource> createMediaSourceForCaptureDeviceWithConstraints(const String& deviceID, CaptureDevice::DeviceType type, const MediaConstraints* constraints, String& invalidConstraint) final {
+    CaptureSourceOrError createVideoCaptureSource(const String& deviceID, const MediaConstraints* constraints) final {
         AVCaptureDeviceTypedef *device = [getAVCaptureDeviceClass() deviceWithUniqueID:deviceID];
-        ASSERT_UNUSED(type, !device || (type == CaptureDevice::DeviceType::Video));
-        return device ? AVVideoCaptureSource::create(device, emptyString(), constraints, invalidConstraint) : nullptr;
+        if (!device)
+            return { };
+        return AVVideoCaptureSource::create(device, emptyString(), constraints);
     }
 };
 
-RefPtr<AVMediaCaptureSource> AVVideoCaptureSource::create(AVCaptureDeviceTypedef* device, const AtomicString& id, const MediaConstraints* constraints, String& invalidConstraint)
+CaptureSourceOrError AVVideoCaptureSource::create(AVCaptureDeviceTypedef* device, const AtomicString& id, const MediaConstraints* constraints)
 {
-    auto source = adoptRef(new AVVideoCaptureSource(device, id));
+    auto source = adoptRef(*new AVVideoCaptureSource(device, id));
     if (constraints) {
         auto result = source->applyConstraints(*constraints);
-        if (result) {
-            invalidConstraint = result.value().first;
-            source = nullptr;
-        }
+        if (result)
+            return WTFMove(result.value().first);
     }
 
-    return source;
+    return CaptureSourceOrError(WTFMove(source));
 }
 
-RealtimeMediaSource::CaptureFactory& AVVideoCaptureSource::factory()
+RealtimeMediaSource::VideoCaptureFactory& AVVideoCaptureSource::factory()
 {
     static NeverDestroyed<AVVideoCaptureSourceFactory> factory;
     return factory.get();
@@ -346,6 +346,39 @@ void AVVideoCaptureSource::applySizeAndFrameRate(std::optional<int> width, std::
         applyFrameRate(frameRate.value());
 }
 
+static inline int sensorOrientation(AVCaptureVideoOrientation videoOrientation)
+{
+#if PLATFORM(IOS)
+    switch (videoOrientation) {
+    case AVCaptureVideoOrientationPortrait:
+        return 180;
+    case AVCaptureVideoOrientationPortraitUpsideDown:
+        return 0;
+    case AVCaptureVideoOrientationLandscapeRight:
+        return 90;
+    case AVCaptureVideoOrientationLandscapeLeft:
+        return -90;
+    }
+#else
+    switch (videoOrientation) {
+    case AVCaptureVideoOrientationPortrait:
+        return 0;
+    case AVCaptureVideoOrientationPortraitUpsideDown:
+        return 180;
+    case AVCaptureVideoOrientationLandscapeRight:
+        return 90;
+    case AVCaptureVideoOrientationLandscapeLeft:
+        return -90;
+    }
+#endif
+}
+
+static inline int sensorOrientationFromVideoOutput(AVCaptureVideoDataOutputType* videoOutput)
+{
+    AVCaptureConnectionType* connection = [videoOutput connectionWithMediaType: getAVMediaTypeVideo()];
+    return connection ? sensorOrientation([connection videoOrientation]) : 0;
+}
+
 void AVVideoCaptureSource::setupCaptureSession()
 {
     NSError *error = nil;
@@ -385,6 +418,8 @@ void AVVideoCaptureSource::setupCaptureSession()
     setPreset(m_pendingPreset.get());
 #endif
 
+    m_sensorOrientation = sensorOrientationFromVideoOutput(m_videoOutput.get());
+    computeSampleRotation();
 }
 
 void AVVideoCaptureSource::shutdownCaptureSession()
@@ -417,6 +452,46 @@ bool AVVideoCaptureSource::updateFramerate(CMSampleBufferRef sampleBuffer)
     return frameRate != m_frameRate;
 }
 
+void AVVideoCaptureSource::monitorOrientation(OrientationNotifier& notifier)
+{
+#if PLATFORM(IOS)
+    notifier.addObserver(*this);
+    orientationChanged(notifier.orientation());
+#else
+    UNUSED_PARAM(notifier);
+#endif
+}
+
+void AVVideoCaptureSource::orientationChanged(int orientation)
+{
+    ASSERT(orientation == 0 || orientation == 90 || orientation == -90 || orientation == 180);
+    m_deviceOrientation = orientation;
+    computeSampleRotation();
+}
+
+void AVVideoCaptureSource::computeSampleRotation()
+{
+    switch (m_sensorOrientation - m_deviceOrientation) {
+    case 0:
+        m_sampleRotation = MediaSample::VideoRotation::None;
+        break;
+    case 180:
+    case -180:
+        m_sampleRotation = MediaSample::VideoRotation::UpsideDown;
+        break;
+    case 90:
+        m_sampleRotation = MediaSample::VideoRotation::Left;
+        break;
+    case -90:
+    case -270:
+        m_sampleRotation = MediaSample::VideoRotation::Right;
+        break;
+    default:
+        ASSERT_NOT_REACHED();
+        m_sampleRotation = MediaSample::VideoRotation::None;
+    }
+}
+
 void AVVideoCaptureSource::processNewFrame(RetainPtr<CMSampleBufferRef> sampleBuffer, RetainPtr<AVCaptureConnectionType> connection)
 {
     // Ignore frames delivered when the session is not running, we want to hang onto the last image
@@ -432,52 +507,18 @@ void AVVideoCaptureSource::processNewFrame(RetainPtr<CMSampleBufferRef> sampleBu
     m_buffer = sampleBuffer;
     m_lastImage = nullptr;
 
-    MediaSample::VideoRotation rotation;
-#if PLATFORM(IOS)
-    switch ([connection videoOrientation]) {
-    case AVCaptureVideoOrientationPortrait:
-        rotation = MediaSample::VideoRotation::UpsideDown;
-        break;
-    case AVCaptureVideoOrientationPortraitUpsideDown:
-        rotation = MediaSample::VideoRotation::None;
-        break;
-    case AVCaptureVideoOrientationLandscapeRight:
-    case AVCaptureVideoOrientationLandscapeLeft:
-        rotation = MediaSample::VideoRotation::Right;
-        break;
-    }
-#else
-    switch ([connection videoOrientation]) {
-    case AVCaptureVideoOrientationPortrait:
-        rotation = MediaSample::VideoRotation::None;
-        break;
-    case AVCaptureVideoOrientationPortraitUpsideDown:
-        rotation = MediaSample::VideoRotation::UpsideDown;
-        break;
-    case AVCaptureVideoOrientationLandscapeRight:
-        rotation = MediaSample::VideoRotation::Right;
-        break;
-    case AVCaptureVideoOrientationLandscapeLeft:
-        rotation = MediaSample::VideoRotation::Left;
-        break;
-    }
-#endif
-
-    bool settingsChanged = false;
     CMVideoDimensions dimensions = CMVideoFormatDescriptionGetDimensions(formatDescription);
-    if (rotation == MediaSample::VideoRotation::Right || rotation == MediaSample::VideoRotation::Left)
+    if (m_sampleRotation == MediaSample::VideoRotation::Left || m_sampleRotation == MediaSample::VideoRotation::Right)
         std::swap(dimensions.width, dimensions.height);
 
     if (dimensions.width != m_width || dimensions.height != m_height) {
         m_width = dimensions.width;
         m_height = dimensions.height;
-        settingsChanged = true;
-    }
 
-    if (settingsChanged)
         settingsDidChange();
+    }
 
-    videoSampleAvailable(MediaSampleAVFObjC::create(m_buffer.get(), rotation, [connection isVideoMirrored]));
+    videoSampleAvailable(MediaSampleAVFObjC::create(m_buffer.get(), m_sampleRotation, [connection isVideoMirrored]));
 }
 
 void AVVideoCaptureSource::captureOutputDidOutputSampleBufferFromConnection(AVCaptureOutputType*, CMSampleBufferRef sampleBuffer, AVCaptureConnectionType* captureConnection)
index 8f24c63..3939ca6 100644 (file)
 
 namespace WebCore {
 
-class CoreAudioCaptureSourceFactory : public RealtimeMediaSource::CaptureFactory {
+class CoreAudioCaptureSourceFactory : public RealtimeMediaSource::AudioCaptureFactory {
 public:
-    RefPtr<RealtimeMediaSource> createMediaSourceForCaptureDeviceWithConstraints(const String& deviceID, CaptureDevice::DeviceType type, const MediaConstraints* constraints, String& invalidConstraint) final {
-        return CoreAudioCaptureSource::create(deviceID, type, constraints, invalidConstraint);
+    CaptureSourceOrError createAudioCaptureSource(const String& deviceID, const MediaConstraints* constraints) final {
+        return CoreAudioCaptureSource::create(deviceID, constraints);
     }
 };
 
 const UInt32 outputBus = 0;
 const UInt32 inputBus = 1;
 
-RefPtr<CoreAudioCaptureSource> CoreAudioCaptureSource::create(const String& deviceID, CaptureDevice::DeviceType type, const MediaConstraints* constraints, String& invalidConstraint)
+CaptureSourceOrError CoreAudioCaptureSource::create(const String& deviceID, const MediaConstraints* constraints)
 {
-    if (type != CaptureDevice::DeviceType::Audio)
-        return nullptr;
-
     String label;
     uint32_t persistentID = 0;
 #if PLATFORM(MAC)
     auto device = CoreAudioCaptureDeviceManager::singleton().coreAudioDeviceWithUID(deviceID);
     if (!device)
-        return nullptr;
+        return { };
 
     label = device->label();
     persistentID = device->deviceID();
 #endif
-    auto source = adoptRef(new CoreAudioCaptureSource(deviceID, label, persistentID));
+    auto source = adoptRef(*new CoreAudioCaptureSource(deviceID, label, persistentID));
 
     if (constraints) {
         auto result = source->applyConstraints(*constraints);
-        if (result) {
-            invalidConstraint = result.value().first;
-            return nullptr;
-        }
+        if (result)
+            return WTFMove(result.value().first);
     }
-
-    return source;
+    return CaptureSourceOrError(WTFMove(source));
 }
 
-RealtimeMediaSource::CaptureFactory& CoreAudioCaptureSource::factory()
+RealtimeMediaSource::AudioCaptureFactory& CoreAudioCaptureSource::factory()
 {
     static NeverDestroyed<CoreAudioCaptureSourceFactory> factory;
     return factory.get();
index 235d5a3..13f416c 100644 (file)
@@ -53,9 +53,9 @@ class CaptureDeviceInfo;
 class CoreAudioCaptureSource : public RealtimeMediaSource {
 public:
 
-    static RefPtr<CoreAudioCaptureSource> create(const String& deviceID, CaptureDevice::DeviceType, const MediaConstraints*, String&);
+    static CaptureSourceOrError create(const String& deviceID, const MediaConstraints*);
 
-    WEBCORE_EXPORT static CaptureFactory& factory();
+    WEBCORE_EXPORT static AudioCaptureFactory& factory();
 
     void addEchoCancellationSource(AudioSampleDataSource&);
     void removeEchoCancellationSource(AudioSampleDataSource&);
index ce4dabb..b0f283b 100644 (file)
@@ -84,13 +84,14 @@ static void addHum(float amplitude, float frequency, float sampleRate, uint64_t
     }
 }
 
-RefPtr<MockRealtimeAudioSource> MockRealtimeAudioSource::create(const String& name, const MediaConstraints* constraints)
+CaptureSourceOrError MockRealtimeAudioSource::create(const String& name, const MediaConstraints* constraints)
 {
-    auto source = adoptRef(new MockRealtimeAudioSourceMac(name));
+    auto source = adoptRef(*new MockRealtimeAudioSourceMac(name));
+    // FIXME: We should report error messages
     if (constraints && source->applyConstraints(*constraints))
-        source = nullptr;
+        return { };
 
-    return source;
+    return CaptureSourceOrError(WTFMove(source));
 }
 
 MockRealtimeAudioSourceMac::MockRealtimeAudioSourceMac(const String& name)
index 4d5143e..95837b1 100644 (file)
  * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
-#ifndef MockRealtimeVideoSourceMac_h
-#define MockRealtimeVideoSourceMac_h
+#pragma once
 
 #if ENABLE(MEDIA_STREAM)
 
 #include "FontCascade.h"
 #include "MockRealtimeVideoSource.h"
+#include "OrientationNotifer.h"
 
 typedef struct __CVBuffer *CVBufferRef;
 typedef CVBufferRef CVImageBufferRef;
@@ -43,9 +43,8 @@ typedef struct __CVPixelBufferPool *CVPixelBufferPoolRef;
 
 namespace WebCore {
 
-class MockRealtimeVideoSourceMac final : public MockRealtimeVideoSource {
+class MockRealtimeVideoSourceMac final : public MockRealtimeVideoSource, private OrientationNotifier::Observer {
 public:
-
     virtual ~MockRealtimeVideoSourceMac() { }
 
 private:
@@ -59,13 +58,14 @@ private:
     void updateSampleBuffer() final;
     bool applySize(const IntSize&) final;
 
+    void orientationChanged(int orientation) final;
+
     mutable RetainPtr<CGImageRef> m_previewImage;
     mutable RetainPtr<PlatformLayer> m_previewLayer;
     mutable RetainPtr<CVPixelBufferPoolRef> m_bufferPool;
+    MediaSample::VideoRotation m_deviceOrientation { MediaSample::VideoRotation::None };
 };
 
 } // namespace WebCore
 
 #endif // ENABLE(MEDIA_STREAM)
-
-#endif // MockRealtimeVideoSourceMac_h
index bc680a8..be4be00 100644 (file)
@@ -50,13 +50,14 @@ namespace WebCore {
 
 static const int videoSampleRate = 90000;
 
-RefPtr<MockRealtimeVideoSource> MockRealtimeVideoSource::create(const String& name, const MediaConstraints* constraints)
+CaptureSourceOrError MockRealtimeVideoSource::create(const String& name, const MediaConstraints* constraints)
 {
-    auto source = adoptRef(new MockRealtimeVideoSourceMac(name));
+    auto source = adoptRef(*new MockRealtimeVideoSourceMac(name));
+    // FIXME: We should report error messages
     if (constraints && source->applyConstraints(*constraints))
-        source = nullptr;
+        return { };
 
-    return source;
+    return CaptureSourceOrError(WTFMove(source));
 }
 
 MockRealtimeVideoSourceMac::MockRealtimeVideoSourceMac(const String& name)
@@ -161,6 +162,27 @@ bool MockRealtimeVideoSourceMac::applySize(const IntSize& newSize)
     return MockRealtimeVideoSource::applySize(newSize);
 }
 
+void MockRealtimeVideoSourceMac::orientationChanged(int orientation)
+{
+    // FIXME: Do something with m_deviceOrientation
+    switch (orientation) {
+    case 0:
+        m_deviceOrientation = MediaSample::VideoRotation::None;
+        break;
+    case 90:
+        m_deviceOrientation = MediaSample::VideoRotation::Right;
+        break;
+    case -90:
+        m_deviceOrientation = MediaSample::VideoRotation::Left;
+        break;
+    case 180:
+        m_deviceOrientation = MediaSample::VideoRotation::UpsideDown;
+        break;
+    default:
+        return;
+    }
+}
+
 } // namespace WebCore
 
 #endif // ENABLE(MEDIA_STREAM)
index 6f4bf61..5c26050 100644 (file)
@@ -136,19 +136,21 @@ void RealtimeMediaSourceCenterMac::createMediaStream(NewMediaStreamHandler compl
     String invalidConstraint;
 
     if (!audioDeviceID.isEmpty() && m_audioFactory) {
-        if (auto audioSource = m_audioFactory->createMediaSourceForCaptureDeviceWithConstraints(audioDeviceID, CaptureDevice::DeviceType::Audio, audioConstraints, invalidConstraint))
-            audioSources.append(audioSource.releaseNonNull());
+        auto audioSource = m_audioFactory->createAudioCaptureSource(audioDeviceID, audioConstraints);
+        if (!!audioSource)
+            audioSources.append(audioSource.source());
 #if !LOG_DISABLED
-        if (!invalidConstraint.isEmpty())
-            LOG(Media, "RealtimeMediaSourceCenterMac::createMediaStream(%p), audio constraints failed to apply: %s", this, invalidConstraint.utf8().data());
+        if (!audioSource.errorMessage.isEmpty())
+            LOG(Media, "RealtimeMediaSourceCenterMac::createMediaStream(%p), audio constraints failed to apply: %s", this, audioSource.errorMessage.utf8().data());
 #endif
     }
     if (!videoDeviceID.isEmpty() && m_videoFactory) {
-        if (auto videoSource = m_videoFactory->createMediaSourceForCaptureDeviceWithConstraints(videoDeviceID, CaptureDevice::DeviceType::Video, videoConstraints, invalidConstraint))
-            videoSources.append(videoSource.releaseNonNull());
+        auto videoSource = m_videoFactory->createVideoCaptureSource(videoDeviceID, videoConstraints);
+        if (!!videoSource)
+            videoSources.append(videoSource.source());
 #if !LOG_DISABLED
-        if (!invalidConstraint.isEmpty())
-            LOG(Media, "RealtimeMediaSourceCenterMac::createMediaStream(%p), video constraints failed to apply: %s", this, invalidConstraint.utf8().data());
+        if (!videoSource.errorMessage.isEmpty())
+            LOG(Media, "RealtimeMediaSourceCenterMac::createMediaStream(%p), video constraints failed to apply: %s", this, videoSource.errorMessage.utf8().data());
 #endif
     }
 
@@ -187,12 +189,18 @@ Vector<String> RealtimeMediaSourceCenterMac::bestSourcesForTypeAndConstraints(Re
         if (!captureDevice.enabled() || captureDevice.type() != deviceType)
             continue;
 
-        RealtimeMediaSource::CaptureFactory* factory = type == RealtimeMediaSource::Type::Video ? m_videoFactory : m_audioFactory;
-        if (!factory)
-            continue;
+        CaptureSourceOrError sourceOrError;
+        if (type == RealtimeMediaSource::Type::Video && m_videoFactory)
+            sourceOrError = m_videoFactory->createVideoCaptureSource(captureDevice.persistentId(), &constraints);
+        else if (type == RealtimeMediaSource::Type::Audio && m_audioFactory)
+            sourceOrError = m_audioFactory->createAudioCaptureSource(captureDevice.persistentId(), &constraints);
 
-        if (auto captureSource = factory->createMediaSourceForCaptureDeviceWithConstraints(captureDevice.persistentId(), deviceType, &constraints, invalidConstraint))
-            bestSources.append(captureSource.leakRef());
+        if (!sourceOrError) {
+            // FIXME: Handle the case of invalid constraints on more than one device.
+            invalidConstraint = WTFMove(sourceOrError.errorMessage);
+            continue;
+        }
+        bestSources.append(sourceOrError.source());
     }
 
     Vector<String> sourceUIDs;
@@ -207,12 +215,12 @@ Vector<String> RealtimeMediaSourceCenterMac::bestSourcesForTypeAndConstraints(Re
     return sourceUIDs;
 }
 
-RealtimeMediaSource::CaptureFactory* RealtimeMediaSourceCenterMac::defaultAudioFactory()
+RealtimeMediaSource::AudioCaptureFactory* RealtimeMediaSourceCenterMac::defaultAudioFactory()
 {
     return &CoreAudioCaptureSource::factory();
 }
 
-RealtimeMediaSource::CaptureFactory* RealtimeMediaSourceCenterMac::defaultVideoFactory()
+RealtimeMediaSource::VideoCaptureFactory* RealtimeMediaSourceCenterMac::defaultVideoFactory()
 {
     return &AVVideoCaptureSource::factory();
 }
index 1bf497f..ff1f897 100644 (file)
@@ -54,8 +54,8 @@ private:
 
     Vector<String> bestSourcesForTypeAndConstraints(RealtimeMediaSource::Type, const MediaConstraints&, String& invalidConstraint);
 
-    RealtimeMediaSource::CaptureFactory* defaultAudioFactory() final;
-    RealtimeMediaSource::CaptureFactory* defaultVideoFactory() final;
+    RealtimeMediaSource::AudioCaptureFactory* defaultAudioFactory() final;
+    RealtimeMediaSource::VideoCaptureFactory* defaultVideoFactory() final;
 };
 
 } // namespace WebCore
index dbc32e5..250b296 100644 (file)
@@ -108,7 +108,7 @@ void RealtimeMediaSourceCenterOwr::createMediaStream(NewMediaStreamHandler compl
     if (!videoDeviceID.isEmpty())
         types |= OWR_MEDIA_TYPE_VIDEO;
 
-    m_completionHandler = completionHandler;
+    m_completionHandler = WTFMove(completionHandler);
 
     owr_get_capture_sources(static_cast<OwrMediaType>(types), mediaSourcesAvailableCallback, this);
 }
index 2aeb3d4..e014852 100644 (file)
 
 namespace WebCore {
 
-class MockRealtimeAudioSourceFactory : public RealtimeMediaSource::CaptureFactory {
+class MockRealtimeAudioSourceFactory : public RealtimeMediaSource::AudioCaptureFactory {
 public:
-    RefPtr<RealtimeMediaSource> createMediaSourceForCaptureDeviceWithConstraints(const String& deviceID, CaptureDevice::DeviceType type, const MediaConstraints* constraints, String&) final {
-        if (type != CaptureDevice::DeviceType::Audio)
-            return nullptr;
-
+    CaptureSourceOrError createAudioCaptureSource(const String& deviceID, const MediaConstraints* constraints) final {
         for (auto& device : MockRealtimeMediaSource::audioDevices()) {
             if (device.persistentId() == deviceID)
                 return MockRealtimeAudioSource::create(device.label(), constraints);
         }
-
-        return nullptr;
+        return { };
     }
 };
 
 #if !PLATFORM(MAC) && !PLATFORM(IOS)
-RefPtr<MockRealtimeAudioSource> MockRealtimeAudioSource::create(const String& name, const MediaConstraints* constraints)
+CaptureSourceOrError MockRealtimeAudioSource::create(const String& name, const MediaConstraints* constraints)
 {
-    auto source = adoptRef(new MockRealtimeAudioSource(name));
+    auto source = adoptRef(*new MockRealtimeAudioSource(name));
     if (constraints && source->applyConstraints(*constraints))
-        source = nullptr;
+        return { };
 
-    return source;
+    return CaptureSourceOrError(WTFMove(source));
 }
 
 RefPtr<MockRealtimeAudioSource> MockRealtimeAudioSource::createMuted(const String& name)
@@ -74,7 +70,7 @@ RefPtr<MockRealtimeAudioSource> MockRealtimeAudioSource::createMuted(const Strin
 }
 #endif
 
-RealtimeMediaSource::CaptureFactory& MockRealtimeAudioSource::factory()
+RealtimeMediaSource::AudioCaptureFactory& MockRealtimeAudioSource::factory()
 {
     static NeverDestroyed<MockRealtimeAudioSourceFactory> factory;
     return factory.get();
index 1331c79..364522b 100644 (file)
@@ -42,10 +42,10 @@ namespace WebCore {
 class MockRealtimeAudioSource : public MockRealtimeMediaSource {
 public:
 
-    static RefPtr<MockRealtimeAudioSource> create(const String&, const MediaConstraints*);
+    static CaptureSourceOrError create(const String&, const MediaConstraints*);
     static RefPtr<MockRealtimeAudioSource> createMuted(const String& name);
 
-    static CaptureFactory& factory();
+    static AudioCaptureFactory& factory();
 
     virtual ~MockRealtimeAudioSource() = default;
 
index 1298348..5dfbba0 100644 (file)
@@ -86,9 +86,9 @@ void MockRealtimeMediaSourceCenter::validateRequestConstraints(ValidConstraintsH
     if (audioConstraints.isValid()) {
         Vector<DeviceInfo> deviceInfo;
         for (const auto& device : MockRealtimeMediaSource::audioDevices()) {
-            auto audioSource = MockRealtimeAudioSource::create(device.label(), nullptr);
-            if (audioSource->supportsConstraints(audioConstraints, invalidConstraint))
-                deviceInfo.append({audioSource->fitnessScore(), device.persistentId()});
+            auto sourceOrError = MockRealtimeAudioSource::create(device.label(), nullptr);
+            if (!!sourceOrError && sourceOrError.captureSource->supportsConstraints(audioConstraints, invalidConstraint))
+                deviceInfo.append({sourceOrError.captureSource->fitnessScore(), device.persistentId()});
         }
 
         if (deviceInfo.isEmpty()) {
@@ -106,9 +106,9 @@ void MockRealtimeMediaSourceCenter::validateRequestConstraints(ValidConstraintsH
     if (videoConstraints.isValid()) {
         Vector<DeviceInfo> deviceInfo;
         for (const auto& device : MockRealtimeMediaSource::videoDevices()) {
-            auto videoSource = MockRealtimeVideoSource::create(device.label(), nullptr);
-            if (videoSource->supportsConstraints(videoConstraints, invalidConstraint))
-                deviceInfo.append({videoSource->fitnessScore(), device.persistentId()});
+            auto sourceOrError = MockRealtimeVideoSource::create(device.label(), nullptr);
+            if (sourceOrError && sourceOrError.captureSource->supportsConstraints(videoConstraints, invalidConstraint))
+                deviceInfo.append({sourceOrError.captureSource->fitnessScore(), device.persistentId()});
         }
 
         if (deviceInfo.isEmpty()) {
@@ -137,9 +137,9 @@ void MockRealtimeMediaSourceCenter::createMediaStream(NewMediaStreamHandler comp
                 continue;
 
             if (audioDeviceID == captureDevice.persistentId()) {
-                auto source = MockRealtimeAudioSource::create(captureDevice.label(), audioConstraints);
-                if (source)
-                    audioSources.append(source.releaseNonNull());
+                auto sourceOrError = MockRealtimeAudioSource::create(captureDevice.label(), audioConstraints);
+                if (!!sourceOrError)
+                    audioSources.append(sourceOrError.source());
             }
         }
     }
@@ -150,9 +150,9 @@ void MockRealtimeMediaSourceCenter::createMediaStream(NewMediaStreamHandler comp
                 continue;
 
             if (videoDeviceID == captureDevice.persistentId()) {
-                auto source = MockRealtimeVideoSource::create(captureDevice.label(), videoConstraints);
-                if (source)
-                    videoSources.append(source.releaseNonNull());
+                auto sourceOrError = MockRealtimeVideoSource::create(captureDevice.label(), videoConstraints);
+                if (!!sourceOrError)
+                    videoSources.append(sourceOrError.source());
             }
         }
     }
@@ -184,12 +184,12 @@ Vector<CaptureDevice> MockRealtimeMediaSourceCenter::getMediaStreamDevices()
     return sources;
 }
 
-RealtimeMediaSource::CaptureFactory* MockRealtimeMediaSourceCenter::defaultAudioFactory()
+RealtimeMediaSource::AudioCaptureFactory* MockRealtimeMediaSourceCenter::defaultAudioFactory()
 {
     return &MockRealtimeAudioSource::factory();
 }
 
-RealtimeMediaSource::CaptureFactory* MockRealtimeMediaSourceCenter::defaultVideoFactory()
+RealtimeMediaSource::VideoCaptureFactory* MockRealtimeMediaSourceCenter::defaultVideoFactory()
 {
     return &MockRealtimeVideoSource::factory();
 }
index bb2de01..b008f70 100644 (file)
@@ -24,8 +24,7 @@
  * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
-#ifndef MockRealtimeMediaSourceCenter_h
-#define MockRealtimeMediaSourceCenter_h
+#pragma once
 
 #if ENABLE(MEDIA_STREAM)
 
@@ -45,8 +44,8 @@ private:
     Vector<CaptureDevice> getMediaStreamDevices() final;
     void createMediaStream(NewMediaStreamHandler, const String& audioDeviceID, const String& videoDeviceID, const MediaConstraints* audioConstraints, const MediaConstraints* videoConstraints) final;
 
-    RealtimeMediaSource::CaptureFactory* defaultAudioFactory() final;
-    RealtimeMediaSource::CaptureFactory* defaultVideoFactory() final;
+    RealtimeMediaSource::AudioCaptureFactory* defaultAudioFactory() final;
+    RealtimeMediaSource::VideoCaptureFactory* defaultVideoFactory() final;
 
     ExceptionOr<void> setDeviceEnabled(const String& persistentID, bool) final;
 };
@@ -54,5 +53,3 @@ private:
 }
 
 #endif // MockRealtimeMediaSourceCenter_h
-
-#endif
index 6a6074a..bd28746 100644 (file)
 
 namespace WebCore {
 
-class MockRealtimeVideoSourceFactory : public RealtimeMediaSource::CaptureFactory {
+class MockRealtimeVideoSourceFactory : public RealtimeMediaSource::VideoCaptureFactory {
 public:
-    RefPtr<RealtimeMediaSource> createMediaSourceForCaptureDeviceWithConstraints(const String& deviceID, CaptureDevice::DeviceType type, const MediaConstraints* constraints, String&) final {
-        if (type != CaptureDevice::DeviceType::Video)
-            return nullptr;
-
+    CaptureSourceOrError createVideoCaptureSource(const String& deviceID, const MediaConstraints* constraints) final {
         for (auto& device : MockRealtimeMediaSource::videoDevices()) {
             if (device.persistentId() == deviceID)
                 return MockRealtimeVideoSource::create(device.label(), constraints);
         }
-
-        return nullptr;
+        return { };
     }
 };
 
 #if !PLATFORM(MAC) && !PLATFORM(IOS)
-RefPtr<MockRealtimeVideoSource> MockRealtimeVideoSource::create(const String& name, const MediaConstraints* constraints)
+CaptureSourceOrError MockRealtimeVideoSource::create(const String& name, const MediaConstraints* constraints)
 {
-    auto source = adoptRef(new MockRealtimeVideoSource(name));
+    auto source = adoptRef(*new MockRealtimeVideoSource(name));
     if (constraints && source->applyConstraints(*constraints))
-        source = nullptr;
+        return { };
 
-    return source;
+    return CaptureSourceOrError(WTFMove(source));
 }
 
 RefPtr<MockRealtimeVideoSource> MockRealtimeVideoSource::createMuted(const String& name)
@@ -81,7 +77,7 @@ RefPtr<MockRealtimeVideoSource> MockRealtimeVideoSource::createMuted(const Strin
 }
 #endif
 
-RealtimeMediaSource::CaptureFactory& MockRealtimeVideoSource::factory()
+RealtimeMediaSource::VideoCaptureFactory& MockRealtimeVideoSource::factory()
 {
     static NeverDestroyed<MockRealtimeVideoSourceFactory> factory;
     return factory.get();
index d79985e..aa60ec7 100644 (file)
@@ -46,10 +46,10 @@ class GraphicsContext;
 class MockRealtimeVideoSource : public MockRealtimeMediaSource {
 public:
 
-    static RefPtr<MockRealtimeVideoSource> create(const String&, const MediaConstraints*);
+    static CaptureSourceOrError create(const String&, const MediaConstraints*);
     static RefPtr<MockRealtimeVideoSource> createMuted(const String& name);
 
-    static CaptureFactory& factory();
+    static VideoCaptureFactory& factory();
 
     virtual ~MockRealtimeVideoSource() { }
 
index 8b1ba27..f1b3002 100644 (file)
@@ -1,3 +1,23 @@
+2017-04-28  Youenn Fablet  <youenn@apple.com>
+
+        getUserMedia video streams should follow device orientation
+        https://bugs.webkit.org/show_bug.cgi?id=171284
+
+        Reviewed by Eric Carlson.
+
+        Refactoring to specialize CaptureFactory in VideoCaptureFactory and AudioCaptureFactory.
+        Refactoring to return a CaptureSourceOrError instead of passing an out parameter in addition to returning a RefPtr.
+
+        * Shared/WebCoreArgumentCoders.h:
+        * UIProcess/Cocoa/UserMediaCaptureManagerProxy.cpp:
+        (WebKit::UserMediaCaptureManagerProxy::createMediaSourceForCaptureDeviceWithConstraints):
+        * UIProcess/Cocoa/UserMediaCaptureManagerProxy.h:
+        * UIProcess/Cocoa/UserMediaCaptureManagerProxy.messages.in:
+        * WebProcess/cocoa/UserMediaCaptureManager.cpp:
+        (WebKit::UserMediaCaptureManager::createCaptureSource):
+        (WebKit::UserMediaCaptureManager::createMediaSourceForCaptureDeviceWithConstraints): Deleted.
+        * WebProcess/cocoa/UserMediaCaptureManager.h:
+
 2017-04-28  Chris Dumez  <cdumez@apple.com>
 
         [iOS] We should not take a background assertion for the UIProcess when app is MobileMail
index 782420d..8ec61d3 100644 (file)
@@ -35,6 +35,7 @@
 #include <WebCore/MediaSelectionOption.h>
 #include <WebCore/NetworkLoadMetrics.h>
 #include <WebCore/PaymentHeaders.h>
+#include <WebCore/RealtimeMediaSource.h>
 #include <WebCore/ScrollSnapOffsetsInfo.h>
 
 namespace WTF {
@@ -729,6 +730,14 @@ template<> struct EnumTraits<WebCore::CaptureDevice::DeviceType> {
         WebCore::CaptureDevice::DeviceType::Video
     >;
 };
+template<> struct EnumTraits<WebCore::RealtimeMediaSource::Type> {
+    using values = EnumValues<
+    WebCore::RealtimeMediaSource::Type,
+    WebCore::RealtimeMediaSource::Type::None,
+    WebCore::RealtimeMediaSource::Type::Audio,
+    WebCore::RealtimeMediaSource::Type::Video
+    >;
+};
 #endif
 
 template<> struct EnumTraits<WebCore::MediaSelectionOption::Type> {
index 838aef1..9c3de77 100644 (file)
@@ -125,14 +125,27 @@ UserMediaCaptureManagerProxy::~UserMediaCaptureManagerProxy()
     m_process.removeMessageReceiver(Messages::UserMediaCaptureManagerProxy::messageReceiverName());
 }
 
-void UserMediaCaptureManagerProxy::createMediaSourceForCaptureDeviceWithConstraints(uint64_t id, const String& deviceID, WebCore::CaptureDevice::DeviceType type, const MediaConstraintsData& constraintsData, bool& succeeded, String& invalidConstraints)
+void UserMediaCaptureManagerProxy::createMediaSourceForCaptureDeviceWithConstraints(uint64_t id, const String& deviceID, WebCore::RealtimeMediaSource::Type type, const MediaConstraintsData& constraintsData, bool& succeeded, String& invalidConstraints)
 {
+    CaptureSourceOrError sourceOrError;
     auto constraints = MediaConstraintsImpl::create(MediaConstraintsData(constraintsData));
-    auto source = RealtimeMediaSourceCenter::singleton().audioFactory()->createMediaSourceForCaptureDeviceWithConstraints(deviceID, type, constraints.ptr(), invalidConstraints);
-    succeeded = !!source;
+    switch (type) {
+    case WebCore::RealtimeMediaSource::Type::Audio:
+        sourceOrError = RealtimeMediaSourceCenter::singleton().audioFactory()->createAudioCaptureSource(deviceID, constraints.ptr());
+        break;
+    case WebCore::RealtimeMediaSource::Type::Video:
+        sourceOrError = RealtimeMediaSourceCenter::singleton().videoFactory()->createVideoCaptureSource(deviceID, constraints.ptr());
+        break;
+    case WebCore::RealtimeMediaSource::Type::None:
+        ASSERT_NOT_REACHED();
+        break;
+    }
 
-    if (source)
-        m_proxies.set(id, std::make_unique<SourceProxy>(id, *this, source.releaseNonNull()));
+    succeeded = !!sourceOrError;
+    if (sourceOrError)
+        m_proxies.set(id, std::make_unique<SourceProxy>(id, *this, sourceOrError.source()));
+    else
+        invalidConstraints = WTFMove(sourceOrError.errorMessage);
 }
 
 void UserMediaCaptureManagerProxy::startProducingData(uint64_t id)
index 0e63518..4139db2 100644 (file)
@@ -49,7 +49,7 @@ private:
     void didReceiveMessage(IPC::Connection&, IPC::Decoder&) final;
     void didReceiveSyncMessage(IPC::Connection&, IPC::Decoder&, std::unique_ptr<IPC::Encoder>&) final;
 
-    void createMediaSourceForCaptureDeviceWithConstraints(uint64_t id, const String& deviceID, WebCore::CaptureDevice::DeviceType, const WebCore::MediaConstraintsData&, bool& succeeded, String& invalidConstraints);
+    void createMediaSourceForCaptureDeviceWithConstraints(uint64_t id, const String& deviceID, WebCore::RealtimeMediaSource::Type, const WebCore::MediaConstraintsData&, bool& succeeded, String& invalidConstraints);
     void startProducingData(uint64_t);
     void stopProducingData(uint64_t);
     void capabilities(uint64_t, WebCore::RealtimeMediaSourceCapabilities&);
index 527dfa6..1fc093e 100644 (file)
@@ -22,7 +22,7 @@
 # THE POSSIBILITY OF SUCH DAMAGE.
 
 messages -> UserMediaCaptureManagerProxy {
-    CreateMediaSourceForCaptureDeviceWithConstraints(uint64_t id, String deviceID, WebCore::CaptureDevice::DeviceType type, struct WebCore::MediaConstraintsData constraints) -> (bool success, String invalidConstraints)
+    CreateMediaSourceForCaptureDeviceWithConstraints(uint64_t id, String deviceID, WebCore::RealtimeMediaSource::Type type, struct WebCore::MediaConstraintsData constraints) -> (bool success, String invalidConstraints)
     StartProducingData(uint64_t id)
     StopProducingData(uint64_t id)
     Capabilities(uint64_t id) -> (WebCore::RealtimeMediaSourceCapabilities capabilities)
index f2e8ecd..78f31ca 100644 (file)
@@ -98,7 +98,7 @@ void WebBackForwardListProxy::addItemFromUIProcess(uint64_t itemID, Ref<HistoryI
 {
     // This item/itemID pair should not already exist in our maps.
     ASSERT(!historyItemToIDMap().contains(item.ptr()));
-    ASSERT(!idToHistoryItemMap().contains(itemID));
+//    ASSERT(!idToHistoryItemMap().contains(itemID));
 
     historyItemToIDMap().set<ItemAndPageID>(item.ptr(), { .itemID = itemID, .pageID = pageID });
     idToHistoryItemMap().set(itemID, item.ptr());
index b31aa18..4d4f026 100644 (file)
@@ -182,10 +182,10 @@ void UserMediaCaptureManager::initialize(const WebProcessCreationParameters& par
         RealtimeMediaSourceCenter::singleton().setAudioFactory(*this);
 }
 
-RefPtr<RealtimeMediaSource> UserMediaCaptureManager::createMediaSourceForCaptureDeviceWithConstraints(const String& deviceID, CaptureDevice::DeviceType type, const MediaConstraints* constraints, String& invalidConstraints)
+WebCore::CaptureSourceOrError UserMediaCaptureManager::createCaptureSource(const String& deviceID, WebCore::RealtimeMediaSource::Type sourceType, const WebCore::MediaConstraints* constraints)
 {
     if (!constraints)
-        return nullptr;
+        return { };
 
     uint64_t id = nextSessionID();
     MediaConstraintsData constraintsData;
@@ -194,24 +194,13 @@ RefPtr<RealtimeMediaSource> UserMediaCaptureManager::createMediaSourceForCapture
     constraintsData.isValid = constraints->isValid();
     bool succeeded;
 
-    m_process.sendSync(Messages::UserMediaCaptureManagerProxy::CreateMediaSourceForCaptureDeviceWithConstraints(id, deviceID, type, constraintsData), Messages::UserMediaCaptureManagerProxy::CreateMediaSourceForCaptureDeviceWithConstraints::Reply(succeeded, invalidConstraints), 0);
-
-    RealtimeMediaSource::Type sourceType;
-    switch (type) {
-    case WebCore::CaptureDevice::DeviceType::Audio:
-        sourceType = WebCore::RealtimeMediaSource::Type::Audio;
-        break;
-    case WebCore::CaptureDevice::DeviceType::Video:
-        sourceType = WebCore::RealtimeMediaSource::Type::Video;
-        break;
-    case WebCore::CaptureDevice::DeviceType::Unknown:
-    default:
-        sourceType = WebCore::RealtimeMediaSource::Type::None;
-        break;
-    }
-    auto source = adoptRef(new Source(String::number(id), sourceType, emptyString(), id, *this));
-    m_sources.set(id, source);
-    return source;
+    String errorMessage;
+    if (!m_process.sendSync(Messages::UserMediaCaptureManagerProxy::CreateMediaSourceForCaptureDeviceWithConstraints(id, deviceID, sourceType, constraintsData), Messages::UserMediaCaptureManagerProxy::CreateMediaSourceForCaptureDeviceWithConstraints::Reply(succeeded, errorMessage), 0))
+        return WTFMove(errorMessage);
+
+    auto source = adoptRef(*new Source(String::number(id), sourceType, emptyString(), id, *this));
+    m_sources.set(id, source.copyRef());
+    return WebCore::CaptureSourceOrError(WTFMove(source));
 }
 
 void UserMediaCaptureManager::sourceStopped(uint64_t id)
index 446da99..a8a93ab 100644 (file)
@@ -42,7 +42,7 @@ namespace WebKit {
 class CrossProcessRealtimeAudioSource;
 class WebProcess;
 
-class UserMediaCaptureManager : public WebProcessSupplement, public IPC::MessageReceiver, public WebCore::RealtimeMediaSource::CaptureFactory {
+class UserMediaCaptureManager : public WebProcessSupplement, public IPC::MessageReceiver, public WebCore::RealtimeMediaSource::AudioCaptureFactory, public WebCore::RealtimeMediaSource::VideoCaptureFactory {
 public:
     explicit UserMediaCaptureManager(WebProcess*);
     ~UserMediaCaptureManager();
@@ -53,8 +53,10 @@ private:
     // WebProcessSupplement
     void initialize(const WebProcessCreationParameters&) final;
 
-    // WebCore::RealtimeMediaSource::Factory:
-    RefPtr<WebCore::RealtimeMediaSource> createMediaSourceForCaptureDeviceWithConstraints(const String& deviceID, WebCore::CaptureDevice::DeviceType, const WebCore::MediaConstraints*, String&) final;
+    // WebCore::RealtimeMediaSource factories
+    WebCore::CaptureSourceOrError createAudioCaptureSource(const String& deviceID, const WebCore::MediaConstraints* constraints) final { return createCaptureSource(deviceID, WebCore::RealtimeMediaSource::Type::Audio, constraints); }
+    WebCore::CaptureSourceOrError createVideoCaptureSource(const String& deviceID, const WebCore::MediaConstraints* constraints) final { return createCaptureSource(deviceID, WebCore::RealtimeMediaSource::Type::Video, constraints); }
+    WebCore::CaptureSourceOrError createCaptureSource(const String& deviceID, WebCore::RealtimeMediaSource::Type, const WebCore::MediaConstraints*);
 
     // IPC::MessageReceiver
     void didReceiveMessage(IPC::Connection&, IPC::Decoder&) final;