Support arbitrary video resolution in getUserMedia API
authoreric.carlson@apple.com <eric.carlson@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 14 Sep 2018 19:01:41 +0000 (19:01 +0000)
committereric.carlson@apple.com <eric.carlson@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 14 Sep 2018 19:01:41 +0000 (19:01 +0000)
https://bugs.webkit.org/show_bug.cgi?id=178109
<rdar://problem/35083128>

Reviewed by Youenn Fablet.

Source/WebCore:

Support arbitrary video resolutions by configuring the camera to capture at the closest
larger size it supports and scaling/cropping frames as necessary.

No new tests, existing tests updated.

* Modules/mediastream/CanvasCaptureMediaStreamTrack.h:
* Modules/webaudio/MediaStreamAudioSource.cpp:
(WebCore::MediaStreamAudioSource::capabilities): Make non-const, it wasn't helpful.
(WebCore::MediaStreamAudioSource::settings): Ditto.
(WebCore::MediaStreamAudioSource::capabilities const): Deleted.
(WebCore::MediaStreamAudioSource::settings const): Deleted.
* Modules/webaudio/MediaStreamAudioSource.h:

* SourcesCocoa.txt: Add PixelBufferResizer.

* WebCore.xcodeproj/project.pbxproj: Ditto.

* platform/cocoa/CoreVideoSoftLink.h:
* platform/cocoa/VideoToolboxSoftLink.cpp:
* platform/cocoa/VideoToolboxSoftLink.h:

* platform/graphics/cv/PixelBufferResizer.h: Added.
(WebCore::PixelBufferResizer::canResizeTo):
* platform/graphics/cv/PixelBufferResizer.mm: Added.
(WebCore::PixelBufferResizer::PixelBufferResizer):
(WebCore::PixelBufferResizer::resize):

* platform/mediastream/RealtimeIncomingAudioSource.cpp:
(WebCore::RealtimeIncomingAudioSource::capabilities): Make non-const, it wasn't helpful.
(WebCore::RealtimeIncomingAudioSource::settings): Ditto.
(WebCore::RealtimeIncomingAudioSource::capabilities const): Deleted.
(WebCore::RealtimeIncomingAudioSource::settings const): Deleted.
* platform/mediastream/RealtimeIncomingAudioSource.h:

* platform/mediastream/RealtimeIncomingVideoSource.cpp:
(WebCore::RealtimeIncomingVideoSource::capabilities): Make non-const, it wasn't helpful.
(WebCore::RealtimeIncomingVideoSource::settings): Ditto.
(WebCore::RealtimeIncomingVideoSource::capabilities const): Deleted.
(WebCore::RealtimeIncomingVideoSource::settings const): Deleted.
* platform/mediastream/RealtimeIncomingVideoSource.h:

* platform/mediastream/RealtimeMediaSource.cpp:
(WebCore::RealtimeMediaSource::supportsConstraint):
(WebCore::RealtimeMediaSource::supportsConstraint const): Deleted.
* platform/mediastream/RealtimeMediaSource.h:

* platform/mediastream/RealtimeVideoSource.cpp:
(WebCore::RealtimeVideoSource::presets):
(WebCore::RealtimeVideoSource::setSupportedPresets):
(WebCore::standardVideoSizes):
(WebCore::RealtimeVideoSource::updateCapabilities): Make non-const, it wasn't helpful.
(WebCore::presetSupportsFrameRate):
(WebCore::RealtimeVideoSource::supportsCaptureSize):
(WebCore::RealtimeVideoSource::shouldUsePreset):
(WebCore::RealtimeVideoSource::bestSupportedSizeAndFrameRate):
(WebCore::RealtimeVideoSource::setSizeAndFrameRate):
(WebCore::RealtimeVideoSource::addSupportedCapabilities const): Deleted.
* platform/mediastream/RealtimeVideoSource.h:
(WebCore::VideoPresetData::encode const):
(WebCore::VideoPresetData::decode):
(WebCore::VideoPreset::create):
(WebCore::VideoPreset::VideoPreset):
(WebCore::RealtimeVideoSource::prefersPreset):
(WebCore::RealtimeVideoSource::canResizeVideoFrames const):
(WebCore::RealtimeVideoSource::setDefaultSize):
(WebCore::RealtimeVideoSource::observedFrameRate const):
(WebCore::VideoPreset::encode const): Deleted.
(WebCore::VideoPreset::decode): Deleted.

* platform/mediastream/mac/AVVideoCaptureSource.h:
(WebCore::AVVideoCaptureSource::videoPresets): Deleted.
* platform/mediastream/mac/AVVideoCaptureSource.mm:
(WebCore::AVVideoPreset::create):
(WebCore::AVVideoPreset::AVVideoPreset):
(WebCore::AVVideoCaptureSource::AVVideoCaptureSource):
(WebCore::AVVideoCaptureSource::settings):
(WebCore::AVVideoCaptureSource::capabilities): Make non-const, it wasn't helpful.
(WebCore::AVVideoCaptureSource::setFrameRate):
(WebCore::AVVideoCaptureSource::prefersPreset):
(WebCore::AVVideoCaptureSource::setSizeAndFrameRateWithPreset):
(WebCore::AVVideoCaptureSource::frameDurationForFrameRate):
(WebCore::AVVideoCaptureSource::setupCaptureSession):
(WebCore::AVVideoCaptureSource::processNewFrame):
(WebCore::AVVideoCaptureSource::captureOutputDidOutputSampleBufferFromConnection):
(WebCore::AVVideoCaptureSource::isFrameRateSupported):
(WebCore::AVVideoCaptureSource::generatePresets):
(WebCore::updateSizeMinMax): Deleted.
(WebCore::updateAspectRatioMinMax): Deleted.
(WebCore::AVVideoCaptureSource::settings const): Deleted.
(WebCore::AVVideoCaptureSource::capabilities const): Deleted.
(WebCore::AVVideoCaptureSource::sizeForPreset): Deleted.
(WebCore::AVVideoCaptureSource::setPreset): Deleted.
(WebCore::AVVideoCaptureSource::setSizeAndFrameRate): Deleted.
(WebCore::AVVideoCaptureSource::bestSessionPresetForVideoDimensions): Deleted.
(WebCore::AVVideoCaptureSource::supportsSizeAndFrameRate): Deleted.

* platform/mediastream/mac/CoreAudioCaptureSource.cpp:
(WebCore::CoreAudioCaptureSource::capabilities): Make non-const, it wasn't helpful.
(WebCore::CoreAudioCaptureSource::settings): Ditto.
(WebCore::CoreAudioCaptureSource::capabilities const): Deleted.
(WebCore::CoreAudioCaptureSource::settings const): Deleted.
* platform/mediastream/mac/CoreAudioCaptureSource.h:

* platform/mediastream/mac/DisplayCaptureSourceCocoa.cpp:
(WebCore::DisplayCaptureSourceCocoa::capabilities): Make non-const, it wasn't helpful.
(WebCore::DisplayCaptureSourceCocoa::settings): Ditto.
(WebCore::DisplayCaptureSourceCocoa::capabilities const): Deleted.
(WebCore::DisplayCaptureSourceCocoa::settings const): Deleted.
* platform/mediastream/mac/DisplayCaptureSourceCocoa.h:

* platform/mediastream/mac/MockRealtimeVideoSourceMac.h:
* platform/mediastream/mac/MockRealtimeVideoSourceMac.mm:
(WebCore::MockRealtimeVideoSourceMac::CMSampleBufferFromPixelBuffer):
(WebCore::MockRealtimeVideoSourceMac::updateSampleBuffer):
(WebCore::MockRealtimeVideoSourceMac::setSizeAndFrameRateWithPreset):
* platform/mock/MockMediaDevice.h:
(WebCore::MockCameraProperties::decode):

* platform/mock/MockRealtimeAudioSource.cpp:
(WebCore::MockRealtimeAudioSource::settings): Make non-const, it wasn't helpful.
(WebCore::MockRealtimeAudioSource::capabilities): Ditto.
(WebCore::MockRealtimeAudioSource::settings const): Deleted.
(WebCore::MockRealtimeAudioSource::capabilities const): Deleted.
* platform/mock/MockRealtimeAudioSource.h:

* platform/mock/MockRealtimeMediaSourceCenter.cpp:
(WebCore::defaultDevices): Change video device presets to trigger resize code more often.

* platform/mock/MockRealtimeVideoSource.cpp:
(WebCore::MockRealtimeVideoSource::MockRealtimeVideoSource):
(WebCore::MockRealtimeVideoSource::generatePresets):
(WebCore::MockRealtimeVideoSource::capabilities): Make non-const, it wasn't helpful.
(WebCore::MockRealtimeVideoSource::settings): Ditto.
(WebCore::MockRealtimeVideoSource::capabilities const): Deleted.
(WebCore::MockRealtimeVideoSource::settings const): Deleted.
* platform/mock/MockRealtimeVideoSource.h:

Source/WebCore/PAL:

* pal/cf/CoreMediaSoftLink.cpp:
* pal/cf/CoreMediaSoftLink.h:

Source/WebKit:

* WebProcess/cocoa/UserMediaCaptureManager.cpp:

LayoutTests:

* fast/mediastream/MediaStreamTrack-getCapabilities-expected.txt:
* fast/mediastream/apply-constraints-advanced-expected.txt:
* fast/mediastream/apply-constraints-advanced.html:
* fast/mediastream/apply-constraints-video.html:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@236015 268f45cc-cd09-0410-ab3c-d52691b4dbfc

53 files changed:
LayoutTests/ChangeLog
LayoutTests/fast/mediastream/MediaStreamTrack-getCapabilities-expected.txt
LayoutTests/fast/mediastream/apply-constraints-advanced-expected.txt
LayoutTests/fast/mediastream/apply-constraints-advanced.html
LayoutTests/fast/mediastream/apply-constraints-video-expected.txt
LayoutTests/fast/mediastream/apply-constraints-video.html
Source/WebCore/ChangeLog
Source/WebCore/Modules/mediastream/CanvasCaptureMediaStreamTrack.h
Source/WebCore/Modules/webaudio/MediaStreamAudioSource.cpp
Source/WebCore/Modules/webaudio/MediaStreamAudioSource.h
Source/WebCore/PAL/ChangeLog
Source/WebCore/PAL/pal/cf/CoreMediaSoftLink.cpp
Source/WebCore/PAL/pal/cf/CoreMediaSoftLink.h
Source/WebCore/SourcesCocoa.txt
Source/WebCore/WebCore.xcodeproj/project.pbxproj
Source/WebCore/platform/cocoa/CoreVideoSoftLink.h
Source/WebCore/platform/cocoa/VideoToolboxSoftLink.cpp
Source/WebCore/platform/cocoa/VideoToolboxSoftLink.h
Source/WebCore/platform/graphics/cv/PixelBufferResizer.h [new file with mode: 0644]
Source/WebCore/platform/graphics/cv/PixelBufferResizer.mm [new file with mode: 0644]
Source/WebCore/platform/mediastream/RealtimeIncomingAudioSource.cpp
Source/WebCore/platform/mediastream/RealtimeIncomingAudioSource.h
Source/WebCore/platform/mediastream/RealtimeIncomingVideoSource.cpp
Source/WebCore/platform/mediastream/RealtimeIncomingVideoSource.h
Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp
Source/WebCore/platform/mediastream/RealtimeMediaSource.h
Source/WebCore/platform/mediastream/RealtimeVideoSource.cpp
Source/WebCore/platform/mediastream/RealtimeVideoSource.h
Source/WebCore/platform/mediastream/VideoPreset.h [new file with mode: 0644]
Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCaptureSource.cpp
Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCaptureSource.h
Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.cpp
Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.h
Source/WebCore/platform/mediastream/gstreamer/MockGStreamerAudioCaptureSource.cpp
Source/WebCore/platform/mediastream/gstreamer/MockGStreamerAudioCaptureSource.h
Source/WebCore/platform/mediastream/gstreamer/MockGStreamerVideoCaptureSource.cpp
Source/WebCore/platform/mediastream/gstreamer/MockGStreamerVideoCaptureSource.h
Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.h
Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.mm
Source/WebCore/platform/mediastream/mac/CoreAudioCaptureSource.cpp
Source/WebCore/platform/mediastream/mac/CoreAudioCaptureSource.h
Source/WebCore/platform/mediastream/mac/DisplayCaptureSourceCocoa.cpp
Source/WebCore/platform/mediastream/mac/DisplayCaptureSourceCocoa.h
Source/WebCore/platform/mediastream/mac/MockRealtimeVideoSourceMac.h
Source/WebCore/platform/mediastream/mac/MockRealtimeVideoSourceMac.mm
Source/WebCore/platform/mock/MockMediaDevice.h
Source/WebCore/platform/mock/MockRealtimeAudioSource.cpp
Source/WebCore/platform/mock/MockRealtimeAudioSource.h
Source/WebCore/platform/mock/MockRealtimeMediaSourceCenter.cpp
Source/WebCore/platform/mock/MockRealtimeVideoSource.cpp
Source/WebCore/platform/mock/MockRealtimeVideoSource.h
Source/WebKit/ChangeLog
Source/WebKit/WebProcess/cocoa/UserMediaCaptureManager.cpp

index 2babe41..b9fffb6 100644 (file)
@@ -1,3 +1,16 @@
+2018-09-14  Eric Carlson  <eric.carlson@apple.com>
+
+        Support arbitrary video resolution in getUserMedia API
+        https://bugs.webkit.org/show_bug.cgi?id=178109
+        <rdar://problem/35083128>
+
+        Reviewed by Youenn Fablet.
+
+        * fast/mediastream/MediaStreamTrack-getCapabilities-expected.txt:
+        * fast/mediastream/apply-constraints-advanced-expected.txt:
+        * fast/mediastream/apply-constraints-advanced.html:
+        * fast/mediastream/apply-constraints-video.html:
+
 2018-09-14  Devin Rousso  <webkit@devinrousso.com>
 
         Web Inspector: Record actions performed on ImageBitmapRenderingContext
index 431f1f0..43e9f42 100644 (file)
@@ -4,12 +4,12 @@ On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE
 
 
 video track capabilities:
-  capabilities.aspectRatio = { max: 1.778, min: 1.222 }
+  capabilities.aspectRatio = { max: 1.778, min: 1 }
   capabilities.deviceId = <UUID>
   capabilities.facingMode = [ user ]
   capabilities.frameRate = { max: 30, min: 5 }
-  capabilities.height = { max: 2160, min: 240 }
-  capabilities.width = { max: 3840, min: 320 }
+  capabilities.height = { max: 720, min: 112 }
+  capabilities.width = { max: 1280, min: 112 }
 
 audio track capabilities:
   capabilities.deviceId = <UUID>
index 75d0bdc..4e22260 100644 (file)
@@ -16,9 +16,9 @@ PASS settings['height'] is 480
 PASS settings['width'] is 640
 PASS settings['height'] is 480
 
-** Constraint: {"width":{"min":640},"height":{"min":480},"advanced":[{"width":6000,"height":6000},{"width":1920,"height":1080}]} - first width and height in advanced are too big, second is used.
-PASS settings['width'] is 1920
-PASS settings['height'] is 1080
+** Constraint: {"width":{"min":640},"height":{"min":480},"advanced":[{"width":6000,"height":6000},{"width":1280,"height":720}]} - first width and height in advanced are too big, second is used.
+PASS settings['width'] is 1280
+PASS settings['height'] is 720
 
 ** Constraint: {"width":320,"height":240} - reset width and height.
 PASS settings['width'] is 320
index 922e135..1eab379 100644 (file)
                                     height: { min: 480 },
                                     advanced: [
                                         { width: 6000, height: 6000 },
-                                        { width: 1920, height: 1080 },
+                                        { width: 1280, height: 720 },
                                     ]
                                  },
-                    expected: { width: 1920, height: 1080 }, 
+                    expected: { width: 1280, height: 720 }, 
                 },
                 {
                     message: "reset width and height.",
index d890d36..37d3ee6 100644 (file)
@@ -32,34 +32,34 @@ PASS settings['width'] is 320
 PASS settings['height'] is 240
 PASS settings['frameRate'] is 30
 
-** Constraint: {"frameRate":{"max":6}} - the 'max' constraint can't be satisfied, promise should reject and no settings should change.
+** Constraint: {"frameRate":{"max":8}} - the 'max' constraint can't be satisfied, promise should reject and no settings should change.
 PASS Promise was rejected
 PASS error.constraint is "frameRate"
 PASS settings['frameRate'] is 30
 
-** Constraint: {"width":{"exact":960}} - the 'exact' constraint can be satisfied.
-PASS settings['width'] is 960
-PASS settings['height'] is 540
+** Constraint: {"width":{"exact":640}} - the 'exact' constraint can be satisfied.
+PASS settings['width'] is 640
+PASS settings['height'] is 480
 
 ** Constraint: {"width":{"min":300,"ideal":5000}} - the 'ideal' constraint can't be satisfied but the 'min' can, maximum value should be chosen.
-PASS settings['width'] is 3840
-PASS settings['height'] is 2160
-
-** Constraint: {"width":{"min":320,"ideal":1280},"height":{"min":480,"ideal":720}} - 'ideal' and 'min' constraints can be satisfied, 'ideal' should be chosen.
 PASS settings['width'] is 1280
 PASS settings['height'] is 720
 
+** Constraint: {"width":{"min":320,"ideal":640},"height":{"min":480,"ideal":720}} - 'ideal' and 'min' constraints can be satisfied, 'ideal' should be chosen.
+PASS settings['width'] is 640
+PASS settings['height'] is 720
+
 ** Constraint: {"width":5000} - ideal width is greater than track capability, should be clamped to the maximum value.
-PASS settings['width'] is 3840
+PASS settings['width'] is 1280
 
-** Constraint: {"width":160,"height":120,"frameRate":4} - all values are less than track capabilities, should be clamped to the minimum values.
-PASS settings['width'] is 320
-PASS settings['height'] is 240
+** Constraint: {"width":100,"height":100,"frameRate":4} - all values are less than track capabilities, should be clamped to the minimum values.
+PASS settings['width'] is 112
+PASS settings['height'] is 112
 PASS settings['frameRate'] is 5
 
 ** Constraint: {"frameRate":20} - set frame rate, width and height should remain unchanged
-PASS settings['width'] is 320
-PASS settings['height'] is 240
+PASS settings['width'] is 112
+PASS settings['height'] is 112
 PASS settings['frameRate'] is 20
 
 ** Constraint: {"facingMode":"xnvironment","height":720} - illegal facing mode value should be ignored, height should change.
index 7765c48..02e112b 100644 (file)
                 },
                 {
                     message: "the 'max' constraint can't be satisfied, promise should reject and no settings should change.",
-                    constraint: { frameRate: {max: 6}, }, 
+                    constraint: { frameRate: {max: 8}, }, 
                     expected: { frameRate: 30 },
                     error: "frameRate",
                 },
                 {
                     message: "the 'exact' constraint can be satisfied.",
-                    constraint: { width: { exact: 960 } },
-                    expected: { width: 960, height: 540 },
+                    constraint: { width: { exact: 640 } },
+                    expected: { width: 640, height: 480 },
                 },
                 {
                     message: "the 'ideal' constraint can't be satisfied but the 'min' can, maximum value should be chosen.",
                     constraint: { width: {min: 300, ideal: 5000} }, 
-                    expected: { width: 3840, height: 2160 },
+                    expected: { width: 1280, height: 720 },
                 },
                 {
                     message: "'ideal' and 'min' constraints can be satisfied, 'ideal' should be chosen.",
-                    constraint: { width: {min: 320, ideal: 1280}, height: {min: 480, ideal: 720}, }, 
-                    expected: { width: 1280, height: 720 },
+                    constraint: { width: {min: 320, ideal: 640}, height: {min: 480, ideal: 720}, }, 
+                    expected: { width: 640, height: 720 },
                 },
                 {
                     message: "ideal width is greater than track capability, should be clamped to the maximum value.",
                     constraint: { width: 5000 },
-                    expected: { width: 3840},
+                    expected: { width: 1280},
                 },
                 {
                     message: "all values are less than track capabilities, should be clamped to the minimum values.",
-                    constraint: { width: 160, height: 120, frameRate: 4 },
-                    expected: { width: 320, height: 240, frameRate: 5 },
+                    constraint: { width: 100, height: 100, frameRate: 4 },
+                    expected: { width: 112, height: 112, frameRate: 5 },
                 },
                 {
                     message: "set frame rate, width and height should remain unchanged",
                     constraint: { frameRate: 20 }, 
-                    expected: { width: 320, height: 240, frameRate: 20 },
+                    expected: { width: 112, height: 112, frameRate: 20 },
                 },
                 {
                     message: "illegal facing mode value should be ignored, height should change.",
index fc89081..e967f0f 100644 (file)
@@ -1,3 +1,148 @@
+2018-09-14  Eric Carlson  <eric.carlson@apple.com>
+
+        Support arbitrary video resolution in getUserMedia API
+        https://bugs.webkit.org/show_bug.cgi?id=178109
+        <rdar://problem/35083128>
+
+        Reviewed by Youenn Fablet.
+
+        Support arbitrary video resolutions by configuring the camera to capture at the closest
+        larger size it supports and scaling/cropping frames as necessary.
+
+        No new tests, existing tests updated.
+
+        * Modules/mediastream/CanvasCaptureMediaStreamTrack.h:
+        * Modules/webaudio/MediaStreamAudioSource.cpp:
+        (WebCore::MediaStreamAudioSource::capabilities): Make non-const, it wasn't helpful.
+        (WebCore::MediaStreamAudioSource::settings): Ditto.
+        (WebCore::MediaStreamAudioSource::capabilities const): Deleted.
+        (WebCore::MediaStreamAudioSource::settings const): Deleted.
+        * Modules/webaudio/MediaStreamAudioSource.h:
+
+        * SourcesCocoa.txt: Add PixelBufferResizer.
+
+        * WebCore.xcodeproj/project.pbxproj: Ditto.
+
+        * platform/cocoa/CoreVideoSoftLink.h:
+        * platform/cocoa/VideoToolboxSoftLink.cpp:
+        * platform/cocoa/VideoToolboxSoftLink.h:
+
+        * platform/graphics/cv/PixelBufferResizer.h: Added.
+        (WebCore::PixelBufferResizer::canResizeTo):
+        * platform/graphics/cv/PixelBufferResizer.mm: Added.
+        (WebCore::PixelBufferResizer::PixelBufferResizer):
+        (WebCore::PixelBufferResizer::resize):
+
+        * platform/mediastream/RealtimeIncomingAudioSource.cpp:
+        (WebCore::RealtimeIncomingAudioSource::capabilities): Make non-const, it wasn't helpful.
+        (WebCore::RealtimeIncomingAudioSource::settings): Ditto.
+        (WebCore::RealtimeIncomingAudioSource::capabilities const): Deleted.
+        (WebCore::RealtimeIncomingAudioSource::settings const): Deleted.
+        * platform/mediastream/RealtimeIncomingAudioSource.h:
+
+        * platform/mediastream/RealtimeIncomingVideoSource.cpp:
+        (WebCore::RealtimeIncomingVideoSource::capabilities): Make non-const, it wasn't helpful.
+        (WebCore::RealtimeIncomingVideoSource::settings): Ditto.
+        (WebCore::RealtimeIncomingVideoSource::capabilities const): Deleted.
+        (WebCore::RealtimeIncomingVideoSource::settings const): Deleted.
+        * platform/mediastream/RealtimeIncomingVideoSource.h:
+
+        * platform/mediastream/RealtimeMediaSource.cpp:
+        (WebCore::RealtimeMediaSource::supportsConstraint):
+        (WebCore::RealtimeMediaSource::supportsConstraint const): Deleted.
+        * platform/mediastream/RealtimeMediaSource.h:
+
+        * platform/mediastream/RealtimeVideoSource.cpp:
+        (WebCore::RealtimeVideoSource::presets):
+        (WebCore::RealtimeVideoSource::setSupportedPresets):
+        (WebCore::standardVideoSizes):
+        (WebCore::RealtimeVideoSource::updateCapabilities): Make non-const, it wasn't helpful.
+        (WebCore::presetSupportsFrameRate):
+        (WebCore::RealtimeVideoSource::supportsCaptureSize):
+        (WebCore::RealtimeVideoSource::shouldUsePreset):
+        (WebCore::RealtimeVideoSource::bestSupportedSizeAndFrameRate):
+        (WebCore::RealtimeVideoSource::setSizeAndFrameRate):
+        (WebCore::RealtimeVideoSource::addSupportedCapabilities const): Deleted.
+        * platform/mediastream/RealtimeVideoSource.h:
+        (WebCore::VideoPresetData::encode const):
+        (WebCore::VideoPresetData::decode):
+        (WebCore::VideoPreset::create):
+        (WebCore::VideoPreset::VideoPreset):
+        (WebCore::RealtimeVideoSource::prefersPreset):
+        (WebCore::RealtimeVideoSource::canResizeVideoFrames const):
+        (WebCore::RealtimeVideoSource::setDefaultSize):
+        (WebCore::RealtimeVideoSource::observedFrameRate const):
+        (WebCore::VideoPreset::encode const): Deleted.
+        (WebCore::VideoPreset::decode): Deleted.
+
+        * platform/mediastream/mac/AVVideoCaptureSource.h:
+        (WebCore::AVVideoCaptureSource::videoPresets): Deleted.
+        * platform/mediastream/mac/AVVideoCaptureSource.mm:
+        (WebCore::AVVideoPreset::create):
+        (WebCore::AVVideoPreset::AVVideoPreset):
+        (WebCore::AVVideoCaptureSource::AVVideoCaptureSource):
+        (WebCore::AVVideoCaptureSource::settings):
+        (WebCore::AVVideoCaptureSource::capabilities): Make non-const, it wasn't helpful.
+        (WebCore::AVVideoCaptureSource::setFrameRate):
+        (WebCore::AVVideoCaptureSource::prefersPreset):
+        (WebCore::AVVideoCaptureSource::setSizeAndFrameRateWithPreset):
+        (WebCore::AVVideoCaptureSource::frameDurationForFrameRate):
+        (WebCore::AVVideoCaptureSource::setupCaptureSession):
+        (WebCore::AVVideoCaptureSource::processNewFrame):
+        (WebCore::AVVideoCaptureSource::captureOutputDidOutputSampleBufferFromConnection):
+        (WebCore::AVVideoCaptureSource::isFrameRateSupported):
+        (WebCore::AVVideoCaptureSource::generatePresets):
+        (WebCore::updateSizeMinMax): Deleted.
+        (WebCore::updateAspectRatioMinMax): Deleted.
+        (WebCore::AVVideoCaptureSource::settings const): Deleted.
+        (WebCore::AVVideoCaptureSource::capabilities const): Deleted.
+        (WebCore::AVVideoCaptureSource::sizeForPreset): Deleted.
+        (WebCore::AVVideoCaptureSource::setPreset): Deleted.
+        (WebCore::AVVideoCaptureSource::setSizeAndFrameRate): Deleted.
+        (WebCore::AVVideoCaptureSource::bestSessionPresetForVideoDimensions): Deleted.
+        (WebCore::AVVideoCaptureSource::supportsSizeAndFrameRate): Deleted.
+
+        * platform/mediastream/mac/CoreAudioCaptureSource.cpp:
+        (WebCore::CoreAudioCaptureSource::capabilities): Make non-const, it wasn't helpful.
+        (WebCore::CoreAudioCaptureSource::settings): Ditto.
+        (WebCore::CoreAudioCaptureSource::capabilities const): Deleted.
+        (WebCore::CoreAudioCaptureSource::settings const): Deleted.
+        * platform/mediastream/mac/CoreAudioCaptureSource.h:
+
+        * platform/mediastream/mac/DisplayCaptureSourceCocoa.cpp:
+        (WebCore::DisplayCaptureSourceCocoa::capabilities): Make non-const, it wasn't helpful.
+        (WebCore::DisplayCaptureSourceCocoa::settings): Ditto.
+        (WebCore::DisplayCaptureSourceCocoa::capabilities const): Deleted.
+        (WebCore::DisplayCaptureSourceCocoa::settings const): Deleted.
+        * platform/mediastream/mac/DisplayCaptureSourceCocoa.h:
+
+        * platform/mediastream/mac/MockRealtimeVideoSourceMac.h:
+        * platform/mediastream/mac/MockRealtimeVideoSourceMac.mm:
+        (WebCore::MockRealtimeVideoSourceMac::CMSampleBufferFromPixelBuffer):
+        (WebCore::MockRealtimeVideoSourceMac::updateSampleBuffer):
+        (WebCore::MockRealtimeVideoSourceMac::setSizeAndFrameRateWithPreset):
+        * platform/mock/MockMediaDevice.h:
+        (WebCore::MockCameraProperties::decode):
+
+        * platform/mock/MockRealtimeAudioSource.cpp:
+        (WebCore::MockRealtimeAudioSource::settings): Make non-const, it wasn't helpful.
+        (WebCore::MockRealtimeAudioSource::capabilities): Ditto.
+        (WebCore::MockRealtimeAudioSource::settings const): Deleted.
+        (WebCore::MockRealtimeAudioSource::capabilities const): Deleted.
+        * platform/mock/MockRealtimeAudioSource.h:
+
+        * platform/mock/MockRealtimeMediaSourceCenter.cpp:
+        (WebCore::defaultDevices): Change video device presets to trigger resize code more often.
+
+        * platform/mock/MockRealtimeVideoSource.cpp:
+        (WebCore::MockRealtimeVideoSource::MockRealtimeVideoSource):
+        (WebCore::MockRealtimeVideoSource::generatePresets):
+        (WebCore::MockRealtimeVideoSource::capabilities): Make non-const, it wasn't helpful.
+        (WebCore::MockRealtimeVideoSource::settings): Ditto.
+        (WebCore::MockRealtimeVideoSource::capabilities const): Deleted.
+        (WebCore::MockRealtimeVideoSource::settings const): Deleted.
+        * platform/mock/MockRealtimeVideoSource.h:
+
 2018-09-14  Frederic Wang  <fwang@igalia.com>
 
         Bug 189541 - Build error in FontDescriptionKey::computeHash when compiling FontTaggedSettings and FontCascadeFonts together
index b006af9..b229059 100644 (file)
@@ -65,8 +65,8 @@ private:
         // RealtimeMediaSource API
         void startProducingData() final;
         void stopProducingData()  final;
-        const RealtimeMediaSourceCapabilities& capabilities() const final { return RealtimeMediaSourceCapabilities::emptyCapabilities(); }
-        const RealtimeMediaSourceSettings& settings() const final { return m_settings; }
+        const RealtimeMediaSourceCapabilities& capabilities() final { return RealtimeMediaSourceCapabilities::emptyCapabilities(); }
+        const RealtimeMediaSourceSettings& settings() final { return m_settings; }
 
         void captureCanvas();
         void requestFrameTimerFired();
index d797b67..56f37c1 100644 (file)
@@ -39,7 +39,7 @@ MediaStreamAudioSource::MediaStreamAudioSource(float sampleRate)
     m_currentSettings.setSampleRate(sampleRate);
 }
 
-const RealtimeMediaSourceCapabilities& MediaStreamAudioSource::capabilities() const
+const RealtimeMediaSourceCapabilities& MediaStreamAudioSource::capabilities()
 {
     // FIXME: implement this.
     // https://bugs.webkit.org/show_bug.cgi?id=122430
@@ -47,7 +47,7 @@ const RealtimeMediaSourceCapabilities& MediaStreamAudioSource::capabilities() co
     return RealtimeMediaSourceCapabilities::emptyCapabilities();
 }
 
-const RealtimeMediaSourceSettings& MediaStreamAudioSource::settings() const
+const RealtimeMediaSourceSettings& MediaStreamAudioSource::settings()
 {
     // FIXME: implement this.
     // https://bugs.webkit.org/show_bug.cgi?id=122430
index 4b90c8a..fc6ddb8 100644 (file)
@@ -43,8 +43,8 @@ public:
 
     ~MediaStreamAudioSource() = default;
 
-    const RealtimeMediaSourceCapabilities& capabilities() const final;
-    const RealtimeMediaSourceSettings& settings() const final;
+    const RealtimeMediaSourceCapabilities& capabilities() final;
+    const RealtimeMediaSourceSettings& settings() final;
 
     const String& deviceId() const { return m_deviceId; }
     void setDeviceId(const String& deviceId) { m_deviceId = deviceId; }
index d038276..d77a76a 100644 (file)
@@ -1,3 +1,14 @@
+2018-09-14  Eric Carlson  <eric.carlson@apple.com>
+
+        Support arbitrary video resolution in getUserMedia API
+        https://bugs.webkit.org/show_bug.cgi?id=178109
+        <rdar://problem/35083128>
+
+        Reviewed by Youenn Fablet.
+
+        * pal/cf/CoreMediaSoftLink.cpp:
+        * pal/cf/CoreMediaSoftLink.h:
+
 2018-09-12  Guillaume Emont  <guijemont@igalia.com>
 
         Add IGNORE_WARNING_.* macros
index b60a645..bda1bf8 100644 (file)
@@ -97,6 +97,7 @@ SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMSampleBufferCreate, OSStatus, (C
 SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMSampleBufferCreateCopy, OSStatus, (CFAllocatorRef allocator, CMSampleBufferRef sbuf, CMSampleBufferRef *sbufCopyOut), (allocator, sbuf, sbufCopyOut))
 SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMSampleBufferCreateCopyWithNewTiming, OSStatus, (CFAllocatorRef allocator, CMSampleBufferRef originalSBuf, CMItemCount numSampleTimingEntries, const CMSampleTimingInfo *sampleTimingArray, CMSampleBufferRef *sBufCopyOut), (allocator, originalSBuf, numSampleTimingEntries, sampleTimingArray, sBufCopyOut))
 SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMSampleBufferCreateReadyWithImageBuffer, OSStatus, (CFAllocatorRef allocator, CVImageBufferRef imageBuffer, CMVideoFormatDescriptionRef formatDescription, const CMSampleTimingInfo* sampleTiming, CMSampleBufferRef* sBufOut), (allocator, imageBuffer, formatDescription, sampleTiming, sBufOut))
+SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMSampleBufferCreateForImageBuffer, OSStatus, (CFAllocatorRef allocator, CVImageBufferRef imageBuffer, Boolean dataReady, CMSampleBufferMakeDataReadyCallback makeDataReadyCallback, void* makeDataReadyRefcon, CMVideoFormatDescriptionRef formatDescription, const CMSampleTimingInfo* sampleTiming, CMSampleBufferRef* sampleBufferOut), (allocator, imageBuffer, dataReady, makeDataReadyCallback, makeDataReadyRefcon, formatDescription, sampleTiming, sampleBufferOut))
 SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMSampleBufferGetDecodeTimeStamp, CMTime, (CMSampleBufferRef sbuf), (sbuf))
 SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMSampleBufferGetDuration, CMTime, (CMSampleBufferRef sbuf), (sbuf))
 SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMSampleBufferGetImageBuffer, CVImageBufferRef, (CMSampleBufferRef sbuf), (sbuf))
index e95fbd3..1a67a3c 100644 (file)
@@ -151,6 +151,8 @@ SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMSampleBufferCreateCopyWithNewTim
 #define CMSampleBufferCreateCopyWithNewTiming softLink_CoreMedia_CMSampleBufferCreateCopyWithNewTiming
 SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMSampleBufferCreateReadyWithImageBuffer, OSStatus, (CFAllocatorRef allocator, CVImageBufferRef imageBuffer, CMVideoFormatDescriptionRef formatDescription, const CMSampleTimingInfo *sampleTiming, CMSampleBufferRef *sBufOut), (allocator, imageBuffer, formatDescription, sampleTiming, sBufOut))
 #define CMSampleBufferCreateReadyWithImageBuffer softLink_CoreMedia_CMSampleBufferCreateReadyWithImageBuffer
+SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMSampleBufferCreateForImageBuffer, OSStatus, (CFAllocatorRef allocator, CVImageBufferRef imageBuffer, Boolean dataReady, CMSampleBufferMakeDataReadyCallback makeDataReadyCallback, void* makeDataReadyRefcon, CMVideoFormatDescriptionRef formatDescription, const CMSampleTimingInfo* sampleTiming, CMSampleBufferRef* sampleBufferOut), (allocator, imageBuffer, dataReady, makeDataReadyCallback, makeDataReadyRefcon, formatDescription, sampleTiming, sampleBufferOut))
+#define CMSampleBufferCreateForImageBuffer softLink_CoreMedia_CMSampleBufferCreateForImageBuffer
 SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMSampleBufferGetDecodeTimeStamp, CMTime, (CMSampleBufferRef sbuf), (sbuf))
 #define CMSampleBufferGetDecodeTimeStamp softLink_CoreMedia_CMSampleBufferGetDecodeTimeStamp
 SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMSampleBufferGetDuration, CMTime, (CMSampleBufferRef sbuf), (sbuf))
index 354d532..6a78661 100644 (file)
@@ -312,6 +312,7 @@ platform/graphics/cocoa/WebGPULayer.mm
 platform/graphics/cocoa/TextTrackRepresentationCocoa.mm
 
 platform/graphics/cv/PixelBufferConformerCV.cpp
+platform/graphics/cv/PixelBufferResizer.mm
 platform/graphics/cv/TextureCacheCV.mm
 platform/graphics/cv/VideoTextureCopierCV.cpp
 
index 6547d0a..684644f 100644 (file)
                074E82BB18A69F0E007EF54C /* PlatformTimeRanges.h in Headers */ = {isa = PBXBuildFile; fileRef = 074E82B918A69F0E007EF54C /* PlatformTimeRanges.h */; settings = {ATTRIBUTES = (Private, ); }; };
                0753860214489E9800B78452 /* CachedTextTrack.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0753860014489E9800B78452 /* CachedTextTrack.cpp */; };
                0753860314489E9800B78452 /* CachedTextTrack.h in Headers */ = {isa = PBXBuildFile; fileRef = 0753860114489E9800B78452 /* CachedTextTrack.h */; };
+               0757B13E214AE79900794B0D /* VideoPreset.h in Headers */ = {isa = PBXBuildFile; fileRef = 0757B13C214AE79700794B0D /* VideoPreset.h */; settings = {ATTRIBUTES = (Private, ); }; };
                075BA84920618AA500FCB4AD /* VideoFullscreenLayerManager.h in Headers */ = {isa = PBXBuildFile; fileRef = 075BA84720618AA500FCB4AD /* VideoFullscreenLayerManager.h */; };
                07638A991884487200E15A1B /* MediaSessionManagerIOS.h in Headers */ = {isa = PBXBuildFile; fileRef = 07638A971884487200E15A1B /* MediaSessionManagerIOS.h */; settings = {ATTRIBUTES = (Private, ); }; };
                07638A9A1884487200E15A1B /* MediaSessionManagerIOS.mm in Sources */ = {isa = PBXBuildFile; fileRef = 07638A981884487200E15A1B /* MediaSessionManagerIOS.mm */; };
                073B87651E43859D0071C0EC /* AudioSampleDataSource.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = AudioSampleDataSource.h; sourceTree = "<group>"; };
                0744ECEB1E0C4AE5000D0944 /* MockRealtimeAudioSourceMac.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = MockRealtimeAudioSourceMac.h; sourceTree = "<group>"; };
                0744ECEC1E0C4AE5000D0944 /* MockRealtimeAudioSourceMac.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = MockRealtimeAudioSourceMac.mm; sourceTree = "<group>"; };
+               0746D30A2146EA37003DDF84 /* PixelBufferResizer.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = PixelBufferResizer.mm; sourceTree = "<group>"; };
+               0746D30C2146EA38003DDF84 /* PixelBufferResizer.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = PixelBufferResizer.h; sourceTree = "<group>"; };
                074E82B818A69F0E007EF54C /* PlatformTimeRanges.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = PlatformTimeRanges.cpp; sourceTree = "<group>"; };
                074E82B918A69F0E007EF54C /* PlatformTimeRanges.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = PlatformTimeRanges.h; sourceTree = "<group>"; };
                0753860014489E9800B78452 /* CachedTextTrack.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = CachedTextTrack.cpp; sourceTree = "<group>"; };
                0753860114489E9800B78452 /* CachedTextTrack.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = CachedTextTrack.h; sourceTree = "<group>"; };
+               0757B13C214AE79700794B0D /* VideoPreset.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = VideoPreset.h; sourceTree = "<group>"; };
                075BA84720618AA500FCB4AD /* VideoFullscreenLayerManager.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = VideoFullscreenLayerManager.h; sourceTree = "<group>"; };
                07638A971884487200E15A1B /* MediaSessionManagerIOS.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = MediaSessionManagerIOS.h; sourceTree = "<group>"; };
                07638A981884487200E15A1B /* MediaSessionManagerIOS.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = MediaSessionManagerIOS.mm; sourceTree = "<group>"; };
                                07221BAD17CF0AD400848E51 /* RTCSessionDescriptionRequest.h */,
                                313591051E7DDC6000F30630 /* RTCSignalingState.h */,
                                07221BB017CF0AD400848E51 /* RTCVoidRequest.h */,
+                               0757B13C214AE79700794B0D /* VideoPreset.h */,
                                070E81D01BF27656001FDA48 /* VideoTrackPrivateMediaStream.h */,
                                41B2A6251EF1BF60002B9D7A /* WebAudioSourceProvider.h */,
                        );
                        children = (
                                CD7D33411C7A123F00041293 /* PixelBufferConformerCV.cpp */,
                                CD7D33421C7A123F00041293 /* PixelBufferConformerCV.h */,
+                               0746D30C2146EA38003DDF84 /* PixelBufferResizer.h */,
+                               0746D30A2146EA37003DDF84 /* PixelBufferResizer.mm */,
                                CD9D82741C7AE535006FF066 /* TextureCacheCV.h */,
                                CD9D82731C7AE535006FF066 /* TextureCacheCV.mm */,
                                CD9D82771C7B8EE1006FF066 /* VideoTextureCopierCV.cpp */,
                                52D5A1A71C57489D00DE34A3 /* VideoFullscreenModel.h in Headers */,
                                52D5A1A81C5748A300DE34A3 /* VideoFullscreenModelVideoElement.h in Headers */,
                                CDE83DB2183C44060031EAA3 /* VideoPlaybackQuality.h in Headers */,
+                               0757B13E214AE79900794B0D /* VideoPreset.h in Headers */,
                                CD9D827A1C7B8EE1006FF066 /* VideoTextureCopierCV.h in Headers */,
                                CDC939A81E9BDFB100BB768D /* VideoToolboxSoftLink.h in Headers */,
                                BE88E0DF1715D2A200658D98 /* VideoTrack.h in Headers */,
index a213c7c..a7d870e 100644 (file)
@@ -62,6 +62,8 @@ SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreVideo, CVPixelBufferPoolCreatePixelBu
 #define CVPixelBufferPoolCreatePixelBuffer softLink_CoreVideo_CVPixelBufferPoolCreatePixelBuffer
 SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreVideo, CVPixelBufferGetIOSurface, IOSurfaceRef, (CVPixelBufferRef pixelBuffer), (pixelBuffer))
 #define CVPixelBufferGetIOSurface softLink_CoreVideo_CVPixelBufferGetIOSurface
+SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreVideo, CVPixelBufferPoolGetPixelBufferAttributes, CFDictionaryRef, (CVPixelBufferPoolRef pool), (pool))
+#define CVPixelBufferPoolGetPixelBufferAttributes softLink_CoreVideo_CVPixelBufferPoolGetPixelBufferAttributes
 
 SOFT_LINK_CONSTANT_FOR_HEADER(WebCore, CoreVideo, kCVPixelBufferPixelFormatTypeKey, CFStringRef)
 #define kCVPixelBufferPixelFormatTypeKey get_CoreVideo_kCVPixelBufferPixelFormatTypeKey()
index 23c4978..23287aa 100644 (file)
@@ -29,6 +29,7 @@
 #include <wtf/SoftLinking.h>
 
 typedef struct OpaqueVTImageRotationSession* VTImageRotationSessionRef;
+typedef struct OpaqueVTPixelTransferSession* VTPixelTransferSessionRef;
 
 SOFT_LINK_FRAMEWORK_FOR_SOURCE(WebCore, VideoToolbox)
 
@@ -50,3 +51,9 @@ SOFT_LINK_CONSTANT_FOR_SOURCE(WebCore, VideoToolbox, kVTDecompressionPropertyKey
 SOFT_LINK_CONSTANT_FOR_SOURCE(WebCore, VideoToolbox, kVTImageRotationPropertyKey_EnableHighSpeedTransfer, CFStringRef)
 SOFT_LINK_CONSTANT_FOR_SOURCE(WebCore, VideoToolbox, kVTImageRotationPropertyKey_FlipHorizontalOrientation, CFStringRef)
 SOFT_LINK_CONSTANT_FOR_SOURCE(WebCore, VideoToolbox, kVTImageRotationPropertyKey_FlipVerticalOrientation, CFStringRef)
+
+SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, VideoToolbox, VTPixelTransferSessionCreate, OSStatus, (CFAllocatorRef allocator, VTPixelTransferSessionRef* pixelTransferSessionOut), (allocator, pixelTransferSessionOut))
+SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, VideoToolbox, VTPixelTransferSessionTransferImage, OSStatus, (VTPixelTransferSessionRef session, CVPixelBufferRef sourceBuffer, CVPixelBufferRef destinationBuffer), (session, sourceBuffer, destinationBuffer))
+SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, VideoToolbox, VTSessionSetProperty, OSStatus, (VTSessionRef session, CFStringRef propertyKey, CFTypeRef propertyValue), (session, propertyKey, propertyValue))
+SOFT_LINK_CONSTANT_FOR_SOURCE(WebCore, VideoToolbox, kVTPixelTransferPropertyKey_ScalingMode, CFStringRef)
+SOFT_LINK_CONSTANT_FOR_SOURCE(WebCore, VideoToolbox, kVTScalingMode_Trim, CFStringRef)
index 8f46a33..7215d3a 100644 (file)
@@ -29,6 +29,7 @@
 #include <wtf/SoftLinking.h>
 
 typedef struct OpaqueVTImageRotationSession* VTImageRotationSessionRef;
+typedef struct OpaqueVTPixelTransferSession* VTPixelTransferSessionRef;
 
 SOFT_LINK_FRAMEWORK_FOR_HEADER(WebCore, VideoToolbox)
 
@@ -68,3 +69,15 @@ SOFT_LINK_CONSTANT_FOR_HEADER(WebCore, VideoToolbox, kVTImageRotationPropertyKey
 #define kVTImageRotationPropertyKey_FlipHorizontalOrientation get_VideoToolbox_kVTImageRotationPropertyKey_FlipHorizontalOrientation()
 SOFT_LINK_CONSTANT_FOR_HEADER(WebCore, VideoToolbox, kVTImageRotationPropertyKey_FlipVerticalOrientation, CFStringRef)
 #define kVTImageRotationPropertyKey_FlipVerticalOrientation get_VideoToolbox_kVTImageRotationPropertyKey_FlipVerticalOrientation()
+
+SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, VideoToolbox, VTPixelTransferSessionCreate, OSStatus, (CFAllocatorRef allocator, VTPixelTransferSessionRef* pixelTransferSessionOut), (allocator, pixelTransferSessionOut))
+#define VTPixelTransferSessionCreate softLink_VideoToolbox_VTPixelTransferSessionCreate
+SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, VideoToolbox, VTPixelTransferSessionTransferImage, OSStatus, (VTPixelTransferSessionRef session, CVPixelBufferRef sourceBuffer, CVPixelBufferRef destinationBuffer), (session, sourceBuffer, destinationBuffer))
+#define VTPixelTransferSessionTransferImage softLink_VideoToolbox_VTPixelTransferSessionTransferImage
+SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, VideoToolbox, VTSessionSetProperty, OSStatus, (VTSessionRef session, CFStringRef propertyKey, CFTypeRef propertyValue), (session, propertyKey, propertyValue))
+#define VTSessionSetProperty softLink_VideoToolbox_VTSessionSetProperty
+
+SOFT_LINK_CONSTANT_FOR_HEADER(WebCore, VideoToolbox, kVTPixelTransferPropertyKey_ScalingMode, CFStringRef)
+#define kVTPixelTransferPropertyKey_ScalingMode get_VideoToolbox_kVTPixelTransferPropertyKey_ScalingMode()
+SOFT_LINK_CONSTANT_FOR_HEADER(WebCore, VideoToolbox, kVTScalingMode_Trim, CFStringRef)
+#define kVTScalingMode_Trim get_VideoToolbox_kVTScalingMode_Trim()
diff --git a/Source/WebCore/platform/graphics/cv/PixelBufferResizer.h b/Source/WebCore/platform/graphics/cv/PixelBufferResizer.h
new file mode 100644 (file)
index 0000000..db19536
--- /dev/null
@@ -0,0 +1,55 @@
+/*
+ * Copyright (C) 2018 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#if USE(VIDEOTOOLBOX)
+
+#include <wtf/RetainPtr.h>
+
+typedef struct OpaqueVTPixelTransferSession* VTPixelTransferSessionRef;
+typedef struct __CVBuffer *CVPixelBufferRef;
+typedef struct __CVPixelBufferPool *CVPixelBufferPoolRef;
+typedef struct opaqueCMSampleBuffer *CMSampleBufferRef;
+
+namespace WebCore {
+
+class PixelBufferResizer {
+public:
+    PixelBufferResizer(IntSize, OSType);
+
+    RetainPtr<CVPixelBufferRef> resize(CVPixelBufferRef);
+    RetainPtr<CMSampleBufferRef> resize(CMSampleBufferRef);
+    bool canResizeTo(IntSize size) { return size == m_size; }
+
+private:
+    RetainPtr<VTPixelTransferSessionRef> m_transferSession;
+    RetainPtr<CVPixelBufferPoolRef> m_bufferPool;
+    IntSize m_size;
+};
+
+}
+
+#endif // USE(VIDEOTOOLBOX)
diff --git a/Source/WebCore/platform/graphics/cv/PixelBufferResizer.mm b/Source/WebCore/platform/graphics/cv/PixelBufferResizer.mm
new file mode 100644 (file)
index 0000000..a53ab1a
--- /dev/null
@@ -0,0 +1,133 @@
+/*
+ * Copyright (C) 2018 Apple Inc. All rights reserved.
+ * 
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "PixelBufferResizer.h"
+
+#if USE(VIDEOTOOLBOX)
+
+#include <wtf/SoftLinking.h>
+
+#include "CoreVideoSoftLink.h"
+#include "VideoToolboxSoftLink.h"
+
+namespace WebCore {
+
+PixelBufferResizer::PixelBufferResizer(IntSize size, OSType videoFormat)
+{
+    VTPixelTransferSessionRef transferSession;
+    VTPixelTransferSessionCreate(NULL, &transferSession);
+    ASSERT(transferSession);
+    m_transferSession = adoptCF(transferSession);
+
+    VTSessionSetProperty(transferSession, kVTPixelTransferPropertyKey_ScalingMode, kVTScalingMode_Trim);
+
+    CFDictionaryRef sourcePixelBufferOptions = (__bridge CFDictionaryRef) @{
+        (__bridge NSString *)kCVPixelBufferWidthKey :@(size.width()),
+        (__bridge NSString *)kCVPixelBufferHeightKey:@(size.height()),
+        (__bridge NSString *)kCVPixelBufferPixelFormatTypeKey:@(videoFormat),
+#if PLATFORM(IOS)
+        (__bridge NSString *)kCVPixelFormatOpenGLESCompatibility : @(YES),
+#else
+        (__bridge NSString *)kCVPixelBufferOpenGLCompatibilityKey : @(YES),
+#endif
+        (__bridge NSString *)kCVPixelBufferIOSurfacePropertiesKey : @{ /*empty dictionary*/ },
+    };
+    CVPixelBufferPoolRef bufferPool;
+    auto status = CVPixelBufferPoolCreate(kCFAllocatorDefault, NULL, sourcePixelBufferOptions, &bufferPool);
+    ASSERT(!status);
+    if (status != kCVReturnSuccess)
+        return;
+
+    m_bufferPool = adoptCF(bufferPool);
+    m_size = size;
+}
+
+RetainPtr<CVPixelBufferRef> PixelBufferResizer::resize(CVPixelBufferRef inputBuffer)
+{
+    RetainPtr<CVPixelBufferRef> result;
+    CVPixelBufferRef outputBuffer = nullptr;
+
+    auto status = CVPixelBufferPoolCreatePixelBuffer(kCFAllocatorDefault, m_bufferPool.get(), &outputBuffer);
+    if (status) {
+        RELEASE_LOG(Media, "PixelBufferResizer::resize, CVPixelBufferPoolCreatePixelBuffer failed with error %d", static_cast<int>(status));
+        return nullptr;
+    }
+    result = adoptCF(outputBuffer);
+
+    auto err = VTPixelTransferSessionTransferImage(m_transferSession.get(), inputBuffer, outputBuffer);
+    if (err) {
+        RELEASE_LOG(Media, "PixelBufferResizer::resize, VTPixelTransferSessionTransferImage failed with error %d", static_cast<int>(err));
+        return nullptr;
+    }
+
+    return result;
+}
+
+RetainPtr<CMSampleBufferRef> PixelBufferResizer::resize(CMSampleBufferRef sampleBuffer)
+{
+    auto convertedPixelBuffer = resize(CMSampleBufferGetImageBuffer(sampleBuffer));
+    if (!convertedPixelBuffer)
+        return nullptr;
+
+    CMVideoFormatDescriptionRef formatDescription = nullptr;
+    auto status = CMVideoFormatDescriptionCreateForImageBuffer(kCFAllocatorDefault, convertedPixelBuffer.get(), &formatDescription);
+    if (status != noErr) {
+        RELEASE_LOG(Media, "PixelBufferResizer::resize: failed to create CMVideoFormatDescription with error code: %d", static_cast<int>(status));
+        return nullptr;
+    }
+
+    CMItemCount itemCount = 0;
+    status = CMSampleBufferGetSampleTimingInfoArray(sampleBuffer, 1, nullptr, &itemCount);
+    if (status != noErr) {
+        RELEASE_LOG(Media, "PixelBufferResizer::resize: CMSampleBufferGetSampleTimingInfoArray failed with error code: %d", static_cast<int>(status));
+        return nullptr;
+    }
+    Vector<CMSampleTimingInfo> timingInfoArray;
+    CMSampleTimingInfo* timeingInfoPtr = nullptr;
+    if (itemCount) {
+        timingInfoArray.grow(itemCount);
+        status = CMSampleBufferGetSampleTimingInfoArray(sampleBuffer, itemCount, timingInfoArray.data(), nullptr);
+        if (status != noErr) {
+            RELEASE_LOG(Media, "PixelBufferResizer::resize: CMSampleBufferGetSampleTimingInfoArray failed with error code: %d", static_cast<int>(status));
+            return nullptr;
+        }
+        timeingInfoPtr = timingInfoArray.data();
+    }
+
+    CMSampleBufferRef resizedSampleBuffer;
+    status = CMSampleBufferCreateReadyWithImageBuffer(kCFAllocatorDefault, convertedPixelBuffer.get(), formatDescription, timeingInfoPtr, &resizedSampleBuffer);
+    CFRelease(formatDescription);
+    if (status != noErr) {
+        RELEASE_LOG(Media, "PixelBufferResizer::resize: failed to create CMSampleBuffer with error code: %d", static_cast<int>(status));
+        return nullptr;
+    }
+
+    return adoptCF(resizedSampleBuffer);
+}
+
+} // namespace WebCore
+
+#endif // USE(VIDEOTOOLBOX)
index 5233138..b376294 100644 (file)
@@ -73,12 +73,12 @@ void RealtimeIncomingAudioSource::setSourceTrack(rtc::scoped_refptr<webrtc::Audi
         m_audioTrack->AddSink(this);
 }
 
-const RealtimeMediaSourceCapabilities& RealtimeIncomingAudioSource::capabilities() const
+const RealtimeMediaSourceCapabilities& RealtimeIncomingAudioSource::capabilities()
 {
     return RealtimeMediaSourceCapabilities::emptyCapabilities();
 }
 
-const RealtimeMediaSourceSettings& RealtimeIncomingAudioSource::settings() const
+const RealtimeMediaSourceSettings& RealtimeIncomingAudioSource::settings()
 {
     return m_currentSettings;
 }
index 044bfa8..9add53a 100644 (file)
@@ -63,8 +63,8 @@ private:
     void startProducingData() final;
     void stopProducingData()  final;
 
-    const RealtimeMediaSourceCapabilities& capabilities() const final;
-    const RealtimeMediaSourceSettings& settings() const final;
+    const RealtimeMediaSourceCapabilities& capabilities() final;
+    const RealtimeMediaSourceSettings& settings() final;
 
     RealtimeMediaSourceSettings m_currentSettings;
     rtc::scoped_refptr<webrtc::AudioTrackInterface> m_audioTrack;
index f760d40..bda76d3 100644 (file)
@@ -70,12 +70,12 @@ void RealtimeIncomingVideoSource::stopProducingData()
         m_videoTrack->RemoveSink(this);
 }
 
-const RealtimeMediaSourceCapabilities& RealtimeIncomingVideoSource::capabilities() const
+const RealtimeMediaSourceCapabilities& RealtimeIncomingVideoSource::capabilities()
 {
     return RealtimeMediaSourceCapabilities::emptyCapabilities();
 }
 
-const RealtimeMediaSourceSettings& RealtimeIncomingVideoSource::settings() const
+const RealtimeMediaSourceSettings& RealtimeIncomingVideoSource::settings()
 {
     return m_currentSettings;
 }
index 06680ab..4bbc07d 100644 (file)
@@ -67,8 +67,8 @@ private:
     void startProducingData() final;
     void stopProducingData()  final;
 
-    const RealtimeMediaSourceCapabilities& capabilities() const final;
-    const RealtimeMediaSourceSettings& settings() const final;
+    const RealtimeMediaSourceCapabilities& capabilities() final;
+    const RealtimeMediaSourceSettings& settings() final;
 
     rtc::scoped_refptr<webrtc::VideoTrackInterface> m_videoTrack;
 };
index f2793f3..b3ab3e0 100644 (file)
@@ -693,7 +693,7 @@ bool RealtimeMediaSource::selectSettings(const MediaConstraints& constraints, Fl
     return true;
 }
 
-bool RealtimeMediaSource::supportsConstraint(const MediaConstraint& constraint) const
+bool RealtimeMediaSource::supportsConstraint(const MediaConstraint& constraint)
 {
     auto& capabilities = this->capabilities();
 
index 057240f..6d8b962 100644 (file)
@@ -191,8 +191,8 @@ public:
     bool echoCancellation() const { return m_echoCancellation; }
     void setEchoCancellation(bool);
 
-    virtual const RealtimeMediaSourceCapabilities& capabilities() const = 0;
-    virtual const RealtimeMediaSourceSettings& settings() const = 0;
+    virtual const RealtimeMediaSourceCapabilities& capabilities() = 0;
+    virtual const RealtimeMediaSourceSettings& settings() = 0;
 
     using SuccessHandler = WTF::Function<void()>;
     using FailureHandler = WTF::Function<void(const String& badConstraint, const String& errorString)>;
@@ -200,7 +200,7 @@ public:
     std::optional<std::pair<String, String>> applyConstraints(const MediaConstraints&);
 
     bool supportsConstraints(const MediaConstraints&, String&);
-    bool supportsConstraint(const MediaConstraint&) const;
+    bool supportsConstraint(const MediaConstraint&);
 
     virtual void settingsDidChange(OptionSet<RealtimeMediaSourceSettings::Flag>);
 
index b3ad6cb..f5db9b0 100644 (file)
@@ -58,22 +58,85 @@ void RealtimeVideoSource::prepareToProduceData()
         setSize(m_defaultSize);
 }
 
-void RealtimeVideoSource::setSupportedPresets(RealtimeVideoSource::VideoPresets&& presets)
+const Vector<Ref<VideoPreset>>& RealtimeVideoSource::presets()
 {
-    m_presets = WTFMove(presets);
-    std::sort(m_presets.begin(), m_presets.end(),
-        [&] (const auto& a, const auto& b) -> bool {
-            return (a.size.width() * a.size.height()) < (b.size.width() * b.size.height());
+    if (m_presets.isEmpty())
+        generatePresets();
+
+    ASSERT(!m_presets.isEmpty());
+    return m_presets;
+}
+
+void RealtimeVideoSource::setSupportedPresets(Vector<VideoPresetData>&& presetData)
+{
+    Vector<Ref<VideoPreset>> presets;
+
+    for (auto& data : presetData)
+        presets.append(VideoPreset::create(WTFMove(data)));
+
+    setSupportedPresets(WTFMove(presets));
+}
+
+void RealtimeVideoSource::setSupportedPresets(const Vector<Ref<VideoPreset>>& presets)
+{
+    m_presets = WTF::map(presets, [](auto& preset) {
+        return preset.copyRef();
     });
-    
+
     for (auto& preset : m_presets) {
-        std::sort(preset.frameRateRanges.begin(), preset.frameRateRanges.end(),
+        std::sort(preset->frameRateRanges.begin(), preset->frameRateRanges.end(),
             [&] (const auto& a, const auto& b) -> bool {
                 return a.minimum < b.minimum;
         });
     }
 }
 
+static const Vector<IntSize>& standardVideoSizes()
+{
+    static const auto sizes = makeNeverDestroyed([] {
+        static IntSize videoSizes[] = {
+            { 112, 112 },
+            { 160, 160 },
+            { 160, 120 }, // 4:3, QQVGA
+            { 192, 192 },
+            { 192, 112 }, // 16:9
+            { 192, 144 }, // 3:4
+            { 240, 240 },
+            { 240, 160 }, // 3:2, HQVGA
+            { 320, 320 },
+            { 320, 176 }, // 16:9
+            { 320, 240 }, // 4:3, QVGA
+            { 352, 288 }, // CIF
+            { 480, 272 }, // 16:9
+            { 480, 360 }, // 4:3
+            { 480, 480 },
+            { 640, 640 },
+            { 640, 368 }, // 16:9
+            { 640, 480 }, // 4:3
+            { 720, 720 },
+            { 800, 600 }, // 4:3, SVGA
+            { 960, 540 }, // 16:9, qHD
+            { 1024, 600 }, // 16:9, WSVGA
+            { 1024, 768 }, // 4:3, XGA
+            { 1280, 960 }, // 4:3
+            { 1280, 1024 }, // 5:4, SXGA
+            { 1280, 720 }, // 16:9, WXGA
+            { 1366, 768 }, // 16:9, HD
+            { 1920, 1080 }, // 16:9, FHD
+            { 2560, 1440 }, // 16:9, QHD
+            { 2592, 1936 },
+            { 3264, 2448 }, // 3:4
+            { 3840, 2160 }, // 16:9, 4K UHD
+        };
+        Vector<IntSize> sizes;
+        for (auto& size : videoSizes)
+            sizes.append(size);
+
+        return sizes;
+    }());
+
+    return sizes.get();
+}
 template <typename ValueType>
 static void updateMinMax(ValueType& min, ValueType& max, ValueType value)
 {
@@ -81,9 +144,9 @@ static void updateMinMax(ValueType& min, ValueType& max, ValueType value)
     max = std::max<ValueType>(max, value);
 }
 
-void RealtimeVideoSource::addSupportedCapabilities(RealtimeMediaSourceCapabilities& capabilities) const
+void RealtimeVideoSource::updateCapabilities(RealtimeMediaSourceCapabilities& capabilities)
 {
-    ASSERT(!m_presets.isEmpty());
+    ASSERT(!presets().isEmpty());
 
     int minimumWidth = std::numeric_limits<int>::max();
     int maximumWidth = 0;
@@ -93,18 +156,28 @@ void RealtimeVideoSource::addSupportedCapabilities(RealtimeMediaSourceCapabiliti
     double maximumAspectRatio = 0;
     double minimumFrameRate = std::numeric_limits<double>::max();
     double maximumFrameRate = 0;
-    for (const auto& preset : m_presets) {
-        const auto& size = preset.size;
+    for (const auto& preset : presets()) {
+        const auto& size = preset->size;
         updateMinMax(minimumWidth, maximumWidth, size.width());
         updateMinMax(minimumHeight, maximumHeight, size.height());
-
         updateMinMax(minimumAspectRatio, maximumAspectRatio, static_cast<double>(size.width()) / size.height());
 
-        for (const auto& rate : preset.frameRateRanges) {
+        for (const auto& rate : preset->frameRateRanges) {
             updateMinMax(minimumFrameRate, maximumFrameRate, rate.minimum);
             updateMinMax(minimumFrameRate, maximumFrameRate, rate.maximum);
         }
     }
+
+    if (canResizeVideoFrames()) {
+        for (auto& size : standardVideoSizes()) {
+            if (size.width() < minimumWidth || size.height() < minimumHeight) {
+                minimumWidth = std::min(minimumWidth, size.width());
+                minimumHeight = std::min(minimumHeight, size.height());
+                minimumAspectRatio = std::min(minimumAspectRatio, static_cast<double>(size.width()) / size.height());
+            }
+        }
+    }
+
     capabilities.setWidth({ minimumWidth, maximumWidth });
     capabilities.setHeight({ minimumHeight, maximumHeight });
     capabilities.setAspectRatio({ minimumAspectRatio, maximumAspectRatio });
@@ -119,55 +192,174 @@ bool RealtimeVideoSource::supportsSizeAndFrameRate(std::optional<int> width, std
     return !!bestSupportedSizeAndFrameRate(width, height, frameRate);
 }
 
-std::optional<RealtimeVideoSource::CaptureSizeAndFrameRate> RealtimeVideoSource::bestSupportedSizeAndFrameRate(std::optional<int> width, std::optional<int> height, std::optional<double> frameRate)
+bool RealtimeVideoSource::frameRateRangeIncludesRate(const FrameRateRange& range, double frameRate)
 {
-    if (!width && !height && !frameRate)
+    const double epsilon = 0.001;
+    return frameRate + epsilon >= range.minimum && frameRate - epsilon <= range.maximum;
+}
+
+bool RealtimeVideoSource::presetSupportsFrameRate(RefPtr<VideoPreset> preset, double frameRate)
+{
+    for (const auto& range : preset->frameRateRanges) {
+        if (frameRateRangeIncludesRate(range, frameRate))
+            return true;
+    }
+
+    return false;
+}
+
+bool RealtimeVideoSource::supportsCaptureSize(std::optional<int> width, std::optional<int> height, const Function<bool(const IntSize&)>&& function)
+{
+    if (width && height)
+        return function({ width.value(), height.value() });
+
+    if (width) {
+        for (auto& size : standardVideoSizes()) {
+            if (width.value() == size.width() && function({ size.width(), size.height() }))
+                return true;
+        }
+
+        return false;
+    }
+
+    for (auto& size : standardVideoSizes()) {
+        if (height.value() == size.height() && function({ size.width(), size.height() }))
+            return true;
+    }
+
+    return false;
+}
+
+bool RealtimeVideoSource::shouldUsePreset(VideoPreset& current, VideoPreset& candidate)
+{
+    return candidate.size.width() <= current.size.width() && candidate.size.height() <= current.size.height() && prefersPreset(candidate);
+}
+
+std::optional<RealtimeVideoSource::CaptureSizeAndFrameRate> RealtimeVideoSource::bestSupportedSizeAndFrameRate(std::optional<int> requestedWidth, std::optional<int> requestedHeight, std::optional<double> requestedFrameRate)
+{
+    if (!requestedWidth && !requestedHeight && !requestedFrameRate)
         return { };
 
-    CaptureSizeAndFrameRate match;
-    for (const auto& preset : m_presets) {
-        const auto& size = preset.size;
-        if ((width && width.value() != size.width()) || (height && height.value() != size.height()))
+    if (!requestedWidth && !requestedHeight && !size().isEmpty()) {
+        requestedWidth = size().width();
+        requestedHeight = size().height();
+    }
+    if (!requestedFrameRate)
+        requestedFrameRate = frameRate();
+
+    CaptureSizeAndFrameRate result;
+    RefPtr<VideoPreset> exactSizePreset;
+    RefPtr<VideoPreset> aspectRatioPreset;
+    IntSize aspectRatioMatchSize;
+    RefPtr<VideoPreset> resizePreset;
+    IntSize resizeSize;
+
+    for (const auto& preset : presets()) {
+        const auto& presetSize = preset->size;
+
+        if (!presetSupportsFrameRate(&preset.get(), requestedFrameRate.value()))
+            continue;
+
+        if (!requestedWidth && !requestedHeight) {
+            result.requestedFrameRate = requestedFrameRate.value();
+            return result;
+        }
+
+        // Don't look at presets smaller than the requested resolution because we never want to resize larger.
+        if ((requestedWidth && presetSize.width() < requestedWidth.value()) || (requestedHeight && presetSize.height() < requestedHeight.value()))
+            continue;
+
+        auto lookForExactSizeMatch = [&] (const IntSize& size) -> bool {
+            return preset->size == size;
+        };
+        if (supportsCaptureSize(requestedWidth, requestedHeight, WTFMove(lookForExactSizeMatch))) {
+            if (!exactSizePreset || prefersPreset(preset))
+                exactSizePreset = &preset.get();
             continue;
+        }
 
-        match.size = size;
-        if (!frameRate) {
-            match.frameRate = preset.frameRateRanges[preset.frameRateRanges.size() - 1].maximum;
-            return match;
+        IntSize encodingSize;
+        auto lookForAspectRatioMatch = [this, &preset, &encodingSize] (const IntSize& size) -> bool {
+            auto aspectRatio = [] (const IntSize size) -> double {
+                return size.width() / static_cast<double>(size.height());
+            };
+            if (std::abs(aspectRatio(preset->size) - aspectRatio(size)) > 10e-7 || !canResizeVideoFrames())
+                return false;
+
+            encodingSize = size;
+            return true;
+        };
+        if (supportsCaptureSize(requestedWidth, requestedHeight, WTFMove(lookForAspectRatioMatch))) {
+            if (!aspectRatioPreset || shouldUsePreset(*aspectRatioPreset, preset)) {
+                aspectRatioPreset = &preset.get();
+                aspectRatioMatchSize = encodingSize;
+            }
         }
 
-        const double epsilon = 0.001;
-        for (const auto& rate : preset.frameRateRanges) {
-            if (frameRate.value() + epsilon >= rate.minimum && frameRate.value() - epsilon <= rate.maximum) {
-                match.frameRate = frameRate.value();
-                return match;
+        if (requestedWidth && requestedHeight) {
+            const auto& minStandardSize = standardVideoSizes()[0];
+            if (requestedWidth.value() >= minStandardSize.width() && requestedHeight.value() >= minStandardSize.height()) {
+                if (!resizePreset || shouldUsePreset(*resizePreset, preset)) {
+                    resizePreset = &preset.get();
+                    resizeSize = { requestedWidth.value(), requestedHeight.value() };
+                }
+            }
+        } else {
+            for (auto& standardSize : standardVideoSizes()) {
+                if (standardSize.width() > preset->size.width() || standardSize.height() > preset->size.height())
+                    break;
+                if ((requestedWidth && requestedWidth.value() != standardSize.width()) || (requestedHeight && requestedHeight.value() != standardSize.height()))
+                    continue;
+
+                if (!resizePreset || shouldUsePreset(*resizePreset, preset)) {
+                    resizePreset = &preset.get();
+                    resizeSize = standardSize;
+                }
             }
         }
+    }
 
-        break;
+    if (!exactSizePreset && !aspectRatioPreset && !resizePreset)
+        return { };
+
+    result.requestedFrameRate = requestedFrameRate.value();
+    if (exactSizePreset) {
+        result.encodingPreset = exactSizePreset;
+        result.requestedSize = exactSizePreset->size;
+        return result;
+    }
+
+    if (aspectRatioPreset) {
+        result.encodingPreset = aspectRatioPreset;
+        result.requestedSize = aspectRatioMatchSize;
+        return result;
     }
 
-    return { };
+    result.encodingPreset = resizePreset;
+    result.requestedSize = resizeSize;
+    return result;
 }
 
 void RealtimeVideoSource::setSizeAndFrameRate(std::optional<int> width, std::optional<int> height, std::optional<double> frameRate)
 {
     std::optional<RealtimeVideoSource::CaptureSizeAndFrameRate> match;
 
-    // If only the framerate is changing, first see if it is supported with the current width and height.
     auto size = this->size();
-    if (!width && !height && !size.isEmpty())
-        match = bestSupportedSizeAndFrameRate(size.width(), size.height(), frameRate);
-
-    if (!match)
-        match = bestSupportedSizeAndFrameRate(width, height, frameRate);
+    if (!width && !height && !size.isEmpty()) {
+        width = size.width();
+        height = size.height();
+    }
 
+    match = bestSupportedSizeAndFrameRate(width, height, frameRate);
     ASSERT(match);
     if (!match)
         return;
 
-    setSize(match->size);
-    setFrameRate(match->frameRate);
+    setSizeAndFrameRateWithPreset(match->requestedSize, match->requestedFrameRate, match->encodingPreset);
+
+    if (!match->requestedSize.isEmpty())
+        setSize(match->requestedSize);
+    setFrameRate(match->requestedFrameRate);
 }
 
 void RealtimeVideoSource::dispatchMediaSampleToObservers(MediaSample& sample)
index 1fb0334..3ba373a 100644 (file)
 #include "ImageBuffer.h"
 #include "MediaSample.h"
 #include "RealtimeMediaSource.h"
+#include "VideoPreset.h"
 #include <wtf/Lock.h>
 #include <wtf/RunLoop.h>
 
 namespace WebCore {
 
-struct FrameRateRange;
-struct VideoPreset;
-
 class RealtimeVideoSource : public RealtimeMediaSource {
 public:
     virtual ~RealtimeVideoSource();
@@ -50,16 +48,19 @@ protected:
     bool supportsSizeAndFrameRate(std::optional<int> width, std::optional<int> height, std::optional<double>) final;
     void setSizeAndFrameRate(std::optional<int> width, std::optional<int> height, std::optional<double>) final;
 
-    using VideoPresets = Vector<VideoPreset>;
-    void setSupportedPresets(VideoPresets&&);
+    virtual void generatePresets() = 0;
+    virtual bool prefersPreset(VideoPreset&) { return true; }
+    virtual void setSizeAndFrameRateWithPreset(IntSize, double, RefPtr<VideoPreset>) { };
+    virtual bool canResizeVideoFrames() const { return false; }
+    bool shouldUsePreset(VideoPreset& current, VideoPreset& candidate);
 
-    struct CaptureSizeAndFrameRate {
-        IntSize size;
-        double frameRate;
-    };
-    std::optional<CaptureSizeAndFrameRate> bestSupportedSizeAndFrameRate(std::optional<int> width, std::optional<int> height, std::optional<double>);
+    void setSupportedPresets(const Vector<Ref<VideoPreset>>&);
+    void setSupportedPresets(Vector<VideoPresetData>&&);
+    const Vector<Ref<VideoPreset>>& presets();
 
-    void addSupportedCapabilities(RealtimeMediaSourceCapabilities&) const;
+    bool frameRateRangeIncludesRate(const FrameRateRange&, double);
+
+    void updateCapabilities(RealtimeMediaSourceCapabilities&);
 
     void setDefaultSize(const IntSize& size) { m_defaultSize = size; }
 
@@ -68,75 +69,21 @@ protected:
     void dispatchMediaSampleToObservers(MediaSample&);
 
 private:
+    struct CaptureSizeAndFrameRate {
+        RefPtr<VideoPreset> encodingPreset;
+        IntSize requestedSize;
+        double requestedFrameRate { 0 };
+    };
+    bool supportsCaptureSize(std::optional<int>, std::optional<int>, const Function<bool(const IntSize&)>&&);
+    std::optional<CaptureSizeAndFrameRate> bestSupportedSizeAndFrameRate(std::optional<int> width, std::optional<int> height, std::optional<double>);
+    bool presetSupportsFrameRate(RefPtr<VideoPreset>, double);
 
-    VideoPresets m_presets;
+    Vector<Ref<VideoPreset>> m_presets;
     Deque<double> m_observedFrameTimeStamps;
     double m_observedFrameRate { 0 };
     IntSize m_defaultSize;
 };
 
-struct FrameRateRange {
-    double minimum;
-    double maximum;
-
-    template<class Encoder> void encode(Encoder&) const;
-    template<class Decoder> static std::optional<FrameRateRange> decode(Decoder&);
-};
-
-template<class Encoder>
-void FrameRateRange::encode(Encoder& encoder) const
-{
-    encoder << minimum;
-    encoder << maximum;
-}
-
-template <class Decoder>
-std::optional<FrameRateRange> FrameRateRange::decode(Decoder& decoder)
-{
-    std::optional<double> minimum;
-    decoder >> minimum;
-    if (!minimum)
-        return std::nullopt;
-
-    std::optional<double> maximum;
-    decoder >> maximum;
-    if (!maximum)
-        return std::nullopt;
-
-    return FrameRateRange { *minimum, *maximum };
-}
-
-struct VideoPreset {
-    IntSize size;
-    Vector<FrameRateRange> frameRateRanges;
-
-    template<class Encoder> void encode(Encoder&) const;
-    template<class Decoder> static std::optional<VideoPreset> decode(Decoder&);
-};
-
-template<class Encoder>
-void VideoPreset::encode(Encoder& encoder) const
-{
-    encoder << size;
-    encoder << frameRateRanges;
-}
-
-template <class Decoder>
-std::optional<VideoPreset> VideoPreset::decode(Decoder& decoder)
-{
-    std::optional<IntSize> size;
-    decoder >> size;
-    if (!size)
-        return std::nullopt;
-
-    std::optional<Vector<FrameRateRange>> frameRateRanges;
-    decoder >> frameRateRanges;
-    if (!frameRateRanges)
-        return std::nullopt;
-
-    return VideoPreset { *size, *frameRateRanges };
-}
-
 } // namespace WebCore
 
 #endif // ENABLE(MEDIA_STREAM)
diff --git a/Source/WebCore/platform/mediastream/VideoPreset.h b/Source/WebCore/platform/mediastream/VideoPreset.h
new file mode 100644 (file)
index 0000000..9f04098
--- /dev/null
@@ -0,0 +1,133 @@
+/*
+ * Copyright (C) 2018 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#if ENABLE(MEDIA_STREAM)
+
+#include "FontCascade.h"
+#include "ImageBuffer.h"
+#include "MediaSample.h"
+#include "RealtimeMediaSource.h"
+#include <wtf/Lock.h>
+#include <wtf/RunLoop.h>
+
+namespace WebCore {
+
+struct FrameRateRange {
+    double minimum;
+    double maximum;
+
+    template<class Encoder> void encode(Encoder&) const;
+    template<class Decoder> static std::optional<FrameRateRange> decode(Decoder&);
+};
+
+template<class Encoder>
+void FrameRateRange::encode(Encoder& encoder) const
+{
+    encoder << minimum;
+    encoder << maximum;
+}
+
+template <class Decoder>
+std::optional<FrameRateRange> FrameRateRange::decode(Decoder& decoder)
+{
+    std::optional<double> minimum;
+    decoder >> minimum;
+    if (!minimum)
+        return std::nullopt;
+
+    std::optional<double> maximum;
+    decoder >> maximum;
+    if (!maximum)
+        return std::nullopt;
+
+    return FrameRateRange { *minimum, *maximum };
+}
+
+struct VideoPresetData {
+    IntSize size;
+    Vector<FrameRateRange> frameRateRanges;
+
+    template<class Encoder> void encode(Encoder&) const;
+    template<class Decoder> static std::optional<VideoPresetData> decode(Decoder&);
+};
+
+template<class Encoder>
+void VideoPresetData::encode(Encoder& encoder) const
+{
+    encoder << size;
+    encoder << frameRateRanges;
+}
+
+template <class Decoder>
+std::optional<VideoPresetData> VideoPresetData::decode(Decoder& decoder)
+{
+    std::optional<IntSize> size;
+    decoder >> size;
+    if (!size)
+        return std::nullopt;
+
+    std::optional<Vector<FrameRateRange>> frameRateRanges;
+    decoder >> frameRateRanges;
+    if (!frameRateRanges)
+        return std::nullopt;
+
+    return VideoPresetData { *size, *frameRateRanges };
+}
+
+class VideoPreset : public RefCounted<VideoPreset> {
+public:
+    static Ref<VideoPreset> create(VideoPresetData&& data)
+    {
+        return adoptRef(*new VideoPreset(data.size, WTFMove(data.frameRateRanges), Base));
+    }
+
+    enum VideoPresetType {
+        Base,
+        AVCapture
+    };
+
+    IntSize size;
+    Vector<FrameRateRange> frameRateRanges;
+    VideoPresetType type;
+
+protected:
+    VideoPreset(IntSize size, Vector<FrameRateRange>&& frameRateRanges, VideoPresetType type)
+        : size(size)
+        , frameRateRanges(WTFMove(frameRateRanges))
+        , type(type)
+    {
+    }
+};
+
+} // namespace WebCore
+
+SPECIALIZE_TYPE_TRAITS_BEGIN(WebCore::VideoPreset)
+    static bool isType(const WebCore::VideoPreset& preset) { return preset.type == WebCore::VideoPreset::VideoPresetType::Base; }
+SPECIALIZE_TYPE_TRAITS_END()
+
+#endif // ENABLE(MEDIA_STREAM)
+
index 7e21673..584f87b 100644 (file)
@@ -136,7 +136,7 @@ void GStreamerAudioCaptureSource::stopProducingData()
     m_capturer->stop();
 }
 
-const RealtimeMediaSourceCapabilities& GStreamerAudioCaptureSource::capabilities() const
+const RealtimeMediaSourceCapabilities& GStreamerAudioCaptureSource::capabilities()
 {
     if (m_capabilities)
         return m_capabilities.value();
@@ -180,7 +180,7 @@ void GStreamerAudioCaptureSource::settingsDidChange(OptionSet<RealtimeMediaSourc
     RealtimeMediaSource::settingsDidChange(settings);
 }
 
-const RealtimeMediaSourceSettings& GStreamerAudioCaptureSource::settings() const
+const RealtimeMediaSourceSettings& GStreamerAudioCaptureSource::settings()
 {
     if (!m_currentSettings) {
         RealtimeMediaSourceSettings settings;
index 985a2e4..759e814 100644 (file)
@@ -33,8 +33,8 @@ public:
     static CaptureSourceOrError create(const String& deviceID, const MediaConstraints*);
     WEBCORE_EXPORT static AudioCaptureFactory& factory();
 
-    const RealtimeMediaSourceCapabilities& capabilities() const override;
-    const RealtimeMediaSourceSettings& settings() const override;
+    const RealtimeMediaSourceCapabilities& capabilities() override;
+    const RealtimeMediaSourceSettings& settings() override;
 
     GstElement* pipeline() { return m_capturer->pipeline(); }
     GStreamerCapturer* capturer() { return m_capturer.get(); }
index 0758e54..30d157b 100644 (file)
@@ -148,7 +148,7 @@ void GStreamerVideoCaptureSource::stopProducingData()
     setSize({ 0, 0 });
 }
 
-const RealtimeMediaSourceCapabilities& GStreamerVideoCaptureSource::capabilities() const
+const RealtimeMediaSourceCapabilities& GStreamerVideoCaptureSource::capabilities()
 {
     if (m_capabilities)
         return m_capabilities.value();
@@ -234,7 +234,7 @@ const RealtimeMediaSourceCapabilities& GStreamerVideoCaptureSource::capabilities
     return m_capabilities.value();
 }
 
-const RealtimeMediaSourceSettings& GStreamerVideoCaptureSource::settings() const
+const RealtimeMediaSourceSettings& GStreamerVideoCaptureSource::settings()
 {
     if (!m_currentSettings) {
         RealtimeMediaSourceSettings settings;
index 1609b00..751296a 100644 (file)
@@ -32,8 +32,8 @@ public:
     static CaptureSourceOrError create(const String& deviceID, const MediaConstraints*);
     WEBCORE_EXPORT static VideoCaptureFactory& factory();
 
-    const RealtimeMediaSourceCapabilities& capabilities() const override;
-    const RealtimeMediaSourceSettings& settings() const override;
+    const RealtimeMediaSourceCapabilities& capabilities() override;
+    const RealtimeMediaSourceSettings& settings() override;
     GstElement* pipeline() { return m_capturer->pipeline(); }
     GStreamerCapturer* capturer() { return m_capturer.get(); }
 
index 995f057..9f01e67 100644 (file)
@@ -83,12 +83,12 @@ void MockGStreamerAudioCaptureSource::startProducingData()
     m_wrappedSource->start();
 }
 
-const RealtimeMediaSourceSettings& MockGStreamerAudioCaptureSource::settings() const
+const RealtimeMediaSourceSettings& MockGStreamerAudioCaptureSource::settings()
 {
     return m_wrappedSource->settings();
 }
 
-const RealtimeMediaSourceCapabilities& MockGStreamerAudioCaptureSource::capabilities() const
+const RealtimeMediaSourceCapabilities& MockGStreamerAudioCaptureSource::capabilities()
 {
     m_capabilities = m_wrappedSource->capabilities();
     m_currentSettings = m_wrappedSource->settings();
index bc11f17..b57181d 100644 (file)
@@ -37,8 +37,8 @@ public:
 private:
     void stopProducingData() final;
     void startProducingData() final;
-    const RealtimeMediaSourceSettings& settings() const final;
-    const RealtimeMediaSourceCapabilities& capabilities() const final;
+    const RealtimeMediaSourceSettings& settings() final;
+    const RealtimeMediaSourceCapabilities& capabilities() final;
 
     void captureFailed();
     std::unique_ptr<RealtimeMediaSource> m_wrappedSource;
index ead7272..44d94cc 100644 (file)
@@ -118,12 +118,12 @@ void MockGStreamerVideoCaptureSource::applyConstraints(const MediaConstraints& c
     m_wrappedSource->applyConstraints(constraints, WTFMove(successHandler), WTFMove(failureHandler));
 }
 
-const RealtimeMediaSourceSettings& MockGStreamerVideoCaptureSource::settings() const
+const RealtimeMediaSourceSettings& MockGStreamerVideoCaptureSource::settings()
 {
     return m_wrappedSource->settings();
 }
 
-const RealtimeMediaSourceCapabilities& MockGStreamerVideoCaptureSource::capabilities() const
+const RealtimeMediaSourceCapabilities& MockGStreamerVideoCaptureSource::capabilities()
 {
     m_capabilities = m_wrappedSource->capabilities();
     m_currentSettings = m_wrappedSource->settings();
index ed3227f..d08aa48 100644 (file)
@@ -42,9 +42,9 @@ public:
 private:
     void stopProducingData() final;
     void startProducingData() final;
-    const RealtimeMediaSourceSettings& settings() const final;
+    const RealtimeMediaSourceSettings& settings() final;
     std::unique_ptr<RealtimeMediaSource> m_wrappedSource;
-    const RealtimeMediaSourceCapabilities& capabilities() const final;
+    const RealtimeMediaSourceCapabilities& capabilities() final;
     void captureFailed() override;
 
     void videoSampleAvailable(MediaSample&) override;
index 9f2fc34..d38e1ae 100644 (file)
 
 #include "IntSizeHash.h"
 #include "OrientationNotifier.h"
-#include "RealtimeMediaSource.h"
+#include "RealtimeVideoSource.h"
+#include <wtf/Lock.h>
 #include <wtf/text/StringHash.h>
 
 typedef struct opaqueCMSampleBuffer* CMSampleBufferRef;
 
 OBJC_CLASS AVCaptureConnection;
 OBJC_CLASS AVCaptureDevice;
+OBJC_CLASS AVCaptureDeviceFormat;
 OBJC_CLASS AVCaptureOutput;
 OBJC_CLASS AVCaptureSession;
 OBJC_CLASS AVCaptureVideoDataOutput;
+OBJC_CLASS AVFrameRateRange;
 OBJC_CLASS NSError;
 OBJC_CLASS NSNotification;
 OBJC_CLASS WebCoreAVVideoCaptureSourceObserver;
 
 namespace WebCore {
 
-class AVVideoCaptureSource : public RealtimeMediaSource, private OrientationNotifier::Observer {
+class AVVideoPreset;
+class PixelBufferResizer;
+
+class AVVideoCaptureSource : public RealtimeVideoSource, private OrientationNotifier::Observer {
 public:
     static CaptureSourceOrError create(const AtomicString&, const MediaConstraints*);
 
@@ -72,13 +78,10 @@ private:
     bool setupCaptureSession();
     void shutdownCaptureSession();
 
-    const RealtimeMediaSourceCapabilities& capabilities() const final;
-    void setSizeAndFrameRate(std::optional<int> width, std::optional<int> height, std::optional<double>) final;
-    void setFrameRate(double);
-    const RealtimeMediaSourceSettings& settings() const final;
+    const RealtimeMediaSourceCapabilities& capabilities() final;
+    const RealtimeMediaSourceSettings& settings() final;
     void startProducingData() final;
     void stopProducingData() final;
-    bool supportsSizeAndFrameRate(std::optional<int> width, std::optional<int> height, std::optional<double>) final;
     void settingsDidChange(OptionSet<RealtimeMediaSourceSettings::Flag>) final;
     void monitorOrientation(OrientationNotifier&) final;
     void beginConfiguration() final;
@@ -86,12 +89,14 @@ private:
     bool isCaptureSource() const final { return true; }
     bool interrupted() const final;
 
+    void setSizeAndFrameRateWithPreset(IntSize, double, RefPtr<VideoPreset>) final;
+    bool prefersPreset(VideoPreset&) final;
+    void generatePresets() final;
+    bool canResizeVideoFrames() const final { return true; }
+
     bool setPreset(NSString*);
     void computeSampleRotation();
-
-    bool isFrameRateSupported(double frameRate);
-
-    NSString* bestSessionPresetForVideoDimensions(std::optional<int> width, std::optional<int> height);
+    AVFrameRateRange* frameDurationForFrameRate(double);
 
     // OrientationNotifier::Observer API
     void orientationChanged(int orientation) final;
@@ -101,34 +106,38 @@ private:
     void processNewFrame(RetainPtr<CMSampleBufferRef>, RetainPtr<AVCaptureConnection>);
     IntSize sizeForPreset(NSString*);
 
-    using VideoPresetMap = HashMap<String, IntSize>;
-    VideoPresetMap& videoPresets() { return m_supportedPresets; }
-
     AVCaptureDevice* device() const { return m_device.get(); }
 
-    RetainPtr<NSString> m_pendingPreset;
     RetainPtr<CMSampleBufferRef> m_buffer;
     RetainPtr<AVCaptureVideoDataOutput> m_videoOutput;
 
-    IntSize m_presetSize;
+    IntSize m_requestedSize;
     int32_t m_width { 0 };
     int32_t m_height { 0 };
     int m_sensorOrientation { 0 };
     int m_deviceOrientation { 0 };
     MediaSample::VideoRotation m_sampleRotation { MediaSample::VideoRotation::None };
 
-    VideoPresetMap m_supportedPresets;
-
-    mutable std::optional<RealtimeMediaSourceSettings> m_currentSettings;
-    mutable std::optional<RealtimeMediaSourceCapabilities> m_capabilities;
+    std::unique_ptr<PixelBufferResizer> m_pixelBufferResizer;
+    std::optional<RealtimeMediaSourceSettings> m_currentSettings;
+    std::optional<RealtimeMediaSourceCapabilities> m_capabilities;
     RetainPtr<WebCoreAVVideoCaptureSourceObserver> m_objcObserver;
     RetainPtr<AVCaptureSession> m_session;
     RetainPtr<AVCaptureDevice> m_device;
+    RefPtr<VideoPreset> m_pendingPreset;
+
+    Lock m_presetMutex;
+    RefPtr<AVVideoPreset> m_currentPreset;
+    IntSize m_pendingSize;
+    double m_pendingFrameRate;
     InterruptionReason m_interruption { InterruptionReason::None };
     bool m_isRunning { false };
-
 };
 
 } // namespace WebCore
 
+SPECIALIZE_TYPE_TRAITS_BEGIN(WebCore::AVVideoPreset)
+    static bool isType(const WebCore::VideoPreset& preset) { return preset.type == WebCore::VideoPreset::VideoPresetType::AVCapture; }
+SPECIALIZE_TYPE_TRAITS_END()
+
 #endif // ENABLE(MEDIA_STREAM)
index f3bd410..adf1ff3 100644 (file)
@@ -33,6 +33,7 @@
 #import "Logging.h"
 #import "MediaConstraints.h"
 #import "MediaSampleAVFObjC.h"
+#import "PixelBufferResizer.h"
 #import "PlatformLayer.h"
 #import "RealtimeMediaSourceCenterMac.h"
 #import "RealtimeMediaSourceSettings.h"
@@ -75,29 +76,14 @@ SOFT_LINK_CLASS(AVFoundation, AVCaptureSession)
 #define AVFrameRateRange getAVFrameRateRangeClass()
 
 SOFT_LINK_CONSTANT(AVFoundation, AVMediaTypeVideo, NSString *)
-SOFT_LINK_CONSTANT_MAY_FAIL(AVFoundation, AVCaptureSessionPreset1280x720, NSString *)
-SOFT_LINK_CONSTANT_MAY_FAIL(AVFoundation, AVCaptureSessionPreset960x540, NSString *)
-SOFT_LINK_CONSTANT_MAY_FAIL(AVFoundation, AVCaptureSessionPreset640x480, NSString *)
-SOFT_LINK_CONSTANT_MAY_FAIL(AVFoundation, AVCaptureSessionPreset352x288, NSString *)
-SOFT_LINK_CONSTANT_MAY_FAIL(AVFoundation, AVCaptureSessionPreset320x240, NSString *)
-
-#define AVCaptureSessionPreset1280x720 getAVCaptureSessionPreset1280x720()
-#define AVCaptureSessionPreset960x540 getAVCaptureSessionPreset960x540()
-#define AVCaptureSessionPreset640x480 getAVCaptureSessionPreset640x480()
-#define AVCaptureSessionPreset352x288 getAVCaptureSessionPreset352x288()
-#define AVCaptureSessionPreset320x240 getAVCaptureSessionPreset320x240()
 
 #if PLATFORM(IOS)
-SOFT_LINK_CONSTANT_MAY_FAIL(AVFoundation, AVCaptureSessionPreset3840x2160, NSString *)
-SOFT_LINK_CONSTANT_MAY_FAIL(AVFoundation, AVCaptureSessionPreset1920x1080, NSString *)
 SOFT_LINK_POINTER_OPTIONAL(AVFoundation, AVCaptureSessionRuntimeErrorNotification, NSString *)
 SOFT_LINK_POINTER_OPTIONAL(AVFoundation, AVCaptureSessionWasInterruptedNotification, NSString *)
 SOFT_LINK_POINTER_OPTIONAL(AVFoundation, AVCaptureSessionInterruptionEndedNotification, NSString *)
 SOFT_LINK_POINTER_OPTIONAL(AVFoundation, AVCaptureSessionInterruptionReasonKey, NSString *)
 SOFT_LINK_POINTER_OPTIONAL(AVFoundation, AVCaptureSessionErrorKey, NSString *)
 
-#define AVCaptureSessionPreset3840x2160 getAVCaptureSessionPreset3840x2160()
-#define AVCaptureSessionPreset1920x1080 getAVCaptureSessionPreset1920x1080()
 #define AVCaptureSessionRuntimeErrorNotification getAVCaptureSessionRuntimeErrorNotification()
 #define AVCaptureSessionWasInterruptedNotification getAVCaptureSessionWasInterruptedNotification()
 #define AVCaptureSessionInterruptionEndedNotification getAVCaptureSessionInterruptionEndedNotification()
@@ -106,6 +92,7 @@ SOFT_LINK_POINTER_OPTIONAL(AVFoundation, AVCaptureSessionErrorKey, NSString *)
 #endif
 
 using namespace WebCore;
+using namespace PAL;
 
 @interface WebCoreAVVideoCaptureSourceObserver : NSObject<AVCaptureVideoDataOutputSampleBufferDelegate> {
     AVVideoCaptureSource* m_callback;
@@ -142,6 +129,22 @@ static dispatch_queue_t globaVideoCaptureSerialQueue()
     return globalQueue;
 }
 
+class AVVideoPreset : public VideoPreset {
+public:
+    static Ref<AVVideoPreset> create(IntSize size, Vector<FrameRateRange>&& frameRateRanges, AVCaptureDeviceFormatType* format)
+    {
+        return adoptRef(*new AVVideoPreset(size, WTFMove(frameRateRanges), format));
+    }
+
+    AVVideoPreset(IntSize size, Vector<FrameRateRange>&& frameRateRanges, AVCaptureDeviceFormatType* format)
+        : VideoPreset(size, WTFMove(frameRateRanges), AVCapture)
+        , format(format)
+    {
+    }
+
+    RetainPtr<AVCaptureDeviceFormatType> format;
+};
+
 CaptureSourceOrError AVVideoCaptureSource::create(const AtomicString& id, const MediaConstraints* constraints)
 {
     AVCaptureDeviceTypedef *device = [getAVCaptureDeviceClass() deviceWithUniqueID:id];
@@ -159,46 +162,10 @@ CaptureSourceOrError AVVideoCaptureSource::create(const AtomicString& id, const
 }
 
 AVVideoCaptureSource::AVVideoCaptureSource(AVCaptureDeviceTypedef* device, const AtomicString& id)
-    : RealtimeMediaSource(id, Type::Video, device.localizedName)
+    : RealtimeVideoSource(id, device.localizedName)
     , m_objcObserver(adoptNS([[WebCoreAVVideoCaptureSourceObserver alloc] initWithCallback:this]))
     , m_device(device)
 {
-    struct VideoPreset {
-        NSString* name;
-        int width;
-        int height;
-    };
-
-    static NeverDestroyed<Vector<VideoPreset>> presets = [] {
-        Vector<VideoPreset> sets;
-#if PLATFORM(IOS)
-        if (canLoadAVCaptureSessionPreset3840x2160())
-            sets.append({ AVCaptureSessionPreset3840x2160, 3840, 2160 });
-        if (canLoadAVCaptureSessionPreset1920x1080())
-            sets.append({ AVCaptureSessionPreset1920x1080, 1920, 1080 });
-#endif
-        if (canLoadAVCaptureSessionPreset1280x720())
-            sets.append({ AVCaptureSessionPreset1280x720, 1280, 720 });
-        if (canLoadAVCaptureSessionPreset960x540())
-            sets.append({ AVCaptureSessionPreset960x540, 960, 540 });
-        if (canLoadAVCaptureSessionPreset640x480())
-            sets.append({ AVCaptureSessionPreset640x480, 640, 480 });
-        if (canLoadAVCaptureSessionPreset352x288())
-            sets.append({ AVCaptureSessionPreset352x288, 352, 288 });
-        if (canLoadAVCaptureSessionPreset320x240())
-            sets.append({ AVCaptureSessionPreset320x240, 320, 240 });
-
-        return sets;
-    }();
-
-    auto* presetsMap = &videoPresets();
-    for (auto& preset : presets.get()) {
-        if (![device supportsAVCaptureSessionPreset:preset.name])
-            continue;
-
-        presetsMap->add(String(preset.name), IntSize(preset.width, preset.height));
-    }
-
 #if PLATFORM(IOS)
     static_assert(static_cast<int>(InterruptionReason::VideoNotAllowedInBackground) == static_cast<int>(AVCaptureSessionInterruptionReasonVideoDeviceNotAvailableInBackground), "InterruptionReason::VideoNotAllowedInBackground is not AVCaptureSessionInterruptionReasonVideoDeviceNotAvailableInBackground as expected");
     static_assert(static_cast<int>(InterruptionReason::VideoNotAllowedInSideBySide) == AVCaptureSessionInterruptionReasonVideoDeviceNotAvailableWithMultipleForegroundApps, "InterruptionReason::VideoNotAllowedInSideBySide is not AVCaptureSessionInterruptionReasonVideoDeviceNotAvailableWithMultipleForegroundApps as expected");
@@ -255,18 +222,6 @@ void AVVideoCaptureSource::stopProducingData()
 #endif
 }
 
-static void updateSizeMinMax(int& min, int& max, int value)
-{
-    min = std::min<int>(min, value);
-    max = std::max<int>(max, value);
-}
-
-static void updateAspectRatioMinMax(double& min, double& max, double value)
-{
-    min = std::min<double>(min, value);
-    max = std::max<double>(max, value);
-}
-
 void AVVideoCaptureSource::beginConfiguration()
 {
     if (m_session)
@@ -285,7 +240,7 @@ void AVVideoCaptureSource::settingsDidChange(OptionSet<RealtimeMediaSourceSettin
     RealtimeMediaSource::settingsDidChange(settings);
 }
 
-const RealtimeMediaSourceSettings& AVVideoCaptureSource::settings() const
+const RealtimeMediaSourceSettings& AVVideoCaptureSource::settings()
 {
     if (m_currentSettings)
         return *m_currentSettings;
@@ -298,10 +253,10 @@ const RealtimeMediaSourceSettings& AVVideoCaptureSource::settings() const
     else
         settings.setFacingMode(RealtimeMediaSourceSettings::Unknown);
 
-    auto maxFrameDuration = [device() activeVideoMaxFrameDuration];
-    settings.setFrameRate(maxFrameDuration.timescale / maxFrameDuration.value);
-    settings.setWidth(m_width);
-    settings.setHeight(m_height);
+    settings.setFrameRate(frameRate());
+    auto& size = this->size();
+    settings.setWidth(size.width());
+    settings.setHeight(size.height());
     settings.setDeviceId(id());
 
     RealtimeMediaSourceSupportedConstraints supportedConstraints;
@@ -319,147 +274,92 @@ const RealtimeMediaSourceSettings& AVVideoCaptureSource::settings() const
     return *m_currentSettings;
 }
 
-const RealtimeMediaSourceCapabilities& AVVideoCaptureSource::capabilities() const
+const RealtimeMediaSourceCapabilities& AVVideoCaptureSource::capabilities()
 {
     if (m_capabilities)
         return *m_capabilities;
 
     RealtimeMediaSourceCapabilities capabilities(settings().supportedConstraints());
     capabilities.setDeviceId(id());
-    AVCaptureDeviceTypedef *videoDevice = device();
 
+    AVCaptureDeviceTypedef *videoDevice = device();
     if ([videoDevice position] == AVCaptureDevicePositionFront)
         capabilities.addFacingMode(RealtimeMediaSourceSettings::User);
     if ([videoDevice position] == AVCaptureDevicePositionBack)
         capabilities.addFacingMode(RealtimeMediaSourceSettings::Environment);
 
-    Float64 lowestFrameRateRange = std::numeric_limits<double>::infinity();
-    Float64 highestFrameRateRange = 0;
-    int minimumWidth = std::numeric_limits<int>::infinity();
-    int maximumWidth = 0;
-    int minimumHeight = std::numeric_limits<int>::infinity();
-    int maximumHeight = 0;
-    double minimumAspectRatio = std::numeric_limits<double>::infinity();
-    double maximumAspectRatio = 0;
-
-    for (AVCaptureDeviceFormatType *format in [videoDevice formats]) {
-
-        for (AVFrameRateRangeType *range in [format videoSupportedFrameRateRanges]) {
-            lowestFrameRateRange = std::min<Float64>(lowestFrameRateRange, range.minFrameRate);
-            highestFrameRateRange = std::max<Float64>(highestFrameRateRange, range.maxFrameRate);
-        }
-    }
-
-    for (auto& preset : m_supportedPresets) {
-        auto values = preset.value;
-        updateSizeMinMax(minimumWidth, maximumWidth, values.width());
-        updateSizeMinMax(minimumHeight, maximumHeight, values.height());
-        updateAspectRatioMinMax(minimumAspectRatio, maximumAspectRatio, static_cast<double>(values.width()) / values.height());
-    }
-    capabilities.setFrameRate(CapabilityValueOrRange(lowestFrameRateRange, highestFrameRateRange));
-    capabilities.setWidth(CapabilityValueOrRange(minimumWidth, maximumWidth));
-    capabilities.setHeight(CapabilityValueOrRange(minimumHeight, maximumHeight));
-    capabilities.setAspectRatio(CapabilityValueOrRange(minimumAspectRatio, maximumAspectRatio));
+    updateCapabilities(capabilities);
 
     m_capabilities = WTFMove(capabilities);
 
     return *m_capabilities;
 }
 
-IntSize AVVideoCaptureSource::sizeForPreset(NSString* preset)
+bool AVVideoCaptureSource::prefersPreset(VideoPreset& preset)
 {
-    if (!preset)
-        return { };
-
-    auto& presets = videoPresets();
-    auto it = presets.find(String(preset));
-    if (it != presets.end())
-        return { it->value.width(), it->value.height() };
+#if PLATFORM(IOS)
+    return ![static_cast<AVVideoPreset*>(&preset)->format.get() isVideoBinned];
+#else
+    UNUSED_PARAM(preset);
+#endif
 
-    return { };
+    return true;
 }
 
-bool AVVideoCaptureSource::setPreset(NSString *preset)
+void AVVideoCaptureSource::setSizeAndFrameRateWithPreset(IntSize requestedSize, double requestedFrameRate, RefPtr<VideoPreset> preset)
 {
-    if (!session()) {
-        m_pendingPreset = preset;
-        return true;
-    }
-
-    auto size = sizeForPreset(preset);
-    if (m_presetSize == size)
-        return true;
-
-    m_presetSize = size;
+    auto* avPreset = preset ? downcast<AVVideoPreset>(preset.get()) : nullptr;
 
-    @try {
-        session().sessionPreset = preset;
-#if PLATFORM(MAC)
-        auto settingsDictionary = @{
-            (__bridge NSString *)kCVPixelBufferPixelFormatTypeKey: @(videoCaptureFormat),
-            (__bridge NSString *)kCVPixelBufferWidthKey: @(size.width()),
-            (__bridge NSString *)kCVPixelBufferHeightKey: @(size.height())
-        };
-        [m_videoOutput setVideoSettings:settingsDictionary];
-#endif
-    } @catch(NSException *exception) {
-        LOG(Media, "AVVideoCaptureSource::setPreset(%p), exception thrown configuring device: <%s> %s", this, [[exception name] UTF8String], [[exception reason] UTF8String]);
-        return false;
+    if (!m_session) {
+        m_pendingPreset = avPreset;
+        m_pendingSize = requestedSize;
+        m_pendingFrameRate = requestedFrameRate;
+        return;
     }
 
-    return true;
-}
+    m_pendingPreset = nullptr;
+    m_pendingFrameRate = 0;
 
-void AVVideoCaptureSource::setFrameRate(double rate)
-{
-    using namespace PAL;
-    double epsilon = 0.00001;
-    AVFrameRateRangeType *bestFrameRateRange = nil;
-    for (AVFrameRateRangeType *frameRateRange in [[device() activeFormat] videoSupportedFrameRateRanges]) {
-        if (rate + epsilon >= [frameRateRange minFrameRate] && rate - epsilon <= [frameRateRange maxFrameRate]) {
-            if (!bestFrameRateRange || CMTIME_COMPARE_INLINE([frameRateRange minFrameDuration], >, [bestFrameRateRange minFrameDuration]))
-                bestFrameRateRange = frameRateRange;
-        }
-    }
+    auto* frameRateRange = frameDurationForFrameRate(requestedFrameRate);
+    ASSERT(frameRateRange);
+    if (!frameRateRange)
+        return;
 
-    if (!bestFrameRateRange || !isFrameRateSupported(rate)) {
-        LOG(Media, "AVVideoCaptureSource::setFrameRate(%p), frame rate %f not supported by video device", this, rate);
+    if (!avPreset)
         return;
-    }
+
+    ASSERT(avPreset->format);
+
+    m_requestedSize = requestedSize;
 
     NSError *error = nil;
+    [m_session beginConfiguration];
     @try {
         if ([device() lockForConfiguration:&error]) {
-            if (bestFrameRateRange.minFrameRate == bestFrameRateRange.maxFrameRate) {
-                [device() setActiveVideoMinFrameDuration:[bestFrameRateRange minFrameDuration]];
-                [device() setActiveVideoMaxFrameDuration:[bestFrameRateRange maxFrameDuration]];
-            } else {
-                [device() setActiveVideoMinFrameDuration: CMTimeMake(1, rate)];
-                [device() setActiveVideoMaxFrameDuration: CMTimeMake(1, rate)];
+            if (!m_currentPreset || ![m_currentPreset->format.get() isEqual:avPreset->format.get()]) {
+                [device() setActiveFormat:avPreset->format.get()];
+#if PLATFORM(MAC)
+                auto settingsDictionary = @{
+                    (__bridge NSString *)kCVPixelBufferPixelFormatTypeKey: @(videoCaptureFormat),
+                    (__bridge NSString *)kCVPixelBufferWidthKey: @(avPreset->size.width()),
+                    (__bridge NSString *)kCVPixelBufferHeightKey: @(avPreset->size.height())
+                };
+                [m_videoOutput setVideoSettings:settingsDictionary];
+#endif
             }
+            [device() setActiveVideoMinFrameDuration:[frameRateRange minFrameDuration]];
+            [device() setActiveVideoMaxFrameDuration:[frameRateRange maxFrameDuration]];
             [device() unlockForConfiguration];
         }
     } @catch(NSException *exception) {
-        LOG(Media, "AVVideoCaptureSource::setFrameRate(%p), exception thrown configuring device: <%s> %s", this, [[exception name] UTF8String], [[exception reason] UTF8String]);
-        return;
+        RELEASE_LOG(Media, "AVVideoCaptureSource::setFrameRate - exception thrown configuring device: <%s> %s", [[exception name] UTF8String], [[exception reason] UTF8String]);
     }
+    [m_session commitConfiguration];
 
-    if (error) {
-        LOG(Media, "AVVideoCaptureSource::setFrameRate(%p), failed to lock video device for configuration: %s", this, [[error localizedDescription] UTF8String]);
-        return;
-    }
-
-    LOG(Media, "AVVideoCaptureSource::setFrameRate(%p) - set frame rate range to %f", this, rate);
-    return;
-}
-
-void AVVideoCaptureSource::setSizeAndFrameRate(std::optional<int> width, std::optional<int> height, std::optional<double> frameRate)
-{
-    if (width || height)
-        setPreset(bestSessionPresetForVideoDimensions(WTFMove(width), WTFMove(height)));
+    m_currentPreset = avPreset;
 
-    if (frameRate)
-        setFrameRate(frameRate.value());
+    if (error)
+        RELEASE_LOG(Media, "AVVideoCaptureSource::setFrameRate - failed to lock video device for configuration: %s", [[error localizedDescription] UTF8String]);
 }
 
 static inline int sensorOrientation(AVCaptureVideoOrientation videoOrientation)
@@ -513,6 +413,22 @@ bool AVVideoCaptureSource::setupSession()
     return success;
 }
 
+AVFrameRateRangeType* AVVideoCaptureSource::frameDurationForFrameRate(double rate)
+{
+    AVFrameRateRangeType *bestFrameRateRange = nil;
+    for (AVFrameRateRangeType *frameRateRange in [[device() activeFormat] videoSupportedFrameRateRanges]) {
+        if (frameRateRangeIncludesRate({ [frameRateRange minFrameRate], [frameRateRange maxFrameRate] }, rate)) {
+            if (!bestFrameRateRange || CMTIME_COMPARE_INLINE([frameRateRange minFrameDuration], >, [bestFrameRateRange minFrameDuration]))
+                bestFrameRateRange = frameRateRange;
+        }
+    }
+
+    if (!bestFrameRateRange)
+        RELEASE_LOG(Media, "AVVideoCaptureSource::frameDurationForFrameRate, no frame rate range for rate %g", rate);
+
+    return bestFrameRateRange;
+}
+
 bool AVVideoCaptureSource::setupCaptureSession()
 {
 #if PLATFORM(IOS)
@@ -534,13 +450,6 @@ bool AVVideoCaptureSource::setupCaptureSession()
 
     m_videoOutput = adoptNS([allocAVCaptureVideoDataOutputInstance() init]);
     auto settingsDictionary = adoptNS([[NSMutableDictionary alloc] initWithObjectsAndKeys: [NSNumber numberWithInt:videoCaptureFormat], kCVPixelBufferPixelFormatTypeKey, nil]);
-    if (m_pendingPreset) {
-#if PLATFORM(MAC)
-        auto size = sizeForPreset(m_pendingPreset.get());
-        [settingsDictionary setObject:@(size.width()) forKey:(__bridge NSString *)kCVPixelBufferWidthKey];
-        [settingsDictionary setObject:@(size.height()) forKey:(__bridge NSString *)kCVPixelBufferHeightKey];
-#endif
-    }
 
     [m_videoOutput setVideoSettings:settingsDictionary.get()];
     [m_videoOutput setAlwaysDiscardsLateVideoFrames:YES];
@@ -552,9 +461,8 @@ bool AVVideoCaptureSource::setupCaptureSession()
     }
     [session() addOutput:m_videoOutput.get()];
 
-#if PLATFORM(IOS)
-    setPreset(m_pendingPreset.get());
-#endif
+    if (m_pendingPreset || m_pendingFrameRate)
+        setSizeAndFrameRateWithPreset(m_pendingSize, m_pendingFrameRate, m_pendingPreset);
 
     m_sensorOrientation = sensorOrientationFromVideoOutput(m_videoOutput.get());
     computeSampleRotation();
@@ -615,25 +523,20 @@ void AVVideoCaptureSource::processNewFrame(RetainPtr<CMSampleBufferRef> sampleBu
     if (!isProducingData() || muted())
         return;
 
-    CMFormatDescriptionRef formatDescription = PAL::CMSampleBufferGetFormatDescription(sampleBuffer.get());
+
+    CMFormatDescriptionRef formatDescription = CMSampleBufferGetFormatDescription(sampleBuffer.get());
     if (!formatDescription)
         return;
 
     m_buffer = sampleBuffer;
-    CMVideoDimensions dimensions = PAL::CMVideoFormatDescriptionGetDimensions(formatDescription);
+    CMVideoDimensions dimensions = CMVideoFormatDescriptionGetDimensions(formatDescription);
     if (m_sampleRotation == MediaSample::VideoRotation::Left || m_sampleRotation == MediaSample::VideoRotation::Right)
         std::swap(dimensions.width, dimensions.height);
 
     if (dimensions.width != m_width || dimensions.height != m_height) {
-        OptionSet<RealtimeMediaSourceSettings::Flag> changed;
-        if (m_width != dimensions.width)
-            changed.add(RealtimeMediaSourceSettings::Flag::Width);
-        if (m_height != dimensions.height)
-            changed.add(RealtimeMediaSourceSettings::Flag::Height);
-
         m_width = dimensions.width;
         m_height = dimensions.height;
-        settingsDidChange(changed);
+        setSize({ dimensions.width, dimensions.height });
     }
 
     videoSampleAvailable(MediaSampleAVFObjC::create(m_buffer.get(), m_sampleRotation, [connection isVideoMirrored]));
@@ -641,55 +544,30 @@ void AVVideoCaptureSource::processNewFrame(RetainPtr<CMSampleBufferRef> sampleBu
 
 void AVVideoCaptureSource::captureOutputDidOutputSampleBufferFromConnection(AVCaptureOutputType*, CMSampleBufferRef sampleBuffer, AVCaptureConnectionType* captureConnection)
 {
-    RetainPtr<CMSampleBufferRef> buffer = sampleBuffer;
-    RetainPtr<AVCaptureConnectionType> connection = captureConnection;
-
-    scheduleDeferredTask([this, buffer, connection] {
-        this->processNewFrame(buffer, connection);
-    });
-}
-
-NSString* AVVideoCaptureSource::bestSessionPresetForVideoDimensions(std::optional<int> width, std::optional<int> height)
-{
-    if (!width && !height)
-        return nil;
-
-    int widthValue = width ? width.value() : 0;
-    int heightValue = height ? height.value() : 0;
+    CMFormatDescriptionRef formatDescription = CMSampleBufferGetFormatDescription(sampleBuffer);
+    if (!formatDescription)
+        return;
 
-    for (auto& preset : videoPresets()) {
-        auto size = preset.value;
-        NSString* name = preset.key;
+    CMVideoDimensions dimensions = CMVideoFormatDescriptionGetDimensions(formatDescription);
+    if (dimensions.width != m_requestedSize.width() || dimensions.height != m_requestedSize.height()) {
+        if (m_pixelBufferResizer && !m_pixelBufferResizer->canResizeTo(m_requestedSize))
+            m_pixelBufferResizer = nullptr;
 
-        if ((!widthValue || widthValue == size.width()) && (!heightValue || heightValue == size.height()))
-            return name;
-    }
+        if (!m_pixelBufferResizer)
+            m_pixelBufferResizer = std::make_unique<PixelBufferResizer>(m_requestedSize, videoCaptureFormat);
+    } else
+        m_pixelBufferResizer = nullptr;
 
-    return nil;
-}
-
-bool AVVideoCaptureSource::isFrameRateSupported(double frameRate)
-{
-    double epsilon = 0.001;
-    for (AVFrameRateRangeType *range in [[device() activeFormat] videoSupportedFrameRateRanges]) {
-        if (frameRate + epsilon >= range.minFrameRate && frameRate - epsilon <= range.maxFrameRate)
-            return true;
+    auto buffer = retainPtr(sampleBuffer);
+    if (m_pixelBufferResizer) {
+        buffer = m_pixelBufferResizer->resize(sampleBuffer);
+        if (!buffer)
+            return;
     }
-    return false;
-}
-
-bool AVVideoCaptureSource::supportsSizeAndFrameRate(std::optional<int> width, std::optional<int> height, std::optional<double> frameRate)
-{
-    if (!height && !width && !frameRate)
-        return true;
 
-    if ((height || width) && !bestSessionPresetForVideoDimensions(WTFMove(width), WTFMove(height)))
-        return false;
-
-    if (!frameRate)
-        return true;
-
-    return isFrameRateSupported(frameRate.value());
+    scheduleDeferredTask([this, buffer, connection = retainPtr(captureConnection)] {
+        this->processNewFrame(buffer, connection);
+    });
 }
 
 void AVVideoCaptureSource::captureSessionIsRunningDidChange(bool state)
@@ -711,6 +589,29 @@ bool AVVideoCaptureSource::interrupted() const
     return RealtimeMediaSource::interrupted();
 }
 
+void AVVideoCaptureSource::generatePresets()
+{
+    Vector<Ref<VideoPreset>> presets;
+    for (AVCaptureDeviceFormatType* format in [device() formats]) {
+
+        CMVideoDimensions dimensions = CMVideoFormatDescriptionGetDimensions(format.formatDescription);
+        IntSize size = {dimensions.width, dimensions.height};
+        auto index = presets.findMatching([&size](auto& preset) {
+            return size == preset->size;
+        });
+        if (index != notFound)
+            continue;
+
+        Vector<FrameRateRange> frameRates;
+        for (AVFrameRateRangeType *range in [format videoSupportedFrameRateRanges])
+            frameRates.append({ range.minFrameRate, range.maxFrameRate});
+
+        presets.append(AVVideoPreset::create(size, WTFMove(frameRates), format));
+    }
+
+    setSupportedPresets(WTFMove(presets));
+}
+
 #if PLATFORM(IOS)
 void AVVideoCaptureSource::captureSessionRuntimeError(RetainPtr<NSError> error)
 {
index 7d727cc..e6cc41f 100644 (file)
@@ -841,7 +841,7 @@ void CoreAudioCaptureSource::stopProducingData()
     unit.stopProducingData();
 }
 
-const RealtimeMediaSourceCapabilities& CoreAudioCaptureSource::capabilities() const
+const RealtimeMediaSourceCapabilities& CoreAudioCaptureSource::capabilities()
 {
     if (!m_capabilities) {
         RealtimeMediaSourceCapabilities capabilities(settings().supportedConstraints());
@@ -854,7 +854,7 @@ const RealtimeMediaSourceCapabilities& CoreAudioCaptureSource::capabilities() co
     return m_capabilities.value();
 }
 
-const RealtimeMediaSourceSettings& CoreAudioCaptureSource::settings() const
+const RealtimeMediaSourceSettings& CoreAudioCaptureSource::settings()
 {
     if (!m_currentSettings) {
         RealtimeMediaSourceSettings settings;
index c40d3c6..476d3d6 100644 (file)
@@ -86,16 +86,16 @@ private:
 
     std::optional<Vector<int>> discreteSampleRates() const final { return { { 8000, 16000, 32000, 44100, 48000, 96000 } }; }
 
-    const RealtimeMediaSourceCapabilities& capabilities() const final;
-    const RealtimeMediaSourceSettings& settings() const final;
+    const RealtimeMediaSourceCapabilities& capabilities() final;
+    const RealtimeMediaSourceSettings& settings() final;
     void settingsDidChange(OptionSet<RealtimeMediaSourceSettings::Flag>) final;
 
     bool interrupted() const final;
 
     uint32_t m_captureDeviceID { 0 };
 
-    mutable std::optional<RealtimeMediaSourceCapabilities> m_capabilities;
-    mutable std::optional<RealtimeMediaSourceSettings> m_currentSettings;
+    std::optional<RealtimeMediaSourceCapabilities> m_capabilities;
+    std::optional<RealtimeMediaSourceSettings> m_currentSettings;
 
     enum class SuspensionType { None, WhilePaused, WhilePlaying };
     SuspensionType m_suspendType { SuspensionType::None };
index 3ecd5ca..119185c 100644 (file)
@@ -60,7 +60,7 @@ DisplayCaptureSourceCocoa::~DisplayCaptureSourceCocoa()
 #endif
 }
 
-const RealtimeMediaSourceCapabilities& DisplayCaptureSourceCocoa::capabilities() const
+const RealtimeMediaSourceCapabilities& DisplayCaptureSourceCocoa::capabilities()
 {
     if (!m_capabilities) {
         RealtimeMediaSourceCapabilities capabilities(settings().supportedConstraints());
@@ -75,7 +75,7 @@ const RealtimeMediaSourceCapabilities& DisplayCaptureSourceCocoa::capabilities()
     return m_capabilities.value();
 }
 
-const RealtimeMediaSourceSettings& DisplayCaptureSourceCocoa::settings() const
+const RealtimeMediaSourceSettings& DisplayCaptureSourceCocoa::settings()
 {
     if (!m_currentSettings) {
         RealtimeMediaSourceSettings settings;
index d8263d5..7e39eee 100644 (file)
@@ -69,13 +69,13 @@ private:
 
     bool isCaptureSource() const final { return true; }
 
-    const RealtimeMediaSourceCapabilities& capabilities() const final;
-    const RealtimeMediaSourceSettings& settings() const final;
+    const RealtimeMediaSourceCapabilities& capabilities() final;
+    const RealtimeMediaSourceSettings& settings() final;
 
     void emitFrame();
 
-    mutable std::optional<RealtimeMediaSourceCapabilities> m_capabilities;
-    mutable std::optional<RealtimeMediaSourceSettings> m_currentSettings;
+    std::optional<RealtimeMediaSourceCapabilities> m_capabilities;
+    std::optional<RealtimeMediaSourceSettings> m_currentSettings;
     RealtimeMediaSourceSupportedConstraints m_supportedConstraints;
 
     MonotonicTime m_startTime { MonotonicTime::nan() };
index 73129e1..0b6e0ab 100644 (file)
@@ -43,6 +43,8 @@ typedef struct __CVPixelBufferPool *CVPixelBufferPoolRef;
 
 namespace WebCore {
 
+class PixelBufferResizer;
+
 class MockRealtimeVideoSourceMac final : public MockRealtimeVideoSource, private OrientationNotifier::Observer {
 public:
     virtual ~MockRealtimeVideoSourceMac() = default;
@@ -57,15 +59,17 @@ private:
     PlatformLayer* platformLayer() const;
     void updateSampleBuffer() final;
     void settingsDidChange(OptionSet<RealtimeMediaSourceSettings::Flag>) final;
+    bool canResizeVideoFrames() const final { return true; }
+    void setSizeAndFrameRateWithPreset(IntSize, double, RefPtr<VideoPreset>) final;
 
     void orientationChanged(int orientation) final;
     void monitorOrientation(OrientationNotifier&) final;
 
-    mutable RetainPtr<CGImageRef> m_previewImage;
-    mutable RetainPtr<PlatformLayer> m_previewLayer;
     mutable RetainPtr<CVPixelBufferPoolRef> m_bufferPool;
     MediaSample::VideoRotation m_deviceOrientation { MediaSample::VideoRotation::None };
     std::unique_ptr<PixelBufferConformerCV> m_pixelBufferConformer;
+    std::unique_ptr<PixelBufferResizer> m_pixelBufferResizer;
+    IntSize m_presetSize;
 };
 
 } // namespace WebCore
index 17b35ba..613f224 100644 (file)
@@ -37,6 +37,7 @@
 #import "MediaConstraints.h"
 #import "MediaSampleAVFObjC.h"
 #import "NotImplemented.h"
+#import "PixelBufferResizer.h"
 #import "PlatformLayer.h"
 #import "RealtimeMediaSourceSettings.h"
 #import <QuartzCore/CALayer.h>
 namespace WebCore {
 using namespace PAL;
 
+#if PLATFORM(MAC)
+const OSType videoCaptureFormat = kCVPixelFormatType_420YpCbCr8Planar;
+#else
+const OSType videoCaptureFormat = kCVPixelFormatType_420YpCbCr8BiPlanarFullRange;
+#endif
 static const int videoSampleRate = 90000;
 
 CaptureSourceOrError MockRealtimeVideoSource::create(const String& deviceID, const String& name, const MediaConstraints* constraints)
@@ -71,23 +77,18 @@ RetainPtr<CMSampleBufferRef> MockRealtimeVideoSourceMac::CMSampleBufferFromPixel
     if (!pixelBuffer)
         return nullptr;
     
-    if (!m_pixelBufferConformer)
-        m_pixelBufferConformer = std::make_unique<PixelBufferConformerCV>((__bridge CFDictionaryRef)@{ (__bridge NSString *)kCVPixelBufferPixelFormatTypeKey: @(kCVPixelFormatType_420YpCbCr8Planar) });
-
-    auto convertedPixelBuffer = m_pixelBufferConformer->convert(pixelBuffer);
-
     CMTime sampleTime = CMTimeMake(((elapsedTime() + 100_ms) * videoSampleRate).seconds(), videoSampleRate);
     CMSampleTimingInfo timingInfo = { kCMTimeInvalid, sampleTime, sampleTime };
 
     CMVideoFormatDescriptionRef formatDescription = nullptr;
-    OSStatus status = CMVideoFormatDescriptionCreateForImageBuffer(kCFAllocatorDefault, (CVImageBufferRef)convertedPixelBuffer, &formatDescription);
+    OSStatus status = CMVideoFormatDescriptionCreateForImageBuffer(kCFAllocatorDefault, (CVImageBufferRef)pixelBuffer, &formatDescription);
     if (status != noErr) {
         LOG_ERROR("Failed to initialize CMVideoFormatDescription with error code: %d", status);
         return nullptr;
     }
 
     CMSampleBufferRef sampleBuffer;
-    status = CMSampleBufferCreateReadyWithImageBuffer(kCFAllocatorDefault, (CVImageBufferRef)convertedPixelBuffer, formatDescription, &timingInfo, &sampleBuffer);
+    status = CMSampleBufferCreateReadyWithImageBuffer(kCFAllocatorDefault, (CVImageBufferRef)pixelBuffer, formatDescription, &timingInfo, &sampleBuffer);
     CFRelease(formatDescription);
     if (status != noErr) {
         LOG_ERROR("Failed to initialize CMSampleBuffer with error code: %d", status);
@@ -148,6 +149,17 @@ void MockRealtimeVideoSourceMac::updateSampleBuffer()
         return;
 
     auto pixelBuffer = pixelBufferFromCGImage(imageBuffer->copyImage()->nativeImage().get());
+    if (m_pixelBufferResizer)
+        pixelBuffer = m_pixelBufferResizer->resize(pixelBuffer.get());
+    else {
+        if (!m_pixelBufferConformer) {
+            m_pixelBufferConformer = std::make_unique<PixelBufferConformerCV>((__bridge CFDictionaryRef)@{
+                (__bridge NSString *)kCVPixelBufferPixelFormatTypeKey: @(videoCaptureFormat)
+            });
+        }
+
+        pixelBuffer = m_pixelBufferConformer->convert(pixelBuffer.get());
+    }
     auto sampleBuffer = CMSampleBufferFromPixelBuffer(pixelBuffer.get());
 
     // We use m_deviceOrientation to emulate sensor orientation
@@ -188,6 +200,21 @@ void MockRealtimeVideoSourceMac::monitorOrientation(OrientationNotifier& notifie
     orientationChanged(notifier.orientation());
 }
 
+void MockRealtimeVideoSourceMac::setSizeAndFrameRateWithPreset(IntSize requestedSize, double, RefPtr<VideoPreset> preset)
+{
+    if (!preset)
+        return;
+
+    if (preset->size != requestedSize) {
+        if (m_pixelBufferResizer && !m_pixelBufferResizer->canResizeTo(requestedSize))
+            m_pixelBufferResizer = nullptr;
+
+        if (!m_pixelBufferResizer)
+            m_pixelBufferResizer = std::make_unique<PixelBufferResizer>(requestedSize, videoCaptureFormat);
+    } else
+        m_pixelBufferResizer = nullptr;
+}
+
 } // namespace WebCore
 
 #endif // ENABLE(MEDIA_STREAM)
index 9136e63..9c96cbb 100644 (file)
@@ -79,7 +79,7 @@ struct MockCameraProperties {
         if (!facingMode)
             return std::nullopt;
 
-        std::optional<Vector<VideoPreset>> presets;
+        std::optional<Vector<VideoPresetData>> presets;
         decoder >> presets;
         if (!presets)
             return std::nullopt;
@@ -94,7 +94,7 @@ struct MockCameraProperties {
 
     double defaultFrameRate { 30 };
     RealtimeMediaSourceSettings::VideoFacingMode facingMode { RealtimeMediaSourceSettings::VideoFacingMode::User };
-    Vector<VideoPreset> presets { { { 640, 480 }, { { 30, 30}, { 15, 15 } } } };
+    Vector<VideoPresetData> presets { { { 640, 480 }, { { 30, 30}, { 15, 15 } } } };
     Color fillColor { Color::black };
 };
 
index 9657270..2e8e55c 100644 (file)
@@ -98,7 +98,7 @@ MockRealtimeAudioSource::~MockRealtimeAudioSource()
 #endif
 }
 
-const RealtimeMediaSourceSettings& MockRealtimeAudioSource::settings() const
+const RealtimeMediaSourceSettings& MockRealtimeAudioSource::settings()
 {
     if (!m_currentSettings) {
         RealtimeMediaSourceSettings settings;
@@ -119,7 +119,7 @@ const RealtimeMediaSourceSettings& MockRealtimeAudioSource::settings() const
     return m_currentSettings.value();
 }
 
-const RealtimeMediaSourceCapabilities& MockRealtimeAudioSource::capabilities() const
+const RealtimeMediaSourceCapabilities& MockRealtimeAudioSource::capabilities()
 {
     if (!m_capabilities) {
         RealtimeMediaSourceCapabilities capabilities(settings().supportedConstraints());
index 79c6678..ded280d 100644 (file)
@@ -59,8 +59,8 @@ protected:
     static Seconds renderInterval() { return 60_ms; }
 
 private:
-    const RealtimeMediaSourceCapabilities& capabilities() const final;
-    const RealtimeMediaSourceSettings& settings() const final;
+    const RealtimeMediaSourceCapabilities& capabilities() final;
+    const RealtimeMediaSourceSettings& settings() final;
 
     void tick();
 
@@ -68,8 +68,8 @@ private:
 
     void delaySamples(Seconds) final;
 
-    mutable std::optional<RealtimeMediaSourceCapabilities> m_capabilities;
-    mutable std::optional<RealtimeMediaSourceSettings> m_currentSettings;
+    std::optional<RealtimeMediaSourceCapabilities> m_capabilities;
+    std::optional<RealtimeMediaSourceSettings> m_currentSettings;
     RealtimeMediaSourceSupportedConstraints m_supportedConstraints;
 
     RunLoop::Timer<MockRealtimeAudioSource> m_timer;
index 642d3ca..ae00d0d 100644 (file)
@@ -53,13 +53,8 @@ static inline Vector<MockMediaDevice> defaultDevices()
             MockCameraProperties {
                 30,
                 RealtimeMediaSourceSettings::VideoFacingMode::User, {
-                    { { 3840, 2160 }, { { 30, 30}, { 27.5, 27.5}, { 25, 25}, { 22.5, 22.5}, { 20, 20}, { 17.5, 17.5}, { 15, 15}, { 12.5, 12.5}, { 10, 10}, { 7.5, 7.5}, { 5, 5} } },
-                    { { 1920, 1080 }, { { 30, 30}, { 27.5, 27.5}, { 25, 25}, { 22.5, 22.5}, { 20, 20}, { 17.5, 17.5}, { 15, 15}, { 12.5, 12.5}, { 10, 10}, { 7.5, 7.5}, { 5, 5} } },
                     { { 1280, 720 }, { { 30, 30}, { 27.5, 27.5}, { 25, 25}, { 22.5, 22.5}, { 20, 20}, { 17.5, 17.5}, { 15, 15}, { 12.5, 12.5}, { 10, 10}, { 7.5, 7.5}, { 5, 5} } },
-                    { { 960, 540 }, { { 30, 30}, { 27.5, 27.5}, { 25, 25}, { 22.5, 22.5}, { 20, 20}, { 17.5, 17.5}, { 15, 15}, { 12.5, 12.5}, { 10, 10}, { 7.5, 7.5}, { 5, 5} } },
-                    { { 640, 480 }, { { 30, 30}, { 27.5, 27.5}, { 25, 25}, { 22.5, 22.5}, { 20, 20}, { 17.5, 17.5}, { 15, 15}, { 12.5, 12.5}, { 10, 10}, { 7.5, 7.5}, { 5, 5} } },
-                    { { 352, 288 }, { { 30, 30}, { 27.5, 27.5}, { 25, 25}, { 22.5, 22.5}, { 20, 20}, { 17.5, 17.5}, { 15, 15}, { 12.5, 12.5}, { 10, 10}, { 7.5, 7.5}, { 5, 5} } },
-                    { { 320, 240 }, { { 30, 30}, { 27.5, 27.5}, { 25, 25}, { 22.5, 22.5}, { 20, 20}, { 17.5, 17.5}, { 15, 15}, { 12.5, 12.5}, { 10, 10}, { 7.5, 7.5}, { 5, 5} } },
+                    { { 640, 480 },  { { 30, 30}, { 27.5, 27.5}, { 25, 25}, { 22.5, 22.5}, { 20, 20}, { 17.5, 17.5}, { 15, 15}, { 12.5, 12.5}, { 10, 10}, { 7.5, 7.5}, { 5, 5} } },
                 },
                 Color::black,
             } },
@@ -68,12 +63,14 @@ static inline Vector<MockMediaDevice> defaultDevices()
             MockCameraProperties {
                 15,
                 RealtimeMediaSourceSettings::VideoFacingMode::Environment, {
-                    { { 1280, 720 }, { { 3, 120 } } },
-                    { { 960, 540 }, { { 3, 60 } } },
-                    { { 640, 480 }, { { 2, 30 } } },
-                    { { 352, 288 }, { { 2, 30 } } },
-                    { { 320, 240 }, { { 2, 30 } } },
-                    { { 160, 120 }, { { 2, 30 } } },
+                    { { 3840, 2160 }, { { 2, 30 } } },
+                    { { 1920, 1080 }, { { 2, 30 } } },
+                    { { 1280, 720 },  { { 3, 120 } } },
+                    { { 960, 540 },   { { 3, 60 } } },
+                    { { 640, 480 },   { { 2, 30 } } },
+                    { { 352, 288 },   { { 2, 30 } } },
+                    { { 320, 240 },   { { 2, 30 } } },
+                    { { 160, 120 },   { { 2, 30 } } },
                 },
                 Color::darkGray,
             } },
index 0e78d56..27b53e4 100644 (file)
@@ -131,11 +131,15 @@ MockRealtimeVideoSource::MockRealtimeVideoSource(const String& deviceID, const S
     auto& properties = WTF::get<MockCameraProperties>(m_device.properties);
     setFrameRate(properties.defaultFrameRate);
     setFacingMode(properties.facingMode);
-    setSupportedPresets(WTFMove(properties.presets));
     m_fillColor = properties.fillColor;
 }
 
-const RealtimeMediaSourceCapabilities& MockRealtimeVideoSource::capabilities() const
+void MockRealtimeVideoSource::generatePresets()
+{
+    setSupportedPresets(WTFMove(WTF::get<MockCameraProperties>(m_device.properties).presets));
+}
+
+const RealtimeMediaSourceCapabilities& MockRealtimeVideoSource::capabilities()
 {
     if (!m_capabilities) {
         RealtimeMediaSourceCapabilities capabilities(settings().supportedConstraints());
@@ -149,13 +153,13 @@ const RealtimeMediaSourceCapabilities& MockRealtimeVideoSource::capabilities() c
             capabilities.setFrameRate(CapabilityValueOrRange(.01, 60.0));
         }
 
-        addSupportedCapabilities(capabilities);
+        updateCapabilities(capabilities);
         m_capabilities = WTFMove(capabilities);
     }
     return m_capabilities.value();
 }
 
-const RealtimeMediaSourceSettings& MockRealtimeVideoSource::settings() const
+const RealtimeMediaSourceSettings& MockRealtimeVideoSource::settings()
 {
     if (m_currentSettings)
         return m_currentSettings.value();
@@ -168,7 +172,7 @@ const RealtimeMediaSourceSettings& MockRealtimeVideoSource::settings() const
         settings.setLogicalSurface(true);
     }
     settings.setFrameRate(frameRate());
-    IntSize size = this->size();
+    auto& size = this->size();
     settings.setWidth(size.width());
     settings.setHeight(size.height());
     if (aspectRatio())
index 91f656a..eec4096 100644 (file)
@@ -63,18 +63,19 @@ protected:
     void settingsDidChange(OptionSet<RealtimeMediaSourceSettings::Flag>) override;
 
 private:
-    const RealtimeMediaSourceCapabilities& capabilities() const final;
-    const RealtimeMediaSourceSettings& settings() const final;
+    const RealtimeMediaSourceCapabilities& capabilities() final;
+    const RealtimeMediaSourceSettings& settings() final;
 
     void startProducingData() final;
     void stopProducingData() final;
+    bool isCaptureSource() const final { return true; }
+
+    void generatePresets() final;
 
     void drawAnimation(GraphicsContext&);
     void drawText(GraphicsContext&);
     void drawBoxes(GraphicsContext&);
 
-    bool isCaptureSource() const final { return true; }
-
     void generateFrame();
     void startCaptureTimer();
 
@@ -98,8 +99,8 @@ private:
 
     unsigned m_frameNumber { 0 };
     RunLoop::Timer<MockRealtimeVideoSource> m_emitFrameTimer;
-    mutable std::optional<RealtimeMediaSourceCapabilities> m_capabilities;
-    mutable std::optional<RealtimeMediaSourceSettings> m_currentSettings;
+    std::optional<RealtimeMediaSourceCapabilities> m_capabilities;
+    std::optional<RealtimeMediaSourceSettings> m_currentSettings;
     RealtimeMediaSourceSupportedConstraints m_supportedConstraints;
     Color m_fillColor { Color::black };
     MockMediaDevice m_device;
index ebf7b39..85e618b 100644 (file)
@@ -1,3 +1,13 @@
+2018-09-14  Eric Carlson  <eric.carlson@apple.com>
+
+        Support arbitrary video resolution in getUserMedia API
+        https://bugs.webkit.org/show_bug.cgi?id=178109
+        <rdar://problem/35083128>
+
+        Reviewed by Youenn Fablet.
+
+        * WebProcess/cocoa/UserMediaCaptureManager.cpp:
+
 2018-09-14  Basuke Suzuki  <Basuke.Suzuki@sony.com>
 
         [Curl][WebKit] Bug fix for continuously retrying with empty credentials.
index bc39dcc..480bad7 100644 (file)
@@ -65,13 +65,15 @@ public:
     }
 
     SharedRingBufferStorage& storage() { return static_cast<SharedRingBufferStorage&>(m_ringBuffer.storage()); }
-    const RealtimeMediaSourceCapabilities& capabilities() const final {
+
+    const RealtimeMediaSourceCapabilities& capabilities() final
+    {
         if (!m_capabilities)
             m_capabilities = m_manager.capabilities(m_id);
         return m_capabilities.value();
     }
 
-    const RealtimeMediaSourceSettings& settings() const final { return m_settings; }
+    const RealtimeMediaSourceSettings& settings() final { return m_settings; }
     void setSettings(RealtimeMediaSourceSettings&& settings)
     {
         auto changed = m_settings.difference(settings);