Support arbitrary video resolution in getUserMedia API
authoreric.carlson@apple.com <eric.carlson@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 14 Sep 2018 19:01:41 +0000 (19:01 +0000)
committereric.carlson@apple.com <eric.carlson@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 14 Sep 2018 19:01:41 +0000 (19:01 +0000)
commitdc6e824824ada2a92d57883454f361d8ae7301da
tree4ef92224f83fb3c28b4b3df933cd4064c2accf13
parent10a520e315a48979bc60f17df622e3a2a8fa5911
Support arbitrary video resolution in getUserMedia API
https://bugs.webkit.org/show_bug.cgi?id=178109
<rdar://problem/35083128>

Reviewed by Youenn Fablet.

Source/WebCore:

Support arbitrary video resolutions by configuring the camera to capture at the closest
larger size it supports and scaling/cropping frames as necessary.

No new tests, existing tests updated.

* Modules/mediastream/CanvasCaptureMediaStreamTrack.h:
* Modules/webaudio/MediaStreamAudioSource.cpp:
(WebCore::MediaStreamAudioSource::capabilities): Make non-const, it wasn't helpful.
(WebCore::MediaStreamAudioSource::settings): Ditto.
(WebCore::MediaStreamAudioSource::capabilities const): Deleted.
(WebCore::MediaStreamAudioSource::settings const): Deleted.
* Modules/webaudio/MediaStreamAudioSource.h:

* SourcesCocoa.txt: Add PixelBufferResizer.

* WebCore.xcodeproj/project.pbxproj: Ditto.

* platform/cocoa/CoreVideoSoftLink.h:
* platform/cocoa/VideoToolboxSoftLink.cpp:
* platform/cocoa/VideoToolboxSoftLink.h:

* platform/graphics/cv/PixelBufferResizer.h: Added.
(WebCore::PixelBufferResizer::canResizeTo):
* platform/graphics/cv/PixelBufferResizer.mm: Added.
(WebCore::PixelBufferResizer::PixelBufferResizer):
(WebCore::PixelBufferResizer::resize):

* platform/mediastream/RealtimeIncomingAudioSource.cpp:
(WebCore::RealtimeIncomingAudioSource::capabilities): Make non-const, it wasn't helpful.
(WebCore::RealtimeIncomingAudioSource::settings): Ditto.
(WebCore::RealtimeIncomingAudioSource::capabilities const): Deleted.
(WebCore::RealtimeIncomingAudioSource::settings const): Deleted.
* platform/mediastream/RealtimeIncomingAudioSource.h:

* platform/mediastream/RealtimeIncomingVideoSource.cpp:
(WebCore::RealtimeIncomingVideoSource::capabilities): Make non-const, it wasn't helpful.
(WebCore::RealtimeIncomingVideoSource::settings): Ditto.
(WebCore::RealtimeIncomingVideoSource::capabilities const): Deleted.
(WebCore::RealtimeIncomingVideoSource::settings const): Deleted.
* platform/mediastream/RealtimeIncomingVideoSource.h:

* platform/mediastream/RealtimeMediaSource.cpp:
(WebCore::RealtimeMediaSource::supportsConstraint):
(WebCore::RealtimeMediaSource::supportsConstraint const): Deleted.
* platform/mediastream/RealtimeMediaSource.h:

* platform/mediastream/RealtimeVideoSource.cpp:
(WebCore::RealtimeVideoSource::presets):
(WebCore::RealtimeVideoSource::setSupportedPresets):
(WebCore::standardVideoSizes):
(WebCore::RealtimeVideoSource::updateCapabilities): Make non-const, it wasn't helpful.
(WebCore::presetSupportsFrameRate):
(WebCore::RealtimeVideoSource::supportsCaptureSize):
(WebCore::RealtimeVideoSource::shouldUsePreset):
(WebCore::RealtimeVideoSource::bestSupportedSizeAndFrameRate):
(WebCore::RealtimeVideoSource::setSizeAndFrameRate):
(WebCore::RealtimeVideoSource::addSupportedCapabilities const): Deleted.
* platform/mediastream/RealtimeVideoSource.h:
(WebCore::VideoPresetData::encode const):
(WebCore::VideoPresetData::decode):
(WebCore::VideoPreset::create):
(WebCore::VideoPreset::VideoPreset):
(WebCore::RealtimeVideoSource::prefersPreset):
(WebCore::RealtimeVideoSource::canResizeVideoFrames const):
(WebCore::RealtimeVideoSource::setDefaultSize):
(WebCore::RealtimeVideoSource::observedFrameRate const):
(WebCore::VideoPreset::encode const): Deleted.
(WebCore::VideoPreset::decode): Deleted.

* platform/mediastream/mac/AVVideoCaptureSource.h:
(WebCore::AVVideoCaptureSource::videoPresets): Deleted.
* platform/mediastream/mac/AVVideoCaptureSource.mm:
(WebCore::AVVideoPreset::create):
(WebCore::AVVideoPreset::AVVideoPreset):
(WebCore::AVVideoCaptureSource::AVVideoCaptureSource):
(WebCore::AVVideoCaptureSource::settings):
(WebCore::AVVideoCaptureSource::capabilities): Make non-const, it wasn't helpful.
(WebCore::AVVideoCaptureSource::setFrameRate):
(WebCore::AVVideoCaptureSource::prefersPreset):
(WebCore::AVVideoCaptureSource::setSizeAndFrameRateWithPreset):
(WebCore::AVVideoCaptureSource::frameDurationForFrameRate):
(WebCore::AVVideoCaptureSource::setupCaptureSession):
(WebCore::AVVideoCaptureSource::processNewFrame):
(WebCore::AVVideoCaptureSource::captureOutputDidOutputSampleBufferFromConnection):
(WebCore::AVVideoCaptureSource::isFrameRateSupported):
(WebCore::AVVideoCaptureSource::generatePresets):
(WebCore::updateSizeMinMax): Deleted.
(WebCore::updateAspectRatioMinMax): Deleted.
(WebCore::AVVideoCaptureSource::settings const): Deleted.
(WebCore::AVVideoCaptureSource::capabilities const): Deleted.
(WebCore::AVVideoCaptureSource::sizeForPreset): Deleted.
(WebCore::AVVideoCaptureSource::setPreset): Deleted.
(WebCore::AVVideoCaptureSource::setSizeAndFrameRate): Deleted.
(WebCore::AVVideoCaptureSource::bestSessionPresetForVideoDimensions): Deleted.
(WebCore::AVVideoCaptureSource::supportsSizeAndFrameRate): Deleted.

* platform/mediastream/mac/CoreAudioCaptureSource.cpp:
(WebCore::CoreAudioCaptureSource::capabilities): Make non-const, it wasn't helpful.
(WebCore::CoreAudioCaptureSource::settings): Ditto.
(WebCore::CoreAudioCaptureSource::capabilities const): Deleted.
(WebCore::CoreAudioCaptureSource::settings const): Deleted.
* platform/mediastream/mac/CoreAudioCaptureSource.h:

* platform/mediastream/mac/DisplayCaptureSourceCocoa.cpp:
(WebCore::DisplayCaptureSourceCocoa::capabilities): Make non-const, it wasn't helpful.
(WebCore::DisplayCaptureSourceCocoa::settings): Ditto.
(WebCore::DisplayCaptureSourceCocoa::capabilities const): Deleted.
(WebCore::DisplayCaptureSourceCocoa::settings const): Deleted.
* platform/mediastream/mac/DisplayCaptureSourceCocoa.h:

* platform/mediastream/mac/MockRealtimeVideoSourceMac.h:
* platform/mediastream/mac/MockRealtimeVideoSourceMac.mm:
(WebCore::MockRealtimeVideoSourceMac::CMSampleBufferFromPixelBuffer):
(WebCore::MockRealtimeVideoSourceMac::updateSampleBuffer):
(WebCore::MockRealtimeVideoSourceMac::setSizeAndFrameRateWithPreset):
* platform/mock/MockMediaDevice.h:
(WebCore::MockCameraProperties::decode):

* platform/mock/MockRealtimeAudioSource.cpp:
(WebCore::MockRealtimeAudioSource::settings): Make non-const, it wasn't helpful.
(WebCore::MockRealtimeAudioSource::capabilities): Ditto.
(WebCore::MockRealtimeAudioSource::settings const): Deleted.
(WebCore::MockRealtimeAudioSource::capabilities const): Deleted.
* platform/mock/MockRealtimeAudioSource.h:

* platform/mock/MockRealtimeMediaSourceCenter.cpp:
(WebCore::defaultDevices): Change video device presets to trigger resize code more often.

* platform/mock/MockRealtimeVideoSource.cpp:
(WebCore::MockRealtimeVideoSource::MockRealtimeVideoSource):
(WebCore::MockRealtimeVideoSource::generatePresets):
(WebCore::MockRealtimeVideoSource::capabilities): Make non-const, it wasn't helpful.
(WebCore::MockRealtimeVideoSource::settings): Ditto.
(WebCore::MockRealtimeVideoSource::capabilities const): Deleted.
(WebCore::MockRealtimeVideoSource::settings const): Deleted.
* platform/mock/MockRealtimeVideoSource.h:

Source/WebCore/PAL:

* pal/cf/CoreMediaSoftLink.cpp:
* pal/cf/CoreMediaSoftLink.h:

Source/WebKit:

* WebProcess/cocoa/UserMediaCaptureManager.cpp:

LayoutTests:

* fast/mediastream/MediaStreamTrack-getCapabilities-expected.txt:
* fast/mediastream/apply-constraints-advanced-expected.txt:
* fast/mediastream/apply-constraints-advanced.html:
* fast/mediastream/apply-constraints-video.html:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@236015 268f45cc-cd09-0410-ab3c-d52691b4dbfc
53 files changed:
LayoutTests/ChangeLog
LayoutTests/fast/mediastream/MediaStreamTrack-getCapabilities-expected.txt
LayoutTests/fast/mediastream/apply-constraints-advanced-expected.txt
LayoutTests/fast/mediastream/apply-constraints-advanced.html
LayoutTests/fast/mediastream/apply-constraints-video-expected.txt
LayoutTests/fast/mediastream/apply-constraints-video.html
Source/WebCore/ChangeLog
Source/WebCore/Modules/mediastream/CanvasCaptureMediaStreamTrack.h
Source/WebCore/Modules/webaudio/MediaStreamAudioSource.cpp
Source/WebCore/Modules/webaudio/MediaStreamAudioSource.h
Source/WebCore/PAL/ChangeLog
Source/WebCore/PAL/pal/cf/CoreMediaSoftLink.cpp
Source/WebCore/PAL/pal/cf/CoreMediaSoftLink.h
Source/WebCore/SourcesCocoa.txt
Source/WebCore/WebCore.xcodeproj/project.pbxproj
Source/WebCore/platform/cocoa/CoreVideoSoftLink.h
Source/WebCore/platform/cocoa/VideoToolboxSoftLink.cpp
Source/WebCore/platform/cocoa/VideoToolboxSoftLink.h
Source/WebCore/platform/graphics/cv/PixelBufferResizer.h [new file with mode: 0644]
Source/WebCore/platform/graphics/cv/PixelBufferResizer.mm [new file with mode: 0644]
Source/WebCore/platform/mediastream/RealtimeIncomingAudioSource.cpp
Source/WebCore/platform/mediastream/RealtimeIncomingAudioSource.h
Source/WebCore/platform/mediastream/RealtimeIncomingVideoSource.cpp
Source/WebCore/platform/mediastream/RealtimeIncomingVideoSource.h
Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp
Source/WebCore/platform/mediastream/RealtimeMediaSource.h
Source/WebCore/platform/mediastream/RealtimeVideoSource.cpp
Source/WebCore/platform/mediastream/RealtimeVideoSource.h
Source/WebCore/platform/mediastream/VideoPreset.h [new file with mode: 0644]
Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCaptureSource.cpp
Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCaptureSource.h
Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.cpp
Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.h
Source/WebCore/platform/mediastream/gstreamer/MockGStreamerAudioCaptureSource.cpp
Source/WebCore/platform/mediastream/gstreamer/MockGStreamerAudioCaptureSource.h
Source/WebCore/platform/mediastream/gstreamer/MockGStreamerVideoCaptureSource.cpp
Source/WebCore/platform/mediastream/gstreamer/MockGStreamerVideoCaptureSource.h
Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.h
Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.mm
Source/WebCore/platform/mediastream/mac/CoreAudioCaptureSource.cpp
Source/WebCore/platform/mediastream/mac/CoreAudioCaptureSource.h
Source/WebCore/platform/mediastream/mac/DisplayCaptureSourceCocoa.cpp
Source/WebCore/platform/mediastream/mac/DisplayCaptureSourceCocoa.h
Source/WebCore/platform/mediastream/mac/MockRealtimeVideoSourceMac.h
Source/WebCore/platform/mediastream/mac/MockRealtimeVideoSourceMac.mm
Source/WebCore/platform/mock/MockMediaDevice.h
Source/WebCore/platform/mock/MockRealtimeAudioSource.cpp
Source/WebCore/platform/mock/MockRealtimeAudioSource.h
Source/WebCore/platform/mock/MockRealtimeMediaSourceCenter.cpp
Source/WebCore/platform/mock/MockRealtimeVideoSource.cpp
Source/WebCore/platform/mock/MockRealtimeVideoSource.h
Source/WebKit/ChangeLog
Source/WebKit/WebProcess/cocoa/UserMediaCaptureManager.cpp