WebRTC MediaStreamTrack Enable / Disable causes video delay / lag
authoryouenn@apple.com <youenn@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 6 Jul 2018 23:47:00 +0000 (23:47 +0000)
committeryouenn@apple.com <youenn@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 6 Jul 2018 23:47:00 +0000 (23:47 +0000)
commitf73374e65b3c409bc7fbdfc9c62af628348339aa
tree7793456fdbffce1c492748b589ef556c9bc2e0af
parentb03cc85b886a7726d702e2e0a4c1d0dfd51c2459
WebRTC MediaStreamTrack Enable / Disable causes video delay / lag
https://bugs.webkit.org/show_bug.cgi?id=186889
<rdar://problem/41370285>

Reviewed by Eric Carlson.

Source/WebCore:

Libwebrtc expects a continuous flow of calls for audio data since the API
does not provide any possiblity to give timestamps.

We were optimizing previously when a source is muted so that we would not transmit audio data.
This breaks synchronization between audio and video frames (which are timestamped).

This patch reverts the optimization and instead makes sure to send zeros for silenced audio tracks.

This requires MediaStreamTrackPrivate to send audio data even if disabled,
so that RealtimeOutgoingAudioSource will continue sending zeros at the correct pace.
This also requires WebAudioSourceProviderAVFObjC to exit early if its track is disabled.

Covered by existing tests.
Manual testing shows that synchronization is kept.

* platform/mediastream/MediaStreamTrackPrivate.cpp:
(WebCore::MediaStreamTrackPrivate::audioSamplesAvailable):
* platform/mediastream/RealtimeOutgoingAudioSource.cpp:
(WebCore::RealtimeOutgoingAudioSource::RealtimeOutgoingAudioSource):
(WebCore::RealtimeOutgoingAudioSource::initializeConverter):
(WebCore::RealtimeOutgoingAudioSource::stop):
(WebCore::RealtimeOutgoingAudioSource::sourceMutedChanged):
(WebCore::RealtimeOutgoingAudioSource::sourceEnabledChanged):
(WebCore::RealtimeOutgoingAudioSource::handleMutedIfNeeded): Deleted.
* platform/mediastream/RealtimeOutgoingAudioSource.h:
(WebCore::RealtimeOutgoingAudioSource::pullAudioData):
(WebCore::RealtimeOutgoingAudioSource::isSilenced const):
(WebCore::RealtimeOutgoingAudioSource::sendSilence): Deleted.
* platform/mediastream/mac/RealtimeOutgoingAudioSourceCocoa.cpp:
(WebCore::RealtimeOutgoingAudioSourceCocoa::pullAudioData):
(WebCore::RealtimeOutgoingAudioSourceCocoa::handleMutedIfNeeded): Deleted.
(WebCore::RealtimeOutgoingAudioSourceCocoa::sendSilence): Deleted.
* platform/mediastream/mac/RealtimeOutgoingAudioSourceCocoa.h:
* platform/mediastream/mac/WebAudioSourceProviderAVFObjC.mm:
(WebCore::WebAudioSourceProviderAVFObjC::audioSamplesAvailable):

LayoutTests:

Test is no longer valid since we are now sending 0 bytes for audio tracks.

* webrtc/audio-muted-stats2-expected.txt: Removed.
* webrtc/audio-muted-stats2.html: Removed.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@233604 268f45cc-cd09-0410-ab3c-d52691b4dbfc
12 files changed:
LayoutTests/ChangeLog
LayoutTests/webrtc/audio-muted-stats2-expected.txt [deleted file]
LayoutTests/webrtc/audio-muted-stats2.html [deleted file]
Source/WebCore/ChangeLog
Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.cpp
Source/WebCore/platform/mediastream/RealtimeOutgoingAudioSource.cpp
Source/WebCore/platform/mediastream/RealtimeOutgoingAudioSource.h
Source/WebCore/platform/mediastream/gstreamer/RealtimeOutgoingAudioSourceLibWebRTC.cpp
Source/WebCore/platform/mediastream/gstreamer/RealtimeOutgoingAudioSourceLibWebRTC.h
Source/WebCore/platform/mediastream/mac/RealtimeOutgoingAudioSourceCocoa.cpp
Source/WebCore/platform/mediastream/mac/RealtimeOutgoingAudioSourceCocoa.h
Source/WebCore/platform/mediastream/mac/WebAudioSourceProviderAVFObjC.mm