[MSE] Implement Append Error algorithm.
authorcommit-queue@webkit.org <commit-queue@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 6 Feb 2015 22:14:20 +0000 (22:14 +0000)
committercommit-queue@webkit.org <commit-queue@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 6 Feb 2015 22:14:20 +0000 (22:14 +0000)
https://bugs.webkit.org/show_bug.cgi?id=139439

Patch by Bartlomiej Gajda <b.gajda@samsung.com> on 2015-02-06
Reviewed by Jer Noble.

If Source Buffer has not received first init segment, then it shall call endOfStream after receiving
Media Segment, as per Media Source spec. (from 17 July 2014) in paragraph 3.5.1 point 6.1.
Source/WebCore:

Based this change on Editor's Draft 12 December 2014, as it clarifies order of events.

Test: media/media-source/media-source-append-media-segment-without-init.html

* Modules/mediasource/MediaSource.cpp:
(WebCore::MediaSource::streamEndedWithError):
* Modules/mediasource/MediaSource.h:
* Modules/mediasource/SourceBuffer.cpp:
(WebCore::SourceBuffer::sourceBufferPrivateAppendComplete):
(WebCore::SourceBuffer::sourceBufferPrivateDidReceiveInitializationSegment):
(WebCore::SourceBuffer::validateInitializationSegment):
(WebCore::SourceBuffer::appendError):
* Modules/mediasource/SourceBuffer.h:

LayoutTests:

Added test which after creating SourceBuffer sends media sample, without any init segments.
Updated existing tests, so they correctly expect updateend and error as per Append Error algorithm.

* media/media-source/media-source-append-failed-expected.txt:
* media/media-source/media-source-append-failed.html:
* media/media-source/media-source-append-media-segment-without-init-expected.txt: Added.
* media/media-source/media-source-append-media-segment-without-init.html: Added.
* media/media-source/media-source-multiple-initialization-segments-expected.txt:
* media/media-source/media-source-multiple-initialization-segments.html:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@179762 268f45cc-cd09-0410-ab3c-d52691b4dbfc

LayoutTests/ChangeLog
LayoutTests/media/media-source/media-source-append-failed-expected.txt
LayoutTests/media/media-source/media-source-append-failed.html
LayoutTests/media/media-source/media-source-append-media-segment-without-init-expected.txt [new file with mode: 0644]
LayoutTests/media/media-source/media-source-append-media-segment-without-init.html [new file with mode: 0644]
LayoutTests/media/media-source/media-source-multiple-initialization-segments-expected.txt
LayoutTests/media/media-source/media-source-multiple-initialization-segments.html
Source/WebCore/ChangeLog
Source/WebCore/Modules/mediasource/SourceBuffer.cpp
Source/WebCore/Modules/mediasource/SourceBuffer.h

index c9909d111f4535ad4f6daf3d6aa3aa2ff51d0df2..00b9ad64b8f5319a1cf9fbbcac23b4eb17903852 100644 (file)
@@ -1,3 +1,22 @@
+2015-02-06  Bartlomiej Gajda  <b.gajda@samsung.com>
+
+        [MSE] Implement Append Error algorithm.
+        https://bugs.webkit.org/show_bug.cgi?id=139439
+
+        Reviewed by Jer Noble.
+
+        If Source Buffer has not received first init segment, then it shall call endOfStream after receiving
+        Media Segment, as per Media Source spec. (from 17 July 2014) in paragraph 3.5.1 point 6.1.
+        Added test which after creating SourceBuffer sends media sample, without any init segments.
+        Updated existing tests, so they correctly expect updateend and error as per Append Error algorithm.
+
+        * media/media-source/media-source-append-failed-expected.txt:
+        * media/media-source/media-source-append-failed.html:
+        * media/media-source/media-source-append-media-segment-without-init-expected.txt: Added.
+        * media/media-source/media-source-append-media-segment-without-init.html: Added.
+        * media/media-source/media-source-multiple-initialization-segments-expected.txt:
+        * media/media-source/media-source-multiple-initialization-segments.html:
+
 2015-02-06  Alexey Proskuryakov  <ap@apple.com>
 
         Correct expectations for inspector/css/selector-dynamic-specificity.html.
index 1311712da8f5ed4e94452af39c4a82034be598c8..61808cff7e35e17e57039e18851436025d76c61b 100644 (file)
@@ -4,6 +4,8 @@ EVENT(sourceopen)
 RUN(sourceBuffer = source.addSourceBuffer("video/mock; codecs=mock"))
 RUN(sourceBuffer.appendBuffer(initSegment))
 EVENT(updatestart)
+EVENT(error)
+EVENT(updateend)
 EVENT(sourceended)
 END OF TEST
 
index 5ea2374a08698764fbdc89b1289cabae37978db8..eabf4c8646ba0b3fcb41b7d85e178f8989b5a317 100644 (file)
@@ -25,6 +25,7 @@
 
         waitForEventOn(sourceBuffer, 'updatestart');
         waitForEventOn(sourceBuffer, 'update');
+        waitForEventOn(sourceBuffer, 'error');
         waitForEventOn(sourceBuffer, 'updateend');
         waitForEventOn(source, 'sourceended', endTest);
         initSegment = makeAnInvalidBox();
diff --git a/LayoutTests/media/media-source/media-source-append-media-segment-without-init-expected.txt b/LayoutTests/media/media-source/media-source-append-media-segment-without-init-expected.txt
new file mode 100644 (file)
index 0000000..0a16818
--- /dev/null
@@ -0,0 +1,8 @@
+
+RUN(video.src = URL.createObjectURL(source))
+EVENT(sourceopen)
+RUN(sourceBuffer = source.addSourceBuffer("video/mock; codecs=mock"))
+RUN(sourceBuffer.appendBuffer(samples))
+EVENT(error)
+END OF TEST
+
diff --git a/LayoutTests/media/media-source/media-source-append-media-segment-without-init.html b/LayoutTests/media/media-source/media-source-append-media-segment-without-init.html
new file mode 100644 (file)
index 0000000..b2dc39e
--- /dev/null
@@ -0,0 +1,48 @@
+<!DOCTYPE html>
+<html>
+<head>
+    <title>mock-media-source</title>
+    <script src="mock-media-source.js"></script>
+    <script src="../video-test.js"></script>
+    <script>
+    var source;
+    var sourceBuffer;
+    var initSegment;
+    var wasError = false;
+
+    if (window.internals)
+        internals.initializeMockMediaSource();
+
+    function runTest() {
+        findMediaElement();
+
+        source = new MediaSource();
+        waitForEventOn(source, 'sourceopen', sourceOpen, false, true);
+        run('video.src = URL.createObjectURL(source)');
+    }
+
+    function sourceOpen() {
+        run('sourceBuffer = source.addSourceBuffer("video/mock; codecs=mock")');
+
+        // Note: In normal usage we should send this line, but this checks what happens if we don't.
+        // initSegment = makeAInit(0, [makeATrack(1, 'mock', TRACK_KIND.VIDEO)]);
+
+        samples = concatenateSamples([
+            makeASample(0, 0, 1, 1, SAMPLE_FLAG.SYNC),
+            makeASample(1, 1, 1, 1, SAMPLE_FLAG.NONE),
+        ]);
+
+        // Note: if code correctly handles sample without init, it will go through Segment Parser Loop
+        // if not, we will receive update event as part of Coded Frame Processing
+        waitForEventOn(sourceBuffer, 'error', null, true, true);
+        failTestIn(2000);
+
+        run('sourceBuffer.appendBuffer(samples)');
+    }
+
+    </script>
+</head>
+<body onload="runTest()">
+    <video></video>
+</body>
+</html>
index 792a7e05585ba8620df1549405dc151e870a2199..8dfc3360b1f0b2147bdfb83ce685d8ff1ee5e77f 100644 (file)
@@ -9,6 +9,8 @@ RUN(sourceBuffer.appendBuffer(initSegment))
 EVENT(updateend)
 Test that a replacement initialization segment containing a track with a different codec but the same trackID fails.
 RUN(sourceBuffer.appendBuffer(initSegment))
+EVENT(error)
+EVENT(updateend)
 EVENT(sourceended)
 END OF TEST
 
index 0c8e2d8adeb979c220bc08cfada306abbd4a0d26..ce6799920d1d8f53550dd5c6fff5d1b973e303b7 100644 (file)
@@ -48,7 +48,8 @@
 
     function secondUpdate() {
         consoleWrite('Test that a replacement initialization segment containing a track with a different codec but the same trackID fails.')
-        waitForEventOn(sourceBuffer, 'updateend', endTest, false, true);
+        waitForEventOn(sourceBuffer, 'error');
+        waitForEventOn(sourceBuffer, 'updateend');
         expected = true;
         initSegment = makeAInit(100, [
             makeATrack(2, '!moc', TRACK_KIND.VIDEO),
index e7a69f42fe314a582748b7ed37f355fabff4e4b4..c947afb415f31e6e448a4ce810f94dda5d078ef6 100644 (file)
@@ -1,3 +1,27 @@
+2015-02-06  Bartlomiej Gajda  <b.gajda@samsung.com>
+
+        [MSE] Implement Append Error algorithm.
+        https://bugs.webkit.org/show_bug.cgi?id=139439
+
+        Reviewed by Jer Noble.
+
+        If Source Buffer has not received first init segment, then it shall call endOfStream after receiving
+        Media Segment, as per Media Source spec. (from 17 July 2014) in paragraph 3.5.1 point 6.1.
+
+        Based this change on Editor's Draft 12 December 2014, as it clarifies order of events.
+
+        Test: media/media-source/media-source-append-media-segment-without-init.html
+
+        * Modules/mediasource/MediaSource.cpp:
+        (WebCore::MediaSource::streamEndedWithError):
+        * Modules/mediasource/MediaSource.h:
+        * Modules/mediasource/SourceBuffer.cpp:
+        (WebCore::SourceBuffer::sourceBufferPrivateAppendComplete):
+        (WebCore::SourceBuffer::sourceBufferPrivateDidReceiveInitializationSegment):
+        (WebCore::SourceBuffer::validateInitializationSegment):
+        (WebCore::SourceBuffer::appendError):
+        * Modules/mediasource/SourceBuffer.h:
+
 2015-02-06  Timothy Horton  <timothy_horton@apple.com>
 
         REGRESSION: Lookup doesn't work in RTL
index 68e397b1b227e969a873a2b72b88db66191652c6..7728a2afc2cd71a42c4e71b855633414540b5e14 100644 (file)
@@ -616,10 +616,10 @@ void SourceBuffer::sourceBufferPrivateAppendComplete(SourceBufferPrivate*, Appen
     // https://dvcs.w3.org/hg/html-media/raw-file/default/media-source/media-source.html#sourcebuffer-buffer-append
 
     // 2. If the input buffer contains bytes that violate the SourceBuffer byte stream format specification,
-    // then run the end of stream algorithm with the error parameter set to "decode" and abort this algorithm.
+    // then run the append error algorithm with the decode error parameter set to true and abort this algorithm.
     if (result == ParsingFailed) {
         LOG(MediaSource, "SourceBuffer::sourceBufferPrivateAppendComplete(%p) - result = ParsingFailed", this);
-        m_source->streamEndedWithError(decodeError(), IgnorableExceptionCode());
+        appendError(true);
         return;
     }
 
@@ -997,8 +997,9 @@ void SourceBuffer::sourceBufferPrivateDidReceiveInitializationSegment(SourceBuff
 
     LOG(MediaSource, "SourceBuffer::sourceBufferPrivateDidReceiveInitializationSegment(%p)", this);
 
-    // 3.5.7 Initialization Segment Received
-    // https://dvcs.w3.org/hg/html-media/raw-file/default/media-source/media-source.html#sourcebuffer-init-segment-received
+    // 3.5.8 Initialization Segment Received (ctd)
+    // https://rawgit.com/w3c/media-source/c3ad59c7a370d04430969ba73d18dc9bcde57a33/index.html#sourcebuffer-init-segment-received [Editor's Draft 09 January 2015]
+
     // 1. Update the duration attribute if it currently equals NaN:
     if (m_source->duration().isInvalid()) {
         // ↳ If the initialization segment contains a duration:
@@ -1009,16 +1010,18 @@ void SourceBuffer::sourceBufferPrivateDidReceiveInitializationSegment(SourceBuff
         m_source->setDurationInternal(newDuration);
     }
 
-    // 2. If the initialization segment has no audio, video, or text tracks, then run the end of stream
-    // algorithm with the error parameter set to "decode" and abort these steps.
+    // 2. If the initialization segment has no audio, video, or text tracks, then run the append error algorithm
+    // with the decode error parameter set to true and abort these steps.
     if (!segment.audioTracks.size() && !segment.videoTracks.size() && !segment.textTracks.size())
-        m_source->streamEndedWithError(decodeError(), IgnorableExceptionCode());
-
+        appendError(true);
 
     // 3. If the first initialization segment flag is true, then run the following steps:
     if (m_receivedFirstInitializationSegment) {
+
+        // 3.1. Verify the following properties. If any of the checks fail then run the append error algorithm
+        // with the decode error parameter set to true and abort these steps.
         if (!validateInitializationSegment(segment)) {
-            m_source->streamEndedWithError(decodeError(), IgnorableExceptionCode());
+            appendError(true);
             return;
         }
         // 3.2 Add the appropriate track descriptions from this initialization segment to each of the track buffers.
@@ -1058,6 +1061,7 @@ void SourceBuffer::sourceBufferPrivateDidReceiveInitializationSegment(SourceBuff
             downcast<InbandTextTrack>(*textTrack).setPrivate(textTrackInfo.track);
         }
 
+        // 3.3 Set the need random access point flag on all track buffers to true.
         for (auto& trackBuffer : m_trackBufferMap.values())
             trackBuffer.needRandomAccessFlag = true;
     }
@@ -1068,13 +1072,14 @@ void SourceBuffer::sourceBufferPrivateDidReceiveInitializationSegment(SourceBuff
     // 5. If the first initialization segment flag is false, then run the following steps:
     if (!m_receivedFirstInitializationSegment) {
         // 5.1 If the initialization segment contains tracks with codecs the user agent does not support,
-        // then run the end of stream algorithm with the error parameter set to "decode" and abort these steps.
+        // then run the append error algorithm with the decode error parameter set to true and abort these steps.
         // NOTE: This check is the responsibility of the SourceBufferPrivate.
 
         // 5.2 For each audio track in the initialization segment, run following steps:
         for (auto& audioTrackInfo : segment.audioTracks) {
             AudioTrackPrivate* audioTrackPrivate = audioTrackInfo.track.get();
 
+            // FIXME: Implement steps 5.2.1-5.2.8.1 as per Editor's Draft 09 January 2015, and reorder this
             // 5.2.1 Let new audio track be a new AudioTrack object.
             // 5.2.2 Generate a unique ID and assign it to the id property on new video track.
             RefPtr<AudioTrack> newAudioTrack = AudioTrack::create(this, audioTrackPrivate);
@@ -1115,6 +1120,7 @@ void SourceBuffer::sourceBufferPrivateDidReceiveInitializationSegment(SourceBuff
         for (auto& videoTrackInfo : segment.videoTracks) {
             VideoTrackPrivate* videoTrackPrivate = videoTrackInfo.track.get();
 
+            // FIXME: Implement steps 5.3.1-5.3.8.1 as per Editor's Draft 09 January 2015, and reorder this
             // 5.3.1 Let new video track be a new VideoTrack object.
             // 5.3.2 Generate a unique ID and assign it to the id property on new video track.
             RefPtr<VideoTrack> newVideoTrack = VideoTrack::create(this, videoTrackPrivate);
@@ -1155,6 +1161,7 @@ void SourceBuffer::sourceBufferPrivateDidReceiveInitializationSegment(SourceBuff
         for (auto& textTrackInfo : segment.textTracks) {
             InbandTextTrackPrivate* textTrackPrivate = textTrackInfo.track.get();
 
+            // FIXME: Implement steps 5.4.1-5.4.8.1 as per Editor's Draft 09 January 2015, and reorder this
             // 5.4.1 Let new text track be a new TextTrack object with its properties populated with the
             // appropriate information from the initialization segment.
             RefPtr<InbandTextTrack> newTextTrack = InbandTextTrack::create(scriptExecutionContext(), this, textTrackPrivate);
@@ -1189,6 +1196,7 @@ void SourceBuffer::sourceBufferPrivateDidReceiveInitializationSegment(SourceBuff
         // 5.5 If active track flag equals true, then run the following steps:
         if (activeTrackFlag) {
             // 5.5.1 Add this SourceBuffer to activeSourceBuffers.
+            // 5.5.2 Queue a task to fire a simple event named addsourcebuffer at activeSourceBuffers
             setActive(true);
         }
 
@@ -1218,11 +1226,11 @@ void SourceBuffer::sourceBufferPrivateDidReceiveInitializationSegment(SourceBuff
 
 bool SourceBuffer::validateInitializationSegment(const InitializationSegment& segment)
 {
-    // 3.5.7 Initialization Segment Received (ctd)
-    // https://dvcs.w3.org/hg/html-media/raw-file/default/media-source/media-source.html#sourcebuffer-init-segment-received
+    // FIXME: ordering of all 3.5.X (X>=7) functions needs to be updated to post-[24 July 2014 Editor's Draft] version
+    // 3.5.8 Initialization Segment Received (ctd)
+    // https://rawgit.com/w3c/media-source/c3ad59c7a370d04430969ba73d18dc9bcde57a33/index.html#sourcebuffer-init-segment-received [Editor's Draft 09 January 2015]
 
-    // 3.1. Verify the following properties. If any of the checks fail then run the end of stream
-    // algorithm with the error parameter set to "decode" and abort these steps.
+    // Note: those are checks from step 3.1
     //   * The number of audio, video, and text tracks match what was in the first initialization segment.
     if (segment.audioTracks.size() != audioTracks()->length()
         || segment.videoTracks.size() != videoTracks()->length()
@@ -1289,11 +1297,45 @@ public:
     }
 };
 
+void SourceBuffer::appendError(bool decodeErrorParam)
+{
+    // 3.5.3 Append Error Algorithm
+    // https://rawgit.com/w3c/media-source/c3ad59c7a370d04430969ba73d18dc9bcde57a33/index.html#sourcebuffer-append-error [Editor's Draft 09 January 2015]
+
+    ASSERT(m_updating);
+    // 1. Run the reset parser state algorithm.
+    resetParserState();
+
+    // 2. Set the updating attribute to false.
+    m_updating = false;
+
+    // 3. Queue a task to fire a simple event named error at this SourceBuffer object.
+    scheduleEvent(eventNames().errorEvent);
+
+    // 4. Queue a task to fire a simple event named updateend at this SourceBuffer object.
+    scheduleEvent(eventNames().updateendEvent);
+
+    // 5. If decode error is true, then run the end of stream algorithm with the error parameter set to "decode".
+    if (decodeErrorParam)
+        m_source->streamEndedWithError(decodeError(), IgnorableExceptionCode());
+}
+
 void SourceBuffer::sourceBufferPrivateDidReceiveSample(SourceBufferPrivate*, PassRefPtr<MediaSample> prpSample)
 {
     if (isRemoved())
         return;
 
+    // 3.5.1 Segment Parser Loop
+    // 6.1 If the first initialization segment received flag is false, then run the append error algorithm
+    //     with the decode error parameter set to true and abort this algorithm.
+    // Note: current design makes SourceBuffer somehow ignorant of append state - it's more a thing
+    //  of SourceBufferPrivate. That's why this check can't really be done in appendInternal.
+    //  unless we force some kind of design with state machine switching.
+    if (!m_receivedFirstInitializationSegment) {
+        appendError(true);
+        return;
+    }
+
     RefPtr<MediaSample> sample = prpSample;
 
     // 3.5.8 Coded Frame Processing
index 5b9f703ffb001d157a8b1d5930360bf54dc39a38..47266bbaf1d641962e4506cd9cc06aa682c4365f 100644 (file)
@@ -92,6 +92,7 @@ public:
     void remove(double start, double end, ExceptionCode&);
     void remove(const MediaTime&, const MediaTime&, ExceptionCode&);
 
+    void appendError(bool);
     void abortIfUpdating();
     void removedFromMediaSource();
     void seekToTime(const MediaTime&);