899878 - Mochitest - Record media with no timeslice for Media Recording API

Status: RESOLVED FIXED

(Core :: Audio/Video: Recording, defect)

Description

[Jason Smith [:jsmith]]

Create a mochitest that records media with no timeslice for the media recording API.

Comment 1

[Jason Smith [:jsmith]]

Attachment: Patch v1

Comment 2

[Jason Smith [:jsmith]]

Attachment: Test Output - First Patch

Comment 3

[Jason Smith [:jsmith]]

Comment on attachment 783526 [details] [diff] [review]
Patch v1

Asking for feedback right now because I can't formally land this patch yet as this mochitest reveals a leak. But I'd at least like to know if this patch is on the right track.

I'll file a bug for the leak and also kick off a try run with this patch shortly.

Comment 4

[Randy Lin [:rlin] - Be Mozillian on 2015/01/26]

The leak issue can be solved by this 
https://bug896353.bugzilla.mozilla.org/attachment.cgi?id=781659
Please open it and I will phase in this solution.

Comment 5

[Jason Smith [:jsmith]]

https://tbpl.mozilla.org/?tree=Try&rev=584c20f4e9e3

Comment 6

[Jason Smith [:jsmith]]

Try run reveals the following:

- I'll need to add a todo for the mime type check after start per bug 898396
- The leak being fixed in bug 896353 needs to land first to get this test to pass without leaks
- There's one timeout on an OS X 10.6 opt build that indicates that ondataavailable never fired. I'll wait for Rob's feedback to see if he knows why we're timing out - is it a bug in the development code or a bug in the test?

Comment 7

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Comment on attachment 783526 [details] [diff] [review]
Patch v1

Review of attachment 783526 [details] [diff] [review]:
-----------------------------------------------------------------

::: content/media/test/test_mediarecorder_record_no_timeslice.html
@@ +89,5 @@
> +    mediaRecorder.start();
> +
> +    is(mediaRecorder.state, 'recording', 'Media recorder should be recording');
> +    is(mediaRecorder.mimeType, expectedMimeType,
> +       'Mime type upon starting recording = ' + expectedMimeType);

I don't think we should expect the type to be available synchronously. This is going to be difficult to ensure in general. I think MediaRecorder.mimeType should actually go away.

@@ +105,5 @@
> +
> +    is(mediaRecorder.state, 'inactive',
> +       'Media recorder is inactive afer being stopped');
> +    is(mediaRecorder.mimeType, expectedMimeType,
> +       'Mime type upon stopping recording = ' + expectedMimeType);

As above.

Comment 8

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Actually there is a bug in the test. It's possible all the timeupdates we're going to get could fire before canplaythrough fires, in which case this test will never complete.

Comment 9

[Jason Smith [:jsmith]]

(In reply to Robert O'Callahan (:roc) (Mozilla Corporation) from comment #8)
> Actually there is a bug in the test. It's possible all the timeupdates we're
> going to get could fire before canplaythrough fires, in which case this test
> will never complete.

Hmm...maybe that's the reasoning why we timed out on one of the runs.

One idea I could do here is ignore the canplaythrough handling entirely and just rely on two timeupdate events firing. When the first fires, we start recording. When the second fires, we stop recording.

Comment 10

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Why not just run the contents of the canplaythrough handler immediately after setting up the element and MediaRecorder?

Comment 11

[Jason Smith [:jsmith]]

(In reply to Robert O'Callahan (:roc) (Mozilla Corporation) from comment #10)
> Why not just run the contents of the canplaythrough handler immediately
> after setting up the element and MediaRecorder?

Good point. That's probably a cleaner approach here to use then relying on multiple timeupdate events.

Comment 12

[Jason Smith [:jsmith]]

Attachment: Rapid Succession Approach Patch

Comment 13

[Jason Smith [:jsmith]]

Updated the patch according to Rob's comments. However, this test currently times out 100%, as onstop and ondataavailable will fail to fire. Filing a bug for this now.

Comment 14

[Jason Smith [:jsmith]]

Attachment: Two Timeupdate Event Approach v2

Comment 15

[Jason Smith [:jsmith]]

The two timeupdate approach apparently works with the known leak in the dependency.

https://tbpl.mozilla.org/?tree=Try&rev=b80519e274fc

Comment 16

[Jason Smith [:jsmith]]

Comment on attachment 786745 [details] [diff] [review]
Two Timeupdate Event Approach v2

Review of attachment 786745 [details] [diff] [review]:
-----------------------------------------------------------------

Given that bug 902318 exists, we can't use the rapid succession approach. So I'm flagging review on the approach that uses two timeupdate events.

Comment 17

[Jason Smith [:jsmith]]

Try run kicked off with the two timeupdate approach post landing of the memory leak fix - https://tbpl.mozilla.org/?tree=Try&rev=9fb7b34fc18b.

Comment 18

[Jason Smith [:jsmith]]

(In reply to Jason Smith [:jsmith] from comment #17)
> Try run kicked off with the two timeupdate approach post landing of the
> memory leak fix - https://tbpl.mozilla.org/?tree=Try&rev=9fb7b34fc18b.

Turns out I forgot to pull the latest changes when I did this try run. On a followup try run Randy kicked off, we're green - https://tbpl.mozilla.org/?tree=Try&rev=f5fe2c48718d.

Comment 19

[Jason Smith [:jsmith]]

Flagging checkin-needed per comment 18's green try run.

Comment 20

[Randy Lin [:rlin] - Be Mozillian on 2015/01/26]

oh, please wait until my another bug 903758 landded for this two line.
+      is(mediaRecorder.state, 'inactive',
+         'Media recorder is inactive afer being stopped');

Comment 21

[Jason Smith [:jsmith]]

(In reply to Randy Lin [:rlin] from comment #20)
> oh, please wait until my another bug 903758 landded for this two line.
> +      is(mediaRecorder.state, 'inactive',
> +         'Media recorder is inactive afer being stopped');

Okay. I'll hold checkin-needed until that lands.

Comment 22

[Jason Smith [:jsmith]]

Dependency is on inbound, so we're good to check this in now.

Comment 23

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/integration/b2g-inbound/rev/635ef1e34417

Comment 24

[Ryan VanderMeulen [:RyanVM]]

Backed out for B2G timeouts.
https://hg.mozilla.org/integration/b2g-inbound/rev/e35b9df27729

https://tbpl.mozilla.org/php/getParsedLog.php?id=26447273&tree=B2g-Inbound

Comment 25

[Jason Smith [:jsmith]]

(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #24)
> Backed out for B2G timeouts.
> https://hg.mozilla.org/integration/b2g-inbound/rev/e35b9df27729
> 
> https://tbpl.mozilla.org/php/getParsedLog.php?id=26447273&tree=B2g-Inbound

Hmm...that implies ondataavailable failed to fire when it should.

Randy - Can you investigate what's failing here?

Comment 26

[Ryan VanderMeulen [:RyanVM]]

And Windows.
https://tbpl.mozilla.org/php/getParsedLog.php?id=26450032&tree=B2g-Inbound

Comment 27

[Jason Smith [:jsmith]]

(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #26)
> And Windows.
> https://tbpl.mozilla.org/php/getParsedLog.php?id=26450032&tree=B2g-Inbound

Okay, so that further confirms ondataavailable is failing to fire intermittently. And it's platform independent likely then.

Comment 28

[Randy Lin [:rlin] - Be Mozillian on 2015/01/26]

In this case, it seems start recording right at end the music playback(media stream is under endding?). The test opus duration is 2.92s, Can we start recording after element.oncanplaythrough() event?

windows. 
14:08:37     INFO -  151805 INFO TEST-PASS | /tests/content/media/test/test_mediarecorder_record_no_timeslice.html | [started detodos.opus-0] Length of array should match number of running tests
14:08:40     INFO -  151806 INFO TEST-PASS | /tests/content/media/test/test_mediarecorder_record_no_timeslice.html | Media recorder should be recording
14:08:40     INFO -  151807 INFO TEST-PASS | /tests/content/media/test/test_mediarecorder_record_no_timeslice.html | Media recorder stream = element stream at the start of recording


b2g emu
19:54     INFO -  08-12 19:29:07.412   693   693 I GeckoDump: 780 INFO TEST-PASS | /tests/content/media/test/test_mediarecorder_record_no_timeslice.html | [started detodos.opus-0] Length of array should match number of running tests
13:19:54     INFO -  08-12 19:29:10.622   693   693 I GeckoDump: 781 INFO TEST-PASS | /tests/content/media/test/test_mediarecorder_record_no_timeslice.html | Media recorder should be recording
13:19:54     INFO -  08-12 19:29:10.642   693   693 I GeckoDump: 782 INFO TEST-PASS | /tests/content/media/test/test_mediarecorder_record_no_timeslice.html | Media recorder stream = element stream at the start of recording

test pass case. MacOSX
12:25:18     INFO -  147537 INFO TEST-INFO | /tests/content/media/test/test_mediarecorder_record_no_timeslice.html | Started Mon Aug 12 2013 12:25:18 GMT-0700 (PDT) (1376335518.692s)
12:25:18     INFO -  147538 INFO TEST-PASS | /tests/content/media/test/test_mediarecorder_record_no_timeslice.html | [started detodos.opus-0] Length of array should match number of running tests
12:25:19     INFO -  147539 INFO TEST-PASS | /tests/content/media/test/test_mediarecorder_record_no_timeslice.html | Media recorder should be recording

Comment 29

[Jason Smith [:jsmith]]

That makes sense. Starting after canplaythrough was what I originally did. However, the one problem that does exist is knowing when to call stop on media recording. Doing the rapid succession approach on canplaythrough won't work due to bug 902318.

I could however wait until onended fires potentially on the media element to stop recording right after onended fires.

An important question here to ask is - if the media element has finished playing, doesn't mean that the stream has ended itself? Wouldn't that imply that onstop should fire if the stream has ended? The try run shows that neither ondataavailable nor onstop fired, which seems strange.

Rob - What do you think?

Comment 30

[Robert O'Callahan (:roc) (email my personal email if necessary)]

It's possible that by the time canplaythrough fires, media element playback has already ended. The same goes for any event, because events can be delayed an arbitrarily long time.

I think we simply have to handle the case where playback has already ended as an additional possible success case.

Comment 31

[Randy Lin [:rlin] - Be Mozillian on 2015/01/26]

Test with " Two Timeupdate Event Approach v2 " patch and remove the recorder.stop() function call. I found when the audio element play end, the EndTrack in TrackUnionStream.h has been called, also the  l->NotifyQueuedTrackChanges invoked, but I found the MediaEncoder.cpp miss receiving this notify, a little strange.

Comment 32

[Jason Smith [:jsmith]]

(In reply to Randy Lin [:rlin] from comment #31)
> Test with " Two Timeupdate Event Approach v2 " patch and remove the
> recorder.stop() function call. I found when the audio element play end, the
> EndTrack in TrackUnionStream.h has been called, also the 
> l->NotifyQueuedTrackChanges invoked, but I found the MediaEncoder.cpp miss
> receiving this notify, a little strange.

Should we file a bug for this issue?

Comment 33

[Randy Lin [:rlin] - Be Mozillian on 2015/01/26]

I think this bug is for this issue. 

Bug 899935 - Stopping the MediaRecorder does not recieve a TRACK_EVENT_ENDED event from callback 
Hi roc. 
Do I need to hook anything in mediaRecorder to trigger the TrackUnionStream::EndTrack?
It seems there are two instances of TrackUnionStream while recording, but the one used for recording isn't trigger the EndTrack method.

Comment 34

[Robert O'Callahan (:roc) (email my personal email if necessary)]

I don't know why EndTrack wouldn't be called. It should be.

Comment 35

[Randy Lin [:rlin] - Be Mozillian on 2015/01/26]

Modify some test case code and try server looks ok. 
https://tbpl.mozilla.org/?tree=Try&rev=97abea53d51d
src = https://hg.mozilla.org/try/raw-rev/97abea53d51d

BTW, should fire another one to look why mochitest can't found EndTrack event from mediaStream and cause timeout.

Comment 36

[Jason Smith [:jsmith]]

(In reply to Randy Lin [:rlin] from comment #35)
> Modify some test case code and try server looks ok. 
> https://tbpl.mozilla.org/?tree=Try&rev=97abea53d51d
> src = https://hg.mozilla.org/try/raw-rev/97abea53d51d
> 
> BTW, should fire another one to look why mochitest can't found EndTrack
> event from mediaStream and cause timeout.

Yup, that was about what I was thinking was needed to fix this initially, but there's one issue - when bug 899935 gets fixed, this test will have onstop to fire twice. What I'd change here is switch mozCaptureStreamUntilEnded to mozCaptureStream so that ending the playback of the element won't cause onstop to fire.

Comment 37

[Jason Smith [:jsmith]]

Just kicked off a try run for using Randy's patch with mozCaptureStream instead here - https://tbpl.mozilla.org/?tree=Try&rev=59d692f64008. If it's green, I'll post the patch up for review.

Comment 38

[Jason Smith [:jsmith]]

Attachment: Patch v3

Comment 39

[Jason Smith [:jsmith]]

We're green on try, so we're ready for review here.

Comment 40

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Comment on attachment 792829 [details] [diff] [review]
Patch v3

Review of attachment 792829 [details] [diff] [review]:
-----------------------------------------------------------------

::: content/media/test/test_mediarecorder_record_no_timeslice.html
@@ +88,5 @@
> +    mediaRecorder.start();
> +    is(mediaRecorder.state, 'recording',
> +       'Media recorder should be recording');
> +    is(mediaRecorder.stream, element.stream,
> +       'Media recorder stream = element stream at the start of recording');

The stream could already have ended by the time canplaythrough fires, so I think this test could fail.

Comment 41

[Jason Smith [:jsmith]]

Comment on attachment 792829 [details] [diff] [review]
Patch v3

Oh right. Forgot about the above comment you mentioned above about that.

Comment 42

[Jason Smith [:jsmith]]

Attachment: Patch v4

Comment 43

[Jason Smith [:jsmith]]

Comment on attachment 793334 [details] [diff] [review]
Patch v4

Is this what you are meaning to imply by stating that we should treat finding out ended = true in canplaythrough as a success condition?

What this does instead is that the test only runs in the case when canplaythrough fires with ended = false at the start of the event handler. If ended = true when canplaythrough fires, we skip the test.

Comment 44

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Comment on attachment 793334 [details] [diff] [review]
Patch v4

Review of attachment 793334 [details] [diff] [review]:
-----------------------------------------------------------------

::: content/media/test/test_mediarecorder_record_no_timeslice.html
@@ +84,5 @@
> +  };
> +
> +  element.oncanplaythrough = function () {
> +    // If content has ended, skip the test
> +    if(element.ended) {

space before (

@@ +86,5 @@
> +  element.oncanplaythrough = function () {
> +    // If content has ended, skip the test
> +    if(element.ended) {
> +      ok(true, 'ended fired before canplaythrough, skipping test');
> +      manager.finished(token);

Yeah, this is what I meant.

Comment 45

[Jason Smith [:jsmith]]

Attachment: Patch v5

Fixed nit on spacing. Carrying forward r+.

Comment 46

[Jason Smith [:jsmith]]

Kicked off a try run again to double check here - https://tbpl.mozilla.org/?tree=Try&rev=84935daa810d. If it's green, then I'll mark checkin-needed for landing.

Comment 47

[Jason Smith [:jsmith]]

Try run is green and we have a r+. Ready for landing.

Comment 48

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/cf3db1cabcb2

Comment 49

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/cf3db1cabcb2