So, we have 4 boolean variables and here is the truth table.
Case 1, 2, 3, 4, 7, 8, 10 and 12 are not possible to happen.
Then, the remaining cases could be clustered into three categories:
(1) Case 5, 9 and 13: no sample is demuxed at all, return 0.
(2) Case 6, 11, 14 and 15: either audio or video is able to be demuxed, return the known value.
(3) Case 15: both audio and video are demuxed, return the minimum of the values.
For simplifying the logic, I will initialize the audioStartTime and videoStartTime
to be INFINITY if we don't have the first-demuxed sample, otherwise, initialize
them to be the real first-demuxed sample's time.
Then, the final calculation will be:
(1) Case 5, 9 and 13: the minimum of two INFINITY values is still INFINITY, return 0.
(2) Case 6, 11, 14 and 15: return the minimum of one real first-demuxed-time and the INFINITY.
(3) Case 15: return the minimum of two real first-demuxed-time values.
Case HasAudio HasVideo HasAudioSample HasVideoSample ExpectedResult
---------------------------------------------------------------------------------------------------
1 F F F F not possible
2 F F F T not possible
3 F F T F not possible
4 F F T T not possible
---------------------------------------------------------------------------------------------------
5 F T F F return 0
6 F T F T return video sample
7 F T T F not possible
8 F T T T not possible
---------------------------------------------------------------------------------------------------
9 T F F F return 0
10 T F F T not possible
11 T F T F return audio sample
12 T F T T not possible
---------------------------------------------------------------------------------------------------
13 T T F F return 0
14 T T F T return videoSample
15 T T T F return audioSample
16 T T T T return min(auidoSample, videoSample)
MozReview-Commit-ID: ANsYDth7slJ
--HG--
extra : transplant_source : %DAj%1A%EC%19%82%7B%B3%05%FE%21%04a%16%9A%9F%18x1%95
Now everything is ready. We can make NotifyQueuedTrackChanges only triggered by TRACK_EVENT_CREATED and TRACK_EVENT_ENDED without breaking anything. Also we make TrackUnionStream no longer copying data in video case.
MozReview-Commit-ID: IgLx1mpBWB3
--HG--
extra : transplant_source : %21M%C5%07%B9%CB%16%96%D6gN%C0g%0A%E8%7D%A0%AB%98%26
Replace |MediaPipelineTransmit::PipelineListener::NotifyQueuedTrackChanges| with |MediaPipelineTransmit::PipelineVideoSink::SetCurrentFrames|. We only need to deal with the video case since audio will be routed to |NotifyQueuedAudioData|.
MozReview-Commit-ID: EVpMVgJynGT
--HG--
extra : transplant_source : %0By%B5%91Fr%5B%BA%F7%D4%EE%FBs7%0C%F2%84%EC%5C5
Add MediaStreamVideoRecorderSink into MediaEncorder. In this patch, I still keep use duration to pass to TrackEncoders. Don't want to make this bug too big and out of control. We can file a new bug to change TrackEncoders use TimeStamp only.
MozReview-Commit-ID: KGftzulZynj
--HG--
extra : transplant_source : %E9%22B%90%D6%CF%08%12X%D1%E2%17%90%99%B2%91%24B%EA%1D
Make CaptureTask to inherite from MediaStreamVideoSink. The main change is to move the logic of |NotifyQueuedTrackChanges| to |SetCurrentFrames|.
The original image capture is not modified for support multiple video MediaStreamTracks. The design still used the track id in owned media stream. The should be fixed in the following bug if we still want to support ImageCapture in multiple video tracks case.
MozReview-Commit-ID: Od4tHoR8Ef
--HG--
extra : transplant_source : %AA%1BD%20%1D%8F%D0%9A%5B%9C%7Ef%A0%1C%E9%D7%EE%AE%E6%93
In this patch, we first deal with the case of MediaElement. Now we replace |PlayVideo| with |VideoFrameContainer::SetCurrentFrames| in |SourceMediaStream::AppendToTrack|. The MSG use TimeStamp::Now() for the TimeStamp of each video frame in most of case except MediaElement case. Becasue the MediaElement has its own VideoQueue, we need to calucalte the correct Timestamp based on the StartTimeStamp of this MediaStream and the elpased time of the video frame in DecodedStream.
MozReview-Commit-ID: 2bm2AHkFXHu
--HG--
extra : transplant_source : %3D%AA%00%CE%A3SV5%8F%84%96%AC%E2%D9%10%EC%85%07N%DF
Replace the pointer of VideoFrameContainer with the pointer of MediaStreamVideoSink.
MozReview-Commit-ID: 5bqEMpemwuR
--HG--
extra : transplant_source : %008z%D8W%EE%87%8E%E9/%2CT%26%EBvo%AE%099%A6
MediaStreamVideoSink is the base class of VideoFrameContainer, CaptureTask(ImageCapture), MediaStreamVideoRecorderSink(MediaRecoreder) and PipelineVideoSink(WebRTC-MediaPipelineTransmit). In this patch, I change VideoFrameContainer only. The rest of cases will be changed in latter patches of this bug.
MozReview-Commit-ID: JNUke3fyCoN
--HG--
extra : transplant_source : %0A%C7q%F3Md%0CO%8C%5DH%90%BBvp%9E%F0%DA%CD%CB
This is a silly bug that start() is called twice in this subtest, one
from the load event of the document (by pointerlock_utils.js), the other
is by onload of the iframe. Given the document's load event comes later,
I removed the onload of iframe.
MozReview-Commit-ID: 7TQJVa3xCb9
--HG--
extra : transplant_source : %88%DFF%5E%CD%84%19%01e%A9%18%7E%85%C75%E4%BAb%EC%03
David Anderson
Eric Faust
Nathan Froyd
Luke Wagner
Nathan Froyd
Terrence Cole
Kim Moir
Alexander Surkov
Miko Mynttinen
Martin Stransky
Ethan Lin
Kaku Kuo
John Dai
Shih-Chiang Chien
Andrew McCreight
Jessica Jong
JerryShih
James Willcox
ctai
Stone Shih
Jonathan Hao
James Cheng
Xidorn Quan