This moves the responsibility for creating MediaStreamTracks from
DecodedStream::Start to MediaDecoder. This let's MediaDecoder create them as
soon as metadata is known. This gives the application guarantees on when tracks
can be expected to exist, and aligns with the spec that says they should be
created when metadata is known.
Differential Revision: https://phabricator.services.mozilla.com/D28473
--HG--
extra : moz-landing-system : lando
This gives us a guarantee that the first frame of a media file can be rendered
with a second media element and mozCaptureStream(), even if the file is very
very short.
With longer video-only files there were also cases where nothing ended up being
rendered. Probably because the MediaStreamGraph ended up switching from an
AudioCallbackDriver to a SystemClockDriver and this took enough time to put the
SourceMediaStream::EndTrack and the SourceMediaStream::AddTrackListener calls
for this video track to be processed in the same iteration. The listener would
then always lose to the ending track and update main thread state too late,
leading to its media element not rendering any frames and nasty intermittent
failures.
Differential Revision: https://phabricator.services.mozilla.com/D27270
--HG--
extra : moz-landing-system : lando
A case where this wasn't working was bipbop-lateaudio.mp4, where the last frame
has duration 0 and starts and ends before the previous frame has ended. The
VideoSink still renders this frame at the end of playback, so this patch
brings DecodedStream closer to that behavior by rendering all frames with a
start time after the previous frame's start time. The track's duration is still
based on absolute times so things don't blow up.
Differential Revision: https://phabricator.services.mozilla.com/D27267
--HG--
extra : moz-landing-system : lando
This is similar to what VideoSink does, and is required for us to end playback
on HAVE_CURRENT_DATA (a frame for currentTime must be available).
Differential Revision: https://phabricator.services.mozilla.com/D27261
--HG--
extra : moz-landing-system : lando
This gives us a guarantee that the first frame of a media file can be rendered
with a second media element and mozCaptureStream(), even if the file is very
very short.
With longer video-only files there were also cases where nothing ended up being
rendered. Probably because the MediaStreamGraph ended up switching from an
AudioCallbackDriver to a SystemClockDriver and this took enough time to put the
SourceMediaStream::EndTrack and the SourceMediaStream::AddTrackListener calls
for this video track to be processed in the same iteration. The listener would
then always lose to the ending track and update main thread state too late,
leading to its media element not rendering any frames and nasty intermittent
failures.
Differential Revision: https://phabricator.services.mozilla.com/D27270
--HG--
extra : moz-landing-system : lando
A case where this wasn't working was bipbop-lateaudio.mp4, where the last frame
has duration 0 and starts and ends before the previous frame has ended. The
VideoSink still renders this frame at the end of playback, so this patch
brings DecodedStream closer to that behavior by rendering all frames with a
start time after the previous frame's start time. The track's duration is still
based on absolute times so things don't blow up.
Differential Revision: https://phabricator.services.mozilla.com/D27267
--HG--
extra : moz-landing-system : lando
This is similar to what VideoSink does, and is required for us to end playback
on HAVE_CURRENT_DATA (a frame for currentTime must be available).
Differential Revision: https://phabricator.services.mozilla.com/D27261
--HG--
extra : moz-landing-system : lando
If the input has been overflow, we don't need to calculate frames and will directly return invalid result to let caller handle the error.
Differential Revision: https://phabricator.services.mozilla.com/D27010
--HG--
extra : moz-landing-system : lando
If the input has been overflow, we don't need to calculate frames and will directly return invalid result to let caller handle the error.
Differential Revision: https://phabricator.services.mozilla.com/D27010
--HG--
extra : moz-landing-system : lando
This patch adds the number of dropped frames for each step of the process
(read/sink/compositor) and gives us more insight about where frames are
dropped, as opposed to the getVideoPlaybackQuality() API which gives the grand
total.
Differential Revision: https://phabricator.services.mozilla.com/D27488
--HG--
extra : moz-landing-system : lando
It's possible to overflow when we do the position calculation, we should only store the position which won't cause the integer overflow when adding it to start time.
Differential Revision: https://phabricator.services.mozilla.com/D25340
--HG--
extra : moz-landing-system : lando
VideoSegments still have durations, and they are still needed by the
MediaStreamGraph as it shuffles MediaSegments around.
They do not have a say in the wall-clock duration of video frames however.
Removing this should prevent any producers starting to add video chunks with
durations in the future.
Differential Revision: https://phabricator.services.mozilla.com/D22914
--HG--
extra : moz-landing-system : lando
DecodedStream has been basing video timestamps on something called
StreamTracksStartTimeStamp in MediaStreams, which call through all the way
down to the GraphDriver.
This removes the entire timestamp mechanism, except for a bit of legit
usage internally in the SystemClockDriver. Video timestamps are instead
based on the audio clock through GetPosition(), the same way the VideoSink
operates.
Differential Revision: https://phabricator.services.mozilla.com/D22896
--HG--
extra : moz-landing-system : lando
Add profiler marker to show when VideoSink sends frames to compositor
and when it discards frames that are outdated according to current
audio position.
Differential Revision: https://phabricator.services.mozilla.com/D22792
--HG--
extra : moz-landing-system : lando
mProcessedQueueLength is int32_t (wrapped around an Atomic) but it is updated from the method FramesToUsecs() which returns int64_t (wrapped around a CheckedInt) thus it is changed to unsigned int to avoid integer overflow.
Differential Revision: https://phabricator.services.mozilla.com/D22949
--HG--
extra : moz-landing-system : lando
It can be determined from the size of the buffer and the number of audio frames. Additionally, it ensures that the duration of the frame is always exactly what the AudioData contains.
Differential Revision: https://phabricator.services.mozilla.com/D20170
--HG--
extra : moz-landing-system : lando
Don't re-create a new trimmed AudioData when we want to remove some content. This remove the need for some copies.
Differential Revision: https://phabricator.services.mozilla.com/D20168
--HG--
extra : moz-landing-system : lando
This will allow to remove mFrames member and calculate from the size of the content, which will dynamically change depending on a cropping filter.
Differential Revision: https://phabricator.services.mozilla.com/D20165
--HG--
extra : moz-landing-system : lando
And we add some strong assertions that we never read passed the end of the buffer.
Differential Revision: https://phabricator.services.mozilla.com/D20162
--HG--
extra : moz-landing-system : lando
Avoid resetting volume in every AudioSink restart to stop unexpected volume change during play/pause and seek. Volume changes could occur if the volume has been modified outside firefox (for example pavucontrol in Linux).
Differential Revision: https://phabricator.services.mozilla.com/D15210
--HG--
extra : moz-landing-system : lando
The VideoSink shares the AudioSink's own EndedPromise to notify its user that it has ended. As such, the MozPromise used must be non-exclusive.
Using the GenericPromise for such purpose only hid that requirement.
We also remove the MediaSink from the media namespace, and clarify the naming of some arguments and class members to accurately describe what they do.
Differential Revision: https://phabricator.services.mozilla.com/D14024
--HG--
extra : moz-landing-system : lando
Andreas Pehrson
Ciure Andrei
Alastor Wu
arthur.iakab
Tarek Ziadé
Sylvestre Ledru
Csoregi Natalia
Narcis Beleuzu
John Lin
Mike Conley
Alex Chronopoulos
Will Hawkins
Chris Pearce
Jean-Yves Avenard
Ehsan Akhgari
Noemi Erli
Coroiu Cristina