VideoSegments still have durations, and they are still needed by the
MediaStreamGraph as it shuffles MediaSegments around.
They do not have a say in the wall-clock duration of video frames however.
Removing this should prevent any producers starting to add video chunks with
durations in the future.
Differential Revision: https://phabricator.services.mozilla.com/D22914
--HG--
extra : moz-landing-system : lando
Long-term we want to lift durations out of video altogether, and only use
wall-clock timestamps. This patch achieves this in VideoTrackEncoder.
When the MediaStreamGraph is audio-only, the equivalent for video will be
completely duration-less. Until we have that, some pieces around the MSG will
still need durations for track-bookkeeping reasons.
This also integrates the DriftCompensator into VideoTrackEncoder, by
compensating for drift when frames are moved from mIncomingBuffer to
mOutgoingBuffer, i.e., when we recalculate time stamps into durations for the
underlying encoder to use.
Differential Revision: https://phabricator.services.mozilla.com/D22909
--HG--
extra : moz-landing-system : lando
Because we sync video frames to audio time before encoding, we suspend and
resume on the audio thread because that's the only place we have acccess to
both clocks at the same time;
- A dispatch to the audio encoder puts the event in the right place in the
audio encoder buffer.
- TimeStamp::Now() is the current video time, and since we capture it on the
audio thread these are in sync. This timestamp needs drift compensation too -
this happens in a later patch on this bug.
Differential Revision: https://phabricator.services.mozilla.com/D22907
--HG--
extra : moz-landing-system : lando
This plumbs the DriftCompensator into the AudioTrackListener and
VideoTrackEncoder. The VideoTrackEncoder is however only finally integrated in
the future patch "Disregard VideoChunk durations in VideoTrackEncoder".
Differential Revision: https://phabricator.services.mozilla.com/D22903
--HG--
extra : moz-landing-system : lando
The webrtc-pc spec says:
> If track is ended, or if the track's output is disabled, i.e. the track is
> disabled and/or muted, the RTCRtpSender MUST send silence (audio),
> black frames (video) or a zero-information-content equivalent.
> In the case of video, the RTCRtpSender SHOULD send one black frame per second.
This patch covers the case when the output is disabled, and the case when no
frames reach the MediaPipeline, for both direct and non-direct video listeners.
Differential Revision: https://phabricator.services.mozilla.com/D22898
--HG--
extra : moz-landing-system : lando
DecodedStream sends video to its video tracks by initially buffering a set
of images, then appending future ones by adding them one by one.
A long time ago we refactored how MediaStreamGraph sends images to the screen,
i.e., to an ImageContainer. It used to send all future frames to
ImageContainer::SetCurrentFrames each time it sent something. After the
refactor we just forward any new frame from a direct listener to
ImageContainer::SetCurrentFrames.
So in case DecodedStream has already sent 10 future frames to its track,
and sends another, we end up calling ImageContainer::SetCurrentFrames(frame11).
However, this is not how ImageContainer works. The refactor was wrong.
Even though the timestamp for frame11 is after a previously buffered frame,
it will be ignored. SetCurrentFrames wipes any previously set frames. Hence
the word "Current" in its name.
This patch largely restores the old behaviour by adding a thin buffering layer
between the MSG (in a direct listener) and the ImageContainer.
This does not give 100% identical frame sync to VideoSink (how we normally
render video), because VideoSink can update the timestamps of already-pushed
images by pushing them again. We can't do that here because the SourceMediaStream
API only allows appending. It does however get in sync for frames appended after
the first frame has been rendered.
Differential Revision: https://phabricator.services.mozilla.com/D22897
--HG--
extra : moz-landing-system : lando
DecodedStream has been basing video timestamps on something called
StreamTracksStartTimeStamp in MediaStreams, which call through all the way
down to the GraphDriver.
This removes the entire timestamp mechanism, except for a bit of legit
usage internally in the SystemClockDriver. Video timestamps are instead
based on the audio clock through GetPosition(), the same way the VideoSink
operates.
Differential Revision: https://phabricator.services.mozilla.com/D22896
--HG--
extra : moz-landing-system : lando
The way it's implemented it only adds plumbing and overhead, no value.
This patch moves it to a thin wrapper around DirectMediaStreamTrackListener,
managed by VideoStreamTrack, instead.
Differential Revision: https://phabricator.services.mozilla.com/D22895
--HG--
extra : moz-landing-system : lando
The current setup uses different ways for different platforms, with
different workarounds, even using extra configuration items for Windows.
Now that there can't be a difference between the host per the build
system and the host per rust, we can get rid of those configuration
items, and use a more common infrastructure.
We cannot, however, avoid using wrapper scripts, because per-target rust
link-arg flags don't work up great.
The downside is that multiplies the number of wrappers, as we now have
to have a different one for host and target, and then we have .bat files
and shell scripts for, respectively, Windows hosts, and other hosts.
Depends on D24321
Differential Revision: https://phabricator.services.mozilla.com/D24322
--HG--
extra : moz-landing-system : lando
While the substitution pattern is kind of awful in make, it will allow
to more straightforwardly deal with the difference between target and
host.
Differential Revision: https://phabricator.services.mozilla.com/D24321
--HG--
extra : moz-landing-system : lando
The code that actually downloads it is behind a condition that isn't set
anywhere.
Differential Revision: https://phabricator.services.mozilla.com/D24215
--HG--
extra : moz-landing-system : lando
A note about the Scalar Replacement Changes: The M{Load,Store}Unboxed*
instructions in theory could be used to manipulate and analyze typed arrays.
However, according to nbp TypedArrays should already be excluded from elibility
because of the potential for cross-thread sharing in a SharedArrayBuffer world,
and so the only support in Scalar Replacement here is for Unboxed Objects, and
so it can all be removed.
Differential Revision: https://phabricator.services.mozilla.com/D24038
--HG--
extra : moz-landing-system : lando
Andreas Pehrson
Mike Hommey
Andreea Pavel
Lars T Hansen
Daniel Holbert
Florian Quèze
Tom Prince
Matthew Gaudet