If MediaEncoder::Shutdown() is called on track end, then track listener
removal has not necessarily been triggered. Waiting on track listener
shutdown waits for removal of track listeners. Invoking
Session::EncoderListener::Shutdown() triggers MediaEncoder::Stop(), which
triggers track listener removal.
This changeset reverts the unnecessary changes in
https://hg.mozilla.org/mozilla-central/rev/509719f98963 that delayed calls to
Session::EncoderListener::Shutdown() until after track listener shutdown,
which would not happen without track listener removal.
IsShutdown() is replaced with a AssertShutdownCalled() because testing
mShutdownPromise would race if the timing of Shutdown() is not known.
Differential Revision: https://phabricator.services.mozilla.com/D65067
--HG--
extra : moz-landing-system : lando
This converts `nsAutoPtr` usage in dom/media to `UniquePtr`. Beyond just a
search and replace we also needed to update assignment and access of the
`UniquePtr`s. This falls into a few categories:
- Assignment from a newly constructed object switches to `MakeUnique`
- Assignment from a raw ptr switches to `UniquePtr::reset`
- Handing out a raw ptr now requires `UniquePtr::get`
- Uses `UniquePtr::release` rather than `nsAutoPtr::forget`
- A few spots are updated to return a `UniquePtr` rather than a raw ptr
Differential Revision: https://phabricator.services.mozilla.com/D60084
--HG--
extra : moz-landing-system : lando
This changeset is a simple find and replace of `MOZ_FALLTHROUGH` and `[[fallthrough]]`.
Unfortunately, the MOZ_FALLTHROUGH_ASSERT macro (to assert on case fallthrough in debug builds) is still necessary after switching from [[clang::fallthrough]] to [[fallthrough]] because:
* MOZ_ASSERT(false) followed by [[fallthrough]] triggers a -Wunreachable-code warning in DEBUG builds
* but MOZ_ASSERT(false) without [[fallthrough]] triggers a -Wimplicit-fallthrough warning in NDEBUG builds.
Differential Revision: https://phabricator.services.mozilla.com/D56440
--HG--
extra : moz-landing-system : lando
The inclusions were removed with the following very crude script and the
resulting breakage was fixed up by hand. The manual fixups did either
revert the changes done by the script, replace a generic header with a more
specific one or replace a header with a forward declaration.
find . -name "*.idl" | grep -v web-platform | grep -v third_party | while read path; do
interfaces=$(grep "^\(class\|interface\).*:.*" "$path" | cut -d' ' -f2)
if [ -n "$interfaces" ]; then
if [[ "$interfaces" == *$'\n'* ]]; then
regexp="\("
for i in $interfaces; do regexp="$regexp$i\|"; done
regexp="${regexp%%\\\|}\)"
else
regexp="$interfaces"
fi
interface=$(basename "$path")
rg -l "#include.*${interface%%.idl}.h" . | while read path2; do
hits=$(grep -v "#include.*${interface%%.idl}.h" "$path2" | grep -c "$regexp" )
if [ $hits -eq 0 ]; then
echo "Removing ${interface} from ${path2}"
grep -v "#include.*${interface%%.idl}.h" "$path2" > "$path2".tmp
mv -f "$path2".tmp "$path2"
fi
done
fi
done
Differential Revision: https://phabricator.services.mozilla.com/D55442
--HG--
extra : moz-landing-system : lando
The inclusions were removed with the following very crude script and the
resulting breakage was fixed up by hand. The manual fixups did either
revert the changes done by the script, replace a generic header with a more
specific one or replace a header with a forward declaration.
find . -name "*.idl" | grep -v web-platform | grep -v third_party | while read path; do
interfaces=$(grep "^\(class\|interface\).*:.*" "$path" | cut -d' ' -f2)
if [ -n "$interfaces" ]; then
if [[ "$interfaces" == *$'\n'* ]]; then
regexp="\("
for i in $interfaces; do regexp="$regexp$i\|"; done
regexp="${regexp%%\\\|}\)"
else
regexp="$interfaces"
fi
interface=$(basename "$path")
rg -l "#include.*${interface%%.idl}.h" . | while read path2; do
hits=$(grep -v "#include.*${interface%%.idl}.h" "$path2" | grep -c "$regexp" )
if [ $hits -eq 0 ]; then
echo "Removing ${interface} from ${path2}"
grep -v "#include.*${interface%%.idl}.h" "$path2" > "$path2".tmp
mv -f "$path2".tmp "$path2"
fi
done
fi
done
Differential Revision: https://phabricator.services.mozilla.com/D55442
--HG--
extra : moz-landing-system : lando
If the VP8 encoder fails to reconfigure in the middle of a stream it can
transition from a state where it was returning `true` for `IsInitialized` to
then returning false. This would trigger an assert in `GetEncodedData` in
`MediaEncoder`. This patch changes to instead handle this case by early
returning if we're in an error state, and ensuring we enter the error state if
a track encoder fails to reconfigure.
Update the TrackEncoder contract to differentiate the error used if encoding has
been canceled or completed, from other, more severe errors.
This also adjusts the logs in the vp8 track encoder to more clearly log if a
reconfigure of the encoder failed or not.
Differential Revision: https://phabricator.services.mozilla.com/D51956
--HG--
extra : moz-landing-system : lando
This does a bit of a cleanup, where changing the notification to waiting for
removal is the major task. It also removes the special handling of not informing
listeners of shutdown on Cancel(), and a bit of cleanup around MozPromise usage.
Differential Revision: https://phabricator.services.mozilla.com/D49416
--HG--
extra : moz-landing-system : lando
This is the last step to making the MediaRecorder's mime type handling up to
spec, and has a real effect only if the constrained mime type is empty, or doesn't
contain any codecs.
Differential Revision: https://phabricator.services.mozilla.com/D17809
--HG--
extra : moz-landing-system : lando
This leaves out support for extending the mime type with the selected container
and codecs, and support in MediaEncoder for using a specific mime type.
Differential Revision: https://phabricator.services.mozilla.com/D46463
--HG--
extra : moz-landing-system : lando
This aligns it better with MediaRecorder's timeslice which was changed from
int32 to uint32 earlier.
Differential Revision: https://phabricator.services.mozilla.com/D41586
--HG--
extra : moz-landing-system : lando
This change mainly removes the `mTracks` member from MediaStream and moves all
associated members up a level, so that a MediaStream in practice represents a
single track.
Classes will be renamed in a future patch to reflect this.
Other changes include:
The new `mEnded` member of MediaStream changes meaning to only become true when
all data in the stream has been processed. It stems from
StreamTracks::Track::mEnded which used to become true as soon as the last bit of
data had been added to a track, and there could still be data in the track that
would get processed in future iterations. We are moving towards not having any
future data in tracks, which is why this change is ok to make -- keeping the old
behavior will soon not make sense.
TrackUnionStream is changed to no longer take a list of streams as input and
forward the union of their tracks to itself. Instead it's limited to having one
track as input at a time.
The autofinishing functionality that TrackUnionStream had before has been
transformed into an autoending functionality to allow it to defer ending until
its been told that it's ok to end through the control API. This lets a single
TrackUnionStream span the lifetime of multiple inputs, which will be useful for
making DecodedStream spec compliant with HTMLMediaElement::CaptureStream(), and
for implementing the currently discussed MediaRecorder::ReplaceTrack(), to name
a few potential use cases.
AudioNodeStreams used to only have a track (and thus an AudioSegment) if the
EXTERNAL_OUTPUT flag was enabled on them. With all MediaStreams now representing
a track, AudioNodeStreams inherently have an AudioSegment as a member. It is
however only used with data if the EXTERNAL_OUTPUT flag is enabled.
Differential Revision: https://phabricator.services.mozilla.com/D45821
--HG--
extra : moz-landing-system : lando
This functionality is not used since we moved to only having a single track per
MediaStream (bug 1493613).
Differential Revision: https://phabricator.services.mozilla.com/D47686
--HG--
extra : moz-landing-system : lando
This change mainly removes the `mTracks` member from MediaStream and moves all
associated members up a level, so that a MediaStream in practice represents a
single track.
Classes will be renamed in a future patch to reflect this.
Other changes include:
The new `mEnded` member of MediaStream changes meaning to only become true when
all data in the stream has been processed. It stems from
StreamTracks::Track::mEnded which used to become true as soon as the last bit of
data had been added to a track, and there could still be data in the track that
would get processed in future iterations. We are moving towards not having any
future data in tracks, which is why this change is ok to make -- keeping the old
behavior will soon not make sense.
TrackUnionStream is changed to no longer take a list of streams as input and
forward the union of their tracks to itself. Instead it's limited to having one
track as input at a time.
The autofinishing functionality that TrackUnionStream had before has been
transformed into an autoending functionality to allow it to defer ending until
its been told that it's ok to end through the control API. This lets a single
TrackUnionStream span the lifetime of multiple inputs, which will be useful for
making DecodedStream spec compliant with HTMLMediaElement::CaptureStream(), and
for implementing the currently discussed MediaRecorder::ReplaceTrack(), to name
a few potential use cases.
AudioNodeStreams used to only have a track (and thus an AudioSegment) if the
EXTERNAL_OUTPUT flag was enabled on them. With all MediaStreams now representing
a track, AudioNodeStreams inherently have an AudioSegment as a member. It is
however only used with data if the EXTERNAL_OUTPUT flag is enabled.
Differential Revision: https://phabricator.services.mozilla.com/D45821
--HG--
extra : moz-landing-system : lando
This functionality is not used since we moved to only having a single track per
MediaStream (bug 1493613).
Differential Revision: https://phabricator.services.mozilla.com/D47686
--HG--
extra : moz-landing-system : lando
To accomodate this, MediaStreamTrackListener::NotifyEnded/Removed get an
additional aGraph argument. NotifyEnabledStateChanged is getting it too, for
completion.
Differential Revision: https://phabricator.services.mozilla.com/D43667
--HG--
extra : moz-landing-system : lando
This first of all does some refactoring of how metadata is encoded in
MediaEncoder. This is now guided by the new Muxer class. If we're ready to pass
data to the muxer and it does not have metadata yet, we provide metadata before
giving it any media data. This metadata is passed to the muxer in a single call.
The metadata provided in this call must stay valid for the entire recording.
This removes MediaEncoder::GetEncodedMetadata().
This also removes the ctor argument from the WebMWriter since it can now rely on
the single SetMetadata() instead.
To comply with the ContainerWriter::SetMetadata() docs,
WebMWriter::SetMetadata() will now also sanity check metadata.
ContainerWriter instances are updated somewhat, to accommodate these changes.
Lastly, and most important, the new Muxer class manages muxing of the (up to)
two tracks into a single container, ensuring that timestamps increase
monotonically throughout a recording.
Differential Revision: https://phabricator.services.mozilla.com/D35306
--HG--
extra : moz-landing-system : lando
Update MediaEncoder to pass frames to the muxer in order of their time stamps.
This should prevent the currently possible scenario where audio and video
frames are written with non-monotonically increasing timestamps (in violation
of the webm spec).
Differential Revision: https://phabricator.services.mozilla.com/D35388
--HG--
extra : moz-landing-system : lando
MediaQueue provides a better interface for interleaving frames when writing to
the muxer (this change will follow in another changeset). The queue interface
provides a nicer abstraction than manually managing a nsTArray.
Differential Revision: https://phabricator.services.mozilla.com/D35387
--HG--
extra : moz-landing-system : lando
This changes EncodedFrame to behave more like MediaData, so that EncodedFrame
can be used with the MediaQueue data structure. It also provides a somewhat
more consistent interface across media data types.
Differential Revision: https://phabricator.services.mozilla.com/D35386
--HG--
extra : moz-landing-system : lando
Move the responsibility of adjusting opus frame timestamps to the MediaEncoder.
This was previously done by the EbmlComposer, but doing so in the MediaEncoder
means we can have greater control over handling of time codes and interleaving
of frames.
Differential Revision: https://phabricator.services.mozilla.com/D35385
--HG--
extra : moz-landing-system : lando
Remove EncodedFrameContainer and clean up areas where it was used.
EncodedFrameContainer provided a wrapper around an
nsTArray<RefPtr<EncodedFrame>>, but it simplifies the code to simply expose
this array. Also clean up unused enums in EncodedFrame, and clean up some of
the outdated comments for our encoded frame handling.
Differential Revision: https://phabricator.services.mozilla.com/D35384
--HG--
rename : dom/media/encoder/EncodedFrameContainer.h => dom/media/encoder/EncodedFrame.h
extra : moz-landing-system : lando
Separating the encode and mux steps allows for better control over interleaving
audio and video data. If encode and mux are done in a single step it's possible
to mux large amounts of audio or video data which should have been interleaved
with the other data type to give correctly ordered time stamps in the target
container.
Differential Revision: https://phabricator.services.mozilla.com/D35383
--HG--
extra : moz-landing-system : lando
If webm is disabled or not compiled in, the video assert would fail if trying
to record a video track -- we'd only have an ogg/opus encoder.
Differential Revision: https://phabricator.services.mozilla.com/D40648
--HG--
extra : moz-landing-system : lando
This first of all does some refactoring of how metadata is encoded in
MediaEncoder. This is now guided by the new Muxer class. If we're ready to pass
data to the muxer and it does not have metadata yet, we provide metadata before
giving it any media data. This metadata is passed to the muxer in a single call.
The metadata provided in this call must stay valid for the entire recording.
This removes MediaEncoder::GetEncodedMetadata().
This also removes the ctor argument from the WebMWriter since it can now rely on
the single SetMetadata() instead.
To comply with the ContainerWriter::SetMetadata() docs,
WebMWriter::SetMetadata() will now also sanity check metadata.
ContainerWriter instances are updated somewhat, to accommodate these changes.
Lastly, and most important, the new Muxer class manages muxing of the (up to)
two tracks into a single container, ensuring that timestamps increase
monotonically throughout a recording.
Differential Revision: https://phabricator.services.mozilla.com/D35306
--HG--
extra : moz-landing-system : lando
Update MediaEncoder to pass frames to the muxer in order of their time stamps.
This should prevent the currently possible scenario where audio and video
frames are written with non-monotonically increasing timestamps (in violation
of the webm spec).
Differential Revision: https://phabricator.services.mozilla.com/D35388
--HG--
extra : moz-landing-system : lando
MediaQueue provides a better interface for interleaving frames when writing to
the muxer (this change will follow in another changeset). The queue interface
provides a nicer abstraction than manually managing a nsTArray.
Differential Revision: https://phabricator.services.mozilla.com/D35387
--HG--
extra : moz-landing-system : lando
This changes EncodedFrame to behave more like MediaData, so that EncodedFrame
can be used with the MediaQueue data structure. It also provides a somewhat
more consistent interface across media data types.
Differential Revision: https://phabricator.services.mozilla.com/D35386
--HG--
extra : moz-landing-system : lando
Move the responsibility of adjusting opus frame timestamps to the MediaEncoder.
This was previously done by the EbmlComposer, but doing so in the MediaEncoder
means we can have greater control over handling of time codes and interleaving
of frames.
Differential Revision: https://phabricator.services.mozilla.com/D35385
--HG--
extra : moz-landing-system : lando
Remove EncodedFrameContainer and clean up areas where it was used.
EncodedFrameContainer provided a wrapper around an
nsTArray<RefPtr<EncodedFrame>>, but it simplifies the code to simply expose
this array. Also clean up unused enums in EncodedFrame, and clean up some of
the outdated comments for our encoded frame handling.
Differential Revision: https://phabricator.services.mozilla.com/D35384
--HG--
rename : dom/media/encoder/EncodedFrameContainer.h => dom/media/encoder/EncodedFrame.h
extra : moz-landing-system : lando
Separating the encode and mux steps allows for better control over interleaving
audio and video data. If encode and mux are done in a single step it's possible
to mux large amounts of audio or video data which should have been interleaved
with the other data type to give correctly ordered time stamps in the target
container.
Differential Revision: https://phabricator.services.mozilla.com/D35383
--HG--
extra : moz-landing-system : lando
This first of all does some refactoring of how metadata is encoded in
MediaEncoder. This is now guided by the new Muxer class. If we're ready to pass
data to the muxer and it does not have metadata yet, we provide metadata before
giving it any media data. This metadata is passed to the muxer in a single call.
The metadata provided in this call must stay valid for the entire recording.
This removes MediaEncoder::GetEncodedMetadata().
This also removes the ctor argument from the WebMWriter since it can now rely on
the single SetMetadata() instead.
To comply with the ContainerWriter::SetMetadata() docs,
WebMWriter::SetMetadata() will now also sanity check metadata.
ContainerWriter instances are updated somewhat, to accommodate these changes.
Lastly, and most important, the new Muxer class manages muxing of the (up to)
two tracks into a single container, ensuring that timestamps increase
monotonically throughout a recording.
Differential Revision: https://phabricator.services.mozilla.com/D35306
--HG--
extra : moz-landing-system : lando
Update MediaEncoder to pass frames to the muxer in order of their time stamps.
This should prevent the currently possible scenario where audio and video
frames are written with non-monotonically increasing timestamps (in violation
of the webm spec).
Differential Revision: https://phabricator.services.mozilla.com/D35388
--HG--
extra : moz-landing-system : lando
Simon Giesecke
Karl Tomlinson
Eric Rahm
Chris Peterson
Gabriele Svelto
Dorel Luca
Csoregi Natalia
Bryce Seager van Dyk
Andreas Pehrson
Gurzau Raul
Andreea Pavel