This is essentially the same code as the interleaved version, but backwards,
since the memory layout is the opposite, we want to take advantage of memory
locality, and only touch audio samples once each.
The `ProcessAudioData` method has been renamed, because of the similarity
between the `AudioData` type and `ProcessAudioData`, since it can now process
`AudioBlock`s.
Differential Revision: https://phabricator.services.mozilla.com/D90431
This is essentially the same code as the interleaved version, but backwards,
since the memory layout is the opposite, we want to take advantage of memory
locality, and only touch audio samples once each.
The `ProcessAudioData` method has been renamed, because of the similarity
between the `AudioData` type and `ProcessAudioData`, since it can now process
`AudioBlock`s.
Differential Revision: https://phabricator.services.mozilla.com/D90431
This patch was generated by running:
```
perl -p -i \
-e 's/^(\s+)([a-zA-Z0-9.]+) = NS_ConvertUTF8toUTF16\((.*)\);/\1CopyUTF8toUTF16(\3, \2);/;' \
-e 's/^(\s+)([a-zA-Z0-9.]+) = NS_ConvertUTF16toUTF8\((.*)\);/\1CopyUTF16toUTF8(\3, \2);/;' \
$FILE
```
against every .cpp and .h in mozilla-central, and then fixing up the
inevitable errors that happen as a result of matching C++ expressions with
regexes. The errors fell into three categories:
1. Calling the convert functions with `std::string::c_str()`; these were
changed to simply pass the string instead, relying on implicit conversion
to `mozilla::Span`.
2. Calling the convert functions with raw pointers, which is not permitted
with the copy functions; these were changed to invoke `MakeStringSpan` first.
3. Other miscellaneous errors resulting from over-eager regexes and/or the
replacement not being type-aware. These changes were reverted.
Differential Revision: https://phabricator.services.mozilla.com/D88903
I want to use `PROFILER_MARKER` in the new C++ marker APIs, so I'm renaming this single (recent) use to something else.
Differential Revision: https://phabricator.services.mozilla.com/D87240
This patch will do :
- use `MediaSampleMarkerPayload` to replace `VideoFrameMarkerPayload`
The advantage of doing so :
- after finishing a support for `MediaSampleMarkerPayload` in profiler front-end side, we can show the sample's start and end time when hovering on the marker icon.
Differential Revision: https://phabricator.services.mozilla.com/D75469
This patch will do :
- add `Media` markers in related codes
- implement `MediaSampleMarkerPayload` to record the media sample
The advantage of doing so :
- using markers can help us know what happens on the media by a glance without expanding the call stack
- adding sample markers allows us compare the speed of decoding sample in `MediaDecoderStataMachine` and rendering sample in `VideoSink`
Differential Revision: https://phabricator.services.mozilla.com/D74174
This patch will do :
- create a profiling category `Media`
- add `Media` profiling labels in related codes
The advantage of doing so :
- allow us to easily see what operations are related to media playback from the profiled report
More details :
According to the description in the `ProfilingCategory.h`, `topmost profiler label frame in the label stack determines the category pair of that stack`. Therefore, most labels I added are the first task would run on the thread, in order to ensure all its following tasks can be marked as the media playback label as well.
Differential Revision: https://phabricator.services.mozilla.com/D74171
`AudioSink` provides a way to let us know if the media data is audible, and we should do the same thing for the `DecodedStream`, in order to update the correct audible state.
Differential Revision: https://phabricator.services.mozilla.com/D70259
This patch also tries to remove the event target entirely if it would
default to the main thread on a null event target.
Depends on D67634
Differential Revision: https://phabricator.services.mozilla.com/D67635
--HG--
extra : moz-landing-system : lando
This removes most dependencies on BlocksRingBuffer, to ease the transition to
the upcoming Fission-friendly profile buffer, including:
- Length type,
- SumBytes(),
- Gecko extensions of serialization.
Differential Revision: https://phabricator.services.mozilla.com/D66722
--HG--
rename : tools/profiler/public/BlocksRingBufferGeckoExtensions.h => tools/profiler/public/ProfileBufferEntrySerializationGeckoExtensions.h
extra : moz-landing-system : lando
The new ProfileBufferEntryReader/Writer are now used everywhere, including in
the profilers and tests.
The old EntryReader/Writer have been removed.
Differential Revision: https://phabricator.services.mozilla.com/D65697
--HG--
extra : moz-landing-system : lando
`mPrincipalHandle` and `mPlaying` listeners were being connected in the ctor of a DecodedStream. However, this is not necessary because their attributes will only be modified after the sink start. In addition to that, it was causing problems if a sink was replaced before being started or stopped (and shutdown). This is a valid scenario, though, that we need to support.
Differential Revision: https://phabricator.services.mozilla.com/D63830
--HG--
extra : moz-landing-system : lando
Bug 1600063 changed the chunk-dropping logic from dropping when we had *started*
playing a chunk, to when we had *finished* playing a chunk -- i.e., started
playing the next chunk.
But when a chunk is the very last chunk in the media resource, we cannot start
playing the next chunk, because there is no next chunk. Making this check
inclusive seems reasonable.
Differential Revision: https://phabricator.services.mozilla.com/D57374
--HG--
extra : moz-landing-system : lando
Rather than applying the volume change to the DecodedStream, which won't be
played until the buffer catches up, instead apply the volume to the segments
as they are consumed by the MediaTrackGraph.
Differential Revision: https://phabricator.services.mozilla.com/D56276
--HG--
extra : moz-landing-system : lando
As we added new files, which affects the build bundle created by unified build. Therefore, add missing scope definition and included header to pass the build.
Differential Revision: https://phabricator.services.mozilla.com/D52430
--HG--
extra : moz-landing-system : lando
As we added new files, which affects the build bundle created by unified build. Therefore, add missing scope definition and included header to pass the build.
Differential Revision: https://phabricator.services.mozilla.com/D52430
--HG--
extra : moz-landing-system : lando
Prior to this patch, promises were always created. Since bug 1172394, creating
DecodedStreamData is conditional on whether the appropriate output tracks are
available. If promises are created and output tracks are *not* available, the
promises would never be resolved or rejected, and would assert on destruction.
This patch makes it so promises are only created if we create DecodedStreamData,
hence only when output tracks are available and the promises are guaranteed to
be resolved or rejected at some point.
Differential Revision: https://phabricator.services.mozilla.com/D53691
--HG--
extra : moz-landing-system : lando
This patches does several minor things:
- Moves SetSink (from setSinkid) to automatic coalescing of multiple calls
through a Canonical/Mirror setup instead of a manual atomic counter.
- Simplifies the logic for when to update the sink in SetSink.
- Removes PlaybackParams as a general MediaSink property, as it only contains
audio params.
- Makes PlaybackParams an internal AudioSink concept, that AudioSinkWrapper
knows about.
- Ensures mMediaSink is only accessed on the decoder TaskQueue, to allow
accessing mirrored members when creating it.
Differential Revision: https://phabricator.services.mozilla.com/D52043
--HG--
extra : moz-landing-system : lando
This patch removes the responsibility of js-facing MediaStreamTracks from the
MediaDecoder stack, and moves the machinery for setting up DecodedStream to
higher order functions like state mirroring and watchables.
OutputStreamManager is completely gone, since it was designed to manage
MediaStreamTracks across multiple output streams for a single decoder,
on main thread. HTMLMediaElement took over its task in the previous patch.
The MediaDecoderStateMachine now has three control points for capturing:
- mOutputCaptured, which, if true, will capture all decoded data into
mOutputTracks. If this is set, but mOutputTracks is empty, we are still
waiting for tracks, and DecodedStream will not play any data. When tracks are
set, a new DecodedStream is created that will play data through
SourceMediaTracks piped into mOutputTracks.
- mOutputTracks, which is the set of tracks data is captured into, for
forwarding to all the output tracks the media element is managing. This set of
tracks is managed by the MediaDecoder owner, and must contain one audio track
if the decoder is decoding audio, and one video track if the decoder is
decoding video. It may be empty since output can be captured before metadata
is loaded, or playback has ended.
- mOutputPrincipal, which is the principal of the decoded data. All data sent
into SourceMediaTracks is tagged with this principal.
Differential Revision: https://phabricator.services.mozilla.com/D52042
--HG--
extra : moz-landing-system : lando
This patches does several minor things:
- Moves SetSink (from setSinkid) to automatic coalescing of multiple calls
through a Canonical/Mirror setup instead of a manual atomic counter.
- Simplifies the logic for when to update the sink in SetSink.
- Removes PlaybackParams as a general MediaSink property, as it only contains
audio params.
- Makes PlaybackParams an internal AudioSink concept, that AudioSinkWrapper
knows about.
- Ensures mMediaSink is only accessed on the decoder TaskQueue, to allow
accessing mirrored members when creating it.
Differential Revision: https://phabricator.services.mozilla.com/D52043
--HG--
extra : moz-landing-system : lando
This patch removes the responsibility of js-facing MediaStreamTracks from the
MediaDecoder stack, and moves the machinery for setting up DecodedStream to
higher order functions like state mirroring and watchables.
OutputStreamManager is completely gone, since it was designed to manage
MediaStreamTracks across multiple output streams for a single decoder,
on main thread. HTMLMediaElement took over its task in the previous patch.
The MediaDecoderStateMachine now has three control points for capturing:
- mOutputCaptured, which, if true, will capture all decoded data into
mOutputTracks. If this is set, but mOutputTracks is empty, we are still
waiting for tracks, and DecodedStream will not play any data. When tracks are
set, a new DecodedStream is created that will play data through
SourceMediaTracks piped into mOutputTracks.
- mOutputTracks, which is the set of tracks data is captured into, for
forwarding to all the output tracks the media element is managing. This set of
tracks is managed by the MediaDecoder owner, and must contain one audio track
if the decoder is decoding audio, and one video track if the decoder is
decoding video. It may be empty since output can be captured before metadata
is loaded, or playback has ended.
- mOutputPrincipal, which is the principal of the decoded data. All data sent
into SourceMediaTracks is tagged with this principal.
Differential Revision: https://phabricator.services.mozilla.com/D52042
--HG--
extra : moz-landing-system : lando
Paul Adenot
Razvan Maries
Cristina Coroiu
Nathan Froyd
Gerald Squelart
Alex Chronopoulos
alwu
Simon Giesecke
Andreas Farre
Andreas Pehrson
Jon Bauman
Noemi Erli
Daniel Varga
Csoregi Natalia
Jim Porter