TrackID has been used to denote tracks that are part of MediaStreams in the
MediaStreamGraph. The TrackInfo::mTrackId IDs are no longer used in the
MediaStream APIs and as such it's not necessary that they have the type TrackID.
This patch changes TrackInfo::mTrackId to be of type uint32_t instead, as that
is the type of Mp4parseTrackInfo::track_id, which is assigned to
TrackInfo::mTrackId as the sole remaining non-static user.
Other users set a static trackId of `1` or `2`, and as such remain supported by
this change.
Differential Revision: https://phabricator.services.mozilla.com/D46763
--HG--
extra : moz-landing-system : lando
TrackID has been used to denote tracks that are part of MediaStreams in the
MediaStreamGraph. The TrackInfo::mTrackId IDs are no longer used in the
MediaStream APIs and as such it's not necessary that they have the type TrackID.
This patch changes TrackInfo::mTrackId to be of type uint32_t instead, as that
is the type of Mp4parseTrackInfo::track_id, which is assigned to
TrackInfo::mTrackId as the sole remaining non-static user.
Other users set a static trackId of `1` or `2`, and as such remain supported by
this change.
Differential Revision: https://phabricator.services.mozilla.com/D46763
--HG--
extra : moz-landing-system : lando
This replaces the instances where Length was being called on an interval set
to determine if it was empty or not empty. IsEmpty makes the intention in these
cases more immediately obvious, and avoids the need to think about ints as
bools in cases like `if(!mySet.Length())`.
Depends on D46638
Differential Revision: https://phabricator.services.mozilla.com/D46639
--HG--
extra : moz-landing-system : lando
This removes all telemetry which expired in Firefox 69 or earlier, with the
exceptions of the following, which we plan to renew:
* AUDIO_TRACK_SILENCE_PROPORTION
* MEDIA_AUTOPLAY_WOULD_BE_ALLOWED_COUNT
* MEDIA_AUTOPLAY_WOULD_NOT_BE_ALLOWED_COUNT
* MEDIACACHESTREAM_LENGTH_KB
* MEDIA_MKV_CANPLAY_REQUESTED
* MEDIA_PAGE_COUNT
* MEDIA_PAGE_HAD_MEDIA_COUNT
* VIDEO_DROPPED_FRAMES_PROPORTION
* VIDEO_PLAY_TIME
* VIDEO_HIDDEN_PLAY_TIME
* VIDEO_HIDDEN_PLAY_TIME_PERCENTAGE
* VIDEO_INFERRED_DECODE_SUSPEND_PERCENTAGE
* VIDEO_INTER_KEYFRAME_AVERAGE_MS
* VIDEO_INTER_KEYFRAME_MAX_MS
* VIDEO_SUSPEND_RECOVERY_TIME_MS
* VIDEO_VP9_BENCHMARK_FPS
* WEB_AUDIO_BECOMES_AUDIBLE_TIME
* WEBVTT_TRACK_KINDS
Differential Revision: https://phabricator.services.mozilla.com/D37313
--HG--
extra : moz-landing-system : lando
This removes all telemetry which expired in Firefox 69 or earlier, with the
exceptions of the following, which we plan to renew:
* AUDIO_TRACK_SILENCE_PROPORTION
* MEDIA_AUTOPLAY_WOULD_BE_ALLOWED_COUNT
* MEDIA_AUTOPLAY_WOULD_NOT_BE_ALLOWED_COUNT
* MEDIACACHESTREAM_LENGTH_KB
* MEDIA_MKV_CANPLAY_REQUESTED
* MEDIA_PAGE_COUNT
* MEDIA_PAGE_HAD_MEDIA_COUNT
* VIDEO_DROPPED_FRAMES_PROPORTION
* VIDEO_PLAY_TIME
* VIDEO_HIDDEN_PLAY_TIME
* VIDEO_HIDDEN_PLAY_TIME_PERCENTAGE
* VIDEO_INFERRED_DECODE_SUSPEND_PERCENTAGE
* VIDEO_INTER_KEYFRAME_AVERAGE_MS
* VIDEO_INTER_KEYFRAME_MAX_MS
* VIDEO_SUSPEND_RECOVERY_TIME_MS
* VIDEO_VP9_BENCHMARK_FPS
* WEB_AUDIO_BECOMES_AUDIBLE_TIME
* WEBVTT_TRACK_KINDS
Differential Revision: https://phabricator.services.mozilla.com/D37313
--HG--
extra : moz-landing-system : lando
This requires replacing inclusions of it with inclusions of more specific prefs
files.
The exception is that StaticPrefsAll.h, which is equivalent to StaticPrefs.h,
and is used in `Codegen.py` because doing something smarter is tricky and
suitable for a follow-up. As a result, any change to StaticPrefList.yaml will
still trigger recompilation of all the generated DOM bindings files, but that's
still a big improvement over trigger recompilation of every file that uses
static prefs.
Most of the changes in this commit are very boring. The only changes that are
not boring are modules/libpref/*, Codegen.py, and ServoBindings.toml.
Differential Revision: https://phabricator.services.mozilla.com/D39138
--HG--
extra : moz-landing-system : lando
We should always append sample by using `AppendSample()` to assert whether a sample is valid, so making `mSamples` private can prevent a direct usage of the nsTarry's `AppendElement()`.
Also provide a method to return non-const mSamples which is only used for the move semantics.
Differential Revision: https://phabricator.services.mozilla.com/D38427
--HG--
extra : moz-landing-system : lando
Return demux error when we get a invalid sample. For mp4 demuxer, we use `MOZ_DIAGNOSTIC_ASSERT` instead because we are pretty sure that it won't happen.
Differential Revision: https://phabricator.services.mozilla.com/D38553
--HG--
extra : moz-landing-system : lando
We don't want to have a sample with invalid time, duration, end time or end timecode, so add a diagnostic assertion to check. And will handle invalid sample for demuxers in next patch.
Differential Revision: https://phabricator.services.mozilla.com/D38426
--HG--
extra : moz-landing-system : lando
This is a refactor of the encryption scheme detection used in Index.cpp aside
from one additional check: the new function verifies that if sample group
information is present that it is consistent with other encryption information.
Introduce a new function that contains all the logic to detect the encryption
scheme used for the current sample. This helps separate the logic of scheme
detection from setting scheme specific data on the current sample. This makes
the steps easier to understand and change in isolation.
This function returns a result, with failure cases containing a string
explaining the error. This is motivated by being able to log or surface the
error for failure cases. Logging is not yet implemented, but will be in
bug 1547328.
Driveby add comment documenting GetSampleEncryptionEntry.
Driveby adjust include order to match Mozilla/Google style.
Differential Revision: https://phabricator.services.mozilla.com/D30111
--HG--
extra : moz-landing-system : lando
This also removes the following prefs, because they're unused:
- media.autoplay.allow-muted pref
- media.autoplay.blackList-override-default
Differential Revision: https://phabricator.services.mozilla.com/D36396
--HG--
extra : rebase_source : 0570540496302b3efedadf4d5115ee5422d5c279
A lot of the overhead in MoofParser::RebuildFragmentedIndex(BoxContext&) is
copying data into the BoxReader's storage. In the MSE case the underlying data
being read is actually buffered in memory, so we may be able to just read the
in-memory data directly, avoiding the copy.
Note that the data may not be stored in a single contiguous ResourceItem in the
SourceBufferResource's ResourQueue. If so, it's not as straightforward to hand
out a uint8_t* to the underlying data. So we just copy it in that case for
simplicity. In most cases, the data in a single MP4 box would be appended in a
single append, and so is likely to be in a single ResourceItem.
Differential Revision: https://phabricator.services.mozilla.com/D33877
--HG--
extra : moz-landing-system : lando
If you look down the call stacks inside MoofParser::RebuildFragmentedIndex(),
you'll see we spend most of the time inside BoxReader, e.g.:
https://perfht.ml/31b42u3
Each Box we create is read by a BoxReader, which makes a copy of the Box's
contents. We spend a bunch of time allocating the memory to store the copy on
the heap.
Since the allocations here are short lived (we delete the boxes at scope exit
of RebuildFragmentedIndex()), we can speed up the allocation by using a bump
allocator here. This means we hit malloc() and free() a lot less.
Note that Twitch only hits this allocator with allocations which are less
than 100 bytes. On YouTube we see a different pattern; there are still a lot
of allocations less than 100 bytes, but also a significant number which are
between 1500-1700 bytes.
The bump allocator lives on the BoxContext, so will be deallocated on scope
exit of MoofParser::RebuildFragmentedIndex().
Differential Revision: https://phabricator.services.mozilla.com/D33876
--HG--
extra : moz-landing-system : lando
Twitch is appending samples with small gaps between them, and the gaps span
Moof boundaries.
This means we end up have buffered ranges with lots of small gaps, and keeping
these up to date is expensive.
So remember the CTS of the last sample in each track, and pass that in when
parsing new Moofs. We can then remove the gaps, which makes our buffered
ranges simpler.
Note that Twitch often has a 1 microsecond gap between audio samples, but up to
1 millisecond gap between video samples; so we use a different threshold for
audio and video.
Differential Revision: https://phabricator.services.mozilla.com/D33875
--HG--
extra : moz-landing-system : lando
Profiling Twitch we see a significant amount of time at the bottom of our call
stacks in the MoofParser creating Box objects. Most of the time spend therein
is allocating and copying into an nsTArray. This is us copying the Box's header
into Box::mHeader. The reallocation and append are expensive. It turns out we
only use the box header in one place; when reporting the PSSH boxes up to EME.
So in the non-EME case, the storage of the Box header is a complete waste of
time and space.
So we can change the Box class to instead read the header from its storage when
we need to retrieve the complete PSSH boxes.
Differential Revision: https://phabricator.services.mozilla.com/D33870
--HG--
extra : moz-landing-system : lando
A lot of the overhead in MoofParser::RebuildFragmentedIndex(BoxContext&) is
copying data into the BoxReader's storage. In the MSE case the underlying data
being read is actually buffered in memory, so we may be able to just read the
in-memory data directly, avoiding the copy.
Note that the data may not be stored in a single contiguous ResourceItem in the
SourceBufferResource's ResourQueue. If so, it's not as straightforward to hand
out a uint8_t* to the underlying data. So we just copy it in that case for
simplicity. In most cases, the data in a single MP4 box would be appended in a
single append, and so is likely to be in a single ResourceItem.
Differential Revision: https://phabricator.services.mozilla.com/D33877
--HG--
extra : moz-landing-system : lando
If you look down the call stacks inside MoofParser::RebuildFragmentedIndex(),
you'll see we spend most of the time inside BoxReader, e.g.:
https://perfht.ml/31b42u3
Each Box we create is read by a BoxReader, which makes a copy of the Box's
contents. We spend a bunch of time allocating the memory to store the copy on
the heap.
Since the allocations here are short lived (we delete the boxes at scope exit
of RebuildFragmentedIndex()), we can speed up the allocation by using a bump
allocator here. This means we hit malloc() and free() a lot less.
Note that Twitch only hits this allocator with allocations which are less
than 100 bytes. On YouTube we see a different pattern; there are still a lot
of allocations less than 100 bytes, but also a significant number which are
between 1500-1700 bytes.
The bump allocator lives on the BoxContext, so will be deallocated on scope
exit of MoofParser::RebuildFragmentedIndex().
Differential Revision: https://phabricator.services.mozilla.com/D33876
--HG--
extra : moz-landing-system : lando
Twitch is appending samples with small gaps between them, and the gaps span
Moof boundaries.
This means we end up have buffered ranges with lots of small gaps, and keeping
these up to date is expensive.
So remember the CTS of the last sample in each track, and pass that in when
parsing new Moofs. We can then remove the gaps, which makes our buffered
ranges simpler.
Note that Twitch often has a 1 microsecond gap between audio samples, but up to
1 millisecond gap between video samples; so we use a different threshold for
audio and video.
Differential Revision: https://phabricator.services.mozilla.com/D33875
--HG--
extra : moz-landing-system : lando
Profiling Twitch we see a significant amount of time at the bottom of our call
stacks in the MoofParser creating Box objects. Most of the time spend therein
is allocating and copying into an nsTArray. This is us copying the Box's header
into Box::mHeader. The reallocation and append are expensive. It turns out we
only use the box header in one place; when reporting the PSSH boxes up to EME.
So in the non-EME case, the storage of the Box header is a complete waste of
time and space.
So we can change the Box class to instead read the header from its storage when
we need to retrieve the complete PSSH boxes.
Differential Revision: https://phabricator.services.mozilla.com/D33870
--HG--
extra : moz-landing-system : lando
Unencrypted samples can carry init data and thus we should set the init data
type regardless of if the sample itself is encrypted (otherwise the init data
info is incomplete for unencrypted samples).
Depends on D32751
Differential Revision: https://phabricator.services.mozilla.com/D32752
--HG--
extra : moz-landing-system : lando
This assert could be triggered with malformed files, so replace it with an early
return.
Differential Revision: https://phabricator.services.mozilla.com/D30027
--HG--
extra : moz-landing-system : lando
When encountering media using the common encryption format (mp4s containing pssh
boxes), the encrypted event we fire should specify the cenc init data type.
Fixes a bug where for cbcs the 'cbcs' init data type was used -- which is a
bogus value.
Differential Revision: https://phabricator.services.mozilla.com/D30095
--HG--
extra : moz-landing-system : lando
It's possible for a malformed mp4 to contain invalid sample description index in
fragments, that do not reference any sample description entries found in the
header. E.g. the header may contain 2 sample description entries (which should
be indexed with indices 1 and 2), but for a fragment to contain an index to 4.
Instead of asserting in this case we should gracefully fail.
Bug 1547328 plans to add logging for this case, so we have a means to still
detect failures here from bad files.
Depends on D29733
Differential Revision: https://phabricator.services.mozilla.com/D29734
--HG--
extra : moz-landing-system : lando
The assert removed here can be triggered by malformed mp4s that lack certain
crypto info (the sinf box), but that still indicate samples are encrypted via
sample description entries.
I plan to follow up the removal here by adding logs so we have some way to
detect this case. This will be done as part of bug 1547328.
Depends on D29692
Differential Revision: https://phabricator.services.mozilla.com/D29693
--HG--
extra : moz-landing-system : lando
I suspect these may have been cargo-culted from elsewhere.
We only need -Wno-sign-compare, which we can add regardless of compiler because clang-cl understands it natively in addition to the -wdNNNN form.
Differential Revision: https://phabricator.services.mozilla.com/D20616
--HG--
extra : moz-landing-system : lando
Consequently, this removes:
- MOZ_LIBPRIO, which is now always enabled.
- non_msvc_compiler, which is now always true.
- The cl.py wrapper, since it's not used anymore.
- CL_INCLUDES_PREFIX, which was only used for the cl.py wrapper.
- NONASCII, which was only there to ensure CL_INCLUDES_PREFIX still
worked in non-ASCII cases.
This however keeps a large part of detecting and configuring for MSVC,
because we still do need it for at least headers, libraries, and midl.
Depends on D19614
Differential Revision: https://phabricator.services.mozilla.com/D19615
--HG--
extra : moz-landing-system : lando
This patch adds telemetry to help us determine if mp4s with certain structures
can occur in the wild. Specifically:
- How often do mp4s have multiple entries in the sample description box?
- Do we ever see mp4s with multiple codecs in the sample description box?
- Do we ever see mp4s with multiple sets of crypto info in the sample
description box?
This information is collected each time we parse mp4 metadata.
Remove some diagnostic asserts now that we're gracefully gathering info.
Reshuffle of DecoderData function order to align with the order in the header.
Differential Revision: https://phabricator.services.mozilla.com/D14426
--HG--
extra : moz-landing-system : lando
The MoofParser didn't have logging on its normal code paths. This patch
implements such logging, covering entry and exit to many functions. Such verbose
logs are helpful for diagnosing failures and the specific file structure leading
to such failures. This should help improve our debugging experience if we can
get logs from end users which is particularly useful for cases where they can't
provide us with media due to geoblocking/proprietary media issues.
Differential Revision: https://phabricator.services.mozilla.com/D19021
--HG--
extra : moz-landing-system : lando
- Use different log levels in MoofParser. Generally Error is used for out of
memory, while warnings are used for bad mp4s.
- Add some new logs for parsing sample description entries.
- Add logs for when we can't find a Moof or fail to read a Moov.
- Have logs be more specific for the Box they're in where possible rather than
just log 'Moof'. E.g., when parsing the stsd box, we now log against that box,
not 'Moof'.
- Log instead of asserting if we're given multiple encrypted sample description
entries. This is valid per the spec, and can happen (though we don't expect
it), so we shouldn't assert.
Differential Revision: https://phabricator.services.mozilla.com/D19020
--HG--
extra : moz-landing-system : lando
Similar to what we do for H264 and for vp9 in webm, we parse the VP9 bytestream and check if a frame is a keyframe there instead.
Differential Revision: https://phabricator.services.mozilla.com/D18790
--HG--
extra : moz-landing-system : lando
Using a variant more clearly indicates how MoofParser works: you cannot request
a specific track id and to parse all tracks. Callers must now explicitly select
one or the other.
Differential Revision: https://phabricator.services.mozilla.com/D18135
--HG--
extra : moz-landing-system : lando
Using track_id 0 is forbidden by the mp4 spec, however, some sites still serve
media using this track_id. We've been using the 0 track ID to trigger special
handling in the MoofParser where we will parse multiple tracks, and this led us
to be tolerant of tracks using this reserved id (though we likely had some bugs
due to this).
Since sites are using this track_id, and as other browsers (and Firefox until I
broke this) tolerate such media, we should too. In order to do so correctly, we
should no longer us track_id=0 as a special case in the MoofParser, and instead
have an explicit flag, which is what this patch does.
Differential Revision: https://phabricator.services.mozilla.com/D16428
--HG--
extra : moz-landing-system : lando
Using track_id 0 is forbidden by the mp4 spec, however, some sites still serve
media using this track_id. We've been using the 0 track ID to trigger special
handling in the MoofParser where we will parse multiple tracks, and this led us
to be tolerant of tracks using this reserved id (though we likely had some bugs
due to this).
Since sites are using this track_id, and as other browsers (and Firefox until I
broke this) tolerate such media, we should too. In order to do so correctly, we
should no longer us track_id=0 as a special case in the MoofParser, and instead
have an explicit flag, which is what this patch does.
Differential Revision: https://phabricator.services.mozilla.com/D16428
--HG--
extra : moz-landing-system : lando
Handle mp4parse-rust providing cbcs data in the track metadata. Explicitly check
the crypto scheme we get in the metadata and error if we encounter something
outside of cenc and cbcs -- catch unexpected data early.
Differential Revision: https://phabricator.services.mozilla.com/D15878
--HG--
extra : moz-landing-system : lando
Rework our mp4 sample iterator to handle cbcs crypto data.
To support this we populate the following new data for samples:
- Crypto pattern information, this is split into a count of encrypted blocks
and a count of clear blocks.
- A constant IV.
This information is available at a track level and a sample group level. The
sample group level supersedes track level information if both a present.
Prior to this patch, some crypto information was written to samples in
the SampleIterator in Index.cpp, and some in the MP4Demuxer (based on if the
SampleIterator had not populated the data). This patch moves all these
operations into the SampleIterator -- the idea being that the sample iterator
should be the component responsible for setting up sample meta data.
Differential Revision: https://phabricator.services.mozilla.com/D15877
--HG--
extra : moz-landing-system : lando
Andreas Pehrson
Gurzau Raul
Bryce Seager van Dyk
Tom Ritter
Sylvestre Ledru
Dan Minor
arthur.iakab
Nicholas Nethercote
alwu
Christian Holler
Chris Pearce
Oana Pop Rus
Jean-Yves Avenard
Ryan Hunt
David Major
Mike Hommey
Ehsan Akhgari
Cosmin Sabou