When seeking, we go to the first IDR prior the seek target. However, if the MP4 sample table is invalid the frame returned may not be an IDR.
It is an error for that frame to not be a keyframe, so we do want to have way to detect them through logging. Change assertion to a soft assertion for that purpose.
MozReview-Commit-ID: EMgwRo1mYMp
--HG--
extra : rebase_source : 363a7e148db166b2130a7862a49f03d419d3fe22
This reverts part of bug 1300296. In the worse case we'll get a decoding error. But we're only trading a bad behaviour for another.
MozReview-Commit-ID: H0gF3FqZsU6
--HG--
extra : rebase_source : 54d38d24b40014351f943bff6860f5de9fc2f418
Should any error occurred while checking the tracks, we could end up with a null pointer stored in the list of track demuxers.
MozReview-Commit-ID: 13FllESrpbg
--HG--
extra : rebase_source : 335de6320c12f758858e462ed22fc9f9ffe8e305
1. using media::TimeUnit to save some typing.
2. replace TimeUnit() with TimeUnit::Zero().
3. replace TimeUnit::FromXXX(0) with TimeUnit::Zero().
4. replace TimeUnit::FromMicroseconds(std::numeric_limits<int64_t>::max()) with TimeUnit::FromInfinity().
5. replace some uses of int64_t with TimeUnit.
6. replace t > TimeUnit() with t.IsPositive().
MozReview-Commit-ID: 6hC94PXx86i
--HG--
extra : rebase_source : 1ea3b409e6ec12915f3e1a00359d6ff4152c8917
extra : intermediate-source : e31a12ad0e7a4840119036f261ed17eaaff85734
extra : source : ae07ee48000c4a52da0e4fd502b4d690ec51ce1f
In addition to the returned MediaResult, a special number-of-tracks value
(not just 0) indicates an unrecoverable error.
For Rust-vs-Stagefright comparison purposes, an error is considered the same
as 0 (because Stagefright never returns errors, but Rust may, so complaining
about that would be too noisy, and useless to us.)
MozReview-Commit-ID: IwadWSOIWr4
--HG--
extra : rebase_source : 29f53ee6a02a0431adb0b615a122a4e7b480108c
The returned MediaResult is used as error or warning in MP4Demuxer::Init().
MozReview-Commit-ID: Bnv4xG8bCJ4
--HG--
extra : rebase_source : c1952ed61396834b0cd7da58c9b64481a5c46ab1
If MP4Demuxer::Init detects some recoverable error (e.g., invalid tracks when
others may still be usable), it will eventually Resolve the promise with the
first warning.
Later on, errors/warnings from the MP4Metadata parser will also be handled, to
provide even better diagnostics.
MozReview-Commit-ID: E9Rly9dhXW3
--HG--
extra : rebase_source : cae214d0c80297bd61156dc1a305a186da0974fe
MP4Metadata::Metadata() contains the same code at the beginning, so calling
HasCompleteMetadata() is unnecessary, so we should just remove it completely.
(Except to get a better error message, but this will be reinstated in an
upcoming bug.)
MozReview-Commit-ID: 2C3GI5fE0Ja
--HG--
extra : rebase_source : 2bdb6199222dcb0ca4012121a3fd4694b440be3e
MP4Demuxer::Init() used to just create a minimal MP4Metadata structure, and
report success/failure from that alone. But other later-called functions
(e.g.: GetNumberTracks, GetTrackDemuxer, etc.) could still fail with no useful
error reporting, when MP4Metadata tried to gather more of the needed
information.
Also, MP4Demuxer needed to keep this MP4Metadata around forever, even though
it could contain an arbitrary amount of extra data that is not needed.
With this patch, MP4Demuxer::Init() fetches all the data that could ever be
needed, and then discards the MP4Metadata.
This ensures that no late-initialization errors could creep in, and also helps
reporting errors early and with better diagnostic information.
This bug focuses on Init(), a later bug will give MP4Metadata the ability
to report its own even-more-detailed errors.
MozReview-Commit-ID: 1NjzOeKa1JI
--HG--
extra : rebase_source : 02781395aa538cf2be984b695a7bc7e2b9b039b7
Thanks to the previous patch, MediaDataDemuxer::Seek and
SkipToNextRandomAccessPoint (and all overrides in derived demuxers) can now
take their TimeUnit parameter by const&.
MozReview-Commit-ID: 6CqfjAXZ7Yk
--HG--
extra : rebase_source : c3453e4432d9e0281cf5eba55217b0c1d6312f5b
There are too many cases where the MP4 is improperly muxed and frames are incorrectly reported as keyframe.
Instead we now look inside the H264 stream and check for IDR frames.
We also ensure that the first frame returned after a seek is always a true keyframe.
For plain MP4, seeking in those broken files will lead to broken A/V sync. The only way to fix this would be to check for the frame type when reading the samples table. However, this would require to read the entire stream which isn't viable.
MozReview-Commit-ID: Cpv5y7HVD0N
--HG--
extra : rebase_source : 2f92032fe39ed6ad6c2b82438f405040b5e7d30c
H264 decoders always use those anyway, so may as well use them in the demuxer if SPS NAL is available.
This guarantees that we have correct dimensions when reading the MP4 metadata, and will have the side benefit that when loadedmetadata is fired, the dimensions provided at the time will be final; not having to wait to decode the first frame.
MozReview-Commit-ID: 3j70Xqw8jJY
--HG--
extra : rebase_source : 6bc0f1fa1c2db35bcaa683cc1a68042d122e2892
We now use an index of samples made of block of samples delimited by keyframes. The search is performed using a binary search. This allows to quickly find which blocks are found within the media cache.
On a 8 core mac pro, this leads to a 67% improvement on CPU time spent playing a long MP4 video (from 112s CPU to 45s CPU)
The optimisation is only possible if all mp4 data chunks are continuous (which they almost always are)
The bulk of this commit was generated with a script, executed at the top
level of a typical source code checkout. The only non-machine-generated
part was modifying MFBT's moz.build to reflect the new naming.
CLOSED TREE makes big refactorings like this a piece of cake.
# The main substitution.
find . -name '*.cpp' -o -name '*.cc' -o -name '*.h' -o -name '*.mm' -o -name '*.idl'| \
xargs perl -p -i -e '
s/nsRefPtr\.h/RefPtr\.h/g; # handle includes
s/nsRefPtr ?</RefPtr</g; # handle declarations and variables
'
# Handle a special friend declaration in gfx/layers/AtomicRefCountedWithFinalize.h.
perl -p -i -e 's/::nsRefPtr;/::RefPtr;/' gfx/layers/AtomicRefCountedWithFinalize.h
# Handle nsRefPtr.h itself, a couple places that define constructors
# from nsRefPtr, and code generators specially. We do this here, rather
# than indiscriminantly s/nsRefPtr/RefPtr/, because that would rename
# things like nsRefPtrHashtable.
perl -p -i -e 's/nsRefPtr/RefPtr/g' \
mfbt/nsRefPtr.h \
xpcom/glue/nsCOMPtr.h \
xpcom/base/OwningNonNull.h \
ipc/ipdl/ipdl/lower.py \
ipc/ipdl/ipdl/builtin.py \
dom/bindings/Codegen.py \
python/lldbutils/lldbutils/utils.py
# In our indiscriminate substitution above, we renamed
# nsRefPtrGetterAddRefs, the class behind getter_AddRefs. Fix that up.
find . -name '*.cpp' -o -name '*.h' -o -name '*.idl' | \
xargs perl -p -i -e 's/nsRefPtrGetterAddRefs/RefPtrGetterAddRefs/g'
if [ -d .git ]; then
git mv mfbt/nsRefPtr.h mfbt/RefPtr.h
else
hg mv mfbt/nsRefPtr.h mfbt/RefPtr.h
fi
--HG--
rename : mfbt/nsRefPtr.h => mfbt/RefPtr.h
In MP4TrackDemuxer constructor, getting the track info and indices could potentially fail, triggering crashing assertions.
The fallible work is now done before calling the constructor, and if it fails a nullptr is returned, which is correctly handled in MediaFormatReader.
Also reordered some MP4TrackDemuxer members for more natural construction.
A MediaDataDemuxer is now not to resolve the init promise until it has all the metadata.
Except MediaSource, all demuxers would be doing blocking read to scan for the metadata, and having partial metadata would be an error.
For MediaSource, we pass the NotifyDataArrived message which will cause the MediaSourceDemuxer to re-attempt to search for the metadata.
When used within MediaSource's TrackBuffersManager, a demuxer will never be created until we have received a complete init segment (this task is performed by the various ContainerParsers)
A typical non-fragmented mp4 would have the ftyp atom located at the beginning of the mp4 and the moov at the end. We would to calculate the location of the metadata by spanning the byte range of the two atoms.
As such, we would typically allocate an amount of memory equivalent to the size of the mp4.
Instead we now reconstruct the metadata to only have the ftyp and moov atoms contiguously.
A typical non-fragmented mp4 would have the ftyp atom located at the beginning of the mp4 and the moov at the end. We would to calculate the location of the metadata by spanning the byte range of the two atoms.
As such, we would typically allocate an amount of memory equivalent to the size of the mp4.
Instead we now reconstruct the metadata to only have the ftyp and moov atoms contiguously.
Jean-Yves Avenard
JW Wang
Gerald Squelart
Alfredo.Yang
Tom Tromey
Sylvestre Ledru
Jay Harris
Jonathan Watt
sajitk
Wes Kocher
Nathan Froyd
Alfredo Yang
Carsten "Tomcat" Book