We provide even further details for the GMP decoder. Other decoders to follow.
MozReview-Commit-ID: 7NxJPec8xWv
--HG--
extra : rebase_source : f44120983070e5c107ecd5cafc762da90aab44bf
The MediaFormatReader will no longer attempt to decode several frames in advance and ahead of the MDSM actually requesting it. The speed advantages were dubious at best, and as most MediaDataDecoders abused the use of InputExhausted callbacks we had to place artificial throttle that would often cause side effects.
As such, it is now expected that the MediaDataDecoder will now always call InputExhausted once Input has been called. InputExhausted indicates that the current decoding session has completed and the MediaDataDecoder is waiting for another input.
MozReview-Commit-ID: 9KUpNP9jozV
--HG--
extra : rebase_source : d261a5eb98de54d5bd29acb738c4205c56abca6b
The MediaFormatReader will no longer attempt to decode several frames in advance and ahead of the MDSM actually requesting it. The speed advantages were dubious at best, and as most MediaDataDecoders abused the use of InputExhausted callbacks we had to place artificial throttle that would often cause side effects.
As such, it is now expected that the MediaDataDecoder will now always either return a decoded sample or call InputExhausted. Never both.
MediaDataDecoder will continue to work as-is, even if they call InpuxExhausted too many times as the MediaFormatReader will only feed a single sample at a time.
MozReview-Commit-ID: 9KUpNP9jozV
--HG--
extra : rebase_source : ebb919fd3f1ce1adf5d08ed3f4292839b84c8321
This member is only assigned but never used now.
MozReview-Commit-ID: IbuAbRaXzc4
--HG--
extra : rebase_source : 166bd3bc7478f725b64b5b2a0e8e258980674d06
This reverts commit 5a949eb358e27
Another more complete solution will follow.
MozReview-Commit-ID: K3lTdrBxW7W
--HG--
extra : rebase_source : d0f9443193b0816ae85972a4a368599b872fd437
ReleaseResources() is called when MDSM enters dormant or during shutdown.
When it is called in response to dormant request, we don't want to clear
current frames so we are able to enter dormant state more aggressively
even when the media element is visible to the user.
When it is called from MediaDecoderReader::Shutdown(), it doesn't really
call ClearCurrentFrame() because MediaFormatReader::Shutdown clears the
|mVideoFrameContainer| pointer. So it doesn't make a difference to remove
the call.
MozReview-Commit-ID: IakGHbSMWTv
--HG--
extra : rebase_source : 7a25de39e04f5c7728bf65fcd447cc67b7a85411
extra : source : 44ff0ffaf63ad51a7a382cf0ee1c16e64ade63b9
Bubble information from SamplesWaitingForKey to an HTMLMediaElement so that we
can emit a waitingForKey event. If we are unable to decode more samples due to
needing a key the event will be signalled. See
http://w3c.github.io/encrypted-media/#dom-evt-waitingforkey for more information
on this event.
The code in place before this patch handles updating readyState when we are
waiting for a key, this patch adds the event which should be emitted in such a
case. The spec defines certain preconditions should be the case before running
the algo to emit this event. For example, the element should potentially be
playing, and it should have at least HAVE_FUTURE_DATA ready state. This is not
strictly true for when the new code is run, due how existing code handles ready
state. We are honoring the spirit of the spec, though the letter of the spec is
lightly gone against in the handling of the preconditions.
MozReview-Commit-ID: LKlDd4wkRSE
--HG--
extra : rebase_source : 2c61fc41636e430afa23240ad5d26c886124d87f
With MSE, the actual duration is always exact as it is amended when data is added. We do not want to fire ended when we attempt to seek to unbuffered data once endOfStream has been called. Instead we will fire the waiting event.
MozReview-Commit-ID: Cl2uBLk2qRQ
--HG--
extra : rebase_source : 6763c6f5a6e15264e276e486fab4d39491ea7f1b
This patch makes most Run() declarations in subclasses of nsIRunnable have the
same form: |NS_IMETHOD Run() override|.
As a result of these changes, I had to add |override| to a couple of other
functions to satisfy clang's -Winconsistent-missing-override warning.
--HG--
extra : rebase_source : 815d0018b0b13329bb5698c410f500dddcc3ee12
So, we have 4 boolean variables and here is the truth table.
Case 1, 2, 3, 4, 7, 8, 10 and 12 are not possible to happen.
Then, the remaining cases could be clustered into three categories:
(1) Case 5, 9 and 13: no sample is demuxed at all, return 0.
(2) Case 6, 11, 14 and 15: either audio or video is able to be demuxed, return the known value.
(3) Case 15: both audio and video are demuxed, return the minimum of the values.
For simplifying the logic, I will initialize the audioStartTime and videoStartTime
to be INFINITY if we don't have the first-demuxed sample, otherwise, initialize
them to be the real first-demuxed sample's time.
Then, the final calculation will be:
(1) Case 5, 9 and 13: the minimum of two INFINITY values is still INFINITY, return 0.
(2) Case 6, 11, 14 and 15: return the minimum of one real first-demuxed-time and the INFINITY.
(3) Case 15: return the minimum of two real first-demuxed-time values.
Case HasAudio HasVideo HasAudioSample HasVideoSample ExpectedResult
---------------------------------------------------------------------------------------------------
1 F F F F not possible
2 F F F T not possible
3 F F T F not possible
4 F F T T not possible
---------------------------------------------------------------------------------------------------
5 F T F F return 0
6 F T F T return video sample
7 F T T F not possible
8 F T T T not possible
---------------------------------------------------------------------------------------------------
9 T F F F return 0
10 T F F T not possible
11 T F T F return audio sample
12 T F T T not possible
---------------------------------------------------------------------------------------------------
13 T T F F return 0
14 T T F T return videoSample
15 T T T F return audioSample
16 T T T T return min(auidoSample, videoSample)
MozReview-Commit-ID: ANsYDth7slJ
--HG--
extra : transplant_source : %DAj%1A%EC%19%82%7B%B3%05%FE%21%04a%16%9A%9F%18x1%95
Replace the pointer of VideoFrameContainer with the pointer of MediaStreamVideoSink.
MozReview-Commit-ID: 5bqEMpemwuR
--HG--
extra : transplant_source : %008z%D8W%EE%87%8E%E9/%2CT%26%EBvo%AE%099%A6
Replace the pointer of VideoFrameContainer with the pointer of MediaStreamVideoSink.
MozReview-Commit-ID: 5bqEMpemwuR
--HG--
extra : amend_source : 7eb1e87fdcbc61f2f9831fa3a6d803cc50306604
Replace the pointer of VideoFrameContainer with the pointer of MediaStreamVideoSink.
MozReview-Commit-ID: 5bqEMpemwuR
--HG--
extra : transplant_source : %9D%86%93%A6%DF%D5%9Ep%20%DF%FD%C1%E2%BA%A3Gq%1A%7E%A3
FrameStatisticsData can now store inter-keyframe information, which is
provided by the MediaFormatReader (based on live decoding).
MozReview-Commit-ID: HhBy6pgT6ZX
--HG--
extra : rebase_source : 6a072623e8a5b0f23b81307e8ea4b19a3e21b252
Decoders now use FrameStatisticsData to gather data for their frame-related
notifications. This will ease introducing new members later on.
MozReview-Commit-ID: DWdOSPX3JM
--HG--
extra : rebase_source : a3e05f34353a397d1c82b3f4d935c0864f90556e
Handle encrypted WebM streams for the clearkey case. Add checking for the
widevine case, though these should currently fail, as not all of the plumping
is in place for widevine.
MozReview-Commit-ID: 5d9fvc5IkZF
--HG--
extra : rebase_source : 9baad2afd7778c350c404c72dcd81426092aa908
Extract all the parameters passed to CreateAudioDecoder/CreateVideoDecoder and
place them into a structure that is passed down to the creation of the actual
decoder, where the relevant parameters can be extracted.
This makes it easier to add more arguments to the Create*Decoder calls in future.
MozReview-Commit-ID: 9LZlcfRVz6A
--HG--
extra : rebase_source : ce3f0c7d0784b96267728697ff5d535ccb8ee7a6
We should only attempt to demux new samples once all pending decoded frames have been returned. Otherwise, the next demuxing attempt once a resolution change has been detected will reset the state and drop all decoded frames.
MozReview-Commit-ID: 2coKQA8lZ8c
--HG--
extra : rebase_source : f8df34bd9ea1b05c251a34d043da19c7d3d5e587
MediaFormatReader::Reset is already resetting the context.
MozReview-Commit-ID: AWIh9aTFA6S
--HG--
extra : rebase_source : 4ad28da651f9d0608d91503b901045a8947f3073
We would only abort early if new data had been received. Which may not always be the case.
MozReview-Commit-ID: HvAUq5CTc7F
--HG--
extra : rebase_source : a24b6c8bf2a31b0c9e69fc748ebb9b80b5c0d286
This currently can't happen as no MediaDataDecoder ever return a null sample.
MozReview-Commit-ID: BucIadubght
--HG--
extra : rebase_source : 4c6f3dc1d5581bdb9d58ba3b7cc1659c57f40d8e
If we're in waiting for data mode, the decoder must have already been drained and we want the waiting promise to be resolved upon the next run of UpdateReceivedNewData.
MozReview-Commit-ID: Hf8pFFyQmjJ
--HG--
extra : rebase_source : c51ed65c7d7c4a9ee877e6f9420f6534480e7cb5
As the decoder was flushed and reset prior the skip to next keyframe started, and future error would be unrecoverable. So only reset the decoder once the skip completes and succeeded. Otherwise we default back to normal decoding.
MozReview-Commit-ID: GEj1i0EsaYO
--HG--
extra : rebase_source : c1d2beafa6e35f571b941e8f173ec10b726ec1a0
currentTime may be past the start of the internal seek time, but before its end, which would have triggered a false positive: we do not want to jump to the next key frame here.
MozReview-Commit-ID: H5O9XXnk84T
--HG--
extra : rebase_source : 0bc96472acfe03ff0c87a6fc3712bd99b96064b6
With the new video-only seek type, we need to ignore the current audio decoding state whenever video-only seeking is in progress.
MozReview-Commit-ID: 5eynmG2UyyA
--HG--
extra : rebase_source : 7e9bdd7c44cb4a80129bfc468da87288e520c984
MediaFormatReader::mOriginalSeekTarget now has different use and the name was no longer reflective of the actual usage.
MozReview-Commit-ID: Fu0fmgUlbgr
--HG--
extra : rebase_source : 7c106fc4832def7c6a4b91e1950ed5b3e1109129
PDMFactory will automatically load and initialize the required frameworks upon first use.
Also fix constness of some methods.
MozReview-Commit-ID: HFbvTMRFAey
--HG--
extra : rebase_source : ce3e7ff6cefc35b32674c9a81c3488fe21b48626
When resuming from suspended video, don't reset the audio queues or
audio demuxing state in MFR.
Extracted the code from ResetDemuxers() and inlined it at calling sites
as recommended by jya.
MozReview-Commit-ID: 3wpH075RrKW
Similar logic with audio tracks, but those are ignored instead.
MozReview-Commit-ID: 6p9ej66aFgD
--HG--
extra : rebase_source : 01406ed958ce9e71d886955cf728d625acc779e1
Similar logic with audio tracks, but those are ignored instead.
MozReview-Commit-ID: 6p9ej66aFgD
--HG--
extra : rebase_source : 0d696fed85bc53feabfe248e9ae16c5380e70261
Implemented by short-circuiting calls to RequestVideoData in MDSM so no
frames are decoded. Resuming playback when video moves to foreground by
using the SeekTask/SeekJob/Seek in MDSM with result of GetMediaTime().
Special consideration is made to only seek the video part of Seek() to
remove an audible glitch in the audio playback when the video becomes
visible again.
MozReview-Commit-ID: 7YFDTanslXu
data promise is only resolved once the decoder has been drained. It was possible for a promise to never be resolved if skipping to the next key frame failed.
MozReview-Commit-ID: GimbQTImH9e
Access to some members is not thread-safe; but the typical use of those informations is when a mochitest has timed out, and by that time the MFR will have been idled for over 5 minutes.
MozReview-Commit-ID: 21BxrSZXVVJ
If the Skip To Next Keyframe logic was activated, the next frame demuxed would have been passed the internal seek target, causing it to be unnecessarily dropped.
MozReview-Commit-ID: DExwMPLXlZu
Some decoders (WMF) keep some internal counters on how many frames have been output and use this to calculate the time of the decoded audio frame. As such, when internally seeking, the next frame decoded would have always been past the seek target.
MozReview-Commit-ID: puzs6ecqbD
While a seek is pending, the demuxer has been reset meaning that ShouldSkip() would almost always incorrectly return true and would corrupt the seeking state.
MozReview-Commit-ID: 6R912Fu6Nul
--HG--
extra : rebase_source : a20a5d0ed1c0d21c04014790bdf5a0c07edb82a4
Pass declared-but-yet-undefined DecoderDoctorDiagnostics pointer to various
routines that contribute to deciding if a media format can be played, and
those that create decoders.
Points where a DecoderDoctorDiagnostics can be injected are currently marked
with "/* DecoderDoctorDiagnostics* */ nullptr", and some will be used in
following patches.
MozReview-Commit-ID: 7u37bvY4CpW
MediaDecoder previously had 3 states within GetSeekable(), media is either
seekable, seekable but not supported by transport, or not seekable. Due to
changes to make cueless webms playable, a 4th option is needed: a file that is
not fully seekable, but may support seeking from the transport, such as these
webms, should only be seekable in the buffered range.
MozReview-Commit-ID: ISeFkngtrGU
MediaDecoderStateMachine's EnqueueFIrstFrameLoadedEvent would previously call
into MediaDecoderReader to update MDR's buffered ranges and enqueue the frame
loaded event. This commit aims to instead have the buffered update take place,
and only afterwards enqueue the event. This should remove the possibility that
the event will be fired and handled before the update of the buffered ranges has
taken place.
MozReview-Commit-ID: GP8w2nF4xmj
--HG--
extra : transplant_source : %A0m%13%95%E3Gs%ACMd%1F%F4%25%B9qE%28J%21R
We would only start to drop frames once we had a request for data. On platforms such as windows WMF, following a drain we would typically hold over 33 decoded frames which would have been held resulting in unnecessarily high memory usage.
This also results in a simplified logic for the following steps.
Decoders are created and initialized automatically when needed.
Also fix issues where we could have a pending initialization promise pending when going into dormant mode.
The bulk of this commit was generated with a script, executed at the top
level of a typical source code checkout. The only non-machine-generated
part was modifying MFBT's moz.build to reflect the new naming.
CLOSED TREE makes big refactorings like this a piece of cake.
# The main substitution.
find . -name '*.cpp' -o -name '*.cc' -o -name '*.h' -o -name '*.mm' -o -name '*.idl'| \
xargs perl -p -i -e '
s/nsRefPtr\.h/RefPtr\.h/g; # handle includes
s/nsRefPtr ?</RefPtr</g; # handle declarations and variables
'
# Handle a special friend declaration in gfx/layers/AtomicRefCountedWithFinalize.h.
perl -p -i -e 's/::nsRefPtr;/::RefPtr;/' gfx/layers/AtomicRefCountedWithFinalize.h
# Handle nsRefPtr.h itself, a couple places that define constructors
# from nsRefPtr, and code generators specially. We do this here, rather
# than indiscriminantly s/nsRefPtr/RefPtr/, because that would rename
# things like nsRefPtrHashtable.
perl -p -i -e 's/nsRefPtr/RefPtr/g' \
mfbt/nsRefPtr.h \
xpcom/glue/nsCOMPtr.h \
xpcom/base/OwningNonNull.h \
ipc/ipdl/ipdl/lower.py \
ipc/ipdl/ipdl/builtin.py \
dom/bindings/Codegen.py \
python/lldbutils/lldbutils/utils.py
# In our indiscriminate substitution above, we renamed
# nsRefPtrGetterAddRefs, the class behind getter_AddRefs. Fix that up.
find . -name '*.cpp' -o -name '*.h' -o -name '*.idl' | \
xargs perl -p -i -e 's/nsRefPtrGetterAddRefs/RefPtrGetterAddRefs/g'
if [ -d .git ]; then
git mv mfbt/nsRefPtr.h mfbt/RefPtr.h
else
hg mv mfbt/nsRefPtr.h mfbt/RefPtr.h
fi
--HG--
rename : mfbt/nsRefPtr.h => mfbt/RefPtr.h
This commit was generated using the following script, executed at the
top level of a typical source code checkout.
# Don't modify select files in mfbt/ because it's not worth trying to
# tease out the dependencies currently.
#
# Don't modify anything in media/gmp-clearkey/0.1/ because those files
# use their own RefPtr, defined in their own RefCounted.h.
find . -name '*.cpp' -o -name '*.h' -o -name '*.mm' -o -name '*.idl'| \
grep -v 'mfbt/RefPtr.h' | \
grep -v 'mfbt/nsRefPtr.h' | \
grep -v 'mfbt/RefCounted.h' | \
grep -v 'media/gmp-clearkey/0.1/' | \
xargs perl -p -i -e '
s/mozilla::RefPtr/nsRefPtr/g; # handle declarations in headers
s/\bRefPtr</nsRefPtr</g; # handle local variables in functions
s#mozilla/RefPtr.h#mozilla/nsRefPtr.h#; # handle #includes
s#mfbt/RefPtr.h#mfbt/nsRefPtr.h#; # handle strange #includes
'
# |using mozilla::RefPtr;| is OK; |using nsRefPtr;| is invalid syntax.
find . -name '*.cpp' -o -name '*.mm' | xargs sed -i -e '/using nsRefPtr/d'
# RefPtr.h used |byRef| for dealing with COM-style outparams.
# nsRefPtr.h uses |getter_AddRefs|.
# Fixup that mismatch.
find . -name '*.cpp' -o -name '*.h'| \
xargs perl -p -i -e 's/byRef/getter_AddRefs/g'
The LayersBackend can be defined at construction time, however if a parent MediaDecoder exists, the value will be overwritten by the MediaDecoderOwner value.
--HG--
extra : rebase_source : fefad83560d5bfa2aee9f665fe138397eb390019
Jean-Yves Avenard
Matt Woodrow
Phil Ringnalda
Kaku Kuo
JW Wang
Iris Hsiao
Bryce Van Dyk
Nicholas Nethercote
Kaku Kuo
ctai
Sebastian Hengst
Carsten "Tomcat" Book
Eugen Sawin
Gerald Squelart
Chris Pearce
Dan Glastonbury
Alfredo Yang
Ryan VanderMeulen
Nathan Froyd
Kyle Huey
Mats Palmgren
Louis Christie
sajitk
Wes Kocher
James Cheng