We need to inform the MediaDataDecoder that a new sample configuration has been detected.
To do so, and to ensure that samples are easily matched to a given configuration, we first drain the decoder prior feeding a new frame.
MozReview-Commit-ID: Hye251CF21i
--HG--
extra : rebase_source : d6e5be74a6678cef0ff4c798fac91d95911be70a
into TrackInfoSharedPtr to better indicate what this class is about.
Adding cast operator to allow transparent conversion from TrackInfoSharedPtr to const TrackInfo*
MozReview-Commit-ID: 6RwXl5CG0fG
--HG--
extra : rebase_source : b5a7a0f06793c609e2eab60aacc4f76d96d6ec32
While reducing total peak memory usage, it also allows to start processing frames as they come without having to wait on the decoder to have processed all frames at once. This is particularly an issue on windows where the decoder will block the entire decode task queue while a YUV->RGB conversion is occurring.
MozReview-Commit-ID: QQDpUoj0PS
--HG--
extra : rebase_source : bc2fc4e29a2b943a71f8c8450aad1cd162375bd6
We now use a generic state rather than various booleans.
We handle partial drains better too.
MozReview-Commit-ID: AJSTcJivfEm
--HG--
extra : rebase_source : 26386cea20b97f1f76df829241b091ba837b301a
It may not be possible for a decoder to return all drained samples in one go. This is particularly a problem on android where there's a limited amount of memory buffers available.
So we no longer expect the operation to complete in one go, we call Drain until there are no more samples returned.
MozReview-Commit-ID: GbfntKNBDaW
--HG--
extra : rebase_source : 6b7dc19e1acf348d41d5f96cf575fa2e7dffdd3b
This prevents the demuxer attempting to use the parent resources once the parent has been shutdown.
MozReview-Commit-ID: LQqPxBkAfAS
--HG--
extra : rebase_source : 1a9c2e870aecb5b0ecb68d5f600c48334ac4d312
This is a big change, and unfortunately impossible to break down with independently functional commits.
There are four main changes being applied here:
* Code cleanup, including making all MediaDataDecoder related code mozilla coding style compliant
* Make MediaDataDecoder use MozPromise
* Making Flush and Shutdown processes fully asynchronous
* Fixing few data races encountered across the code, in particular in the Android PDM
MozReview-Commit-ID: DpiZucGofJT
--HG--
extra : rebase_source : 80bd6c6f9726d536b6f306c40d9af6df27333be9
This is not compatible with promise based decoders. We'll re-implement it in a different fashion later on.
MozReview-Commit-ID: HYBNKVxEUmB
--HG--
extra : rebase_source : fe9f26e47b77a64099a422be844d20f4108663d0
ScheduleUpdate() will be called when we have new decoding requests.
MozReview-Commit-ID: FkOc4zQmfSo
--HG--
extra : rebase_source : c36133d7fd1a64daabfd6d5fba445ad42b1a47cf
extra : intermediate-source : a610694bfb7a7d58c7633913d4ac1ce33f46bcd2
extra : source : 36f4a436f1750087f848e4cb68104015ef0284f6
We runs all demuxing operations on a dedicated task queue.
MediaDataDemuxer's members using a synchronous API are handled via thread-safe copy that are updated along the operations.
The buffered range calculation is now handled separately and the entire operation is made asynchronous.
MozReview-Commit-ID: Gd4DCC8Ix6n
--HG--
extra : rebase_source : b90bad0a386c2a1e30acc00e3db9db6b6762aa3b
We runs all demuxing operations on a dedicated task queue.
MediaDataDemuxer's members using a synchronous API are handled via thread-safe copy that are updated along the operations.
The buffered range calculation is now handled separately and the entire operation is made asynchronous.
MozReview-Commit-ID: Gd4DCC8Ix6n
--HG--
extra : rebase_source : 6a18ce2552bf4cbf88e9b8db1c9a87e70623fd15
We runs all demuxing operations on a dedicated task queue.
MediaDataDemuxer's members using a synchronous API are handled via thread-safe copy that are updated along the operations.
The buffered range calculation is now handled separately and the entire operation is made asynchronous.
MozReview-Commit-ID: Gd4DCC8Ix6n
--HG--
extra : rebase_source : fb157f58e38c2f3199ed96095634bf8e4ab6065b
Because it will change the ref-count which might cause destructor to run before exiting the constructor.
MozReview-Commit-ID: MMIea970Dv
--HG--
extra : rebase_source : faf898a3107706bc0ea750e738d8c3a7a3824264
Bug 1313635 ensures buffer ranges are updated before resolving metadata.
So we can revert the changes in bug 1251460.
MozReview-Commit-ID: Ek0sgPpv92s
--HG--
extra : rebase_source : 79b457167d5b987280083364afe31b31860b48fe
extra : intermediate-source : 9205ca61bb2255e33744af7ec4e13059f275960c
extra : source : 14d155274dc4e0d2daa8a90b1bbb15f975d7ac24
Thanks to this, MediaDecoderReader::Seek and overrides can take SeekTarget by
const&.
MozReview-Commit-ID: C5D26jKrW3p
--HG--
extra : rebase_source : 6f6278654ff3c51f6331cbf683c22622223a8cd2
So m{Audio,Video}.mDecoder will never reference a half-baked decoder.
MozReview-Commit-ID: FwuzaRR9ugf
--HG--
extra : rebase_source : 32b13b5e3041072bb5e48e6a633281bd8df76a41
extra : intermediate-source : 3d5bc0ca507717f09e1429a5ea05f98e18b48dfc
extra : source : 039ed0007b2e63c381c1ad0f638eed5b4c53266f
While it's unlikely to have been a problem as the decoder would have been idled at this stage. During the time the TrackInfo was reset and the decoder actually be shut down, the reference to the object would have been invalid causing a potential UAF.
MozReview-Commit-ID: 7pGJtYRy2Yr
--HG--
extra : rebase_source : d32b8ff393f55685ef182b00f286aea1d1d75464
The issue is particularly problematic with the Apple audio decoder. The Apple decoder relies on the sampling rate to configure the CoreAudio transform.
The actual output rate may be different (such as with HE-AAC). Should the decoder ever need to be reset again, future initialization would have failed as the initial rate was now incorrect.
MozReview-Commit-ID: 7kTiaUYuOgf
--HG--
extra : rebase_source : 00119a13205c610542a2f690ce6c3b9caeb1b69c
Some encrypted MP4 streams don't contain PSSH boxes in their MOOV boxes, but
they still have tracks with valid TENC boxes which we still parse. So we need
to consider media encrypted if any of its tracks have crypto meta data, rather
than only if the media has crypto init data.
The WebM demuxer's crypto init data is all the tracks' crypto meta data, so
WebM isn't broken by this change.
MozReview-Commit-ID: 1qOi4uTtCE3
--HG--
extra : amend_source : 0f266976a3d65fb8dc720cd40d70bd6da1712420
Warnings were provided earlier on and got removed. So we instead make it consistent across all decoders.
MozReview-Commit-ID: 53obtEClq98
--HG--
extra : rebase_source : f2e22e0d8ff33e8198b39f5a955b8f233a97daa6
We provide even further details for the GMP decoder. Other decoders to follow.
MozReview-Commit-ID: 7NxJPec8xWv
--HG--
extra : rebase_source : f44120983070e5c107ecd5cafc762da90aab44bf
The MediaFormatReader will no longer attempt to decode several frames in advance and ahead of the MDSM actually requesting it. The speed advantages were dubious at best, and as most MediaDataDecoders abused the use of InputExhausted callbacks we had to place artificial throttle that would often cause side effects.
As such, it is now expected that the MediaDataDecoder will now always call InputExhausted once Input has been called. InputExhausted indicates that the current decoding session has completed and the MediaDataDecoder is waiting for another input.
MozReview-Commit-ID: 9KUpNP9jozV
--HG--
extra : rebase_source : d261a5eb98de54d5bd29acb738c4205c56abca6b
The MediaFormatReader will no longer attempt to decode several frames in advance and ahead of the MDSM actually requesting it. The speed advantages were dubious at best, and as most MediaDataDecoders abused the use of InputExhausted callbacks we had to place artificial throttle that would often cause side effects.
As such, it is now expected that the MediaDataDecoder will now always either return a decoded sample or call InputExhausted. Never both.
MediaDataDecoder will continue to work as-is, even if they call InpuxExhausted too many times as the MediaFormatReader will only feed a single sample at a time.
MozReview-Commit-ID: 9KUpNP9jozV
--HG--
extra : rebase_source : ebb919fd3f1ce1adf5d08ed3f4292839b84c8321
This member is only assigned but never used now.
MozReview-Commit-ID: IbuAbRaXzc4
--HG--
extra : rebase_source : 166bd3bc7478f725b64b5b2a0e8e258980674d06
This reverts commit 5a949eb358e27
Another more complete solution will follow.
MozReview-Commit-ID: K3lTdrBxW7W
--HG--
extra : rebase_source : d0f9443193b0816ae85972a4a368599b872fd437
ReleaseResources() is called when MDSM enters dormant or during shutdown.
When it is called in response to dormant request, we don't want to clear
current frames so we are able to enter dormant state more aggressively
even when the media element is visible to the user.
When it is called from MediaDecoderReader::Shutdown(), it doesn't really
call ClearCurrentFrame() because MediaFormatReader::Shutdown clears the
|mVideoFrameContainer| pointer. So it doesn't make a difference to remove
the call.
MozReview-Commit-ID: IakGHbSMWTv
--HG--
extra : rebase_source : 7a25de39e04f5c7728bf65fcd447cc67b7a85411
extra : source : 44ff0ffaf63ad51a7a382cf0ee1c16e64ade63b9
Bubble information from SamplesWaitingForKey to an HTMLMediaElement so that we
can emit a waitingForKey event. If we are unable to decode more samples due to
needing a key the event will be signalled. See
http://w3c.github.io/encrypted-media/#dom-evt-waitingforkey for more information
on this event.
The code in place before this patch handles updating readyState when we are
waiting for a key, this patch adds the event which should be emitted in such a
case. The spec defines certain preconditions should be the case before running
the algo to emit this event. For example, the element should potentially be
playing, and it should have at least HAVE_FUTURE_DATA ready state. This is not
strictly true for when the new code is run, due how existing code handles ready
state. We are honoring the spirit of the spec, though the letter of the spec is
lightly gone against in the handling of the preconditions.
MozReview-Commit-ID: LKlDd4wkRSE
--HG--
extra : rebase_source : 2c61fc41636e430afa23240ad5d26c886124d87f
With MSE, the actual duration is always exact as it is amended when data is added. We do not want to fire ended when we attempt to seek to unbuffered data once endOfStream has been called. Instead we will fire the waiting event.
MozReview-Commit-ID: Cl2uBLk2qRQ
--HG--
extra : rebase_source : 6763c6f5a6e15264e276e486fab4d39491ea7f1b
This patch makes most Run() declarations in subclasses of nsIRunnable have the
same form: |NS_IMETHOD Run() override|.
As a result of these changes, I had to add |override| to a couple of other
functions to satisfy clang's -Winconsistent-missing-override warning.
--HG--
extra : rebase_source : 815d0018b0b13329bb5698c410f500dddcc3ee12
So, we have 4 boolean variables and here is the truth table.
Case 1, 2, 3, 4, 7, 8, 10 and 12 are not possible to happen.
Then, the remaining cases could be clustered into three categories:
(1) Case 5, 9 and 13: no sample is demuxed at all, return 0.
(2) Case 6, 11, 14 and 15: either audio or video is able to be demuxed, return the known value.
(3) Case 15: both audio and video are demuxed, return the minimum of the values.
For simplifying the logic, I will initialize the audioStartTime and videoStartTime
to be INFINITY if we don't have the first-demuxed sample, otherwise, initialize
them to be the real first-demuxed sample's time.
Then, the final calculation will be:
(1) Case 5, 9 and 13: the minimum of two INFINITY values is still INFINITY, return 0.
(2) Case 6, 11, 14 and 15: return the minimum of one real first-demuxed-time and the INFINITY.
(3) Case 15: return the minimum of two real first-demuxed-time values.
Case HasAudio HasVideo HasAudioSample HasVideoSample ExpectedResult
---------------------------------------------------------------------------------------------------
1 F F F F not possible
2 F F F T not possible
3 F F T F not possible
4 F F T T not possible
---------------------------------------------------------------------------------------------------
5 F T F F return 0
6 F T F T return video sample
7 F T T F not possible
8 F T T T not possible
---------------------------------------------------------------------------------------------------
9 T F F F return 0
10 T F F T not possible
11 T F T F return audio sample
12 T F T T not possible
---------------------------------------------------------------------------------------------------
13 T T F F return 0
14 T T F T return videoSample
15 T T T F return audioSample
16 T T T T return min(auidoSample, videoSample)
MozReview-Commit-ID: ANsYDth7slJ
--HG--
extra : transplant_source : %DAj%1A%EC%19%82%7B%B3%05%FE%21%04a%16%9A%9F%18x1%95
Replace the pointer of VideoFrameContainer with the pointer of MediaStreamVideoSink.
MozReview-Commit-ID: 5bqEMpemwuR
--HG--
extra : transplant_source : %008z%D8W%EE%87%8E%E9/%2CT%26%EBvo%AE%099%A6
Replace the pointer of VideoFrameContainer with the pointer of MediaStreamVideoSink.
MozReview-Commit-ID: 5bqEMpemwuR
--HG--
extra : amend_source : 7eb1e87fdcbc61f2f9831fa3a6d803cc50306604
Replace the pointer of VideoFrameContainer with the pointer of MediaStreamVideoSink.
MozReview-Commit-ID: 5bqEMpemwuR
--HG--
extra : transplant_source : %9D%86%93%A6%DF%D5%9Ep%20%DF%FD%C1%E2%BA%A3Gq%1A%7E%A3
FrameStatisticsData can now store inter-keyframe information, which is
provided by the MediaFormatReader (based on live decoding).
MozReview-Commit-ID: HhBy6pgT6ZX
--HG--
extra : rebase_source : 6a072623e8a5b0f23b81307e8ea4b19a3e21b252
Decoders now use FrameStatisticsData to gather data for their frame-related
notifications. This will ease introducing new members later on.
MozReview-Commit-ID: DWdOSPX3JM
--HG--
extra : rebase_source : a3e05f34353a397d1c82b3f4d935c0864f90556e
Handle encrypted WebM streams for the clearkey case. Add checking for the
widevine case, though these should currently fail, as not all of the plumping
is in place for widevine.
MozReview-Commit-ID: 5d9fvc5IkZF
--HG--
extra : rebase_source : 9baad2afd7778c350c404c72dcd81426092aa908
Extract all the parameters passed to CreateAudioDecoder/CreateVideoDecoder and
place them into a structure that is passed down to the creation of the actual
decoder, where the relevant parameters can be extracted.
This makes it easier to add more arguments to the Create*Decoder calls in future.
MozReview-Commit-ID: 9LZlcfRVz6A
--HG--
extra : rebase_source : ce3f0c7d0784b96267728697ff5d535ccb8ee7a6
We should only attempt to demux new samples once all pending decoded frames have been returned. Otherwise, the next demuxing attempt once a resolution change has been detected will reset the state and drop all decoded frames.
MozReview-Commit-ID: 2coKQA8lZ8c
--HG--
extra : rebase_source : f8df34bd9ea1b05c251a34d043da19c7d3d5e587
MediaFormatReader::Reset is already resetting the context.
MozReview-Commit-ID: AWIh9aTFA6S
--HG--
extra : rebase_source : 4ad28da651f9d0608d91503b901045a8947f3073
We would only abort early if new data had been received. Which may not always be the case.
MozReview-Commit-ID: HvAUq5CTc7F
--HG--
extra : rebase_source : a24b6c8bf2a31b0c9e69fc748ebb9b80b5c0d286
This currently can't happen as no MediaDataDecoder ever return a null sample.
MozReview-Commit-ID: BucIadubght
--HG--
extra : rebase_source : 4c6f3dc1d5581bdb9d58ba3b7cc1659c57f40d8e
If we're in waiting for data mode, the decoder must have already been drained and we want the waiting promise to be resolved upon the next run of UpdateReceivedNewData.
MozReview-Commit-ID: Hf8pFFyQmjJ
--HG--
extra : rebase_source : c51ed65c7d7c4a9ee877e6f9420f6534480e7cb5
As the decoder was flushed and reset prior the skip to next keyframe started, and future error would be unrecoverable. So only reset the decoder once the skip completes and succeeded. Otherwise we default back to normal decoding.
MozReview-Commit-ID: GEj1i0EsaYO
--HG--
extra : rebase_source : c1d2beafa6e35f571b941e8f173ec10b726ec1a0
currentTime may be past the start of the internal seek time, but before its end, which would have triggered a false positive: we do not want to jump to the next key frame here.
MozReview-Commit-ID: H5O9XXnk84T
--HG--
extra : rebase_source : 0bc96472acfe03ff0c87a6fc3712bd99b96064b6
With the new video-only seek type, we need to ignore the current audio decoding state whenever video-only seeking is in progress.
MozReview-Commit-ID: 5eynmG2UyyA
--HG--
extra : rebase_source : 7e9bdd7c44cb4a80129bfc468da87288e520c984
MediaFormatReader::mOriginalSeekTarget now has different use and the name was no longer reflective of the actual usage.
MozReview-Commit-ID: Fu0fmgUlbgr
--HG--
extra : rebase_source : 7c106fc4832def7c6a4b91e1950ed5b3e1109129
PDMFactory will automatically load and initialize the required frameworks upon first use.
Also fix constness of some methods.
MozReview-Commit-ID: HFbvTMRFAey
--HG--
extra : rebase_source : ce3e7ff6cefc35b32674c9a81c3488fe21b48626
When resuming from suspended video, don't reset the audio queues or
audio demuxing state in MFR.
Extracted the code from ResetDemuxers() and inlined it at calling sites
as recommended by jya.
MozReview-Commit-ID: 3wpH075RrKW
Similar logic with audio tracks, but those are ignored instead.
MozReview-Commit-ID: 6p9ej66aFgD
--HG--
extra : rebase_source : 01406ed958ce9e71d886955cf728d625acc779e1
Similar logic with audio tracks, but those are ignored instead.
MozReview-Commit-ID: 6p9ej66aFgD
--HG--
extra : rebase_source : 0d696fed85bc53feabfe248e9ae16c5380e70261
Implemented by short-circuiting calls to RequestVideoData in MDSM so no
frames are decoded. Resuming playback when video moves to foreground by
using the SeekTask/SeekJob/Seek in MDSM with result of GetMediaTime().
Special consideration is made to only seek the video part of Seek() to
remove an audible glitch in the audio playback when the video becomes
visible again.
MozReview-Commit-ID: 7YFDTanslXu
data promise is only resolved once the decoder has been drained. It was possible for a promise to never be resolved if skipping to the next key frame failed.
MozReview-Commit-ID: GimbQTImH9e
Access to some members is not thread-safe; but the typical use of those informations is when a mochitest has timed out, and by that time the MFR will have been idled for over 5 minutes.
MozReview-Commit-ID: 21BxrSZXVVJ
If the Skip To Next Keyframe logic was activated, the next frame demuxed would have been passed the internal seek target, causing it to be unnecessarily dropped.
MozReview-Commit-ID: DExwMPLXlZu
Some decoders (WMF) keep some internal counters on how many frames have been output and use this to calculate the time of the decoded audio frame. As such, when internally seeking, the next frame decoded would have always been past the seek target.
MozReview-Commit-ID: puzs6ecqbD
While a seek is pending, the demuxer has been reset meaning that ShouldSkip() would almost always incorrectly return true and would corrupt the seeking state.
MozReview-Commit-ID: 6R912Fu6Nul
--HG--
extra : rebase_source : a20a5d0ed1c0d21c04014790bdf5a0c07edb82a4
Pass declared-but-yet-undefined DecoderDoctorDiagnostics pointer to various
routines that contribute to deciding if a media format can be played, and
those that create decoders.
Points where a DecoderDoctorDiagnostics can be injected are currently marked
with "/* DecoderDoctorDiagnostics* */ nullptr", and some will be used in
following patches.
MozReview-Commit-ID: 7u37bvY4CpW
Gerald Squelart
JW Wang
Tom Tromey
Jean-Yves Avenard
Andi-Bogdan Postelnicu
Bevis Tseng
Kaku Kuo
Sebastian Hengst
Carsten "Tomcat" Book
James Cheng
Eugen Sawin
Matt Woodrow
Phil Ringnalda
Ryan VanderMeulen
Chris Pearce
Iris Hsiao
Bryce Van Dyk
Nicholas Nethercote
Kaku Kuo
ctai
Dan Glastonbury
Alfredo Yang
Nathan Froyd
Kyle Huey
Mats Palmgren