Adding tests to ensure that when cues with overlapping times, the one with earlier end timestamp should disappear when the media time reaches its end time. In this test, we have two cues with overlapping time, when the video starts, both cues should be displayed. When the time passes 1 seconds, the first cue should disappear and the second cues should be still displayed.
Differential Revision: https://phabricator.services.mozilla.com/D31172
--HG--
extra : moz-landing-system : lando
If the amount of cues which are going to be displayed is different from the one we displayed last time, we have to compute cues' display state again because cue's position might be affected by other cues.
Differential Revision: https://phabricator.services.mozilla.com/D31170
--HG--
extra : moz-landing-system : lando
We can actually let `processCue()` to handle rendering cues or cleaning displayed cues, no need to use another way to clear the cue.
The advantages is to make the code cleaner and easier to read, now we just need to know JS side would handle all rendering stuffs for us. We don't need to have different behavior when there is no showing cue.
The way we clear displayed cues are intuitive, we would remove all child nodes under the overlay, which are used to display cues.
Differential Revision: https://phabricator.services.mozilla.com/D31171
--HG--
extra : moz-landing-system : lando
If we end up taking the gap branch of the if in the patch, and the media file
in the future from the point we branched at contains no gaps or overlaps, we'll
still end up taking the gap branch until the end. This is because it only
appends up to the start of the current frame, but not the frame itself. I.e.,
when the next frame gets processed, there's still a gap in appended data.
Depends on D32107
Differential Revision: https://phabricator.services.mozilla.com/D32108
--HG--
extra : moz-landing-system : lando
Refactor those tests' structure in order to make them more readable, and add the comment to show what the test purpose is for each test.
Differential Revision: https://phabricator.services.mozilla.com/D31914
--HG--
extra : moz-landing-system : lando
This patch do two things in order to trigger loading for track element and wait for correct event to check track's and cues' status after loading finished.
(1) listen track element's load event
There are some tests listening video's loadedmetadata, but it's wrong. The loading process of media element and track element are completely non-related.
If you would like to check track element's status, you should wait for track element's load event.
(2) enable track explictly
If the text track which has default attribute is added to the media element before the media element starts running automatic track selection [1], then it would be enable by the media element.
Otherwise, you have to enable track explicitly by changing its track mode.
[1] https://html.spec.whatwg.org/multipage/media.html#sourcing-out-of-band-text-tracks:text-track-7
Differential Revision: https://phabricator.services.mozilla.com/D31913
--HG--
extra : moz-landing-system : lando
Create test elements in HTML beforehand, which can remove unnecessary JS code and make test cleaner.
Differential Revision: https://phabricator.services.mozilla.com/D31911
--HG--
extra : moz-landing-system : lando
And with some tidying some comments and removing stray #include "gfxPrefs.h"
Differential Revision: https://phabricator.services.mozilla.com/D31468
--HG--
extra : moz-landing-system : lando
gfxPrefs Live preferences are almost identical to StaticPrefs.
We leave aside for now those that set a custom change callback as this feature isn't yet supported in StaticPrefs.
Differential Revision: https://phabricator.services.mozilla.com/D31256
--HG--
extra : moz-landing-system : lando
We should resolve video playing promise when we finish playing the last frame. Otherwise, it would cause playback to be ended earlier than its original duration.
Differential Revision: https://phabricator.services.mozilla.com/D32051
--HG--
extra : moz-landing-system : lando
Previous patch in Bug 1552712 actually didn't set the correct upper bound; it
failed to set the upper bound in cases like:
intersect({[0,10]}, {[0,1], [2,3], [4,5]})
That's clearly the maximum of the lengths of the array, not the minimum.
Differential Revision: https://phabricator.services.mozilla.com/D31946
--HG--
extra : moz-landing-system : lando
Profiling shows we can spend a lot of time in IntervalSet::Intersection().
Much of this time is spent appending elements to a temporary buffer as we
loop over the two IntervalSets' arrays comparing them. As we append, the
storage of the temporary array is reallocated as the storage must grow to
hold more elements. These reallocations are expensive, as we must copy the
old elements to the new storage.
So before starting the loop, reserve enough capacity to store the upper
bound; the minimum of the two IntervalSets' lengths. This may waste memory,
but will be faster, as expensive reallocations are avoided.
Differential Revision: https://phabricator.services.mozilla.com/D31744
--HG--
extra : moz-landing-system : lando
This assert could be triggered with malformed files, so replace it with an early
return.
Differential Revision: https://phabricator.services.mozilla.com/D30027
--HG--
extra : moz-landing-system : lando
Mp4s using the 'cenc' encryption scheme should contain auxiliary information
containing the initialization vector to be used to decode that sample. However,
it's possible for malformed mp4s to contain other relevant crypto information,
but to omit this aux info. This adds such a file to our crashtests.
Differential Revision: https://phabricator.services.mozilla.com/D30025
--HG--
extra : moz-landing-system : lando
`IsStarted` may not be updated at the time `Revive()` method is executed since the update of the flag happens on the async operation. This will be the case if `Revive` is executed right after `Start`. When that happens the revive method asserts that the stream is started.
Differential Revision: https://phabricator.services.mozilla.com/D30889
--HG--
extra : moz-landing-system : lando
On MochCubeb add a fake audio thread and the corresponding methods for stream_{init,start,stop,destroy}.
Differential Revision: https://phabricator.services.mozilla.com/D30888
--HG--
extra : moz-landing-system : lando
Extract the existing MockCubeb logic from TestAudioDeviceEnumerator to a separate reusable header.
Differential Revision: https://phabricator.services.mozilla.com/D30887
--HG--
extra : moz-landing-system : lando
Now adding or removing cue would trigger processing cues only when media element's `show-poster` flag is false, so we have to modify the test to reset the flag in order to test the correct behavior.
Differential Revision: https://phabricator.services.mozilla.com/D31511
--HG--
extra : moz-landing-system : lando
When `RemoteMediaDataDecoder::Shutdown` is finished it is not necessary to hold a reference of the `self` any more. Keep the `mChild` alive, which is the only one needed to destroy the IPDL. In addition to that, deleting the IPDL and destroying the child will be happening at the task queue similar to what was happening before Bug 1545416.
Differential Revision: https://phabricator.services.mozilla.com/D31261
--HG--
extra : moz-landing-system : lando
Instead of deleteing a RefPtr directly copy it in a local variable in order to ensure that the pointer will be alive till the end of the method. In addition to that, on RemoteMediaDataDecpder::Shutdown promise use a reference of the child object instead of the whole `self` object since this is the only one needed. Finally, one style change.
Differential Revision: https://phabricator.services.mozilla.com/D30637
--HG--
extra : moz-landing-system : lando
When encountering media using the common encryption format (mp4s containing pssh
boxes), the encrypted event we fire should specify the cenc init data type.
Fixes a bug where for cbcs the 'cbcs' init data type was used -- which is a
bogus value.
Differential Revision: https://phabricator.services.mozilla.com/D30095
--HG--
extra : moz-landing-system : lando
encrypted-track-with-bad-sample-description-index.mp4 appears to be causing
issues when running under older Android versions. Skip this when running crash
tests while we investigate and add a comment to indicate as such. Bug 1550912
has been raised to follow this up and fix as needed.
Differential Revision: https://phabricator.services.mozilla.com/D30836
--HG--
extra : moz-landing-system : lando
We should add some debug logs, which should be easily enable and disable, in vtt.jsm in order to increase the efficiency of debugging.
Differential Revision: https://phabricator.services.mozilla.com/D30798
--HG--
extra : moz-landing-system : lando
Add test to to ensure that the looping video (without audio track) which has been suspended can continute to playback correctly after we resume video decoding.
Differential Revision: https://phabricator.services.mozilla.com/D30419
--HG--
extra : moz-landing-system : lando
The normal looping process is that, goes to `completed` state first, notify playback ended, and finally media element would call seek to the start position in order to start looping again.
However, if we're in the seamless looping mode, we can stay in `loopingDecoding` state and repeating the looping without going to other states. Otherwise, we should go to `completed` state if decoding has ended.
Differential Revision: https://phabricator.services.mozilla.com/D30307
--HG--
extra : moz-landing-system : lando
When adjusting box's position, we would choose a axis first and move the box along this axis to see whether we can place this box without overlapping with other boxes and fully inside the rendering area.
If the box has been over the boundary along the moving direction, we should move box back to the original position and change the moving direction to see whether we can find another best place for the box.
Although the adjustment can run without timeout now, it still doesn't match the result of the reference, so change the state to `fail`.
Differential Revision: https://phabricator.services.mozilla.com/D29765
--HG--
extra : moz-landing-system : lando
According to the spec [1] 7.2.10.2, we should use the first line box's height or width as positioning unit to adjust box's position.
We will also use this value to adjust box when `snap-to-line` is false.
There, we implement a new chrome-only API to acquire this information, which would return the first line box's size in the `block` frame.
[1] https://www.w3.org/TR/webvtt1/#ref-for-webvtt-cue-snap-to-lines-flag-12
Differential Revision: https://phabricator.services.mozilla.com/D29745
--HG--
extra : moz-landing-system : lando
When adjusting cue's position, the goal is to place cue boxes fully inside the rendering area, and place in the right place.
In order to check whether the box is correctly inside the rendering area, we have to know the correct size of the cue box.
Therefore, we implement the new chrome-only APIs to acquire this information (box's height and width), and this returned value won't be affected by CSS transformation, which is good for positioning.
Differential Revision: https://phabricator.services.mozilla.com/D27178
--HG--
extra : moz-landing-system : lando
Create a new IPDL message `ShutdownComplete`, direction from parent (RDD) to child (content), to inform the child when the decoder shutdown has been completed. The remote decoder child uses that message to resolve the shutdown promise.
Differential Revision: https://phabricator.services.mozilla.com/D28340
--HG--
extra : moz-landing-system : lando
According to the spec [1], the cue's end time might be negative and be smaller than its start time.
In this case, when we reach the cue's start on the media time line, we should treat it as a `missing cue` (which won't be actually displayed, but will receive events) when we run the `TimeMarchesOn`.
Therefore, we have to add this kind of cue into `otherCue` and let `TimeMarchesOn` handles it properly, to dispatch `enter` and `exit` event for it.
[1] https://html.spec.whatwg.org/multipage/media.html#text-track-cue-end-time
Differential Revision: https://phabricator.services.mozilla.com/D30242
--HG--
extra : moz-landing-system : lando
When under pressure, the MediaCache evicts data before the last read on a
stream. We typically have two demuxers reading from different offsets in a
stream. So if the MediaCache is under pressure, it may end up evicting data
between the two demuxers.
The MediaDecoderStateMachine currently goes into buffering state if there's
insufficient data available beginning at the start of its queue of decoded
samples. However since the MediaCache evicts data behind the streams read
cursor, the data after the beginning of the sample queue may have already been
evicted by the media cache. This will cause the MediaDecoderStateMachine to
enter a buffering state, and if its sample queues are full, there will be no
demuxers reading to cause the MediaCache to download the data between the two
demuxers, and we'll get stuck in buffering state indefinitely.
So change the MediaDecoderStateMachine to instead check whether there's
insufficient data available at the end of the decoded sample queues. This means
we won't get stuck in buffering state. Note the MediaCache may still evict data
which the other demuxer needed, causing us to re-request it, but at least we
won't get stuck in buffering state.
Differential Revision: https://phabricator.services.mozilla.com/D30310
--HG--
extra : moz-landing-system : lando
`rv` is not used anymore out of Nightly, because diagnostic asserts are
disabled.
Differential Revision: https://phabricator.services.mozilla.com/D30059
--HG--
extra : moz-landing-system : lando
Normally when downloading media data we throttle the download only if we're
ahead of the read cursor more than the "readahead limit", and if we estimate
that the connection is fast enough that we'll be able to download at a rate
fast enough to playback in real time if we resume it later.
On mobile we additionally override this so that we always throttle the download
once we're ahead of the read cursor by the readahead limit. This is to save
data. I think we can relax this to only do this override if we're on a
cellular connection; if we're on WiFi we can assume data is cheap.
Differential Revision: https://phabricator.services.mozilla.com/D26234
--HG--
extra : moz-landing-system : lando
In GeckoView the nsINetworkLinkService doesn't work in the content process, as
we don't seem to have an AndroidBridge there, so just maintain the network
connection type on the ContentChild.
(I had considered keeping this on the NeckoChild, but the creation of that is
initiated from the content process side, and there's not an easy and clean way
to have the parent process send us the connection type after construction of
the NeckoParent, other than have the NeckoChild request it either
synchronously, or doing it async and hoping it's not asked for the value before
the response comes in.)
Differential Revision: https://phabricator.services.mozilla.com/D26232
--HG--
extra : moz-landing-system : lando
`rv` is not used anymore out of Nightly, because diagnostic asserts are
disabled.
Differential Revision: https://phabricator.services.mozilla.com/D30059
--HG--
extra : moz-landing-system : lando
This patch structurizes the media debug information via webidl dictionaries
that are returned by HTMLMediaElement::GetMozRequestDebugInfo() and
MediaSource::GetMozDebugReaderData().
Differential Revision: https://phabricator.services.mozilla.com/D27893
--HG--
extra : moz-landing-system : lando
With changes from bug 1548555, some of the gtests previously disabled on Android
can be re-enabled.
Differential Revision: https://phabricator.services.mozilla.com/D30084
--HG--
extra : moz-landing-system : lando
Consumers that just care about this boolean state should use this instead of
getting the JSObject* directly.
Differential Revision: https://phabricator.services.mozilla.com/D29705
--HG--
extra : moz-landing-system : lando
It's possible for a malformed mp4 to contain invalid sample description index in
fragments, that do not reference any sample description entries found in the
header. E.g. the header may contain 2 sample description entries (which should
be indexed with indices 1 and 2), but for a fragment to contain an index to 4.
Instead of asserting in this case we should gracefully fail.
Bug 1547328 plans to add logging for this case, so we have a means to still
detect failures here from bad files.
Depends on D29733
Differential Revision: https://phabricator.services.mozilla.com/D29734
--HG--
extra : moz-landing-system : lando
Add an mp4 with a bad sample description index to crashtests. When samples in a
fragment are encountered, they should reference a sample description entry found
in the mp4 header. However, it's possible that the index contained in the
fragment may refer to an entry that doesn't exist in the header, as in this
file.
Differential Revision: https://phabricator.services.mozilla.com/D29733
--HG--
extra : moz-landing-system : lando
This can cause 1proc tests to fail because no decoder ends up supporting
a format. The particular case was enabling Vorbis decoding on RDD.
Differential Revision: https://phabricator.services.mozilla.com/D29766
--HG--
extra : moz-landing-system : lando
The assert removed here can be triggered by malformed mp4s that lack certain
crypto info (the sinf box), but that still indicate samples are encrypted via
sample description entries.
I plan to follow up the removal here by adding logs so we have some way to
detect this case. This will be done as part of bug 1547328.
Depends on D29692
Differential Revision: https://phabricator.services.mozilla.com/D29693
--HG--
extra : moz-landing-system : lando
Add an mp4 with a mangled tenc box to the crashtests. This file has been fuzzed
and appears to have had the tenc box for its second track changed to a '.enc'
box, which means it cannot be parsed. The tenc box is contained in the sinf box,
so this will bust the sinf parse. We should tolerate such bad files and not
assert and/or crash.
Driveby add a comment to another line in crashtests list to make it clear the
bug the file originally relates to.
Differential Revision: https://phabricator.services.mozilla.com/D29692
--HG--
extra : moz-landing-system : lando
Normally when downloading media data we throttle the download only if we're
ahead of the read cursor more than the "readahead limit", and if we estimate
that the connection is fast enough that we'll be able to download at a rate
fast enough to playback in real time if we resume it later.
On mobile we additionally override this so that we always throttle the download
once we're ahead of the read cursor by the readahead limit. This is to save
data. I think we can relax this to only do this override if we're on a
cellular connection; if we're on WiFi we can assume data is cheap.
Differential Revision: https://phabricator.services.mozilla.com/D26234
--HG--
extra : moz-landing-system : lando
In GeckoView the nsINetworkLinkService doesn't work in the content process, as
we don't seem to have an AndroidBridge there, so just maintain the network
connection type on the ContentChild.
(I had considered keeping this on the NeckoChild, but the creation of that is
initiated from the content process side, and there's not an easy and clean way
to have the parent process send us the connection type after construction of
the NeckoParent, other than have the NeckoChild request it either
synchronously, or doing it async and hoping it's not asked for the value before
the response comes in.)
Differential Revision: https://phabricator.services.mozilla.com/D26232
--HG--
extra : moz-landing-system : lando
Detect content process that already has connection to the RDD process
and don't attempt to relaunch or needlessly call
RDDProcessManager::CreateContentBridge. Calling CreateContentBridge
unnecessarily causes churn by recreating RemoteDecoderManagerParents.
Differential Revision: https://phabricator.services.mozilla.com/D29013
--HG--
extra : moz-landing-system : lando
As the purpose of adjusting time is to make the start of the playback to align to zero, so the result should not be negative.
However, we can't always ensure the demuxed start time is correct, the decoded first frame sample time might be different with the time demuxer provides.
Even if we try to update the start time when we get the first decoded sample, there is still a problem because a decoder might return samples with different order, which means the first frame the decoder returns is probably not a real first frame.
Therefore, using warning to help us knows how many this situaion would be.
Differential Revision: https://phabricator.services.mozilla.com/D28432
--HG--
extra : moz-landing-system : lando
Considering that the audio sample's time is always increased, the decoded sample's time decoder returns should always be equal or larger than its demuxed time.
When the decoded sample's time is smaller than the time of first demuxed sample, that time would probably cause a problem so we should throw an error for that.
Differential Revision: https://phabricator.services.mozilla.com/D28167
--HG--
extra : moz-landing-system : lando
There functions won't change any internal variables, so they should be marked as const.
Differential Revision: https://phabricator.services.mozilla.com/D29590
--HG--
extra : moz-landing-system : lando
Replace the MacSandboxType_Plugin sandbox type with MacSandboxType_Flash and MacSandboxType_GMP so that there is a 1:1 association between MacSandboxType values and sandbox policies.
Remove the MacSandboxPluginType enum. Instead of having different MacSandboxPluginTypes, we will just have MacSandboxType_GMP. We only use GMP for two plugin types, Widevine and OpenH264, and they only differ in that Widevine requires accss to the WindowServer.
Remove the MacSandboxPluginInfo struct and move the two needed fields pluginPath and pluginBinaryPath to MacSandboxInfo.
Differential Revision: https://phabricator.services.mozilla.com/D29585
--HG--
extra : moz-landing-system : lando
Normally when downloading media data we throttle the download only if we're
ahead of the read cursor more than the "readahead limit", and if we estimate
that the connection is fast enough that we'll be able to download at a rate
fast enough to playback in real time if we resume it later.
On mobile we additionally override this so that we always throttle the download
once we're ahead of the read cursor by the readahead limit. This is to save
data. I think we can relax this to only do this override if we're on a
cellular connection; if we're on WiFi we can assume data is cheap.
Differential Revision: https://phabricator.services.mozilla.com/D26234
--HG--
extra : moz-landing-system : lando
In GeckoView the nsINetworkLinkService doesn't work in the content process, as
we don't seem to have an AndroidBridge there, so just maintain the network
connection type on the ContentChild.
(I had considered keeping this on the NeckoChild, but the creation of that is
initiated from the content process side, and there's not an easy and clean way
to have the parent process send us the connection type after construction of
the NeckoParent, other than have the NeckoChild request it either
synchronously, or doing it async and hoping it's not asked for the value before
the response comes in.)
Differential Revision: https://phabricator.services.mozilla.com/D26232
--HG--
extra : moz-landing-system : lando
float(INT32_MAX); gets compiled into 2.14748365E+9 using clang, which is slightly bigger than INT32_MAX, as such 2.14748365E+9 <= INT32_MAX will return true (as INT32_MAX gets converted to a float)
Differential Revision: https://phabricator.services.mozilla.com/D29464
--HG--
extra : moz-landing-system : lando
This removes all references to application capture except from MediaSourceEnum.
That was left in place so as to not change the enumerated values used
for WEBRTC_GET_USER_MEDIA_TYPE telemetry.
Differential Revision: https://phabricator.services.mozilla.com/D28089
--HG--
extra : moz-landing-system : lando
These are added in start, and we get into unexpected state if they notify us
after Stop() (when MediaEncoder shuts down and internally removes the tracks
it is encoding) but before Shutdown() when we remove the listeners. This is not
symmetrical. The proper thing to do is to remove these listeners in Stop() as
well.
Depends on D29080
Differential Revision: https://phabricator.services.mozilla.com/D29081
--HG--
extra : moz-landing-system : lando
The mp4 demuxer already does this, so have the WebM case follow suit. This
avoids asserts with creating 0 size shmems in the case where the demuxer is
being used with the remote data decoder.
Differential Revision: https://phabricator.services.mozilla.com/D25846
--HG--
extra : moz-landing-system : lando
As the purpose of adjusting time is to make the start of the playback to align to zero, so the result should not be negative.
However, we can't always ensure the demuxed start time is correct, the decoded first frame sample time might be different with the time demuxer provides.
Even if we try to update the start time when we get the first decoded sample, there is still a problem because a decoder might return samples with different order, which means the first frame the decoder returns is probably not a real first frame.
Therefore, using warning to help us knows how many this situaion would be.
Differential Revision: https://phabricator.services.mozilla.com/D28432
--HG--
extra : moz-landing-system : lando
Considering that the audio sample's time is always increased, the decoded sample's time decoder returns should always be equal or larger than its demuxed time.
When the decoded sample's time is smaller than the time of first demuxed sample, that time would probably cause a problem so we should throw an error for that.
Differential Revision: https://phabricator.services.mozilla.com/D28167
--HG--
extra : moz-landing-system : lando
Per the discussion in:
https://groups.google.com/d/msg/mozilla.dev.platform/P79pwa9z5m8/iPYPAWPHCAAJ
They should be CamelCase, and that's what most of them already do. This converts
the rest, which are a few.
For the ones that already used `e` or `k` prefixes, I've mostly done:
for file in $(rg Type::e layout | cut -d : -f 1 | sort | uniq); do sed -i 's#Type::e#Type::#g' $file; done
For the ones that used uppercase, I've removed the prefix if it was already in
the type name, and turn them into CamelCase.
Depends on D28680
Differential Revision: https://phabricator.services.mozilla.com/D28681
--HG--
extra : moz-landing-system : lando
We're rarely getting nullptrs out of MediaQueues. It's not clear where these are
coming from, as we have many guards against them. Upgrade this assert to a
diagnostic to help us track the source and determine if the value is null before
entering the queue.
Differential Revision: https://phabricator.services.mozilla.com/D28866
--HG--
extra : moz-landing-system : lando
We don't currently know any ways in which external code can make the
suspendCount negative, but this will keep us safe should a future bug enable it.
Depends on D28805
Differential Revision: https://phabricator.services.mozilla.com/D28806
--HG--
extra : moz-landing-system : lando
This patch moves remaining public `enum` of `nsIPresShell` to `mozilla`
namespace in `mozilla/PresShellForwards.h` and make them `enum class`es.
Additionally, some methods which use the moving `enum`s from `nsIPresShell`
to `PresShell`.
Differential Revision: https://phabricator.services.mozilla.com/D28607
--HG--
extra : moz-landing-system : lando
The value of `mAudioDecodedDuration` can be larger than `int32`, so we should use the modular function which accepts `TimeUnit`as a input.
Differential Revision: https://phabricator.services.mozilla.com/D28536
--HG--
extra : moz-landing-system : lando
We should only use valid time to construct a `SeekTarget`, because we can't get the value from the invalid time.
This can help us to prevent having a crash when request data value from an invalid time.
Differential Revision: https://phabricator.services.mozilla.com/D28564
--HG--
extra : moz-landing-system : lando
As the purpose of adjusting time is to make the start of the playback to align to zero, so the result should not be negative.
Differential Revision: https://phabricator.services.mozilla.com/D28432
--HG--
extra : moz-landing-system : lando
Considering that the audio sample's time is always increased, the decoded sample's time decoder returns should always be equal or larger than its demuxed time.
When the decoded sample's time is smaller than the time of first demuxed sample, that time would probably cause a problem so we should throw an error for that.
Differential Revision: https://phabricator.services.mozilla.com/D28167
--HG--
extra : moz-landing-system : lando
This moves the responsibility for creating MediaStreamTracks from
DecodedStream::Start to MediaDecoder. This let's MediaDecoder create them as
soon as metadata is known. This gives the application guarantees on when tracks
can be expected to exist, and aligns with the spec that says they should be
created when metadata is known.
Differential Revision: https://phabricator.services.mozilla.com/D28473
--HG--
extra : moz-landing-system : lando
This gives us a guarantee that the first frame of a media file can be rendered
with a second media element and mozCaptureStream(), even if the file is very
very short.
With longer video-only files there were also cases where nothing ended up being
rendered. Probably because the MediaStreamGraph ended up switching from an
AudioCallbackDriver to a SystemClockDriver and this took enough time to put the
SourceMediaStream::EndTrack and the SourceMediaStream::AddTrackListener calls
for this video track to be processed in the same iteration. The listener would
then always lose to the ending track and update main thread state too late,
leading to its media element not rendering any frames and nasty intermittent
failures.
Differential Revision: https://phabricator.services.mozilla.com/D27270
--HG--
extra : moz-landing-system : lando
`streamCurrentTime` is relative to the start of the stream, as is the track's
segment. We need to account for the track's start time in the stream when
notifying of output.
Differential Revision: https://phabricator.services.mozilla.com/D27268
--HG--
extra : moz-landing-system : lando
A case where this wasn't working was bipbop-lateaudio.mp4, where the last frame
has duration 0 and starts and ends before the previous frame has ended. The
VideoSink still renders this frame at the end of playback, so this patch
brings DecodedStream closer to that behavior by rendering all frames with a
start time after the previous frame's start time. The track's duration is still
based on absolute times so things don't blow up.
Differential Revision: https://phabricator.services.mozilla.com/D27267
--HG--
extra : moz-landing-system : lando
This is necessary to get an accurate total track duration in case DecodedStream
does a lot of seeking and pausing.
This also removes previous code for avoiding blocking a video track, as this new
code handles that case too.
Differential Revision: https://phabricator.services.mozilla.com/D27264
--HG--
extra : moz-landing-system : lando
Null frames could still be valid. It's how DecodedStream signals a pause. They
could also be valid and have an intrinsic size and the force-black flag set.
Differential Revision: https://phabricator.services.mozilla.com/D27263
--HG--
extra : moz-landing-system : lando
This is similar to what VideoSink does, and is required for us to end playback
on HAVE_CURRENT_DATA (a frame for currentTime must be available).
Differential Revision: https://phabricator.services.mozilla.com/D27261
--HG--
extra : moz-landing-system : lando
This tests HTMLMediaElement::MozCaptureStream on all of our play-tests, rather
than just a single test, in order to test more corner cases in the capture code.
Because this test checks that the media element that's playing the
mozCaptureStream()-stream ends after at least the duration of the original media
file, and because media elements playing a MediaStream have a `currentTime`
based on the MediaStreamGraph's monotonically increasing clock, we need to have
accurate metadata for how long the duration of the content of the original media
file is. There are several cases among the play-tests where the duration
metadata doesn't match what the decoder gives us -- the reason is often preroll
or corrupt containers. This patch updates manifest.js with `contentDuration` for
the tests among `gPlayTests` where try detected mismatches.
Differential Revision: https://phabricator.services.mozilla.com/D27257
--HG--
extra : moz-landing-system : lando
When we suspend or resume the `AudioContext`, it should affect ALL media streams which belong to or are related to the `AudioNode` that are created by this `AudioContext`.
As `AudioNode::OutputParams()` can only return the connected AudioParams, it doesn't return the AudioParams which are belong to itself. That means we would miss to apply the suspend/resume operation for those streams, and it would cause imbalancing suspended count.
Therefore, we let `AudioNode` to keep the reference of all its AudioParam, and return them to `AudioContext` in order to do the operation for all streams.
Differential Revision: https://phabricator.services.mozilla.com/D28008
--HG--
extra : moz-landing-system : lando
Create a new IPDL message `ShutdownComplete`, direction from parent (RDD) to child (content), to inform the child when the decoder shutdown has been completed. The remote decoder child uses that message to resolve the shutdown promise.
Differential Revision: https://phabricator.services.mozilla.com/D28340
--HG--
extra : moz-landing-system : lando
Changes:
- added comments for tests being disabled
- disabled two additional tests in order to green the run
Differential Revision: https://phabricator.services.mozilla.com/D28085
--HG--
extra : moz-landing-system : lando
The attributes for an interface should be on the line right before the
interface.
Interface attributes should be separated by spaces.
Clean up some trailing whitespace in widget/.
Differential Revision: https://phabricator.services.mozilla.com/D28234
--HG--
extra : moz-landing-system : lando
If the RDD process is enabled and fails to start properly for some reason,
make sure we cannot accidentally fall back to decoding AV1 on the content
process.
Differential Revision: https://phabricator.services.mozilla.com/D28204
--HG--
extra : moz-landing-system : lando
As operator`%` would need to handle `TimeUnit` as a input as well, we add new operator function.
Also use `int64` as a input parameter for another two operator functions `%` and `/` in order to add assertion to ensure that the input won't overflow.
Differential Revision: https://phabricator.services.mozilla.com/D27644
--HG--
extra : moz-landing-system : lando
CloseAudioInut method posts a message, to the graph thread, in order to close the input asynchonously. When CloseAudioInput method was being executed from a thread other than the main thread, a runnable would be posted to main thread in order to post the async message from there. That was a risky path because when the graph was shutting down there were no guarantee that the close-input message would reach the graph thread before destroy takes place. By limiting CloseAudioInput to main thread it is ensured that the close-input message will be executed before destroy.
Differential Revision: https://phabricator.services.mozilla.com/D27751
--HG--
extra : moz-landing-system : lando
Revive method of AudioCallbackDriver was wrong because it was re-initializing an already initialized driver. That was hitting an assert. Instead of that it should stop the drained stream and start it again. Also, mStarted member should be reset properly on the stop method.
Differential Revision: https://phabricator.services.mozilla.com/D26678
--HG--
extra : moz-landing-system : lando
This gives us a guarantee that the first frame of a media file can be rendered
with a second media element and mozCaptureStream(), even if the file is very
very short.
With longer video-only files there were also cases where nothing ended up being
rendered. Probably because the MediaStreamGraph ended up switching from an
AudioCallbackDriver to a SystemClockDriver and this took enough time to put the
SourceMediaStream::EndTrack and the SourceMediaStream::AddTrackListener calls
for this video track to be processed in the same iteration. The listener would
then always lose to the ending track and update main thread state too late,
leading to its media element not rendering any frames and nasty intermittent
failures.
Differential Revision: https://phabricator.services.mozilla.com/D27270
--HG--
extra : moz-landing-system : lando
`streamCurrentTime` is relative to the start of the stream, as is the track's
segment. We need to account for the track's start time in the stream when
notifying of output.
Differential Revision: https://phabricator.services.mozilla.com/D27268
--HG--
extra : moz-landing-system : lando
A case where this wasn't working was bipbop-lateaudio.mp4, where the last frame
has duration 0 and starts and ends before the previous frame has ended. The
VideoSink still renders this frame at the end of playback, so this patch
brings DecodedStream closer to that behavior by rendering all frames with a
start time after the previous frame's start time. The track's duration is still
based on absolute times so things don't blow up.
Differential Revision: https://phabricator.services.mozilla.com/D27267
--HG--
extra : moz-landing-system : lando
This is necessary to get an accurate total track duration in case DecodedStream
does a lot of seeking and pausing.
This also removes previous code for avoiding blocking a video track, as this new
code handles that case too.
Differential Revision: https://phabricator.services.mozilla.com/D27264
--HG--
extra : moz-landing-system : lando
Null frames could still be valid. It's how DecodedStream signals a pause. They
could also be valid and have an intrinsic size and the force-black flag set.
Differential Revision: https://phabricator.services.mozilla.com/D27263
--HG--
extra : moz-landing-system : lando
This is similar to what VideoSink does, and is required for us to end playback
on HAVE_CURRENT_DATA (a frame for currentTime must be available).
Differential Revision: https://phabricator.services.mozilla.com/D27261
--HG--
extra : moz-landing-system : lando
This tests HTMLMediaElement::MozCaptureStream on all of our play-tests, rather
than just a single test, in order to test more corner cases in the capture code.
Because this test checks that the media element that's playing the
mozCaptureStream()-stream ends after at least the duration of the original media
file, and because media elements playing a MediaStream have a `currentTime`
based on the MediaStreamGraph's monotonically increasing clock, we need to have
accurate metadata for how long the duration of the content of the original media
file is. There are several cases among the play-tests where the duration
metadata doesn't match what the decoder gives us -- the reason is often preroll
or corrupt containers. This patch updates manifest.js with `contentDuration` for
the tests among `gPlayTests` where try detected mismatches.
Differential Revision: https://phabricator.services.mozilla.com/D27257
--HG--
extra : moz-landing-system : lando
- `Array.map` becomes `Array.from`
- Array copying via `Array.slice` becomes `Array.from`.
- `Array.forEach` that did not rely on closures becomes `for`-`of` loops.
- Anything else: `Array.X` becomes `Array.prototype.X`.
Complex cases:
dom/bindings/test/TestInterfaceJS.js and
dom/bindings/test/test_exception_options_from_jsimplemented.html
use `Array.indexOf` to generate an error with a specific error message.
Switched to `Array.prototype.forEach` to generate the same error.
js/src/jit-test/tests/basic/exception-column-number.js
In this test `Array.indexOf()` is used to generate an error. Since the
exact message doesn't matter, I switched to `Array.from()`.
Intentionally not changed:
editor/libeditor/tests/browserscope/lib/richtext/richtext/js/range.js
Did not modify because this is 3rd-party code and the code uses
feature detection as a fall back when Array generics are not used.
testing/talos/talos/tests/dromaeo/lib/mootools.js
Did not modify because mootools adds the `Array.slice` method to the
`Array` object.
Not changed because they check the implementation of Array generics:
js/src/jit-test/tests/basic/arrayNatives.js
js/src/jit-test/tests/basic/bug563243.js
js/src/jit-test/tests/basic/bug618853.js
js/src/jit-test/tests/basic/bug830967.js
js/src/jit-test/tests/jaeger/recompile/bug656753.js
js/src/jit-test/tests/self-hosting/alternate-static-and-instance-array-extras.js
js/src/tests/non262/Array/generics.js
js/src/tests/non262/Array/regress-415540.js
js/src/tests/non262/extensions/regress-355497.js
js/src/tests/non262/extensions/typedarray-set-neutering.js
Depends on D27802
Differential Revision: https://phabricator.services.mozilla.com/D27803
--HG--
extra : moz-landing-system : lando
Shutting down the taskqueue early prevents the decoder's tasks to be queued.
A TaskQueue no longer requires to be explicitly shutdown it will shutdown when ref counter drops to zero.
Differential Revision: https://phabricator.services.mozilla.com/D26929
--HG--
extra : moz-landing-system : lando
Changes:
- most tests are skipped using `moz.build` configuration file.
- `MultiWriterQueue` had to be skipped with `define` clauses in the test file due to build bustages when its `moz.build` file was used.
Differential Revision: https://phabricator.services.mozilla.com/D27944
--HG--
extra : moz-landing-system : lando
The spec referenced in the comments in code has been updated since the code was
originally written. As such, some of the sections referenced in code comments
now have different numbering. Update the comments to reflect this.
Differential Revision: https://phabricator.services.mozilla.com/D27726
--HG--
extra : moz-landing-system : lando
If the input has been overflow, we don't need to calculate frames and will directly return invalid result to let caller handle the error.
Differential Revision: https://phabricator.services.mozilla.com/D27010
--HG--
extra : moz-landing-system : lando
If the input has been overflow, we don't need to calculate frames and will directly return invalid result to let caller handle the error.
Differential Revision: https://phabricator.services.mozilla.com/D27010
--HG--
extra : moz-landing-system : lando
This patch adds the number of dropped frames for each step of the process
(read/sink/compositor) and gives us more insight about where frames are
dropped, as opposed to the getVideoPlaybackQuality() API which gives the grand
total.
Differential Revision: https://phabricator.services.mozilla.com/D27488
--HG--
extra : moz-landing-system : lando
This is split from the previous changeset since if we include dom/ the file size is too
large for phabricator to handle.
This is an autogenerated commit to handle scripts loading mochitest harness files, in
the simple case where the script src is on the same line as the tag.
This was generated with https://bug1544322.bmoattachments.org/attachment.cgi?id=9058170
using the `--part 2` argument.
Differential Revision: https://phabricator.services.mozilla.com/D27457
--HG--
extra : moz-landing-system : lando
When unplugging a camera in use in a video-only gUM capture, we see a crash in
GetUserMediaWindowListener::Remove when trying to use `aListener` after its last
reference was removed from an nsRefPtrHashTable at the beginning of the method.
As `aListener` was a rawptr, neither this method itself, nor its caller, kept it
alive.
Making this method take a RefPtr instead of a rawptr solves this issue.
This patch also converts other similar rawptrs for good measure.
Differential Revision: https://phabricator.services.mozilla.com/D27520
--HG--
extra : moz-landing-system : lando
I missed this intermittent when enabling Android gtests in bug 1318091. Disable the failing test on Android only, to green up Android gtest.
Differential Revision: https://phabricator.services.mozilla.com/D27556
--HG--
extra : moz-landing-system : lando
I missed this intermittent when enabling Android gtests in bug 1318091. Disable
the failing test on Android only, to green up Android gtest.
Differential Revision: https://phabricator.services.mozilla.com/D27536
--HG--
extra : moz-landing-system : lando
The Picture-in-Picture toggle buttons are now part of the video controls UAWidget
bindings, so we need to construct a UAWidget for the no-controls case for Desktop
to make that toggle available.
Up until now, we've never needed a no-controls UAWidget for Desktop, since we
never needed to show UI in that case.
Depends on D26809
Differential Revision: https://phabricator.services.mozilla.com/D26803
--HG--
extra : moz-landing-system : lando
This patch makes `nsDocShell::GetPresShell()` and
`nsDocShell::GetEldestPresShell()` return `mozilla::PresShell*` and
some non-public methods use `mozilla::PresShell*` directly.
Differential Revision: https://phabricator.services.mozilla.com/D26424
--HG--
extra : moz-landing-system : lando
This ignores warnings in endpointer.cc/h and energy_endpointer.cc/h as they
appear to be from Chromium.
Differential Revision: https://phabricator.services.mozilla.com/D26725
--HG--
extra : moz-landing-system : lando
This will let us clear the SpeechTrackListener's ref-cycle and avoid leaking.
Differential Revision: https://phabricator.services.mozilla.com/D26724
--HG--
extra : moz-landing-system : lando
Disable test_isTypeSupported on android version >= 28 (Android 9.0) since our
infra there doesn't support cross origins there, at least until bug 1335740
is resolved.
Differential Revision: https://phabricator.services.mozilla.com/D27215
--HG--
extra : moz-landing-system : lando
There are slight differences produced between the earlier 422 output surface and the 420; interestingly the image is sharper with 420.
Differential Revision: https://phabricator.services.mozilla.com/D27020
--HG--
extra : moz-landing-system : lando
No color space was defined for these videos, making any correct rendering impossible.
We modify the h264 stream metadata, by marking it as being bt601.
This is done with the ffmpeg command:
ffmpeg -i dom/media/test/short.mp4 -codec copy -bsf:v h264_metadata=colour_primaries=6:transfer_characteristics=6:matrix _coefficients=6 -map 0
Differential Revision: https://phabricator.services.mozilla.com/D26871
--HG--
extra : moz-landing-system : lando
It allows for more readable code, not having to store multiple times different storage type across multiple objects.
Now each class does one task and only deal with a single texture data type.
Differential Revision: https://phabricator.services.mozilla.com/D26473
--HG--
extra : moz-landing-system : lando
YUVColorSpace is inseparable from the bit depth as the matrix coefficients to be calculated need the bit depth information.
So let's put the two types together. gfx namespace also makes more sense as that's where we find IntRect, IntSize and other.
The extent of the changes highlight how much similar data structures are duplicated across the code, to the point it's scary.
Differential Revision: https://phabricator.services.mozilla.com/D25347
--HG--
extra : moz-landing-system : lando
All compositors support 10/12 bits images now.
Additionally, add BT2020 support to AOM decoder.
Differential Revision: https://phabricator.services.mozilla.com/D25344
--HG--
extra : moz-landing-system : lando
There is no need to cast to `TimeUnit` to `int64` and then cast to `TimeUnit` again.
Differential Revision: https://phabricator.services.mozilla.com/D26758
--HG--
extra : moz-landing-system : lando
Now the caller of `AdjustForStartTime()` should check the return value to know whether the adjustment is succeeded.
If the error happens the caller should handle it and return an error.
Differential Revision: https://phabricator.services.mozilla.com/D26756
--HG--
extra : moz-landing-system : lando
alwu
Andreas Pehrson
Mihai Alexandru Michis
Gurzau Raul
Michael Froman
Jean-Yves Avenard
Nico Grunbaum
Chris Pearce
Nika Layzell
Paul Adenot
nils@vulndev.org, Karl Tomlinson
Bryce Van Dyk
Jan Beich
Jan-Ivar Bruaroey
Alex Chronopoulos
Ryan VanderMeulen
Jim Mathies
Daniel Varga
Andreea Pavel
Byron Campen [:bwc]
shindli
Dorel Luca
Tarek Ziadé
Geoff Brown
Boris Zbarsky
Valentin Millet
Haik Aftandilian
Noemi Erli
Dan Minor
Sylvestre Ledru
Cosmin Sabou
Emilio Cobos Álvarez
Masayuki Nakano
Razvan Maries
Edwin Gao
Narcis Beleuzu
Andrew McCreight
Ciure Andrei
Rob Wu
Brian Grinstead
arthur.iakab
Markus Stange
Coroiu Cristina
Mike Conley
Oana Pop Rus
Csoregi Natalia