Currently we didn't consider the engine's output volume when we determine the audible state for the destination node, doing that would help us to more correctly deide the actual audible state for users.
Differential Revision: https://phabricator.services.mozilla.com/D94182
The main purposes of this patch is to
- changes the way the destination node holding the audio channel agent (holding the agent all the time)
That would help us
- earlier mute the destination node in order to prevent sound leaking
- request wakelock based on the actual audible state (considering more factors)
- improve the readability of handling audible state (the old way contains some weird workaround, eg. setting the node as audible in the beginning)
Differential Revision: https://phabricator.services.mozilla.com/D94181
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
When a page doesn't have a non-empty media session metadata, we would use its title as a default metadata, which would display as an artist name on the virtual control interface.
Therefore, when a page changes its title, we should also notify media controller (if it's been created) in order to update the default metadata as well.
Differential Revision: https://phabricator.services.mozilla.com/D93670
GeckoView is seemingly the last user depending on MediaManager iterating over
a all iframes of a window when queried for its capture state.
It is fine however, since GeckoViewMedia already iterates over all windows in
MediaManagerService.activeMediaCaptureWindows, which includes all subframes that
are actively captured.
This patch removes IterateWindowListeners altogether, and the last callsite is
simplified.
Differential Revision: https://phabricator.services.mozilla.com/D93079
And we remove unnecessary checks, BackgroundParent only run in the parent process and if e10s is on. Also RecvLauchRDDProcess will only ever be called if the rdd pref is on already.
By streamlining the call we also reduce the number of sync dispatch to 1.
Depends on D93317
Differential Revision: https://phabricator.services.mozilla.com/D93476
Following bug 1566389, an AudioBuffer can be created only to be then truncated if we need to trim it back to a size of 0.
When the resulting AudioBuffer is transferred over IPC it will be rebuilt, and be null then making EnsureAudioBuffer fail.
Depends on D92345
Differential Revision: https://phabricator.services.mozilla.com/D92714
The code always assumed that the size of the image with the Y plane dimensions, which, while often the case, isn't correct.
We remove the assertions that the display offset was always (0,0) and properly carry the actual data over IPC.
Remoting the theora decoder and enabling fast video copy exposed several other related issues in the various D3D11 image types.
Various WPT uses theora YUV44 images (because we do not support YUV444 H264 ones). Those images are made of 32 pixels planes with a display size set to 20 pixels. Prior P1D the backend image was a ShareYCbCrPlanar image which correctly handled the size settings.
Like the image serializer, the various D3D11 images always assumed that the Y plane size was the image size.
This however expose existing issues where the offset position of the display is completely ignored for some image type. See bug 1669054
All those problems explain why sometimes we displayed more pixels than we should have.
Depends on D91914
Differential Revision: https://phabricator.services.mozilla.com/D92233
We unfortunately must skip the fastpath tests on mac and linux as they will succeed on M-gli platform (WebGL in GPU) and we can't filter those out.
Depends on D91700
Differential Revision: https://phabricator.services.mozilla.com/D91914
Attempting to initialize a PDM can cause the process to be killed due to sandbox violations.
For now, only FFVPX decoders is working and enabled in the RDD process.
One downside is that changing the ffvpx/ffmpeg rdd pref now requires a restart.
Will be able to remove such issue once all sandbox issues have been resolved.
Depends on D91693
Differential Revision: https://phabricator.services.mozilla.com/D91694
Ultimately, we should be able to remove everything that got added to the RDD sandbox from the content's one.
Fly-by fix; allow checking if AVX512 is supported in content sandbox.
Splitting sandbox setting from Utility to a new RDD one as per review comment in P12 considering it's no longer an IPC only sandbox.
Depends on D91688
Differential Revision: https://phabricator.services.mozilla.com/D91689
Add a synchronous Supports message to the IPDL PRemoteDecoderManager protocol so
decoder modules can query for playback support in the actual process that will
attempt to do the decoding.
Depends on D54878
Differential Revision: https://phabricator.services.mozilla.com/D54879
This is a subset of the parameters passed via CreateDecoderParams and is so
Supports() calls have access to KnowsCompositor and Options when determining
if decoding is supported.
Depends on D54877
Differential Revision: https://phabricator.services.mozilla.com/D54878
This moves most of the JsepTrack functionality for dealing with duplicate
payload types to a new EnsurePayloadTypeNotDuplicate method in
JsepCodecDescription. It also adds a virtual EnsureNoDuplicatePayloadTypes
method that checks the appropriate payload types for each codec type.
Differential Revision: https://phabricator.services.mozilla.com/D92112
There is a case where two VideoOutputs are feeding the same VideoFrameContainer
with data: VideoStreamTrack::AddVideoOutput(VideoFrameContainer) and
VideoStreamTrack::AddVideoOutput(FirstFrameVideoOutput) in HTMLMediaElement.
This is racy since they both call VideoFrameContainer::NewFrameID().
These VideoOutputs use different ProducerIDs however, so it makes sense for them
to also use separate FrameID generators.
Differential Revision: https://phabricator.services.mozilla.com/D94054
The purpose for those reftest at present isn't to verify that our image conversion is correct (it isn't), just that decoding produce something not completely broken
Differential Revision: https://phabricator.services.mozilla.com/D94065
We have reports of VPx WMF failures in the wild causing performance issues, but
it can be hard to detect these failures. Having explicit markers in the profiler
with more information will make it easier to see if users are running into these
issues if they can provide profiles. I've also added logs to enrich cases where
we will use logging to debug.
Driveby fix a comment that says we're asserting when we no longer are.
Differential Revision: https://phabricator.services.mozilla.com/D93672
We have reports of VPx WMF failures in the wild causing performance issues, but
it can be hard to detect these failures. Having explicit markers in the profiler
with more information will make it easier to see if users are running into these
issues if they can provide profiles. I've also added logs to enrich cases where
we will use logging to debug.
Driveby fix a comment that says we're asserting when we no longer are.
Differential Revision: https://phabricator.services.mozilla.com/D93672
We have reports of VPx WMF failures in the wild causing performance issues, but
it can be hard to detect these failures. Having explicit markers in the profiler
with more information will make it easier to see if users are running into these
issues if they can provide profiles. I've also added logs to enrich cases where
we will use logging to debug.
Driveby fix a comment that says we're asserting when we no longer are.
Differential Revision: https://phabricator.services.mozilla.com/D93672
It would be good for us to know how many websites actually use MediaSession API to control (play/pause/stop) their media playback, an how many websites use our default handler.
Differential Revision: https://phabricator.services.mozilla.com/D93283
Each platform has different ways to allow users to use media control, adding a telemetry probe to detect that would be good for us to know the usage among different platforms.
Differential Revision: https://phabricator.services.mozilla.com/D93282
It would be good for us to know how many websites actually use MediaSession API to control (play/pause/stop) their media playback, an how many websites use our default handler.
Differential Revision: https://phabricator.services.mozilla.com/D93283
Each platform has different ways to allow users to use media control, adding a telemetry probe to detect that would be good for us to know the usage among different platforms.
Differential Revision: https://phabricator.services.mozilla.com/D93282
This interface is never used directly, and the only consumers of the virtual functions are by the derived classes themselves.
Differential Revision: https://phabricator.services.mozilla.com/D93180
Since the semantics of `ContentParent::MarkAsDead` are significantly different
from `GeckoProcessManager::MarkAsDead`, let's rename the latter to better
reflect what it actually does.
Differential Revision: https://phabricator.services.mozilla.com/D92649
- What this patch does:
Send a "download-suspended-by-cache" signal to the media-element paired
with a cloned cache-stream right after the cache-stream is cloned. That
signal help the media-element to decide if it's ready to enter the
HAVE_ENOUGH_DATA state and fire a "canplaythrough" event.
- Why this patch is needed:
It solves a WPT timeout issue. That WPT test waits a "canplaythrough"
event.
- What problem this patch solves is:
This patch addresses the problem mentioned in [1].
Each media-element pairs with a cache-stream downloading the data it
needs. When the data-download of a cache-stream gets suspended, the
media-element paired with the cache-stream should be notified. This
notification is one of the factor to move the ready-state of the
media-element to HAVE_ENOUGH_DATA. However, the media-element paired
with a cloned cache-stream may never receive this notification. And the
worst is that it may never have a chance to download enough data it
needs to move the ready-state to HAVE_ENOUGH_DATA as well at the same
time, in the case mentioned below.
This can happen when a media-element paired with a cloned cache-stream
is created after the cache is full, and all of the cache-streams are
suspended. (Cloned cache-stream is a cache-stream cloned from another
cache-stream that shares the same underlying data with it since their
paired media-elements have the same `src` (of `HTMLMediaElement`)).
"canplaythrough" event is fired when the ready-state is transited to
HAVE_ENOUGH_DATA. As a result, it will never be fired in this case.
In usual case, if the cache-stream gets suspended from non-suspended, it
will send a "download-suspended-by-cache=true" signal to its paired
media-element when running `MediaCache::Update()`. In fact, all other
cache-streams sharing the same underlying data will send this signal at
the same time if necessary. (Later, once the cache-stream is resumed
from suspended to non-suspended it will send a
"download-suspended-by-cache=false" signal to its paired media-element.
All other cache-streams sharing the same underlying data will do the
same if necessary.) The media-element keeps tracking that signal it
receives. After the first-frame of the media-element is loaded, the
ready-state of the media-element will be transited to HAVE_ENOUGH_DATA
by force if the signal is true. (Otherwise, the ready-state will be
inferred by other information.)
When cloning a cache-stream from another one, the cloned cache-stream is
suspended by default. If it's added to a jammed cache that all of the
cache-streams are suspended since the cache is full, then it never has a
chance to fire the "download-suspended-by-cache" signal. Both its
source-stream and itself have no status-change between suspended and
non-suspended, so `MediaCache::Update` is unable to send the signal.
In this case, we should force the media-element paired with the newly
cloned cache-stream transits its ready-state to HAVE_ENOUGH_DATA, which
follows the existing mechanism, by queueing a status-change-update when
cloning the stream. The status-change-update will be run in the
`MediaCache::Update` and it will check what the signal should be sent.
Once the "download-suspended-by-cache=true" is sent, the
"canplaythrough" event of the media-element can be dispatched after
its first-frame is loaded. (The event listeners can possibly make the
media-element starts playing, which is likely to cause a cache-seek that
can revitalize the cache eventually.)
[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1646719#c5
Differential Revision: https://phabricator.services.mozilla.com/D92125
Add some logs that can help us to check how many cache-streams are
suspended and the working status in `MediaCache::Update`
Differential Revision: https://phabricator.services.mozilla.com/D92124
Without this patch, the FrameEncode gtest encoder won't encode the last bit of
data, making the total duration 20ms too short.
When passing EOS we encode the lookahead worth of silence, so this patch also
accounts for that.
Differential Revision: https://phabricator.services.mozilla.com/D91957
Without this patch, there could be an input rate leading to the use of a
resampler *and* framesLeft being 0 (not rounding up). Then we end up with too
little data to feed the resampler, and we fail an assert.
Differential Revision: https://phabricator.services.mozilla.com/D91956
Without this patch, the lookahead silence would be added to the source segment,
which is sampled at the input rate.
The lookahead is in the output rate.
Converting the lookahead to the input rate, and letting our regular encoding
logic convert it back to the output rate can lead to a rounding error for some
input rates. A rounding error here would lead to the last packet being too
short.
Differential Revision: https://phabricator.services.mozilla.com/D91725
There should be no accumulating rounding error here since packet durations are
exactly 200ms which does not lead to a rounding error when converting to
microseconds.
But this is for sanity, since the behavior prior to this patch is exactly how
you get an accumulating rounding error.
Differential Revision: https://phabricator.services.mozilla.com/D91955
Certain logic, like AudioOutputFramesPerPacket(), is hinged off whether a
resampler exists. The resampler is destroyed when encoding is completed, making
such logic flawed from that point. To avoid this potential footgun we can decide
the output rate at construction time, since the input rate is known then.
Differential Revision: https://phabricator.services.mozilla.com/D91953
AudioTrackEncoder uses GetPacketDuration() for signaling upwards that data is
available to be encoded. Data to be encoded is sampled at the input rate while
GetPacketDuration() is the duration in the output rate.
Meanwhile, OpusTrackEncoder uses GetPacketDuration() internally for deciding how
much data to encode. This is after resampling so correctly in the output rate.
To support both these cases, this patch adds NumOutputFramesPerPacket(), modeled
on GetOutputSampleRate(), denoting the packet duration in the output rate.
GetPacketDuration() is renamed to NumInputFramesPerPacket() and changed to be
the packet duration in the input rate.
Differential Revision: https://phabricator.services.mozilla.com/D91952
Without this patch there was a gap between the default-ctor and when real values
got set. If setting a member was forgotten, it would have needed an audit to be
found. With this patch the compiler will make sure all values have been
explicitly handed to the ctor.
mFrameData (nsTArray) becomes Refcountable to allow VP8TrackEncoder to extend
the duration of an EncodedFrame by copying the last frame's ref and constructing
a new EncodedFrame with a longer duration than the last one's.
Differential Revision: https://phabricator.services.mozilla.com/D91724
This was originally handled by EbmlComposer. Since bug 1014393 this was handled
by MediaEncoder. By doing it in OpusTrackEncoder we can avoid reading hardcoded
fields in the opus metadata to get the codec delay value.
Differential Revision: https://phabricator.services.mozilla.com/D91723
When close the event source, it should be responsible to clear up and reset the virtual control interface, rather than doing so by `Media Control Server` via setting some empty results.
Differential Revision: https://phabricator.services.mozilla.com/D92116
The old way to open/close the event source, which is triggered by controller amount change event, is less intuitive, and we do the extra clean up when close the event source by assigning some parameters [1] that causes an issue on Windows where the control interface can't be clear up completely.
Each platform has its own way to clean the interface. For example, on Windows, we can simply call `ISystemMediaTransportControlsDisplayUpdater::ClearAll()`. So calling those functions actually helps nothing. The best way to do that is to ask the event source to do the clean up, rathering than setting those unnecessary parameters.
Therefore, we make it happen closer to when we determine or clear main controller and ask the event source to take a responsible to clean up when it closes.
[1] https://searchfox.org/mozilla-central/rev/35245411b9e8a911fe3f5adb0632c3394f8b4ccb/dom/media/mediacontrol/MediaControlService.cpp#410-413
Differential Revision: https://phabricator.services.mozilla.com/D92115
As we've chosen another way for GeckoView implementation, so `SetControlledTabBrowsingContextId` is no longer needed.
Differential Revision: https://phabricator.services.mozilla.com/D92114
When close the event source, it should be responsible to clear up and reset the virtual control interface, rather than doing so by `Media Control Server` via setting some empty results.
Differential Revision: https://phabricator.services.mozilla.com/D92116
The old way to open/close the event source, which is triggered by controller amount change event, is less intuitive, and we do the extra clean up when close the event source by assigning some parameters [1] that causes an issue on Windows where the control interface can't be clear up completely.
Each platform has its own way to clean the interface. For example, on Windows, we can simply call `ISystemMediaTransportControlsDisplayUpdater::ClearAll()`. So calling those functions actually helps nothing. The best way to do that is to ask the event source to do the clean up, rathering than setting those unnecessary parameters.
Therefore, we make it happen closer to when we determine or clear main controller and ask the event source to take a responsible to clean up when it closes.
[1] https://searchfox.org/mozilla-central/rev/35245411b9e8a911fe3f5adb0632c3394f8b4ccb/dom/media/mediacontrol/MediaControlService.cpp#410-413
Differential Revision: https://phabricator.services.mozilla.com/D92115
As we've chosen another way for GeckoView implementation, so `SetControlledTabBrowsingContextId` is no longer needed.
Differential Revision: https://phabricator.services.mozilla.com/D92114
This avoids us risking an overflow when we convert encrypted media with
subsamples to AnnexB (since that conversion can grow the clear sizes of the
sample). See the test in the preceding patch for an example of how and why this
happens.
Differential Revision: https://phabricator.services.mozilla.com/D92300
Add test code to ensure AnnexB conversions behave as we expect. This adds some
coverage for non-encrypted conversions that we only tested with broader tests
until now. It also adds a test to ensure we don't overflow our subsample sizes
when dealing with encrypted media with very large subsamples. This latter test
covers the issue seen in bug 1618529.
Differential Revision: https://phabricator.services.mozilla.com/D92299
`mBlockOwnersWatermark` is introduced in bug 1366936 for telemetry `MEDIACACHE_BLOCKOWNERS_WATERMARK` but `MEDIACACHE_BLOCKOWNERS_WATERMARK` is removed in bug 1356046
Depends on D92106
Differential Revision: https://phabricator.services.mozilla.com/D92107
`mIndexWatermark` was introduced in bug 1366929 for telemetry `MEDIACACHE_WATERMARK_KB` but `MEDIACACHE_WATERMARK_KB` was removed in bug 1356046
Depends on D92105
Differential Revision: https://phabricator.services.mozilla.com/D92106
alwu
Paul Adenot
Jean-Yves Avenard
Frédéric Wang
Lee Salzman
Dorel Luca
Ricky Stewart
Matthew Gregan
smolnar
Andreas Pehrson
Nils Ohlmeier [:drno]
Dan Glastonbury
Razvan Maries
Dan Minor
Mihai Alexandru Michis
Cosmin Sabou
Narcis Beleuzu
Bryce Seager van Dyk
Byron Campen [:bwc]
Matt Woodrow
Bogdan Tara
Aaron Klotz
Kartikaya Gupta
Chun-Min Chang
Jon Bauman
Nico Grunbaum
Kershaw Chang
Noemi Erli