Also we drop support for an independent-of-scroll/viewport capture, which
the old Tab Sharing supported, for security reasons (and we don't need it).
Differential Revision: https://phabricator.services.mozilla.com/D80974
Also we drop support for an independent-of-scroll/viewport capture, which
the old Tab Sharing supported, for security reasons (and we don't need it).
Differential Revision: https://phabricator.services.mozilla.com/D80974
Sync dispatch are being used; due to how the background taskqueue is setup this could cause a hang if if also called from a background taskqueue.
Differential Revision: https://phabricator.services.mozilla.com/D82500
Also we drop support for an independent-of-scroll/viewport capture, which
the old Tab Sharing supported, for security reasons (and we don't need it).
Differential Revision: https://phabricator.services.mozilla.com/D80974
Also we drop support for an independent-of-scroll/viewport capture, which
the old Tab Sharing supported, for security reasons (and we don't need it).
Differential Revision: https://phabricator.services.mozilla.com/D80974
The MediaStreamRenderer handles the AudioOutput of an HTMLMediaElement. it register/unregister the AudioOutput according to the rendering status. The sink-change follows the logic of AudioOutput thus it has been moved in it.
The previous way created an error when the element had been muted and the sink had been changed before the source MediaStream was attached to the element. The error occured because it was possible, the same entry to be registered more than once in the AudioOutput's list, which resulted in the track to be unmuted when it should not.
Differential Revision: https://phabricator.services.mozilla.com/D82529
The DISPATCH_QUEUE_PRIORITY_DEFAULT queue currently in use starts tasks as they
arrive rather than running tasks sequentially. By using a dedicated serial queue,
frames will be encoded and delivered in the order in which they are added to the
queue. It also makes it possible to post a task to the queue when video capture
is stopped, which can guarantee that all frames have been delivered before stop
completes. This should fix a crash we're seeing in DeliverCapturedFrame which
appears to be caused by racing between the main thread in Stop() and the capture
thread in captureOutput. The new queue is set to target the
DISPATCH_QUEUE_PRIORITY_DEFAULT, so tasks will still run there. This should give
us serial execution without the overhead of starting a new thread to manage our
queue.
The new upstream implementation creates a serial queue rather than using
DISPATCH_QUEUE_PRIORITY_DEFAULT. It also raises the priority of the queue above
default, but I think that should wait for a separate bug.
Differential Revision: https://phabricator.services.mozilla.com/D82721
Rename PostTask methods into Dispatch to be more in-line with the current naming convention as PostTask pretty much always referred to the MessageLoop's method.
Differential Revision: https://phabricator.services.mozilla.com/D82159
On Windows the CubebDeviceEnumerator needs to run on an MTA thread; to achieve that goal we use the EnsureMTA utility.
EnsureMTA relies on a MTA thread to exist which will be destroyed when XPCOM shuts down ; as such we must shutdown the CubebDeviceEnumerator during the same shutdown phase.
ClearOnShutdown registrars are processed in LIFO order, so as the CubebDeviceEnumerator constructor uses EnsureMTA we are guaranteed that the CubebDeviceEnumerator clearOnShutdown will be processed before the EnsureMTA thread shutdown.
The CubebDeviceEnumerator needs to be shutdown before cubeb; which is also guaranteed as the cubeb instance is destroyed during the final XPCOM shutdown stage.
Fix GetSafety() thread-safety.
CubebDeviceEnumerator should be a static non-refcounted singleton on the stack really which would made the code much simpler, unfortunately its gtests rely on having the Shutdown method present to force re-scanning the devices and work with the mock cubeb.
Differential Revision: https://phabricator.services.mozilla.com/D82158
Ths helps not having to worry about how to create the thread; which could be probablematic when running off a thread pool.
Differential Revision: https://phabricator.services.mozilla.com/D82142
Sync dispatch are being used; due to how the background taskqueue is setup this could cause a hang if if also called from a background taskqueue.
Differential Revision: https://phabricator.services.mozilla.com/D82500
Sync dispatch are being used; due to how the background taskqueue is setup this could cause a hang if if also called from a background taskqueue.
Differential Revision: https://phabricator.services.mozilla.com/D82500
We can have different autoplay permission setting for specific sites, and those setting can override the global autoplay setting.
Eg. Global permission setting is `Block Audio & Video` but site A is being set to `Block Audio Only`, then when inaudible video wants to start on siteA, it would be allowed to autoplay.
Returning `DENY_ACTION` is like specifically modifying the site permission setting, but in those cases, we should return `UNKNOWN_ACTION` because we are actually not able to check the site permission.
Differential Revision: https://phabricator.services.mozilla.com/D82570
Rename PostTask methods into Dispatch to be more in-line with the current naming convention as PostTask pretty much always referred to the MessageLoop's method.
Differential Revision: https://phabricator.services.mozilla.com/D82159
On Windows the CubebDeviceEnumerator needs to run on an MTA thread; to achieve that goal we use the EnsureMTA utility.
EnsureMTA relies on a MTA thread to exist which will be destroyed when XPCOM shuts down ; as such we must shutdown the CubebDeviceEnumerator during the same shutdown phase.
ClearOnShutdown registrars are processed in LIFO order, so as the CubebDeviceEnumerator constructor uses EnsureMTA we are guaranteed that the CubebDeviceEnumerator clearOnShutdown will be processed before the EnsureMTA thread shutdown.
The CubebDeviceEnumerator needs to be shutdown before cubeb; which is also guaranteed as the cubeb instance is destroyed during the final XPCOM shutdown stage.
Fix GetSafety() thread-safety.
CubebDeviceEnumerator should be a static non-refcounted singleton on the stack really which would made the code much simpler, unfortunately its gtests rely on having the Shutdown method present to force re-scanning the devices and work with the mock cubeb.
Differential Revision: https://phabricator.services.mozilla.com/D82158
Ths helps not having to worry about how to create the thread; which could be probablematic when running off a thread pool.
Differential Revision: https://phabricator.services.mozilla.com/D82142
Sync dispatch are being used; due to how the background taskqueue is setup this could cause a hang if if also called from a background taskqueue.
Differential Revision: https://phabricator.services.mozilla.com/D82500
While the duration returned by the demuxer is typically equal to the decoded size; recent cases have shown that this isn't always the case.
Now, the only way you can encounter this bug is if the user has set some pref. the audio BlankDecoderModule is never used by default; only the video one.
Differential Revision: https://phabricator.services.mozilla.com/D82138
In all those cases, the current nsISerialEventTarget is either the main thread or the MessageChannel's nsISerialEventTarget (since bug 1634846)
Differential Revision: https://phabricator.services.mozilla.com/D81966
What this patch do are
- add `onpositionstatechange` event handler on MediaController
- `PositionStateEvent` would be sent to `positionstatechange` event handler
The advantage of doing so is
- to allow us to listen to the position change on the media controller interface (that can be used for testing and the future plan, the media hub)
Differential Revision: https://phabricator.services.mozilla.com/D80791
What this patch do are
- propagate the position state change from the media session
The advantage of doing so is
- to allow us to notify this change to `MediaController` and eventually would notify that to `MediaControlKeySource`
Differential Revision: https://phabricator.services.mozilla.com/D80790
What this patch do are
- to add `SetPositionState()` on `MediaControlKeySource`
- notify `MediaControlKeySource` when the main controller's position state changes
The advantage of doing so is
- to allow `MediaControlKeySource` to know the position state change so that it can update the display content (eg. showing the duration or the progress bar of playback)
Differential Revision: https://phabricator.services.mozilla.com/D80789
This fix was required for subsequent patches to build successfully, because adding new source files exposed missing dependencies in unified sources.
Differential Revision: https://phabricator.services.mozilla.com/D75290
In order to implement VA-API playback we need a VADisplay connection.
According to preference load libva-wayland.so.2 or libva-drm.so.2 libraries
and dlsym vaGetDisplayWl or vaGetDisplayDRM to get VADisplay for VA-API decoding.
Differential Revision: https://phabricator.services.mozilla.com/D81967
There's a small race that can happen when the remote decoder gets shutdown during xpcom shutdown; that would cause GetCurrentSerialEventTarget to return null. Leading to an assertion failure in ActorLifecycleProxy thread-safety check when PRemoteDecoderManagerParent gets destroyed.
So we use a background taskqueue instead and cleanup a bit the threading code in there allowed thanks to the TaskQueue ability to not require an explicit shutdown.
Differential Revision: https://phabricator.services.mozilla.com/D81287
We forgot to implement those two methods on `MediaControlKeyManager`, which would result in not able to propagate those update to the real key source that handle the platform level APIs, such as MPRIS, SMTC.
Differential Revision: https://phabricator.services.mozilla.com/D81882
On GeckoView, when we update the property on the key source, such as the playback state, metadata. We want to know which tab that those properties change belong to.
Therefore, we create a method `SetControlledTabBrowsingContextId()` to tell the key source which tab is being controlled and all properties update should belong to that tab.
Differential Revision: https://phabricator.services.mozilla.com/D81881
In AudioResampler the resampling of a buffer happens on pieces of 128 or 256 frames. Before resampling, the number of available frames is checked for the whole buffer and for the individual smaller pieces. When the whole buffer is checked we need to check for one extra frame in order to compensate the case that one individual resampling has consumed an extra frame, something that is normal and happens periodically.
Differential Revision: https://phabricator.services.mozilla.com/D81321
There's a small race that can happen when the remote decoder gets shutdown during xpcom shutdown; that would cause GetCurrentSerialEventTarget to return null. Leading to an assertion failure in ActorLifecycleProxy thread-safety check when PRemoteDecoderManagerParent gets destroyed.
So we use a background taskqueue instead and cleanup a bit the threading code in there allowed thanks to the TaskQueue ability to not require an explicit shutdown.
Differential Revision: https://phabricator.services.mozilla.com/D81287
Currently that thread is always the main thread; but really it doesn't have to be.
We make this use a generic nsISerialEventTarget and rename some members to better reflect what thread is doing what.
Differential Revision: https://phabricator.services.mozilla.com/D81079
The error indicates that test timeouts because there is no audio in the output. As a workaround the the test will wait for audio in the input and then will start recording and checking the output.
Differential Revision: https://phabricator.services.mozilla.com/D81116
This patch would
- introduce new methods `SetEnableFullScreen()` and `SetEnablePictureInPictureMode()` on `MediaControlKeySource`
- notify the change of enabling/disabling the fullscreen and picture-in-picture mode to `MediaControlKeySource`
The advantage of doing this is
- to allow the key source to do corresponding operations when media enters/leaves the fullscreen/picture-in-picture
eg. GeckoView would use them to implement its API, `onFullscreen()` and `onPictureInPicture()`
Differential Revision: https://phabricator.services.mozilla.com/D79785
This patch would
- active the controller when it enters fullscreen or picture-in-picture mode
The advantage of doing this is
- allow to control media even if media doesn't start
Differential Revision: https://phabricator.services.mozilla.com/D79766
This patch would
- notify media controller when media enters/leaves fullscreen
The advantage of doing this is
- prework of being able to control media when media enters fullscreen
Differential Revision: https://phabricator.services.mozilla.com/D79765
This patch would
- stop notifying media key to media in content process is the controller is inactive
The advantage of doing this is
- to prevent unexpectedly controlling an inactive controller
Differential Revision: https://phabricator.services.mozilla.com/D79235
This patch would
- move the mechanism of creating a stop timer from media element to media controller
The advantage of doing this is
- to remove redundant timers in the content process, because we only need to maintain one global timer for a whole tab
Differential Revision: https://phabricator.services.mozilla.com/D79233
If XPCOM has been shutdown, then we should not access the `BrowsingContext::Get()` because `sBrowsingContexts` has also been destroyed.
Differential Revision: https://phabricator.services.mozilla.com/D80559
This patch changes the type of mAudioThreadId because `std:🧵:id` is
non-memovable and AudioThreadRegistry uses an nsTArray, that needs elements to
be memovable.
Differential Revision: https://phabricator.services.mozilla.com/D80466
This keeps track of how many users of an audio thread there is, and takes care
of registring or unregistring those threads to the Gecko Profiler.
Differential Revision: https://phabricator.services.mozilla.com/D80464
All changes in these files are a result of code formatting being applied.
When these files were under webrtc, they were treated as third party code and
not formatting. By moving them, formatting is now applied.
Differential Revision: https://phabricator.services.mozilla.com/D80544
Before P1, GetCurrentThreadSerialEventTarget would have always returned the same data as NS_GetCurrentThread, making the comment incorrect Now it will properly return the running TaskQueue if any.
This change of name more clearly exposes what they are doing, as we aren't always dealing with threads directly; but a nsISerialEventTarget
Differential Revision: https://phabricator.services.mozilla.com/D80354
When the default sink (sink-id equal to an empty string) has been requested, null device-id is returned. The device-id is propagated all the way down to cubeb. When cubeb is configured with null device-id the default device is chosen. In addition to that, on default device change the new default will be followed. This aligns with the expected behavior for the default sink.
Differential Revision: https://phabricator.services.mozilla.com/D77810
A new method has been added in AudioStreamTrack to allow the change of the output device. Also, the methods that add/remove the AudioOutput or Volume have been enhanced to use the CrossGraphManager, when available, in order to set the AudioOutput or volume to the correct MediaTrack.
Differential Revision: https://phabricator.services.mozilla.com/D77808
The name of the two new tracks is CrossGraphTransmitter and CrossGraphReceiver. They are used together to transfer the audio data of the transmitter to the receiver which belongs to different MTG. In addition to that a CrossGraphManager class has been created that creates the connection between the transmitter and the receiver and can redirect to the correct track some operations like the volume change etc.
Differential Revision: https://phabricator.services.mozilla.com/D77807
Prepare ffmpeg decoding module to X11 dmabuf implementation. In order to cover both X11 and Wayland backend do:
- Use DMABUFSurfaceImage instead of WaylandDMABUFSurfaceImage
- Use DMABufSurface instead of WaylandDMABufSurface
- Rename former DMABufSurface to DMABufSurfaceWrapper to state it actually wraps the DMABufSurface.
- Use DMABufSurfaceYUV instead of WaylandDMABufSurfaceNV12
Differential Revision: https://phabricator.services.mozilla.com/D79638
The FetchImageHelp doesn't need to decode the fetched image before
handing the image to its caller since the decoded raw data won't be used
in the caller. The ImagePromise can be resolved upon image is fetched
(ready) instead.
Differential Revision: https://phabricator.services.mozilla.com/D79736
This patch does the following things:
1. Use `FetchImageHelper` to fetch the MediaImage defined in
media-session
2. Upon the above image is fetched, set it to the SMTC's thumbnail
Differential Revision: https://phabricator.services.mozilla.com/D77893
Still can not figure out how did we enable the same action twice, because we would only update the action change when the action handler is set from `null` to `something` or from `something` to `nothing` [1]. In addition, the browsing context Id for a media session is always unchanged, so it's guarantee to set the action on the correct `MediaSessionInfo`.
However, in order to avoid having more crashes, I have to change the assertions to early returns and add logs to help us be aware of this issue if it happens again.
[1] https://searchfox.org/mozilla-central/rev/fac90408bcf52ca88a3dcd2ef30a379b68ab24e2/dom/media/mediasession/MediaSession.cpp#71-75
Differential Revision: https://phabricator.services.mozilla.com/D78947
- Use WaylandDMABufSurface for SW decoded frames when they can be converted to NV12 format.
- Rename VAAPIFrameHolder to DMABufSurface and use it as general placeholder for WaylandDMABufSurface surface.
It's used for VA-API/SW video playback and holds decoded video images until thery are used by gecko rendering engine.
- Implmenet a linked list of DMABufSurface where recently used frames are stored. The frames are recycled by ffmpeg
decoder for VA-API and SW video playback.
Differential Revision: https://phabricator.services.mozilla.com/D78292
The AAC MFT has a latency of one packet (1024 frames); when we drain the decoder to retrieve that last data, it sets the the decoded data start time to the time of the end of the previous data.
this is the right thing to do under most cases; however when dealing with content with broken timestamps it can lead to bad A/V sync.
Differential Revision: https://phabricator.services.mozilla.com/D79456
- Use WaylandDMABufSurface for SW decoded frames when they can be converted to NV12 format.
- Rename VAAPIFrameHolder to DMABufSurface and use it as general placeholder for WaylandDMABufSurface surface.
It's used for VA-API/SW video playback and holds decoded video images until thery are used by gecko rendering engine.
- Implmenet a linked list of DMABufSurface where recently used frames are stored. The frames are recycled by ffmpeg
decoder for VA-API and SW video playback.
Differential Revision: https://phabricator.services.mozilla.com/D78292
The allocation still needs to be a power of two so the ergonomics are not
amazing, but this will do until we replace with a buffer-based MPSC ringbuffer.
Differential Revision: https://phabricator.services.mozilla.com/D78499
and re-enable OnStateChanged() transition assertions.
Rejecting the promise avoids having AudioContextOperation promises resolve out
of order when AudioContextOperationControlMessage::RunDuringShutdown()
resolves a Close operation shortly after Run() for a previous operation has
queued its update via mUpdateRunnables. mUpdateRunnables are run after
controlMessagesToRunDuringShutdown.
https://searchfox.org/mozilla-central/rev/ea7f70dac1c5fd18400f6d2a92679777d4b21492/dom/media/MediaTrackGraph.cpp#1793,1803
AudioContext::Shutdown() takes care of rejecting JS Promises when the window
is being destroyed.
Differential Revision: https://phabricator.services.mozilla.com/D79055
The promises are rejected before shutting down threads so that they don't need
to wait for nested event loops to exit.
Differential Revision: https://phabricator.services.mozilla.com/D76807
The key change here is that AudioContextOperation promises for the same
AudioContext are resolved in order as long as the graph is running.
There is one remaining case where AudioContextOperation promises can be
resolved out of order, which is when
AudioContextOperationControlMessage::RunDuringShutdown() resolves a Close
operation shortly after Run() has queued its update via mUpdateRunnables.
mUpdateRunnables are run after controlMessagesToRunDuringShutdown.
https://searchfox.org/mozilla-central/rev/ea7f70dac1c5fd18400f6d2a92679777d4b21492/dom/media/MediaTrackGraph.cpp#1793,1803
Pending resume operations are now stored on the graph instead of on
mNextDriver. This means there is no need to move operations between drivers
when switching from one mNextDriver to another, the code for which was
previously missing.
Suspend operations are performed immediately, because even a callback driver
can render nothing. Tracks are not resumed until either there is an
AudioCallbackDriver to deliver the rendered audio or the Resume operation is
canceled by a subsequent Suspend.
While the graph is running, DispatchToMainThreadStableState() is used
consistently, whereas previously this was mixed with direct Dispatch() to the
main thread, which could have the runnable Run() before main thread state had
been updated for rendering up to the time of a Suspend.
Differential Revision: https://phabricator.services.mozilla.com/D76806
This provides that a new track does not start processing before existing
tracks in the AudioContext, which may not be resumed until an
AudioCallbackDriver is running.
Depends on D79048
Differential Revision: https://phabricator.services.mozilla.com/D79049
This will be helpful when delaying resumption of tracks until there is an AudioCallbackDriver.
The memory for the array is also transferred to the callee to save an allocation.
Differential Revision: https://phabricator.services.mozilla.com/D76801
Because we store each plane's data pointer we don't need to store a separate
offset, we can just roll that offset into the pointer for that plane.
This helps remove a lot of code where we're not using the offset anyway, and in
the Chromium CDM case we simply roll the offset into our data pointer.
Differential Revision: https://phabricator.services.mozilla.com/D78320
This patch will
- remove `MediaControlKeysEvent` and use `MediaControlKey` to replace it
- rename names for all `MediaControlKey` related methods, functions, classes and descriptions
The advantage of doing so are
- remove the duplicated type so that we only need to maintain `MediaControlKey`
Differential Revision: https://phabricator.services.mozilla.com/D78140
This patch will
- create a chrome-only webdil interface `MediaController`
- expose supported keys via `MediaController` webidl interface
The advantage of doing so are
- to have a dedicated interface that is only used for MediaController that can be used for testing and our future plan (media hub)
More Details :
Currently, we access media controller's from `ChromeUtils` [1], but it causes a problem of creating a duplicated enum of the enum which we want to expose into Chrome JS.
Instead, we should create a media controller interface to access all its attibutes, which is more easier and clean.
In addition, we're planning to have a something like Chrome's media hub [2]. In order to do that, we have to expose some JS methods to allow us to control playback directly from Chrome JS.
[1] https://searchfox.org/mozilla-central/rev/559b25eb41c1cbffcb90a34e008b8288312fcd25/dom/chrome-webidl/ChromeUtils.webidl#485-493
[2] https://blog.google/products/chrome/manage-audio-and-video-in-chrome/
Differential Revision: https://phabricator.services.mozilla.com/D77757
This patch will
- prevent dispatching metadata change event for non-active media session (we only care about the event from the active media session)
The advantage of doing so are
- reduce some unnecessary code running
Differential Revision: https://phabricator.services.mozilla.com/D77756
This patch will
- add a method `SetSupportedMediaKeys()` on `MediaControlKeysEventSource`
- set main controller's supported key to the event source
The advantage of doing so are
- to allow the event source knowing which key is supported in order to determine the displayed UI button
Differential Revision: https://phabricator.services.mozilla.com/D77200
This patch will
- tell the media controll supported action changes when media session updates its action handler
The advantage of doing so are
- to sync the status between media session in content process and the `MediaSessionInfo` in chrome process
Differential Revision: https://phabricator.services.mozilla.com/D77199
This patch will
- add methods to allow media controller to enable/disable action for specific media session
The advantage of doing so are
- to know the supported actions for each media session
Differential Revision: https://phabricator.services.mozilla.com/D77198
This patch will
- `ContentMediaAgent` as the only class to propagate the information to chrome process
The advantage of doing so are
- to group all methods involving IPC in one place, that would be easier to see what information would be propagated to chrome process
Differential Revision: https://phabricator.services.mozilla.com/D77197
This patch will
- make `ContentMediaAgent` inherit from `IMediaInfoUpdater`
- move `MediaPlaybackState` and `MediaAudibleState` to `MediaPlaybackStatus.h`
The advantage of doing so are
- to force all methods which are related with updating information from content process to chrome process to be manged by `IMediaInfoUpdater`. It can help us only put the descriptive comment for methods on one place. (on `IMediaInfoUpdater`)
Differential Revision: https://phabricator.services.mozilla.com/D77196
The CDM header bump has moved some enums, as well as using enum classes instead
of old style enums. This patch updates consumers of these enums to be compatible
with the new headers.
Drive by remove `using namespace cdm` from a couple of files as
- In some places I'd already been using fully qualified names.
- I prefer the fully qualified names as they make it clear when enums are coming
from the cdm namespace and `cdm::` is not a particularly more verbose thing to
have on identifiers.
Differential Revision: https://phabricator.services.mozilla.com/D78343
With RTX and transport-cc enabled, it is possible to send padding packets prior
to any media being sent (see comments in RTPSender::SendPadData). But padding
bytes are counted separate from payload bytes and so do not show up in
bytesSent, which means that is possible that we've sent packets, but not bytes,
as far as the stats are concerned. This leads to frequent intermittent failures
while running tests on Android hardware.
Depends on D77815
Differential Revision: https://phabricator.services.mozilla.com/D78287
Randell Jesup
Bogdan Tara
alwu
Narcis Beleuzu
Andrea Marchesini
Mihai Alexandru Michis
Jean-Yves Avenard
Csoregi Natalia
Tom Prince
Alex Chronopoulos
Razvan Maries
Dan Minor
Paul Adenot
Sylvestre Ledru
Corentin Arnould
Subhamoy Sengupta
Martin Stransky
David Major
Simon Giesecke
Nico Grunbaum
Junior Hsu
Emilio Cobos Álvarez
Coroiu Cristina
Jan-Ivar Bruaroey
Butkovits Atila
Chris Fronk
Karl Tomlinson
Noemi Erli
Andrew McCreight
Danny
tkhan
Chun-Min Chang
Cosmin Sabou
Nicholas Nethercote
Bryce Seager van Dyk