For an AudioCallbackDriver, the number of input channels is immutable, and
passed at construction, so that it's less necessary to rely on global state.
MozReview-Commit-ID: F9TL1H92z3W
--HG--
extra : rebase_source : 5193488592ca97273eb2b6f43d4c7a0e4beb0a33
The MSG now can feed microphone data to all its input listeners. This paves the
way for multiple input device, if we feel it's needed at some point, but does
not implement it.
The method for adding/removing inputs are also cleaned up.
MozReview-Commit-ID: 9OX4Da6Gjq2
--HG--
extra : rebase_source : 043c486e53f9220ae61fd788ed86064ba723f1a4
Add new log module which allow us to debug by using "MOZ_LOG=Autoplay:5".
MozReview-Commit-ID: 9CG5JyCw21G
--HG--
extra : rebase_source : c71c4bbed88b07a7803d93b661bfeac37cb94035
The logic here intends to (as is written in the comment) append one block of
silence to the track to allow for us to underrun one full scratch buffer.
The code doesn't match this behavior however, because if we are not underrunning
by less than a block, we end up appending *less* than a block. This causes us to
append at a later time as the scratch buffer can swallow more (up to a full
block) than we appended.
Without processing this seems to work because of timing and ordering, but
with processing (aec/agc/ns) we tend to add 71
(512 for an iteration - 441 packed) samples of silence,
leaving us to hit the assert with a 44% ((128-71)/128) chance during subsequent
iterations.
Differential Revision: https://phabricator.services.mozilla.com/D2644
--HG--
extra : moz-landing-system : lando
This results in a speed improvement of about 6% on the "Convolution reverb"
webaudio-benchmark (measured with Linux x86_64 build on Skylake).
MozReview-Commit-ID: 94W6h3qE1tB
--HG--
extra : rebase_source : f6fe4f3fa0338a90a1e389450134027d9389af09
Various web authors have expressed desire to know in advance whether autoplay
will work.
They want this in order to avoid paying the price for downloading media that
won't play. Or they want to take other action such as showing a poster image
instead.
This is of particular interest to Firefox, as we're planning on showing a
prompt to ask the user whether they would like a site to play. If sites want to
determine whether they can autoplay but avoid the prompt showing, they won't be
able to just call play() in Firefox and see whether it works, as that would
likely show the prompt if the user doesn't already have a stored permission.
We've been working out a spec here:
https://github.com/whatwg/html/issues/3617#issuecomment-398613484
This implements what is the consensus to date there;
HTMLMediaElement.allowedToPlay, which returns true when a play() call would not
be blocked with NotAllowedError by autoplay blocking policies.
MozReview-Commit-ID: AkBu0G7uCJ0
--HG--
extra : rebase_source : 3f31db79aa1e570fdd9fc7062d0ddac7c96a8931
In present web extension design (both Firefox and Chrome), background script can autoplay.
We don't want to break this design, so we would always allow it to autoplay.
MozReview-Commit-ID: 9BfWgll7PNx
--HG--
extra : rebase_source : 7996daa06c31a84a2aea9008daa9bddfcde98c74
When a transceiver is stopped, its mid should not be reused by a new transceiver.
Differential Revision: https://phabricator.services.mozilla.com/D2518
--HG--
extra : moz-landing-system : lando
Presumably the Rust portion of this will have to land externally first and
then be imported, but I have no idea how or where to submit it.
MozReview-Commit-ID: 2gzQbRKxaZ9
--HG--
extra : rebase_source : 582e41200e69ff3722585c7664ddd122eb0de2fe
extra : intermediate-source : e0a021b27d2c66d46ba973d66d1360678411da26
extra : source : 6d18a8bd5ee351da1a0cdfaa63f49706a2f95ba3
Presumably the Rust portion of this will have to land externally first and
then be imported, but I have no idea how or where to submit it.
MozReview-Commit-ID: 2gzQbRKxaZ9
--HG--
extra : source : 6d18a8bd5ee351da1a0cdfaa63f49706a2f95ba3
extra : histedit_source : aa7995595e2699d53f1dc60410b90cfd0d4a5c4e
Presumably the Rust portion of this will have to land externally first and
then be imported, but I have no idea how or where to submit it.
MozReview-Commit-ID: 2gzQbRKxaZ9
--HG--
extra : rebase_source : 05924114b7ff1c48ce0c4584469b3b2ef0bc26cb
Allow autoplay for video document when user is directly viewing a video/audio by visiting
its url.
MozReview-Commit-ID: GnSxx32h6hm
--HG--
extra : rebase_source : 0890f7f3e80711b35c65a6f32d7ce960de76d20e
We'd like to add telemetry to help inform the decision as to how enabling
block autoplay will affect video playback in the wild.
Our data science team would also like some input to help them estimate the
rate at which our shield study would receive pings, and the telemetry
collected here will help them estimate that.
We'd like to collect the following, on a per session basis:
* Count of the number of top level content documents loaded, as denominator for
other stats collected here.
* Count of the number of top level content documents which contained (directly
or in a descendant document) playback of an audible media element.
* Count of the number of top level content documents which contained (directly
or in a descendant document) a muted media element that was paused by the
autoplay policy because it tried to unmute and it wasn't allowed to autoplay
audibly.
* Count of the total number of audible autoplay videos that would have not been
allowed to play if block autoplay was enabled. We'd either prompt for
permission on these videos, or block outright depending on user's settings.
* Count of the total number of audible autoplay videos that would have been
allowed to play if block autoplay was enabled.
MozReview-Commit-ID: vHWJPyqHjT
--HG--
extra : rebase_source : e1f22ec83fda8b65d78f1de9f02cf060d424019c
MediaRawData is never subclassed, so it's pointless to have these
methods be virtual.
--HG--
extra : rebase_source : ef55196831f5ff96e75b46f304541e1cb8fafada
extra : source : ccef9e2643a19668dc06426aa0e55ed5d61ae535
AV1 support is behind a pref, as such, the result of canPlayType should depends on the value of that pref.
Additionally to this change we remove AOMDecoder::IsSupportedCodec as it implied confusion on what a codec mimetype is. There are two type of codec mimetype: the one describing the container content ("av1") and the one describing the codec itself "video/av1")
AOMDecoder shouldn't know anything about containers (e.g. mp4 or webm)
--HG--
extra : rebase_source : 72ebe662f76fb6c9d8be1f753890601aac440061
AV1 support is behind a pref, as such, the result of canPlayType should depends on the value of that pref.
Additionally to this change we remove AOMDecoder::IsSupportedCodec as it implied confusion on what a codec mimetype is. There are two type of codec mimetype: the one describing the container content ("av1") and the one describing the codec itself "video/av1")
AOMDecoder shouldn't know anything about containers (e.g. mp4 or webm)
Summary:
We've hit stack overflows while decoding, in particular for av1. This increases
the thread size for the platform decoder threads, while leaving the others at
their default values.
Reviewers: jya
Tags: #secure-revision
Bug #: 1474684
Differential Revision: https://phabricator.services.mozilla.com/D2226
--HG--
extra : rebase_source : d03a91e93910fac5458f18f43398d76b736bbee6
Pending figuring out how we want to block autoplay of WebAudio content, we
should just not block it by default for the initial release of block autoplay,
and follow up once we've figured out how to not break the web.
MozReview-Commit-ID: ClfdrHcugLs
--HG--
extra : rebase_source : 54f61b0765f1d0ed9c754c90da9c2809a7de8676
The PeerConnection mochitest wrapper had checks to make sure that the ssrc was reported as a string.
It is now an unsigned long and those tests needed to reflect that.
MozReview-Commit-ID: b7TzzG0I4J
--HG--
extra : rebase_source : 8c1665b1c8d8e2f0418a0be70d37c8ee10507c35
Both for mochitest (simply an expectation adjustment), and in mochitest (align
with the code).
MozReview-Commit-ID: 2UIq4zrcd02
--HG--
extra : rebase_source : 1858e73b3ad89aade3219ee8556c653c55c85bed
The front end code can't always guarantee to give us an allow/cancel response
to a permission request. In particular in these cases:
* if we close a tab while showing a doorhanger, or
* if we navigate a tab while showing a doorhanger, or
* if the permission prompt requested in a background tab and never shown.
Handling all of these cases is problematic; we don't get events for all of
these where it's easy and cheap to determine that we should cancel the
permission request.
Canceling the permission request is important in the autoplay-media permission
request case as there's objects waiting on the resolution of the permission
request, and they leak in ASan builds while running chrome tests if the Gecko
size of the permission request doesn't get a notification telling it to stop
waiting.
But we can however rely on the doorhanger code to drop its reference to the
nsIContentPermissionRequest object that we pass to it when the doorhanger goes
away. So we can cancel the permission request in our
nsIContentPermissionRequest's implementation's destructor in order to easily
catch all the above cases.
In order to do that, we need to split AutoplayRequest into two; one part being
the implementation of nsIContentPermissionRequest (AutoplayPermissionRequest),
and the other part being the code to own the PromiseHolder and manage the
permission request (AutoplayPermissionManager).
AutoplayPermissionRequest keeps a weak reference to AutoplayPermissionManager,
so that it can tell the AutoplayPermissionManager to reject the request promise
when it's destroyed.
This fixes the ASan leak for which I got backed out from earlier in this bug,
and also fixes the cases above.
MozReview-Commit-ID: KoVkgIqDleW
--HG--
rename : dom/html/AutoplayRequest.cpp => dom/html/AutoplayPermissionManager.cpp
rename : dom/html/AutoplayRequest.h => dom/html/AutoplayPermissionManager.h
extra : rebase_source : dbca520a93d8c416f6d64c2da027630181bb5910
This cannot happen anymore, because we're using sequence<float> and not
Float32Array in WebIDL, and sequence<float> throws when it contains Infinity or
NaN.
MozReview-Commit-ID: 9ZUbXa0viSk
--HG--
extra : rebase_source : e1783b6873caeefa1f09caf938e342f591da0056
extra : source : 5d0cce417e56935badf1e7a503f348079c8a9435
Link to the standard: https://webaudio.github.io/web-audio-api/#dom-audioparam-setvaluecurveattime
MozReview-Commit-ID: 8GwaIbQkfr2
--HG--
extra : rebase_source : aa8dd5e653de51768ff81d855fe1b8b398baa586
extra : intermediate-source : 9d34c85e0ec166fb7a117b2a85ca7cd4e98b1ceb
extra : source : d752fc72a9a35fdc0ce7b8bce94b29149eaf7639
A more involved test exists as a web-platform-tests, but we can't run it because
it makes use of AudioListener AudioParam, that we don't have right now.
MozReview-Commit-ID: 8QJ12cGVRbQ
--HG--
extra : rebase_source : 6161e33d7d8ef83eb5e16a4570a770401dd672cd
In the panning formula, one of the channels is always left untouched by the
panning gain, so the current setup didn't work: it would not apply the gain to
one of the channels.
MozReview-Commit-ID: LjrTlTT2z9r
--HG--
extra : rebase_source : 29aabddc7caf16427330687acbab91f9c3047d32
In bug 1404042, we determined that video playback of software based surface was very slow and power hungry when using the intel GPU, but was working okay when using the dGPU (either AMD or nVidia)
So always return false in IsVP9DecodeFast when the current GPU is an Intel, as well as return a benchmark score of 0
Force a new run of the benchmarking by bumping the version number.
Differential Revision: https://phabricator.services.mozilla.com/D2075
--HG--
extra : moz-landing-system : lando
In bug 1404042, we determined that video playback of software based surface was very slow and power hungry when using the intel GPU, but was working okay when using the dGPU (either AMD or nVidia)
So always return false in IsVP9DecodeFast when the current GPU is an Intel, as well as return a benchmark score of 0
Force a new run of the benchmarking by bumping the version number.
Differential Revision: https://phabricator.services.mozilla.com/D2075
--HG--
extra : moz-landing-system : lando
nsGlobalWindowInner::GetExtantDoc() can potentially return null, and
we're not null checking its return value in
MediaManaager::IsActivelyCapturingOrHasAPermission() which I think is the
cause of this crash.
So instead of getting the principal from the window's extant doc, we can
get the principal directly from the nsGlobalWindowInner.
MozReview-Commit-ID: BUwiJGLss2a
--HG--
extra : rebase_source : 5135a2ef524044c44046705f4a7c7516f01f6b34
Summary:
Additionally, consider all videos <= 480p to be smooth and power efficient as:
1- Hardware decoding it typically not used for those
2- We can't do any better
3- Any machines should be able to do 480p
Depends on D1794
Reviewers: bryce
Tags: #secure-revision
Bug #: 1409664
Differential Revision: https://phabricator.services.mozilla.com/D1796
Summary:
MediaCapabilities provide a finer detail on VP9 being supported or not. YouTube will use that information to determine which resolutions to support when using VP9
Depends on D1772
Reviewers: bryce
Tags: #secure-revision
Bug #: 1409664
Differential Revision: https://phabricator.services.mozilla.com/D1794
Summary:
If the benchmark task hasn't run yet, we will assume smoothness for now.
Depends on D1771
Reviewers: bryce
Tags: #secure-revision
Bug #: 1409664
Differential Revision: https://phabricator.services.mozilla.com/D1772
Summary:
This will allow to be called from the MediaCapabilities taskqueue if we find that a decoder won't be hardware accelerated.
It is still assumed that Benchmark::Init() was called at least once on the main thread.
Depends on D1628
Tags: #secure-revision
Differential Revision: https://phabricator.services.mozilla.com/D1767
Summary:
The Apple VT decoder requires SPS+PPS at construction time. If not provided, in earlier macOS it used to give an error. In the current 10.13 it appears to work, however the decoder always report to be software only.
To properly determine the decoder capabilities, we construct a SPS NAL from the codec mimetype provided.
Details on the structure of the mimetype can be found in https://tools.ietf.org/html/rfc6381#section-3.3 and is a 1:1 match with the data found in the SPS.
Depends on D1718
Reviewers: bryce
Tags: #secure-revision
Bug #: 1409664
Differential Revision: https://phabricator.services.mozilla.com/D1719
Summary:
We'll need it to properly build a SPS/PPS extradata later. Also, change the types used. The original data is stored on two bytes ASCII, it will always fit in a uint8_t. Additionally, this is how those values are stored in a SPS.
Depends on D1678
Reviewers: bryce
Tags: #secure-revision
Bug #: 1409664
Differential Revision: https://phabricator.services.mozilla.com/D1718
Summary:
By default, when creating a H264 decoder it is wrapped into a H264Converter which will only create the actual decoder once a valid SPS/PPS has been seen.
As creating valid SPS/PPS NALs isn't trivial, when all we care about are capabilities of such decoder, we do not wrap the decoder so that it will be immediately created.
We can then test its capabilities.
We only enable this on windows, as on mac we need to generate a SPS/PPS, otherwise the mac decoder always report that HW decoding is not enabled.
Depends on D1632
Tags: #secure-revision
Bug #: 1409664
Differential Revision: https://phabricator.services.mozilla.com/D1633
Summary:
To properly determine if a decoder is hardware accelerated, we must pass information about the compositor to the decoder.
Depends on D1631
Tags: #secure-revision
Bug #: 1409664
Differential Revision: https://phabricator.services.mozilla.com/D1632
Summary:
For flac, mp3 and adts, if a codec was provided but wasn't supported in the container, it would have reported Maybe instead of No
Depends on D1628
Tags: #secure-revision
Bug #: 1409664
Differential Revision: https://phabricator.services.mozilla.com/D1629
Summary:
We now provides more detailed information for audio (check sampling rate and channels if provided).
And check for the power efficient attribute. We directly correlate this information with the decoder being hardware accelerated or not. All audio codecs are deemed to be power efficient.
Depends on D1626
Tags: #secure-revision
Bug #: 1409664
Differential Revision: https://phabricator.services.mozilla.com/D1627
Summary:
We can't create a H264 VT decoder until we have all SPS/PPS NALs, which makes it tricky to generate when we only want to check if H264 is supported.
On mac, we can reasonable assume that hardware acceleration is always supported (though on a mac pro 2013 that isn't the case or hackintosh with nvidia cards).
Depends on D1625
Tags: #secure-revision
Bug #: 1409664
Differential Revision: https://phabricator.services.mozilla.com/D1626
Summary:
We know those sampling rate aren't supported and cause initialization errors later.
Depends on D1624
Tags: #secure-revision
Bug #: 1409664
Differential Revision: https://phabricator.services.mozilla.com/D1625
Summary:
Allows to build an array ot TrackInfo built from the mimetype provided. This will allow to create dummy decoder to check that if they are supported and how well the decoder will perform.
Depends on D1623
Tags: #secure-revision
Bug #: 1409664
Differential Revision: https://phabricator.services.mozilla.com/D1624
Summary:
Addtionally, change the framerate to be of type double and allow to create a MediaExtendedMIMEType based on the new dom VideoConfiguration and AudioConfiguration object.
Depends on D1622
Tags: #secure-revision
Bug #: 1409664
Differential Revision: https://phabricator.services.mozilla.com/D1623
Summary:
TaskQueue no longer requires an explicit call to BeginShutdown() as such we no longer have a need for AutoTaskQueue.
Depends on D1621
Tags: #secure-revision
Bug #: 1409664
Differential Revision: https://phabricator.services.mozilla.com/D1622
Summary:
The information returned is identical to
1- canPlayType() for file
2- MediaSource.isTypeSupported for media-source
3- MediaRecorder.isTypeRecorder() for recordings.
Depends on D1616
Tags: #secure-revision
Bug #: 1409664
Differential Revision: https://phabricator.services.mozilla.com/D1617
Prior to the changes in this bug, a document would request autoplay permission
for its own origin, and not use the top level document's origin for the
permission check. Since now we use the top level document's origin for
requesting autoplay permission, some mochitests need to change, namely:
* test_autoplay_policy_permission.html can use
file_autoplay_policy_activation_frame.html directly.
The test was failing because its helper page was loaded same origin and the
helper page was testing if cross origin iframes could play; since we use the
top level document's permission request, this no longer effectively tests
whether the cross origin child can autoplay, as the cross origin child just
uses the top level window's origin for requests.
So we can instead load the helper window cross orgin instead, and remove one
helper layer.
Also an issue here is the way I was waiting for a new window to load was racy,
so now we wait for loading windows to send us a "ready" message.
* test_autoplay_policy_activation.html; this test's helper needs to wait for
loading windows to send it a "ready" message, as its helper is shared with the
above test.
MozReview-Commit-ID: LvRa4G7tqFw
--HG--
extra : rebase_source : 8ecd0e58200d79f0065a6d7b146d1d110d35953d
Add an implementation of nsIContentPermissionRequest to encapsulate requesting
permission from the user to autoplay audible media.
All documents in the tab request permission using the top level document's
origin, so the AutoplayRequest instance for a tab is stored on the top level
content window of the tab.
AutoplayRequest ensures that there's only a single prompt shown at once.
MozReview-Commit-ID: 2u3aLnEa21z
--HG--
extra : rebase_source : a3db3a1ef87a09442b47ae1b1034b4a0143289fb
extra : source : ae572bf618155a92b136fc5f12c1f8c9ab31f89c
Some MSE streams will resend the same init segment. In these cases we can treat
the following data as belonging to the already existing stream. We do this by
reusing the same stream id. This stops us tearing down and restarting decoders
in these cases. This has the benefit of stopping us introducing gaps to audio
and/or video.
Differential Revision: https://phabricator.services.mozilla.com/D1837
--HG--
extra : moz-landing-system : lando
Paul Adenot
alwu
Andreas Pehrson
Karl Tomlinson
Chris Pearce
Mark Banner
Byron Campen [:bwc]
Jan-Ivar Bruaroey
Kris Maglione
Jan de Mooij
Boris Zbarsky
Cosmin Sabou
Coroiu Cristina
Gurzau Raul
Brian Hackett
Nathan Froyd
Jean-Yves Avenard
Dan Minor
Tom Schuster
Noemi Erli
Alex Chronopoulos
Margareta Eliza Balazs
Nico Grunbaum
Andreas Pehrson
Dale Harvey
Bogdan Tara
Andrea Marchesini
Aaron Klotz
shindli
Arnaud Bienner
Bryce Van Dyk
sotaro
Jeff Gilbert
Emilio Cobos Álvarez