By using the soundtouch library, this patch implements the time stretching on the samples in AudioDecoderInputTrack when its playback rate is not 1.0f, in order to support changing playback rate on the captured media stream.
As the time stretcher has to be initialized by a fixed channel count, we would perform a realtime up-mix/down-mix for those audio chunks which have different channel count thant AudioDecoderInputTrack's initial channel count.
Differential Revision: https://phabricator.services.mozilla.com/D114560
Each AudioInputTrack has its own AudioSegments storing the input audio
data. When the AudioInputTrack is in pass-through mode, the AudioSegment
is just the data de-interleaved from its raw data, without any audio
processing magic applied on it. If there are multiple AudioInputTracks
in pass-through mode exist in the same graph, then all of their
AudioSegments are same.
Before this patch, each of these AudioInputTracks allocates its own
space to store its own AudioSegments even those data are same. This
patch makes it possible for these AudioInputTracks to share the same
AudioSegment data they need. By creating the AudioSegment in the
NativeInputTrack, which is mapped to one specific device and is
connected to the AudioInputTrack, the AudioInputTrack can fetch the
AudioSegment data when they need and then append shared-references of
the AudioChunk, inside the fetched AudioSegment, into their own
AudioSegment. Therefore, we can have some AudioChunks created by the
NativeInputTrack and shared among the AudioInputTracks in pass-through
mode.
Differential Revision: https://phabricator.services.mozilla.com/D114801
When an AudioCallbackDriver starts a fallback SystemClockDriver is running in
its stead. The AudioTrack getting fed data from the input stream of the driver
will get real data while the AudioCallbackDriver is running, and silence while
the fallback driver is running.
If the MediaStreamTrack representing the microphone source is part of a
MediaStream being played by an HTMLMediaElement; and another MediaStreamTrack
representing another source with a lower channel count than the microphone is
part of another MediaStream being played by another HTMLMediaElement; then:
1. We start the graph with the other source's output channel count, and a
fallback driver.
2. Once the audio driver has started, it adds data at a higher channel count
than the graph's to its MediaTrack. The driver switches audio driver to
match the new channel count.
3. The new driver starts with a fallback driver, which adds silence to the
track. Silence has no channel count, so the graph sees only the channel
count of the other source and switches audio driver to match this.
4. Go to 1.
This patch fixes makes us memoize a previously returned max channel count for an
AudioSegment for use when there is only null data (e.g., silence) present in the
segment. This applies to step 3 above, where no audio driver would be switched
because the graph still sees the mic's channel count.
Differential Revision: https://phabricator.services.mozilla.com/D95931
Specifically, this renames
* nsTArray_CopyChooser to nsTArray_RelocationStrategy
* the Copy template argument of nsTArray_base to RelocationStrategy
* nsTArray_CopyWithConstructors to nsTArray_RelocateUsingMoveConstructor
* nsTArray_CopyWithMemutils to nsTArray_RelocateUsingMemutils
* DECLARE_USE_COPY_CONSTRUCTORS to MOZ_DECLARE_RELOCATE_USING_MOVE_CONSTRUCTOR
Differential Revision: https://phabricator.services.mozilla.com/D66243
--HG--
extra : moz-landing-system : lando
It is technically possible to have different channel count per chunk, so we
should reinitialize the resampler when that happens. This is a bit awkward
because the rates are on the track, but the data is in the chunks, and the track
cannot access the chunks, so we have to plumb the resampler state down.
Differential Revision: https://phabricator.services.mozilla.com/D53806
--HG--
extra : moz-landing-system : lando
It is now the max channel count of the underlying chunks or 0 if there are no
segment.
Differential Revision: https://phabricator.services.mozilla.com/D51865
--HG--
extra : moz-landing-system : lando
It's not maintained and probably does not work anymore.
Differential Revision: https://phabricator.services.mozilla.com/D5438
--HG--
extra : rebase_source : ccd622e40844dda5d16266e49991462d4ea94224
Add method to help us know whether audio block is audible or not, so that we won't
show the sound indicator for silent web audio.
Differential Revision: https://phabricator.services.mozilla.com/D7819
--HG--
extra : moz-landing-system : lando
This was done automatically replacing:
s/mozilla::Move/std::move/
s/ Move(/ std::move(/
s/(Move(/(std::move(/
Removing the 'using mozilla::Move;' lines.
And then with a few manual fixups, see the bug for the split series..
MozReview-Commit-ID: Jxze3adipUh
This allows static analysis to pass for AudioNodeExternalInputStream.cpp.
MozReview-Commit-ID: 9dvllnnODed
--HG--
extra : rebase_source : 0c7665a3240422d52b82c5c2eaa4942be522dcb9
This avoids any risk of undefined behavior when evaluating uninitialized
members, during copies for example, and makes it safe to test mBufferFormat
when null.
MozReview-Commit-ID: IMAyZ1CSHbk
--HG--
extra : rebase_source : b02431634732cf63d6fe9ede5eb1400a2baa6308
This allows us to see if all chunks in a MediaSegment are null.
When this is true for the MediaSegment passed to NotifyQueuedChanges, we can
assume that the track's input is blocked for the given time.
MozReview-Commit-ID: zf7V7aiohg
--HG--
extra : rebase_source : 5dd8442f85e6011fc0015460c6a7fedf04fb956e
This helps simplify code in places where we want to move all chunks in one
segment to another, new, segment.
For clarity this also explicitly forbids copying a MediaSegment.
MozReview-Commit-ID: 49rzUwFqE7V
--HG--
extra : rebase_source : 3fc7d733cb70419bcf5eedb4a0dd8c0dc90f7054
extra : source : 737ce537e48c84b5e0aa4d06ee5332ae09a99897
The new name makes the sense of the condition much clearer. E.g. compare:
NS_WARN_IF_FALSE(!rv.Failed());
with:
NS_WARNING_ASSERTION(!rv.Failed());
The new name also makes it clearer that it only has effect in debug builds,
because that's standard for assertions.
--HG--
extra : rebase_source : 886e57a9e433e0cb6ed635cc075b34b7ebf81853
PrincipalHandle is a thread safe pointer to a holder of (the main-thread-only
nsIPrincipal) that can be passed around the MSG.
A MediaStreamTrack whose source has just updated its principal, sets the new
principal aside (as its "pending principal"), and combines the new principal
into its current principal.
Then the source starts passing the new principal to the MediaStreamGraph as
a PrincipalHandle.
Changes to a track's PrincipalHandle on the MSG will be surfaced through the
MediaStreamTrackListener API. These changes are dispatched to main thread
and compared to a MediaStreamTrack's pending principal. In case of a match
the track knows the correct principal is flowing and can move the pending
principal to be the current principal and update any main thread principal
observers.
MozReview-Commit-ID: D0JXGWhQFFU
--HG--
extra : rebase_source : 296e269bb46fc5a85a9c3f90dfc0dc40e53572bc
alwu
Noemi Erli
Chun-Min Chang
Andreas Pehrson
Paul Adenot
Razvan Maries
Simon Giesecke
shindli
Sylvestre Ledru
Gurzau Raul
Ehsan Akhgari
Emilio Cobos Álvarez
Andreas Pehrson
Sebastian Hengst
Karl Tomlinson
Andreas Pehrson
Phil Ringnalda
Nicholas Nethercote
Eric Rahm
Wes Kocher
Birunthan Mohanathas