The core stream data that will be touched on the different threads when
the stream is created, reinitialized, destoryed should be used in the
critical section. Those core data are initialized in stream-setup and
destroyed in stream-close. The stream-setup and stream-close is run in
the critical section so they won't be executed at the same time.
Currently, the critical section is created by our custom mutex. However,
this custom mutex will be removed in the later mutex replacement.
Instead of running these two operations in the critical sections created
by the custom mutex, they should be moved into a struct wrapped by a
standard Rust mutex. As a result, at the end when the custom mutex is
removed, these two operations are still in the critical sections that
are created by the Rust mutex.
Differential Revision: https://phabricator.services.mozilla.com/D34073
--HG--
extra : moz-landing-system : lando
The listeners will be registered and unregistered on the different threads,
so the listeners should be used in the critical sections. We do create
critical sections by our own mutex. However, this custom mutex will be
gradually replaced by the standard Rust mutex in the following patches.
To replace the custom mutex, we put the listeners to the struct wrapped
by a Rust mutex and register or unregister the listeners in the critical
section created by this struct. At the end when the custom mutex is
removed, those operations are still in the critical sections.
Differential Revision: https://phabricator.services.mozilla.com/D34072
--HG--
extra : moz-landing-system : lando
Calling configure_input requires borrowing AudioUnitStream as a
mutable. Merging configure_input into setup will help to avoid a
potential borrowing-twice issue in the later mutex replacement. There
will be a critical section created by a Rust mutex in the setup, which
will borrow the AudioUnitStream as an immutable.
Differential Revision: https://phabricator.services.mozilla.com/D34071
--HG--
extra : moz-landing-system : lando
Calling configure_output requires borrowing AudioUnitStream as a
mutable. Merging configure_output into setup will help to avoid a
potential borrowing-twice issue in the later mutex replacement. There
will be a critical section created by a Rust mutex in the setup, which
will borrow the AudioUnitStream as an immutable.
Differential Revision: https://phabricator.services.mozilla.com/D34070
--HG--
extra : moz-landing-system : lando
Avoid calling minimum_resampling_input_frames by a borrowing from
AudioUnitStream
Differential Revision: https://phabricator.services.mozilla.com/D34069
--HG--
extra : moz-landing-system : lando
We store global layout info and channels info in the cubeb context.
However, the streams might output to different devices. We should store
the layout info of the output device in the stream directly instead of
in the context. On the other hand, the `channels` value is a duplicate
of `output_desc.mChannelsPerFrame` or `mixer.out_channels` so we don't
need to store this value.
Differential Revision: https://phabricator.services.mozilla.com/D34068
--HG--
extra : moz-landing-system : lando
1. Avoid calling layout_init by wrong AudioUnit value
2. Avoid calling layout getting/setting APIs by borrowing the
AudioUnitStream as a mutable. It will help to avoid the potential
borrowing-twice issues in the later mutex replacement.
Differential Revision: https://phabricator.services.mozilla.com/D34067
--HG--
extra : moz-landing-system : lando
1. The code readability is slightly better if we can register and
unregister the callbacks in the same places. The device-chaned callbacks
are registered when stream setup and unregistered when stream close.
2. In the later mutex replacement, the core stream variables that may be
touched by different threads when reinitializing or destroying the
stream, including {in, out}_unit, will be wrapped in a Rust mutex.
Moving these callbacks (un)registration into the same places makes it
easier to find where the critical sections are.
Differential Revision: https://phabricator.services.mozilla.com/D34066
--HG--
extra : moz-landing-system : lando
Before reporting the error by the state callback, closing the stream to
make sure the data callback and device-changed callback are
unregistered.
Differential Revision: https://phabricator.services.mozilla.com/D34065
--HG--
extra : moz-landing-system : lando
1. If AudioUnitRender return kAudioUnitErr_CannotDoInCurrentContext
within a input-only stream, the input-only stream will keep feed silence
data into the buffer instead of reporting the error. With this change,
the error will be rendered as the returned value of the data callback to
the underlying CoreAudio framework.
2. By merging the render_input into audiounit_input_callback and adjust
the timing to call reinit_async, now the reinit_async is called at the
line that is out of the main logic for feeding buffer data. In the scope
of the main logic, there will be a critical section created by locking a
Rust mutex within AudioUnitStream in the later mutex replacement.
Without moving the reinit_async, which borrows AudioUnitStream as a
mutable, out of the critical section, there will be a borrowing-twice
issue.
Differential Revision: https://phabricator.services.mozilla.com/D34064
--HG--
extra : moz-landing-system : lando
The mixer of the stream will be created, reinitialized, used or
destroyed on different threads, so its operations should be in the
critical sections. We do create critical sections by our custom mutex.
However, this custom mutex will be gradually replaced by the standard
Rust mutex in the following patches.
To replace the custom mutex, we put the mixer to the struct wrapped by a
Rust mutex and do all the mixer operations in the critical section
created by this struct. At the end when the custom mutex is removed,
those operations are still in critical sections.
Calling notify_state_changed needs to borrow AudioUnitStream as a
mutuable. To avoid the borrowing-twice issue, the notify_state_changed
calling is moved out the scope of the critical section created in the
output data callback.
Differential Revision: https://phabricator.services.mozilla.com/D34062
--HG--
extra : moz-landing-system : lando
Using an Mixer struct to operate all the mixing APIs is a way to make
the code clearer. In addition, we can avoid calling mix function by
borrowing the AudioUnitStream as a mutable. It will help to avoid
potential borrowing-twice issues in the later mutex replacement.
Differential Revision: https://phabricator.services.mozilla.com/D34061
--HG--
extra : moz-landing-system : lando
The resampler of the stream will be created, reinitialized, used or
destroyed on different threads, so its operations should be in the
critical sections. We do create critical sections by our custom mutex.
However, this custom mutex will be gradually replaced by the standard
mutex in the following patches. To replace the custom mutex, we put the
resampler to the struct wrapped by a Rust mutex and do all the resampler
operations in the critical section created by this struct. At the end
when the custom mutex is removed, those operations are still in critical
sections.
Differential Revision: https://phabricator.services.mozilla.com/D34060
--HG--
extra : moz-landing-system : lando
Using a resample struct to operate all its related operations will make
the code clearer.
Differential Revision: https://phabricator.services.mozilla.com/D34059
--HG--
extra : moz-landing-system : lando
The aggregate device of the stream may be created, reinitialized, or
destroyed on different threads, so its operations should be in the
critical sections. We do create critical sections by our custom mutex.
However, this custom mutex will be gradually replaced by the standard
Rust mutex in the following patches.
To replace the custom mutex, we create a struct wrapped by a Rust mutex
and create critical sections by this Rust mutex to do operations for
aggregate-device settings. At the end, not only aggregate-device
calls, but also other operations that needs to be locked will be
operated in the critical section created by this struct.
This is the beginning patch to replace the custom mutex in
AudioUnitStream.
Differential Revision: https://phabricator.services.mozilla.com/D34058
--HG--
extra : moz-landing-system : lando
Using an aggregate-device struct to operate all the settings for the
aggregate device is a way to make the code clearer. In addition, we can
avoid calling some aggregate-device APIs by borrowing the
AudioUnitStream as a mutable. It will help to avoid potential
borrowing-twice issues in the later mutex replacement.
Differential Revision: https://phabricator.services.mozilla.com/D34057
--HG--
extra : moz-landing-system : lando
The mutex is considered poisoned whenever a thread panics while holding
the mutex. Registering a second device changed callback without unregistering
the original callback makes the thread panics while holding the mutex
for the device_changed_callback. This mutex will be used when device
property changed callback. If the mutex is poisoned then a device
property is changed, we will get another panic when firing the callback
because of using a poisoned mutex.
Differential Revision: https://phabricator.services.mozilla.com/D34056
--HG--
extra : moz-landing-system : lando
The registration, unregistration, and notification for the device
property changed is done in a critical section created by locking our
own custom mutex. To replace our own custom mutex by standard Rust
mutex, the critical section should be created by locking a standard Rust
mutex. This can be done by simply merging device_changed_callback and
device_changed_callback_lock into a Rust mutex variable.
Differential Revision: https://phabricator.services.mozilla.com/D34055
--HG--
extra : moz-landing-system : lando
By moving out this API from the AudioUnitStream, we can avoid calling
it by borrowing the AudioUnitStream variable as a mutable. This will
help to avoid the potential borrowing-twice issues in the later mutex
replacement
In addition, the change applies an idiomatic way to wait for the system
callback event without consuming CPU time.
Differential Revision: https://phabricator.services.mozilla.com/D34054
--HG--
extra : moz-landing-system : lando
This change provides two benefits:
1. Setting variable by the Result value directly is rustier than setting
the variable by a mutual reference with a dummy Result that only
contains an error.
2. By moving out this API from the AudioUnitStream, we can avoid calling
it by borrowing the AudioUnitStream variable as a mutable. This will
help to avoid the potential borrowing-twice issues in the later mutex
replacement.
Differential Revision: https://phabricator.services.mozilla.com/D34053
--HG--
extra : moz-landing-system : lando
Setting variable by the Result value directly is rustier than setting
the variable by a mutual reference with a dummy Result that only
contains an error.
Differential Revision: https://phabricator.services.mozilla.com/D34052
--HG--
extra : moz-landing-system : lando
Setting variable by the Result value directly is rustier than setting
the variable by a mutual reference with a dummy Result that only
contains an error.
Differential Revision: https://phabricator.services.mozilla.com/D34051
--HG--
extra : moz-landing-system : lando
By moving get_volume out of AudioUnitStream, we can avoid unnecessary
borrowing from the AudioUnitStream variable. This change will make the
get_volume become symmetric to the set_volume.
Differential Revision: https://phabricator.services.mozilla.com/D34050
--HG--
extra : moz-landing-system : lando
By narrowing down the parameters that set_volume API needs, we can avoid
the unnecessary mutual borrowing from the AudioUnitStream variable. This
will help to avoid potential borrowing-twice issues in the later mutex
replacement.
Differential Revision: https://phabricator.services.mozilla.com/D34049
--HG--
extra : moz-landing-system : lando
Setting variable by the Result value directly is rustier than setting
the variable by a mutual reference with a dummy Result that only
contains the error.
Differential Revision: https://phabricator.services.mozilla.com/D34048
--HG--
extra : moz-landing-system : lando
The code will be clearer if we can wrap the state-callback that notify
the state changed into a function. It will hide some details about
rendering the callback to C interface.
Differential Revision: https://phabricator.services.mozilla.com/D34047
--HG--
extra : moz-landing-system : lando
When the stream is re-initialized, the input unit and/or output unit
should be reset if they are currently used, no matter what scope (input
or output or in-output) the parameter requests.
Differential Revision: https://phabricator.services.mozilla.com/D34046
--HG--
extra : moz-landing-system : lando
The OwnedCriticalSection is a custom mutex directly translated from the
C code of the Cubeb library. Replacing the custom mutex by standard Rust
mutex make the code fit in the Rust design better and make debugging the
code easier. The variables protected by the custom mutex in the cubeb
context is moved to the Rust standard mutex in previous patches. The
custom mutex in the cubeb context is able to be removed since we no
longer rely on it.
Differential Revision: https://phabricator.services.mozilla.com/D29983
--HG--
extra : moz-landing-system : lando
The mutex is considered poisoned whenever a thread panics while holding
the mutex. Registering a second device-collection-changed callback
without unregistering the original callback makes the thread panics while
holding the mutex for the device-collection-changed variables. This
mutex will be used when the cubeb context is dropped. If the mutex is
poisoned, we will get another panic when dropping the cubeb context
because of using a poisoned mutex.
Differential Revision: https://phabricator.services.mozilla.com/D29982
--HG--
extra : moz-landing-system : lando
The registration, unregistration, and notification for the device
collection changed is done in a critical section created by locking our
own custom mutex. To replace our own custom mutex by standard Rust
mutex, the critical section should be created by locking a standard Rust
mutex. This can be done by simply putting those varaibles for
device-collection-changed to a struct within a Rust mutex.
Differential Revision: https://phabricator.services.mozilla.com/D29981
--HG--
extra : moz-landing-system : lando
The global latency frames in the cubeb context is the latency frames of
the first cubeb stream created in that cubeb context. The reason is
because latency frames cannot be changed when there is an active stream
operating in parallel. To make sure the latency won't be changed, the
latency frames of the streams after the first stream will be overwritten
by the global latency frames. Since we always use the global latency
frames, instead of overwritting latency frames of the later streams, we
could, we could simply initialize all the streams with the global
latency frames.
Differential Revision: https://phabricator.services.mozilla.com/D29980
--HG--
extra : moz-landing-system : lando
The counting of the active streams in a cubeb context is calculated in a
critical section created by locking our own custom mutex. To replace our
own mutex by standard Rust mutex, the critical section calculating the
active streams should be created by locking a standard Rust mutex. This
can be done by simply putting the active_streams to a struct with a Rust
mutex.
Differential Revision: https://phabricator.services.mozilla.com/D29979
--HG--
extra : moz-landing-system : lando
In the current implementation, we assume the aggregate device is created
immediately after calling the system API to create it, and we also
assume the sub-devices of the aggregate device are added immediately
upon we call the system API to set the sub-devices. Unfortunately, these
assumptions are not correct. Occasionally these settings are not
executed immediately, especially when they are not called on the main
thread (e.g., we may create an aggregate device off the main thread when
switching devices). Using the listeners monitoring the devices-changed
events can make sure those settings are done.
Differential Revision: https://phabricator.services.mozilla.com/D29978
--HG--
extra : moz-landing-system : lando
The clamp_latency function is only called when the cubeb context has one
stream only. In that case, clamp_latency will clamp the stream latency
frames in a safe range and then return the clamped value directly. The
code in clamp_latency for more than one streams doesn't be executed
since clamp_latency isn't called when there are more than one streams
within cubeb context. It's easier to read the code if the unused code is
removed.
Differential Revision: https://phabricator.services.mozilla.com/D29011
--HG--
extra : moz-landing-system : lando
To import cubeb-coreaudio-rs crate later, creating a dummy folder in
advance in libcubeb. It should be the mirrow of cubeb-coreaudio-rs crate
on github. The information of cubeb-coreaudio-rs is in README_MOZILLA.
Differential Revision: https://phabricator.services.mozilla.com/D23429
--HG--
extra : moz-landing-system : lando
Because IPC call runs asynchronously in both remote decoder process and
content process, ProcessOutput() for buffers prior to Flush() could be
scheduled to run after the flush promise is resolved, and Codec.queueInput()
could be preempted and processes prior sample after flush.
To help check the validness of buffers, a session ID increased by flush
is added to both RemoteDataDecoder and remote codec service and will be
passed through IPC. If the passed ID doesn't agree with current session
ID, it means the buffer doesn't belong to current session and should be
discard.
Differential Revision: https://phabricator.services.mozilla.com/D36382
--HG--
extra : moz-landing-system : lando
Because IPC call runs asynchronously in both remote decoder process and
content process, ProcessOutput() for buffers prior to Flush() could be
scheduled to run after the flush promise is resolved, and Codec.queueInput()
could be preempted and processes prior sample after flush.
To help check the validness of buffers, a session ID increased by flush
is added to both RemoteDataDecoder and remote codec service and will be
passed through IPC. If the passed ID doesn't agree with current session
ID, it means the buffer doesn't belong to current session and should be
discard.
Differential Revision: https://phabricator.services.mozilla.com/D36382
--HG--
extra : moz-landing-system : lando
This also removes the following prefs, because they're unused:
- media.autoplay.allow-muted pref
- media.autoplay.blackList-override-default
Differential Revision: https://phabricator.services.mozilla.com/D36396
--HG--
extra : rebase_source : 0570540496302b3efedadf4d5115ee5422d5c279
In particular:
* trait objects without an explicit `dyn` are deprecated
* `...` range patterns are deprecated
I think these shouldn't really warn by default and should be clippy / opt-in
lints, but anyway, doesn't hurt.
Differential Revision: https://phabricator.services.mozilla.com/D35135
--HG--
extra : moz-landing-system : lando
This must have been broken while addressing review comments in Bug 1548841,
the first commit has it right and the last commit in the series breaks it.
Since we don't yet generate mDNS addresses in Firefox, the unit tests would not
catch this problem, and interoperability with other browsers would continue to
work due to peer reflex candidates.
We will have test coverage for this once we land generating our own mDNS
candidates. ICE will fail in the mochitests if we don't support resolving
mDNS addresses properly, which is how I discovered this bug.
Differential Revision: https://phabricator.services.mozilla.com/D34579
--HG--
extra : moz-landing-system : lando
This adds a scalar for the codecs used when receiving and sending video in
a WebRTC call.
Differential Revision: https://phabricator.services.mozilla.com/D33840
--HG--
extra : moz-landing-system : lando
In bug 1538508 we changed the MediaChangeMonitor to raise an error when we try
to create a decoder and we don't yet have the H.264 extra data, to maintain the
assumptions made by OpenH264/WebRTC. This however doesn't play nice with the
HLS demuxer, which won't always give you extra data after a seek.
So change the MediaChangeMonitor to by default assume that it should just drop
frames until it has the extra data it needs to create a decoder. We can flag
this mode off in the WebRTC MediaDataDecoder.
Differential Revision: https://phabricator.services.mozilla.com/D34079
--HG--
extra : moz-landing-system : lando
A few small changes are required to backport the updated windows capture code
to our version of webrtc.org. These are largely path changes, but
DetachFromThread has been renamed to Detach, and they seem to have a working
overload for ostream operator<< for VideoType that I have not been able to
track down. It is also necessary to add our local modification for pid_t back
to the GetDeviceName method, but it is not used for video capture anyways.
Depends on D33337
Differential Revision: https://phabricator.services.mozilla.com/D33338
--HG--
extra : moz-landing-system : lando
This was added in commit 74395345e8d2fecd504cb687eb79deee4ef394c3, but this
only cherry picks the ToHex part of that commit.
Depends on D33336
Differential Revision: https://phabricator.services.mozilla.com/D33337
--HG--
extra : moz-landing-system : lando
This updates just video_capture/windows to the tip of webrtc.org, from commit
8581877121f58435d683c2a39a73e72e5dcd558a.
This removes the locking which was leading to the deadlock in Bug 1552755. It
also removes the reliance upon the DirectShow example code which lead to our
local implementations of BaseFilter, BaseInputPin, BasePin, and MediaType.
These can now be removed.
Differential Revision: https://phabricator.services.mozilla.com/D33336
--HG--
extra : moz-landing-system : lando
This is a cherry pick of upstream commit c1187ab34bdf836bd33f7f050d525184eba4cd20.
Differential Revision: https://phabricator.services.mozilla.com/D32826
--HG--
extra : moz-landing-system : lando
This adds a mdns_addr field to nICEr ICE candidates to track the mDNS address
associated with a candidate, if any. This is used to hide the real address
when generating ICE stats. This potentially could be handled at the
MediaTransportHandler level, but we need to know if a candidate is mDNS to
prevent pairing it with relay candidates, which is part of the next commit.
This adds a unit tests to check that the mDNS addresses are handled properly.
As part of doing this, TestBogusCandidate was fixed. As written, it was never
parsing the bogus candidate because it was in the wrong ICE state.
Differential Revision: https://phabricator.services.mozilla.com/D30935
--HG--
extra : moz-landing-system : lando
This adds code to use the DNSService to resolve incoming mDNS candidates.
This could potentially be done within nICEr itself, but the delays
associated with resolving mDNS addresses make the timing there quite
awkward.
An internal DNSListener class is used to proxy requests back to the
MediaTransportHandler to avoid problems with multiple inheritance. Once
resolved, mDNS candidates are passed into nICEr as though they were
normal candidates. Avoiding leaking the actual address is done in the
second commit in this series.
Calling ActivateTransport can also potentially results in a call to
nr_ice_ctx_parse_candidate which would fail to handle mDNS candidates, but
we don't specify any candidates at that point.
Differential Revision: https://phabricator.services.mozilla.com/D29861
--HG--
extra : moz-landing-system : lando
This change pulls in some changes necessary to work with newer versions
of Rust.
Differential Revision: https://phabricator.services.mozilla.com/D32650
--HG--
extra : moz-landing-system : lando
This removes all references to application capture except from MediaSourceEnum.
That was left in place so as to not change the enumerated values used
for WEBRTC_GET_USER_MEDIA_TYPE telemetry.
Differential Revision: https://phabricator.services.mozilla.com/D28089
--HG--
extra : moz-landing-system : lando
Pick commits:
3570749 - wasapi: tie monitor lifetime with notification client lifetime. BMO 1545279 (#505)
162625a - test: add option to get the posotion of a stream (#504)
Differential Revision: https://phabricator.services.mozilla.com/D28796
--HG--
extra : moz-landing-system : lando