cubeb-coreaudio-rs/README.md

276 строки
19 KiB
Markdown
Исходник Обычный вид История

2018-09-10 21:18:00 +03:00
# cubeb-coreaudio-rs
2019-02-14 02:36:49 +03:00
*Rust* implementation of [Cubeb][cubeb] on the MacOS platform.
2018-09-10 21:20:11 +03:00
2018-10-08 01:58:11 +03:00
## Current Goals
2019-02-14 02:36:49 +03:00
- Rewrite the [C code][cubeb-au] into *Rust* on a line-by-line basis
- The coding style is in *C* style rather than *Rust* so it's easier to review
(and it's easy to re-format the style later by running `rustfmt`)
- Create some tests for later refactoring
2019-01-09 04:04:09 +03:00
## Branches
- *trailblazer*: Draft *Rust* code without being reviewed. Commits are scribbled.
- *release*: The offical version. All the commits are reviewed.
- *dev*: All the commits are cherry-picked from *trailblazer* branch.
This branch is used to create pull-requests to *release* branch.
2019-02-19 21:23:12 +03:00
## Development Pipeline / Timeline
2019-02-26 22:12:06 +03:00
| phase | 1 | 2 | 3 | 4 | 5 |
| ------- | --------- | --------- | ----------------------- | --------------- | ------------- |
| draft | translate | translate | test in gecko, fix bugs | | x |
| review | | review | review and land | refactor | refactor loop |
| release | | | | ride the trains | release loop |
2019-02-19 21:23:12 +03:00
The draft code is in *trailblazer* branch, the reviewing code are in the pull requests,
which comes from *dev* branch, and the reviewed code is in *release* branch.
2019-01-11 04:33:06 +03:00
## Status
2019-02-14 02:36:49 +03:00
2019-02-26 22:12:06 +03:00
Thew project is in phase 2. All the lines in [*cubeb_audiounit.cpp*][cubeb-au] are translated.
2019-02-19 21:23:12 +03:00
2019-02-26 22:12:06 +03:00
By applying the [patch][integrate-with-cubeb] to integrate within [Cubeb][cubeb],
it can pass all the tests under *cubeb/test*
and it's able to switch devices when the stream is working
(we are unable to test this automatically yet).
2019-02-14 02:36:49 +03:00
2019-02-26 22:12:06 +03:00
Now the draft version is tested within *gecko*.
It can be tracked on [*bugzilla* 1530715][bugzilla-cars].
(Commits to import and build this within *gecko* can be found [here][build-within-gecko])
<!--
2019-01-11 04:33:06 +03:00
- 🥚 : Not implemented.
- 🐣 : Work in progress. May be implemented partially or blocked by dependent APIs.
- 🐥 : Implemented.
2019-01-29 01:52:45 +03:00
- 🐓 : Already ride the trains.
2019-01-11 04:33:06 +03:00
### Cubeb APIs (Public APIs)
2019-02-05 01:15:31 +03:00
- 🥚 : 0/20 (0%)
- 🐣 : 0/20 (0%)
- 🐥 : 20/20 (100%)
2019-01-11 04:33:06 +03:00
| Cubeb APIs | status |
| --------------------------------------------- | ------ |
| cubub_init | 🐥 |
| cubub_get_backend_id | 🐥 |
| cubub_get_max_channel_count | 🐥 |
| cubub_get_min_latency | 🐥 |
| cubub_get_preferred_sample_rate | 🐥 |
| cubub_enumerate_devices | 🐥 |
| cubeb_device_collection_destroy | 🐥 |
| cubeb_stream_init | 🐥 |
| cubeb_stream_destroy | 🐥 |
2019-02-04 23:36:57 +03:00
| cubeb_stream_start | 🐥 |
2019-02-05 01:15:31 +03:00
| cubeb_stream_stop | 🐥 |
2019-01-11 04:33:06 +03:00
| cubeb_reset_default_device | 🐥 |
| cubeb_stream_get_position | 🐥 |
| cubeb_stream_get_latency | 🐥 |
| cubeb_stream_set_volume | 🐥 |
| cubeb_stream_set_panning | 🐥 |
2019-01-11 04:33:06 +03:00
| cubeb_stream_get_current_device | 🐥 |
| cubeb_stream_device_destroy | 🐥 |
| cubeb_stream_register_device_changed_callback | 🐥 |
| cubub_register_device_collection_changed | 🐥 |
2019-01-11 23:59:00 +03:00
### Interanl APIs
- 🥚 : 0/75 (0%)
2019-02-22 01:59:56 +03:00
- 🐣 : 0/75 (0%)
- 🐥 : 74/75 (100%)
2019-01-11 23:59:00 +03:00
| Interanl AudioUnit APIs | status |
| ------------------------------------------- | ------ |
| make_sized_audio_channel_layout | 🐥 |
2019-01-11 23:59:00 +03:00
| to_string | 🐥 |
| has_input | 🐥 |
| has_output | 🐥 |
| channel_label_to_cubeb_channel | 🐥 |
| cubeb_channel_to_channel_label | 🐥 |
2019-01-11 23:59:00 +03:00
| audiounit_increment_active_streams | 🐥 |
| audiounit_decrement_active_streams | 🐥 |
| audiounit_active_streams | 🐥 |
| audiounit_set_global_latency | 🐥 |
2019-02-13 02:16:14 +03:00
| audiounit_make_silent | 🐥 |
| audiounit_render_input | 🐥 |
| audiounit_input_callback | 🐥 |
2019-02-21 00:52:25 +03:00
| audiounit_mix_output_buffer | 🐥 |
| minimum_resampling_input_frames | 🐥 |
2019-02-21 00:52:25 +03:00
| audiounit_output_callback | 🐥 |
2019-01-11 23:59:00 +03:00
| audiounit_set_device_info | 🐥 |
2019-02-22 01:59:56 +03:00
| audiounit_reinit_stream | 🐥 |
| audiounit_reinit_stream_async | 🐥 |
2019-01-11 23:59:00 +03:00
| event_addr_to_string | 🐥 |
| audiounit_property_listener_callback | 🐥 |
| audiounit_add_listener | 🐥 |
| audiounit_remove_listener | 🐥 |
| audiounit_install_device_changed_callback | 🐥 |
| audiounit_install_system_changed_callback | 🐥 |
| audiounit_uninstall_device_changed_callback | 🐥 |
| audiounit_uninstall_system_changed_callback | 🐥 |
| audiounit_get_acceptable_latency_range | 🐥 |
| audiounit_get_default_device_id | 🐥 |
| audiounit_convert_channel_layout | 🐥 |
| audiounit_get_preferred_channel_layout | 🐥 |
| audiounit_get_current_channel_layout | 🐥 |
| audiounit_destroy | 🐥 |
2019-01-11 23:59:00 +03:00
| audio_stream_desc_init | 🐥 |
2019-02-21 00:52:25 +03:00
| audiounit_init_mixer | 🐥 |
2019-02-20 02:54:22 +03:00
| audiounit_set_channel_layout | 🐥 |
2019-02-20 21:55:56 +03:00
| audiounit_layout_init | 🐥 |
2019-01-11 23:59:00 +03:00
| audiounit_get_sub_devices | 🐥 |
| audiounit_create_blank_aggregate_device | 🐥 |
| get_device_name | 🐥 |
| audiounit_set_aggregate_sub_device_list | 🐥 |
| audiounit_set_master_aggregate_device | 🐥 |
| audiounit_activate_clock_drift_compensation | 🐥 |
| audiounit_workaround_for_airpod | 🐥 |
| audiounit_create_aggregate_device | 🐥 |
| audiounit_destroy_aggregate_device | 🐥 |
| audiounit_new_unit_instance | 🐥 |
| audiounit_enable_unit_scope | 🐥 |
| audiounit_create_unit | 🐥 |
| audiounit_init_input_linear_buffer | 🐥 |
2019-01-11 23:59:00 +03:00
| audiounit_clamp_latency | 🐥 |
2019-01-12 04:21:39 +03:00
| buffer_size_changed_callback | 🐥 |
| audiounit_set_buffer_size | 🐥 |
2019-02-04 22:01:54 +03:00
| audiounit_configure_input | 🐥 |
2019-02-21 00:52:25 +03:00
| audiounit_configure_output | 🐥 |
| audiounit_setup_stream | 🐥 |
| audiounit_close_stream | 🐥 |
| audiounit_stream_destroy_internal | 🐥 |
| audiounit_stream_destroy | 🐥 |
2019-02-04 22:53:49 +03:00
| audiounit_stream_start_internal | 🐥 |
| audiounit_stream_stop_internal | 🐥 |
2019-01-11 23:59:00 +03:00
| audiounit_stream_get_volume | 🐥 |
| convert_uint32_into_string | 🐥 |
| audiounit_get_default_device_datasource | 🐥 |
| audiounit_get_default_device_name | 🐥 |
| audiounit_strref_to_cstr_utf8 | 🐥 |
| audiounit_get_channel_count | 🐥 |
| audiounit_get_available_samplerate | 🐥 |
| audiounit_get_device_presentation_latency | 🐥 |
| audiounit_create_device_from_hwdev | 🐥 |
| is_aggregate_device | 🐥 |
| audiounit_get_devices_of_type | 🐥 |
| audiounit_collection_changed_callback | 🐥 |
| audiounit_add_device_listener | 🐥 |
| audiounit_remove_device_listener | 🐥 |
2019-02-26 22:12:06 +03:00
-->
2019-01-11 23:59:00 +03:00
2018-10-08 20:24:56 +03:00
## TODO
- Use `Atomic{I64, U32, U64}` instead of `Atomic<{i64, u32, u64}>`, once they are stable.
- Integration Tests
- Add a test-only API to change the default audio devices
- Use above API to test the device-changed callback
2019-02-14 02:36:49 +03:00
- Rewrite some tests under _cubeb/test/*_ in _Rust_ as part of the integration tests
- Add tests for capturing/recording, output, duplex streams
2018-11-02 21:33:27 +03:00
- Move issues below to github issues.
2018-10-09 01:52:54 +03:00
- Test aggregate devices
2019-02-14 02:36:49 +03:00
- Test with AirPods, bluethooth devices, or other devices that with special workarounds.
- Unit test for stream operations
2018-10-09 01:52:54 +03:00
- Clean up the tests. Merge the duplicated pieces in to a function.
2018-10-20 02:21:39 +03:00
- Find a way to catch memory leaks
2018-10-20 02:41:26 +03:00
- Try Instrument on OSX
- Some of bugs are found when adding tests. Search *FIXIT* to find them.
2018-12-06 16:48:09 +03:00
- Maybe it's better to move all `fn some_func(stm: &AudioUnitStream, ...)` functions into `impl AudioUnitStream`.
2018-12-17 23:28:28 +03:00
- Add comments for APIs in `utils`
2018-12-28 04:01:35 +03:00
- Fail to run `test_create_blank_aggregate_device` with `test_add_device_listeners_dont_affect_other_scopes_with_*` at the same time
- I guess `audiounit_create_blank_aggregate_device` will fire the callbacks in `test_add_device_listeners_dont_affect_other_scopes_with_*`
- Fail to run `test_ops_context_register_device_collection_changed_twice_*` on my MacBook Air.
- A panic in `capi_register_device_collection_changed` causes `EXC_BAD_INSTRUCTION`.
- Works fine if replacing `register_device_collection_changed: Option<unsafe extern "C" fn(..,) -> c_int>` to `register_device_collection_changed: unsafe extern "C" fn(..,) -> c_int`
- Test them in `AudioUnitContext` directly instead of calling them via `OPS` for now.
- Fail to run `test_configure_input_with_zero_latency_frames` and `test_configure_input` at the same time.
- `audiounit_set_buffer_size` cannot be called in parallel
- We should not set `kAudioDevicePropertyBufferFrameSize` in parallel when another stream using the same device with smaller buffer size is active. See [here][chg-buf-sz] for reference.
- *Buffer frame size* within same device may be overwritten (no matter the *AudioUnit*s are different or not) ?
- Check the input `StreamParams` parameters properly, or we will set a invalid format into `AudioUnit`.
- In fact, we should check **all** the parameters properly so we can make sure we don't mess up the streams/devices settings!
- Find a reliable way to verify `enumerate_devices`
2019-02-20 02:54:22 +03:00
- Make a list pairing (device-uid/device-name, available channel layouts) so we can check the layout-related APIs properly!
- A prototype is in [`test_set_channel_layout_output`](src/backend/test.rs).
2019-02-20 03:19:07 +03:00
- Make a black/white list for those devices cannot/can get the *datasource*,
so the tests for `audiounit_get_default_device_datasource` and those APIs based on `audiounit_get_default_device_datasource` can work on different devices.
2019-02-22 01:59:56 +03:00
- Replace `if let Err(_) = some_func(...) { ... }` by `if some_func(...).is_err() { ... }`
- Replace `fn audiounit_set_device_info(..., devtype: DeviceType)` with `fn audiounit_set_device_info(..., side: io_side)`
2019-02-14 02:36:49 +03:00
- [cubeb-rs][cubeb-rs]
- Implement `to_owned` in [`StreamParamsRef`][cubeb-rs-stmparamsref]
2018-10-24 02:52:33 +03:00
## Issues
2019-02-26 22:12:06 +03:00
- See discussion [here][discussion]
2018-11-02 21:27:17 +03:00
- Mutex: Find a replacement for [`owned_critical_section`][ocs]
2019-02-13 19:32:13 +03:00
- A dummy mutex like `Mutex<()>` should work (see [`test_dummy_mutex_multithread`][ocs-rust]) as what `owned_critical_section` does in [_C_ version][ocs], but it doens't has equivalent API for `assert_current_thread_owns`.
2018-11-02 21:27:17 +03:00
- We implement a [`OwnedCriticalSection` around `pthread_mutex_t`][ocs-rust] like what we do in [_C_ version][ocs] for now.
- It's hard to debug with the variables using `OwnedCriticalSection`. Within a test with a variable using `OwnedCriticalSection`, if the `OwnedCriticalSection` used in the test isn't be dropped in a correct order, then the test will get a crash in `OwnedCriticalSection`. The examples are [`test_stream_drop_mutex_(in)correct`](src/backend/test.rs). The tests must be created very carefully.
2018-12-06 00:07:39 +03:00
- Atomic:
- The stable atomic types only support `bool`, `usize`, `isize`, and `ptr`, but we need `u64`, `i64`, and `f32`.
- Using [atomic-rs](https://github.com/Amanieu/atomic-rs) instead.
- *Rust-Nightly* supports `AtomicU32` and `AtomicU64` so we use that.
2018-11-14 03:10:03 +03:00
- Unworkable API: [`dispatch_async`][dis-async] and [`dispatch_sync`][dis-sync]
- The second parameter of [`dispatch_async`][dis-async] and [`dispatch_sync`][dis-sync] is [`dispatch_block_t`][dis-block], which is defined by `typedef void (^dispatch_block_t)(void)`.
2018-11-02 21:16:32 +03:00
- The caret symbol `^` defines a [block][c-ext-block].
- The _block_ is a lambda expression-like syntax to create closures. (See Apple's document: [Working with Blocks][apple-block])
- Not sure if _Rust_ can work with it. _Rust_ has its own [_closure_][rs-closure].
2018-11-14 03:10:03 +03:00
- For now, we implement an API [`async_dispatch`][async-dis] and [`sync_dispatch`][sync-dis] to replace [`dispatch_async`][dis-async] and [`dispatch_sync`][dis-sync] (prototype on [gist][osx-dis-gist].)
2018-11-02 21:16:32 +03:00
- [`async_dispatch`][async-dis] is based on [`dispatch_async_f`][dis-async-f].
2018-11-14 03:10:03 +03:00
- [`sync_dispatch`][sync-dis] is based on [`dispatch_sync_f`][dis-sync-f].
- [`async_dispatch`][async-dis] and [`sync_dispatch`][sync-dis] take [_Rust closures_][rs-closure], instead of [Apple's _block_][apple-block], as one of their parameters.
2018-11-02 21:16:32 +03:00
- The [_Rust closure_][rs-closure] (it's actually a struct) will be `box`ed, which means the _closure_ will be moved into heap, so the _closure_ cannot be optimized as _inline_ code. (Need to find a way to optimize it?)
- Since the _closure_ will be run on an asynchronous thread, we need to move the _closure_ to heap to make sure it's alive and then it will be destroyed after the task of the _closure_ is done.
- Borrowing Issues
1. Pass `AudioUnitContext` across threads. In _C_ version, we [pass the pointer to `cubeb` context across threads][cubeb-au-ptr-across-threads], but it's forbidden in _Rust_. A workaround here is to
2018-11-02 21:27:17 +03:00
1. Cast the pointer to a `cubeb` context into a `usize` value
2018-11-02 21:32:15 +03:00
2. Pass that value to threads. The value is actually be **copied** into the code-block that will be run on another thread
2018-11-02 21:27:17 +03:00
3. When the task on another thread is run, the value is casted to a pointer to a `cubeb` context
2. We have a [`mutex`][ocs-rust] in `AudioUnitContext`, and we have a _reference_ to `AudioUnitContext` in `AudioUnitStream`. To sync what we do in [_C version_][cubeb-au-init-stream], we need to _lock_ the `mutex` in `AudioUnitContext` then pass a _reference_ to `AudioUnitContext` to `AudioUnitStream::new(...)`. To _lock_ the `mutex` in `AudioUnitContext`, we call `AutoLock::new(&mut AudioUnitContext.mutex)`. That is, we will borrow a reference to `AudioUnitContext` as a mutable first then borrow it again. It's forbidden in _Rust_. Some workarounds are
1. Replace `AutoLock` by calling `mutex.lock()` and `mutex.unlock()` explicitly.
2. Save the pointer to `mutex` first, then call `AutoLock::new(unsafe { &mut (*mutex_ptr) })`.
3. Cast immutable reference to a `*const` then to a `*mut`: `pthread_mutex_lock(&self.mutex as *const pthread_mutex_t as *mut pthread_mutex_t)`
- Complexity of creating unit tests
- We have lots of dependent APIs, so it's hard to test one API only, specially for those APIs using mutex(`OwnedCriticalSection` actually)
- It's better to split them into several APIs so it's easier to test them
- APIs that cannot be called in parallel
- The APIs depending on `audiounit_set_buffer_size` cannot be called in parallel
- `kAudioDevicePropertyBufferFrameSize` cannot be set when another stream using the same device with smaller buffer size is active. See [here][chg-buf-sz] for reference.
- The *buffer frame size* within same device may be overwritten (no matter the *AudioUnit*s are different or not) ?
2018-10-08 20:24:56 +03:00
2018-10-08 02:38:43 +03:00
[cubeb]: https://github.com/kinetiknz/cubeb "Cross platform audio library"
2018-11-02 21:16:32 +03:00
[cubeb-au]: https://github.com/kinetiknz/cubeb/blob/master/src/cubeb_audiounit.cpp "Cubeb AudioUnit"
2019-02-14 02:36:49 +03:00
[integrate-with-cubeb]: https://github.com/ChunMinChang/cubeb-coreaudio-rs/commit/e84c554f18ef054376134c79a112a84cb8f923b4 "patch for integrating within cubeb"
2018-11-02 21:16:32 +03:00
[ocs]: https://github.com/kinetiknz/cubeb/blob/master/src/cubeb_utils_unix.h "owned_critical_section"
[ocs-rust]: src/backend/owned_critical_section.rs "OwnedCriticalSection"
2018-11-14 03:10:03 +03:00
[dis-sync]: https://developer.apple.com/documentation/dispatch/1452870-dispatch_sync "dispatch_sync"
2018-11-02 21:16:32 +03:00
[dis-async]: https://developer.apple.com/documentation/dispatch/1453057-dispatch_async "dispatch_async"
[dis-async-f]: https://developer.apple.com/documentation/dispatch/1452834-dispatch_async_f "dispatch_async_f"
2018-11-14 03:10:03 +03:00
[dis-sync-f]: https://developer.apple.com/documentation/dispatch/1453123-dispatch_sync_f "dispatch_sync_f"
2018-11-02 21:16:32 +03:00
[dis-block]: https://developer.apple.com/documentation/dispatch/dispatch_block_t?language=objc "dispatch_block_t"
[c-ext-block]: https://en.wikipedia.org/wiki/Blocks_(C_language_extension) "Blocks: C language extension"
[apple-block]: https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/ProgrammingWithObjectiveC/WorkingwithBlocks/WorkingwithBlocks.html "Working with Blocks"
[rs-closure]: https://doc.rust-lang.org/book/second-edition/ch13-01-closures.html "Closures"
2018-11-14 03:10:03 +03:00
[sync-dis]: src/backend/dispatch_utils.rs
[async-dis]: src/backend/dispatch_utils.rs
[osx-dis-gist]: https://gist.github.com/ChunMinChang/8d13946ebc6c95b2622466c89a0c9bcc "gist"
2018-11-02 21:16:32 +03:00
[cubeb-au-ptr-across-threads]: https://github.com/kinetiknz/cubeb/blob/9a7a55153e7f9b9e0036ab023909c7bc4a41688b/src/cubeb_audiounit.cpp#L3454-L3480 "Pass pointers across threads"
2018-11-14 21:05:26 +03:00
[cubeb-au-init-stream]: https://github.com/kinetiknz/cubeb/blob/9a7a55153e7f9b9e0036ab023909c7bc4a41688b/src/cubeb_audiounit.cpp#L2745-L2748 "Init stream"
[cubeb-rs]: https://github.com/djg/cubeb-rs "cubeb-rs"
[cubeb-rs-stmparamsref]: https://github.com/djg/cubeb-rs/blob/78ed9459b8ac2ca50ea37bb72f8a06847eb8d379/cubeb-core/src/stream.rs#L61 "StreamParamsRef"
[cubeb-rs-capi-stm-reg-dev-chg-callback]: https://github.com/djg/cubeb-rs/blob/78ed9459b8ac2ca50ea37bb72f8a06847eb8d379/cubeb-backend/src/capi.rs#L56 "stream_register_device_changed_callback"
[cubeb-backend]: https://github.com/djg/cubeb-rs/tree/master/cubeb-backend "cubeb-backend"
[cubeb-pulse-rs]: https://github.com/djg/cubeb-pulse-rs "cubeb-pulse-rs"
[cubeb-backend-stm-reg-dev-chg-cb]: cubeb-backend-stream_register_device_changed_callback.diff "Implementation of stream_register_device_changed_callback"
[cubeb-pulse-rs-reg-dev-chg-cb]: cubeb-pulse-rs-register_device_changed_callback.diff "Impelement of register_device_changed_callback"
2019-02-26 22:12:06 +03:00
[chg-buf-sz]: https://cs.chromium.org/chromium/src/media/audio/mac/audio_manager_mac.cc?l=982-989&rcl=0207eefb445f9855c2ed46280cb835b6f08bdb30 "issue on changing buffer size"
[bugzilla-cars]: https://bugzilla.mozilla.org/show_bug.cgi?id=1530715 "Bug 1530715 - Implement CoreAudio backend for Cubeb in Rust"
[build-within-gecko]: https://github.com/ChunMinChang/gecko-dev/commits/cubeb-coreaudio-rs
[discussion]: https://docs.google.com/document/d/1ZP6R7d5S9I_8bXOXhplnO6qFM1X4VokWtE7w8ExgJEQ/edit?ts=5c6d5f09