cubeb-coreaudio-rs/README.md

cubeb-coreaudio-rs

Implementation of MacOS Audio backend in CoreAudio framework for Cubeb written in Rust. Currently it can only be built by rust-nightly since we use nightly-only atomic types(AtomicU32 and AtomicU64).

Current Goals

• Translate C code line by line into Rust (so the coding style is almost same as C)
• Create tests for later refactoring

Branches

• trailblazer: Draft Rust code without being reviewed. Commits are scribbled.
• release: The offical version. All the commits are reviewed.
• dev: All the commits are cherry-picked from trailblazer branch. This branch is used to create pull-requests to release branch.

Status

• 🥚 : Not implemented.
• 🐣 : Work in progress. May be implemented partially or blocked by dependent APIs.
• 🐥 : Implemented.
• 🐓 : Already ride the trains.

Cubeb APIs (Public APIs)

• 🥚 : 2/20 (10%)
• 🐣 : 2/20 (10%)
• 🐥 : 16/20 (80%)

Cubeb APIs	status
cubub_init	🐥
cubub_get_backend_id	🐥
cubub_get_max_channel_count	🐥
cubub_get_min_latency	🐥
cubub_get_preferred_sample_rate	🐥
cubub_enumerate_devices	🐥
cubeb_device_collection_destroy	🐥
cubeb_stream_init	🐣
cubeb_stream_destroy	🐣
cubeb_stream_start	🥚
cubeb_stream_stop	🥚
cubeb_reset_default_device	🐥
cubeb_stream_get_position	🐥
cubeb_stream_get_latency	🐥
cubeb_stream_set_volume	🐥
cubeb_stream_set_panning	🐥
cubeb_stream_get_current_device	🐥
cubeb_stream_device_destroy	🐥
cubeb_stream_register_device_changed_callback	🐥
cubub_register_device_collection_changed	🐥

Interanl APIs

• 🥚 : 19/75 (25.3%)
• 🐣 : 8/75 (10.6%)
• 🐥 : 48/75 (64%)

Interanl AudioUnit APIs	status
make_sized_audio_channel_layout	🥚
to_string	🐥
has_input	🐥
has_output	🐥
channel_label_to_cubeb_channel	🥚
cubeb_channel_to_channel_label	🥚
audiounit_increment_active_streams	🐥
audiounit_decrement_active_streams	🐥
audiounit_active_streams	🐥
audiounit_set_global_latency	🐥
audiounit_make_silent	🥚
audiounit_render_input	🥚
audiounit_input_callback	🥚
audiounit_mix_output_buffer	🥚
minimum_resampling_input_frames	🥚
audiounit_output_callback	🥚
audiounit_set_device_info	🐥
audiounit_reinit_stream	🥚
audiounit_reinit_stream_async	🐣
event_addr_to_string	🐥
audiounit_property_listener_callback	🐥
audiounit_add_listener	🐥
audiounit_remove_listener	🐥
audiounit_install_device_changed_callback	🐥
audiounit_install_system_changed_callback	🐥
audiounit_uninstall_device_changed_callback	🐥
audiounit_uninstall_system_changed_callback	🐥
audiounit_get_acceptable_latency_range	🐥
audiounit_get_default_device_id	🐥
audiounit_convert_channel_layout	🥚
audiounit_get_preferred_channel_layout	🥚
audiounit_get_current_channel_layout	🥚
audiounit_destroy	🥚
audio_stream_desc_init	🐥
audiounit_init_mixer	🥚
audiounit_set_channel_layout	🥚
audiounit_layout_init	🥚
audiounit_get_sub_devices	🐥
audiounit_create_blank_aggregate_device	🐥
get_device_name	🐥
audiounit_set_aggregate_sub_device_list	🐥
audiounit_set_master_aggregate_device	🐥
audiounit_activate_clock_drift_compensation	🐥
audiounit_workaround_for_airpod	🐥
audiounit_create_aggregate_device	🐥
audiounit_destroy_aggregate_device	🐥
audiounit_new_unit_instance	🐥
audiounit_enable_unit_scope	🐥
audiounit_create_unit	🐥
audiounit_init_input_linear_buffer	🐣
audiounit_clamp_latency	🐥
buffer_size_changed_callback	🐥
audiounit_set_buffer_size	🐥
audiounit_configure_input	🐣
audiounit_configure_output	🐣
audiounit_setup_stream	🐣
audiounit_close_stream	🐣
audiounit_stream_destroy_internal	🐣
audiounit_stream_destroy	🐣
audiounit_stream_start_internal	🥚
audiounit_stream_stop_internal	🥚
audiounit_stream_get_volume	🐥
convert_uint32_into_string	🐥
audiounit_get_default_device_datasource	🐥
audiounit_get_default_device_name	🐥
audiounit_strref_to_cstr_utf8	🐥
audiounit_get_channel_count	🐥
audiounit_get_available_samplerate	🐥
audiounit_get_device_presentation_latency	🐥
audiounit_create_device_from_hwdev	🐥
is_aggregate_device	🐥
audiounit_get_devices_of_type	🐥
audiounit_collection_changed_callback	🐥
audiounit_add_device_listener	🐥
audiounit_remove_device_listener	🐥

TODO

• cubeb-rs
  • Implement to_owned in StreamParamsRef
• Integration Tests
  • Add a test-only API to change the default audio devices
  • Use above API to test the device-changed callback
• Move issues below to github issues.
• Test aggregate devices
  • Test with AirPods
• Test for stream operations
• Clean up the tests. Merge the duplicated pieces in to a function.
• Find a way to catch memory leaks
  • Try Instrument on OSX
• Some of bugs are found when adding tests. Search FIXIT to find them.
• Maybe it's better to move all fn some_func(stm: &AudioUnitStream, ...) functions into impl AudioUnitStream.
• Add comments for APIs in utils
• Fail to run test_create_blank_aggregate_device with test_add_device_listeners_dont_affect_other_scopes_with_* at the same time
  • I guess audiounit_create_blank_aggregate_device will fire the callbacks in test_add_device_listeners_dont_affect_other_scopes_with_*
• Fail to run test_ops_context_register_device_collection_changed_twice_* on my MacBook Air.
  • A panic in capi_register_device_collection_changed causes EXC_BAD_INSTRUCTION.
  • Works fine if replacing register_device_collection_changed: Option<unsafe extern "C" fn(..,) -> c_int> to register_device_collection_changed: unsafe extern "C" fn(..,) -> c_int
  • Test them in AudioUnitContext directly instead of calling them via OPS for now.
• Fail to run test_configure_input_with_zero_latency_frames and test_configure_input at the same time.
  • audiounit_set_buffer_size cannot be called in parallel
  • We should not set kAudioDevicePropertyBufferFrameSize in parallel when another stream using the same device with smaller buffer size is active. See here for reference.
  • Buffer frame size within same device may be overwritten (no matter the AudioUnits are different or not) ?
• Implement auto_array_wrapper and auto_array (by trait and generic I think)

Issues

• Mutex: Find a replacement for owned_critical_section
  • A dummy mutex like Mutex<()> should work (see test_dummy_mutex_multithread) as what owned_critical_section does in C version, but it doens't has equivalent API for assert_current_thread_owns.
  • We implement a OwnedCriticalSection around pthread_mutex_t like what we do in C version for now.
  • It's hard to debug with the variables using OwnedCriticalSection. Within a test with a variable using OwnedCriticalSection that will get a panic, if the OwnedCriticalSection used in the test isn't be dropped before where the code get a panic, then the test might get a crash in OwnedCriticalSection rather than the line having a panic. One example is test_stream_get_panic_before_releasing_mutex. The tests must be created very carefully.
• Atomic:
  • The stable atomic types only support bool, usize, isize, and ptr, but we need u64, i64, and f32.
  • Using atomic-rs instead.
  • Rust-Nightly supports AtomicU32 and AtomicU64 so we use that.
• Unworkable API: dispatch_async and dispatch_sync
  • The second parameter of dispatch_async and dispatch_sync is dispatch_block_t, which is defined by typedef void (^dispatch_block_t)(void).
  • The caret symbol ^ defines a block.
  • The block is a lambda expression-like syntax to create closures. (See Apple's document: Working with Blocks)
  • Not sure if Rust can work with it. Rust has its own closure.
  • For now, we implement an API async_dispatch and sync_dispatch to replace dispatch_async and dispatch_sync (prototype on gist.)
    • async_dispatch is based on dispatch_async_f.
    • sync_dispatch is based on dispatch_sync_f.
    • async_dispatch and sync_dispatch take Rust closures, instead of Apple's block, as one of their parameters.
    • The Rust closure (it's actually a struct) will be boxed, which means the closure will be moved into heap, so the closure cannot be optimized as inline code. (Need to find a way to optimize it?)
    • Since the closure will be run on an asynchronous thread, we need to move the closure to heap to make sure it's alive and then it will be destroyed after the task of the closure is done.
• Borrowing Issues
  1. Pass AudioUnitContext across threads. In C version, we pass the pointer to cubeb context across threads, but it's forbidden in Rust. A workaround here is to
     1. Cast the pointer to a cubeb context into a usize value
     2. Pass that value to threads. The value is actually be copied into the code-block that will be run on another thread
     3. When the task on another thread is run, the value is casted to a pointer to a cubeb context
  2. We have a mutex in AudioUnitContext, and we have a reference to AudioUnitContext in AudioUnitStream. To sync what we do in C version, we need to lock the mutex in AudioUnitContext then pass a reference to AudioUnitContext to AudioUnitStream::new(...). To lock the mutex in AudioUnitContext, we call AutoLock::new(&mut AudioUnitContext.mutex). That is, we will borrow a reference to AudioUnitContext as a mutable first then borrow it again. It's forbidden in Rust. Some workarounds are
     1. Replace AutoLock by calling mutex.lock() and mutex.unlock() explicitly.
     2. Save the pointer to mutex first, then call AutoLock::new(unsafe { &mut (*mutex_ptr) }).
     3. Cast immutable reference to a *const then to a *mut: pthread_mutex_lock(&self.mutex as *const pthread_mutex_t as *mut pthread_mutex_t)
• Complexity of creating unit tests
  • We have lots of dependent APIs, so it's hard to test one API only, specially for those APIs using mutex(OwnedCriticalSection actually)
  • It's better to split them into several APIs so it's easier to test them
• APIs that cannot be called in parallel
  • The APIs depending on audiounit_set_buffer_size cannot be called in parallel
    • kAudioDevicePropertyBufferFrameSize cannot be set when another stream using the same device with smaller buffer size is active. See here for reference.
    • The buffer frame size within same device may be overwritten (no matter the AudioUnits are different or not) ?