cubeb-coreaudio-rs

Rust implementation of Cubeb on the MacOS platform.

Current Goals

• Rewrite the C code into Rust on a line-by-line basis
  • The coding style is in C style rather than Rust so it's easier to review (and it's easy to re-format the style later by running rustfmt)
• Create some tests for later refactoring

Branches

• trailblazer: Draft Rust code without being reviewed. Commits are scribbled.
• release: The offical version. All the commits are reviewed.
• dev: All the commits are cherry-picked from trailblazer branch. This branch is used to create pull-requests to release branch.

Development Pipeline / Timeline

phase	1	2	3	4	5
draft	translate	translate	test in gecko, fix bugs		x
review		review	review and land	refactor	refactor loop
release				ride the trains	release loop

The draft code is in trailblazer branch, the reviewing code are in the pull requests, which comes from dev branch, and the reviewed code is in release branch.

Status

Thew project is in phase 2. All the lines in cubeb_audiounit.cpp are translated.

By applying the patch to integrate within Cubeb, it can pass all the tests under cubeb/test and it's able to switch devices when the stream is working (we are unable to test this automatically yet).

Now the draft version is tested within gecko. It can be tracked on bugzilla 1530715. (Commits to import and build this within gecko can be found here)

TODO

• Remove #[allow(non_camel_case_types)], #![allow(unused_assignments)], #![allow(unused_must_use)] and apply rust coding styles
• Use Atomic{I64, U32, U64} instead of Atomic<{i64, u32, u64}>, once they are stable.
• Integration Tests
  • Add a test-only API to change the default audio devices
  • Use above API to test the device-changed callback
  • Rewrite some tests under cubeb/test/* in Rust as part of the integration tests
  • Add tests for capturing/recording, output, duplex streams
• Move issues below to github issues.
• Test aggregate devices
  • Test with AirPods, bluethooth devices, or other devices that with special workarounds.
• Unit tests for stream operations
• Clean up the tests. Merge the duplicated pieces in to a function.
• Find a way to catch memory leaks
  • Try Instrument on OSX
• Some of bugs are found when adding tests. Search FIXIT to find them.
• Maybe it's better to move all fn some_func(stm: &AudioUnitStream, ...) functions into impl AudioUnitStream.
• Add comments for APIs in utils
• Fail to run test_create_blank_aggregate_device with test_add_device_listeners_dont_affect_other_scopes_with_* at the same time
  • I guess audiounit_create_blank_aggregate_device will fire the callbacks in test_add_device_listeners_dont_affect_other_scopes_with_*
• Fail to run test_ops_context_register_device_collection_changed_twice_* on my MacBook Air.
  • A panic in capi_register_device_collection_changed causes EXC_BAD_INSTRUCTION.
  • Works fine if replacing register_device_collection_changed: Option<unsafe extern "C" fn(..,) -> c_int> to register_device_collection_changed: unsafe extern "C" fn(..,) -> c_int
  • Test them in AudioUnitContext directly instead of calling them via OPS for now.
• Fail to run test_configure_input_with_zero_latency_frames and test_configure_input at the same time.
  • audiounit_set_buffer_size cannot be called in parallel
  • We should not set kAudioDevicePropertyBufferFrameSize in parallel when another stream using the same device with smaller buffer size is active. See here for reference.
  • Buffer frame size within same device may be overwritten (no matter the AudioUnits are different or not) ?
• Find a reliable way to verify enumerate_devices
• Make a list pairing (device-uid/device-name, available channel layouts) so we can check the layout-related APIs properly!
  • A prototype is in test_set_channel_layout_output.
• Make a black/white list for those devices cannot/can get the datasource, so the tests for audiounit_get_default_device_datasource and those APIs based on audiounit_get_default_device_datasource can work on different devices.
• cubeb-rs
  • Implement to_owned in StreamParamsRef
  • Check the passed parameters like what cubeb.c does!
    • Check the input StreamParams parameters properly, or we will set a invalid format into AudioUnit. In fact, we should check all the parameters properly so we can make sure we don't mess up the streams/devices settings!

Issues

• See discussion here
• Mutex: Find a replacement for owned_critical_section
  • A dummy mutex like Mutex<()> should work (see test_dummy_mutex_multithread) as what owned_critical_section does in C version, but it doens't has equivalent API for assert_current_thread_owns.
  • We implement a OwnedCriticalSection around pthread_mutex_t like what we do in C version for now.
  • It's hard to debug with the variables using OwnedCriticalSection. Within a test with a variable using OwnedCriticalSection, if the OwnedCriticalSection used in the test isn't be dropped in a correct order, then the test will get a crash in OwnedCriticalSection. The examples are test_stream_drop_mutex_(in)correct. The tests must be created very carefully.
• Atomic:
  • The stable atomic types only support bool, usize, isize, and ptr, but we need u64, i64, and f32.
  • Using atomic-rs instead.
  • Rust-Nightly supports AtomicU32 and AtomicU64 so we use that.
• Unworkable API: dispatch_async and dispatch_sync
  • The second parameter of dispatch_async and dispatch_sync is dispatch_block_t, which is defined by typedef void (^dispatch_block_t)(void).
  • The caret symbol ^ defines a block.
  • The block is a lambda expression-like syntax to create closures. (See Apple's document: Working with Blocks)
  • Not sure if Rust can work with it. Rust has its own closure.
  • For now, we implement an API async_dispatch and sync_dispatch to replace dispatch_async and dispatch_sync (prototype on gist.)
    • async_dispatch is based on dispatch_async_f.
    • sync_dispatch is based on dispatch_sync_f.
    • async_dispatch and sync_dispatch take Rust closures, instead of Apple's block, as one of their parameters.
    • The Rust closure (it's actually a struct) will be boxed, which means the closure will be moved into heap, so the closure cannot be optimized as inline code. (Need to find a way to optimize it?)
    • Since the closure will be run on an asynchronous thread, we need to move the closure to heap to make sure it's alive and then it will be destroyed after the task of the closure is done.
• Borrowing Issues
  1. Pass AudioUnitContext across threads. In C version, we pass the pointer to cubeb context across threads, but it's forbidden in Rust. A workaround here is to
     1. Cast the pointer to a cubeb context into a usize value
     2. Pass that value to threads. The value is actually be copied into the code-block that will be run on another thread
     3. When the task on another thread is run, the value is casted to a pointer to a cubeb context
  2. We have a mutex in AudioUnitContext, and we have a reference to AudioUnitContext in AudioUnitStream. To sync what we do in C version, we need to lock the mutex in AudioUnitContext then pass a reference to AudioUnitContext to AudioUnitStream::new(...). To lock the mutex in AudioUnitContext, we call AutoLock::new(&mut AudioUnitContext.mutex). That is, we will borrow a reference to AudioUnitContext as a mutable first then borrow it again. It's forbidden in Rust. Some workarounds are
     1. Replace AutoLock by calling mutex.lock() and mutex.unlock() explicitly.
     2. Save the pointer to mutex first, then call AutoLock::new(unsafe { &mut (*mutex_ptr) }).
     3. Cast immutable reference to a *const then to a *mut: pthread_mutex_lock(&self.mutex as *const pthread_mutex_t as *mut pthread_mutex_t)
• Complexity of creating unit tests
  • We have lots of dependent APIs, so it's hard to test one API only, specially for those APIs using mutex(OwnedCriticalSection actually)
  • It's better to split them into several APIs so it's easier to test them
• APIs that cannot be called in parallel
  • The APIs depending on audiounit_set_buffer_size cannot be called in parallel
    • kAudioDevicePropertyBufferFrameSize cannot be set when another stream using the same device with smaller buffer size is active. See here for reference.
    • The buffer frame size within same device may be overwritten (no matter the AudioUnits are different or not) ?